Monday 29 April 2013

Writing Working Code at Hyperspeed

Image courtesy of Cycling Cosmonaut

Today I thought I would share with you something I had to do recently at work. To give some background I work for Talent Q who are a provider of online psychometric tests; I am one of the developers who works on all the various systems we have such as the candidate tests, the administration side and all sorts of backend services to keep everything working as it should.

In our last sprint we were working on adding a new suite of tests to our product list. In fact we’ve already got them running now as part of some trials before global release and have already witnessed tens of thousands of completions – they clearly work so that’s always a good start! Me and my colleagues did want to go back over the code though before the final release as developers always have that feeling of wanting to perfect their code; we also had a view of making sure long-term maintainability could be achieved.

So we looked through what we had already and found some areas that we wanted to improve. The main part was that the code was trying to be too clever for its own good:

  1. Most classes were trying to be very generic but ended up not incorporating the specifics of each individual test, resulting in some code smells such as lots of object casting which was unnecessary.
  2. We also had several God objects that tried to contain every conceivable detail you could imagine yet at the end of day even missing some vital details.
  3. Due to all the various classes being very generic it was difficult to follow through the logic of the code. There was clearly a design in there to not have too many strong dependencies interlinked with each other – something that I believe is definitely worthwhile – but you could also get easily lost down a codepath; bear in mind that these tests were built up in an organic manner as most things are so the end result does tend to have more than you actually need (which I will discuss later on).

So I was the one tasked with implementing the changes we wanted. We decided that we would not make all these generic types which somehow had to be tied together, instead each test would have a dedicated class with a clear focus even if it meant some duplication of logic – as long as we have working code we would be able to compare the differences and refactor later if required.

All seems reasonable so far. Oh, did I mention that due to a number of planning issues I had to do all of this in a matter of days?

Umm, OK…

But I managed it, all within working hours (no overtime) and it all worked first time! Here’s how I did it…

Plan Your Work First

You might think that with a tight deadline in front of you that you should not waste any time and start writing code immediately, right?

Wrong!

You could do this but without a clear focus on what you are trying to achieve you’ll struggle to keep up with what you are doing; you don’t want to be thinking of what you need to do and how to translate that into code at the same time.

So first of all I spent a few hours going over what I needed to do. I looked at each of the new tests we were creating and wrote down notes to help me understand the high-level logic that drove them, such as:

  • How do I load the questions?
  • How do I keep track of state and where the candidate is in the test?
  • What if the candidate quits a session and restarts from the last question they answered? Will we be able to know where to pick up the test from?
  • How do I get a final result from the test?

Fortunately out of the three new test types we were creating two of them were similar to previous assessments we had already released so it wasn’t too hard to work out the details for them. By the end of this I had a one-page document of notes for each test which explained to me everything they needed to do. With this now written down and not cluttering my mind I could finally get down to writing the code.

Test Driven Development is Your Friend

Now I knew what to do I had to figure out how I was going to get my code written down and make sure it all worked quickly. The trial code we currently had was not designed to be testable; it was all written under the assumption that there would always be a database and always have a user interface driving it, meaning that testing that code was manual, laborious and the feedback loop was painfully slow.

I’m a big fan of Test Driven Development though; the idea of writing lots of little tests to quickly verify that your code is working is something I’ve jumped onto with gusto. In the end, although I don’t particularly like the idea of rewriting code from scratch, I decided that I was going to have to write the code for these assessments from first principles again and this time focus on making them work in an isolated environment meaning no database or website required.

Start Typing

Image originally from Tumblr

With all this in place it was time for me to start typing, and fast! One of the benefits of Test Driven Development is that because unit tests are so small you can clearly focus on one tiny bit at a time. For example if I wrote the constructor for the assessment then under the scenario of a brand new assessment being run I knew that the outcome of creating the object would mean:

  1. The internal state would be set to “Running Test” instead of “Completed”
  2. That the first question number would be one
  3. That the total number of questions available would be a specific number
  4. That the first question was loaded into the test
  5. etc

So I simply wrote one test at a time, used the test runner to make sure it passed, and moved to the next one. I managed to keep up this pace and produce quite a number of test cases quickly, all the while improving the code and verifying that it all worked as I expected it to.

Getting “In the Zone”

My blog is called “In the Coding Zone” as it refers to that mythical place where developers love to roam and simply be lost in their own thoughts without any kind of interruption. When “in the zone” developers are able to churn out vast amounts of code and be hugely productive.

Getting there is a challenge in itself, especially when working in an open office environment. Fortunately I got very few distractions during this time so that I really could just ignore everything around me and focus purely on hitting this deadline.

The End Result

After a lot of hard work I hit that deadline but what else did I have to show for it?

  1. With the code rewritten with testing principles in mind we now had our first ever set of assessments which we could run using test automation. This means we can now verify that these assessments work even without a real database and just using dummy data, or build upon the unit tests to create integration tests using a real bank of questions.
  2. For the first time our assessments finally make sense; figuring out how our assessments worked used to be a daunting task but I realised this is because it was down to the sheer amount of extraneous code that was written, not due to the complexity of the logic itself. Test Driven Development is really good at focusing you on what you need to achieve to get your tests running; once all tests pass then you don’t need to write any more code so your final implementation tends to be very lean and ultimately simpler to understand.
  3. One drawback though is that I was mentally tired after this exercise. Keeping up a relentless pace of writing code/tests almost non-stop for days does take it’s toll on you and you can’t keep it up forever.

Developing software seems to have this aurora of always being delayed and not working until you test it over and over again but I wanted to prove that it is possible to have something work first time (and on time!) as long as you plan your work effectively, focus on the job in hand and basically test as you go, not leaving it as an afterthought.

Thursday 25 April 2013

Is JavaScript Really So Bad?

I’ve had some pretty strong opinions on JavaScript in the past, most of them negative as a previous post has explained. Having been forced into using it more fully recently on several mini-projects though my rigid opinions on it have softened slightly.

Despite working on web technologies I don’t really consider myself a web developer; I am most comfortable in working on server-side code, usually in C#. So when the time comes when I am faced with HTML, CSS and JavaScript I do tend to get a little overwhelmed; I can be extremely productive in C# but tying server-side code to a front-end interface can take me hours, more likely days, to get an end result that I’m not even that happy with. It is this mindset I think that clouds my judgement with JavaScript; I’m simply not that proficient in using it and don’t continue developing this skillset to allow me to improve.

Part of the problem I think is that I am most familiar with ASP.NET WebForms. When I first started web development proper I worked on WebForms and was able to bypass a lot of issues around learning good HTML/CSS/JavaScript practices because WebForms managed most of the functionality for you; just tie up server controls, set some properties and the final rendered result would somehow magically do what you wanted it to. I still work in WebForms but now I’m trying to get more involved with ASP.NET MVC, both in a professional and recreational sense, but there’s a problem; MVC is an excellent framework but it does less for you than WebForms. You have full control over the HTML you output but now I need to know the finer details that WebForms so successfully hid from me all these years, including even simple things like what happens when a HTML form sends a POST request back to the server.

So given that I’ve been working a bit more on MVC websites recently in various capacities I took the opportunity to try and “reset” my thinking process and try and work with JavaScript as it was intended and not as an afterthought or constant hindrance. Over the past few weeks I’ve now started to form these opinions instead:

Separation of Concerns

Below is a typical example of the kind of webpage I’m most used to seeing:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>My webpage</title>
<style type="text/css">
.banner
{
background-color
: red;
}
</style>
<script type="text/javascript">
function firstLoad() {
alert(
"Hello World!");
}
</script>
</head>
<body onload="firstLoad()">
<div class="banner">
Welcome!
</div>
<a href="Next.html" style="padding: 10px" onclick="javascript: alert('You clicked me!')">Next page</a>
</body>

Yuck! See how it mixes HTML, CSS and JavaScript into one file? Unfortunately this is what I see a lot of and, I’m ashamed to say, even do myself because it is easy to do. But then I’ll run into problems like how to test or debug that JavaScript. Take the example of the hardcoded onclick handler on that anchor tag; it’s a simple example but what if it was something more complicated? How could I test that when I find it doesn’t work like I expected?


So instead I’ve tried to decouple the parts of the webpage into a file each for HTML, CSS and JavaScript as nearly every web developer recommends. From this I’ve found it is now easier to edit and view my work; if I want to change the style of an element I go to my CSS file and not hunt for it in the monolithic file and similarly if I need to make a code change I update the script file and reload.


jQuery


OK, now I’ve got my code separated from everything else so now I can just focus on the functionality of the page, but how do I do <insert cool fancy feature here>? On it’s own JavaScript doesn’t provide a lot for you. Sure, I can use getElementById() to find various parts of the page but after that do I then write a huge amount of code to do a relatively simple thing? No, I would use one of the bazillion JavaScript frameworks out there. To my mind jQuery is the top framework that everyone thinks of.


I spent about a week of playing around with jQuery and using selectors and effects to manipulate the page, calling back to the server to process some JSON data and so on. As a newbie I did have to search around on the internet quite a bit to figure out how to do some simple things but the advantage of jQuery is that, because it is so prolific, there are tons of resources and samples out there to use.


All in all I now can’t imagine trying to do anything in JavaScript now without using jQuery as it just makes things so much simpler.


JavaScript IntelliSense


I will freely admit that I completely rely on IntelliSense when writing .NET code. Why bother memorising thousands of APIs when a simple list can show you what is available? IntelliSense works brilliantly for statically typed languages but JavaScript is a dynamic language; at any point during execution the floor could potentially be pulled up beneath you and the API you thought was available could be changed. The dynamic nature of JavaScript is definitely a positive in a lot of cases but not when it comes to trying to discover what APIs are available, going back to relying on *shudder* reading documentation.


As it turns out though Visual Studio now has a way of trying to provide IntelliSense some information when writing JavaScript code by including reference comments in your file. These seem to work a little like import statements or C-style #include instructions but instead of pulling in code dependencies it looks for source code commenting and adds that information to IntelliSense. I tried it on jQuery and suddenly I can see everything that jQuery is capable of. Amazing!


Having this in JavaScript is such a big time saver for me because:



  1. It saves round-trips to reading documentation and determining what each function is and what it requires. A simple tooltip which shows what parameters are needed and what they do is a great help to me.
  2. It helps a lot in discovering the APIs. jQuery as an example has lots of functions available but how many? And what do they do? Sometimes just having a simple list of functions available is enough to make you wonder what one does and could end up helping you figure out your solution.

Function Callbacks


And now onto some minor gripes I still have with JavaScript!


One of the little annoyances I’m coming across, especially with jQuery’s constant need of function parameters, is that the syntax for creating a nameless function (or sometimes referred to as a lambda function in other languages) is particularly verbose. Here is what I mean:


$(document).ready(function () {
$(
".tag").each(function (index, element) {
// Do something here
});
});

Having to write the word function gets a bit tiring when doing it over and over again – especially as JavaScript can be minified, you would think that you would want something shorter to use. Whereas in C# using LINQ I could write something like this:


var results = list.Where(x => x.Age > 10).Select(x => x.Name);

Notice how this “fat arrow” syntax makes things more concise. It does appear though that this is being included in the ES 6 specification so hopefully I will see this in JavaScript one day, I just wish it was now!


But It’s Still Not Working…


By biggest bug-bear with JavaScript though is still the fact that I can write lots of lovely code and it may still not work and the browser won’t tell me it doesn’t work! Rather than tell me anything useful it will simply ignore anything that won’t work and just silently carry on, leaving me bewildered as to why what I expect to happen doesn’t.


Here is a typical conversation I end up having with my computer:


Me: OK, let’s run this and check my new feature works.


Computer:


Me: Er, hello? Did you do anything? Nothing has happened.


Computer: Sorry, I can’t find that new function you wrote.


Me: But it’s in my source file right here!


Computer: Well it’s not in my cached version which didn’t update on the last page load.


Me: Oh right, the browser cache. *sigh* Let me clear it and reload.


Computer: Ah, now I see your new function!


Me: Great, but still nothing is happening.


Computer: Well no, I can’t find an element with an ID called “banner”.


Me: Er, that’s meant to be “Banner”. With an uppercase “B”.


Computer: Well if you can’t spell that’s your own fault. *snigger*


Me: *grumble* Fine… *fixes spelling* There, happy now?


Computer: No, now I can’t find your new function. Have you tried clearing your browser cache?


Me: *bangs head on table*


…and so on.


Unfortunately this feedback loop happens all the time with me. Perhaps it will improve with experience but at the moment I do end up getting frustrated that no useful developer information is conveyed to me when something goes wrong.



Conclusion


Overall though I am starting to find JavaScript more useful now. When it is coupled with a powerful framework to assist you and you accept it for what it is and not try and compare it to another language I realise now that you can achieve quite a lot with it. Which is useful as it seems that we’ll all be using JavaScript for a long time to come.