JavaScript: Behavior Driven Development / Test Driven Development Tools Roundup

14Jul08

I’m sure with size, complexity and sheer number of JavaScript applications and frameworks that have been poring into production over the last few years, there must be a lot of developers out there putting serious effort into testing. Despite this there seems to be quite a low amount of information on the web about the tools and frameworks that are available. So I thought I would share the with you my own personal experiences when it comes to the tools that are out there.

Spec and Testing Frameworks:

This is an area JavaScript has pretty much got nailed with two great offering for both the xUnit and xSpec/xBehave crowds.

JSSpec

I really don’t know what I would do without this great BDD framework written by 강규영 (a.k.a Alan Kang). It follows the standard semantics of BDD and has an extremely natural DSL which I’ve seen developers get up and running with in under ten minutes.

describe('Plus operation', {
    'should concatenate two strings': function() {
        value_of("Hello " + "World").should_be("Hello World");
    },
    'should add two numbers': function() {
        value_of(1 + 2).should_be(3);
    }
});

The spec runner is also quite something to behold as it runs through each spec within the suite for your story. I still believe that a GUI or web interface as is the case in JSSpec is a very powerful tool when conveying the value of thorough testing to the product owner. I even had a product owner bring stakeholders into the office who have no idea about BDD or any software development practices for that matter just to show them the tests.

There are a few ‘quirks’, such as extending some of JavaScript’s base type classes and a small sprinkling of global variables but that’s a small price to pay in the name of elegant DSL, besides the code is very readable which means you can easily get in there and make some tweaks if you would prefer a slightly less flowing DSL in return for a clean namespace.

JSUnit

I used this JavaScript port of JUnit for a long time before I discovered BDD and then JSSpec. It is a very well supported framework and is very good at what it does, but as is the problem with all of the xUnit family of test suites it doesn’t have a clearly defined set of semantics for testing other than assertThis and assertThat so therefore can’t really have a DSL, so instead we have a collection of functions and a lot of room for developers to fill in the blanks.

function testPlusOperation() {
    assertEquals("Hello " + "World", "Hello World");
    assertEquals(1 + 2, 3);
}

For people who are experienced at testing it works great, for those who aren’t things can get very ugly, very quickly. It also has a nice test runner, but the green progress bar just doesn’t have the same impact in conveying the sheer weight of  the tests it is running through in the same way JSSpec does, it is very functional but isn’t going to impress the stakeholders. To sum up as I said already I can’t knock the software itself, it does exactly what is says on the box.

Mocking Frameworks:

Unfortunately the two offerings for mocking frameworks aren’t nearly as strong as their spec/test counterparts discussed already. They both have their ups and downs but one major flaw they both share is an inability to reliably get the public methods from your class (function) constructors and this is in part a JavaScript problem. In order for the mocking frameworks to retrieve the public methods which may be contained within your classes constructor rather than its prototype, the class has to be instantiated so that the the properties can be iterated through to see if the expected method can be found there. This is great, but the problem comes when your class is dependent on an argument passed in to the constructor. Like so:

function MyClass(dependancyObject){
    this.foo = dependancyObject.someMethod();
}

When we pass MyClass over to be mocked, the mocking framework is not going to pass in an argument that implements the interface of the dependency object and therefore will raise an exception. Yes I already know there are umpteen good JavaScript coding practice that will avoid this problem, but the point is that a mocking framework should work as long as the code for my class is valid, regardless of whether it follows good practices or not.

JSMock

JSMock is cool little mocking framework with a very sexy DSL, the only problem is that it isn’t fully featured. It allows for you to add return value expectations to your mock object but does not allow you to place an expectation on the number of times the method will be called. This often results in this kind of thing:

myMock.expects().myVeryWellUsedMethod().andReturns(something)
myMock.expects().myVeryWellUsedMethod().andReturns(something)
myMock.expects().myVeryWellUsedMethod().andReturns(something)
myMock.expects().myVeryWellUsedMethod().andReturns(something)
myMock.expects().myVeryWellUsedMethod().andReturns(something)
myMock.expects().myVeryWellUsedMethod().andReturns(something)
myMock.expects().myVeryWellUsedMethod().andReturns(something)
myMock.expects().myVeryWellUsedMethod().andReturns(something)

When what I REALLY REALLY want to do is this:

myMock.expects().myVeryWellUsedMethod().times(8).andReturns(something)

Or if we were a bit naughty and had extended the Number.prototype, maybe even something like this:

myMock.expects().myVeryWellUsedMethod().8.times().andReturns(something)

But what we all to often end up with is this:

// 😦
var myStub = function(){
    myVeryWellUsedMethod: function(){
        return something;
    }
};

There is also no support giving expectations to the parameters passed to methods of the mock object. Personally with my normal usage patterns I seem to be able to live with that, but I know that for other people it is a big issue. I know I sound like I’ve come down quite hard on JSMock but that’s only because I really love the DSL and know that with just a couple more features it could have been perfect.

Mock4js

While JSMock falls short on features, I very much doubt you will hear many people laying the same charge against Mock4js. It has everything you would expect from a fully featured mocking framework, we can set expectations for our mock methods on the arguments passed, the return values, the number of times we expect it to be called and can also quickly knock up stubs if you are that way inclined.
But don’t start getting too excited quite yet. The DSL is not very smooth, meaning it will take a few days before you are writing mocks without having to check the docs, but this is a small issue compared to the havoc it causes in your namespace. In order to be able to create mocks you need to make a call like so:

Mock4JS.addMockSupport(this);

This method then proceeds to decorate whichever namespace ‘this’ is referring to (usually global or that of your testing framework) with a sh!t load of methods. The first time I used it was within in the JSSpec namespace and straight away there was a collision.
“And that’s all I have to say about that”. Forrest Gump.

Code Coverage Tools

Although I’m not anywhere as religious about code coverage as I am to other aspects of testing, especially in JavaScript where inconsistencies often mean you have whole modules that were designed only for a single browser, SVG/VML type applications spring to mind. This means a file can receive 100% coverage in one browser and 0% in another. Despite these glaring inconsistencies code coverage is still useful to find any large gaps, especially if you are a lead developer working on a large code base with a large team where monitoring manually can be a real pain.

JSCoverage

JSCoverage is a cool command line app that runs through your projects folder and recreates the project in a new folder only with some added ‘special sauce’. You will now be able to point there web interface to the HTML page of you test suite and get back a detailed coverage report.

Advertisements


7 Responses to “JavaScript: Behavior Driven Development / Test Driven Development Tools Roundup”

  1. 1 Ben

    I agree with your first para and I’m surprised you have no feedback other than this rather pointless note. But thanks a lot, the analysis and the pointers are good to have.

  2. Hi Chris. Nice little article here. Totally with you on JSSpec, it’s really helped with catching problems when working on JazzRecord.

    I find BDD to be much more sensible than testing all the internals and things users of your code will never touch. I think of it as black box testing and feel it really makes for better stability without greatly affecting time to test. Adding a few JSSpec tests for a new feature (or something as yet untested) takes a few minutes – not much longer than simply verifying something works for myself in Firebug – but lasts for the life of the project.

    It’s more a mindset change than anything else. Before doing BDD (this applies to all languages) I was testing internals unnecessarily and re-reading old tests was such a pain. With the more declarative nature of BDD test frameworks (like `value_of(X).should_be(Y)` in JSSpec) is immediately recognizable for what it is testing, and makes digging into the details that much easier.

    The only thing that could improve the JSSpec experience would be if it could somehow be integrated in the deployment process automatically as an enforcement of rules. I’ve made a few deployments now where I had fixed bugs in component files of JazzRecord, but forgot to build the actual release file, which JSSpec tests against. If someone knows how to do this I’d greatly appreciate the tip!

  3. 3 Chris

    Hi Nick,

    Sorry blog, I had nearly forgotten all about you!

    Thanks for the feedback Nick. Yep I don’t know where I’d be without BDD, well actually I do know as testing is something I’ve learned about the hard way in the past. We had a project where we reverse engineered the Google Maps API and plugged it in to an open source mapping engine. That was a project where testing was extremely important as at that time the Google Maps API was changing so frequently, so we used every JavaScript testing product we could lay our hands on at one point or another.

    Since writing the article above there have been some new developments, there’s been a new BDD framework released called ScrewUnit (http://github.com/nkallen/screw-unit/tree/master), looks quite cool but I really hate it when developers try making JavaScript look like Ruby, trying to recreate symbols and such, so I’ve stuck with JSSpec.

    Also I’ve been in contact with Alan from JSSpec and he informed me JSSpec2 is at alpha stage, this new version has a HMTL, Rhino and Windows Script Host runners, so now it’s possible to start setting up a continuous integration process for your JavaScript projects, which I am extremely excited about. Unfortunately the current release only has an xUnit style DSL, but the BDD dsl is in the todo list. You can get it out of the JSSpec SVN on Google Code.

    Also there’s a new mocking framework called (amok) which I wrote, I haven’t publicised it as I wrote it mainly for internal use, but it’s open source and available at Google Code (http://code.google.com/p/amok/).

    Cheers,

    Chris

  4. Yeah, I know a lot of the Rails world is using ScrewUnit, or so my local grapevine tells me. I looked over it, it seemed pretty similar.

    A few things from Ruby are nice in JS (the each method, for instance) but I’m with you on not re-creating a language when the existing tools do just fine.

    I emailed Alan a couple times before I implemented any tests for Jazz with JSSpec, hoping there was a concise, non-kludgey-feeling way of using `before` and `before_all`, etc setup/teardown of data for test sets. He said that for the now I’d have to stick with globals and delete them in the `after` functions, which is what I’m doing for the time being. As long as you’re careful it’s not a big deal, but it doesn’t /feel/ right, I know.

    I’d heard something about JSSpec2 but yeah, don’t feel the need to try it at this stage. I will be trying to get my existing tests to run in AIR and Titanium shortly, as we advertise those as supported platforms and I’d like to automated test more thoroughly.

    Looked over the amok stuff. Pretty cool. Seems fairly similar to JSMock with diff language.

  5. 5 TJ

    You should check out JSpec http://visionmedia.github.com/jspec/
    it is by far the cleanest and lightest BDD framework (one file). Tons of features and over 50 matchers

  6. 6 vinay

    I don’t know you people whether, discussing about Technology behind this or the behavior test. Anew behavior test available online here: http://www.toolsacademy.net/tools_btest/


  1. 1 ipod video

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: