Asynchronous Testing with Node.js
This post has been updated at 09/07/2011.
Do you remember projects when your tests run for 30 minutes or even an hour? I do, actually it's nearly turning into a standard. So when CI send you notification that something went wrong, you are already working on a new task, so you have to switch context to fix the stupid bug and then switch context back ... pretty annoying and distracting. Fortunatelly it doesn't have to be so slow, mainly in a web application when you can make a lot of code to run asynchronously since you are always waiting for database ...
Do you remember projects when your tests run for 30 minutes or even an hour? I do, actually it's nearly turning into a standard. So when CI send you notification that something went wrong, you are already working on a new task, so you have to switch context to fix the stupid bug and then switch context back ... pretty annoying and distracting. Fortunatelly it doesn't have to be so slow, mainly in a web application when you can make a lot of code to run asynchronously since you are always waiting for database ...
Well to be honest with you, it's not a choice you can make. Since Node.js itself is asynchronous you simply can't write synchronous tests. Consider following code:
function list (request, response) {
db.query("SELECT * from posts", function (error, posts) {
response.writeHead(200, {"Content-Type": "text/json"});
response.write(JSON.stringify(posts));
});
};
We aren't using return anywhere since methods which takes callback returns undefined or something else, but never the actual result since the result isn't available when you return from the query function (it was returning a promise object in previous versions of Node.js). Take a look at this example:
// BAD, this can't work!
function list (request, response) {
var posts = db.query("SELECT * from posts", function (err, posts) {
// do something
});
return posts;
};
This means you can simply do posts.should(eql(list(request, response)) but you actually need to put the test to the callback:
function list (request, response, callback) {
db.query("SELECT * from posts", function (error, posts) {
response.writeHead(200, {"Content-Type": "text/json"});
response.write(JSON.stringify(posts));
callback(error, posts);
});
};
// test
it("it should returns something", function () {
list(request, response, function (error, posts) {
assert.ok(posts);
});
};
Easy-peasy, right? Hmm, but what if the callback isn't fired? Then the spec will pass but the code won't actually work at all!
it("it should returns something", function (test) {
list(request, response, function (error, posts) {
assert.ok(posts);
test.finished();
});
};
So the test is considered to be finished only if the finished method is called. Something similar has to be supported by all frameworks if they claim they are asynchronous. If they don't provide this, don't use them, because you won't be able to write solid tests.
So Which Frameworks Support Asynchronous Testing?
JSpec
JSpec claims to support async testing, but according to the examples in JSpec README there isn't actually full support for asynchronous testing, just for setTimeout and some others and you have to know how long you have to wait what you obviously at most cases don't know. I found the author to be very nice and responsive, and JSpec looks pretty nice if you are guy who likes things like Rails, Prototype etc. It has a lot of functionality, it seems to be quite magic and it has some problems (i. e. you don't have to use JS syntax, but some custom one, however when you have an error in the syntax, it just silently fails). As I prefer lightweight, straightforward & as simple as possible tools, JSpec isn't for me.
Vows.js
Vows.js took different approach than I showed in my example. It uses Node.js events and if the question() function emits success event, the test is considered to be completed successfully. Take a look at an example (borrowed from their README):
var vows = require('vows'),
assert = require('assert');
vows.tell('Deep Thought', function () {
question('what is the answer to the universe?').addVow(function (answer) {
assert.equals(answer, 42);
}, 'it should know the answer to the ultimate question of life');
});
I like the idea with events, but I don't like the syntax.
Node Async Testing
Node Async Testing has much nicer syntax, at least in my opinion, but I don't like the way how it works. It use global exceptionHandler to catch exceptions from specs which means that only one test can run in time, so tests won't be so fast. It is OK for a small projects but when you start to write a big projects in Node.js, this won't be good enough. But yes, even if the test running in this test framework is synchronous, it still can test asynchronous code.
NodeUnit
NodeUnit is a nice, minimalistic test framework. It's rather explicit and really simple and I have to admin, I like it. This is how the syntax looks like:
exports.testListResponse = function(test) {
view.list(request, response, function () {
test.ok(false, "this assertion should fail");
test.done();
});
};
Currently I'm using my own 2 methods for testing and I use the assert module from Node.js core, but my application is getting more complex, so I'm going to use NodeUnit. So no, I didn't try any of these projects, but I made sure they are up-to-date, that they don't use promises and similar deprecated API.
I don't consider myself an expert to testing asynchronous code nor Node.js testing frameworks. If you know about others testing frameworks which support asynchronous testing, please let me know.
PS: I found another interesting post on this topic: Unit Testing with Node.js by Debuggable.com.
blog comments powered by Disqus