A modern web page does a lot of things, hence the term web application has become more commonly used over the last year. A consequence of this is that we are crafting widgets and entire modules using JavaScript, or at least in some language that eventually ends up being “compiled” to JavaScript.
Where there is code there should also be tests. And the tests need to go green every time the code is changed. You are testing your client side code, right?
JavaScript testing
As with everything else, there has been an evolution of testing frameworks for JavaScript. From tests that run in a static HTML page like QUnit, to the innovative js-test-driver that captures (multiple) browsers and runs tests on them using a server component, and finally to the newest kid on the block, Buster.js, that does everything and more.
Using Buster, you can have your tests run very quickly on PhantomJS, a headless WebKit browser based on the V8 JavaScript engine. (It uses JavaScriptCore with JIT compiler). It is also easy to capture multiple browsers, and thus run the tests on multiple platforms at once.
We have recently set up our Continuous Integration server, Jenkins, to run our tests written in Buster. In addition, we added linting and checkstyle reports using JSHint.
Configuring Jenkins CI
To accomplish this, go configure your project in Jenkins, find the Build section, and add the following build steps (in order) using Execute Shell:
1. Start the Buster server, and make the Phantom browser connect to it. (The buster_phantom.sh script is included at the end of this post.)
2. Run the tests, and make Buster output xUnit formatted XML.
3. Cleanup by stopping Buster and Phantom.
4. Run JSHint with the —jslint-reporter flag.
5. Run JSHint with the –checkstyle-reporter flag.
Finally, under Post-build Actions, add these steps:
1. Publish Checkstyle analysis results and Publish JUnit test result report.
2. To mark the build as unstable when one of the JSHint checks fail, add Report Violations.
3. If you have not done so already, you should add email notifications when the build goes unstable.
The script used to start Buster and Phantom, are embedded below from https://gist.github.com/4285257:
| #!/bin/bash | |
| # | |
| # Depends on buster.js and phantom.js being installed | |
| # | |
| BUSTERPATH_OSX="/usr/local/lib/node_modules/buster" | |
| BUSTERPATH_GNU="/usr/lib/node_modules/buster" | |
| BUSTER_PHANTOMSCRIPT="script/phantom.js" | |
| BUSTER_CMD="buster-server" | |
| PHANTOM_CMD="phantomjs" | |
| command -v $BUSTER_CMD > /dev/null || { echo "Buster not found"; exit 1; } | |
| command -v $PHANTOM_CMD > /dev/null || { echo "Phantomjs not found"; exit 1; } | |
| if [ -d $BUSTERPATH_GNU ]; then | |
| BUSTER_PHANTOMSCRIPT_PATH="$BUSTERPATH_GNU/$BUSTER_PHANTOMSCRIPT"; | |
| elif [ -d $BUSTERPATH_OSX ]; then | |
| BUSTER_PHANTOMSCRIPT_PATH="$BUSTERPATH_OSX/$BUSTER_PHANTOMSCRIPT" | |
| else | |
| echo "Buster script not found" | |
| exit 1 | |
| fi | |
| get_buster_pid (){ | |
| echo `ps aux|grep $BUSTER_CMD|grep node|awk '{ print $2 }'` | |
| } | |
| get_phantom_pid (){ | |
| echo `ps aux|grep $PHANTOM_CMD|grep buster|awk '{ print $2 }'` | |
| } | |
| case "$1" in | |
| "start") | |
| echo "Starting buster server…" | |
| buster-server & | |
| sleep 4 # It takes a while for buster server to start | |
| echo "Starting phantomjs and pointing it at buster…" | |
| phantomjs $BUSTER_PHANTOMSCRIPT_PATH &>/dev/null & | |
| ;; | |
| "stop") | |
| echo "Killing buster and phantom servers" | |
| kill `get_buster_pid` | |
| kill `get_phantom_pid` | |
| ;; | |
| *) | |
| echo "Usage: | |
| $0 start|stop" | |
| ;; | |
| esac |
Final thoughts
It should be quite easy™ to run the tests on more platforms, perhaps even using a browser provider such as Browserstack or Testling.
If you have an Open Source project going on, you can run the tests using Travis CI, see this guide by QMetric.
And the result? Beautiful graphs on the project page, sure, but more importantly, the tests are run every time your code changes, and you get feedback when something is wrong.
How do you run your JavaScript tests? Let us know in the comments!
The Iterate Blog 
is there anyone pay attention on Testacular? The idea of Testacular is based on js-test-driver and made a great improvement. Testacular is based on Nodejs and Socket.io
It also support Jenkins and multiple browsers and of coz PhantomJS.
Unless you’re really enjoying setting up all those things in Jenkins, I recommend Semaphore (https://semaphoreapp.com). We have many users running their tests against PhantomJS, plus all kinds of JavaScript tests. It takes no set up at all.
Minor correction: PhantomJS is not based on V8.
Wow, do not know where I got that from. It is corrected in the post now.
As a sidenote, I see that someone has posted a question on the PhantomJS site on changing JavaScriptCore with V8.
Anyways, thank you for the great work you have done on PhantomJS!
this blog would be greatly improved if there were no ads.
Thanks! The blog is upgraded as of one minute ago, no more ads 🙂
This won’t work if you have multiple Jobs running your Javascript tests right? You could have Job B kill the buster server before Job A is still running the tests. (Which causes buster test to hang forever (https://github.com/busterjs/buster/issues/323))
Instead we take a different approach and utilise the PhantomJS web server, see the following pseudo shell script:
`get the pidof buster-server
if the pidof buster-server is nothing
killall instances of buster-test
# restart buster server
nohup start buster server &
fi
# Try and start phantomjs if not already
pidof phantomjs || nohup phantomjs ourcustomscript.js > /dev/null &
# make a ‘reset’ request to PhantomJS. Internally this ONLY takes action if it is not captured, essentially this call captures PhantomJS if it is not already.
curl “http://jenkins:1112/?reset -o /dev/null`
This works really well, and we are able to run at least 4 Jobs in parallel without any conflict. Very occasionally Buster gets into a state that Jenkins can’t recover from, usually down to issue 323. (But at the moment the benefits outweigh the issue of restarting a Job every couple of days :P)
That’s a really good point, thanks for commenting!
This has not yet been an issue for us yet, as we have a separate slave set up with number of executors set to 1.
We will definitively look into your tip though, thank you!