Browser automation testing in javascript- selenium, webdriverJs

I am adding automation testing the for my application and I have also been the online testing “champion” in my workplace (Java).

As usual, Concept first

This is stil a relative fresh field. There are quite a lot of concerns, I try to write one by one and be concise.

Why browser automation testing

I am not very expereinced in unit-testing or javascript. IMO, it may be a bit too expensive, especially when you have simple code and what you focus is the webpage being shown to users and things work there. It is expensive in a sense that comparing to JAVA, again IMO, it is harder to yield in automated refactoring and it is easily coupled with DOM.

There are nice behaviour testing framework like Jasmine. Some of them are also possible to run in browsers. But still that may not be the same as browser automation testing, which test against USER ACTIONS in browsers. You use the real page to test can care less about the fixtures.

Selenium

Nowadays selenium, from Google, is the de facto standard for that. I am used to it in Java and it is roboust, easy to write and scalable. Even run in headless Linux is possible . Alternatives: JsTestServer

Why browser automation testing with JS

I am building couchdb app (with node.js tools) and simpler, javascript is the language of web.

Selenium Javascript port (client/binding)

There are ports to use selenium in javascript. Almost all are based on the OfficialJsonWireProtocol

So the story is: your javascript test code run with the selenium webdriver client, which communicates with the selenium server via the JsonWireProtocol, which then issue requests to the connected browsers and carry out the actions.

Frameworks out there..

At first I was quite confused as they all name as webDriverJs. So some quick facts.

All are available as node component, and in NPM:

official selenium-webdriver-js – actively developing

Camme/WebdriverJs – last commit Sep2012

**Update: per author’s comment it is now active again :)**

WD.js – actively developing

Some footprint analysis in github: not very popular in wild

  • selenium-webdriver 38 code results
  • webdriverjs: 142 code results

Which to use?

Sometimes it is not the matter of which API to use, rather it is more on understanding the tradeoff and how you make them decouple. IMO things are still not very mature today and I am thus suggesting for a even simpler framework. Lets dive in more…

Browser Automation Testing Concerns

These CONCERNS should be DECOUPLED.

client API

Basically all the clients framework are async and used a queue-like implementation. So your script will finish but it actually just registered the actions for selenium. Then selenium will wait and call the callbacks at right time.

Comparing the official one and Camme/WebdriverJs, the former one is more sophisticated with concept of “Promise” and “Defer”, but this may also be harder to understand and write. Meanwhile the later has no explicit wait method and this is quite a large limitation in testing against ajax.

As you can see in the WebDriverJs wiki, concerns are problem of state, API too verbose, too many nested callback and how they are currently handled. I think the official client did a great job.

It is also for sure that if you use the official API, it is expected to be more updated with selenium..

Selector Style

There are many methods to select an element in selenium. Name/Id/Css Selectors.

The official webdriverJs’s basic API use the common selenium API driver.findElement(webdriver.By.name('q')); where the advantange is it is consistent even you come from Java.

On the other hand it may be quite strange to write in javascript, esp content assist is unlikely in place and the builder pattern makes you hard to debug & see available methods. There is also lack of documentation.

On this I prefer Camme/WebdriverJs which is more jQuery-like.

Also it may be to your surprise, it is possible for selenium to execute javascript. By this I mean you can have the script as a string and let it be executed by selenium server for you, no matter from Java/Python/etc, of course in javascript. This is quite handy in some situations, but some common use case like getting the value back into the place you call could be hard.

jQuery?

Then one might ask is that possible to use jQuery?

First we need to separate the questions into 2: 1. to use in test script 2. to run in browser

For 1, you can always make jquery available in Node.js by require('jquery'). The point is you cant use jQuery selector directly to write the test flow, as actual actions are done by selenium instead

For 2, it may be hard as you can imagine the dependency.

You may argue that jQuery is not necessary because selenium support css selectors already. but some methods are not, like .first() or .show(). I believe this can greatly save developer’s time.

I just tried to check if a element is present in the official webdriver, which is quite hard.

So the tradeoff:consistency amond languages vs native style. I will prefer and suggest to have a selenium client like jQuery as much as possible.

Testing – PageObject

It is imporant to separate test code and automation code.

This official reference to PageObject is extremely helpful.

I tried to write my own js page-object, but that may be harder as to duel with the scope and state.

this js-page-object project also seems interesting.

Testing – assert Style

Afer all this is a test case. Camme/WebdriverJs provides a test mode. This may be very convenient to keep scripts compact and with less dependencies.

Example

    client
    .testMode()
    .init()
    .url("https://github.com")
    .tests.cssPropertyEquals(".login a", "color", "#4183c4", "Color of .login a is #4183c4")
    .tests.titleEquals("Secure source code hosting and collaborative development - GitHub", "Title of the page is 'Secure source code hosting and collaborative development - GitHub'")
    .click(".pricing a")
    .tests.titleEquals("Plans & Pricing - GitHub", "Title of the page is 'Plans & Pricing - GitHub'")
    .tests.visible(".pagehead", true, ".pagehead is visible after click")
    .end();

For me, I used in hybrid with nodeunit.

For the general assert(actual expected) vs assert(expected, actual) war link1 , the former is what in Node and nodeunit. I am from Java but I think I will stick with Node’s approach from now on.

One can use the tearDown() method to make sure the browser is killed after test, even in case of exceptions.

Conclusion

I will try to write more and make it a “white paper”

— suggestion is lets use the official API and look forward to its API wrappers ang page-object implementatio. May make one myself if I have time and I really need it.