This talk is concerned with automated testing of Web applications. It looks at testing Web apps in general, its goals and challenges; it will present Selenium and Selenium RC in particular as a testing platform; and will then focus on adaptions made to Selenium to ease the effort to test apps made with qooxdoo, a JavaScript framework.

1. Selenium RC: Automated Testing
of Modern Web Applications
Thomas Herchenröder, 1&1 Internet AG

2. Overview
● Testing Web Apps
● Selenium RC
● Extending Selenium RC

3. Overview
● Testing Web Apps
● Selenium RC
● Extending Selenium RC

4. Web Testing: Goals and
Challenges
Goals:
● create tests
● run automatically, repetitively
● regression testing
● compare results

5. Web Testing: Goals and
Challenges
Challenges:
● GUI app testing
● OS/browser/platform
● JS engines/versions/levels, plug-ins

6. Web Testing: Approaches
● GUI Automation
● locating by coordinates
● using generic OS mouse/keyboard events
● suitable for any GUI app
● HTML Automation
● parsing HTML
● evaluating HTML elements (forms, ...)
● “batch browsers”
● DOM Automation
● DOM as presentation of application
● DOM traversal (getElementByName, ...)
● interacting with DOM tree
(target.dispatchEvent,...)

7. A Simple Example: GUI
Automation

8. A Simple Example: HTML
Automation

9. A Simple Example: DOM
Automation

10. A Complex Example: GUI
Automation

11. A Complex Example: HTML
Automation

12. A Complex Example: DOM
Automation

13. Web Testing: Approaches (Rev.)
● GUI Automation
● constant complexity
● platform-dependent
● fragile
● HTML Automation
● insufficient content capture
● DOM Automation
● DOM tree represents application
● sufficient content capture

14. Web Testing: DOM Automation
● full access to app elements
● full capture of dynamic content
● issue: locating elements (reliably)
● issue: non-interference with AUT
● you can embed test code with AUT
● issue: reliability of results
● (i)frames
● cross-frame interaction

15. Overview
● Testing Web Apps
● Selenium RC
● Extending Selenium RC

16. Selenium: selenium.openqa.org
● Selenium Core
● Selenium IDE
● Selenium on Rails
● Selenium RC
● Selenium Grid
● open source, OpenQA hosted,
ThoughtWorks

17. Selenium RC: Architecture

18. Selenium RC
● overcome SOP restriction
● drive tests programmatically
● language bindings (Java, Python, Perl,
...)
● access to OS features (timers, threads,
screen shots, ...)
● user-extensions.js

19. Selenium RC: The Browser

20. Selenium RC: An Interactive
Session
● Starting the server:
● java -jar selenium-server.jar -interactive
● Interactive commands:
● cmd=getNewBrowserSession&1=*firefox&2=http://d
emo.qooxdoo.org
● cmd=open&1=http://demo.qooxdoo.org/current/sho
wcase
● cmd=qxClick&1=//div[text()=quot;Tabquot;]
● cmd=testComplete
● quit

21. Overview
● Testing Web Apps
● Selenium RC
● Extending Selenium RC

22. Selenium and qooxdoo
● crucial issues:
● object identification
● event generation
● qooxdoo rendering: a world of <div>s
● Selenium commands
● Selenium locators
● the need for (and feasibility of)
qooxdoo specific extensions

23. user­extensions.js: commands
Selenium.prototype.doQxClick =
function(locator, eventParams)
{
var element =
this.page().findElement(locator);
this.clickElementQx(element,
eventParams);
};

24. user­extensions.js: locators
PageBot.prototype.locateElementByQx =
function(qxLocator, inDocument, inWindow)
{
LOG.info(quot;Locate Element by qooxdoo UserData
Locator”);
var qxObject =
this._findQxObjectInWindow(qxLocator, inWindow);
if (qxObject) {
return qxObject.getElement();
}
};

25. user­extensions.js: language
bindings
● new custom extensions:
● support for qx locator
● just a string passed as parameter of
command
● no change to client driver API
● support for qxClick command
● must be supported in language
bindings
● so client drivers can use it
● Sample:
● qxClick(“qx:foo/bar/baz”)

26. user­extensions.js: language
bindings
● Python:
● def qxClick(self, locator, opts=None):
return self.do_command(quot;qxClickquot;, [locator,])
● sel.qxClick(r'//div[text()=quot;Formquot;]')
● Java
● public String qxClick(String locator) {
return this.commandProcessor.doCommand(quot;qxClickquot;,
new String[] {locator,}); }
● sel.qxClick('qxh=*/[@label=quot;Tabquot;]/[@page]/*/[@label=quot;Fi
ndquot;]');
● JavaScript using Rhino
● uses Java infrastructure, so no additional overhead

27. A Client Driver Script
importClass(Packages.com.thoughtworks.selenium.QxS
elenium);
var sel = new QxSelenium(quot;localhostquot;,4444,quot;*firefoxquot;,
quot;http://demo.qooxdoo.orgquot;);
sel.setSpeed(quot;1000quot;);
sel.start();
sel.open(quot;http://demo.qooxdoo.org/0.7.3/showcase/quot;);
Packages.java.lang.Thread.sleep(2000);
sel.qxClick('//div[text()=quot;Tabquot;]');
sel.qxClick('qxh=*/[@label=quot;Tabquot;]/[@page]/*/[@label=
quot;Findquot;]');
sel.stop();

28. Conclusion
● Interacting with the DOM seems to be
the best way to test Web applications
● Selenium RC provides a distributed
environment to run tests on a broad
range of platforms
● Various language bindings exist, so
tests can be written as stand-alone,
client-server programms in a preferred
language
● Custom commands and locators can be
added to the Selenium environment, to
suite specific testing needs

29. Selenium RC: Automated Testing of
Modern Web Applications
Thank you!
Thomas Herchenröder, 1&1 Internet AG

30. Selenium RC: Automated Testing
of Modern Web Applications
Thomas Herchenröder, 1&1 Internet AG
This talk is concerned with automated testing of Web applications. It
looks at testing Web apps in general, its goals and challenges; it will
present Selenium and Selenium RC in particular as a testing platform;
and will then focus on adaptions made to Selenium to ease the effort
to test apps made with qooxdoo, a JavaScript framework.

31. Overview
● Testing Web Apps
● Selenium RC
● Extending Selenium RC

32. Overview
● Testing Web Apps
● Selenium RC
● Extending Selenium RC
Part I: An Introduction to testing Web applications and its approaches.

33. Web Testing: Goals and
Challenges
Goals:
● create tests
● run automatically, repetitively
● regression testing
● compare results
●When we speak of creating automated tests for Web applications
(“application” in the broadest sense – from a static HTML page
through CGI-driven applications to full-blown single-page apps), what
do we mean by the term 'test'?!
●All test start by retrieving a specific web page.
●The simplest test might then just look at its static contents,
inspecting the page and checking for specific properties (e.g. the
correct title, presence of keywords or phrases, etc.).
●More advanced tests will want to look at the dynamic aspects of the
web page, and therefore need some means to interact with it. This
encompasses detecting and following hyperlinks, automatically filling
in and submitting forms, or use other interactive elements. In all
cases, the outcome of such interaction (new page, change of current
page content, ...) has to be checked against some expected result.
●Depending on these requirements, test methodology and effort can
vary a lot.
●Once you have created a test in a suitable form, its purpose is to be
run repetitively and, hopefully, automatically, i.e. without user
intervention. Results need to be verified, and probably captured, to be
compared wit subsequent test runs.
●This allows for regression testing and test-driven development, to
ensure existing functionality is not broken while the web app is further
developed.

34. Web Testing: Goals and
Challenges
Challenges:
● GUI app testing
● OS/browser/platform
● JS engines/versions/levels, plug-ins
●Every tester loves APIs! A clear defined interface that can be driven
programmatically and in a defined and reproducible way. Results can
be compared, collected, logged, standardized, or put in a database.
The scope of testing is clearly defined (list of API calls), and though
parameter ranges and call protocols (sequencing of calls) make things
more complex, there often is hope for test coverage (getting all –
important – test cases).
●GUI apps evade from such a close grip. Although they often have a
number of distinct interaction elements (buttons, boxes, fields, ...),
they rely on event handling that in most cases defies protocols and a
systematic way of testing. Events can be triggered in arbitrary order,
and might – or might not – lead to the same results.
●Moreover, any GUI can rely on one of the most versatile interpreters
available – the human brain. Pictures with colored dots are recognized
as numbers, technically unrelated geometrical elements are
recognized as text, context is maintained across pages or parts of a
page, across time and across any change happening to a page. With
this kind of semantic processing, correlations can be established and
information can be provided that is very hard to capture by any known
automaton. How can 'correctness' or its absence be detected in such
a context?
●To worsen things, in the realm of web applications the tester is
confronted with the 'plague of platforms': multiple OSs, multiple
desktops, multiple browsers, multiple configurations, multiple ... To
devise tests in this scattered landscape is hard, to devise tests that are
maintainable and provide reliable results in all situations is very hard.

35. Web Testing: Approaches
● GUI Automation
● locating by coordinates
● using generic OS mouse/keyboard events
● suitable for any GUI app
● HTML Automation
● parsing HTML
● evaluating HTML elements (forms, ...)
● “batch browsers”
● DOM Automation
● DOM as presentation of application
● DOM traversal (getElementByName, ...)
● interacting with DOM tree
(target.dispatchEvent,...)
●There are mainly three approaches to tackle the problem of web
testing:
●GUI Automation:
● uses the least common denominator of GUIs: generic mouse and
keyboard events.
● uses OS-level I/O event generation.
● doesn't care whether it is Web browser, Chat client, File
browser, MP3 player, etc.
● but is entirely platform-dependent, since it relies on OS-level
APIs.
● result verification is hard (comparing bitmaps?!)
● fragile: depends on correct coordinates for mouse actions,
keyboard layouts, themes in place, current key bindings, ...

36. A Simple Example: GUI
Automation
Click to add text
●Locate input field, push search button - GUI automation would
certainly work
●But: what about result verification?!

37. A Simple Example: HTML
Automation
Click to add text
●fair reflection of the rendered page:
● opening “body” tag
● navigational links (Images, Maps, ...), with embedded menu
(“more”)
● “form” element with input field, buttons
● further information links
●functional “meat” is present: input field, buttons

38. A Simple Example: DOM
Automation
Click to add text
●Interesting elements: HTML->Body->Center->Form->...->TD:
● Input: hidden
● Input: text field for query string
● Input: submit button
● Input: I'm feeling lucky button
●uses name attribute for input fields ('q' for text field, 'btnG' for
submit etc.)
●makes it easy to locate (getElementByName(), Xpath, ...)
●Conclusion:
● good “semantic” representation of the page and its interactive
elements (not layout dependend!)
● easy and robust to drive through DOM manipulations

39. A Complex Example: GUI
Automation
Click to add text
●Complex GUI:
● buttons, tabs, panes
● multiple embedded “windows”, independently movable
● check-/radio-buttons, pop-ups
●Tip: Try imagining this in HTML!
●GUI Automation:
● easy, doesn't care about complexity
● problematic: exact locating
● (influence of screen resolution?)
● problematic: re-arranging of elements (e.g. tabs)
●(Next Slide: corresp. HTML - “Prepare for disappointment!”)

40. A Complex Example: HTML
Automation
Click to add text
●HTML: That's all there is to it! :-(
●nothing to work with
●all HTML-processing programms will fail
●Conclusion: completely unusable

41. A Complex Example: DOM
Automation
Click to add text
●again, full access to page elements
●harder to relate DOM elements to visual result
● DOM tree as “assembler”
●problematic: locating elements (“flood of divs”)
●remedy: “id”, “name” etc. attributes
●remedy: Xpath queries for “robust” features (unique labels, images
etc.)
● still not comfortable, but works

42. Web Testing: Approaches (Rev.)
● GUI Automation
● constant complexity
● platform-dependent
● fragile
● HTML Automation
● insufficient content capture
● DOM Automation
● DOM tree represents application
● sufficient content capture
●GUI Automation:
● use the least common denominator of GUIs: generic mouse and
keyboard events.
● Use OS-level I/O event generation.
● Doesn't care whether it is Web browser, Chat client, File
browser, MP3 player, etc.
● But entirely platform-dependent, since it relies on OS-level APIs.
● Result verification is hard (comparing bitmaps?!)
● Fragile: Depends on correct coordinates for mouse actions,
keyboard layouts, themes in place, current key bindings, ...
● Impossible to write one test that runs the same on
Win/Linux/OSX/... in IE/FF/Safari/...
●HTML Automation:
● just not feasible for modern applications
●DOM Automation:
● provides everything needed
● (more next slide)

43. Web Testing: DOM Automation
● full access to app elements
● full capture of dynamic content
● issue: locating elements (reliably)
● issue: non-interference with AUT
● you can embed test code with AUT
● issue: reliability of results
● (i)frames
● cross-frame interaction
Overview of some features of the DOM automation approach:
●everything in the app is accessible through its DOM representation
●even across dynamic changes
●finding the element to interact with can be a challenge, esp. across
various revisions of the same app
●non-interference: if testing code changes DOM, the app is no longer
the same; danger of introducing subtle changes in behaviour
● test code could be part of the app (and this is actually used!)
● will AUT behaviour be the same as when driven by a real user?
● one solution: separation of AUT and testing logic in different
frames
● issue here: cross-frame interaction

44. Overview
● Testing Web Apps
● Selenium RC
● Extending Selenium RC
Part II: An Introduction to Selenium and Selenium RC in particular.

45. Selenium: selenium.openqa.org
● Selenium Core
● Selenium IDE
● Selenium on Rails
● Selenium RC
● Selenium Grid
● open source, OpenQA hosted,
ThoughtWorks
●Selenium Core:
● JavaScript engine
● runs in browser
● controls AUT in other frame
● exposes API
●Selenium IDE:
● FF plug-in
● directly communicates with Selenium Core in the browser
● record/replay features
● great to get started (interactively)
●Selenium RC:
● remote control
● server component, proxy
● allows test automation in various ways
● this is what we will focus on
● has a capability called Selenium Grid, to drive multiple instances
of the SRC server
●project frame: open source (Apache 2.0 lic.), OpenQA run by
collaboration of a few companies

46. Selenium RC: Architecture
●Selenium RC server (blue)
● starts browser instance with modified profile
● promotes itself as proxy for all requests
● injects Selenium Core with startup page
● all requests go through server, dispatches on URL
●browser instance (orange)
● initial page installs Core, multi-(i)frame
● Core maintains connection with server (comet-like, long polling)
● receives commands and performs them in browser (load URL in
AUT frame, click, type, check results, ...)
●web server hosting the AUT (brown)
● SRC server appears as normal client
●client drivers (green)
● multiple language bindings (Java, Python, Perl, ...)
● completely control SRC server
● starting of browser
● loading of AUT
● any (test) interaction
● ending of browser session

47. Selenium RC
● overcome SOP restriction
● drive tests programmatically
● language bindings (Java, Python, Perl,
...)
● access to OS features (timers, threads,
screen shots, ...)
● user-extensions.js
●SOP: same-origin-policy:
● JS can only make XHR requests to domain it was loaded from
● browser only sees a single domain:
http://<domain_of_aut>/selenium-
server/core/RemoteRunner.html?...
● SRC server dispatches on URL path
● even handles all requests to
http://<domain_of_aut>/<aut_path> (due to proxy)
● control ends when requesting a page in another domain
●user-extensions.js
● main topic of part III
●Hint: some more 'hands-on' material before

48. Selenium RC: The Browser
Click to add text
●default appearance
●three parts:
● AUT frame
● Log frame (command history)
● Selenium frame
● “show log” button, for message log
● important for debug messages
●Hint: next: log of interactive session to bring this up

49. Selenium RC: An Interactive
Session
● Starting the server:
● java -jar selenium-server.jar -interactive
● Interactive commands:
● cmd=getNewBrowserSession&1=*firefox&2=http://d
emo.qooxdoo.org
● cmd=open&1=http://demo.qooxdoo.org/current/sho
wcase
● cmd=qxClick&1=//div[text()=quot;Tabquot;]
● cmd=testComplete
● quit
●Starting the server:
● Selenium RC is a Java application (built atop of Jetty)
● So invoke Java, pass Selenium Server Jar as argument
● SRC takes arguments itself (-interactive for interactive shell,
others for AUT window handling, proxying, ... see SRC command
line reference)
●Interactive Commands:
● First, create a new browser session for this test.
● SRC can handle multiple sessions
● Yields SessionID
● (which is mandatory as a parameter for later commands
“...&ssessionId=4711”, but can be omitted in single-session
mode)
● is automatically kept in the Selenium object in client driver
code
● This will create a new browser instance running on the local
machine (FF: creating profile, IE: manipulating the Registry).
● Then, open the AUT in this browser session.
● This will load the web app in iframe/window, controlled by
Selenium Core.
● Perform various actions on the AUT, possibly checking outcomes
(mouse clicks, typing, loading pages, ...)
● Finally, end the browser session.
● Eventually, end SRC server.

50. Overview
● Testing Web Apps
● Selenium RC
● Extending Selenium RC
Part III: Extending Selenium RC through its user-extensions.js and
language bindings.

51. Selenium and qooxdoo
● crucial issues:
● object identification
● event generation
● qooxdoo rendering: a world of <div>s
● Selenium commands
● Selenium locators
● the need for (and feasibility of)
qooxdoo specific extensions
●event generation:
● Selenium: synthetic events on elements
● Sel events on an element (think 'div') might not match qooxdoo's
expected events on the element
● 'click' vs. 'mousedown', 'mouseup'
● background:
● qooxdoo establishes unified event model over AUT
● only top-most widget registers for all events
● qooxdoo handles event capture and bubbling internally
●Selenium commands:
● tell the browser what to do
●Selenium locators:
● determine where to apply a command
●qooxdoo specific extensions:
● exactly in these two areas
● new commands
● new locators
●Rest of talk: practical examples for these

52. user­extensions.js: commands
Selenium.prototype.doQxClick =
function(locator, eventParams)
{
var element =
this.page().findElement(locator);
this.clickElementQx(element,
eventParams);
};
●doQxClick:
● naming convention for extension commands: “doXxYy” if
command is “xxYy”
●locator: locator string from command
●eventParams: Selenium-provided eventParam object, filled from
command or defaults (think: mouse button, shift key, ...)
●findElement(): Sel function, returning DOM object reference
●clickElementQx(): custom method that triggers mouseover,
mousedown, mouseup, ... events on element (element.dispatchEvent(),
element.fireEvent())

53. user­extensions.js: locators
PageBot.prototype.locateElementByQx =
function(qxLocator, inDocument, inWindow)
{
LOG.info(quot;Locate Element by qooxdoo UserData
Locator”);
var qxObject =
this._findQxObjectInWindow(qxLocator, inWindow);
if (qxObject) {
return qxObject.getElement();
}
};
●locateElementByQx:
● naming conventions for locators: “locateElementByXxx” if
locator is “xxx:”
●qxLocator: locator string from Selenium command, specific to locator
“qx:”
●LOG: another Selenium interface object
●_findQxObjectInWindow():
● custom method
● parses qxLocator
● supports parent/child relation between widgets and “userData”
property of widgets
● depends on qooxdoo object hierarchy:
qx.ui.core.ClientDocument etc.
● returns JS object (not DOM!)
●qxObject.getElement(): stand. qooxdoo object method to return
corresp. DOM object
● to satisfy Sel locator interface

54. user­extensions.js: language
bindings
● new custom extensions:
● support for qx locator
● just a string passed as parameter of
command
● no change to client driver API
● support for qxClick command
● must be supported in language
bindings
● so client drivers can use it
● Sample:
● qxClick(“qx:foo/bar/baz”)
●you want to support the extended features in your test scripts
●new locator: just use in locator strings
●new command: need to define in language-specific API, in order to
use it in test scripts
●then use as method of Selenium test object

55. user­extensions.js: language
bindings
● Python:
● def qxClick(self, locator, opts=None):
return self.do_command(quot;qxClickquot;, [locator,])
● sel.qxClick(r'//div[text()=quot;Formquot;]')
● Java
● public String qxClick(String locator) {
return this.commandProcessor.doCommand(quot;qxClickquot;,
new String[] {locator,}); }
● sel.qxClick('qxh=*/[@label=quot;Tabquot;]/[@page]/*/[@label=quot;Fi
ndquot;]');
● JavaScript using Rhino
● uses Java infrastructure, so no additional overhead
●here are two examples of doing this:
●Python:
● the first item shows how to define the new API method
● the second how to use it in testing code
●Java:
● define the new method
● use it in testing code
● this one uses yet another locator that provides XPath-like
syntax to traverse the qooxdoo object hierarchy
●you can use JavaScript as implementation language for your tests,
atop of the Java language binding, through Rhino
(www.mozilla.org/rhino/)
●Hint: complete JS example on next slide

56. A Client Driver Script
importClass(Packages.com.thoughtworks.selenium.QxS
elenium);
var sel = new QxSelenium(quot;localhostquot;,4444,quot;*firefoxquot;,
quot;http://demo.qooxdoo.orgquot;);
sel.setSpeed(quot;1000quot;);
sel.start();
sel.open(quot;http://demo.qooxdoo.org/0.7.3/showcase/quot;);
Packages.java.lang.Thread.sleep(2000);
sel.qxClick('//div[text()=quot;Tabquot;]');
sel.qxClick('qxh=*/[@label=quot;Tabquot;]/[@page]/*/[@label=
quot;Findquot;]');
sel.stop();
●importClass() Rhino built-in to load Java classes; QxSelenium is
derived class from standard Selenium class, to hold user extensions
●create interface object
●set properties
●start() method will launch browser
●open the AUT
●use a standard platform (Java) call (“waiting” is inherently difficult in
browser-side JavaScript)
●invoke test commands with locators
●stop() brings down the browser

57. Conclusion
● Interacting with the DOM seems to be
the best way to test Web applications
● Selenium RC provides a distributed
environment to run tests on a broad
range of platforms
● Various language bindings exist, so
tests can be written as stand-alone,
client-server programms in a preferred
language
● Custom commands and locators can be
added to the Selenium environment, to
suite specific testing needs
● draw some conclusions

58. Selenium RC: Automated Testing of
Modern Web Applications
Thank you!
Thomas Herchenröder, 1&1 Internet AG