Testing Ajax Applications
SPA 2009
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
INTRODUCTIONS
2
© 2009 CAPLIN SYSTEMS

Who we are
 Patrick Myles
Head of Engineering, Caplin Systems
 Andrew MacLeod
JavaScript Architect, Caplin Systems
3
© 2009 CAPLIN SYSTEMS

How this talk came about
 We have a big Ajax application
 We use Agile
 We needed a way to test each release
This is the overview we wish
we’d had 2 years ago
4
© 2009 CAPLIN SYSTEMS

This is not a Selenium HOWTO
 Just enough Selenium to:
 understand the grammar
 promote a discussion on frameworks
 see how good Selenium is!
 Not enough Selenium to:
 lose non-programmers (e.g. QA engineers)
 get bogged down in Selenium nuts-and-bolts
 see how frustrating Selenium is!
5
© 2009 CAPLIN SYSTEMS

Frustration
Browsers were never designed as
platforms for automated tests...
...so browser test tools are a hack!
6
© 2009 CAPLIN SYSTEMS

We would like you to leave with...
 Confidence
 automated testing
 hands-on experience
 Ideas
 designing tests
 tool selection
 building a framework
 planning a test programme
7
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
THE PROBLEM
8
© 2009 CAPLIN SYSTEMS

The Problem
 How can we ensure that each release of
our application works everywhere?
9
© 2009 CAPLIN SYSTEMS

Manual Testing
DOM
Test Browser
Pack Application
Tester
(acting as the User)
 Repeat for each new feature
 Repeat for all supported platforms .
10
© 2009 CAPLIN SYSTEMS

Automated Testing
DOM
Test Browser
Script Application
Robot
(acting as the User)
 How intelligent is the robot?
 How human readable is the script?
 How much effort does the robot consume? .
11
© 2009 CAPLIN SYSTEMS

Tools
Test Behaviour Driven
FIT
Fitnesse
Framework Development
COM
Cucumber
WebTest
browser control Domain Specific
rspec Language (DSL)
Watir
Selenium hamcrest
TestNG
WebDriver
Mercury JSUnit
HttpUnit
WindowLicker
Windmill
Auto-Hotkey LoadRunner
DOM
browser control
Screen coordinates
12
© 2009 CAPLIN SYSTEMS

Workshop
Why are browser applications
so difficult to test?
13
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
CAPLIN: 5 MINUTE CASE STUDY
14
© 2009 CAPLIN SYSTEMS

Caplin Trader
 Quick demonstration

Caplin’s Timeline
2007 2008 2009
Scope
Smoke Test (continuous integration)
Functional Tests (nightly)
Performance Tests
Authors Apps
Reference Implementation
Bespoke Implementations (up to 3 in parallel)
Development
QA
Framework
Selenium Framework v1 Selenium Framework v2
16
© 2009 CAPLIN SYSTEMS

Our first test...

... a more recent test
goal: Use product finder portlet
do Login (${username}=”user1\", ${password}=\"tester\")
do CreateNewLayout
do Insert (${portlet}=“Product Finder”)
do AssertPortletPresent (${portlet}=“Product Finder”)
do Logout

Testing AJAX Applications
EXERCISE 1:
SELENIUM IDE: SEND AN EMAIL
19
© 2009 CAPLIN SYSTEMS

Objective
 To get familiar with the Selenium IDE
 To produce a simple test script
 To review the script afterwards
 Experience some Ajax testing pitfalls

Schema
interact
record
playback

User Story
“As a user,
I want to send an e-mail message
so that I can stay in touch with my friend”
22
© 2009 CAPLIN SYSTEMS

Golden Rule
Put yourself in the user’s position
 write the test first
 watch it fail
 fix it
23
© 2009 CAPLIN SYSTEMS

Tools: Firefox Add-ons
 Firefox Selenium IDE
 easy authoring tool
 Firebug debugger
 reverse engineering tool
 XPather
 locates UI elements
24
© 2009 CAPLIN SYSTEMS

Demo
 Open Firefox IDE
 Record
 Save
 Playback
25
© 2009 CAPLIN SYSTEMS

Setup…
 Disconnect from LAN
 Connect to CaplinSpa2009 wireless
network
 Plug USB key
 Open Firefox + Selenium IDE
 (set ?:\\Examples\\user-extensions.js)
 Navigate to http://spa2009.com/webmail
26
© 2009 CAPLIN SYSTEMS

Your Turn…
 Login:
 user: spa1
 password: caplin2009
 Send mail To: patrick@spa2009.com
 Insert appropriate actions:
 pause
 waitForElementPresent
 waitForEditable
 Save in /Exercise1-Basic/
27
© 2009 CAPLIN SYSTEMS

Selenium IDE Review
Pros Cons
 
easy to set up no explicit verification
 
easy to write locators are unintelligible
 
easy to run brittle
 
little Selenium knowledge unstructured
required  desktop orientated
 generates code!  feature must exist
28
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
SELENIUM: 5 MINUTE TUTORIAL
29
© 2009 CAPLIN SYSTEMS

Selenium Test Script
One step
per line...
action thing to optional
act on argument
30
© 2009 CAPLIN SYSTEMS

Actions
 Selenium
Selenium.prototype.doType(sInputLocator, sInputText)
Selenium.prototype.doClick(sLocator)
 Custom Framework
Selenium.prototype.doClickControl(sLocator)
31
© 2009 CAPLIN SYSTEMS

Locators
 Selenium
xpath
id
link
...
 Custom Framework
PageBot.prototype.locateElementByMenuBarItem(sLocatorText, oDocument)
32
© 2009 CAPLIN SYSTEMS

Accessors
 Selenium
waitFor
store ElementPresent
assert
verify
Selenium.prototype.isElementPresent(sLocator)
 Custom
waitFor
store
MessageCount
assert
verify
Selenium.prototype.getMessageCount(sLocator)
33
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
SCRIPTS:
INTERPRETERS AND RUNNERS
34
© 2009 CAPLIN SYSTEMS

Script: Machine vs. Human readable
Pianola
(acting as pianist)
Piano Roll
35
© 2009 CAPLIN SYSTEMS

After the break...
Linguistic Application
Constructs Abstractions
• locator
strategies
• structure
• application
• functions
operations
• control flow
• state
transitions
DOM
Source Browser
compiler Interpreter Runner
Code COM Application
36
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
COFFEE BREAK (10 MINUTES)
37
© 2009 CAPLIN SYSTEMS

Design Principles
Testing Ajax applications
38
© 2009 CAPLIN SYSTEMS

Design Principles
 Domain Hierarchies
 how to make my tests more readable?
 modeling vs. maintainability?
 how can I isolate application changes?
 Test structure and composition
 how do I avoid duplication?
 how do my tests map to my requirements?
 how can I make my framework scalable?
39
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
DOMAIN HIERARCHIES
40
© 2009 CAPLIN SYSTEMS

?
Test framework
Verification Domain
(Locate, input, verify, etc.)
Test tool
Browser Domain
(DOM, text nodes, clicks, etc.)
Application
System Domain
(OO language, may model business domain)
41
© 2009 CAPLIN SYSTEMS

What to use for framework domain?
1. Don’t bother! ...
42
© 2009 CAPLIN SYSTEMS

?
Test framework
Verification Domain Test framework
(Locate, input, verify, etc.)
Test tool
Browser Domain
(DOM, text nodes, clicks, etc.)
Application
System Domain
(OO language, may model business domain)
43
© 2009 CAPLIN SYSTEMS

What to use for framework domain?
1. Don’t bother...
Quick to get going
Can use off-the-shelf tools (e.g. Selenium
IDE)
Lots of duplication in tests
Maintenance nightmare
Brittle tests
Rapidly become hard to read and author
44
© 2009 CAPLIN SYSTEMS

What to use for framework domain?
2. Mirror of system domain...
45
© 2009 CAPLIN SYSTEMS

??
System Domain (again)
Test framework
Verification Domain
(Locate, input, verify, etc.)
Test tool
Browser Domain
(DOM, text nodes, clicks, etc.)
Application
System Domain
(OO language, may model business domain)
46
© 2009 CAPLIN SYSTEMS

What to use for framework domain?
2. Mirror of system domain...
Usually in same language as System
Seems attractive – particularly if System is
well modelled on problem domain
Can “share” domain objects
Often ends up with either tight coupling...
...or duplicated code / abstractions on both
sides of the Browser/Verification divide
Using same language as System doesn’t
reinforce the domain separation
47
© 2009 CAPLIN SYSTEMS

What to use for framework domain?
3. Model or DSL that reinforces user’s
interactions and use-cases...
48
© 2009 CAPLIN SYSTEMS

?
Business (User) Domain / DSL
(Inbox, compose, check mail, logout, etc.)
Test framework
Verification Domain
(Locate, input, verify, etc.)
Test tool
Browser Domain
(DOM, text nodes, clicks, etc.)
Application
System Domain
(OO language, may model business domain)
49
© 2009 CAPLIN SYSTEMS

What to use for framework domain?
3. Model or DSL that reinforces user’s
interactions and use-cases...
Easier to understand for non-programmers
Less brittle when underlying domains change
More natural abstractions
Requires more maintenance of different
abstractions
Can make it hard to implement complex
interactions
50
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
EXERCISE 2:
USING A FRAMEWORK
51
© 2009 CAPLIN SYSTEMS

Objectives
 Move test script closer to application domain
 Introduce logical names for screen controls
 Remove physical references to screen controls
 Show Selenium extensions
 Make tests more readable
52
© 2009 CAPLIN SYSTEMS

Selenium Framework
Application
Abstractions
• actions
• locators
DOM
Source Browser
Interpreter Runner
Code Application
COM
53
© 2009 CAPLIN SYSTEMS

Exercise 2 - Framework
 Demo
54
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
UCD TEST STRUCTURE
55
© 2009 CAPLIN SYSTEMS

User Centered Design (UCD)
 A powerful way to model and design a
system
 Like Agile, focuses on Users – their needs
and goals
 Typically used for UI/UX design, but
provides great models for test frameworks
 We use with a simple text DSL
56
© 2009 CAPLIN SYSTEMS

User Centered Design (UCD)
 Brings focus back to users and their
 Goals
 Tasks
• Actions
 Fits well with user story structure:
As a [type of user]
I want to [a task or sequence of tasks]
so that I can [achieve some goal]
57
© 2009 CAPLIN SYSTEMS

Use-Cases and UCD
UCD concepts map
well to use-case
levels
Goal
Task
Action
58
© 2009 CAPLIN SYSTEMS

Story Structure
Tasks
Goal Actions
Send Email Login Action
Action
Send Message Action
Action
Action
Logout Action
Action
59
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
EXERCISE 3:
RUNNING A UCD TEST
60
© 2009 CAPLIN SYSTEMS

Objectives
 Demonstrate UCD structure
 Show task re-use
 Implement a new task
61
© 2009 CAPLIN SYSTEMS

Old Structure
Test Action
Action
Action
Action
Action
Action
Action
Action
62
© 2009 CAPLIN SYSTEMS

UCD Structure
Goal Task Action
Action
Task Action
Action
Action
Task Action
Action
63
© 2009 CAPLIN SYSTEMS

Demonstration
 Goal/Task/Action hierarchy
 Task level convenient for
 Reuse
 Maintenance
 Communication
 Action level too small to be reusable
64
© 2009 CAPLIN SYSTEMS

UCD Structure
Send Email Login Action
Action
Send Message Action
Action
Action
Verify Sent Action
Action
Logout Action
Action
65
© 2009 CAPLIN SYSTEMS

A new story…
“As a user,
I want to receive an email
so that I can prove I have a friend!”
66
© 2009 CAPLIN SYSTEMS

Your Turn...
Receive Email Login Action
Action
Action
Verify Inbox
non-empty
Action
Logout Action
Action
67
© 2009 CAPLIN SYSTEMS

Testing AJAX Applications
SUMMARY
68
© 2009 CAPLIN SYSTEMS

Summary
 Selenium is not a perfect testing tool, but…
 Selenium extensions are effective
 actions
 locators
 accessors
 UCD is a good way to structure test cases
 goals
 tasks
 actions

Taking things forward…
 Develop a simple DSL for testers to use
 invoke tasks with parameters
 get rid of ugly HTML “Selenese”
 use a PATH mechanism to locate tasks
-- goal: use product finder portlet
do Login (${username}=”user1\", ${password}=\"tester\")
do CreateNewLayout
do Insert (${portlet}=“Product Finder”)
do AssertPortletPresent (${portlet}=“Product Finder”)
do Logout