• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
SPA 2009 - Acceptance Testing AJAX Web Applications through the GUI
 

SPA 2009 - Acceptance Testing AJAX Web Applications through the GUI

on

  • 2,793 views

These are the slides that Andrew MacLeod and Patrick Myles presented at the SPA conference in London, April 2009

These are the slides that Andrew MacLeod and Patrick Myles presented at the SPA conference in London, April 2009

http://www.spaconference.org/spa2009/sessions/session219.html

Statistics

Views

Total Views
2,793
Views on SlideShare
2,785
Embed Views
8

Actions

Likes
0
Downloads
55
Comments
0

2 Embeds 8

http://www.slideshare.net 5
http://blog.caplin.com 3

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    SPA 2009 - Acceptance Testing AJAX Web Applications through the GUI SPA 2009 - Acceptance Testing AJAX Web Applications through the GUI Presentation Transcript

    • Testing Ajax Applications SPA 2009 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications INTRODUCTIONS 2 © 2009 CAPLIN SYSTEMS
    • Who we are  Patrick Myles Head of Engineering, Caplin Systems  Andrew MacLeod JavaScript Architect, Caplin Systems 3 © 2009 CAPLIN SYSTEMS
    • How this talk came about  We have a big Ajax application  We use Agile  We needed a way to test each release This is the overview we wish we’d had 2 years ago 4 © 2009 CAPLIN SYSTEMS
    • This is not a Selenium HOWTO  Just enough Selenium to:  understand the grammar  promote a discussion on frameworks  see how good Selenium is!  Not enough Selenium to:  lose non-programmers (e.g. QA engineers)  get bogged down in Selenium nuts-and-bolts  see how frustrating Selenium is! 5 © 2009 CAPLIN SYSTEMS
    • Frustration Browsers were never designed as platforms for automated tests... ...so browser test tools are a hack! 6 © 2009 CAPLIN SYSTEMS
    • We would like you to leave with...  Confidence  automated testing  hands-on experience  Ideas  designing tests  tool selection  building a framework  planning a test programme 7 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications THE PROBLEM 8 © 2009 CAPLIN SYSTEMS
    • The Problem  How can we ensure that each release of our application works everywhere? 9 © 2009 CAPLIN SYSTEMS
    • Manual Testing DOM Test Browser Pack Application Tester (acting as the User)  Repeat for each new feature  Repeat for all supported platforms . 10 © 2009 CAPLIN SYSTEMS
    • Automated Testing DOM Test Browser Script Application Robot (acting as the User)  How intelligent is the robot?  How human readable is the script?  How much effort does the robot consume? . 11 © 2009 CAPLIN SYSTEMS
    • Tools Test Behaviour Driven FIT Fitnesse Framework Development COM Cucumber WebTest browser control Domain Specific rspec Language (DSL) Watir Selenium hamcrest TestNG WebDriver Mercury JSUnit HttpUnit WindowLicker Windmill Auto-Hotkey LoadRunner DOM browser control Screen coordinates 12 © 2009 CAPLIN SYSTEMS
    • Workshop Why are browser applications so difficult to test? 13 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications CAPLIN: 5 MINUTE CASE STUDY 14 © 2009 CAPLIN SYSTEMS
    • Caplin Trader  Quick demonstration
    • Caplin’s Timeline 2007 2008 2009 Scope Smoke Test (continuous integration) Functional Tests (nightly) Performance Tests Authors Apps Reference Implementation Bespoke Implementations (up to 3 in parallel) Development QA Framework Selenium Framework v1 Selenium Framework v2 16 © 2009 CAPLIN SYSTEMS
    • Our first test...
    • ... a more recent test goal: Use product finder portlet do Login (${username}=”user1quot;, ${password}=quot;testerquot;) do CreateNewLayout do Insert (${portlet}=“Product Finder”) do AssertPortletPresent (${portlet}=“Product Finder”) do Logout
    • Testing AJAX Applications EXERCISE 1: SELENIUM IDE: SEND AN EMAIL 19 © 2009 CAPLIN SYSTEMS
    • Objective  To get familiar with the Selenium IDE  To produce a simple test script  To review the script afterwards  Experience some Ajax testing pitfalls
    • Schema interact record playback
    • User Story “As a user, I want to send an e-mail message so that I can stay in touch with my friend” 22 © 2009 CAPLIN SYSTEMS
    • Golden Rule Put yourself in the user’s position  write the test first  watch it fail  fix it 23 © 2009 CAPLIN SYSTEMS
    • Tools: Firefox Add-ons  Firefox Selenium IDE  easy authoring tool  Firebug debugger  reverse engineering tool  XPather  locates UI elements 24 © 2009 CAPLIN SYSTEMS
    • Demo  Open Firefox IDE  Record  Save  Playback 25 © 2009 CAPLIN SYSTEMS
    • Setup…  Disconnect from LAN  Connect to CaplinSpa2009 wireless network  Plug USB key  Open Firefox + Selenium IDE  (set ?:Examplesuser-extensions.js)  Navigate to http://spa2009.com/webmail 26 © 2009 CAPLIN SYSTEMS
    • Your Turn…  Login:  user: spa1  password: caplin2009  Send mail To: patrick@spa2009.com  Insert appropriate actions:  pause  waitForElementPresent  waitForEditable  Save in /Exercise1-Basic/ 27 © 2009 CAPLIN SYSTEMS
    • Selenium IDE Review Pros Cons   easy to set up no explicit verification   easy to write locators are unintelligible   easy to run brittle   little Selenium knowledge unstructured required  desktop orientated  generates code!  feature must exist 28 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications SELENIUM: 5 MINUTE TUTORIAL 29 © 2009 CAPLIN SYSTEMS
    • Selenium Test Script One step per line... action thing to optional act on argument 30 © 2009 CAPLIN SYSTEMS
    • Actions  Selenium Selenium.prototype.doType(sInputLocator, sInputText) Selenium.prototype.doClick(sLocator)  Custom Framework Selenium.prototype.doClickControl(sLocator) 31 © 2009 CAPLIN SYSTEMS
    • Locators  Selenium xpath id link ...  Custom Framework PageBot.prototype.locateElementByMenuBarItem(sLocatorText, oDocument) 32 © 2009 CAPLIN SYSTEMS
    • Accessors  Selenium waitFor store ElementPresent assert verify Selenium.prototype.isElementPresent(sLocator)  Custom waitFor store MessageCount assert verify Selenium.prototype.getMessageCount(sLocator) 33 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications SCRIPTS: INTERPRETERS AND RUNNERS 34 © 2009 CAPLIN SYSTEMS
    • Script: Machine vs. Human readable Pianola (acting as pianist) Piano Roll 35 © 2009 CAPLIN SYSTEMS
    • After the break... Linguistic Application Constructs Abstractions • locator strategies • structure • application • functions operations • control flow • state transitions DOM Source Browser compiler Interpreter Runner Code COM Application 36 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications COFFEE BREAK (10 MINUTES) 37 © 2009 CAPLIN SYSTEMS
    • Design Principles Testing Ajax applications 38 © 2009 CAPLIN SYSTEMS
    • Design Principles  Domain Hierarchies  how to make my tests more readable?  modeling vs. maintainability?  how can I isolate application changes?  Test structure and composition  how do I avoid duplication?  how do my tests map to my requirements?  how can I make my framework scalable? 39 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications DOMAIN HIERARCHIES 40 © 2009 CAPLIN SYSTEMS
    • ? Test framework Verification Domain (Locate, input, verify, etc.) Test tool Browser Domain (DOM, text nodes, clicks, etc.) Application System Domain (OO language, may model business domain) 41 © 2009 CAPLIN SYSTEMS
    • What to use for framework domain? 1. Don’t bother! ... 42 © 2009 CAPLIN SYSTEMS
    • ? Test framework Verification Domain Test framework (Locate, input, verify, etc.) Test tool Browser Domain (DOM, text nodes, clicks, etc.) Application System Domain (OO language, may model business domain) 43 © 2009 CAPLIN SYSTEMS
    • What to use for framework domain? 1. Don’t bother... Quick to get going Can use off-the-shelf tools (e.g. Selenium IDE) Lots of duplication in tests Maintenance nightmare Brittle tests Rapidly become hard to read and author 44 © 2009 CAPLIN SYSTEMS
    • What to use for framework domain? 2. Mirror of system domain... 45 © 2009 CAPLIN SYSTEMS
    • ?? System Domain (again) Test framework Verification Domain (Locate, input, verify, etc.) Test tool Browser Domain (DOM, text nodes, clicks, etc.) Application System Domain (OO language, may model business domain) 46 © 2009 CAPLIN SYSTEMS
    • What to use for framework domain? 2. Mirror of system domain... Usually in same language as System Seems attractive – particularly if System is well modelled on problem domain Can “share” domain objects Often ends up with either tight coupling... ...or duplicated code / abstractions on both sides of the Browser/Verification divide Using same language as System doesn’t reinforce the domain separation 47 © 2009 CAPLIN SYSTEMS
    • What to use for framework domain? 3. Model or DSL that reinforces user’s interactions and use-cases... 48 © 2009 CAPLIN SYSTEMS
    • ? Business (User) Domain / DSL (Inbox, compose, check mail, logout, etc.) Test framework Verification Domain (Locate, input, verify, etc.) Test tool Browser Domain (DOM, text nodes, clicks, etc.) Application System Domain (OO language, may model business domain) 49 © 2009 CAPLIN SYSTEMS
    • What to use for framework domain? 3. Model or DSL that reinforces user’s interactions and use-cases... Easier to understand for non-programmers Less brittle when underlying domains change More natural abstractions Requires more maintenance of different abstractions Can make it hard to implement complex interactions 50 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications EXERCISE 2: USING A FRAMEWORK 51 © 2009 CAPLIN SYSTEMS
    • Objectives  Move test script closer to application domain  Introduce logical names for screen controls  Remove physical references to screen controls  Show Selenium extensions  Make tests more readable 52 © 2009 CAPLIN SYSTEMS
    • Selenium Framework Application Abstractions • actions • locators DOM Source Browser Interpreter Runner Code Application COM 53 © 2009 CAPLIN SYSTEMS
    • Exercise 2 - Framework  Demo 54 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications UCD TEST STRUCTURE 55 © 2009 CAPLIN SYSTEMS
    • User Centered Design (UCD)  A powerful way to model and design a system  Like Agile, focuses on Users – their needs and goals  Typically used for UI/UX design, but provides great models for test frameworks  We use with a simple text DSL 56 © 2009 CAPLIN SYSTEMS
    • User Centered Design (UCD)  Brings focus back to users and their  Goals  Tasks • Actions  Fits well with user story structure: As a [type of user] I want to [a task or sequence of tasks] so that I can [achieve some goal] 57 © 2009 CAPLIN SYSTEMS
    • Use-Cases and UCD UCD concepts map well to use-case levels Goal Task Action 58 © 2009 CAPLIN SYSTEMS
    • Story Structure Tasks Goal Actions Send Email Login Action Action Send Message Action Action Action Logout Action Action 59 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications EXERCISE 3: RUNNING A UCD TEST 60 © 2009 CAPLIN SYSTEMS
    • Objectives  Demonstrate UCD structure  Show task re-use  Implement a new task 61 © 2009 CAPLIN SYSTEMS
    • Old Structure Test Action Action Action Action Action Action Action Action 62 © 2009 CAPLIN SYSTEMS
    • UCD Structure Goal Task Action Action Task Action Action Action Task Action Action 63 © 2009 CAPLIN SYSTEMS
    • Demonstration  Goal/Task/Action hierarchy  Task level convenient for  Reuse  Maintenance  Communication  Action level too small to be reusable 64 © 2009 CAPLIN SYSTEMS
    • UCD Structure Send Email Login Action Action Send Message Action Action Action Verify Sent Action Action Logout Action Action 65 © 2009 CAPLIN SYSTEMS
    • A new story… “As a user, I want to receive an email so that I can prove I have a friend!” 66 © 2009 CAPLIN SYSTEMS
    • Your Turn... Receive Email Login Action Action Action Verify Inbox non-empty Action Logout Action Action 67 © 2009 CAPLIN SYSTEMS
    • Testing AJAX Applications SUMMARY 68 © 2009 CAPLIN SYSTEMS
    • Summary  Selenium is not a perfect testing tool, but…  Selenium extensions are effective  actions  locators  accessors  UCD is a good way to structure test cases  goals  tasks  actions
    • Taking things forward…  Develop a simple DSL for testers to use  invoke tasks with parameters  get rid of ugly HTML “Selenese”  use a PATH mechanism to locate tasks -- goal: use product finder portlet do Login (${username}=”user1quot;, ${password}=quot;testerquot;) do CreateNewLayout do Insert (${portlet}=“Product Finder”) do AssertPortletPresent (${portlet}=“Product Finder”) do Logout