SPA 2009 - Acceptance Testing AJAX Web Applications through the GUI

1,536 views

Published on

These are the slides that Andrew MacLeod and Patrick Myles presented at the SPA conference in London, April 2009

http://www.spaconference.org/spa2009/sessions/session219.html

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,536
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
59
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

SPA 2009 - Acceptance Testing AJAX Web Applications through the GUI

  1. 1. Testing Ajax Applications SPA 2009 © 2009 CAPLIN SYSTEMS
  2. 2. Testing AJAX Applications INTRODUCTIONS 2 © 2009 CAPLIN SYSTEMS
  3. 3. Who we are  Patrick Myles Head of Engineering, Caplin Systems  Andrew MacLeod JavaScript Architect, Caplin Systems 3 © 2009 CAPLIN SYSTEMS
  4. 4. How this talk came about  We have a big Ajax application  We use Agile  We needed a way to test each release This is the overview we wish we’d had 2 years ago 4 © 2009 CAPLIN SYSTEMS
  5. 5. This is not a Selenium HOWTO  Just enough Selenium to:  understand the grammar  promote a discussion on frameworks  see how good Selenium is!  Not enough Selenium to:  lose non-programmers (e.g. QA engineers)  get bogged down in Selenium nuts-and-bolts  see how frustrating Selenium is! 5 © 2009 CAPLIN SYSTEMS
  6. 6. Frustration Browsers were never designed as platforms for automated tests... ...so browser test tools are a hack! 6 © 2009 CAPLIN SYSTEMS
  7. 7. We would like you to leave with...  Confidence  automated testing  hands-on experience  Ideas  designing tests  tool selection  building a framework  planning a test programme 7 © 2009 CAPLIN SYSTEMS
  8. 8. Testing AJAX Applications THE PROBLEM 8 © 2009 CAPLIN SYSTEMS
  9. 9. The Problem  How can we ensure that each release of our application works everywhere? 9 © 2009 CAPLIN SYSTEMS
  10. 10. Manual Testing DOM Test Browser Pack Application Tester (acting as the User)  Repeat for each new feature  Repeat for all supported platforms . 10 © 2009 CAPLIN SYSTEMS
  11. 11. Automated Testing DOM Test Browser Script Application Robot (acting as the User)  How intelligent is the robot?  How human readable is the script?  How much effort does the robot consume? . 11 © 2009 CAPLIN SYSTEMS
  12. 12. Tools Test Behaviour Driven FIT Fitnesse Framework Development COM Cucumber WebTest browser control Domain Specific rspec Language (DSL) Watir Selenium hamcrest TestNG WebDriver Mercury JSUnit HttpUnit WindowLicker Windmill Auto-Hotkey LoadRunner DOM browser control Screen coordinates 12 © 2009 CAPLIN SYSTEMS
  13. 13. Workshop Why are browser applications so difficult to test? 13 © 2009 CAPLIN SYSTEMS
  14. 14. Testing AJAX Applications CAPLIN: 5 MINUTE CASE STUDY 14 © 2009 CAPLIN SYSTEMS
  15. 15. Caplin Trader  Quick demonstration
  16. 16. Caplin’s Timeline 2007 2008 2009 Scope Smoke Test (continuous integration) Functional Tests (nightly) Performance Tests Authors Apps Reference Implementation Bespoke Implementations (up to 3 in parallel) Development QA Framework Selenium Framework v1 Selenium Framework v2 16 © 2009 CAPLIN SYSTEMS
  17. 17. Our first test...
  18. 18. ... a more recent test goal: Use product finder portlet do Login (${username}=”user1quot;, ${password}=quot;testerquot;) do CreateNewLayout do Insert (${portlet}=“Product Finder”) do AssertPortletPresent (${portlet}=“Product Finder”) do Logout
  19. 19. Testing AJAX Applications EXERCISE 1: SELENIUM IDE: SEND AN EMAIL 19 © 2009 CAPLIN SYSTEMS
  20. 20. Objective  To get familiar with the Selenium IDE  To produce a simple test script  To review the script afterwards  Experience some Ajax testing pitfalls
  21. 21. Schema interact record playback
  22. 22. User Story “As a user, I want to send an e-mail message so that I can stay in touch with my friend” 22 © 2009 CAPLIN SYSTEMS
  23. 23. Golden Rule Put yourself in the user’s position  write the test first  watch it fail  fix it 23 © 2009 CAPLIN SYSTEMS
  24. 24. Tools: Firefox Add-ons  Firefox Selenium IDE  easy authoring tool  Firebug debugger  reverse engineering tool  XPather  locates UI elements 24 © 2009 CAPLIN SYSTEMS
  25. 25. Demo  Open Firefox IDE  Record  Save  Playback 25 © 2009 CAPLIN SYSTEMS
  26. 26. Setup…  Disconnect from LAN  Connect to CaplinSpa2009 wireless network  Plug USB key  Open Firefox + Selenium IDE  (set ?:Examplesuser-extensions.js)  Navigate to http://spa2009.com/webmail 26 © 2009 CAPLIN SYSTEMS
  27. 27. Your Turn…  Login:  user: spa1  password: caplin2009  Send mail To: patrick@spa2009.com  Insert appropriate actions:  pause  waitForElementPresent  waitForEditable  Save in /Exercise1-Basic/ 27 © 2009 CAPLIN SYSTEMS
  28. 28. Selenium IDE Review Pros Cons   easy to set up no explicit verification   easy to write locators are unintelligible   easy to run brittle   little Selenium knowledge unstructured required  desktop orientated  generates code!  feature must exist 28 © 2009 CAPLIN SYSTEMS
  29. 29. Testing AJAX Applications SELENIUM: 5 MINUTE TUTORIAL 29 © 2009 CAPLIN SYSTEMS
  30. 30. Selenium Test Script One step per line... action thing to optional act on argument 30 © 2009 CAPLIN SYSTEMS
  31. 31. Actions  Selenium Selenium.prototype.doType(sInputLocator, sInputText) Selenium.prototype.doClick(sLocator)  Custom Framework Selenium.prototype.doClickControl(sLocator) 31 © 2009 CAPLIN SYSTEMS
  32. 32. Locators  Selenium xpath id link ...  Custom Framework PageBot.prototype.locateElementByMenuBarItem(sLocatorText, oDocument) 32 © 2009 CAPLIN SYSTEMS
  33. 33. Accessors  Selenium waitFor store ElementPresent assert verify Selenium.prototype.isElementPresent(sLocator)  Custom waitFor store MessageCount assert verify Selenium.prototype.getMessageCount(sLocator) 33 © 2009 CAPLIN SYSTEMS
  34. 34. Testing AJAX Applications SCRIPTS: INTERPRETERS AND RUNNERS 34 © 2009 CAPLIN SYSTEMS
  35. 35. Script: Machine vs. Human readable Pianola (acting as pianist) Piano Roll 35 © 2009 CAPLIN SYSTEMS
  36. 36. After the break... Linguistic Application Constructs Abstractions • locator strategies • structure • application • functions operations • control flow • state transitions DOM Source Browser compiler Interpreter Runner Code COM Application 36 © 2009 CAPLIN SYSTEMS
  37. 37. Testing AJAX Applications COFFEE BREAK (10 MINUTES) 37 © 2009 CAPLIN SYSTEMS
  38. 38. Design Principles Testing Ajax applications 38 © 2009 CAPLIN SYSTEMS
  39. 39. Design Principles  Domain Hierarchies  how to make my tests more readable?  modeling vs. maintainability?  how can I isolate application changes?  Test structure and composition  how do I avoid duplication?  how do my tests map to my requirements?  how can I make my framework scalable? 39 © 2009 CAPLIN SYSTEMS
  40. 40. Testing AJAX Applications DOMAIN HIERARCHIES 40 © 2009 CAPLIN SYSTEMS
  41. 41. ? Test framework Verification Domain (Locate, input, verify, etc.) Test tool Browser Domain (DOM, text nodes, clicks, etc.) Application System Domain (OO language, may model business domain) 41 © 2009 CAPLIN SYSTEMS
  42. 42. What to use for framework domain? 1. Don’t bother! ... 42 © 2009 CAPLIN SYSTEMS
  43. 43. ? Test framework Verification Domain Test framework (Locate, input, verify, etc.) Test tool Browser Domain (DOM, text nodes, clicks, etc.) Application System Domain (OO language, may model business domain) 43 © 2009 CAPLIN SYSTEMS
  44. 44. What to use for framework domain? 1. Don’t bother... Quick to get going Can use off-the-shelf tools (e.g. Selenium IDE) Lots of duplication in tests Maintenance nightmare Brittle tests Rapidly become hard to read and author 44 © 2009 CAPLIN SYSTEMS
  45. 45. What to use for framework domain? 2. Mirror of system domain... 45 © 2009 CAPLIN SYSTEMS
  46. 46. ?? System Domain (again) Test framework Verification Domain (Locate, input, verify, etc.) Test tool Browser Domain (DOM, text nodes, clicks, etc.) Application System Domain (OO language, may model business domain) 46 © 2009 CAPLIN SYSTEMS
  47. 47. What to use for framework domain? 2. Mirror of system domain... Usually in same language as System Seems attractive – particularly if System is well modelled on problem domain Can “share” domain objects Often ends up with either tight coupling... ...or duplicated code / abstractions on both sides of the Browser/Verification divide Using same language as System doesn’t reinforce the domain separation 47 © 2009 CAPLIN SYSTEMS
  48. 48. What to use for framework domain? 3. Model or DSL that reinforces user’s interactions and use-cases... 48 © 2009 CAPLIN SYSTEMS
  49. 49. ? Business (User) Domain / DSL (Inbox, compose, check mail, logout, etc.) Test framework Verification Domain (Locate, input, verify, etc.) Test tool Browser Domain (DOM, text nodes, clicks, etc.) Application System Domain (OO language, may model business domain) 49 © 2009 CAPLIN SYSTEMS
  50. 50. What to use for framework domain? 3. Model or DSL that reinforces user’s interactions and use-cases... Easier to understand for non-programmers Less brittle when underlying domains change More natural abstractions Requires more maintenance of different abstractions Can make it hard to implement complex interactions 50 © 2009 CAPLIN SYSTEMS
  51. 51. Testing AJAX Applications EXERCISE 2: USING A FRAMEWORK 51 © 2009 CAPLIN SYSTEMS
  52. 52. Objectives  Move test script closer to application domain  Introduce logical names for screen controls  Remove physical references to screen controls  Show Selenium extensions  Make tests more readable 52 © 2009 CAPLIN SYSTEMS
  53. 53. Selenium Framework Application Abstractions • actions • locators DOM Source Browser Interpreter Runner Code Application COM 53 © 2009 CAPLIN SYSTEMS
  54. 54. Exercise 2 - Framework  Demo 54 © 2009 CAPLIN SYSTEMS
  55. 55. Testing AJAX Applications UCD TEST STRUCTURE 55 © 2009 CAPLIN SYSTEMS
  56. 56. User Centered Design (UCD)  A powerful way to model and design a system  Like Agile, focuses on Users – their needs and goals  Typically used for UI/UX design, but provides great models for test frameworks  We use with a simple text DSL 56 © 2009 CAPLIN SYSTEMS
  57. 57. User Centered Design (UCD)  Brings focus back to users and their  Goals  Tasks • Actions  Fits well with user story structure: As a [type of user] I want to [a task or sequence of tasks] so that I can [achieve some goal] 57 © 2009 CAPLIN SYSTEMS
  58. 58. Use-Cases and UCD UCD concepts map well to use-case levels Goal Task Action 58 © 2009 CAPLIN SYSTEMS
  59. 59. Story Structure Tasks Goal Actions Send Email Login Action Action Send Message Action Action Action Logout Action Action 59 © 2009 CAPLIN SYSTEMS
  60. 60. Testing AJAX Applications EXERCISE 3: RUNNING A UCD TEST 60 © 2009 CAPLIN SYSTEMS
  61. 61. Objectives  Demonstrate UCD structure  Show task re-use  Implement a new task 61 © 2009 CAPLIN SYSTEMS
  62. 62. Old Structure Test Action Action Action Action Action Action Action Action 62 © 2009 CAPLIN SYSTEMS
  63. 63. UCD Structure Goal Task Action Action Task Action Action Action Task Action Action 63 © 2009 CAPLIN SYSTEMS
  64. 64. Demonstration  Goal/Task/Action hierarchy  Task level convenient for  Reuse  Maintenance  Communication  Action level too small to be reusable 64 © 2009 CAPLIN SYSTEMS
  65. 65. UCD Structure Send Email Login Action Action Send Message Action Action Action Verify Sent Action Action Logout Action Action 65 © 2009 CAPLIN SYSTEMS
  66. 66. A new story… “As a user, I want to receive an email so that I can prove I have a friend!” 66 © 2009 CAPLIN SYSTEMS
  67. 67. Your Turn... Receive Email Login Action Action Action Verify Inbox non-empty Action Logout Action Action 67 © 2009 CAPLIN SYSTEMS
  68. 68. Testing AJAX Applications SUMMARY 68 © 2009 CAPLIN SYSTEMS
  69. 69. Summary  Selenium is not a perfect testing tool, but…  Selenium extensions are effective  actions  locators  accessors  UCD is a good way to structure test cases  goals  tasks  actions
  70. 70. Taking things forward…  Develop a simple DSL for testers to use  invoke tasks with parameters  get rid of ugly HTML “Selenese”  use a PATH mechanism to locate tasks -- goal: use product finder portlet do Login (${username}=”user1quot;, ${password}=quot;testerquot;) do CreateNewLayout do Insert (${portlet}=“Product Finder”) do AssertPortletPresent (${portlet}=“Product Finder”) do Logout

×