Your SlideShare is downloading. ×
Atd agile automation
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Atd agile automation


Published on

  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Making TestAutomation Work in Agile Projects Agile Testing Days 2012 Lisa Crispin With Material from Janet Gregory 1
  • 2. Introductions: Experience, Goals§  Form groups: §  New to automation §  One to two years experience w/ automation §  More than two years experience§  Talk with at least two other people §  Tell each other your learning goals for today §  Note the most interesting one you hear 2
  • 3. About me…Programmer, tester, agile team member Ever-evolving views on automation 3 Copyright 2012: Lisa Crispin
  • 4. Introduction - You 4 Copyright 2012: Lisa Crispin
  • 5. TakeawaysSuccessful test automation is an innovation!•  Let go of traditional roles, reach across them•  Visualize how tests should look•  Be smart about when to automate•  Experiment to overcome obstacles•  Find/build the right toolsWe won’t do any hands-on automation, but will demo some examples 5 Copyright 2012: Lisa Crispin
  • 6. Exercise: Your Learning Goals§  Write one interesting learning goal you heard from another participant at the start of the class on a sticky note.§  Write your number one learning goal for today on a sticky note.§  Put the sticky notes on your table group “learning goals” sheet§  Group similar ones together 6
  • 7. Let’s start by defining “agile”Agile teams: q  Deliver business value frequently q  at a sustainable pace q  while adapting to the changing needs of the business Source: Elisabeth Hendrickson 7 Copyright 2012: Lisa Crispin
  • 8. The key is “sustainable pace”Technical debt slows us down 8 Copyright 2012: Lisa Crispin
  • 9. The DeathSpiral
  • 10. High technical debt +insufficient automation =even less time Copyright 2012: Lisa Crispin
  • 11. Barriers to Test AutomationWhat’s holding you back? 11 Copyright 2012: Lisa Crispin
  • 12. Exercise: Your barriersIndividually and silently – write one barrier hindering your team per sticky notePut these on the “impediments” wall chart for your table groupWork with your group to identify patterns, group similar items 12 Copyright 2012: Lisa Crispin
  • 13. Questions? 13 Copyright 2012: Lisa Crispin
  • 14. The Whole-Team Approach§  Automated tests are code§  Respecting the tests§  Collaborating§  Commitment to quality§  Return on investment 14 Copyright 2012: Lisa Crispin
  • 15. Automated tests are codepublic class CalculatorFixture extends ColumnFixture { public String startDate; public String endDate; public double startBalance; public double endBalance; public String irrTarget;private Calculator calculator = new Calculator(); private Double irr; 15 Copyright 2012: Lisa Crispin
  • 16. Source: Gojko Adzic, StarEast 2011 keynote
  • 17. Source: Gojko Adzic, StarEast 2011 keynote
  • 18. Testers are especially good at…§ Eliciting examples§ Turning them into tests§ Ensuring the right testing gets done§ Exploratory testing 18 Copyright 2012: Lisa Crispin
  • 19. Give testers time to do what we do best How? 19
  • 20. 20The Whole-Team Approach
  • 21. Experiment: Iteration 1§  Pair up: one will be tester, one programmer§  Sit back to back so you face away from each other§  Tester: You have a drawing which you need to instruct your developer to draw§  Programmer: Remain silent while working, only do what you’re instructed. §  No talking! §  No peeking!When “done”:§  Programmer – pass your drawing to the tester§  Tester – note problems on separate card/paperWill the customer be happy? 21 Copyright 2012: Lisa Crispin
  • 22. Experiment: Iteration 2§  Now, turn to face each other.§  Tester: §  Still don’t show your picture! §  Tell the programmer what to draw, but - §  Watch what the programmer draws, and point out any mistakes immediately§  Programmer: §  Still no peeking! §  Ask questions, ask for feedback, make corrections immediately Thanks to the members of the agile-games group and Kane Mar for ideas & pictures for this game 22 Copyright 2012: Lisa Crispin
  • 23. Ways to collaborate Pair: Tester-Coder Tester-Tester Any role – any role 23 Copyright 2012: Lisa Crispin
  • 24. Team responsibilityAutomate all regression tests 24 Copyright 2012: Lisa Crispin
  • 25. 25Copyright 2012: Lisa Crispin
  • 26. Commitment to qualityWhat’s your team’s commitment?The best possible software product? 26 Copyright 2012: Lisa Crispin
  • 27. MeaningfulCommitment Experiment Learn 27 Copyright 2012: Lisa Crispin
  • 28. Under-commitPlan less work than you think you can doIncluding all test automation Copyright 2012: Lisa Crispin
  • 29. Learn to write maintainable tests§  Get over the “hump of pain” From Gerard Meszaros’ XUnit Test Patterns 29 Copyright 2012: Lisa Crispin
  • 30. Whole-team software developmentCreate a Expand user tests – story Story Tests Automate Write Q2 Tests Pair,Customer Start “Show (Q2) thinking Me” Tests how to code TDD Exploratory testing Product owner Product owner/ Tester Tester Tester/Programmer Programmer Customer User Acceptance 30
  • 31. Exercise • Think of an experiment to get your whole team engaged in automating tests • Share with your table group • Pick two to share with the class Copyright 2012: Lisa Crispin
  • 32. Getting Over the Hump§  The test automation pyramid§  The agile testing quadrants§  What should be automated§  What shouldnt§  Difficult areas 32 Copyright 2012: Lisa Crispin
  • 33. Test Automation Pyramid 33 Copyright 2012: Lisa Crispin
  • 34. Test Automation VolcanoFrom MattBarcomb Copyright 2012: Lisa Crispin
  • 35. Agile Testing Quadrants 35 Copyright 2012: Lisa Crispin
  • 36. Shared Understanding§  Start with tests§  Collaborate 36 Copyright 2012: Lisa Crispin
  • 37. What Should We Automate?§  Quadrant 1 tests §  Unit, component, TDD§  Quadrant 2 tests §  API, service-level§  Quadrant 4 tests §  Load, performance, stress§  Quadrant 3 tests? §  Leverage automation where useful 37 Copyright 2012: Lisa Crispin
  • 38. What Shouldn’t We Automate?§  Quadrant 2 tests §  Wizard of Oz, prototyping§  Quadrant 3 tests §  Usability, UAT, ET§  Tests that will never fail? §  Assess risk§  ROI not enough §  One-off tests 38 Copyright 2012: Lisa Crispin
  • 39. Where Should We Be Careful?§  GUI tests §  Watch ROI§  End-to-End tests §  Push testing down to lowest level§  Remember the Pyramid 39 Copyright 2012: Lisa Crispin
  • 40. Hard to Automate?§  Legacy code §  Hard to automate, or just lack of skill? §  “Working Effectively with Legacy Code” – Feathers §  “Strangling” – Fowler, Thomas 40 Copyright 2012: Lisa Crispin
  • 41. Exercise: Low-Hanging Fruit Any “easy wins”? 41 Copyright 2012: Lisa Crispin
  • 42. Agile Automation Strategy§  What hurts the most§  Layered approach§  Applying agile principles §  Small chunks/thin slices §  Smart test design§  Choosing the right tools 42 Copyright 2012: Lisa Crispin
  • 43. What Hurts the Most§  Use retrospectives§  Keep an impediment backlog 43 Copyright 2012: Lisa Crispin
  • 44. Multi-Layered ApproachExample: §  Learn TDD at unit level §  Automate GUI smoke tests 44 Copyright 2012: Lisa Crispin
  • 45. Simplicity §  Address one or two needs at a time §  Understand the problem first §  Try simplest approach first §  Work in small chunks, thin slices §  Incremental & iterative 45 Copyright 2012: Lisa Crispin
  • 46. Automate a Slice at a TimeExample: 4-step UI to validate, upload profit sharing contribution data•  Thread 1: All four pages with navigation•  Thread 2: Select year, enter description on page 1, display on page 2, browse and upload file on page 2•  Thread 3: Validate data in file, display on page 3•  Thread 4: Persist data, display ‘success’ message on page 4 46 Copyright 2012: Lisa Crispin
  • 47. Thin Slice Example 47 Copyright 2012: Lisa Crispin
  • 48. Exercise: Thin SlicesHere’s our user story (or theme):As an internet shopper, I want to create anaccount so that I do not have to enter my addressand billing information each time I make apurchaseDraw a mind map for this feature on a big sheet ofpaperIdentify a basic end-to-end slice of functionalitythat can be coded, tested, and automated.If you have time, identify additional slices. 48 Copyright 2012: Lisa Crispin
  • 49. Test Design Patterns/Principles §  Code design patterns §  One clear purpose §  Don’t Repeat Yourself 49
  • 50. Page Object PatternA “page object” is a test object that holdsthe details of all the elements on a webpage that might be involved in anautomated test. 50 Copyright 2012: Lisa Crispin
  • 51. More on the Page Object Pattern 51 Copyright 2012: Lisa Crispin
  • 52. Demo: semi-automating exploratory tests 52 Copyright 2012: Lisa Crispin
  • 53. Iterative Feedback§  Spike two different approaches§  Pick one to try for N # of iterations§  Use retrospectives to evaluate 53 Copyright 2012: Lisa Crispin
  • 54. Learn by Doing§  Courage – don’t be afraid to fail§  Production code practices for test code§  Incremental, thin slices§  Experiment 54 Copyright 2012: Lisa Crispin
  • 55. Questions About Automation Strategy? 55 Copyright 2012: Lisa Crispin
  • 56. Choosing Tools, Frameworks, Drivers§  Team effort§  Time§  Requirements§  Focus on goals, problems, not tools.§  Experiment 56 Copyright 2012: Lisa Crispin
  • 57. Understand the Purpose§  Who’s using the tests? What for?§  What’s being automated?§  Existing tools, environment§  Who’s doing what for automating? 57 Copyright 2012: Lisa Crispin
  • 58. What Fits Your Situation•  Existing skills•  Language of application under test•  Collaboration needs•  What’s being automated•  Life span, future use of tests 58 Copyright 2012: Lisa Crispin
  • 59. Test Drivers/Frameworks§  Divide into sub-teams and prototype solutions§  Have a “bake-off” showing each prototype 59 Copyright 2012: Lisa Crispin
  • 60. Where To Find Tools§§§§§§  Grow your own! 60 Copyright 2012: Lisa Crispin
  • 61. Example: One Team’s Tool Choices•  IntelliJ Idea•  Jenkins, ant, Maven•  JUnit•  FitNesse•  Canoo WebTest•  Watir•  Robot Framework•  JMeter•  Selenium 2.0 / WebDriver with Geb framework 61 Copyright 2012: Lisa Crispin
  • 62. Exercise: Test FrameworksWhat format would work for your team? EG,BDD style? Tables? Assertions? Scenarios?Think of the test format that might work foryour team. Write examples on sticky notes.Write down any particular types of testpatterns, frameworks and driverss you wantto learn more about.and place these sticky notes next to animpediment it might help overcome..Discuss with group how that tool might helpsolve the problem. Copyright 2012: Lisa Crispin 62
  • 63. Making Test Automation Work§  Time to do it right§  Experiments, bake-offs§  Testable architecture§  Test data§  Managing tests 63 Copyright 2012: Lisa Crispin
  • 64. Time To Do It Right§  Limit scope, don’t over-commit§  Write automation task cards§  Plan test code maintenance/ refactoring time§  Quality must be team goal§  Long-term, will let you go faster 64 Copyright 2012: Lisa Crispin
  • 65. Bake-offs§  Have sub-teams/pairs experiment with prototypes§  Budget lots of time for learning, experimenting§  Look at pros & cons 65 Copyright 2012: Lisa Crispin
  • 66. Testable Architecture•  Layered architecture •  eg. UI, business logic, data access•  Ports and Adapters pattern •  App can work without UI or database •  Ports accept outside events •  Adapters convert for human or automated users 66 Copyright 2012: Lisa Crispin
  • 67. Test Data§  Avoid database access when possible§  Setup/Teardown §  Independent, rerunnable tests§  Canonical data §  Refresh before each test run§  Customizable data for ET§  Production-like data §  Get customers to provide example data 67 Copyright 2012: Lisa Crispin
  • 68. Managing Automated Tests§  Tests as Living Documentation§  Continuous Integration§  Reporting results§  Metrics 68 Copyright 2012: Lisa Crispin
  • 69. Tests as Living Documentation§  Understandable§  Who will really use them?§  Once passing, must always pass 69 Copyright 2012: Lisa Crispin
  • 70. Any Example Can Become a Test 70 Copyright 2012: Lisa Crispin
  • 71. Given/Then/When ExampleScenario: Valid name search returns resultsGIVEN that Kant is a supervisor with employeesAND Kant has an employee named SmithWHEN Kant navigates to the employee name search pageAND enters the value “S”THEN Kant will see a search result that includes Smith 71 Copyright 2012: Lisa Crispin
  • 72. Continuous integration/testing……short feedback loop 72 Copyright 2012: Lisa Crispin
  • 73. Keep tests passingStop the line to fix problems 73 Copyright 2012: Lisa Crispin
  • 74. Test Management Tools§  Manage tests, code together§  Some tools have own management§  What problem are you trying to solve? 74 Copyright 2012: Lisa Crispin
  • 75. Exercise: Tests as Documentation 75 Copyright 2012: Lisa Crispin
  • 76. Key Success Factors Thin Slices Specification by Example Whole Team ApproachShortFeedbackLoops Experiments 76
  • 77. Exercise: Breaking Barriers 77 Copyright 2012: Lisa Crispin
  • 78. Remember§  It’s a team problem!§  Tackle automation problems with diversity§  Try small experiments§  Baby steps – start simple 78 Copyright 2012: Lisa Crispin
  • 79. Questions? “Aha” Moments? 79 Copyright 2012: Lisa Crispin
  • 80. Agile Testing: A Practical Guide for Testers and AgileTeamsBy Lisa Crispin and Janet Copyright 2012: Lisa Crispin 80
  • 81. Experiences of Test AutomationDorothy Graham and Mark Fewster Copyright 2012: Lisa Crispin 81
  • 82. Beautiful Testing: Leading Professionals Reveal HowThey Improve SoftwareEdited by Tim Riley, Adam GoucherIncludes chapter by yours truly Copyright 2012: Lisa Crispin 82
  • 83. Test PatternsXunit Test Patterns: Refactoring Test CodeBy Gerard Meszaros Copyright 2012: Lisa Crispin 83
  • 84. Specification by ExampleHow successful teams deliver the rightsoftwareGojko AdzicCase studies from > 50 teams 84 Copyright 2012: Lisa Crispin Copyright 2008 Janet Gregory, DragonFire
  • 85. Agile Test Automation Resources§§§§§§§§§§ 85 Copyright 2012: Lisa Crispin