Your SlideShare is downloading. ×
Belgium Testing Days - Making Test Automation Work in Agile Projects
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Belgium Testing Days - Making Test Automation Work in Agile Projects


Published on

Slides from tutorial. Note that the most important part of the tutorial is the exercises, and I can't capture that in the slide deck. Please do not use these for public paid courses, I'm tired of our …

Slides from tutorial. Note that the most important part of the tutorial is the exercises, and I can't capture that in the slide deck. Please do not use these for public paid courses, I'm tired of our stuff being ripped off for agile testing classes.

Published in: Technology, Education

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Making TestAutomation Work in Agile ProjectsBelgium Testing Days 2012 Lisa Crispin With Material from Janet Gregory 1
  • 2. Introductions: Experience, Goals Form groups:  New to automation  One to two years experience w/ automation  More than two years experience Talk with at least two other people  Tell each other your learning goals for today  Note the most interesting one you hear 2
  • 3. About me…Programmer, tester, agile team member 3 Copyright 2012: Lisa Crispin
  • 4. Introduction - You Main role on team? Programming, automation experience? Using agile approach? What have you automated? (Test, CI,deployment...) 4 Copyright 2012: Lisa Crispin
  • 5. TakeawaysFoundation for successful test automation “Whole Team” approach What/when to automate Identifying, overcoming barriers Choosing, implementing tools First stepsWe won’t do any hands-on automation, but will demo some examples 5 Copyright 2012: Lisa Crispin
  • 6. Exercise: Your Learning Goals Write one interesting learning goal you heard from another participant at the start of the class on a sticky note. Write your number one learning goal for today on a sticky note. Put the sticky notes on your table group “learning goals” sheet Group similar ones together 6
  • 7. Let‟s start by defining “agile”Agile teams:  Deliver business value frequently  at a sustainable pace  while adapting to the changing needs of the business Source: Elisabeth Hendrickson 7 Copyright 2012: Lisa Crispin
  • 8. The key is “sustainable pace”Technical debt slows us down 8 Copyright 2012: Lisa Crispin
  • 9. The DeathSpiral
  • 10. High technical debt +insufficient automation = evenless time Copyright 2012: Lisa Crispin
  • 11. Barriers to Test AutomationWhat‟s holding you back? 11 Copyright 2012: Lisa Crispin
  • 12. Exercise: Your barriersIndividually and silently – write one barrier hindering your team per sticky notePut these on the “impediments” wall chart for your table groupTalk to your fellow group members – do you see any patterns? 12 Copyright 2012: Lisa Crispin
  • 13. Questions? 13 Copyright 2012: Lisa Crispin
  • 14. The Whole-Team Approach Automated tests are code Respecting the tests Collaborating Commitment to quality Return on investment 14 Copyright 2012: Lisa Crispin
  • 15. Automated tests are codepublic class CalculatorFixture extends ColumnFixture { public String startDate; public String endDate; public double startBalance; public double endBalance; public String irrTarget;private Calculator calculator = new Calculator(); private Double irr; 15 Copyright 2012: Lisa Crispin
  • 16. Test code deserves the same respect asproduction code! (Rodney Dangerfield, a comedian whose tag line was “I don’t get no respect… )
  • 17. Source: Gojko Adzic, StarEast 2011 keynote
  • 18. Source: Gojko Adzic, StarEast 2011 keynote
  • 19. Testers are especially good at… Eliciting examples Turning them into tests Ensuring the right testing gets done Exploratory testing 19 Copyright 2012: Lisa Crispin
  • 20. Give testers time to do what we do best How? 20
  • 21. 21The Whole-Team Approach
  • 22. Experiment: Iteration 1 Pair up: one will be tester, one programmer Sit back to back so you face away from each other Tester gets a drawing which needs to be replicated Tester tells the programmer what to draw, all at one time. Programmer draws the shapes based on what the tester explained.  No talking during „coding‟! Tester “tests” the drawing, reports “bugs” on index cards Programmer fixes the “bugs” How long did it take? Will the customer be 22 happy? Copyright 2012: Lisa Crispin
  • 23. Experiment: Iteration 2 Collaborate! Tester tells programmer what to draw, watches the programmer draw, answers questions, points out „defects‟ for programmer to fix immediately  (Don‟t show the programmer the drawing, that makes it too easy, we‟re trying to simulate real coding) How long did it take? Will the customer be happy? Thanks to the members of the agile-games group and Kane Mar for ideas & pictures for this game 23 Copyright 2012: Lisa Crispin
  • 24. Ways to collaborate Pair 24 Copyright 2012: Lisa Crispin
  • 25. Team responsibilityAutomate all regression tests 25 Copyright 2012: Lisa Crispin
  • 26. 26Copyright 2012: Lisa Crispin
  • 27. Commitment to qualityWhat‟s your team‟s commitment?The best possible software product? 27 Copyright 2012: Lisa Crispin
  • 28. MeaningfulCommitment Experimen t Learn 28 Copyright 2012: Lisa Crispin
  • 29. Under-commitPlan less work than you think you can doIncluding all test automation Copyright 2012: Lisa Crispin
  • 30. Learn to write maintainable tests Get over the “hump of pain” From Gerard Meszaros’ XUnit Test Patterns 30 Copyright 2012: Lisa Crispin
  • 31. Whole-team software developmentCreate a Expand user tests – story Story Tests Automate Write Q2 Tests Pair,Customer Start “Show (Q2) thinking Me” Tests how to code TDD Exploratory testing Product owner Product owner/ Tester Tester Tester/Programmer Programmer Customer User Acceptance 31
  • 32. Exercise •Think of an experiment to get your whole team engaged in automating tests •Share with your table group •Pick two to share with the class Copyright 2012: Lisa Crispin
  • 33. Getting Over the Hump The test automation pyramid The agile testing quadrants What should be automated What shouldnt Difficult areas 33 Copyright 2012: Lisa Crispin
  • 34. Test Automation Pyramid 34 Copyright 2012: Lisa Crispin
  • 35. Agile Testing Quadrants 35 Copyright 2012: Lisa Crispin
  • 36. Shared Understanding Start with tests Collaborate 36 Copyright 2012: Lisa Crispin
  • 37. What Should We Automate? Quadrant 1 tests  Unit, component, TDD Quadrant 2 tests  API, service-level Quadrant 4 tests  Load, performance, stress Quadrant 3 tests?  Leverage automation where useful 37 Copyright 2012: Lisa Crispin
  • 38. What Shouldn‟t We Automate? Quadrant 2 tests  Wizard of Oz, prototyping Quadrant 3 tests  Usability, UAT, ET Tests that will never fail?  Assess risk ROI not enough  One-off tests 38 Copyright 2012: Lisa Crispin
  • 39. Where Should We Be Careful? GUI tests  Watch ROI End-to-End tests  Push testing down to lowest level Remember the Pyramid 39 Copyright 2012: Lisa Crispin
  • 40. Hard to Automate? Legacy code  Hard to automate, or just lack of skill?  “Working Effectively with Legacy Code” – Feathers  “Strangling” – Fowler, Thomas 40 Copyright 2012: Lisa Crispin
  • 41. Exercise: Low-Hanging Fruit 41 Copyright 2012: Lisa Crispin
  • 42. Agile Automation Strategy What hurts the most Layered approach Applying agile principles  Small chunks/thin slices  Smart test design Choosing the right tools 42 Copyright 2012: Lisa Crispin
  • 43. What Hurts the Most Use retrospectives Keep an impediment backlog 43 Copyright 2012: Lisa Crispin
  • 44. Multi-Layered ApproachExample:  Learn TDD at unit level  Automate GUI smoke tests 44 Copyright 2012: Lisa Crispin
  • 45. Simplicity  Address one or two needs at a time  Understand the problem first  Try simplest approach first  Work in small chunks, thin slices  Incremental & iterative 45 Copyright 2012: Lisa Crispin
  • 46. Automate a Slice at a TimeExample: 4-step UI to validate, upload profit sharing contribution data• Thread 1: All four pages with navigation• Thread 2: Select year, enter description on page 1, display on page 2, browse and upload file on page 2• Thread 3: Validate data in file, display on page 3• Thread 4: Persist data, display „success‟ message on page 4 46 Copyright 2012: Lisa Crispin
  • 47. Thin Slice Example 47 Copyright 2012: Lisa Crispin
  • 48. Exercise: Thin SlicesHere‟s our user story:As an internet shopper, I want to create anaccount so that I do not have to enter my addressand billing information each time I make apurchaseDraw a mind map for this story on a big sheet ofpaperIdentify a basic end-to-end slice of functionalitythat can be coded, tested, and automated.If you have time, identify additional slices. 48 Copyright 2012: Lisa Crispin
  • 49. Test Design Patterns/Principles  Code design patterns  One clear purpose  Don‟t Repeat Yourself 49
  • 50. Demo 50 Copyright 2012: Lisa Crispin
  • 51. Iterative Feedback Spike two different approaches Pick one to try for N # of iterations Use retrospectives to evaluate 51 Copyright 2012: Lisa Crispin
  • 52. Learn by Doing Courage – don‟t be afraid to fail Production code practices for test code Incremental, thin slices Experiment 52 Copyright 2012: Lisa Crispin
  • 53. Questions About Automation Strategy? 53 Copyright 2012: Lisa Crispin
  • 54. Choosing Tools Team effort Time Requirements Focus on goals, problems, not tools. Experiment 54 Copyright 2012: Lisa Crispin
  • 55. Understand the Purpose Who‟s using the tests? What for? What‟s being automated? Existing tools, environment Who‟s doing what for automating? 55 Copyright 2012: Lisa Crispin
  • 56. What Fits Your Situation• Existing skills• Language of application under test• Collaboration needs• What‟s being automated• Life span, future use of tests 56 Copyright 2012: Lisa Crispin
  • 57. Test Drivers/Frameworks Automation layer Fit for everyone on team Try out more than one Let‟s look at more examples 57 Copyright 2012: Lisa Crispin
  • 58. Where To Find Tools - aa-ftt spreadsheet 58 Copyright 2012: Lisa Crispin
  • 59. Example: My Team‟s Tool Choices• IntelliJ Idea• Jenkins, ant, Maven• JUnit• FitNesse• Canoo WebTest• Watir• JMeter• Selenium 2.0 / WebDriver with Geb framework 59 Copyright 2012: Lisa Crispin
  • 60. Exercise: Tools 60 Copyright 2012: Lisa Crispin
  • 61. Making Test Automation Work Time to do it right Learning culture Testable architecture Test data Managing tests 61 Copyright 2012: Lisa Crispin
  • 62. Time To Do It Right Limit scope, don‟t over-commit Write automation task cards Quality must be team goal Long-term, will let you go faster 62 Copyright 2012: Lisa Crispin
  • 63. Learning Culture OK to make mistakes Lots of small experiments Slack Evolve right design 63 Copyright 2012: Lisa Crispin
  • 64. Testable Architecture• Layered architecture • eg. UI, business logic, data access• Ports and Adapters pattern • App can work without UI or database • Ports accept outside events • Adapters convert for human or automated users 64 Copyright 2012: Lisa Crispin
  • 65. Test Data Avoid database access when possible Setup/Teardown  Independent, rerunnable tests Canonical data  Refresh before each test run Customizable data for ET Production-like data  Get customers to provide example data 65 Copyright 2012: Lisa Crispin
  • 66. Managing Automated Tests Tests as Documentation Continuous Integration Reporting results Metrics 66 Copyright 2012: Lisa Crispin
  • 67. Tests as Documentation Understandable Who will really use them? Once passing, must always pass 67 Copyright 2012: Lisa Crispin
  • 68. Any Example Can Become a Test 68 Copyright 2012: Lisa Crispin
  • 69. Given/Then/When ExampleScenario: Valid name search returns resultsGIVEN that Kant is a supervisor with employeesAND Kant has an employee named SmithWHEN Kant navigates to the employee name search pageAND enters the value “S”THEN Kant will see a search result that includes Smith 69 Copyright 2012: Lisa Crispin
  • 70. Continuous integration/testing……short feedback loop 70 Copyright 2012: Lisa Crispin
  • 71. Keep tests passingStop the line to fix problems 71 Copyright 2012: Lisa Crispin
  • 72. Test Management Tools Manage tests, code together Some tools have own management What problem are you trying to solve? 72 Copyright 2012: Lisa Crispin
  • 73. Exercise: Tests as Documentation 73 Copyright 2012: Lisa Crispin
  • 74. Key Success Factors Thin Slices Specificatio n by Whole Team Example ApproachShortFeedbackLoops Experiments 74
  • 75. Exercise: Breaking Barriers 75 Copyright 2012: Lisa Crispin
  • 76. Remember It‟s a team problem! Tackle automation problems with diversity Try small experiments Baby steps – start simple 76 Copyright 2012: Lisa Crispin
  • 77. Questions? “Aha” Moments? 77 Copyright 2012: Lisa Crispin
  • 78. Agile Testing: A Practical Guide for Testers and AgileTeamsBy Lisa Crispin and Janet Copyright 2012: Lisa Crispin 78
  • 79. Experiences of Test AutomationDorothy Graham and Mark Fewster Copyright 2012: Lisa Crispin 79
  • 80. Beautiful Testing: Leading Professionals Reveal HowThey Improve SoftwareEdited by Tim Riley, Adam GoucherIncludes chapter by yours truly Copyright 2012: Lisa Crispin 80
  • 81. Test PatternsXunit Test Patterns: Refactoring Test CodeBy Gerard Meszaros Copyright 2012: Lisa Crispin 81
  • 82. Specification by ExampleHow successful teams deliver the rightsoftwareGojko AdzicCase studies from > 50 teams 82 Copyright 2012: Lisa Crispin Copyright 2008 Janet Gregory, DragonFire
  • 83. Agile Test Automation Resources 83 Copyright 2012: Lisa Crispin