Your SlideShare is downloading. ×
Belgium Testing Days - Making Test Automation Work in Agile Projects
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Belgium Testing Days - Making Test Automation Work in Agile Projects


Published on

Slides from tutorial. Note that the most important part of the tutorial is the exercises, and I can't capture that in the slide deck. Please do not use these for public paid courses, I'm tired of our …

Slides from tutorial. Note that the most important part of the tutorial is the exercises, and I can't capture that in the slide deck. Please do not use these for public paid courses, I'm tired of our stuff being ripped off for agile testing classes.

Published in: Technology, Education

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Making TestAutomation Work in Agile ProjectsBelgium Testing Days 2012 Lisa Crispin With Material from Janet Gregory 1
  • 2. Introductions: Experience, Goals Form groups:  New to automation  One to two years experience w/ automation  More than two years experience Talk with at least two other people  Tell each other your learning goals for today  Note the most interesting one you hear 2
  • 3. About me…Programmer, tester, agile team member 3 Copyright 2012: Lisa Crispin
  • 4. Introduction - You Main role on team? Programming, automation experience? Using agile approach? What have you automated? (Test, CI,deployment...) 4 Copyright 2012: Lisa Crispin
  • 5. TakeawaysFoundation for successful test automation “Whole Team” approach What/when to automate Identifying, overcoming barriers Choosing, implementing tools First stepsWe won’t do any hands-on automation, but will demo some examples 5 Copyright 2012: Lisa Crispin
  • 6. Exercise: Your Learning Goals Write one interesting learning goal you heard from another participant at the start of the class on a sticky note. Write your number one learning goal for today on a sticky note. Put the sticky notes on your table group “learning goals” sheet Group similar ones together 6
  • 7. Let‟s start by defining “agile”Agile teams:  Deliver business value frequently  at a sustainable pace  while adapting to the changing needs of the business Source: Elisabeth Hendrickson 7 Copyright 2012: Lisa Crispin
  • 8. The key is “sustainable pace”Technical debt slows us down 8 Copyright 2012: Lisa Crispin
  • 9. The DeathSpiral
  • 10. High technical debt +insufficient automation = evenless time Copyright 2012: Lisa Crispin
  • 11. Barriers to Test AutomationWhat‟s holding you back? 11 Copyright 2012: Lisa Crispin
  • 12. Exercise: Your barriersIndividually and silently – write one barrier hindering your team per sticky notePut these on the “impediments” wall chart for your table groupTalk to your fellow group members – do you see any patterns? 12 Copyright 2012: Lisa Crispin
  • 13. Questions? 13 Copyright 2012: Lisa Crispin
  • 14. The Whole-Team Approach Automated tests are code Respecting the tests Collaborating Commitment to quality Return on investment 14 Copyright 2012: Lisa Crispin
  • 15. Automated tests are codepublic class CalculatorFixture extends ColumnFixture { public String startDate; public String endDate; public double startBalance; public double endBalance; public String irrTarget;private Calculator calculator = new Calculator(); private Double irr; 15 Copyright 2012: Lisa Crispin
  • 16. Test code deserves the same respect asproduction code! (Rodney Dangerfield, a comedian whose tag line was “I don’t get no respect… )
  • 17. Source: Gojko Adzic, StarEast 2011 keynote
  • 18. Source: Gojko Adzic, StarEast 2011 keynote
  • 19. Testers are especially good at… Eliciting examples Turning them into tests Ensuring the right testing gets done Exploratory testing 19 Copyright 2012: Lisa Crispin
  • 20. Give testers time to do what we do best How? 20
  • 21. 21The Whole-Team Approach
  • 22. Experiment: Iteration 1 Pair up: one will be tester, one programmer Sit back to back so you face away from each other Tester gets a drawing which needs to be replicated Tester tells the programmer what to draw, all at one time. Programmer draws the shapes based on what the tester explained.  No talking during „coding‟! Tester “tests” the drawing, reports “bugs” on index cards Programmer fixes the “bugs” How long did it take? Will the customer be 22 happy? Copyright 2012: Lisa Crispin
  • 23. Experiment: Iteration 2 Collaborate! Tester tells programmer what to draw, watches the programmer draw, answers questions, points out „defects‟ for programmer to fix immediately  (Don‟t show the programmer the drawing, that makes it too easy, we‟re trying to simulate real coding) How long did it take? Will the customer be happy? Thanks to the members of the agile-games group and Kane Mar for ideas & pictures for this game 23 Copyright 2012: Lisa Crispin
  • 24. Ways to collaborate Pair 24 Copyright 2012: Lisa Crispin
  • 25. Team responsibilityAutomate all regression tests 25 Copyright 2012: Lisa Crispin
  • 26. 26Copyright 2012: Lisa Crispin
  • 27. Commitment to qualityWhat‟s your team‟s commitment?The best possible software product? 27 Copyright 2012: Lisa Crispin
  • 28. MeaningfulCommitment Experimen t Learn 28 Copyright 2012: Lisa Crispin
  • 29. Under-commitPlan less work than you think you can doIncluding all test automation Copyright 2012: Lisa Crispin
  • 30. Learn to write maintainable tests Get over the “hump of pain” From Gerard Meszaros’ XUnit Test Patterns 30 Copyright 2012: Lisa Crispin
  • 31. Whole-team software developmentCreate a Expand user tests – story Story Tests Automate Write Q2 Tests Pair,Customer Start “Show (Q2) thinking Me” Tests how to code TDD Exploratory testing Product owner Product owner/ Tester Tester Tester/Programmer Programmer Customer User Acceptance 31
  • 32. Exercise •Think of an experiment to get your whole team engaged in automating tests •Share with your table group •Pick two to share with the class Copyright 2012: Lisa Crispin
  • 33. Getting Over the Hump The test automation pyramid The agile testing quadrants What should be automated What shouldnt Difficult areas 33 Copyright 2012: Lisa Crispin
  • 34. Test Automation Pyramid 34 Copyright 2012: Lisa Crispin
  • 35. Agile Testing Quadrants 35 Copyright 2012: Lisa Crispin
  • 36. Shared Understanding Start with tests Collaborate 36 Copyright 2012: Lisa Crispin
  • 37. What Should We Automate? Quadrant 1 tests  Unit, component, TDD Quadrant 2 tests  API, service-level Quadrant 4 tests  Load, performance, stress Quadrant 3 tests?  Leverage automation where useful 37 Copyright 2012: Lisa Crispin
  • 38. What Shouldn‟t We Automate? Quadrant 2 tests  Wizard of Oz, prototyping Quadrant 3 tests  Usability, UAT, ET Tests that will never fail?  Assess risk ROI not enough  One-off tests 38 Copyright 2012: Lisa Crispin
  • 39. Where Should We Be Careful? GUI tests  Watch ROI End-to-End tests  Push testing down to lowest level Remember the Pyramid 39 Copyright 2012: Lisa Crispin
  • 40. Hard to Automate? Legacy code  Hard to automate, or just lack of skill?  “Working Effectively with Legacy Code” – Feathers  “Strangling” – Fowler, Thomas 40 Copyright 2012: Lisa Crispin
  • 41. Exercise: Low-Hanging Fruit 41 Copyright 2012: Lisa Crispin
  • 42. Agile Automation Strategy What hurts the most Layered approach Applying agile principles  Small chunks/thin slices  Smart test design Choosing the right tools 42 Copyright 2012: Lisa Crispin
  • 43. What Hurts the Most Use retrospectives Keep an impediment backlog 43 Copyright 2012: Lisa Crispin
  • 44. Multi-Layered ApproachExample:  Learn TDD at unit level  Automate GUI smoke tests 44 Copyright 2012: Lisa Crispin
  • 45. Simplicity  Address one or two needs at a time  Understand the problem first  Try simplest approach first  Work in small chunks, thin slices  Incremental & iterative 45 Copyright 2012: Lisa Crispin
  • 46. Automate a Slice at a TimeExample: 4-step UI to validate, upload profit sharing contribution data• Thread 1: All four pages with navigation• Thread 2: Select year, enter description on page 1, display on page 2, browse and upload file on page 2• Thread 3: Validate data in file, display on page 3• Thread 4: Persist data, display „success‟ message on page 4 46 Copyright 2012: Lisa Crispin
  • 47. Thin Slice Example 47 Copyright 2012: Lisa Crispin
  • 48. Exercise: Thin SlicesHere‟s our user story:As an internet shopper, I want to create anaccount so that I do not have to enter my addressand billing information each time I make apurchaseDraw a mind map for this story on a big sheet ofpaperIdentify a basic end-to-end slice of functionalitythat can be coded, tested, and automated.If you have time, identify additional slices. 48 Copyright 2012: Lisa Crispin
  • 49. Test Design Patterns/Principles  Code design patterns  One clear purpose  Don‟t Repeat Yourself 49
  • 50. Demo 50 Copyright 2012: Lisa Crispin
  • 51. Iterative Feedback Spike two different approaches Pick one to try for N # of iterations Use retrospectives to evaluate 51 Copyright 2012: Lisa Crispin
  • 52. Learn by Doing Courage – don‟t be afraid to fail Production code practices for test code Incremental, thin slices Experiment 52 Copyright 2012: Lisa Crispin
  • 53. Questions About Automation Strategy? 53 Copyright 2012: Lisa Crispin
  • 54. Choosing Tools Team effort Time Requirements Focus on goals, problems, not tools. Experiment 54 Copyright 2012: Lisa Crispin
  • 55. Understand the Purpose Who‟s using the tests? What for? What‟s being automated? Existing tools, environment Who‟s doing what for automating? 55 Copyright 2012: Lisa Crispin
  • 56. What Fits Your Situation• Existing skills• Language of application under test• Collaboration needs• What‟s being automated• Life span, future use of tests 56 Copyright 2012: Lisa Crispin
  • 57. Test Drivers/Frameworks Automation layer Fit for everyone on team Try out more than one Let‟s look at more examples 57 Copyright 2012: Lisa Crispin
  • 58. Where To Find Tools - aa-ftt spreadsheet 58 Copyright 2012: Lisa Crispin
  • 59. Example: My Team‟s Tool Choices• IntelliJ Idea• Jenkins, ant, Maven• JUnit• FitNesse• Canoo WebTest• Watir• JMeter• Selenium 2.0 / WebDriver with Geb framework 59 Copyright 2012: Lisa Crispin
  • 60. Exercise: Tools 60 Copyright 2012: Lisa Crispin
  • 61. Making Test Automation Work Time to do it right Learning culture Testable architecture Test data Managing tests 61 Copyright 2012: Lisa Crispin
  • 62. Time To Do It Right Limit scope, don‟t over-commit Write automation task cards Quality must be team goal Long-term, will let you go faster 62 Copyright 2012: Lisa Crispin
  • 63. Learning Culture OK to make mistakes Lots of small experiments Slack Evolve right design 63 Copyright 2012: Lisa Crispin
  • 64. Testable Architecture• Layered architecture • eg. UI, business logic, data access• Ports and Adapters pattern • App can work without UI or database • Ports accept outside events • Adapters convert for human or automated users 64 Copyright 2012: Lisa Crispin
  • 65. Test Data Avoid database access when possible Setup/Teardown  Independent, rerunnable tests Canonical data  Refresh before each test run Customizable data for ET Production-like data  Get customers to provide example data 65 Copyright 2012: Lisa Crispin
  • 66. Managing Automated Tests Tests as Documentation Continuous Integration Reporting results Metrics 66 Copyright 2012: Lisa Crispin
  • 67. Tests as Documentation Understandable Who will really use them? Once passing, must always pass 67 Copyright 2012: Lisa Crispin
  • 68. Any Example Can Become a Test 68 Copyright 2012: Lisa Crispin
  • 69. Given/Then/When ExampleScenario: Valid name search returns resultsGIVEN that Kant is a supervisor with employeesAND Kant has an employee named SmithWHEN Kant navigates to the employee name search pageAND enters the value “S”THEN Kant will see a search result that includes Smith 69 Copyright 2012: Lisa Crispin
  • 70. Continuous integration/testing……short feedback loop 70 Copyright 2012: Lisa Crispin
  • 71. Keep tests passingStop the line to fix problems 71 Copyright 2012: Lisa Crispin
  • 72. Test Management Tools Manage tests, code together Some tools have own management What problem are you trying to solve? 72 Copyright 2012: Lisa Crispin
  • 73. Exercise: Tests as Documentation 73 Copyright 2012: Lisa Crispin
  • 74. Key Success Factors Thin Slices Specificatio n by Whole Team Example ApproachShortFeedbackLoops Experiments 74
  • 75. Exercise: Breaking Barriers 75 Copyright 2012: Lisa Crispin
  • 76. Remember It‟s a team problem! Tackle automation problems with diversity Try small experiments Baby steps – start simple 76 Copyright 2012: Lisa Crispin
  • 77. Questions? “Aha” Moments? 77 Copyright 2012: Lisa Crispin
  • 78. Agile Testing: A Practical Guide for Testers and AgileTeamsBy Lisa Crispin and Janet Copyright 2012: Lisa Crispin 78
  • 79. Experiences of Test AutomationDorothy Graham and Mark Fewster Copyright 2012: Lisa Crispin 79
  • 80. Beautiful Testing: Leading Professionals Reveal HowThey Improve SoftwareEdited by Tim Riley, Adam GoucherIncludes chapter by yours truly Copyright 2012: Lisa Crispin 80
  • 81. Test PatternsXunit Test Patterns: Refactoring Test CodeBy Gerard Meszaros Copyright 2012: Lisa Crispin 81
  • 82. Specification by ExampleHow successful teams deliver the rightsoftwareGojko AdzicCase studies from > 50 teams 82 Copyright 2012: Lisa Crispin Copyright 2008 Janet Gregory, DragonFire
  • 83. Agile Test Automation Resources 83 Copyright 2012: Lisa Crispin