Making TestAutomation Work in  Agile ProjectsBelgium Testing Days 2012          Lisa Crispin With Material from Janet Greg...
Introductions: Experience, Goals Form groups:    New to automation    One to two years experience w/ automation    Mor...
About me…Programmer, tester, agile team member                                              3               Copyright 2012...
Introduction - You   Main role on team?   Programming, automation experience?   Using agile approach?   What have you ...
TakeawaysFoundation for successful test automation “Whole Team” approach What/when to automate Identifying, overcoming ...
Exercise: Your Learning Goals Write one interesting learning goal you heard  from another participant at the start of the...
Let‟s start by defining “agile”Agile teams:     Deliver business value      frequently     at a sustainable pace     wh...
The key is “sustainable pace”Technical debt slows us down                                                8                ...
The DeathSpiral
High technical debt +insufficient automation = evenless time          Copyright 2012: Lisa Crispin
Barriers to Test AutomationWhat‟s holding you back?                                               11                Copyri...
Exercise: Your barriersIndividually and silently – write one barrier  hindering your team per sticky notePut these on the ...
Questions?                                            13             Copyright 2012: Lisa Crispin
The Whole-Team Approach   Automated tests are code   Respecting the tests   Collaborating   Commitment to quality   R...
Automated tests are codepublic class CalculatorFixture extends ColumnFixture {  public String startDate;  public String en...
Test code deserves the same respect asproduction code!                           (Rodney                           Dangerf...
Source: Gojko Adzic, StarEast 2011 keynote
Source: Gojko Adzic, StarEast 2011 keynote
Testers are especially good at… Eliciting examples Turning them into tests Ensuring the right testing gets done Explor...
Give testers time to do what we do best                       How?                                      20
21The Whole-Team Approach
Experiment: Iteration 1   Pair up: one will be tester, one programmer   Sit back to back so you face away from each    o...
Experiment: Iteration 2   Collaborate!   Tester tells programmer what to draw, watches    the programmer draw, answers q...
Ways to collaborate                                        Pair                                               24          ...
Team responsibilityAutomate all regression tests                                              25               Copyright 2...
26Copyright 2012: Lisa Crispin
Commitment to qualityWhat‟s your team‟s commitment?The best possible software product?                                    ...
MeaningfulCommitment                                            Experimen                                            t    ...
Under-commitPlan less work than you think you can doIncluding all test automation                Copyright 2012: Lisa Cris...
Learn to write maintainable tests   Get over the “hump of pain”                                    From Gerard Meszaros’ ...
Whole-team software developmentCreate a       Expand  user         tests – story          Story                Tests      ...
Exercise •Think of an experiment to get your whole team engaged in automating tests •Share with your table group •Pick two...
Getting Over the Hump   The test automation pyramid   The agile testing quadrants   What should be automated   What sh...
Test Automation Pyramid                                             34              Copyright 2012: Lisa Crispin
Agile Testing Quadrants                                              35               Copyright 2012: Lisa Crispin
Shared Understanding Start with tests Collaborate                                                    36                 ...
What Should We Automate? Quadrant 1 tests    Unit, component, TDD Quadrant 2 tests    API, service-level Quadrant 4 t...
What Shouldn‟t We Automate? Quadrant 2 tests    Wizard of Oz, prototyping Quadrant 3 tests    Usability, UAT, ET Test...
Where Should We Be Careful? GUI tests    Watch ROI End-to-End tests    Push testing down to lowest level Remember the...
Hard to Automate? Legacy code    Hard to automate, or just lack of skill?    “Working Effectively with Legacy Code” –  ...
Exercise: Low-Hanging Fruit                                              41               Copyright 2012: Lisa Crispin
Agile Automation Strategy What hurts the most Layered approach Applying agile principles    Small chunks/thin slices  ...
What Hurts the Most   Use retrospectives   Keep an impediment backlog                                                   ...
Multi-Layered ApproachExample:   Learn TDD at unit level   Automate GUI smoke tests                                     ...
Simplicity    Address one or two needs at a time    Understand the problem first    Try simplest approach first    Wor...
Automate a Slice at a TimeExample: 4-step UI to validate, upload profit  sharing contribution data• Thread 1: All four pag...
Thin Slice Example                                             47              Copyright 2012: Lisa Crispin
Exercise: Thin SlicesHere‟s our user story:As an internet shopper, I want to create anaccount so that I do not have to ent...
Test Design Patterns/Principles                    Code design patterns                    One clear purpose            ...
Demo                                      50       Copyright 2012: Lisa Crispin
Iterative Feedback   Spike two different approaches   Pick one to try for N # of iterations   Use retrospectives to eva...
Learn by Doing   Courage – don‟t be afraid to fail   Production code practices for test code   Incremental, thin slices...
Questions About Automation Strategy?                                             53              Copyright 2012: Lisa Cris...
Choosing Tools   Team effort   Time   Requirements   Focus on goals, problems, not tools.   Experiment               ...
Understand the Purpose   Who‟s using the tests? What for?   What‟s being automated?   Existing tools, environment   Wh...
What Fits Your Situation•   Existing skills•   Language of application under test•   Collaboration needs•   What‟s being a...
Test Drivers/Frameworks   Automation layer   Fit for everyone on team   Try out more than one   Let‟s look at more exa...
Where To Find Tools   www.softwareqatest.com/qattls1.html   www.testingfaqs.org   www.opensourcetesting.org   awta.wik...
Example: My Team‟s Tool Choices•   IntelliJ Idea•   Jenkins, ant, Maven•   JUnit•   FitNesse•   Canoo WebTest•   Watir•   ...
Exercise: Tools                                                 60                  Copyright 2012: Lisa Crispin
Making Test Automation Work   Time to do it right   Learning culture   Testable architecture   Test data   Managing t...
Time To Do It Right   Limit scope, don‟t over-commit   Write automation task cards   Quality must be team goal   Long-...
Learning Culture   OK to make mistakes   Lots of small experiments   Slack   Evolve right design                      ...
Testable Architecture• Layered architecture  •   eg. UI, business logic, data access• Ports and Adapters pattern  •   App ...
Test Data Avoid database access when possible Setup/Teardown   Independent, rerunnable tests Canonical data   Refresh...
Managing Automated Tests   Tests as Documentation   Continuous Integration   Reporting results   Metrics              ...
Tests as Documentation Understandable Who will really use them? Once passing, must always pass                         ...
Any Example Can Become a Test                                            68             Copyright 2012: Lisa Crispin
Given/Then/When ExampleScenario: Valid name search returns resultsGIVEN that Kant is a supervisor with employeesAND Kant h...
Continuous integration/testing……short feedback loop                                              70               Copyrigh...
Keep tests passingStop the line to fix problems                                              71               Copyright 20...
Test Management Tools Manage tests, code together Some tools have own management What problem are you trying to solve? ...
Exercise: Tests as Documentation                                             73              Copyright 2012: Lisa Crispin
Key Success Factors           Thin Slices                           Specificatio                           n by           ...
Exercise: Breaking Barriers                                              75               Copyright 2012: Lisa Crispin
Remember   It‟s a team problem!   Tackle automation problems with diversity   Try small experiments   Baby steps – sta...
Questions? “Aha” Moments?                                            77             Copyright 2012: Lisa Crispin
Agile Testing: A Practical Guide for Testers and AgileTeamsBy Lisa Crispin and Janet Gregorywww.agiletester.ca            ...
Experiences of Test AutomationDorothy Graham and Mark Fewster                          Copyright 2012: Lisa Crispin       ...
Beautiful Testing: Leading Professionals Reveal HowThey Improve SoftwareEdited by Tim Riley, Adam GoucherIncludes chapter ...
Test PatternsXunit Test Patterns: Refactoring Test CodeBy Gerard Meszaros                     Copyright 2012: Lisa Crispin...
Specification by ExampleHow successful teams deliver the rightsoftwareGojko AdzicCase studies from > 50 teams             ...
Agile Test Automation Resourcesdhemery.com/pdf/writing_maintainable_automated_acceptance_tests.pdflisacrispin.comjanetg...
Upcoming SlideShare
Loading in...5
×

Belgium Testing Days - Making Test Automation Work in Agile Projects

3,352

Published on

Slides from tutorial. Note that the most important part of the tutorial is the exercises, and I can't capture that in the slide deck. Please do not use these for public paid courses, I'm tired of our stuff being ripped off for agile testing classes.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,352
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
111
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Belgium Testing Days - Making Test Automation Work in Agile Projects

  1. 1. Making TestAutomation Work in Agile ProjectsBelgium Testing Days 2012 Lisa Crispin With Material from Janet Gregory 1
  2. 2. Introductions: Experience, Goals Form groups:  New to automation  One to two years experience w/ automation  More than two years experience Talk with at least two other people  Tell each other your learning goals for today  Note the most interesting one you hear 2
  3. 3. About me…Programmer, tester, agile team member 3 Copyright 2012: Lisa Crispin
  4. 4. Introduction - You Main role on team? Programming, automation experience? Using agile approach? What have you automated? (Test, CI,deployment...) 4 Copyright 2012: Lisa Crispin
  5. 5. TakeawaysFoundation for successful test automation “Whole Team” approach What/when to automate Identifying, overcoming barriers Choosing, implementing tools First stepsWe won’t do any hands-on automation, but will demo some examples 5 Copyright 2012: Lisa Crispin
  6. 6. Exercise: Your Learning Goals Write one interesting learning goal you heard from another participant at the start of the class on a sticky note. Write your number one learning goal for today on a sticky note. Put the sticky notes on your table group “learning goals” sheet Group similar ones together 6
  7. 7. Let‟s start by defining “agile”Agile teams:  Deliver business value frequently  at a sustainable pace  while adapting to the changing needs of the business Source: Elisabeth Hendrickson 7 Copyright 2012: Lisa Crispin
  8. 8. The key is “sustainable pace”Technical debt slows us down 8 Copyright 2012: Lisa Crispin
  9. 9. The DeathSpiral
  10. 10. High technical debt +insufficient automation = evenless time Copyright 2012: Lisa Crispin
  11. 11. Barriers to Test AutomationWhat‟s holding you back? 11 Copyright 2012: Lisa Crispin
  12. 12. Exercise: Your barriersIndividually and silently – write one barrier hindering your team per sticky notePut these on the “impediments” wall chart for your table groupTalk to your fellow group members – do you see any patterns? 12 Copyright 2012: Lisa Crispin
  13. 13. Questions? 13 Copyright 2012: Lisa Crispin
  14. 14. The Whole-Team Approach Automated tests are code Respecting the tests Collaborating Commitment to quality Return on investment 14 Copyright 2012: Lisa Crispin
  15. 15. Automated tests are codepublic class CalculatorFixture extends ColumnFixture { public String startDate; public String endDate; public double startBalance; public double endBalance; public String irrTarget;private Calculator calculator = new Calculator(); private Double irr; 15 Copyright 2012: Lisa Crispin
  16. 16. Test code deserves the same respect asproduction code! (Rodney Dangerfield, a comedian whose tag line was “I don’t get no respect… )
  17. 17. Source: Gojko Adzic, StarEast 2011 keynote
  18. 18. Source: Gojko Adzic, StarEast 2011 keynote
  19. 19. Testers are especially good at… Eliciting examples Turning them into tests Ensuring the right testing gets done Exploratory testing 19 Copyright 2012: Lisa Crispin
  20. 20. Give testers time to do what we do best How? 20
  21. 21. 21The Whole-Team Approach
  22. 22. Experiment: Iteration 1 Pair up: one will be tester, one programmer Sit back to back so you face away from each other Tester gets a drawing which needs to be replicated Tester tells the programmer what to draw, all at one time. Programmer draws the shapes based on what the tester explained.  No talking during „coding‟! Tester “tests” the drawing, reports “bugs” on index cards Programmer fixes the “bugs” How long did it take? Will the customer be 22 happy? Copyright 2012: Lisa Crispin
  23. 23. Experiment: Iteration 2 Collaborate! Tester tells programmer what to draw, watches the programmer draw, answers questions, points out „defects‟ for programmer to fix immediately  (Don‟t show the programmer the drawing, that makes it too easy, we‟re trying to simulate real coding) How long did it take? Will the customer be happy? Thanks to the members of the agile-games group and Kane Mar for ideas & pictures for this game 23 Copyright 2012: Lisa Crispin
  24. 24. Ways to collaborate Pair 24 Copyright 2012: Lisa Crispin
  25. 25. Team responsibilityAutomate all regression tests 25 Copyright 2012: Lisa Crispin
  26. 26. 26Copyright 2012: Lisa Crispin
  27. 27. Commitment to qualityWhat‟s your team‟s commitment?The best possible software product? 27 Copyright 2012: Lisa Crispin
  28. 28. MeaningfulCommitment Experimen t Learn 28 Copyright 2012: Lisa Crispin
  29. 29. Under-commitPlan less work than you think you can doIncluding all test automation Copyright 2012: Lisa Crispin
  30. 30. Learn to write maintainable tests Get over the “hump of pain” From Gerard Meszaros’ XUnit Test Patterns 30 Copyright 2012: Lisa Crispin
  31. 31. Whole-team software developmentCreate a Expand user tests – story Story Tests Automate Write Q2 Tests Pair,Customer Start “Show (Q2) thinking Me” Tests how to code TDD Exploratory testing Product owner Product owner/ Tester Tester Tester/Programmer Programmer Customer User Acceptance 31
  32. 32. Exercise •Think of an experiment to get your whole team engaged in automating tests •Share with your table group •Pick two to share with the class Copyright 2012: Lisa Crispin
  33. 33. Getting Over the Hump The test automation pyramid The agile testing quadrants What should be automated What shouldnt Difficult areas 33 Copyright 2012: Lisa Crispin
  34. 34. Test Automation Pyramid 34 Copyright 2012: Lisa Crispin
  35. 35. Agile Testing Quadrants 35 Copyright 2012: Lisa Crispin
  36. 36. Shared Understanding Start with tests Collaborate 36 Copyright 2012: Lisa Crispin
  37. 37. What Should We Automate? Quadrant 1 tests  Unit, component, TDD Quadrant 2 tests  API, service-level Quadrant 4 tests  Load, performance, stress Quadrant 3 tests?  Leverage automation where useful 37 Copyright 2012: Lisa Crispin
  38. 38. What Shouldn‟t We Automate? Quadrant 2 tests  Wizard of Oz, prototyping Quadrant 3 tests  Usability, UAT, ET Tests that will never fail?  Assess risk ROI not enough  One-off tests 38 Copyright 2012: Lisa Crispin
  39. 39. Where Should We Be Careful? GUI tests  Watch ROI End-to-End tests  Push testing down to lowest level Remember the Pyramid 39 Copyright 2012: Lisa Crispin
  40. 40. Hard to Automate? Legacy code  Hard to automate, or just lack of skill?  “Working Effectively with Legacy Code” – Feathers  “Strangling” – Fowler, Thomas 40 Copyright 2012: Lisa Crispin
  41. 41. Exercise: Low-Hanging Fruit 41 Copyright 2012: Lisa Crispin
  42. 42. Agile Automation Strategy What hurts the most Layered approach Applying agile principles  Small chunks/thin slices  Smart test design Choosing the right tools 42 Copyright 2012: Lisa Crispin
  43. 43. What Hurts the Most Use retrospectives Keep an impediment backlog 43 Copyright 2012: Lisa Crispin
  44. 44. Multi-Layered ApproachExample:  Learn TDD at unit level  Automate GUI smoke tests 44 Copyright 2012: Lisa Crispin
  45. 45. Simplicity  Address one or two needs at a time  Understand the problem first  Try simplest approach first  Work in small chunks, thin slices  Incremental & iterative 45 Copyright 2012: Lisa Crispin
  46. 46. Automate a Slice at a TimeExample: 4-step UI to validate, upload profit sharing contribution data• Thread 1: All four pages with navigation• Thread 2: Select year, enter description on page 1, display on page 2, browse and upload file on page 2• Thread 3: Validate data in file, display on page 3• Thread 4: Persist data, display „success‟ message on page 4 46 Copyright 2012: Lisa Crispin
  47. 47. Thin Slice Example 47 Copyright 2012: Lisa Crispin
  48. 48. Exercise: Thin SlicesHere‟s our user story:As an internet shopper, I want to create anaccount so that I do not have to enter my addressand billing information each time I make apurchaseDraw a mind map for this story on a big sheet ofpaperIdentify a basic end-to-end slice of functionalitythat can be coded, tested, and automated.If you have time, identify additional slices. 48 Copyright 2012: Lisa Crispin
  49. 49. Test Design Patterns/Principles  Code design patterns  One clear purpose  Don‟t Repeat Yourself 49
  50. 50. Demo 50 Copyright 2012: Lisa Crispin
  51. 51. Iterative Feedback Spike two different approaches Pick one to try for N # of iterations Use retrospectives to evaluate 51 Copyright 2012: Lisa Crispin
  52. 52. Learn by Doing Courage – don‟t be afraid to fail Production code practices for test code Incremental, thin slices Experiment 52 Copyright 2012: Lisa Crispin
  53. 53. Questions About Automation Strategy? 53 Copyright 2012: Lisa Crispin
  54. 54. Choosing Tools Team effort Time Requirements Focus on goals, problems, not tools. Experiment 54 Copyright 2012: Lisa Crispin
  55. 55. Understand the Purpose Who‟s using the tests? What for? What‟s being automated? Existing tools, environment Who‟s doing what for automating? 55 Copyright 2012: Lisa Crispin
  56. 56. What Fits Your Situation• Existing skills• Language of application under test• Collaboration needs• What‟s being automated• Life span, future use of tests 56 Copyright 2012: Lisa Crispin
  57. 57. Test Drivers/Frameworks Automation layer Fit for everyone on team Try out more than one Let‟s look at more examples 57 Copyright 2012: Lisa Crispin
  58. 58. Where To Find Tools www.softwareqatest.com/qattls1.html www.testingfaqs.org www.opensourcetesting.org awta.wikispaces.com/2009ToolsList groups.yahoo.com/group/agile-testing http://bit.ly/AgileTestTools - aa-ftt spreadsheet 58 Copyright 2012: Lisa Crispin
  59. 59. Example: My Team‟s Tool Choices• IntelliJ Idea• Jenkins, ant, Maven• JUnit• FitNesse• Canoo WebTest• Watir• JMeter• Selenium 2.0 / WebDriver with Geb framework 59 Copyright 2012: Lisa Crispin
  60. 60. Exercise: Tools 60 Copyright 2012: Lisa Crispin
  61. 61. Making Test Automation Work Time to do it right Learning culture Testable architecture Test data Managing tests 61 Copyright 2012: Lisa Crispin
  62. 62. Time To Do It Right Limit scope, don‟t over-commit Write automation task cards Quality must be team goal Long-term, will let you go faster 62 Copyright 2012: Lisa Crispin
  63. 63. Learning Culture OK to make mistakes Lots of small experiments Slack Evolve right design 63 Copyright 2012: Lisa Crispin
  64. 64. Testable Architecture• Layered architecture • eg. UI, business logic, data access• Ports and Adapters pattern • App can work without UI or database • Ports accept outside events • Adapters convert for human or automated users 64 Copyright 2012: Lisa Crispin
  65. 65. Test Data Avoid database access when possible Setup/Teardown  Independent, rerunnable tests Canonical data  Refresh before each test run Customizable data for ET Production-like data  Get customers to provide example data 65 Copyright 2012: Lisa Crispin
  66. 66. Managing Automated Tests Tests as Documentation Continuous Integration Reporting results Metrics 66 Copyright 2012: Lisa Crispin
  67. 67. Tests as Documentation Understandable Who will really use them? Once passing, must always pass 67 Copyright 2012: Lisa Crispin
  68. 68. Any Example Can Become a Test 68 Copyright 2012: Lisa Crispin
  69. 69. Given/Then/When ExampleScenario: Valid name search returns resultsGIVEN that Kant is a supervisor with employeesAND Kant has an employee named SmithWHEN Kant navigates to the employee name search pageAND enters the value “S”THEN Kant will see a search result that includes Smith 69 Copyright 2012: Lisa Crispin
  70. 70. Continuous integration/testing……short feedback loop 70 Copyright 2012: Lisa Crispin
  71. 71. Keep tests passingStop the line to fix problems 71 Copyright 2012: Lisa Crispin
  72. 72. Test Management Tools Manage tests, code together Some tools have own management What problem are you trying to solve? 72 Copyright 2012: Lisa Crispin
  73. 73. Exercise: Tests as Documentation 73 Copyright 2012: Lisa Crispin
  74. 74. Key Success Factors Thin Slices Specificatio n by Whole Team Example ApproachShortFeedbackLoops Experiments 74
  75. 75. Exercise: Breaking Barriers 75 Copyright 2012: Lisa Crispin
  76. 76. Remember It‟s a team problem! Tackle automation problems with diversity Try small experiments Baby steps – start simple 76 Copyright 2012: Lisa Crispin
  77. 77. Questions? “Aha” Moments? 77 Copyright 2012: Lisa Crispin
  78. 78. Agile Testing: A Practical Guide for Testers and AgileTeamsBy Lisa Crispin and Janet Gregorywww.agiletester.ca Copyright 2012: Lisa Crispin 78
  79. 79. Experiences of Test AutomationDorothy Graham and Mark Fewster Copyright 2012: Lisa Crispin 79
  80. 80. Beautiful Testing: Leading Professionals Reveal HowThey Improve SoftwareEdited by Tim Riley, Adam GoucherIncludes chapter by yours truly Copyright 2012: Lisa Crispin 80
  81. 81. Test PatternsXunit Test Patterns: Refactoring Test CodeBy Gerard Meszaros Copyright 2012: Lisa Crispin 81
  82. 82. Specification by ExampleHow successful teams deliver the rightsoftwareGojko AdzicCase studies from > 50 teams 82 Copyright 2012: Lisa Crispin Copyright 2008 Janet Gregory, DragonFire
  83. 83. Agile Test Automation Resourcesdhemery.com/pdf/writing_maintainable_automated_acceptance_tests.pdflisacrispin.comjanetgregory.cagokjo.netexampler.comagile-testing@yahoogroups.comtestobsessed.comtestingreflections.compairwith.us 83 Copyright 2012: Lisa Crispin
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×