Making TestAutomation Work in  Agile Projects Agile Testing Days 2012          Lisa Crispin With Material from Janet Grego...
Introductions: Experience, Goals§  Form groups:     §  New to automation     §  One to two years experience w/ automati...
About me…Programmer, tester, agile team member  Ever-evolving views on automation                                         ...
Introduction - You                                               4                Copyright 2012: Lisa Crispin
TakeawaysSuccessful test automation is an innovation!•  Let go of traditional roles, reach across them•  Visualize how tes...
Exercise: Your Learning Goals§  Write one interesting learning goal you heard    from another participant at the start of...
Let’s start by defining “agile”Agile teams:  q    Deliver business value        frequently  q    at a sustainable pace  ...
The key is “sustainable pace”Technical debt slows us down                                                8                ...
The DeathSpiral
High technical debt +insufficient automation =even less time          Copyright 2012: Lisa Crispin
Barriers to Test AutomationWhat’s holding you back?                                               11                Copyri...
Exercise: Your barriersIndividually and silently – write one barrier  hindering your team per sticky notePut these on the ...
Questions?                                            13             Copyright 2012: Lisa Crispin
The Whole-Team Approach§    Automated tests are code§    Respecting the tests§    Collaborating§    Commitment to qual...
Automated tests are codepublic class CalculatorFixture extends ColumnFixture {  public String startDate;  public String en...
Source: Gojko Adzic, StarEast 2011 keynote
Source: Gojko Adzic, StarEast 2011 keynote
Testers are especially good at…§ Eliciting examples§ Turning them into tests§ Ensuring the right testing gets done§ Ex...
Give testers time to do what we do best                       How?                                      19
20The Whole-Team Approach
Experiment: Iteration 1§  Pair up: one will be tester, one programmer§  Sit back to back so you face away from each othe...
Experiment: Iteration 2§    Now, turn to face each other.§    Tester:       §  Still don’t show your picture!       § ...
Ways to collaborate                           Pair:                           Tester-Coder                           Teste...
Team responsibilityAutomate all regression tests                                              24               Copyright 2...
25Copyright 2012: Lisa Crispin
Commitment to qualityWhat’s your team’s commitment?The best possible software product?                                    ...
MeaningfulCommitment                                            Experiment                                            Lear...
Under-commitPlan less work than you think you can doIncluding all test automation                Copyright 2012: Lisa Cris...
Learn to write maintainable tests§    Get over the “hump of pain”                                      From Gerard Meszar...
Whole-team software developmentCreate a       Expand  user         tests – story          Story                Tests      ...
Exercise • Think of an experiment to get your whole team engaged in automating tests • Share with your table group • Pick ...
Getting Over the Hump§    The test automation pyramid§    The agile testing quadrants§    What should be automated§   ...
Test Automation Pyramid                                             33              Copyright 2012: Lisa Crispin
Test Automation VolcanoFrom MattBarcomb            Copyright 2012: Lisa Crispin
Agile Testing Quadrants                                              35               Copyright 2012: Lisa Crispin
Shared Understanding§  Start with tests§  Collaborate                                                      36           ...
What Should We Automate?§  Quadrant 1 tests     §  Unit, component, TDD§  Quadrant 2 tests     §  API, service-level§...
What Shouldn’t We Automate?§  Quadrant 2 tests     §  Wizard of Oz, prototyping§  Quadrant 3 tests     §  Usability, U...
Where Should We Be Careful?§  GUI tests     §  Watch ROI§  End-to-End tests     §  Push testing down to lowest level§...
Hard to Automate?§  Legacy code     §  Hard to automate, or just lack of skill?     §  “Working Effectively with Legacy...
Exercise: Low-Hanging Fruit  Any “easy wins”?                                              41               Copyright 2012...
Agile Automation Strategy§  What hurts the most§  Layered approach§  Applying agile principles     §  Small chunks/thi...
What Hurts the Most§    Use retrospectives§    Keep an impediment backlog                                               ...
Multi-Layered ApproachExample:  §  Learn TDD at unit level  §  Automate GUI smoke tests                                 ...
Simplicity §  Address one or two needs at a time §  Understand the problem first §  Try simplest approach first §  Wor...
Automate a Slice at a TimeExample: 4-step UI to validate, upload profit   sharing contribution data•  Thread 1: All four p...
Thin Slice Example                                             47              Copyright 2012: Lisa Crispin
Exercise: Thin SlicesHere’s our user story (or theme):As an internet shopper, I want to create anaccount so that I do not ...
Test Design Patterns/Principles                 §    Code design patterns                 §    One clear purpose        ...
Page Object PatternA “page object” is a test object that holdsthe details of all the elements on a webpage that might be i...
More on the Page Object Patternhttp://www.beer30.org/?p=54                                               51               ...
Demo: semi-automating exploratory tests                                               52                Copyright 2012: Li...
Iterative Feedback§    Spike two different approaches§    Pick one to try for N # of iterations§    Use retrospectives ...
Learn by Doing§    Courage – don’t be afraid to fail§    Production code practices for test code§    Incremental, thin ...
Questions About Automation Strategy?                                             55              Copyright 2012: Lisa Cris...
Choosing Tools, Frameworks, Drivers§    Team effort§    Time§    Requirements§    Focus on goals, problems, not tools....
Understand the Purpose§    Who’s using the tests? What for?§    What’s being automated?§    Existing tools, environment...
What Fits Your Situation•  Existing skills•  Language of application under test•  Collaboration needs•  What’s being autom...
Test Drivers/Frameworks§  Divide into sub-teams and prototype    solutions§  Have a “bake-off” showing each prototype   ...
Where To Find Tools§    www.softwareqatest.com/qattls1.html§    http://bit.ly/AgileTestTools§    www.testingfaqs.org§ ...
Example: One Team’s Tool Choices•    IntelliJ Idea•    Jenkins, ant, Maven•    JUnit•    FitNesse•    Canoo WebTest•    Wa...
Exercise: Test FrameworksWhat format would work for your team? EG,BDD style? Tables? Assertions? Scenarios?Think of the te...
Making Test Automation Work§    Time to do it right§    Experiments, bake-offs§    Testable architecture§    Test data...
Time To Do It Right§    Limit scope, don’t over-commit§    Write automation task cards§    Plan test code maintenance/ ...
Bake-offs§  Have sub-teams/pairs experiment with    prototypes§  Budget lots of time for learning,    experimenting§  L...
Testable Architecture•  Layered architecture  •  eg. UI, business logic, data access•  Ports and Adapters pattern  •  App ...
Test Data§  Avoid database access when possible§  Setup/Teardown  §  Independent, rerunnable tests§  Canonical data  §...
Managing Automated Tests§    Tests as Living Documentation§    Continuous Integration§    Reporting results§    Metric...
Tests as Living Documentation§  Understandable§  Who will really use them?§  Once passing, must always pass            ...
Any Example Can Become a Test                                            70             Copyright 2012: Lisa Crispin
Given/Then/When ExampleScenario: Valid name search returns resultsGIVEN that Kant is a supervisor with employeesAND Kant h...
Continuous integration/testing……short feedback loop                                              72               Copyrigh...
Keep tests passingStop the line to fix problems                                              73               Copyright 20...
Test Management Tools§  Manage tests, code together§  Some tools have own management§  What problem are you trying to s...
Exercise: Tests as Documentation                                             75              Copyright 2012: Lisa Crispin
Key Success Factors           Thin Slices                            Specification                            by Example   ...
Exercise: Breaking Barriers                                              77               Copyright 2012: Lisa Crispin
Remember§    It’s a team problem!§    Tackle automation problems with diversity§    Try small experiments§    Baby ste...
Questions? “Aha” Moments?                                            79             Copyright 2012: Lisa Crispin
Agile Testing: A Practical Guide for Testers and AgileTeamsBy Lisa Crispin and Janet Gregorywww.agiletester.ca            ...
Experiences of Test AutomationDorothy Graham and Mark Fewster                          Copyright 2012: Lisa Crispin       ...
Beautiful Testing: Leading Professionals Reveal HowThey Improve SoftwareEdited by Tim Riley, Adam GoucherIncludes chapter ...
Test PatternsXunit Test Patterns: Refactoring Test CodeBy Gerard Meszaros                     Copyright 2012: Lisa Crispin...
Specification by ExampleHow successful teams deliver the rightsoftwareGojko AdzicCase studies from > 50 teams             ...
Agile Test Automation Resources§ dhemery.com/pdf/writing_maintainable_automated_acceptance_tests.pdf§ http://bit.ly/Agil...
Upcoming SlideShare
Loading in...5
×

Atd agile automation

775

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
775
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
41
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Atd agile automation

  1. 1. Making TestAutomation Work in Agile Projects Agile Testing Days 2012 Lisa Crispin With Material from Janet Gregory 1
  2. 2. Introductions: Experience, Goals§  Form groups: §  New to automation §  One to two years experience w/ automation §  More than two years experience§  Talk with at least two other people §  Tell each other your learning goals for today §  Note the most interesting one you hear 2
  3. 3. About me…Programmer, tester, agile team member Ever-evolving views on automation 3 Copyright 2012: Lisa Crispin
  4. 4. Introduction - You 4 Copyright 2012: Lisa Crispin
  5. 5. TakeawaysSuccessful test automation is an innovation!•  Let go of traditional roles, reach across them•  Visualize how tests should look•  Be smart about when to automate•  Experiment to overcome obstacles•  Find/build the right toolsWe won’t do any hands-on automation, but will demo some examples 5 Copyright 2012: Lisa Crispin
  6. 6. Exercise: Your Learning Goals§  Write one interesting learning goal you heard from another participant at the start of the class on a sticky note.§  Write your number one learning goal for today on a sticky note.§  Put the sticky notes on your table group “learning goals” sheet§  Group similar ones together 6
  7. 7. Let’s start by defining “agile”Agile teams: q  Deliver business value frequently q  at a sustainable pace q  while adapting to the changing needs of the business Source: Elisabeth Hendrickson 7 Copyright 2012: Lisa Crispin
  8. 8. The key is “sustainable pace”Technical debt slows us down 8 Copyright 2012: Lisa Crispin
  9. 9. The DeathSpiral
  10. 10. High technical debt +insufficient automation =even less time Copyright 2012: Lisa Crispin
  11. 11. Barriers to Test AutomationWhat’s holding you back? 11 Copyright 2012: Lisa Crispin
  12. 12. Exercise: Your barriersIndividually and silently – write one barrier hindering your team per sticky notePut these on the “impediments” wall chart for your table groupWork with your group to identify patterns, group similar items 12 Copyright 2012: Lisa Crispin
  13. 13. Questions? 13 Copyright 2012: Lisa Crispin
  14. 14. The Whole-Team Approach§  Automated tests are code§  Respecting the tests§  Collaborating§  Commitment to quality§  Return on investment 14 Copyright 2012: Lisa Crispin
  15. 15. Automated tests are codepublic class CalculatorFixture extends ColumnFixture { public String startDate; public String endDate; public double startBalance; public double endBalance; public String irrTarget;private Calculator calculator = new Calculator(); private Double irr; 15 Copyright 2012: Lisa Crispin
  16. 16. Source: Gojko Adzic, StarEast 2011 keynote
  17. 17. Source: Gojko Adzic, StarEast 2011 keynote
  18. 18. Testers are especially good at…§ Eliciting examples§ Turning them into tests§ Ensuring the right testing gets done§ Exploratory testing 18 Copyright 2012: Lisa Crispin
  19. 19. Give testers time to do what we do best How? 19
  20. 20. 20The Whole-Team Approach
  21. 21. Experiment: Iteration 1§  Pair up: one will be tester, one programmer§  Sit back to back so you face away from each other§  Tester: You have a drawing which you need to instruct your developer to draw§  Programmer: Remain silent while working, only do what you’re instructed. §  No talking! §  No peeking!When “done”:§  Programmer – pass your drawing to the tester§  Tester – note problems on separate card/paperWill the customer be happy? 21 Copyright 2012: Lisa Crispin
  22. 22. Experiment: Iteration 2§  Now, turn to face each other.§  Tester: §  Still don’t show your picture! §  Tell the programmer what to draw, but - §  Watch what the programmer draws, and point out any mistakes immediately§  Programmer: §  Still no peeking! §  Ask questions, ask for feedback, make corrections immediately Thanks to the members of the agile-games group and Kane Mar for ideas & pictures for this game 22 Copyright 2012: Lisa Crispin
  23. 23. Ways to collaborate Pair: Tester-Coder Tester-Tester Any role – any role 23 Copyright 2012: Lisa Crispin
  24. 24. Team responsibilityAutomate all regression tests 24 Copyright 2012: Lisa Crispin
  25. 25. 25Copyright 2012: Lisa Crispin
  26. 26. Commitment to qualityWhat’s your team’s commitment?The best possible software product? 26 Copyright 2012: Lisa Crispin
  27. 27. MeaningfulCommitment Experiment Learn 27 Copyright 2012: Lisa Crispin
  28. 28. Under-commitPlan less work than you think you can doIncluding all test automation Copyright 2012: Lisa Crispin
  29. 29. Learn to write maintainable tests§  Get over the “hump of pain” From Gerard Meszaros’ XUnit Test Patterns 29 Copyright 2012: Lisa Crispin
  30. 30. Whole-team software developmentCreate a Expand user tests – story Story Tests Automate Write Q2 Tests Pair,Customer Start “Show (Q2) thinking Me” Tests how to code TDD Exploratory testing Product owner Product owner/ Tester Tester Tester/Programmer Programmer Customer User Acceptance 30
  31. 31. Exercise • Think of an experiment to get your whole team engaged in automating tests • Share with your table group • Pick two to share with the class Copyright 2012: Lisa Crispin
  32. 32. Getting Over the Hump§  The test automation pyramid§  The agile testing quadrants§  What should be automated§  What shouldnt§  Difficult areas 32 Copyright 2012: Lisa Crispin
  33. 33. Test Automation Pyramid 33 Copyright 2012: Lisa Crispin
  34. 34. Test Automation VolcanoFrom MattBarcomb Copyright 2012: Lisa Crispin
  35. 35. Agile Testing Quadrants 35 Copyright 2012: Lisa Crispin
  36. 36. Shared Understanding§  Start with tests§  Collaborate 36 Copyright 2012: Lisa Crispin
  37. 37. What Should We Automate?§  Quadrant 1 tests §  Unit, component, TDD§  Quadrant 2 tests §  API, service-level§  Quadrant 4 tests §  Load, performance, stress§  Quadrant 3 tests? §  Leverage automation where useful 37 Copyright 2012: Lisa Crispin
  38. 38. What Shouldn’t We Automate?§  Quadrant 2 tests §  Wizard of Oz, prototyping§  Quadrant 3 tests §  Usability, UAT, ET§  Tests that will never fail? §  Assess risk§  ROI not enough §  One-off tests 38 Copyright 2012: Lisa Crispin
  39. 39. Where Should We Be Careful?§  GUI tests §  Watch ROI§  End-to-End tests §  Push testing down to lowest level§  Remember the Pyramid 39 Copyright 2012: Lisa Crispin
  40. 40. Hard to Automate?§  Legacy code §  Hard to automate, or just lack of skill? §  “Working Effectively with Legacy Code” – Feathers §  “Strangling” – Fowler, Thomas 40 Copyright 2012: Lisa Crispin
  41. 41. Exercise: Low-Hanging Fruit Any “easy wins”? 41 Copyright 2012: Lisa Crispin
  42. 42. Agile Automation Strategy§  What hurts the most§  Layered approach§  Applying agile principles §  Small chunks/thin slices §  Smart test design§  Choosing the right tools 42 Copyright 2012: Lisa Crispin
  43. 43. What Hurts the Most§  Use retrospectives§  Keep an impediment backlog 43 Copyright 2012: Lisa Crispin
  44. 44. Multi-Layered ApproachExample: §  Learn TDD at unit level §  Automate GUI smoke tests 44 Copyright 2012: Lisa Crispin
  45. 45. Simplicity §  Address one or two needs at a time §  Understand the problem first §  Try simplest approach first §  Work in small chunks, thin slices §  Incremental & iterative 45 Copyright 2012: Lisa Crispin
  46. 46. Automate a Slice at a TimeExample: 4-step UI to validate, upload profit sharing contribution data•  Thread 1: All four pages with navigation•  Thread 2: Select year, enter description on page 1, display on page 2, browse and upload file on page 2•  Thread 3: Validate data in file, display on page 3•  Thread 4: Persist data, display ‘success’ message on page 4 46 Copyright 2012: Lisa Crispin
  47. 47. Thin Slice Example 47 Copyright 2012: Lisa Crispin
  48. 48. Exercise: Thin SlicesHere’s our user story (or theme):As an internet shopper, I want to create anaccount so that I do not have to enter my addressand billing information each time I make apurchaseDraw a mind map for this feature on a big sheet ofpaperIdentify a basic end-to-end slice of functionalitythat can be coded, tested, and automated.If you have time, identify additional slices. 48 Copyright 2012: Lisa Crispin
  49. 49. Test Design Patterns/Principles §  Code design patterns §  One clear purpose §  Don’t Repeat Yourself 49
  50. 50. Page Object PatternA “page object” is a test object that holdsthe details of all the elements on a webpage that might be involved in anautomated test. 50 Copyright 2012: Lisa Crispin
  51. 51. More on the Page Object Patternhttp://www.beer30.org/?p=54 51 Copyright 2012: Lisa Crispin
  52. 52. Demo: semi-automating exploratory tests 52 Copyright 2012: Lisa Crispin
  53. 53. Iterative Feedback§  Spike two different approaches§  Pick one to try for N # of iterations§  Use retrospectives to evaluate 53 Copyright 2012: Lisa Crispin
  54. 54. Learn by Doing§  Courage – don’t be afraid to fail§  Production code practices for test code§  Incremental, thin slices§  Experiment 54 Copyright 2012: Lisa Crispin
  55. 55. Questions About Automation Strategy? 55 Copyright 2012: Lisa Crispin
  56. 56. Choosing Tools, Frameworks, Drivers§  Team effort§  Time§  Requirements§  Focus on goals, problems, not tools.§  Experiment 56 Copyright 2012: Lisa Crispin
  57. 57. Understand the Purpose§  Who’s using the tests? What for?§  What’s being automated?§  Existing tools, environment§  Who’s doing what for automating? 57 Copyright 2012: Lisa Crispin
  58. 58. What Fits Your Situation•  Existing skills•  Language of application under test•  Collaboration needs•  What’s being automated•  Life span, future use of tests 58 Copyright 2012: Lisa Crispin
  59. 59. Test Drivers/Frameworks§  Divide into sub-teams and prototype solutions§  Have a “bake-off” showing each prototype 59 Copyright 2012: Lisa Crispin
  60. 60. Where To Find Tools§  www.softwareqatest.com/qattls1.html§  http://bit.ly/AgileTestTools§  www.testingfaqs.org§  www.opensourcetesting.org§  groups.yahoo.com/group/agile-testing§  Grow your own! 60 Copyright 2012: Lisa Crispin
  61. 61. Example: One Team’s Tool Choices•  IntelliJ Idea•  Jenkins, ant, Maven•  JUnit•  FitNesse•  Canoo WebTest•  Watir•  Robot Framework•  JMeter•  Selenium 2.0 / WebDriver with Geb framework 61 Copyright 2012: Lisa Crispin
  62. 62. Exercise: Test FrameworksWhat format would work for your team? EG,BDD style? Tables? Assertions? Scenarios?Think of the test format that might work foryour team. Write examples on sticky notes.Write down any particular types of testpatterns, frameworks and driverss you wantto learn more about.and place these sticky notes next to animpediment it might help overcome..Discuss with group how that tool might helpsolve the problem. Copyright 2012: Lisa Crispin 62
  63. 63. Making Test Automation Work§  Time to do it right§  Experiments, bake-offs§  Testable architecture§  Test data§  Managing tests 63 Copyright 2012: Lisa Crispin
  64. 64. Time To Do It Right§  Limit scope, don’t over-commit§  Write automation task cards§  Plan test code maintenance/ refactoring time§  Quality must be team goal§  Long-term, will let you go faster 64 Copyright 2012: Lisa Crispin
  65. 65. Bake-offs§  Have sub-teams/pairs experiment with prototypes§  Budget lots of time for learning, experimenting§  Look at pros & cons 65 Copyright 2012: Lisa Crispin
  66. 66. Testable Architecture•  Layered architecture •  eg. UI, business logic, data access•  Ports and Adapters pattern •  App can work without UI or database •  Ports accept outside events •  Adapters convert for human or automated users 66 Copyright 2012: Lisa Crispin
  67. 67. Test Data§  Avoid database access when possible§  Setup/Teardown §  Independent, rerunnable tests§  Canonical data §  Refresh before each test run§  Customizable data for ET§  Production-like data §  Get customers to provide example data 67 Copyright 2012: Lisa Crispin
  68. 68. Managing Automated Tests§  Tests as Living Documentation§  Continuous Integration§  Reporting results§  Metrics 68 Copyright 2012: Lisa Crispin
  69. 69. Tests as Living Documentation§  Understandable§  Who will really use them?§  Once passing, must always pass 69 Copyright 2012: Lisa Crispin
  70. 70. Any Example Can Become a Test 70 Copyright 2012: Lisa Crispin
  71. 71. Given/Then/When ExampleScenario: Valid name search returns resultsGIVEN that Kant is a supervisor with employeesAND Kant has an employee named SmithWHEN Kant navigates to the employee name search pageAND enters the value “S”THEN Kant will see a search result that includes Smith 71 Copyright 2012: Lisa Crispin
  72. 72. Continuous integration/testing……short feedback loop 72 Copyright 2012: Lisa Crispin
  73. 73. Keep tests passingStop the line to fix problems 73 Copyright 2012: Lisa Crispin
  74. 74. Test Management Tools§  Manage tests, code together§  Some tools have own management§  What problem are you trying to solve? 74 Copyright 2012: Lisa Crispin
  75. 75. Exercise: Tests as Documentation 75 Copyright 2012: Lisa Crispin
  76. 76. Key Success Factors Thin Slices Specification by Example Whole Team ApproachShortFeedbackLoops Experiments 76
  77. 77. Exercise: Breaking Barriers 77 Copyright 2012: Lisa Crispin
  78. 78. Remember§  It’s a team problem!§  Tackle automation problems with diversity§  Try small experiments§  Baby steps – start simple 78 Copyright 2012: Lisa Crispin
  79. 79. Questions? “Aha” Moments? 79 Copyright 2012: Lisa Crispin
  80. 80. Agile Testing: A Practical Guide for Testers and AgileTeamsBy Lisa Crispin and Janet Gregorywww.agiletester.ca Copyright 2012: Lisa Crispin 80
  81. 81. Experiences of Test AutomationDorothy Graham and Mark Fewster Copyright 2012: Lisa Crispin 81
  82. 82. Beautiful Testing: Leading Professionals Reveal HowThey Improve SoftwareEdited by Tim Riley, Adam GoucherIncludes chapter by yours truly Copyright 2012: Lisa Crispin 82
  83. 83. Test PatternsXunit Test Patterns: Refactoring Test CodeBy Gerard Meszaros Copyright 2012: Lisa Crispin 83
  84. 84. Specification by ExampleHow successful teams deliver the rightsoftwareGojko AdzicCase studies from > 50 teams 84 Copyright 2012: Lisa Crispin Copyright 2008 Janet Gregory, DragonFire
  85. 85. Agile Test Automation Resources§ dhemery.com/pdf/writing_maintainable_automated_acceptance_tests.pdf§ http://bit.ly/AgileTestTools§ lisacrispin.com§ janetgregory.ca§ gokjo.net§ exampler.com§ agile-testing@yahoogroups.com§ testobsessed.com§ testingreflections.com§ pairwith.us 85 Copyright 2012: Lisa Crispin
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×