Trends in Agile Testing by Lisa Crispin


Published on

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • When I talk about Agile, I will usually refer to XP or SCRUM because those are the 2 methodologies I am the most familiar with. When I talk about traditional methods, I mean phased or gated methodologies such as waterfall. Phased Approach - waterfall Coding occurs after requirements and specifications are complete Testing occurs after coding is complete Application not delivered until all phases are complete Some problems with waterfall: missing and out of date requirements gaps between what the customer is expecting and what is delivered Agile = Testing occurs on a feature by feature basis Each feature (or story) is delivered incrementally Business value is delivered on a regular basis
  • Trends in Agile Testing by Lisa Crispin

    1. 1. Trends in Agile Testing Code Chef Talks 2009 Lisa Crispin With Material from Janet Gregory
    2. 2. Introduction <ul><li>Tester on agile teams since 2000 </li></ul><ul><li>My teams: </li></ul><ul><ul><li>Delight customers </li></ul></ul><ul><ul><li>Deliver production-ready value every 2 weeks </li></ul></ul><ul><ul><li>Drive development with tests, examples </li></ul></ul><ul><ul><li>100% regression test automation </li></ul></ul>
    3. 3. Your Agile Experience <ul><li>How about you? </li></ul><ul><li>How many on agile teams? </li></ul><ul><li>How many in transition? </li></ul><ul><li>How many planning a transition? </li></ul><ul><li>How many on more traditional teams? </li></ul><ul><ul><li>Agile testing works on those, too. </li></ul></ul><ul><li>Managers, testers, developers...? </li></ul>
    4. 4. Takeaways <ul><li>Transitions in testing </li></ul><ul><li>How testing works on agile teams </li></ul><ul><li>Agile testing tools and techniques </li></ul><ul><li>What it means for us </li></ul>
    5. 5. Traditional vs. Agile SDLC Phased or gated – ex. Waterfall It 1 It 2 It 3 A B A B A B C C D Time Agile : iterative and incremental - Each story is expanded, coded and tested - Possible release after each iteration Requirements Specifications Code Testing Release Copyright 2009 Lisa Crispin and Janet Gregory
    6. 6. Testing as a Phase <ul><li>Code, then test </li></ul><ul><li>Separate, siloed teams </li></ul><ul><li>Heavyweight process </li></ul><ul><ul><li>Requirements, design documents </li></ul></ul><ul><ul><li>Test plans </li></ul></ul><ul><ul><li>“ Traceability” </li></ul></ul><ul><ul><li>Change review boards </li></ul></ul><ul><li>Testing “bottleneck” at end </li></ul>
    7. 7. Automation - After Coding <ul><li>Vendor tools </li></ul><ul><li>Record/Playback </li></ul><ul><li>Done by testers or automation team </li></ul><ul><li>If developers automate unit tests – it’s after coding </li></ul><ul><li>Expensive </li></ul>
    8. 8. Exploratory Testing – “agile” <ul><li>Simultaneous learning, test design, test execution [source: James Bach] </li></ul><ul><li>Aka “Context-Driven” </li></ul><ul><li>Critical thinking, no requirements doc needed </li></ul><ul><li>Diverse ideas </li></ul><ul><li>Rich resources </li></ul><ul><ul><li>Tools, data, people </li></ul></ul><ul><ul><li>[source: Jon Hagar] </li></ul></ul>
    9. 9. How Agile Changes Things <ul><li>Whole Team Approach </li></ul><ul><li>Coding and Testing are One Process </li></ul><ul><li>Feedback, collaboration key </li></ul><ul><li>TDD/ATDD practices </li></ul><ul><li>Test-infected developers </li></ul>
    10. 10. Test Approach - The Agile Way Copyright 2007: Lisa Crispin and Janet Gregory Project Initiation Get an understanding of the project Participate in estimating stories Create Test Plan Release Planning Each Iteration 1 … . X Write and execute story tests Write and execute new functional test cases Pair test with other testers, developers Automate new functional test cases Run automated regression test cases Perform Load Test Complete Regression Test Perform UAT Perform Mock Deploy Participate in Release Readiness System Test / End Game Participate in Release to Prod Participate in Retrospectives Release to Prod/ Support
    11. 11. Team Organization - Traditional vs. Agile <ul><li>Agile : works as one team </li></ul>Copyright 2007: Lisa Crispin and Janet Gregory Traditional : independent functional groups Dev BAs Tester Tester Programmer Domain Expert
    12. 12. The Whole Team Approach <ul><li>Team commitment to quality </li></ul><ul><li>Team responsible for testing </li></ul><ul><li>Testers get support, training </li></ul><ul><li>Anyone can do any task </li></ul><ul><li>Testers transfer testing knowledge </li></ul><ul><li>Get the right people </li></ul><ul><li>Collaboration </li></ul>
    13. 13. Testing is Not a Phase <ul><li>Development = testing + coding </li></ul><ul><ul><li>Write test, write code, test </li></ul></ul><ul><li>Unit and Acceptance Test levels </li></ul><ul><li>Automated tests </li></ul><ul><ul><li>Free time for exploratory testing </li></ul></ul>
    14. 14. Agile Testing Quadrants
    15. 15. Test Automation Pyramid
    16. 16. Feedback Guides Development <ul><li>Team uses feedback to improve </li></ul><ul><li>Testers are expert feedback providers </li></ul><ul><li>Feedback lets team make course corrections </li></ul>
    17. 17. Feedback Examples <ul><li>Big Visible Charts </li></ul><ul><li>Retrospectives </li></ul><ul><li>CI process sends results in emails </li></ul><ul><li>Short iterations facilitate improvement </li></ul><ul><li>Discoveries during exploratory testing </li></ul><ul><li>Collaboration with users, customers </li></ul>
    18. 18. Core Practices for Success <ul><li>Continuous integration </li></ul><ul><ul><li>Frequent and fast feedback </li></ul></ul><ul><li>Coding and testing one process </li></ul><ul><li>Test environments </li></ul><ul><li>Manage technical debt </li></ul><ul><li>Work incrementally </li></ul><ul><li>Synergy </li></ul>
    19. 19. Where Is It Taking Us? <ul><li>Lines between roles blurred </li></ul><ul><li>More focus on tests to drive coding </li></ul><ul><li>Examples as requirements </li></ul><ul><li>Find ways to control technical debt </li></ul><ul><li>Tests pushed to lower levels </li></ul><ul><ul><li>Less end-to-end automation </li></ul></ul>
    20. 20. Current Trends in Testing/Development <ul><li>Pomodoro technique for teams </li></ul><ul><ul><li> </li></ul></ul><ul><li>More exploratory testing </li></ul><ul><ul><li>As automated regression tests free time </li></ul></ul><ul><ul><li> </li></ul></ul><ul><li>Lean approaches </li></ul><ul><ul><li>Kanban, scrumban </li></ul></ul><ul><ul><li>Poppendiecks’ books </li></ul></ul>
    21. 21. More Current Trends <ul><li>Continuous Integration is a given </li></ul><ul><li>Mix and match frameworks/drivers </li></ul><ul><li>More low-tech early prototyping </li></ul><ul><li>More early collaboration </li></ul><ul><li>Behavior-Driven Development (BDD) </li></ul>
    22. 22. BDD/Narrative/Scenario Example Given that the FitNesse wiki is a Wiki Given that the FitNesse User is a Web User Given that the FitNesse wiki was able to start for the first time When the FitNesse User attempts to go to the Home Page Then the content of the Home Page for the FitNesse User should be Welcome to the Wonderful World of FitNesse! From: Antony Marcano, Andy Palmer,,
    23. 23. Test Tools <ul><li>Open source, agile frameworks/drivers </li></ul><ul><li>Vendor tools getting lighter weight </li></ul><ul><li>FitNesse/Slim/Fit ubiquitous </li></ul><ul><ul><li>Framework for many other tools </li></ul></ul><ul><li>BDD tools </li></ul><ul><ul><li>Cucumber, easyb, nbehave </li></ul></ul><ul><li>GUI test drivers </li></ul><ul><ul><li>Watir, Selenium </li></ul></ul>
    24. 24. Testing Tool Spectrum Agile2009 - © ASERT 2006-2009 Database Drivers DbUnit DataSets SqlUnit groovy.sql JPA JDO BigTable JDBC SOAP / REST Drivers GroovyWS XML-RPC CXF Axis2 JAX-WS JAX-RS Utilities All Pairs Combinations Polyglot languages Logic programming Threads, Parallel concurrency libraries Data-driven libraries Networking libraries XML Processing Read/write files, Excel / Word / CSV, Reporting, Logging Other Drivers FEST Email FTP AntUnit Telnet SSH Exec Tools iTest2, SoapUI, Twist, IDEs, JMeter, Text editors, Recorders, Build Tools, CI Web Drivers WebTest WebDriver JWebUnit Tellurium Selenium HtmlUnit Watij HttpBuilder Cyberneko Runners Native Groovy, JUnit, TestNG, Spock, EasyB, JBehave, Cucumber, Robot Framework, Slim
    25. 25. Testing Techniques <ul><li>Team writes acceptance tests together </li></ul><ul><li>Programmers and testers pair to automate tests </li></ul><ul><li>ROI guides automation </li></ul><ul><li>Testers do exploratory testing </li></ul><ul><li>Customers participate </li></ul><ul><ul><li>Pair with customers for ET </li></ul></ul><ul><ul><li>Frequent demos </li></ul></ul>
    26. 26. Trends in Test Automation <ul><li>Record/Playback as “trainer”, debugger </li></ul><ul><li>Test architecture/design critical </li></ul><ul><li>Planning and tracking tools – online and low-tech </li></ul><ul><li>Open source, agile tools explosion </li></ul><ul><ul><li>Plenty of good browser test drivers </li></ul></ul><ul><ul><li>Other problems now being solved </li></ul></ul><ul><li>Scenario, Narrative style </li></ul>
    27. 27. What Does It Mean for Us? <ul><li>More integration with development teams </li></ul><ul><li>More collaboration with customers </li></ul><ul><li>Helping programmers with testing </li></ul><ul><li>We probably need to be more technical </li></ul><ul><li>More creative thinking, problem-solving </li></ul>
    28. 28. Do Your Best Work <ul><li>Whole Team approach </li></ul><ul><ul><li>Collaborate with customers, developers </li></ul></ul><ul><li>Business examples, turned into tests </li></ul><ul><li>Agile Testing Quadrants </li></ul><ul><li>Test Automation Pyramid </li></ul><ul><li>Exploratory testing </li></ul>
    29. 29. The Future is What You Make It <ul><li>Get out of your chair – go talk to programmers and customers </li></ul><ul><li>Learn a new test framework or scripting language today - build your skills </li></ul><ul><ul><li>Lots of free resources, just takes time </li></ul></ul><ul><li>Bring testing problems to the whole team </li></ul><ul><ul><li>Benefit from multiple skill sets </li></ul></ul><ul><li>How can your expertise help the team? </li></ul>
    30. 30. Questions?
    31. 31. Some Agile Testing Resources <ul><li> </li></ul><ul><li> </li></ul><ul><li> </li></ul><ul><li>[email_address] </li></ul><ul><li> </li></ul><ul><li>A </li></ul>
    32. 32. Copyright 2008 Janet Gregory, DragonFire Bridging the Communication Gap Specification By Example and Acceptance Testing Gojko Adzic
    33. 33. Now Available Agile Testing: A Practical Guide for Testers and Agile Teams By Lisa Crispin and Janet Gregory