Planning Your Agile Testing: A Practical Guide

1,084 views
877 views

Published on

Traditional test plans are incompatible with agile software development because we don't know all the details about all the requirements up front. However, in an agile software release, you still must decide what types of testing activities will be required—and when you need to schedule them. Janet Gregory explains how to use the Agile Testing Quadrants, a model identifying the different purposes of testing, to help your team understand your testing needs as you plan the next release. Janet introduces you to alternative, lightweight test planning tools that allow you to plan and communicate your big picture testing needs and risks. Learn how to decide who does what testing—and when. Determine what types of testing to consider when planning an agile release, the infrastructure and environments needed for testing, what goes into an agile “test plan,” how to plan for acquiring test data, and lightweight approaches for documenting your tests and recording test results.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,084
On SlideShare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
45
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Planning Your Agile Testing: A Practical Guide

  1. 1. 1/22/2013 Star Canada 2013 Janet Gregory, DragonFire Inc. Copyright 2013 With material from Lisa Crispin My experience comes … As a tester, working on agile teams Coaching and training, learning Agile Testing: A Practical Guide for Testers and Agile Teams; Addison Wesley 2009 Copyright 2013 DragonFire Inc. 1
  2. 2. 1/22/2013 How many years experience with agile? ◦ ◦ ◦ ◦ ◦ At least 2 years Less than 2 years Less than 1 year Have done some reading, starting in a team Completely new to the concept Are you a ... ◦ ◦ ◦ ◦ Tester Test Manager Programmer Other Copyright 2013 DragonFire Inc. 3 10 minutes ~1 minute / person Name Where you are from? Why you are here? Let everyone have a chance to introduce themselves Copyright 2013 DragonFire Inc. 4 2
  3. 3. 1/22/2013 Copyright 2013 DragonFire Inc. • • • • • • 5 have short iterations encourage active customer participation demand whole team collaboration test features & stories as they are coded deliver business value at regular intervals adapt their processes based on feedback Copyright 2013 DragonFire Inc. 6 3
  4. 4. 1/22/2013 Sequential, phased – eg. waterfall Requirements Specifications Code Testing Release Time D C C A It 0 B It 1 A B It 2 A Agile: iterative, incremental F E D C B It 3 A – Each story is expanded, coded and tested – Possible release after each iteration B It 4 7 What concerns you about agile and test planning? Copyright 2013 DragonFire Inc. 4
  5. 5. 1/22/2013 Test Approach – The Agile Way Project Initiation Release/Project Planning Each Iteration 1 …. X System Test / End Game Release to Prod/ Support Get an understanding of the project (business) Participate in sizing stories Create Test Plan Estimate tasks, Run regression tests Collaborate with customers on acceptance tests Write, automate and execute new story tests Pair test with other testers, developers Perform exploratory testing Perform Final Load Test Complete Final Regression Test Perform UAT Perform Mock Deploy Participate in Release Readiness Participate in Release to Prod Participate in Retrospectives Copyright 2013 DragonFire Inc. 9 Project Test Plan Release Test Plan Copyright 2013 DragonFire Inc. Story Tests 10 5
  6. 6. 1/22/2013 • Product Roadmap ◦ High level feature ideas • Release Planning ◦ Backlog of sized and prioritized stories • Iteration Planning ◦ Backlog of estimated tasks ◦ Estimations are in hours: how long will it take Copyright 2013 DragonFire Inc. 11 • Types of testing needed • Automation • Complexity • Size More details later … Copyright 2013 DragonFire Inc. 12 6
  7. 7. 1/22/2013 Instead of saying NO, or being the gatekeeper Be the information provider so business can make the decisions Questions… on agile approach to planning Copyright 2013 DragonFire Inc. 13 Copyright 2013 DragonFire Inc. 14 7
  8. 8. 1/22/2013 Let’s talk a little testing …. What is enough to keep you out of trouble? Copyright 2013 DragonFire Inc. 15 • Find bugs • Check correctness • Test reliability • Check usability • Answer “Is it done?” • Learn about the application • Feedback into future stories What else??? Copyright 2013 DragonFire Inc. 16 8
  9. 9. 1/22/2013 • Each group, collaborate • Write different types of testing that you do • One type per sticky • 5 minutes Copyright 2013 DragonFire Inc. 17 • Model for classifying tests • Looks at the purpose of the tests – the ‘why’ Copyright 2013 DragonFire Inc. 18 9
  10. 10. 1/22/2013 Agile Testing Quadrants Brain child of Brian Marick 19 • Take your tests and put them in the quadrant you think they belong in. • Were you able to get them all classified? • Add tests as you think about them. Copyright 2013 DragonFire Inc. 20 10
  11. 11. 1/22/2013 Agile Testing Quadrants (Brian Marick) 21 • Can be used as a communication tool o to explain testing in a common language • Emphasize whole-team responsibility o o focus on collaboration whole team participation • Help plan what tests to automate o and tools and infrastructure needed Copyright 2013 DragonFire Inc. 11
  12. 12. 1/22/2013 For each story / feature ◦ No story is done until tested ◦ Customer needs captured as passing tests ◦ Automated regression tests For release readiness ◦ Delivers value ◦ “Doneness” in all quadrants Copyright 2013 DragonFire Inc. 23 Copyright 2013 DragonFire Inc. 24 12
  13. 13. 1/22/2013 Unit Tests o o o o Tests developer intent - program design Tests a small piece of code Makes sure it does what it should TDD Component Tests o o Tests architecture intent – system design Tests that components work together correctly Connectivity Tests Copyright 2013 DragonFire Inc. • Builds quality in • Focus on internal code quality • Builds testability into code • Go faster, do more o Unit tests provide refactoring support • Provides instant feedback • TDD increases confidence in design • Forms the foundation of automation suite Copyright 2013 DragonFire Inc. 13
  14. 14. 1/22/2013 Can you think of anything else for Quadrant 1? 27 Copyright 2013 DragonFire Inc. 28 14
  15. 15. 1/22/2013 • Use to elicit requirements • Acceptance Test (or Example) Driven Development o Allows developers to code until the tests pass • Capture examples, express as executable tests • User experience o wire frames; mock-ups / prototypes Copyright 2013 DragonFire Inc. • Help customers achieve advance clarity • Gain shared common understanding of stories • Drive development with business-facing tests • Obtain enough requirements to start coding • Focus is external quality • Customer – developer – tester collaboration ◦ Power of Three Copyright 2013 DragonFire Inc. 15
  16. 16. 1/22/2013 Can you think of anything else for Quadrant 3? Discussions? 31 Copyright 2013 DragonFire Inc. 32 16
  17. 17. 1/22/2013 • Exploratory testing o what didn’t we think about • Test for usability o understand end users, personas • Tours • User Acceptance Testing (UAT) Copyright 2013 DragonFire Inc. • Iteration reviews / demos o Builds confidence o Quick feedback loop • Pair test with customers • Informal demos o Pair exploratory testing with customer Copyright 2013 DragonFire Inc. 17
  18. 18. 1/22/2013 • Provide feedback …. o Turn learnings into tests that drive new features o Change process as needed • Evaluation of the product • Recreate actual user experiences • Realistic use Copyright 2013 DragonFire Inc. • Know your customers • Make them real • Plan your exploratory testing using them • Picture – from Jeff Patton’s Pragmatic Personas weekly column on Sticky Minds (1/25/2010) Copyright 2013 DragonFire Inc. 18
  19. 19. 1/22/2013 • Think of some personas or characters and devise exploratory scenarios which that role might get into, for example: Shopping website - Amazon o Senior citizen who never shopped on the web o Hacker looking to cause trouble o Working mom in a rush o Internet-savvy teen 37 Copyright 2013 DragonFire Inc. 37 Any questions about Quadrant 3? 38 19
  20. 20. 1/22/2013 39 Copyright 2013 DragonFire Inc. • Non-functional tests • Performance, scalability, stress, load • “ility” testing • Automated deployments • Memory management • Infrastructure testing • Security testing o Roles & permissions , system ‘hacking’ • Data migration • Recovery Copyright 2013 DragonFire Inc. 20
  21. 21. 1/22/2013 • May be higher priority than functional req’ts • Makes the ‘finished’ product • Transfer specialized expertise Consider the four quadrants • Would you add anything? • Consider your quality attributes • Do you need to move any tests? • What tests are you missing? • What is your team is not doing now? Copyright 2013 DragonFire Inc. 42 21
  22. 22. 1/22/2013 Automate at the feature level push the tests lower Automate at the story level Automate at the task level Sequential, phased – eg. waterfall Requirements All automation done here Specifications Code Testing Release Time D C C A It 0 B It 1 A B It 2 A Agile: iterative, incremental F E D C B It 3 A – Each story is expanded, coded and tested – Possible release after each iteration B Automation here It 4 Automation Automation here Automation here here 44 22
  23. 23. 1/22/2013 Instead of saying NO, or being the gatekeeper Be the information provider so business can make the decisions Questions… on types of testing? Copyright 2013 DragonFire Inc. 45 Copyright 2013 DragonFire Inc. 46 23
  24. 24. 1/22/2013 Release Level Test Planning Copyright 2013 DragonFire Inc. 47 • Consider scope, priorities, risks • Document only what is useful • Consider all types of testing • Budget time for: o o o infrastructure tools automation needs • Strive for simplicity Copyright 2013 DragonFire Inc. 48 24
  25. 25. 1/22/2013 49 • Sizing of stories ◦ think about impacts to system • Ask questions ◦ that may impact the ‘bigness’ of the story ◦ to uncover hidden assumptions ◦ such as ‘what if?’, or “what happens if?” • Not the time for details • Create a project “test plan” Copyright 2013 DragonFire Inc. 50 25
  26. 26. 1/22/2013 • Are we working with a vendor? ◦ If so, how do we coordinate testing? • Are there dependencies between … ◦ features? ◦ teams? ◦ Stories? Copyright 2013 DragonFire Inc. 51 26
  27. 27. 1/22/2013 • Should be project specific • High-light critical information ◦ risks ◦ assumptions ◦ constraints • Focus on what is really needed • Put static information –in a Quality Management Strategy document Copyright 2013 DragonFire Inc. 53 Copyright 2013 DragonFire Inc. 54 27
  28. 28. 1/22/2013 Copyright 2013 DragonFire Inc. 55 A test matrix (one example) ◦ provides a view of the release ◦ can also be used as visible progress But, remember ◦ the value is in the planning Let`s look at an example Copyright 2013 DragonFire Inc. 56 28
  29. 29. 1/22/2013 Copyright 2009 Janet Copyright 2013 DragonFire Inc. Gregory, DragonFire 57 Let’s do some collaboration, & work together to find a solution Copyright 2013 DragonFire Inc. 58 29
  30. 30. 1/22/2013 rules Sub topic rules user name save password Sub topic first time Sub topic encryption MAIN new account TOPIC Sub topic change Sub topic Copyright 2013 DragonFire Inc. 59 Mind map testing ideas for a feature How many ideas can you generate in 5 minutes? Copyright 2013 DragonFire Inc. 30
  31. 31. 1/22/2013 Feature A • As an on-line shopper, I want to provide my shipping address when I check out so that my order goes to the right location Feature B • As an on-line shopper, I want to provide payment information when I check out so I am billed correctly. Copyright 2013 DragonFire Inc. • Create a test matrix for the stories on the next slide. • Functionality down the left side • Test conditions across the top o Think heuristics o Use mnemonics (ex. SFDPOT) • Gray out the squares that are not applicable. Copyright 2013 DragonFire Inc. 31
  32. 32. 1/22/2013 Feature A • As an on-line shopper, I want to provide my shipping address when I check out so that my order goes to the right location Feature B • As an on-line shopper, I want to provide payment information when I check out so I am billed correctly. Copyright 2013 DragonFire Inc. • What value do you see in something like this? • What did you learn? • How would you use it? Copyright 2013 DragonFire Inc. 32
  33. 33. 1/22/2013 Instead of saying NO, or being the gatekeeper Be the information provider so business can make the decisions Questions on test planning at the release level? Copyright 2013 DragonFire Inc. 65 Copyright 2013 DragonFire Inc. 66 33
  34. 34. 1/22/2013 Iteration Planning: • Stories tell us about the tip of the iceberg • What don’t we know? • What questions should we ask to find out? Copyright 2013 DragonFire Inc. 68 34
  35. 35. 1/22/2013 Create a user story Write Customer (Q2) Tests Expand tests – Story Tests Automate Q2 Tests Start thinking how to code Pair, “Show Me” TDD Product owner Product owner/ Tester Tester Tester/Programmer Programmer Exploratory testing Customer User Acceptance Copyright 2013 DragonFire Inc. 69 • Be proactive – preplanning • Try specifications workshops – Gojko Adzic • Help customer achieve ‘advance clarity’ on stories ◦ ◦ ◦ ◦ Customers speak with “one voice” Testable stories Steel threads Create acceptance tests Copyright 2013 DragonFire Inc. 70 35
  36. 36. 1/22/2013 • Define high level story tests or examples • Find hidden assumptions • Define and estimate testing tasks – consider: o o o o automation needs test data, exploratory testing Q4 tests (‘ilities’, security, performance, etc.) Copyright 2013 DragonFire Inc. 71 Ask questions • What's the business goal? • Can the user mess up? • What’s the best thing? • What’s the worst thing that can happen? • Watch for scope creep or ‘bling’ • Is this story testable? Copyright 2013 DragonFire Inc. 72 36
  37. 37. 1/22/2013 To Review Copyright 2013 DragonFire Inc. Picture from Mike Cohn’s website 73 73 • • • • Express the intent of the story Use examples as specific instances of a scenario Think expected and unexpected behaviors Gives a shared common understanding of the story • Feeds into TDD (Test Driven Development) Copyright 2013 DragonFire Inc. 74 37
  38. 38. 1/22/2013 As a new user, I want to create an account with a user name and password so that only I can access my information. =============================== Copyright 2013 DragonFire Inc. 75 75 • To create my account, I enter a valid user name and password, my information is saved and I am logged into the system and on the home page. • If I enter an invalid user name, I get an error message “Invalid User Name”, and I am able to try again. • If I enter an invalid password, I get an error message “Invalid Password”, and I am able to try again. Copyright 2013 DragonFire Inc. 76 38
  39. 39. 1/22/2013 BDD – Behavioural Driven Development Given the user has no existing account When she requests to create a new account, Then she enters a valid user name and valid password (rules defined) And the information is saved upon submitting . 77 Copyright 2013 DragonFire Inc. 77 returnValue TestLogin ( userName, password) User Name Password Expected result comments JanetGregory Password Access system as JanetGregory Valid combo saved Janet Gregory Password Error Space in user name JanetGregory Abc Error Invalid password Copyright 2013 DragonFire Inc. 78 39
  40. 40. 1/22/2013 • Write Acceptance Test(s) for one of the two stories – any format you like • To help, try mind-mapping or draw a flow diagram • Ask your customer for examples. • Think “Amazon” • (~15 min) Copyright 2013 DragonFire Inc. Story a – Part of Feature A • As the company shipper, I need to verify the city, state and postal code so that the order goes to the right location. Story b – Part of Feature B • As the company accountant, I need to ensure the credit card information is correct so the on-line shopper is billed correctly. Copyright 2013 DragonFire Inc. 40
  41. 41. 1/22/2013 We walk out of the iteration planning meeting .... Now what? Copyright 2013 DragonFire Inc. 82 41
  42. 42. 1/22/2013 • Start simple with the high level acceptance tests ◦ Add boundary, edge conditions, etc. ◦ Add non-functional tests • Experiment to find the right detail level • Review with programmers ◦ Then automate • Keep adding until story is complete • Consider exploratory test scenarios Copyright 2013 DragonFire Inc. 83 Acceptance Tests – Fit Automation Style -- Third thread User Name Password Expected result Comments JanetGregory Password Access system as JanetGregory Valid combo saved Janet Gregory Password Error Space in user name Janet#Gregory Password Error Special char not allowed Password Error Blank user name JanetGregory Password Error User already exists JanetGregory Abc Error Not enough char in password Error Blank password JanetGregory Copyright 2013 DragonFire Inc. 84 42
  43. 43. 1/22/2013 How do you know you are done? Copyright 2013 DragonFire Inc. Description 85 Completed 1 Story tests reviewed, automated and pass Yes 2 Exploratory testing complete Yes 3 Unit tests reviewed and pass Yes 4 Q4 tests complete Yes 5 Acceptance tests pass Yes Copyright 2013 DragonFire Inc. 86 43
  44. 44. 1/22/2013 Instead of saying NO, or being the gatekeeper Be the information provider so business can make the decisions Questions on test planning at the story level? Copyright 2013 DragonFire Inc. 87 Copyright 2013 DragonFire Inc. 88 44
  45. 45. 1/22/2013 • Understand your context • Understand the purpose • Consider ROI (return on investment) • Push the tests lower • Automate the repetitive and boring tests • Plan, but document simply • Plan, but plan for the appropriate level Copyright 2013 DragonFire Inc. • How much is too much? • What is not enough? • Why do you need them? • What is the right information? • Who looks at them? • What do they use them for? • What is the simplest way you can capture them? Copyright 2013 DragonFire Inc. 90 45
  46. 46. 1/22/2013 • • • • • • • • What problem are you trying to solve? What measurements make sense? What is the simplest way to gather them? Who needs to see them? What is the simplest way to make them visible? How are you going to use them? When will you know to stop? Be Aware!!! o o The wrong measurements can be deadly Metrics can be misinterpreted Copyright 2013 DragonFire Inc. • How are you going to know when you are done? • What is enough? • Collaborate to decide what coverage you want • Think risk – include the customer • Functional coverage • Lines of code? … be careful • What tools can you use? o o Heuristics Mnemonics Copyright 2013 DragonFire Inc. 46
  47. 47. 1/22/2013 • Understand the level of precision you need ◦ Think big picture in release planning ◦ Think tasks during iteration planning ◦ Think tests during story test planning • Use ATDD for feature and story testing • Make it visible • Make it simple • Make it valuable • You want to be testing, not documenting Copyright 2013 DragonFire Inc. Instead of saying NO, or being the gatekeeper Be the information provider so business can make the decisions Are there still unanswered questions.. Concerns? Copyright 2013 DragonFire Inc. 94 47
  48. 48. 1/22/2013 Now Available Agile Testing: A Practical Guide for Testers and Agile Teams By Lisa Crispin and Janet Gregory www.agiletester.ca My contact info www.janetgregory.ca http://janetgregory.blogspot.com/ Email: janet@agiletester.ca www.lisacrispin.com http://lisacrispin.com Email: lisa@agiletester.ca Copyright 2013 DragonFire Inc. 95 • • • • • • • • www.lisacrispin.com agile-testing@yahoogroups.com http://www.exampler.com - Brian Marick’s web site www.testobsessed.com (heuristics cheat sheet) – Elisabeth Hendrickson http://www.shino.de/blog/ - Markus Gärtner’s blog Gojko Adzic, Bridging the Communication Gap, 2009 Gojko Adzic, Executable Specifications, Manning, 2011 vwww.mountaingoatsoftware.com – Mike Cohn’s web site (and all his books) • http://www.developsense.com/2009/04/of-testing-tours-anddashboards.html • James Whittaker, Exploratory Software Testing, 2009 Addison-Wesley • http://www.stickyminds.com/ Pragmatic Personas - Jeff Patton’s weekly column 1/25/2010 • Jean Tabaka, Collaboration Explained, 2006 Addison-Wesley • Copyright 2013Manifesto: http://agilemanifesto.org/ Agile DragonFire Inc. 96 48

×