Agile Testing

1,927 views
1,816 views

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,927
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
79
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Agile Testing

  1. 1. Agile Testing Elisabeth Hendrickson Quality Tree Software, Inc. www.qualitytree.com Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 1
  2. 2. Poll: How Agile Is Your Testing Now? How many of you… Would call your test practices “Agile”? Participate in defining the acceptance criteria? Routinely test incomplete software to provide feedback to your stakeholders sooner? Have adopted a lightweight documentation style? Have found that your test artifacts are relatively easy to update even when there are radical changes to the code? Have access to the source code and to the same set of tools as the programmers? Have ensured the programmers have full access to all the artifacts and tools you use? Are co-located with the project team? Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 2
  3. 3. What’s “Agile?” Agile is more than a buzzword. It is a relentless focus on providing business value, usually by adopting one or more Agile methodologies such as Scrum or XP. – See the Agile Manifesto: http://www.agilemanifesto.org – And the Agile Alliance: http://www.agilealliance.org Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 3
  4. 4. Examples of Agile Methodologies Lean Scrum Lean manufacturing Lightweight concepts applied to management software framework. development. Crystal Extreme Lightweight set of Programming (XP) development Rigorous set of practices. practices designed to keep both the code and team agile. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 4
  5. 5. Agile Synthesized • Communication and • Feedback collaboration • Whole team thinking • Visible indicators • Short iterations • Disciplined development • Low overhead, high practices productivity Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 5
  6. 6. Calling It “Agile” Doesn’t Make It So This is NOT Agile: 1. Compress the schedule 2. Toss out the documentation 3. Code up to the last minute The organization may gain short term speed but at the cost of long term pain. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 6
  7. 7. How Traditional Test Practices Evolved With great optimism and the best of intentions, The Project Plan is announced: Analyze Design Code Test/Bug Fix Requirements Completed Code Release handed off to Dev handed off to Test Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 7
  8. 8. How Traditional Test Practices Evolved Inevitably, The Project Plan is revised: Analyze, Design, & Code Test/Bug Fix Completed Code Release handed off to Test Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 8
  9. 9. The Result: Practices Intended to Control Chaos Traditional test practices attempt to manage the chaos (or at least avoid the blame): • “Last Defender of Quality” stance • Strict change management • Detailed preparation and up front planning • Heavyweight documentation suitable for outsourcing the test effort • Strict entrance and exit criteria with signoffs • Heavyweight test automation focused on regression Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 9
  10. 10. Becoming Agile: Shifting Roles “Fear not! I’ll protect you!” “Hey, would this help? from last line of defense… …to team support Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 10
  11. 11. Testing Directions: Marick’s Model Business Facing Support Programming Critique Product Technology Facing Source: http://www.testing.com/cgi-bin/blog/2003/08/21#agile-testing-project-1 Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 11
  12. 12. Traditional & Agile Testing Contrasted Traditional Testing Agile Testing Change Manage & control it. Accept it. Planning Comprehensive up front test Plan as you go. design. Documentation Verbose. Only as much as absolutely necessary. Handoffs Formal entrance and exit criteria It’s not a relay race. with signoffs. Collaborate. Automation System-level, built by tool All levels, built by anyone, specialists, created after the code an integral part of the project. is “done.” Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 12
  13. 13. Change Embrace Change: Plan Tests for Maintainability When creating test artifacts… • Minimize duplication. Planning Thought exercise: if a feature were removed from your software, how many Documentation test artifacts would have to be updated? • Use tools designed for change. Hint: if the vendor says “stabilize the Handoffs interface first,” run away! Automation Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 13
  14. 14. Change How Not to Plan a Test Effort Unfortunately, this is common practice: • Download a generic, standard, formal test plan Planning template from the Internet. • Fill in every section in the template even if you Documentation don’t understand it and have to make stuff up. • Kill the maximum possible number of trees by distributing copies to every team member. Handoffs The result is a large and mostly irrelevant plan that must now be maintained. Yuck. Automation Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 14
  15. 15. Plan as Little as Possible A little planning is good. More is not better. Change • Know who you serve (primarily). Planning Technology-facing or business-facing? • Plan for the current iteration. Documentation Speculative planning means rework. • Have a strategy that fits on one page. If it is still relevant in 6 months, it’s probably at the right level of detail. Handoffs • Keep an up-to-date, prioritized risks list. What kind of information are the testers looking Automation for? The risks list covers it. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 15
  16. 16. Change Favor Informal, Collaborative Tools Informal Whiteboards Planning Sticky Notes Index Cards Documentation Wikis Formal Databases Gantt/PERT Charts Handoffs Polished Documents Automation Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 16
  17. 17. Change Monitor Documentation Costs How Much Time Are We Spending on Test Documentation? 40 Planning # Testers Reporting 35 30 25 20 Documentation 15 10 5 0 0 10 20 30 40 50 60 70 80 90 100 % Time Handoffs How Much Does Documentation Cost? Informal polling of 162 software testers from 65 companies revealed that most spend more Automation than 33% of their time documenting tests. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 17
  18. 18. Change Keep the Documentation Simple • Capture the essence, not the details. Step-by-step instructions cost time without Planning providing value (usually). • Point to other project documents. Documentation If it’s in the user guide, requirements, specs, etc., leave it there. • Centralize generic tests in a checklist. Handoffs Try this: count the number of times you find a common condition, like invalid dates or null strings, in the test docs. More than Automation once is too many. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 18
  19. 19. Watch Verbose Wording Which would you rather maintain? Change Choose the Import option from the File menu. Planning A dialog titled “Import File” appears. Navigate to xyzlong.dat and click Open. A dialog Documentation titled “Importing…” appears with the current status, a progress bar, and a button labeled “Cancel.” Click on the Cancel button. Choose OK on the confirmation dialog. Verify that the Handoffs import stops. Or: Automation Start a long import. Cancel it in the middle. Verify it stops. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 19
  20. 20. Change A Lightweight Example Planning Documentation Handoffs Automation Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 20
  21. 21. Change Do Away with the Signoff Sheet Hint: when people joke about signing the handoff form in Planning blood, the signoff process is more about blame than about Documentation producing good software. Handoffs Automation Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 21
  22. 22. Integrate Testing Testing is not a phase. It’s a way of life. Change • Agile teams are test infected. Planning The question, “How will we test it?” is as important as “How will we build it?” Documentation • Co-locate testers and programmers. But sitting side by side does not ensure communication. • Track testing status and programming Handoffs status all together. Show tests run-passed-failed together with Automation features/stories done and left to do. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 22
  23. 23. Leverage Automation Investments Automate everything you can, but invest wisely. Change • Collaborate with programmers on test Planning infrastructure code. The programmers have already automated the Documentation unit tests. Why not reuse the investment where possible? • Use different types of automated tests for different purposes. Handoffs Automated system tests should cover end-to-end sequences. Unit tests detect unintended change. Automation Don’t substitute one for the other. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 23
  24. 24. Change Automation & Exploratory Testing Use test automation to support early exploratory testing. Planning Traditional test wisdom says we can’t start testing a feature until it’s accessible from Documentation an external interface (like a GUI). But we don’t have to wait. Test automation can facilitate manual exploration. Handoffs Automation Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 24
  25. 25. Agile Testing Synthesized • Communication and • Early involvement collaboration • Leveraged automation • Whole team thinking • Focus on providing rapid • Low overhead, high feedback to key stakeholders productivity Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 25
  26. 26. Necessary Conditions for Agile Testing 1. The organization is willing to embrace agility as defined by the Agile Manifesto. Saying “Be More Agile” or “Test Faster” isn’t enough. 2. The whole team is responsible for quality, not just the testers or people with “Quality” in their title. Which are you more likely to hear: “How did you miss that bug?!?” or “How did we not catch that?” 3. Everyone tests, not just designated testers. Agile teams are “test infected.” 4. Managers focus on fixing problems, not blame. Agile practices don’t provide CYA paper trails and are unlikely to succeed in a high-blame, high-fear environment. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 26
  27. 27. Agile Testers Wanted: A Job Description Responsibilities: • Perform manual/exploratory tests on early-stage code • Create automated acceptance tests • Advise the team about overall risks and trends • Assist the business stakeholders define acceptance criteria • Facilitate communication between the technical & business stakeholders Qualifications: • Experience designing and executing tests • Scripting skills in at least one of the following languages: Ruby, Perl, Python, or JavaScript. Experience programming in Java, C#, etc. a plus. • Strong analysis skills • Ability to work in a team (bullpen) environment Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 27
  28. 28. Test Teams that Embrace Agility… • Focus on providing value to our key stakeholders (both business-facing and technology-facing) • Shift from being the last line of defense to providing an information service. • Aggressively reduce time and resources spent on anything that does not directly contribute to providing information. • Collaborate with programmers to improve testability and leverage test automation efforts. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 28
  29. 29. Further Reading… Beck, K. (1999). Extreme Programming Explained: Embrace Change. Addison-Wesley. Cockburn, A. (2004). Crystal Clear: A Human- Powered Methodology for Small Teams. Addison- Wesley. Crispin, L., & House, T. (2002). Testing Extreme Programming. Addison-Wesley. Poppendieck, M. & Poppendieck, T. (2003). Lean Software Development. Addison-Wesley. Schwaber, K. & Beedle, M. (2001). Agile Software Development with SCRUM. Prentice Hall. Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 29
  30. 30. Acknowledgements Many thanks to early reviewers of the ideas presented here: Brian Marick, William Wake, Jonathan Kohl, Jeffrey Fredrick, Daniel Knierim, Marc Kellogg, Danny Faught, Ron Jeffries, Hubert Smits, Rob Mee, Sherry Erskine, Amy Jo Esser, Gunjan Doshi, Dave Liebreich, Janet Gregory, Chris McMahon Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 30

×