Rapid Software Testing: Strategy


Published on

A test strategy is the set of ideas that guides your test design. It's what explains why you test this instead of that, and why you test this way instead of that way. Strategic thinking matters because testers must make quick decisions about what needs testing right now and what can be left alone. You must be able to work through major threads without being overwhelmed by tiny details. James Bach describes how test strategy is organized around risk but is not defined before testing begins. Rather, it evolves alongside testing as we learn more about the product. We start with a vague idea of our strategy, organize it quickly, and document as needed in a concise way. In the end, the strategy can be as formal and detailed as you want it to be. In the beginning, though, we start small. If you want to focus on testing and not paperwork, this approach is for you.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Rapid Software Testing: Strategy

  1. 1. MH Half-day Tutorials 5/5/2014 8:30:00 AM Rapid Software Testing: Strategy Presented by: James Bach Satisfice, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. James Bach Satisfice, Inc. James Bach is founder and principal consultant of Satisfice, Inc., a software testing and quality assurance company. In the eighties, James cut his teeth as a programmer, tester, and SQA manager in Silicon Valley in the world of market-driven software development. For nearly ten years, he has traveled the world teaching rapid software testing skills and serving as an expert witness on court cases involving software testing. James is the author of Lessons Learned in Software Testing and Secrets of a Buccaneer-Scholar: How Self- Education and the Pursuit of Passion Can Lead to a Lifetime of Success.
  3. 3. Rapid Software Testing: Strategy James Bach, Satisfice, Inc. james@satisfice.com www.satisfice.com
  4. 4. What is Rapid Testing? Rapid testing is a mind-set and a skill-set of testing focused on how to do testing more quickly, less expensively, with excellent results. This is a general testing methodology. It adapts to any kind of project or product.
  5. 5. The Premises of Rapid Testing 1. Software projects and products are relationships between people, who are creatures both of emotion and rational thought. 2. Each project occurs under conditions of uncertainty and time pressure. 3. Despite our best hopes and intentions, some degree of inexperience, carelessness, and incompetence is normal. 4. A test is an activity; it is performance, not artifacts. 5. Testing’s purpose is to discover the status of the product and any threats to its value, so that our clients can make informed decisions about it. 6. We commit to performing credible, cost-effective testing, and we will inform our clients of anything that threatens that commitment. 7. We will not knowingly or negligently mislead our clients and colleagues. 8. Testers accept responsibility for the quality of their work, although they cannot control the quality of the product.
  6. 6. What is a test strategy? Test strategy is the set of ideas that guide your choice of tests. A set of ideas does not necessarily mean a document. The test strategy may be entirely in your head. Or it may be in several heads, and emerge through discussion, over time. It may be documented partially on a whiteboard or Post-Its or in a mindmap. Or it could be in a formal document all dressed like Cinderella at the royal ball.
  7. 7. What is a test strategy? Test strategy is the set of ideas that guide your choice of tests. To guide is to influence but not necessarily to determine. Testing is shaped by many factors in addition to strategy, including opportunities, skills, mistakes, time pressures, limitations of tools, testability, and unconscious biases.
  8. 8. What is a test strategy? Test strategy is the set of ideas that guide your choice of tests. I mean choice in the most expansive sense of the word, not simply the selection of existing test cases. Choice of tests includes choices of what tests to design and how to design them and all decisions made during test design. It includes choices made during test execution, too, including how to perform tests and what mix of tests to perform in response to which perceived risks.
  9. 9. Why have a test strategy? If you test, then you already have a test strategy, so that’s not a meaningful question… Here are some better questions: Why have an explicit test strategy? Why worry about your test strategy? Why explain it? Why document it? Why not let the tests “speak for themselves?” 1. To get more credibility, control, agility, and accountability, for less time and effort. 2. Tests don’t talk.
  10. 10. (Test Strategy is a Middle Level Tool)
  11. 11. Strategic Thinking Begins with the Context Test Team Expertise Loading Cohesion Motivation Leadership Project Integration Test Lab Test Platforms Test Tools Test Library Problem Tracking System Office Facilities Product Mission Stakeholders Quality Criteria Reference Material Requirements Product Project Lifecycle Project Management Configuration Management Defect Prevention Development Team Development Mission Find Important Problems Assess Quality Certify to Standard Fulfill Process Mandates Satisfy Stakeholders Assure Accountability Advise about QA Advise about Testing Advise about Quality Maximize Efficiency Minimize Cost Minimize Time Test Process Strategy Logistics Work-products
  12. 12. Ask yourself  What testing is easy (even if not very important)?  What testing is important (to find big bugs)?  What testing is expected (by people who matter)?  How can tools help?  Have we advocated for testability?  Are we minimizing administrative costs?
  13. 13.  Happy Path  Tour the Product – Sample Data – Variables – Files – Complexity – Menus & Windows – Keyboard & Mouse A quick test is a cheap test that has some value but requires little preparation, knowledge, or time to perform.  Interruptions  Undermining  Adjustments  Dog Piling  Continuous Use  Feature Interactions  Click on Help Cost as a Simplifying Factor Try quick tests as well as careful tests
  14. 14.  Input Constraint Attack  Click Frenzy  Shoe Test  Blink Test  Error Message Hangover A quick test is a cheap test that has some value but requires little preparation, knowledge, or time to perform.  Resource Starvation  Multiple Instances  Crazy Configs  Cheap Tools Cost as a Simplifying Factor Try quick tests as well as careful tests
  15. 15. Value (or Risk) as a Simplifying Factor Find problems that matter Instead of thinking pass vs. fail, consider thinking problem vs. no problem.  In general it can vastly simplify testing if we focus on whether the product has a problem that matters, rather than whether the product merely satisfies all relevant standards.  Effective testing requires that we understand standards as they relate to how our clients value the product.
  16. 16. One way to make a strategy… 1. Learn the product. 2. Think of important potential problems. 3. Think of how to search the product for those problems. 4. Think of how to search the product, in general. Think of ways that:  will take advantage of the resources you have.  comprise a mix of different techniques.  comprise something that you really can actually do.  serve the specific mission you are expected to fulfill.
  17. 17. Test Strategy for DecideRight
  18. 18. Test Strategy Text
  19. 19. Risk-Based Test Project Cycle: Testing itself is risk analysis. Experience Problems In the Field Analyze Potential Risks Analyze Actual Risks Short loop Long loop New Project ship Exploratory vs. ScriptedPerform Appropriate Testing Experience Problems & Potentialities
  20. 20. 18 Tests Project Environment Product Elements Quality Criteria Perceived Quality A Heuristic Test Strategy Model
  21. 21. 19 Project Environment Tests Product Elements Quality Criteria Perceived Quality A Heuristic Test Strategy Model
  22. 22. Example: Medical Device