Downloaded 12 times
More Related Content
Similar to Rediscover Exploratory Testing
Rediscover Exploratory Testing
- 1.
- 2. Ingo Phillip After several years working as a scientist in particle physics and computational fluid dynamics, Ingo Philipp immersed himself in the multifaceted worlds of functional software testing and development. Ingo is on the product management team at Tricentis where his responsibilities range from product development and product marketing to test management, test conception, test design, and test automation. His experiences with software testing embrace the application of agile as well as classic testing methodologies in the insurance, banking, commerce, telecommunications, and energy sectors.
- 3. Ingo Philipp © 2017by . Rediscover Exploratory Testing Thursday, May 11, 2017 - 9:45am to 10:45am
- 4. © 2017 by Agenda 1.Why you should go for it? 2. What is exploratory testing? Common confusions, misconceptions & controversies. Relation & distinction to other approaches. 3. Who should do it? 4. How to practice it? Heuristics & techniques. 5. When to practice it? How to integrate it into the software lifecycle?
- 5. © 2017 by 80% ManualTesting 20% Automated Testing UI API Today The reliance on manual testing is the top technical challenge in app development. World Quality Report, 2015/16 Test automation requires developers. Testing Present
- 6. © 2017 by UI API TodayTomorrow Exploratory Tests Automated UI Tests API Tests Orchestrated Service Virtualization 80% Manual Testing +85% 20% Automated Testing - key enabler for high automation rates - Testing Future
- 7. © 2017 by HistoricalContext 2016 Is the term a source of inspiration or confusion? Is it an activity or a technique? Is it something that you do? A way or style of doing something? Is it an approach? Isn’t then all testing exploratory testing? Is the term helping or hurting the testing craft? 2010 2000 1988 Recognition Controversy Hostility Tolerance Acceptance Cluelessness Ignorance Nescience Unawareness Cem Kaner, Testing Computer Software *Glenford J. Myers, The Art of Software Testing, 1979
- 8. © 2017 by Nobodyreally knows how to do it. Everyone thinks everyone else is doing it. So everyone claims they are doing it. Everyone talks about it. Natural Consequence
- 9. © 2017 by Exploratorytesting is not defined by any particular example of exploratory testing. James Bach, What Exploratory Testing Isn’t, 2011 Formal TestingInformal Testing Exploratory Testing Manual Testing Automated Testing Ad-hoc Testing Testing Continuum
- 10. © 2017 by Exploratorytesting is about learning the product, designing test cases, executing test cases & interpreting the test results at the same time. Cem Kaner, Exploratory Testing Research Summit, 2006 Official Definition
- 11. © 2017 by Testing,especially exploratory testing, is always a search for information. Cem Kaner Core Conclusion
- 12. © 2017 by Risks Yoursystem under test Performance Issue Usability Issue Functional Issue Stability Issue Reliability IssueScalability Issue Coherence Issue Understandability Issue Testability Issue Convenience Issue Security Issue Accessibility Issue Why are we missing these issues? ≫ What do we have to do to proactively find them? ≪
- 13. © 2017 by Risks AutomatedTesting I’m an automated test case. Usability Issue Stability Issue Reliability IssueScalability Issue Coherence Issue Understandability Issue Testability Issue Convenience Issue Security Issue Accessibility Issue Performance Issue Functional Issue Automation is doing, what automation does.
- 14. © 2017 by12 3 4 5 … … … … … … … Risks The narrower the view, the wider the ignorance. ≫ The miss-the-forest-for-the-trees problem ≪ Automated testing is not automated manual testing. ≫ The no-brain-in-the-loop problem ≪ I think what I am doing is hard to automate in any meaningful way. ≫ The unbalanced-effort-versus-value problem ≪ Developer testing is like a lawn service. I could do it myself, I simply choose not to. ≫ The unwilling-tester problem ≪ You don’t write automation you should write, you write the automation you know how to write. ≫ The line-of-the-least-resistance problem ≪ Automated Testing
- 15. © 2017 by I’man automated test case. Manual Testing I’m the same test case executed manually.
- 16. © 2017 by Planas you test! Story-Based Motivating Credible Exploratory Testing Exploratory Branching The art of deciding on what to test next. New Testing Idea Test Cases vs. Scenarios Pre-specified inputs vs. hypothetical situation
- 17. © 2017 by Specification BasedTesting Exploratory Testing Main Purpose Analyze Potential Risks Think outside the system boundaries. Monitor Known Risks Think within the system. *That’s an illustration. Don’t confuse it with reality.
- 18. © 2017 by AnalyzePotential Risks Think outside the system boundaries. Specification Based Testing Exploratory Testing Monitor Known Risks Think within the system. Agile Testing Law Checked ≫ Efficient Automated Testing ≪ Explored ≫ Effective Exploratory Testing ≪ Tested ≫ Productive Testing ≪ + = Agile Equation Of Motion
- 19. © 2017 by Session Specification BasedTesting Analyze Potential Risks Problem vs. No Problem Session-Based Testing Testing Mission How to explore? Goal Scope Sprint Start Sprint End2h 2h 2h2h Business Product Owner Subject Matter Expert Business Analyst Product Manager Development Developer Support Engineer Documentation Expert Tester Operations System Engineer System Administrator Network Engineer Database Administrator Release Engineer
- 20. © 2017 by Testing Mission Howto explore? Goal Scope Session-Based Testing Sprint Start Sprint End2h 2h 2h2h Past. What happened during the exploratory test? Results. What was achieved during the exploratory test? Obstacles. What got in the way of good testing? Outlook. What still needs to be done? Feelings. How does the tester feel about all this? ≫ PROOF ≪ Debriefing
- 21. © 2017 by Session-basedtesting can be thought of as structured exploratory testing. Michael Bolton *Exploratory testing is not directionless. A session sets the agenda for exploratory testing via its charter. Session-Based Testing
- 22. © 2017 by Exploratorytesting starts with a focus on the things we don’t know & illusions we’re holding true without empirical evidence. Maaret Pyhäjärvi Session-Based Testing *Explore the invisible to avoid the unthinkable happening to the anonymous.
- 23. © 2017 by A B VisitingLondon ~100 Billion Billion Billion Billion Billion Billion Billion Combinations = 50
- 24. © 2017 by Settinggoals is the first step in turning the invisible into the visible. A B Visiting London Tony Robbins
- 25. © 2017 byBeinvolved in every stage of the data’s life cycle. FedEx Tour A tour is an exploration of a product that is organized around a theme.
- 26. © 2017 byThinksuperficially! Whatever you do, don’t go beyond skin deep. Supermodel Tour
- 27. © 2017 by TourTesting A uniquely portable magic* Money Tour Focus on money-generating features. Landmark Tour Focus on key feature combinations. FedEx Tour Track data through various stages. Back Alley Tour Focus on the least-used features. All-Nighter Tour Run scenarios over & over again. The Saboteur Sabotage system resources. Intellectual Tour Ask the hardest questions to test the limits. Supermodel Tour Take a single-minded focus on the UI. Collector’s Tour Generate & document all possible outputs. Rained-Out Tour Start, stop & cancel time-consuming tasks. Museum Tour Focus on legacy code in new environments. Couch Potato Tour Do as little as possible, process default values. Guidebook Tour Follow the user manual’s advice to its letter. Antisocial Tour Focus on least-likely & known bad inputs. After-Hours Tour Focus on batch functionality.
- 28. © 2017 by PolychromeTesting How we think, collaborate, communicate and empathize have massive effects on our work and the outcomes of what we do; testing. Julian Harty
- 29. © 2017 by CriticalEmotionalFactual PositiveCreative Control www.debonogroup.com Polychrome Testing
- 30. © 2017 by Factual TheDetective It calls for information known or needed. It concentrates on facts and figures, and things we believe but don’t yet know. It’s about questioning facts, just the facts. www.debonogroup.com Factual white hat thinking Polychrome Testing A detective searches for clues, for evidence and for facts that will help them to effectively solve the case they are working on.
- 31. © 2017 by Itsignifies feelings, hunches and intuition. With this hat we express emotions, feelings and share fears, likes, dislikes, loves, passion, anger and hates. Emotional The Heart www.debonogroup.com Emotional red hat thinking Polychrome Testing A heart is a very intuitive organ that senses subtle changes in feeling and emotion when circumstances present themselves.
- 32. © 2017 by It’sall about judgment. With this hat we are devil's advocates, and spot the difficulties and dangers, where things might go wrong. Probably the most powerful of all hats but a problem if overused. Critical The Reaper www.debonogroup.com Critical black hat thinking Polychrome Testing A Reaper is a mythical creature who brings death and destruction to the living.
- 33. © 2017 by Itsymbolizes sunshine, optimism. Under this hat you actively seeking the positives and probe for value and benefit. It’s about constructive thinking and encouraging. Positive The Sun www.debonogroup.com Positive yellow hat thinking Polychrome Testing A sun is bright, happy and powerful, helping to instill life into everything it touches.
- 34. © 2017 by Itfocuses on creativity, on possibilities, on fresh alternatives, and on new ideas without criticism. It's an opportunity to express new concepts and new perceptions. It’s all about shaping ideas. Creative The Seedling www.debonogroup.com Creative green hat thinking Polychrome Testing A seedling sprouts from the ground and grows persistently - expanding its leaves and branches in many unexpected directions.
- 35. © 2017 by Control TheMovie Director It manages the entire thinking process. It provides the focus, purpose & and controls the use of other hats. With this hat we think about our thinking. www.debonogroup.com Polychrome Testing
- 36. © 2017 by SessionPlanning Single Session Single Scope & Multiple Goals Multiple Sessions Multiple Scopes & Multiple Goals Session Owner Session Owner Session Owner Session Owner
- 37. © 2017 by CoreElements 1. Session-Based Testing Structure exploratory testing; make it plannable & applicable for larger teams. 2. Requirements-Based Testing Limit the scope; make it manageable for skilled & unskilled testers; then iterate. 3. Tour-Based Testing Set crystal-clear goals to turn the invisible into the visible. 4. Polychrome Testing Diversify your exploration; explore the product from different viewpoints. 5. Scenario-Based Testing Capture testing ideas in units of scenarios; translate your findings into a reviewable documentation.
- 38. Sprint Start Sprint End PackageBuild 2 N-1 N NN-1 Unit& Smoke Tests 1 2 1 Regression Tests User Story Sprint Acceptance Test One Day Life-Cycle Integration Exploratory Session Test Case Design Automation
- 39. Sprint Start Sprint End 2 User Story Life-Cycle Integration 1.Whatshould be explored? Business Use Case, System Use Case, Epic, Theme, User Story, Technical Component, … 2. What are the core characteristics? Performance, Usability, Stability, Security, Safety, Reliability, Understandability, … 3. Which tour(s) are required? Supermodel Tour, FedEx Tour, Money Tour, Landmark Tour, </ Custom Tour/>, … 4. Which hat(s) should be considered? Black Hat, White Hat, Red Hat, Green Hat, Yellow Hat, </ Custom Hat /> 5. Who is required, and who takes which hat? 6. Who owns the session(s)? 7. How long should the session(s) last? 1 2 1 Exploratory Session
- 40. © 2017 by ConcludingRemarks Exploratory testing is not so much a thing that you do, it’s far more a way you think. Michael Bolton *The eyes are useless, when the mind is blind.