Exploratory testing is an approach that emphasizes freedom and responsibility of individual testers in a process where continuous learning, test design, and execution occur simultaneously. It is a disciplined, planned, and controlled form of testing that focuses on continuous learning. Research has shown there is no significant difference in results between exploratory testing and preplanned test cases, but exploratory testing requires significantly less effort overall. Effective exploratory testing requires skills like making models, keeping an open mind, and risk-based testing approaches. Both the strengths and potential blind spots of exploratory testing are discussed.

1. ExploratoryTesting –Explained and ExperiencedMaaret PyhäjärviEmail: <maaret@iki.fi> | Gsm: 040-8233777"The purpose of testing is largely to force everyone to see the detailed implications of their vague daydreams." -- James Bach Turku Agile Day 2011(2011-03-17) Erkki Pöyhönen & Maaret Pyhäjärvi Attribution (Tekijä mainittava) http://creativecommons.org/licenses/by/3.0/

2. About the Speaker: Experiences in testing and being a tester on varied basisStarted with testing -95: localization testing for greek version of an English software systemSubcontractor-side on various projects: moving from localizations to functional testingTried being in a developer role after thinking testers are not respected: the problem is not per roleTeaching testing at HUT, giving public presentations and courses on testingResearcher in testing, consulting various organizations, teaching, being a test manager and a tester, with and without test automationAt F-Secure with product business 2005 – 2008 Moved to insurance pension sector, first with subcontractor side, now on customer side (Ilmarinen) as test manager / test specialist. Doing testing-related training on the side of a normal day-to-day job to get to meet brilliant testers in Finland. TESTER AND A WALKING TESTING DICTIONARYAGILE PROJECTS @ F-SECUREPLAN-DRIVEN PROJECTS @ ILMARINEN

3. RealizationsaboutNature of Testing201416395±23

4. ”Testing mathematics” Extra Short Curriculum201416395±2A kids game ”20 questions teaching something relevant about testing. With two optional strategies to the game: 1) listing questions beforehand at a time you know the least or 2) asking questions one by one learning from the answer to the question already asked. Why in testing we continuously play with the foolish strategy?Common advice in testing is to think the expected results beforehand – in avoidance of blindness. Testing targets the unknown problems. Why do we invest a significant portion of limited project budgets actually to minimize the probability of finding the unexpectedproblems?http://viscog.beckman.illinois.edu/flashmovie/15.php

5. http://www.dothetest.co.uk/whodunnit.htmlThe samemaynotbe the sameifyouchangeanyvariable. Testing is notaboutcoveringall of itagainst a specification, butthinkingabout ”is there a problem, is itrelevant”. Human memory is limited – ”human envelope”. We need documentation as support, especially when working under pressure. Documentation is huge expense, which only passes small portion of information. Why not create the documentation at the time we understand the needs best – towards the end of project.

6. Exploratory Testing:Better tests, better testers! An approach to software testingEmphasized freedom and responsibility of an individual in a process where continuous optimization of value of information is important“Any testing process that involves simultaneous learning, test design, and execution.” – James Bach & Cem KanerDisciplined, planned and controlled testing that emphasizes continuous learningResearch in Finland, Itkonen et al. 2007No significant difference in results for preplanned test cases and exploratory testingMore false alarms with test casesComparing overall effort: significantly more in test-case based testingUnknown territoryTest-related learningDesign of new testsTest executionResult interpretation5

7. There’s a Difference!A comment I hear often on the ET course I give:”I’ve always been doing this just did not give it a name”I must emphasize that:I require more out of exploratory testing than just ”going where you feel like while testing”Ask yourself:Can you defend your choices of strategy and tactics?Can you explain what you’ve done in words that don’t just explain numbers of bugs found? How do you know if you’re done or not?6”My testing would be unsystematic ad hoc testing if I could not tell the story of my tests, remember what I’ve tested or what my strategy was, or relate that back to my mission” –James Bach

8. The Scale of Tester’s FreedomRole-based sandboxesVague scriptsFragment test cases (scenarios)Pure scriptedFreestyleCharters”To what extent am I in control of the test and from where did the idea originate?”7

9. Two Sides of Exploratory TestingFor the testerFor the managerValueFunFreedomControllabilityFlexibilityReliabilityProfessionalismVisibilityRespect8

10. Framework of ManagementVision (“Sandbox”)Current CharterOther ChartersDetailsCoachingQuality ReportPerception of quality and coverageDebriefingCharter backlog of the future testing”A day’s work”PastResultsObstaclesOutlookFeelingsNext in importance!Test ManagerMetrics summary#?xOut of budget#, ?, x, +20:20:60Session sheets of the past testingBug ReportsIdea of explorationTester9

11. The Pieces in Management FrameworkA disciplined tester replanning on various levelsSession with charter that provides a reportClassification of information created as metricsPrioritizing of what test idea comes nextSupporting reporting by debriefingSupporting skills development by coachingCreating a combined judgement of quality by quality reporting10

12. Testautomation / tools in ETAnyform of acquiringquality-relatedinformationfits into exploratorytestingIt’snotmanual, it’sbrain-engaged – and for makingthathappen, youneed to besmart with automation and tools! Example: youmightnot at firstknowwhatyou’relooking for...Search "<ns1:Koodi>" (443 hits in 169 files) in VE Y:\ELLU\LAPA\Testaus\Kokonaiseläketurvan-ote\Ansaintatiedot-VE\HaeAnsaintatiedot-VE__ 20110307 14-22-15.xml (4 hits) Line 1313: <ns1:Koodi>YL130I</ns1:Koodi> Line 1317: <ns1:Koodi>LAPA_172_011</ns1:Koodi> Line 1321: <ns1:Koodi>67</ns1:Koodi> Line 1324: <ns1:Koodi>67</ns1:Koodi> A lot of textcutawayfromhere…

13. Potentially Useful Tools for Exploratory TestingSnapzProhttp://www.ambrosiasw.com/utilities/snapzprox/HyperSnap http://www.hyperionics.com/hsdx/TestExplorer http://www.sirius-sqa.com/BBTestAssistant http://www.bbsoftware.co.uk/Spector http://www.spector.com/spectresoft.htmlCygwin http://www.cygwin.com/UnixUtils http://unxutils.sourceforge.net/Regular Expressionshttp://www.ilovejackdaniels.com/cheat-sheets/regular-expressions-cheat-sheet/Process Monitorhttp://www.microsoft.com/technet/sysinternals/processesandthreads/processmonitor.mspxProcess Explorerhttp://www.microsoft.com/technet/sysinternals/processesandthreads/processexplorer.mspxHolodeckhttp://www.securityinnovation.com/holodeck/index.shtmlperlclip http://www.satisfice.com/tools.shtmlDataGraphhttp://www.visualdatatools.com/DataPlot/index.htmlVermont High Test http://www.vtsoft.com/Firebug http://www.getfirebug.com/iMacros http://www.iopus.com/imacros/firefox/JMeter http://jakarta.apache.org/jmeter/VirtualPChttp://www.microsoft.com/windows/products/winfamily/virtualpc/default.mspxVMWare http://www.vmware.com/Parallels http://www.parallels.com/

14. TestIdeas / Quick-and-DirtyDownload the full 2-page CheatSheet with ideasfromElisabeth Hendrickson, James Lyndsay, and Dale Emery on Qualitytree.com

15. (Exploratory) Testing Dynamics Source: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing Dynamics. v.2.2. 2009”…skills that comprise professional and cost effective exploration of technology. Each is distinctly observable and learnable, and each is necessary to exploratory work.””Exploratory testing spirals upward toward a complete and professional set of test artifacts””A set ofconsiderations designed to help you test robustly or evaluate someone else’s testing.”” To develop ideas or search a complex space quickly yet thoroughly, not only must you look at theworld from many points of view and perform many kinds of activities”

16. Exploration SkillsSource: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing Dynamics. v.2.2. 2009DoneTo DoIssuesCoverageAll sources available Best use of time – effective and efficient workMaking modelsTool support – creative solutionsRisk-based testing – scientific approachKeeping one’s eyes open

17. To EndThis With: ExploratoryTestingStrenghts and BlindSpotsStrengthsValue-focused: stakeholders, risks, good-enoughquality and knowledge of qualityLeverageseachtester’sstrengthsMakestestersbettercontinuouslyEnablesadapting to changes and surprisesRevealsinformationthatwasnotavailablewithoutlearningwhileusing the systemAvoidnon-valuableduplicativework in testingthroughfocus on resultsHighbugfindingratesBlindSpotsRequirescalendartime for learning with the software: the lessweknow, the morewe’relikely to missLimited byeachtester’sweaknessesifnotmitigatedbyoverlappingassignmentsbased on individual’scompetenciesSkilled and disciplinedwork – canbelearned with mentoring, nottransferredthroughdocumentation