Your SlideShare is downloading. ×
0
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Exploratory Testing Explained and Experienced

1,481

Published on

A 45 minute presentation delivered in Turku Agile Days for a mixed audience of mostly non-testers.

A 45 minute presentation delivered in Turku Agile Days for a mixed audience of mostly non-testers.

Published in: Spiritual
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,481
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
44
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Lisähuomiona: perinteiset testausvaiheet väärin nimetty, ovat korjausvaiheita. Hyväksymistestaus on ainoa testausvaihe – ja vain mikäli järjestelmätestaus on tuotannonkaltaista. Boltoninsertifiointi: "I test. You observe. We talk. You decide".
  • The slogan ”Better tests, better testers!” comes from a presentation by Cem Kaner. Itkonen, J., M.V. Mäntylä, C. Lassenius: Defect Detection Efficiency: Test Case Based vs. Exploratory Testing, in proceedings of ESEM, pp. 61-70, 2007Starts with an idea: a chest of goldEnds with a perception: nothing to find; earth ends here. Know your mission!Tunnustetaan ohjelmistokehityksen ja –testauksen luonne tietotyönä ja hyödynnetään ihmisiä tekemisen selkärankanaItseohjautuvat tiimitIhmisen kyky oppiaIhmiset voi laittaa jatkamaan toisen jäljiltä muutenkin kuin rakentamalla ensin dokumentaation jolla tieto ei kuitenkaan välity!Kotimaista tutkimusta Itkonen et al. Samat tulokset pienemmillä kustannuksillaKaksi isoa haastetta: testauksesta ei saada tuloksia (ei osata ajatella monipuolisesti) ja testausta ei osata lopettaa ajoissa.
  • The difference between people who do ad hoc testing (a starting form of exploratory testing) is significant. It is especially visible for a manager in a scrum meeting. When you, as a tester, answer the question: ”what did you do since yesterday and what do you plan on doing by tomorrow”, you’re not doing a very good job in ET if you tell that ”Found a problem in X, will find another one” – you’re not enabling your team to learn about testing, most likely since you did not learn yourself yet. Learning is a key element in exploratory testing!
  • Esimerkki FSC:ltä – areas, subareas, kuva riskiperusteisen testauksen kurssilta jne.
  • Kaikkea saa ja voi muuttaa, mutta ei hallitsemattomasti vaan tiedostaen koko ajan!Isompi konteksti: budjetointi jne. Tehtävänantolista vs. testiloki. Muutoksen hanskaaminen testauksen aikana.
  • Transcript

    • 1. ExploratoryTesting –Explained and Experienced
      Maaret Pyhäjärvi
      Email: <maaret@iki.fi> | Gsm: 040-8233777
      "The purpose of testing is largely to force everyone to see the detailed implications of their vague daydreams." -- James Bach
      Turku Agile Day 2011
      (2011-03-17)
      Erkki Pöyhönen & Maaret Pyhäjärvi
      Attribution (Tekijä mainittava) http://creativecommons.org/licenses/by/3.0/
    • 2. About the Speaker: Experiences in testing and being a tester on varied basis
      Started with testing -95: localization testing for greek version of an English software system
      Subcontractor-side on various projects: moving from localizations to functional testing
      Tried being in a developer role after thinking testers are not respected: the problem is not per role
      Teaching testing at HUT, giving public presentations and courses on testing
      Researcher in testing, consulting various organizations, teaching, being a test manager and a tester, with and without test automation
      At F-Secure with product business 2005 – 2008
      Moved to insurance pension sector, first with subcontractor side, now on customer side (Ilmarinen) as test manager / test specialist.
      Doing testing-related training on the side of a normal day-to-day job to get to meet brilliant testers in Finland.
      TESTER AND A WALKING TESTING DICTIONARY
      AGILE PROJECTS @ F-SECURE
      PLAN-DRIVEN PROJECTS @ ILMARINEN
    • 3. RealizationsaboutNature of Testing
      20
      14
      1639
      5±2
      3
    • 4. ”Testing mathematics” Extra Short Curriculum
      20
      14
      1639
      5±2
      A kids game ”20 questions teaching something relevant about testing. With two optional strategies to the game: 1) listing questions beforehand at a time you know the least or 2) asking questions one by one learning from the answer to the question already asked.
      Why in testing we continuously play with the foolish strategy?
      Common advice in testing is to think the expected results beforehand – in avoidance of blindness. Testing targets the unknown problems.
      Why do we invest a significant portion of limited project budgets actually to minimize the probability of finding the unexpectedproblems?
      • http://viscog.beckman.illinois.edu/flashmovie/15.php
      • 5. http://www.dothetest.co.uk/whodunnit.html
      The samemaynotbe the sameifyouchangeanyvariable. Testing is notaboutcoveringall of itagainst a specification, butthinkingabout ”is there a problem, is itrelevant”.
      Human memory is limited – ”human envelope”. We need documentation as support, especially when working under pressure. Documentation is huge expense, which only passes small portion of information. Why not create the documentation at the time we understand the needs best – towards the end of project.
    • 6. Exploratory Testing:Better tests, better testers!
      An approach to software testing
      Emphasized freedom and responsibility of an individual in a process where continuous optimization of value of information is important
      “Any testing process that involves simultaneous learning, test design, and execution.” – James Bach & Cem Kaner
      Disciplined, planned and controlled testing that emphasizes continuous learning
      Research in Finland, Itkonen et al. 2007
      No significant difference in results for preplanned test cases and exploratory testing
      More false alarms with test cases
      Comparing overall effort: significantly more in test-case based testing
      Unknown territory
      Test-related learning
      Design of new tests
      Test execution
      Result interpretation
      5
    • 7. There’s a Difference!
      A comment I hear often on the ET course I give:
      ”I’ve always been doing this just did not give it a name”
      I must emphasize that:
      I require more out of exploratory testing than just ”going where you feel like while testing”
      Ask yourself:
      Can you defend your choices of strategy and tactics?
      Can you explain what you’ve done in words that don’t just explain numbers of bugs found?
      How do you know if you’re done or not?
      6
      ”My testing would be unsystematic ad hoc testing if I could not tell the story of my tests, remember what I’ve tested or what my strategy was, or relate that back to my mission” –James Bach
    • 8. The Scale of Tester’s Freedom
      Role-based sandboxes
      Vague scripts
      Fragment test cases (scenarios)
      Pure scripted
      Freestyle
      Charters
      ”To what extent am I in control of the test and from where did the idea originate?”
      7
    • 9. Two Sides of Exploratory Testing
      For the tester
      For the manager
      Value
      Fun
      Freedom
      Controllability
      Flexibility
      Reliability
      Professionalism
      Visibility
      Respect
      8
    • 10. Framework of Management
      Vision (“Sandbox”)
      Current Charter
      Other Charters
      Details
      Coaching
      Quality Report
      Perception of quality and coverage
      Debriefing
      Charter backlog of the future testing
      ”A day’s work”
      Past
      Results
      Obstacles
      Outlook
      Feelings
      Next in importance!
      Test Manager
      Metrics summary
      #
      ?
      x
      Out of budget
      #, ?, x, +
      20:20:60
      Session sheets of the past testing
      Bug Reports
      Idea of exploration
      Tester
      9
    • 11. The Pieces in Management Framework
      A disciplined tester replanning on various levels
      Session with charter that provides a report
      Classification of information created as metrics
      Prioritizing of what test idea comes next
      Supporting reporting by debriefing
      Supporting skills development by coaching
      Creating a combined judgement of quality by quality reporting
      10
    • 12. Testautomation / tools in ET
      Anyform of acquiringquality-relatedinformationfits into exploratorytesting
      It’snotmanual, it’sbrain-engaged – and for makingthathappen, youneed to besmart with automation and tools!
      Example: youmightnot at firstknowwhatyou’relooking for...
      Search "<ns1:Koodi>" (443 hits in 169 files) in VE
        Y:ELLULAPATestausKokonaiseläketurvan-oteAnsaintatiedot-VEHaeAnsaintatiedot-VE__ 20110307 14-22-15.xml (4 hits)
                      Line 1313:                <ns1:Koodi>YL130I</ns1:Koodi>
                      Line 1317:                <ns1:Koodi>LAPA_172_011</ns1:Koodi>
                      Line 1321:                <ns1:Koodi>67</ns1:Koodi>
                      Line 1324:                <ns1:Koodi>67</ns1:Koodi>
       A lot of textcutawayfromhere…
    • 13. Potentially Useful Tools for Exploratory Testing
      SnapzPro
      http://www.ambrosiasw.com/utilities/snapzprox/
      HyperSnap
      http://www.hyperionics.com/hsdx/
      TestExplorer
      http://www.sirius-sqa.com/
      BBTestAssistant
      http://www.bbsoftware.co.uk/
      Spector
      http://www.spector.com/spectresoft.html
      Cygwin
      http://www.cygwin.com/
      UnixUtils
      http://unxutils.sourceforge.net/
      Regular Expressions
      http://www.ilovejackdaniels.com/cheat-sheets/regular-expressions-cheat-sheet/
      Process Monitor
      http://www.microsoft.com/technet/sysinternals/processesandthreads/processmonitor.mspx
      Process Explorer
      http://www.microsoft.com/technet/sysinternals/processesandthreads/processexplorer.mspx
      Holodeck
      http://www.securityinnovation.com/holodeck/index.shtml
      perlclip
      http://www.satisfice.com/tools.shtml
      DataGraph
      http://www.visualdatatools.com/DataPlot/index.html
      Vermont High Test
      http://www.vtsoft.com/
      Firebug
      http://www.getfirebug.com/
      iMacros
      http://www.iopus.com/imacros/firefox/
      JMeter
      http://jakarta.apache.org/jmeter/
      VirtualPC
      http://www.microsoft.com/windows/products/winfamily/virtualpc/default.mspx
      VMWare
      http://www.vmware.com/
      Parallels
      http://www.parallels.com/
    • 14. TestIdeas / Quick-and-Dirty
      Download the full 2-page CheatSheet with ideasfromElisabeth Hendrickson, James Lyndsay, and Dale Emery on Qualitytree.com
    • 15. (Exploratory) Testing Dynamics Source: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing Dynamics. v.2.2. 2009
      ”…skills that comprise professional and cost effective exploration of technology. Each is distinctly observable and learnable, and each is necessary to exploratory work.”
      ”Exploratory testing spirals upward toward a complete and professional set of test artifacts”
      ”A set of
      considerations designed to help you test robustly or evaluate someone else’s testing.”
      ” To develop ideas or search a complex space quickly yet thoroughly, not only must you look at the
      world from many points of view and perform many kinds of activities”
    • 16. Exploration SkillsSource: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing Dynamics. v.2.2. 2009
      Done
      To Do
      Issues
      Coverage
      All sources available
      Best use of time – effective and efficient work
      Making models
      Tool support – creative solutions
      Risk-based testing – scientific approach
      Keeping one’s eyes open
    • 17. To EndThis With: ExploratoryTestingStrenghts and BlindSpots
      Strengths
      Value-focused: stakeholders, risks, good-enoughquality and knowledge of quality
      Leverageseachtester’sstrengths
      Makestestersbettercontinuously
      Enablesadapting to changes and surprises
      Revealsinformationthatwasnotavailablewithoutlearningwhileusing the system
      Avoidnon-valuableduplicativework in testingthroughfocus on results
      Highbugfindingrates
      BlindSpots
      Requirescalendartime for learning with the software: the lessweknow, the morewe’relikely to miss
      Limited byeachtester’sweaknessesifnotmitigatedbyoverlappingassignmentsbased on individual’scompetencies
      Skilled and disciplinedwork – canbelearned with mentoring, nottransferredthroughdocumentation
    • 18. WHY do testers look both ways when crossing a one-way street?
      It may be one-way on this corner, two ways on that corner.
      You may only *think* it's a one-way street.
      Someone might be backing up
      Ears don't warn you when a bicycle is coming the wrong way.
      They realize people can be fooled by preconceptions.
      When helping someone, you've extra duty to seek improbables.
      Source: Various tweets by testers in 2010.

    ×