Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Exploratory Testing


Published on

Presentation on Exploratory Testing delivered by Anutthara Baradwaj at the June 09 Knowledge sharing session hosted at Oracle India

Published in: Technology
  • Login to see the comments

Exploratory Testing

  1. 1. GSJGD Exploratory Testing Anutthara Bharadwaj Microsoft
  2. 2. if there was an Olympics for FAILURE
  3. 3. Or a World Cup for that which fails the most
  4. 4. surely SOFTWARE would reign supreme
  5. 5. GSJGD How do we find the *%&^ bugs?!!
  6. 6. Manual v. automated • Automation is cool, it gets respect … but … – Automation is code • like any code it will take time, have bugs, cause false positives and require maintenance • it will have an imperfect oracle • Manual testing is tedious … but … – It finds over 90% of the bugs in the system
  7. 7. Manual testing • Scripted manual testing – From rigid (click this button, enter this value) to flexible (achieve this result) – Requires preparatory work – Scripts act as their own documentation • Exploratory testing – Minimizes preparatory work – Documentation of testing is extra work
  8. 8. Manual testing • We need to get better at it … A LOT better at it • What is the right prep time v. test time? • What are the right parts of the app to focus on? – Can we pinpoint problematic functionality? – Can we focus on changed code? – Should we focus on buggy parts? • Where is the method in the madness?
  9. 9. Types of exploratory testing • Freestyle • Scenario • Strategy • Feedback
  10. 10. Exploratory testing • Freestyle exploratory testing – Ad hoc exploration of an application’s features – No rules, not necessary to account for coverage, past tests, etc. – Good for: • familiarizing yourself with an application to prepare for more sophisticated methods • quick smoke tests, sometimes finds bugs
  11. 11. Exploratory testing • Scenario-based exploratory testing – An extension of traditional scenario testing – Start with: • user stories • e2e scenarios • test scenarios – Then widen the scope and inject variation as you execute the scenarios
  12. 12. Exploratory testing • Strategy-based exploratory testing – Freestyle testing combined with known bug finding techniques • documented strategies like boundary values and combinatorial testing • intuitive instinct from experience or familiarity with the application – Focus is on learning tips and tricks and recognizing when to apply them
  13. 13. Exploratory testing • Feedback-based exploratory testing – Starts out freestyle until a test history is built up – That feedback then guides future exploration • Test history • Coverage • Churn • … – Tools are often an integral part of this type of testing as they ‘remember’ history for us
  14. 14. The tourist metaphor • You’re visiting Hyderabad for the first time – You can go it alone – You can buy a guidebook – You can hire a guide – You can take the bus tour • On return visits you can – Repeat the best tours – Choose to visit new places
  15. 15. The tourist metaphor • You’re testing an application – What do you use as a guide? – Don’t just wander the app • Always have a goal in mind • When presented with a choice, allow the goal to guide you • Track which goals work for you – Coverage of code, features, UI – Find important bugs – Force the software to best exhibit its functionality
  16. 16. The guidebook tour • Guidebooks help tourists navigate choices – Boil down the possibilities to a ‘best of’ list – Many tourist follow this advice exclusively • Use – User manual, follow its advice – Online help (the F1 tour) – Third party advice (the blogger’s tour) – Detractors (the pundit’s tour) – Competing products (the competitor’s tour)
  17. 17. The antisocial tour • Tours have rules and there are places off limits • Antisocial tourists ignore those rules – Enter illegal input (the crime spree tour) – Enter the least likely input (the opposite tour) – Choose an input, change your mind, use undo, use cancel, then do it all over again
  18. 18. The back alley tour • Explore the back alleys of your app • Do you track feature usage? – Tour the least popular features – Mix and match popular and unpopular features (mixed destination tour)
  19. 19. The couch potato tour • Being a nonparticipant on a tour is a waste of money • Sometimes for a tester, it finds bugs – Do as little as possible on every screen – Leave fields blank, enter as little info as possible, always take the shortest path • Why? – Default values have to be set, used – Decisions have to be made by the software – Just because you aren’t working hard doesn’t mean the software isn’t either
  20. 20. The obsessive compulsive tour • OCD tours would likely be quite unpopular for real tourists – Avoiding sidewalk cracks – Walking the same street over and over obsessively • But testers can use repetition to their advantage – Developers often code based on an expected path – Repeating inputs individually and in sequence can often lead to bugs
  21. 21. The all nighter tour • The ‘clubbing tour’ is about staying out all night, a real test of the constitution • Software should stay out all night too – Keep it working, never shut it down – Keep files open without saving or closing them • This tests time-out functionality, finds memory problems and leaks that are masked by shutdown-restart
  22. 22. The TOGOF tour • Test-One-Get-One-Free • Run multiple copies of the app • Put them through their paces • Try and open the same files, access the same resources … can they coexist? • Why TOGOF? Because if you find a bug in one copy, you’ve broken them all 
  23. 23. The saboteur • The sabotage tour • Ask the software to do something – Access a resource – Open a file – Perform a tasks • Understand the resource it uses – Memory – Files – Network • Then prevent it from doing so – Take away memory – Delete/rename files – Turn off the network
  24. 24. The collector’s tour • Some people need to do it all – Pictures of landmarks, samples at a wine tasting, signatures from the Disney characters – They don’t want to miss anything • Testers can learn from this – Force the software to produce every output • Buy items from every department • Put every special character in an email • Fill a document with every possible graphic • …
  25. 25. The supermodel tour • This tour goes only skin deep – Ignore the function and meaning and look only at the interface – It’s about first impressions • Watch the interface elements – Do they look good and render properly? – Are interface elements consistent across the app? – As you make changes does the UI refresh properly?
  26. 26. Types of exploratory testing • Scenario • Then inject variation – Start with a scenario – Freestyle guidance • Strategy – Tour metaphors – Start with accumulated testing knowledge • Feedback – Test for a while – Build up a history – Use the history to guide additional testing
  27. 27. Obstacles to exploratory testing • Where’s the accountability? • How do we track completeness? • What metrics are important? WE NEED TOOLS!
  28. 28. GSJGD Q/A