Your SlideShare is downloading. ×
  • Like
TestIstanbul May 2013 Keynote Experiences With Exploratory Testing
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

TestIstanbul May 2013 Keynote Experiences With Exploratory Testing


Slides for Keynote: Experiences With Exploratory Testing …

Slides for Keynote: Experiences With Exploratory Testing

Delivered at TestIstanbul May 2013 by Alan Richardson

Published in Real Estate , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Experiences with Exploratory Testing Alan Richardson @eviltester
  • 2. Future of Testing: New Techniques and Methodologies ● Exploratory Testing is not new, ● but people still don't do it. ● The future will be the same, as now, ● unless we take responsibility for it. ● Learn from experience, ● base experiments on observation.
  • 3. Some things I've learned by doing and teaching Exploratory Testing ● Adopt a personal approach ● Exploration & experiment is natural ● Environment can kill exploratory testing ● Evidence based documentation ● Use Automation in your exploration
  • 4. What does "Exploratory Testing" mean? "An Interactive Process of Simultaneous learning, test design, and test execution" James Bach
  • 5. "If you master the source, but not the teachings, whenever you open your mouth you will speak at random; if you master the teachings but not the source, you will be like a one-eyed dragon." The story of Chinese Zen, by Nan Huai-Chin
  • 6. Adopt a personal approach to Testing Take responsibility for your test approach: ● What does it mean to me? ● How do I do it? ● Why did I start? ● Why do I do it?
  • 7. What do I mean by exploratory testing? ● Continual Evaluation & Re-evaluation ● Observation & Evidence ● Tool Augmented ● Low Overhead & Fast ● Skill Based ● Team Based ● Intent Characterized by:
  • 8. Low Overhead & Fast ● Fast paced environments, becoming norm ● Success based on outcome, not compliance ● Maximize the time spend "doing" Techniques: ● Think through risks and target those. System risks, business risks. ● Chunk testing into 'charters' and 'sessions'. Log the testing against those. Raise defects.
  • 9. Skill Based Rather than just relying on 'Techniques'... ● Assume the tester knows the techniques and applies them on an adhoc skill based basis ● Train people where risks and bugs exist and how to look for them ● Train people to be adaptable ● Trust people, give them responsibility
  • 10. Team Based ● Use the whole team to review everyone else's work ● Write logs that others can read ● Review coverage of others testing and suggest new scope ● Pair to come up with new ideas ● Debrief rather than report ● The Manager tests as well
  • 11. Continual Evaluation & Re-evaluation ● Start with minimal scope ● Test to learn what you need to test ● Write down what you know and don't know ● Continually revisit your approach and testing scope ● Revisit your use of tools ● Question your effectiveness ● Model the system and the process
  • 12. Intent ● Know what you are testing for. ● Decide on what to test next. ● What risk are you concerned about? ● What requirement are you investigating? ● What data are you covering? ● Don't do it because you planned to. ● Do it because it is important now.
  • 13. Observation & Evidence ● Focus / Defocus ● Go Meta ● Go Deeper ● Peripheral Observation ● Relate to existing evidence ● Produce Evidence ● screenshots, system logs, ● recordings, manual logs
  • 14. Tool Augmented ● Tools augment our testing to help us test better ● Tools should never drive or dictate our process ● Exploratory Testing has heavy use of tools for: Observation, Data Creation, Logging, State Manipulation
  • 15. My Exploratory Process ● Modelling ● Observing ● Reflective ● Intentional ● Manipulative
  • 16. Why should you do exploratory testing? • Because you liked the definition? • Because you liked the characteristics? • Because you want to test more effectively? Personal Approach I don't know why you should do it. I only know why I do it, and why I started doing it.
  • 17. Why did I start doing exploratory testing? "There is a risk that the test team will waste a lot of time and money by creating scripts that are wrong and will never be run" • Frustration Mainly • Heavily Scripted environment which wasn't working well
  • 18. In order to be effective we had to Subvert the process Step # Description Expected Result 1 I've only added this because the tool made me I'll test the conditions in the case and write down my observations
  • 19. Why do I keep doing Exploratory Testing? Because: ● It works ● I get better at it, the more I do it ● I can train people to do it ● It adapts to different environments, and processes
  • 20. ● Stop Following ● Do and Learn ● Change What you do Take Responsibility to succeed
  • 21. Develop Ability & enable Response Ability Response Able Adapt Able Adapt Ability Account Ability Account Able
  • 22. What stops Exploratory Testing? ● Environment ● Personal Fear ● Existing Learning
  • 23. Automation & Exploratory Testing ● Tool Augmented ● Exploratory Automated Testing ● Models & Checks ● Automation leads to Exploration
  • 24. An example: Random Coverage of  Defined Scope ● New Registration Process ● Manual Testing kept finding problems, ● Every app change introduced new errors ● Randomize Coverage of a Model ● Model registration as a process of equivalent steps ● Fill Name, Fill Password, Fill PostCode etc. ● Model data as a set of templates to randomly fill ● {a-z,A-Z, }*3-25, {a-z,A-Z,0-9,!"£$%^&*()}*6-18 ● Leave to run, Automated Checks flag issues ● When run is clean, perform exploratory testing
  • 25. When Automation Leads to exploration ● Sometimes you want to get somewhere known fast, ● so that you can explore unknowns ● Automate the 'getting to' ● Randomise to maximise chance that this shows a problem ● When there, engage brain and test
  • 26. Explore and Experiment with your Testing ● The Future of Testing requires us to Unlearn. ● Change our existing beliefs based on evidence. ● Make it easy to experiment with new approaches. ● Evaluate Qualitatively as well as Quantitatively. ● Take responsibility, own your test approach.
  • 27. 27 Blogs and Web Sites ● ● ● Testing Papers and Tools – @eviltester Online Training Courses ● Technical Web Testing 101 – ● Intro to Selenium – ● Selenium 2 WebDriver API – Videos Books Selenium Simplified Alan Richardson is an Independent Test Consultant based in the UK. He offers training and consultancy in Selenium WebDriver, exploratory and technical web testing. – Contact Alan for training and consultancy tailored to your needs: