TestIstanbul May 2013 Keynote Experiences With Exploratory Testing


Published on

Slides for Keynote: Experiences With Exploratory Testing

Delivered at TestIstanbul May 2013 by Alan Richardson

Published in: Real Estate, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

TestIstanbul May 2013 Keynote Experiences With Exploratory Testing

  1. 1. Experiences with Exploratory Testing Alan Richardson @eviltester www.eviltester.com www.compendiumdev.co.uk www.seleniumsimplified.com
  2. 2. Future of Testing: New Techniques and Methodologies ● Exploratory Testing is not new, ● but people still don't do it. ● The future will be the same, as now, ● unless we take responsibility for it. ● Learn from experience, ● base experiments on observation.
  3. 3. Some things I've learned by doing and teaching Exploratory Testing ● Adopt a personal approach ● Exploration & experiment is natural ● Environment can kill exploratory testing ● Evidence based documentation ● Use Automation in your exploration
  4. 4. What does "Exploratory Testing" mean? "An Interactive Process of Simultaneous learning, test design, and test execution" James Bach http://www.satisfice.com/glossary.shtml
  5. 5. "If you master the source, but not the teachings, whenever you open your mouth you will speak at random; if you master the teachings but not the source, you will be like a one-eyed dragon." The story of Chinese Zen, by Nan Huai-Chin
  6. 6. Adopt a personal approach to Testing Take responsibility for your test approach: ● What does it mean to me? ● How do I do it? ● Why did I start? ● Why do I do it?
  7. 7. What do I mean by exploratory testing? ● Continual Evaluation & Re-evaluation ● Observation & Evidence ● Tool Augmented ● Low Overhead & Fast ● Skill Based ● Team Based ● Intent Characterized by:
  8. 8. Low Overhead & Fast ● Fast paced environments, becoming norm ● Success based on outcome, not compliance ● Maximize the time spend "doing" Techniques: ● Think through risks and target those. System risks, business risks. ● Chunk testing into 'charters' and 'sessions'. Log the testing against those. Raise defects.
  9. 9. Skill Based Rather than just relying on 'Techniques'... ● Assume the tester knows the techniques and applies them on an adhoc skill based basis ● Train people where risks and bugs exist and how to look for them ● Train people to be adaptable ● Trust people, give them responsibility
  10. 10. Team Based ● Use the whole team to review everyone else's work ● Write logs that others can read ● Review coverage of others testing and suggest new scope ● Pair to come up with new ideas ● Debrief rather than report ● The Manager tests as well
  11. 11. Continual Evaluation & Re-evaluation ● Start with minimal scope ● Test to learn what you need to test ● Write down what you know and don't know ● Continually revisit your approach and testing scope ● Revisit your use of tools ● Question your effectiveness ● Model the system and the process
  12. 12. Intent ● Know what you are testing for. ● Decide on what to test next. ● What risk are you concerned about? ● What requirement are you investigating? ● What data are you covering? ● Don't do it because you planned to. ● Do it because it is important now.
  13. 13. Observation & Evidence ● Focus / Defocus ● Go Meta ● Go Deeper ● Peripheral Observation ● Relate to existing evidence ● Produce Evidence ● screenshots, system logs, ● recordings, manual logs
  14. 14. Tool Augmented ● Tools augment our testing to help us test better ● Tools should never drive or dictate our process ● Exploratory Testing has heavy use of tools for: Observation, Data Creation, Logging, State Manipulation
  15. 15. My Exploratory Process ● Modelling ● Observing ● Reflective ● Intentional ● Manipulative
  16. 16. Why should you do exploratory testing? • Because you liked the definition? • Because you liked the characteristics? • Because you want to test more effectively? Personal Approach I don't know why you should do it. I only know why I do it, and why I started doing it.
  17. 17. Why did I start doing exploratory testing? "There is a risk that the test team will waste a lot of time and money by creating scripts that are wrong and will never be run" • Frustration Mainly • Heavily Scripted environment which wasn't working well
  18. 18. In order to be effective we had to Subvert the process Step # Description Expected Result 1 I've only added this because the tool made me I'll test the conditions in the case and write down my observations
  19. 19. Why do I keep doing Exploratory Testing? Because: ● It works ● I get better at it, the more I do it ● I can train people to do it ● It adapts to different environments, and processes
  20. 20. ● Stop Following ● Do and Learn ● Change What you do Take Responsibility to succeed
  21. 21. Develop Ability & enable Response Ability Response Able Adapt Able Adapt Ability Account Ability Account Able
  22. 22. What stops Exploratory Testing? ● Environment ● Personal Fear ● Existing Learning
  23. 23. Automation & Exploratory Testing ● Tool Augmented ● Exploratory Automated Testing ● Models & Checks ● Automation leads to Exploration
  24. 24. An example: Random Coverage of  Defined Scope ● New Registration Process ● Manual Testing kept finding problems, ● Every app change introduced new errors ● Randomize Coverage of a Model ● Model registration as a process of equivalent steps ● Fill Name, Fill Password, Fill PostCode etc. ● Model data as a set of templates to randomly fill ● {a-z,A-Z, }*3-25, {a-z,A-Z,0-9,!"£$%^&*()}*6-18 ● Leave to run, Automated Checks flag issues ● When run is clean, perform exploratory testing
  25. 25. When Automation Leads to exploration ● Sometimes you want to get somewhere known fast, ● so that you can explore unknowns ● Automate the 'getting to' ● Randomise to maximise chance that this shows a problem ● When there, engage brain and test
  26. 26. Explore and Experiment with your Testing ● The Future of Testing requires us to Unlearn. ● Change our existing beliefs based on evidence. ● Make it easy to experiment with new approaches. ● Evaluate Qualitatively as well as Quantitatively. ● Take responsibility, own your test approach.
  27. 27. 27 Blogs and Web Sites ● SeleniumSimplified.com ● EvilTester.com ● Testing Papers and Tools – CompendiumDev.co.uk @eviltester Online Training Courses ● Technical Web Testing 101 – Unow.be/at/udemy101 ● Intro to Selenium – Unow.be/at/udemystart ● Selenium 2 WebDriver API – Unow.be/at/udemyapi Videos youtube.com/user/EviltesterVideos Books Selenium Simplified Unow.be/rc/selsimp Alan Richardson is an Independent Test Consultant based in the UK. He offers training and consultancy in Selenium WebDriver, exploratory and technical web testing. – uk.linkedin.com/in/eviltester Contact Alan for training and consultancy tailored to your needs: alan@compendiumdev.co.uk