The document outlines a pilot program for exploratory testing (ET) led by Henrik Andersson, highlighting its benefits in simultaneous learning, test design, and execution. It details the process of implementing ET, including scheduling, session guidelines, and the role of exploratory testing champions (ETCs) who facilitate knowledge sharing within teams. The results indicate that ET discovered more errors than traditional regression testing, emphasizing the importance of flexibility in testing approaches and documentation.

1. Exploratory Testing Champions
Henrik Andersson
henrik.andersson@houseoftest.se

2. The Client

3. Background

4. What is Exploratory Testing
Exploratory testing is simultaneous learning, test design, and test execution.
"...a way of thinking about testing“
Programs that pass certain tests tend to continue to pass the same tests and are more likely to fail other tests that are yet to be explored
Diversity is good!

5. The ET pilot
”ET could also be a good complement when debugging or investigating failures in the platform. Now when we know the tools it should be quite easy to set up and explore the platform.”
”One shouldn’t forget that we are primarily test developers with diverse knowledge in different fields such as Signal Processing and programming.”
”Good with the scheduling, otherwise it's easy to stop after i.e. 30 or 60 min”
”It's a little bit hard to find good missions.” ”The testers really liked this way of working. They felt like it was really productive when actually performing test cases without writing extensive documentation beforehand.”
”When workiing in a more agile way, we have to look at ET and actually think about what kind of documentation we need and what we can skip doing.”

6. Statistics from One Pilot
Errors found during ET
•4 resources for 1 day
•The purpose was to more extensively test areas with poor test coverage, stress the system and to find new bugs.
Total: 3-4 platform ERs were found. These ERs wouldn't been found by the regression TCs. 6 issues to be further investigated
Errors found by regression test
•Approx four (4) times more time spent then during ET
•The purpose of the regression test was to see if new labels are stable and to find degrades. Total: 3 platform ERs were found
Disclamer: It is not possible to make any conclusions based on this limited number of data.
Same application of the Sw
Same build/sw version

7. The Decision

8. Schedule
•Day 1
–First ET meetingwith ETC, go through the schedule
–One ET session
–Debriefing
–Preparation
•Day 2
–WorkshopDesigning the Process –Output draft of the ET process and reviewed templates for test design
–Mini Workshopsession discipline –Output Session guidelines
–One ET sessionwith the defined process
–Debriefing
•Day 3
–Workshop Heuristics Bug taxonomy -Output checklists
–Two ET sessionsusing heuristics and Test design techniques
–Debriefing
•Day 4
–Workshop, Test environment –Output list on what changes needs to be done to the test tools and which areas can the tool be used for as is.
–One ET session
–Debriefing
–Mini workshop–input to testing
–Summary discussion
•Day 5
–Hand over
–Department meeting
–Close down

9. Output
•Hands on experience of Exploratory testing
•A process
•Check list -Exploratory Test Session
•Input to testing
•Tool Improvements
•Real examples of ET Session reports
•Session Report Template

10. Exploratory Testing Champions
The ETC will spend one day/week within the projects.
The ETC will coach, tutor and share their knowledge about ET, test techniques, tools and testing strategies to everyone in their teams.
The ETC will drive, develop, build and explore the craftsmanship of testing. Everyone in the team is allowed to and are encouraged spend time together with the advisor and take usage of his knowledge.
The ETCs will meet up every second week to exchange experience, discuss issues and try out new thoughts.

11. About the process
•Suggestion is to run two (2) test sessions every 2nd week.
•Each session about 90 minute long.
•Setup and preparation shall have been done before the session ( Check list - Exploratory Test Session)
•Alter testing alone and testing two and two (pair testing)
•To start with only reporting done is session reports and defects.We do not use any tool or metrics to track the sessions.
•Have a flexible approach to debriefings. Can be performed one on one, in group, after every session, after a day of sessions, tester to tester or tester to test manager. Important is that this does not become routine and the value from it decreases: That is one reason for variation.
•Use a wikki accessible for the whole team to enter mission ideas
•Common folder for each team to save session reports
•Common folder for everyone to save test scripts the helps us to get to a certain state quick.

12. What was left to do?

13. Where are they today?

14. Thank you for your attention!