A new generation of testing is emerging, led by testers who believe test automation is an inseparable part of how we do exploratory testing. Yes, you read that right: exploratory testing with test automation, and on unit, api, ui and e2e levels all at our reach with team collaboration. We work on acceptance criteria and examples in our short increments, yet people are at their most creative on what could go wrong when they use the product or a piece of the product as their external imagination. When we’re creative, we need to do things we can’t do by hand, using programming to extend our reach. When we’re learning, we need to take notes now and to keep for later, again using programming to replenish information we deem worthwhile. We want to work on exploring a category of bugs before implementing, and a category of bugs before releasing, and a category of bugs after releasing. Last year I worked with a team that was transforming from a traditional manual test case + test automation -based approach to contemporary exploratory testing. This year I work with another team that takes this style of testing further than we’ve taken it before with whole-team testing. Let’s learn from a practitioner’s examples of ideas leading into this, changes enabling this, and results we are seeing with contemporary exploratory testing.

1. @maaretp
A Practitioner’s Guide to
Contemporary Exploratory
Testing
by Maaret Pyhäjärvi

2. @maaretp
ideas leading into this,
changes enabling this, and
results we are seeing
with contemporary
exploratory testing

3. @maaretp
1988
style of skilled multidisciplinary testing common in SiliconValley
* Spotted live in Finland 1973 (Kela, the Social Insurance Institution in Finland)

4. @maaretp
Exploratory Testing
TEST DESIGN
TEST
EXECUTION
LEARNING,
AGENCY
IMPACTFUL RESULTS
* power to decide with knowledge
and attitude that goes with power

5. @maaretp
Tier boundary testing
Cross-tenant impact testing
API throttling testing
* From Jyothi Mishra’s Cloud Testing Talk today

6. @maaretp
Contemporary
Exploratory Testing
MANUAL AUTOMATED
LEARNING,
AGENCY
IMPACTFUL RESULTS
* power to decide with knowledge
and attitude that goes with power

7. @maaretp

8. @maaretp
ATTENDED
UNATTENDED
* You can't automate well without exploring. You can't explore well without automating.

9. @maaretp
Time used on warning about
test automation is away from
succeeding with it.
* opportunity cost

10. @maaretp

11. @maaretp
before implementing
before releasing
after releasing
a category of bugs

12. @maaretp
Finding and Sharing New Information Matters
Investment Awareness
Time Dimensions: Now vs. Future
From Reactive to Just-in-Time Reactive
Learn to Learn *really well*
Grow as Testers

13. @maaretp
Good Neighbor
-principle
Know
more than
your team

14. @maaretp
Continuous Releases: Shorter Increments over Session
Based Test Management
Co-Design Workshops over Status Meetings
Pair Testing over Reviewing Test Ideas
Regular Review of Investment and Results
Protect Agency

15. @maaretp
Evidence
as Automation
*You know it is a small part but sufficient for evidence
Automation over
Mindmaps over
Title test cases over
Test cases with steps

16. @maaretp
Visualized with gource.io

17. @maaretp
Perceived Success Factors
Dimension Factor
Human Whole team effort
Expert team members
Self-motivated team members
Organizing Allow time for learning curve
Internal open-source community mindset
Technical Test tool choice and architecture
Testlab infrastructure
Product testability
Telemetry
Process Incremental approach
Process observation and optimization

18. @maaretp
Conflict with previous research!
• No explicit test automation strategy
• No careful tool selection
• No measuring quality and performance of TA
• No explicit guidelines
“…relaxed, verbally communicated strategy
without strict rules and processes relying on
developers’ voluntary participation.We
consider this an unusual success with TAPI …
https://www.academia.edu/49495368/Test_Automation_Process_Improvement_in_a_DevOps_Team_Experience_Report

19. @maaretp
Since then…
WB
(fail)
DMU
(success)
AMS
(success)
AVMT
(inconclusive)
SND
(fail)
EPT
(inconclusive)
NM
(inconclusive)
PSU
(fail)
FIR
(fail)
Success /
Failure is a
Snapshot
VSP
(success)
ENTROPHY
INERTIA

20. @maaretp

21. @maaretp
(Exploratory)
Tester
(Application)
Programmer
Test Automator
/ Programmer
Testing is too important to be left for only testers!

22. @maaretp
Everyone
can test

23. @maaretp
Even a broken clock is right twice a day.

24. @maaretp
Find (some of) what
others may have missed!
Testing
… and never be bored.

25. @maaretp
2020 2016
MIATPP
Most Influential Agile Testing
Professional Person
#PayToSpeak #TechVoices
#EnsembleTesting #EnsembleProgramming #StrongStylePairing
#ExploratoryTesting #TestAutomation
#ModernAgile
#AwesomeTesters
Maaret Pyhäjärvi (from Finland)
Email: maaret@iki.fi
Twitter: @maaretp
Web: maaretp.com
Blog: visible-quality.blogspot.fi
(please connect with me through
Twitter or LinkedIn)
2019, 2020, 2021
https://exploratorytestingacademy.com
Ohjelmistotestaus ry
https://techvoices.org