The document discusses exploratory testing with scripted support. It describes a common agile development model and suggests that testing preparation and execution can occur during feature investigation, planning, and design phases. Exploratory testing requires elements like domain knowledge, test environment understanding, rapid learning, and communication. The document warns against common misconceptions that exploratory testing means less documentation, structure, and traceability when in fact it requires more responsibility and communication from skilled testers.

1. (Exploratory) Testing
With Scripted Support
Experience & Considerations
Simon Morley October 2014
Saturday 25 October 14

2. My Situation Context
• Complex Environment
• Meaning overhead & set-up costs
• Meaning preparation, tooling or
ordering
• Many signaling protocols
• Meaning script support for signal
injection and check
• Agile & Incremental SW Development
Saturday 25 October 14

3. Testing in Agile
• A common “agile” development model:
• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Saturday 25 October 14

4. Testing in Agile
• A common “agile” development model:
• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep &
execution happens
Saturday 25 October 14

5. Testing in Agile
• A common “agile” development model:
• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep &
execution happens
Saturday 25 October 14

6. Testing in Agile
• A common “agile” development model:
• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep &
execution happens
Saturday 25 October 14

7. Testing in Agile
• A common “agile” development model:
• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep &
execution happens
Saturday 25 October 14

8. Testing in Agile
• A common “agile” development model:
• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep &
execution happens
Saturday 25 October 14

9. Testing in Agile
• A common “agile” development model:
• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep &
execution happens
Saturday 25 October 14

10. Testing in Agile
• A common “agile” development model:
• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep &
execution happens
Saturday 25 October 14

11. Testing before “testing”
• Testing in the “Opportunity Analysis”,
“Planning” & “Design” “phases” might be:
• Feature Investigation - from a testing
perspective
• Testable?
• Creates issues with existing framework,
tools and maintenance?
• Feature Walkthrough
Saturday 25 October 14

12. Possible Testing Approach
• Investigate the new feature
development
• Develop framework and tools, develop
scripts, comms, test scripts &
framework & feature, comms, debug
scripts, comms, debug environment,
comms, test feature, comms, report
completion, archiving.
Saturday 25 October 14

13. Feature Investigation
• Feature Walkthrough
• Test environment impacts &
walkthrough
• Test Analysis summary
Saturday 25 October 14

14. Test Analysis Summary
• Describe* or reference the current
feature, test environment and
framework situation
• Describe* impacts to test environment
and framework
• Describe* approach
• Include coverage goals for product,
feature and testing
Saturday 25 October 14

15. Comms?
• Communication
• Can be: Verbal & Written
• Can be: Formal & Informal
• Always: Two-way!
• Reporting - stand-ups, progress
boards, meetings
• Reflect & decide about next steps
Saturday 25 October 14

16. Reflect & Decide?
• What has the testing so far revealed?
• Product coverage goals met?
• Testing coverage goals met?
• Perceived risk acceptable?
• Stakeholder view on these?
Saturday 25 October 14

17. Reflect & Decide #2
• Next steps?
• Additional test steps?
• Enough already?
• Change direction or approach?
Saturday 25 October 14

18. Reflect & Decide #3
• Pause & Reflect
• Control Q: “What don’t we know?”
• Silent Evidence
• Note: This pause & reflection is
important - see thinking about heuristic
discovery.
Saturday 25 October 14

19. Necessary support?
• Can the tools & framework support
changes?
• Is the tool & framework architecture
modular?
• Remember to test the tool architecture!!!
• Do the tools & framework allow quick
prototyping?
• If not -> fix it!
Saturday 25 October 14

20. Describe*
• Describe: Meaning the level sufficient in
the team or project.
• If coordination is needed between
teams this should be a format that is
commonly understood (picture/
symbols, dashboard or text
document).
• You agree what’s appropriate!
Saturday 25 October 14

21. Exploratory Testing Traps
• Sometimes people think of ET means
less:-
• Documentation
• Structure
• Traceability
• And more
• Random banging on a keyboard
Saturday 25 October 14

22. Exploratory Testing Traps
• Sometimes people think of ET means
less:-
• Documentation
• Structure
Wrong !!!
• Traceability
• And more
• Random banging on a keyboard
Saturday 25 October 14

23. ET Traps #2
• Actually ET is about more
responsibility in skilled testing
• The skilled tester is responsible for
communicating his/her approach,
thinking, results and any mandated
archiving
Saturday 25 October 14

24. ET Traps #2
• Actually ET is about more
responsibility in skilled testing
• The skilled tester is responsible for
communicating his/her approach,
thinking, results and any mandated
archiving
I.E. Plenty of structure,
traceability & documentation
where appropriate!
Saturday 25 October 14

25. ET Traps #3
• No support for skilled testers and
testing.
• Support for skilled testing is a pre-requisite
(hand-in-hand) to realizing
emphasized tester responsibility.
Saturday 25 October 14

26. ET Traps #3
• No support for skilled testers and
testing.
• Support for skilled testing is a pre-requisite
(hand-in-hand) to realizing
emphasized tester responsibility.
Fix these if you want good &
valuable testing!
Saturday 25 October 14

27. Discussion
• Exploratory testing can be performed
with scripting support
• Good testing (in complex
environments) requires elements of:
• Domain knowledge - built through
early investigation
• Test environment knowledge &
understanding
Saturday 25 October 14

28. Discussion #2
• Plus
• Rapid learning
• System and problem analysis
• Decision making
• Framing, Consensus &
Commumnication
Saturday 25 October 14

29. References
• Experience Report: Introducing Exploratory
Testing
• Deliberated Exploration - A Walk in the Woods
• Experiences with Semi-Scripted Exploratory
Testing
• Framing: Some Decision Analysis and Frames in
Testing
• Silent Evidence in Testing
• On Thinking about Heuristic Discovery
Saturday 25 October 14

30. Thanks!
• Simon Morley
• Twitter: @YorkyAbroad
• Blog: testers-headache.blogspot.com
• Questions?
• Get in touch!
Saturday 25 October 14

31. Feature Walkthrough
Saturday 25 October 14