Automate Exploratory Testing

W18

Special
Topics

10/5/16
15:00

It's
Time
to
Automate
Your
Exploratory

Testing

Presented
by:

Jim
Trentadue

Original
Software

Brought
to
you
by:

350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073

888-‐-‐-‐268-‐-‐-‐8770
·∙·∙
904-‐-‐-‐278-‐-‐-‐0524
-‐
info@techwell.com
-‐
http://www.starwest.techwell.com/

Jim
Trentadue

Jim
Trentadue
has
more
thanÊseventeenÊyears
of
experience
as
a

coordinator/manager
in
the
software
testing
field.
Jim's
various
career
roles
in

testing
have
focused
on
test
execution,
automation,
management,
environment

management,
standards
deployment,
and
test
tool
implementation.
In
the
area
of

offshore
testing,
Jim
has
worked
with
multiple
large
firms
to
develop
and
coordinate

cohesive
relationships.
As
a
guest
speaker
at
the
University
of
South
Florida's

software
testing
class,
Jim
mentors
students
on
the
testing
industry
and
trends
for

future
job
searches
and
continued
training.ÊMost
recently,
Jim
has
worked
at
testing

solutions
companies,
focusing
on
quality
management,
test
automation,
manual

testing,
UAT
and
test
data
management.

9/21/16
1
www.origsoft.com | © Original Software 1
Exploratory Test Automation
Jim Trentadue
Software Quality Consulting Director
Original Software
jtrentadue@origsoft.com
Agenda
2 What drives ET
3 Good automation practice with ET
4
5 Case Study
Enabling Test Automation for ET
What is ET1
Session Recap6

9/21/16
2
What is ET?
What is Exploratory Testing?
Only for the context of automation and this presentation
Exploratory Testing is simultaneous
learning, test design and test execution
- “Exploratory Testing Explained”, James Bach 2003
Simultaneously designing and executing tests
to learn about the system using your insights
from the last experiment to inform the next
- “Explore It!”, Elisabeth Hendrickson
Original Modified
↔ ↔
EvaluateEngineer

9/21/16
3
Structured vs. Unstructured?
What does ET look like?
Ad hoc or
Test Case?
Chartered or
Uncharted?
Bounded or
Unbounded?
Define the scope of Exploratory Testing
Exploratory Tester types
Deciding skills and experience of doing ET
Testing skill level
System skill level
IT Role
High Low
High
High
Low
Low
Experienced
vs.
New Testers?
App. SME
vs.
New to app.?
Developer
vs.
Testers?

9/21/16
4
What drives ET?
ET mission
Drivers for Exploratory Tests
ET
Risk:
Module,
App., etc.
Examine
App.
Behavior
Cause-
Effect
Break
Software!
Observe
Concern
Curiosity
Destruct

9/21/16
5
Mood for ET
Using ‘Six Thinking Hats’ as a reference
ET Release Planning
Highlighting the need for automation
Data
Navigation
Timing
Test Case 1
Facts:
▪ Executed 12 times
in release
▪ Module has been
modified in each
sprint (10)
▪ TC1 passes, but
related complex
tests are failing
Bolstering by
Automation
▪ Frequent & random
data injection
▪ Navigation paved
more paths
▪ Sporadic timeouts
on forms / pages are
captured precisely

9/21/16
6
Good Automation Practice with ET
Decision to automate
When should an exploratory test be automated?
Dina Hart, “A Casual Approach to Testing” on ministryoftesting.com

9/21/16
7
Exploratory Automation structure
Where to begin with automating these tests?
“TOURING TESTS”
‘Exploratory Software Testing’ by James Whitaker
Business
District
Historical
District
Entertainment
District
Tourist
District
Hotel District Seedy District
“CHARTER YOUR EXPLORATIONS”
Simple
Template
•  Explore (target), with (resources)
to discover (information)
Good
Charters
•  Direction without over specifying
test actions
Generating
Charters
•  Good time to draft charters is
during requirements discussions
Nightmare
Headline
Game
•  Inspiring from our worst fears
Planning
Charters
•  Chartering is more fluid than
typical test planning
Practice
Sessions
•  Draft charters for exploring a
software package
“Explore It!” by Elisabeth Hendrickson
ET Automated Test Types
Three types that can be covered with automation
Planned, but randomData
•  Field Data Injection, Data Reusability, Mapping Data Patterns
Objects identified, but sporadicNavigation
•  Back and Forth, Object State Transition, Object Touches
Events known, but timings unknownTimings
•  Varying Queries, Page Events (both fast and slow), Additional Instances

9/21/16
8
Enabling Test Automation for ET
Data
Bloodline of your application
Analyzing data
output; map to
existing or new
automated test
cases
Mapping
Leveraging this
for subsequent
transactions
Reuse
Random Data
injected in
page / form
fields, both
good and bad
Injection

9/21/16
9
Navigation
Touring the world…in your system
Random touch
on an object to
see what’s
available and
what isn’t
Touch
Determine an
object's state in
expected or
unexpected
transition
Transition
Back and forth
actions to see
if object / data
pattern emerge
Seesaw
Timings
How responsive is your design
Open another
instance of the
app on same
PC or same
server
Again
By executing
these, does the
performance
fluctuate on
other events?
Events
Run large
queries, small,
and something
in between
Queries

9/21/16
10
Case Study
CS - Exploratory Test Automation
Introduction to Model-Based Automation
Exploratory Test Automation in ‘Experiences of Test Automation’ by Graham & Fewster
Background
Testing each critical scenario in a
Trouble Manager system for a major
telecommunications company seemed
impossible & harder yet was when
certain conditions would appear
Solution
Implement a Model-Based Automation
approach to each major function
Trouble Manager Functions
Create ticket to track problem
Run electrical test on phone line
Diagnose the problem
Fix the problem
Notify customer the issue is fixed
Close the ticket

9/21/16
11
CS – Defining Workflows
Functions that flow through the system
Start
Created
Basic Workflow
CreateTicket
Created
AnalyzeTicket
Diagnosed
DiagnoseTicket
Repaired
Closed
RepairTicket CloseTicket
Improved Workflow
AnalyzeTicket
CloseTicket
AnalyzeTicket
Original Path
1.  CreateTicket
2.  AnalyzeTicket
3.  Diagnose Ticket
4.  RepairTicket
5.  CloseTicket
Second Path
1.  CreateTicket
2.  AnalyzeTicket
3.  AnalyzeTicket
5.  RepairTicket
6.  CloseTicket
Third Path
1.  CreateTicket
2.  AnalyzeTicket
4.  CloseTicket
Fourth Path
1.  CreateTicket
2.  AnalyzeTicket
4.  RepairTicket
5.  AnalyzeTicket
6.  DiagnoseTicket
7.  RepairTicket
8.  CloseTicket
Exploratory Test Automation in ‘Experiences of Test Automation’ by Graham & Fewster
CS – Key Points
Summarizing key points in this case study
"   Partial knowledge of a test result if automated, can enable for more
tests to be run and checked automatically
"   Contain work within the module for what can possibly be done
"   Draw different logical ‘models’ of what can happen in the application
"   Automate the tour of the modelled paths through your system

9/21/16
12
Session Recap
Presentation Recap
"   Define Exploratory Testing only for the context of your automation
"   Understand the drivers for ET in your testing organization
"   Review your structure and types for Automated ET to decide appropriately
"   Create your automated ET ‘tour’ in three areas: Data, Navigation & Timings
"   Reference this Case Study or others for practical examples
Key takeaways for Automating Exploratory Tests

9/21/16
13
Thank you for attending
this presentation!
Jim Trentadue
Software Quality Consulting Director
Original Software
jtrentadue@origsoft.com

Automate Exploratory Testing

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (20)

Similar to Automate Exploratory Testing

Similar to Automate Exploratory Testing (20)

More from TechWell

More from TechWell (20)

Recently uploaded

Recently uploaded (20)

Automate Exploratory Testing