SlideShare a Scribd company logo
1 of 28
Today’s webinar is presented by Michael Kelly. He will discuss
Tips for Writing Better Charters for Exploratory Testing
Sessions.
Welcome to the EuroSTAR Webinar
Series
www.eurostarconferences.com
This webinar is due to start at 2pm. Make sure you stick around at the end for the
Q&A session and continue the conversation with the speaker on Twitter after the
show!
@esconfs
#esconfs
Tips for Writing Better Charters for
Exploratory Testing Sessions
Michael Kelly, DeveloperTown
Mike Kelly is a managing partner at DeveloperTown, a
venture development firm. Mike is a contributing author to
“How to Reduce the Cost of Software Testing”, is a past
director and president for the Association for Software
Testing, and a co-founder of the Indianapolis Workshops on
Software Testing, a series of ongoing meetings on topics in
software testing.
www.eurostarconferences.com
Join the conversation on Twitter
#esconfs
@esconfs
@michael_d_kelly
www.eurostarconferences.com
Continue the conversation with the speaker on Twitter after the show!
MICHAEL KELLY
Managing Partner, DeveloperTown
www.DeveloperTown.com
Past President, Association for Software Testing
www.AssociationForSoftwareTesting.org
Articles and Blogs
www.MichaelDKelly.com
www.eurostarconferences.com
@esconfs
#esconfs
“The tester, the way they work,
and their ability to work in the
future, is as important as the
product being tested.”
www.eurostarconferences.com
@esconfs
#esconfs
“One of the most difficult aspects of
software testing is coming up with good
test ideas. It doesn't matter how you're
doing your testing: scripted vs.
exploratory, manual vs. automated, or
performance vs. functional.”
EXPLORATORY TESTING
“Simultaneous learning, test design, and test execution.” – James Bach
www.eurostarconferences.com
@esconfs
#esconfs
www.eurostarconferences.com
@esconfs
#esconfs
pure scripted freestyle exploratory
chartersvague scripts
fragmentary
test cases
(scenarios)
roles
-image from Jon Bach
SESSION BASED TEST
MANAGEMENT
A method for managing exploratory testing. Created by Jon and James
Bach, refined by others.
www.eurostarconferences.com
@esconfs
#esconfs
• charters
• time-boxed
• session notes
• debriefs
• team prioritization
• ad-hoc test documentation
• ad-hoc test automation
• dynamic metrics and reporting
www.eurostarconferences.com
@esconfs
#esconfs
CHARTERING
Making your own decisions about what you will work on and how you
will work. Understanding your client’s needs, the problems you must
solve, and assuring that your work is on target.
www.eurostarconferences.com
@esconfs
#esconfs
What the tester thinks
• What am I suppose to be
doing?
• I need more time!
• This is boring…
• What data should I be using
for this?
• Didn’t someone already test
this?
• Is this really the most
important thing for me to be
focused on?
What happens on the project
• Areas of the product don’t
get covered.
• Entire categories of risk
don’t get tested.
• Duplication of effort.
• The team does not work
based on priority.
• Testers get frustrated.
• Poor visibility into actual
progress.
www.eurostarconferences.com
@esconfs
#esconfs
BAD CHARTERING
ESSENTIAL ELEMENTS OF A
CHARTER
www.eurostarconferences.com
@esconfs
#esconfs
Risk, Coverage, and Timeframe
COVERAGE
www.eurostarconferences.com
@esconfs
#esconfs
What am I testing? Which features, screens, artifacts, or areas of
the product should I be touching while testing?
RISK
Why am I running this test? What could go wrong? What am I
looking for?
TIMEFRAME
How long should it take to perform this testing? Are there setup tasks I
should do before I officially start? What do I need to do when I’m done?
When you look at a list of 10 charters,
you should see 10 distinct testing missions,
equating to around eight to 10 hours
of heads-down testing.
www.eurostarconferences.com
@esconfs
#esconfs
BETTER CHARTERING
www.eurostarconferences.com
@esconfs
#esconfs
List specific risks and coverage targets
Leverage mnemonics for risk and coverage ideas
Develop a knowledge base for risk/coverage summaries
Compare missions
Try developing a couple of templates
Charter for smaller sessions, then affinity map to create larger sessions
Thumb vote for priority
Use testing polarities
Let charters emerge over time
Track metrics
LIST SPECIFIC RISK AND
COVERAGE TARGETS
Test the portal for reporting accuracy.
Which reports?
What do you mean by accuracy?
Test reports X, Y, and Z for errors related to start
and end time selection criteria, summing,
totaling, and rounding.
www.eurostarconferences.com
@esconfs
#esconfs
MNEMONICS FOR RISK
AND COVERAGE IDEAS
• Application Touring: FFC CUTS VIDS
• Coverage: SFDPOT
• Coverage and Risk: FIBLOTS
• Quality Criteria: CRUSSPICSTMPL
• Test Techniques: FDSFSCURA
• Oracles: HICCUPPS
• Session Notes: MCOASTER
www.eurostarconferences.com
@esconfs
#esconfs
RISK AND COVERAGE
KNOWLEDGE BASE
As a team, build out a wiki detailing the nuance of
various coverage areas and specific areas of risk.
Commonly captured items will include:
• Detailed feature listings/checklists
• Integration points and data flow
• Example and common test data / test beds
• Known issues/gotchas
• Application-specific or feature-specific test
techniques
www.eurostarconferences.com
@esconfs
#esconfs
COMPARE
MISSIONS
• Test feature X
• Stress test feature X
• Performance test feature X
What’s the difference?
www.eurostarconferences.com
@esconfs
#esconfs
My mission is to test
<insert risk here> for
<insert coverage here>
Some examples:
• My mission is to test for
various boundary errors for
Microsoft Word’s bullets and
numbering feature.
• My mission is to test for
accurate error messaging
pop-ups for Ford Motor
Vehicle's Build and Price
website.
• My mission is to test for SQL
injection vulnerabilities for
application login and
administration screens.
www.eurostarconferences.com
@esconfs
#esconfs
CHARTER
TEMPLATE
KiwanisInternationalApp
(possible10to15mincharters)
1. Test for boundary related issues related to projects
2. Test for boundary related issues related to project reports
3. Test for deliverability related issues related to project emails
4. Test for data quality issues with clubs available on the app
5. Test for data accuracy issues related to calculations on project reports
6. Test for data accuracy issues related to calculations on service and
fundraising summary stats
7. Test for data accuracy issues related to Piggy Bank commitment
calculations
8. Perform basic link validation for items in the news feed
9. Perform a copy review of the various descriptive dialogs in the app
looking for typos and grammar issues
10. Test for performance related issues related to mapping capabilities
11. Stress test the application to identify conditions that crash the app
www.eurostarconferences.com
@esconfs
#esconfs
CHARTER FOR
SMALLER SESSIONS
Walk your charters quickly as a team
Use thumb voting to see if there is
agreement around clarity and scope
The particularly contentious charters
will lead the team to discussion and
debate (and new charters)
Thumb Vote 101
High Priority
“We need to run this
charter.”
Medium Priority
“If we have time, we
should run this
charter.”
Low Priority
“It’s a test, and we
could run it, but likely
there are better uses
of our time.”
www.eurostarconferences.com
@esconfs
#esconfs
THUMB
VOTE
TESTING
POLARITIES
Example Polarities:
• Warming up vs. cruising vs. cooling down
• Doing vs. describing
• Careful vs. quick
• Data gathering vs. data analysis
• Working with the product vs. reading about the product
• Working with the product vs. working with the developer
• Training (or learning) vs. performing
• Product focus vs. project focus
• Solo work vs. team effort
• Your ideas vs. other peoples’ ideas
• Lab conditions vs. field conditions
• Current version vs. old versions
• Feature vs. feature
• Requirement vs. requirement
• Coverage vs. oracles
• Testing vs. touring
• Individual tests vs. general lab procedures and infrastructure
• Testing vs. resting
• Playful vs. serious
The Process:
• When you charter your tests, include
polarities explicitly in your mission.
• This practice is particularly effective for
shorter test charters (20-30 minutes).
• Or, when you charter, use the polarities to
help you identify new charter ideas
www.eurostarconferences.com
@esconfs
#esconfs
Source: http://www.satisfice.com/blog/wp-
content/uploads/2009/10/et-dynamics22.pdf
IN YOUR CHARTER:
• Testing vs. Touring
• Feature vs. Feature
• Requirement vs. Requirement
WHILE CHARTERING:
Individual Tests vs. General Lab Procedures
Coverage vs. Oracles
Lab Conditions vs. Field Conditions
LET CHARTER
EMERGE OVER TIME
www.eurostarconferences.com
@esconfs
#esconfs
1. Pull down a
charter for testing
2. Execute that
charter
3. Debrief
4. After your
debrief, add new
charters and
reprioritize
existing charters
5. Repeat
Time management is critical to chartering.
Metrics help provide visibility into what’s
really happening. Tagging allows for easier
metrics gathering.
I regularly tag the following:
• bugs or issues for isolation (b)
• possible new charters (c)
• open questions (?)
• tasks for automation (a)
• tasks for documentation (doc)
• setup, testing, investigation #...
Some examples:
• Actual execution time vs.
estimated time
• Confidence factor of tester
on their coverage of the
stated risks
• Number of follow up
charters resulting from lack
of time
• Percentage of session time
spent setting up for testing
• Percentage of session time
spent testing
• Percentage of session time
spent investigating
problems
www.eurostarconferences.com
@esconfs
#esconfs
TRACK
METRICS
BETTER CHARTERING
www.eurostarconferences.com
@esconfs
#esconfs
List specific risks and coverage targets
Leverage mnemonics for risk and coverage ideas
Develop a knowledge base for risk/coverage summaries
Compare missions
Try developing a couple of templates
Charter for smaller sessions, then affinity map to create larger sessions
Thumb vote for priority
Use testing polarities
Let charters emerge over time
Track metrics
EuroSTAR Webinar Archive
Access almost 50 on-demand software testing webinars by the world's leading
testing experts. Topics include….
www.eurostarconferences.com
Agile, Automation, Test Management, Process Improvements, Risk-based Testing, Performance
Testing, Test Documentation, Cloud Testing, Session-based Testing, Weekend Testers
Visit the archive!
http://www.eurostarconferences.com/community/member/webinar-archive
@esconfs
#esconfs
The EuroSTAR Community.
Are you a member?
Did you know we have an online resource area with almost …
www.eurostarconferences.com
600 software testing presentations, 60 topical
webinars, over 40 videos and podcasts and 20 expert
eBooks
Do you know that membership is completely free and you can access these
premium testing resources anytime?
Join today!
http://www.eurostarconferences.com/community/
member
@esconfs
#esconfs

More Related Content

What's hot

Basic software-testing-concepts
Basic software-testing-conceptsBasic software-testing-concepts
Basic software-testing-concepts
medsherb
 
powerpoint template for testing training
powerpoint template for testing trainingpowerpoint template for testing training
powerpoint template for testing training
John Roddy
 
Quality Assurance and Software Testing
Quality Assurance and Software TestingQuality Assurance and Software Testing
Quality Assurance and Software Testing
pingkapil
 
Testing concepts ppt
Testing concepts pptTesting concepts ppt
Testing concepts ppt
Rathna Priya
 

What's hot (20)

A Taste of Exploratory Testing
A Taste of Exploratory TestingA Taste of Exploratory Testing
A Taste of Exploratory Testing
 
Basic software-testing-concepts
Basic software-testing-conceptsBasic software-testing-concepts
Basic software-testing-concepts
 
Exploratory test
Exploratory testExploratory test
Exploratory test
 
ISTQB - What's testing
ISTQB - What's testingISTQB - What's testing
ISTQB - What's testing
 
Exploratory Testing Explained
Exploratory Testing ExplainedExploratory Testing Explained
Exploratory Testing Explained
 
Risk-based Testing
Risk-based TestingRisk-based Testing
Risk-based Testing
 
QA process Presentation
QA process PresentationQA process Presentation
QA process Presentation
 
What is this exploratory testing thing
What is this exploratory testing thingWhat is this exploratory testing thing
What is this exploratory testing thing
 
Agile testing - Testing From Day 1
Agile testing - Testing From Day 1Agile testing - Testing From Day 1
Agile testing - Testing From Day 1
 
Agile Quality and Risk Management
Agile Quality and Risk ManagementAgile Quality and Risk Management
Agile Quality and Risk Management
 
powerpoint template for testing training
powerpoint template for testing trainingpowerpoint template for testing training
powerpoint template for testing training
 
Performance testing : An Overview
Performance testing : An OverviewPerformance testing : An Overview
Performance testing : An Overview
 
Test cases
Test casesTest cases
Test cases
 
Software Testing Life Cycle – A Beginner’s Guide
Software Testing Life Cycle – A Beginner’s GuideSoftware Testing Life Cycle – A Beginner’s Guide
Software Testing Life Cycle – A Beginner’s Guide
 
Test cases and bug report v3.2
Test cases and bug report v3.2Test cases and bug report v3.2
Test cases and bug report v3.2
 
QA Best Practices in Agile World_new
QA Best Practices in Agile World_newQA Best Practices in Agile World_new
QA Best Practices in Agile World_new
 
Manual software-testing-interview-questions-with-answers
Manual software-testing-interview-questions-with-answersManual software-testing-interview-questions-with-answers
Manual software-testing-interview-questions-with-answers
 
Quality Assurance and Software Testing
Quality Assurance and Software TestingQuality Assurance and Software Testing
Quality Assurance and Software Testing
 
Exploratory Testing in Practice
Exploratory Testing in PracticeExploratory Testing in Practice
Exploratory Testing in Practice
 
Testing concepts ppt
Testing concepts pptTesting concepts ppt
Testing concepts ppt
 

Viewers also liked

Standardized risks & charters in exploratory testing
Standardized risks & charters in exploratory testingStandardized risks & charters in exploratory testing
Standardized risks & charters in exploratory testing
Johan Hoberg
 

Viewers also liked (13)

Qa exploratory test charter template
Qa exploratory test charter templateQa exploratory test charter template
Qa exploratory test charter template
 
Exploratory Testing Explained
Exploratory Testing ExplainedExploratory Testing Explained
Exploratory Testing Explained
 
Fear the psychology of testing rik marselis
Fear the psychology of testing rik marselisFear the psychology of testing rik marselis
Fear the psychology of testing rik marselis
 
Test Lead of my Dreams
Test Lead of my Dreams   Test Lead of my Dreams
Test Lead of my Dreams
 
Standardized risks & charters in exploratory testing
Standardized risks & charters in exploratory testingStandardized risks & charters in exploratory testing
Standardized risks & charters in exploratory testing
 
STARWest 2013 Courage and Freedom in Exploratory Testing
STARWest 2013 Courage and Freedom in Exploratory TestingSTARWest 2013 Courage and Freedom in Exploratory Testing
STARWest 2013 Courage and Freedom in Exploratory Testing
 
Exploratory Testing Explained (Tampere Goes Agile - 2013)
Exploratory Testing Explained (Tampere Goes Agile - 2013)Exploratory Testing Explained (Tampere Goes Agile - 2013)
Exploratory Testing Explained (Tampere Goes Agile - 2013)
 
Santa Barbara Agile: Exploratory Testing Explained and Experienced
Santa Barbara Agile: Exploratory Testing Explained and ExperiencedSanta Barbara Agile: Exploratory Testing Explained and Experienced
Santa Barbara Agile: Exploratory Testing Explained and Experienced
 
Test Heuristics for Exploratory Testing
Test Heuristics for Exploratory TestingTest Heuristics for Exploratory Testing
Test Heuristics for Exploratory Testing
 
Exploratory testing and the mobile tester : A presentation by Jon Hagar
Exploratory testing and the mobile tester : A presentation by Jon HagarExploratory testing and the mobile tester : A presentation by Jon Hagar
Exploratory testing and the mobile tester : A presentation by Jon Hagar
 
Test design made easy (and fun) Rik Marselis EuroSTAR
Test design made easy (and fun) Rik Marselis EuroSTARTest design made easy (and fun) Rik Marselis EuroSTAR
Test design made easy (and fun) Rik Marselis EuroSTAR
 
Exploratory testing in practice, short story how approach influenced on strategy
Exploratory testing in practice, short story how approach influenced on strategyExploratory testing in practice, short story how approach influenced on strategy
Exploratory testing in practice, short story how approach influenced on strategy
 
Exploratory Session Based Testing…With A Twist Star East 2009
Exploratory Session Based Testing…With A Twist   Star East 2009Exploratory Session Based Testing…With A Twist   Star East 2009
Exploratory Session Based Testing…With A Twist Star East 2009
 

Similar to Tips for Writing Better Charters for Exploratory Testing Sessions by Michael D Kelly

"ISO 29119 - The New Set of International Standards on Software Testing" with...
"ISO 29119 - The New Set of International Standards on Software Testing" with..."ISO 29119 - The New Set of International Standards on Software Testing" with...
"ISO 29119 - The New Set of International Standards on Software Testing" with...
TEST Huddle
 
Extreme Programming Talk Wise Consulting Www.Talkwiseconsulting
Extreme  Programming    Talk Wise  Consulting   Www.TalkwiseconsultingExtreme  Programming    Talk Wise  Consulting   Www.Talkwiseconsulting
Extreme Programming Talk Wise Consulting Www.Talkwiseconsulting
talkwiseone
 
Extreme programming talk wise consulting - www.talkwiseconsulting
Extreme programming   talk wise consulting - www.talkwiseconsultingExtreme programming   talk wise consulting - www.talkwiseconsulting
Extreme programming talk wise consulting - www.talkwiseconsulting
talkwiseone
 
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity Software Ireland
 

Similar to Tips for Writing Better Charters for Exploratory Testing Sessions by Michael D Kelly (20)

TLC2018 Thomas Haver: The Automation Firehose - Be Strategic and Tactical
TLC2018 Thomas Haver: The Automation Firehose - Be Strategic and TacticalTLC2018 Thomas Haver: The Automation Firehose - Be Strategic and Tactical
TLC2018 Thomas Haver: The Automation Firehose - Be Strategic and Tactical
 
The Automation Firehose: Be Strategic & Tactical With Your Mobile & Web Testing
The Automation Firehose: Be Strategic & Tactical With Your Mobile & Web TestingThe Automation Firehose: Be Strategic & Tactical With Your Mobile & Web Testing
The Automation Firehose: Be Strategic & Tactical With Your Mobile & Web Testing
 
Continuous Testing - The New Normal
Continuous Testing - The New NormalContinuous Testing - The New Normal
Continuous Testing - The New Normal
 
Solving the 3 Biggest Questions in Continuous Testing
Solving the 3 Biggest Questions in Continuous TestingSolving the 3 Biggest Questions in Continuous Testing
Solving the 3 Biggest Questions in Continuous Testing
 
The Automation Firehose: Be Strategic and Tactical by Thomas Haver
The Automation Firehose: Be Strategic and Tactical by Thomas HaverThe Automation Firehose: Be Strategic and Tactical by Thomas Haver
The Automation Firehose: Be Strategic and Tactical by Thomas Haver
 
Software testing
Software testingSoftware testing
Software testing
 
How to Guarantee Continuous Value from your Test Automation
How to Guarantee Continuous Value from your Test AutomationHow to Guarantee Continuous Value from your Test Automation
How to Guarantee Continuous Value from your Test Automation
 
So You Think You Can Write a Test Case - XBOSoft Webinar
So You Think You Can Write a Test Case - XBOSoft WebinarSo You Think You Can Write a Test Case - XBOSoft Webinar
So You Think You Can Write a Test Case - XBOSoft Webinar
 
Beyond "Quality Assurance"
Beyond "Quality Assurance"Beyond "Quality Assurance"
Beyond "Quality Assurance"
 
Software Development Lifecycle Presentation
Software Development Lifecycle PresentationSoftware Development Lifecycle Presentation
Software Development Lifecycle Presentation
 
Agile testing
Agile testingAgile testing
Agile testing
 
Exploratory Testing Explained
Exploratory Testing ExplainedExploratory Testing Explained
Exploratory Testing Explained
 
Context Driven Automation Gtac 2008
Context Driven Automation Gtac 2008Context Driven Automation Gtac 2008
Context Driven Automation Gtac 2008
 
"ISO 29119 - The New Set of International Standards on Software Testing" with...
"ISO 29119 - The New Set of International Standards on Software Testing" with..."ISO 29119 - The New Set of International Standards on Software Testing" with...
"ISO 29119 - The New Set of International Standards on Software Testing" with...
 
Extreme Programming Talk Wise Consulting Www.Talkwiseconsulting
Extreme  Programming    Talk Wise  Consulting   Www.TalkwiseconsultingExtreme  Programming    Talk Wise  Consulting   Www.Talkwiseconsulting
Extreme Programming Talk Wise Consulting Www.Talkwiseconsulting
 
Extreme programming talk wise consulting - www.talkwiseconsulting
Extreme programming   talk wise consulting - www.talkwiseconsultingExtreme programming   talk wise consulting - www.talkwiseconsulting
Extreme programming talk wise consulting - www.talkwiseconsulting
 
A Method for Evaluating End-User Development Technologies
A Method for Evaluating End-User Development TechnologiesA Method for Evaluating End-User Development Technologies
A Method for Evaluating End-User Development Technologies
 
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
 
How to scale your Test Automation
How to scale your Test AutomationHow to scale your Test Automation
How to scale your Test Automation
 
Cloud and Network Transformation using DevOps methodology : Cisco Live 2015
Cloud and Network Transformation using DevOps methodology : Cisco Live 2015Cloud and Network Transformation using DevOps methodology : Cisco Live 2015
Cloud and Network Transformation using DevOps methodology : Cisco Live 2015
 

More from TEST Huddle

More from TEST Huddle (20)

Why We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- AccentureWhy We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- Accenture
 
Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar
 
Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway
 
Being a Tester in Scrum
Being a Tester in ScrumBeing a Tester in Scrum
Being a Tester in Scrum
 
Leveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional TestsLeveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional Tests
 
Using Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test WorkUsing Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test Work
 
Big Data: The Magic to Attain New Heights
Big Data:  The Magic to Attain New HeightsBig Data:  The Magic to Attain New Heights
Big Data: The Magic to Attain New Heights
 
Will Robots Replace Testers?
Will Robots Replace Testers?Will Robots Replace Testers?
Will Robots Replace Testers?
 
TDD For The Rest Of Us
TDD For The Rest Of UsTDD For The Rest Of Us
TDD For The Rest Of Us
 
Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)
 
Creating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger EnterprisesCreating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger Enterprises
 
Is There A Risk?
Is There A Risk?Is There A Risk?
Is There A Risk?
 
Are Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test CoverageAre Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test Coverage
 
Growing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for TestersGrowing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for Testers
 
Do we need testers on agile teams?
Do we need testers on agile teams?Do we need testers on agile teams?
Do we need testers on agile teams?
 
How to use selenium successfully
How to use selenium successfullyHow to use selenium successfully
How to use selenium successfully
 
Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey
 
Practical Test Strategy Using Heuristics
Practical Test Strategy Using HeuristicsPractical Test Strategy Using Heuristics
Practical Test Strategy Using Heuristics
 
Thinking Through Your Role
Thinking Through Your RoleThinking Through Your Role
Thinking Through Your Role
 
Using Selenium 3 0
Using Selenium 3 0Using Selenium 3 0
Using Selenium 3 0
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Peter Udo Diehl
 

Recently uploaded (20)

Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
THE BEST IPTV in GERMANY for 2024: IPTVreel
THE BEST IPTV in  GERMANY for 2024: IPTVreelTHE BEST IPTV in  GERMANY for 2024: IPTVreel
THE BEST IPTV in GERMANY for 2024: IPTVreel
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 

Tips for Writing Better Charters for Exploratory Testing Sessions by Michael D Kelly

  • 1. Today’s webinar is presented by Michael Kelly. He will discuss Tips for Writing Better Charters for Exploratory Testing Sessions. Welcome to the EuroSTAR Webinar Series www.eurostarconferences.com This webinar is due to start at 2pm. Make sure you stick around at the end for the Q&A session and continue the conversation with the speaker on Twitter after the show! @esconfs #esconfs
  • 2. Tips for Writing Better Charters for Exploratory Testing Sessions Michael Kelly, DeveloperTown Mike Kelly is a managing partner at DeveloperTown, a venture development firm. Mike is a contributing author to “How to Reduce the Cost of Software Testing”, is a past director and president for the Association for Software Testing, and a co-founder of the Indianapolis Workshops on Software Testing, a series of ongoing meetings on topics in software testing. www.eurostarconferences.com
  • 3. Join the conversation on Twitter #esconfs @esconfs @michael_d_kelly www.eurostarconferences.com Continue the conversation with the speaker on Twitter after the show!
  • 4. MICHAEL KELLY Managing Partner, DeveloperTown www.DeveloperTown.com Past President, Association for Software Testing www.AssociationForSoftwareTesting.org Articles and Blogs www.MichaelDKelly.com www.eurostarconferences.com @esconfs #esconfs
  • 5. “The tester, the way they work, and their ability to work in the future, is as important as the product being tested.” www.eurostarconferences.com @esconfs #esconfs “One of the most difficult aspects of software testing is coming up with good test ideas. It doesn't matter how you're doing your testing: scripted vs. exploratory, manual vs. automated, or performance vs. functional.”
  • 6. EXPLORATORY TESTING “Simultaneous learning, test design, and test execution.” – James Bach www.eurostarconferences.com @esconfs #esconfs
  • 7. www.eurostarconferences.com @esconfs #esconfs pure scripted freestyle exploratory chartersvague scripts fragmentary test cases (scenarios) roles -image from Jon Bach
  • 8. SESSION BASED TEST MANAGEMENT A method for managing exploratory testing. Created by Jon and James Bach, refined by others. www.eurostarconferences.com @esconfs #esconfs
  • 9. • charters • time-boxed • session notes • debriefs • team prioritization • ad-hoc test documentation • ad-hoc test automation • dynamic metrics and reporting www.eurostarconferences.com @esconfs #esconfs
  • 10. CHARTERING Making your own decisions about what you will work on and how you will work. Understanding your client’s needs, the problems you must solve, and assuring that your work is on target. www.eurostarconferences.com @esconfs #esconfs
  • 11. What the tester thinks • What am I suppose to be doing? • I need more time! • This is boring… • What data should I be using for this? • Didn’t someone already test this? • Is this really the most important thing for me to be focused on? What happens on the project • Areas of the product don’t get covered. • Entire categories of risk don’t get tested. • Duplication of effort. • The team does not work based on priority. • Testers get frustrated. • Poor visibility into actual progress. www.eurostarconferences.com @esconfs #esconfs BAD CHARTERING
  • 12. ESSENTIAL ELEMENTS OF A CHARTER www.eurostarconferences.com @esconfs #esconfs Risk, Coverage, and Timeframe
  • 13. COVERAGE www.eurostarconferences.com @esconfs #esconfs What am I testing? Which features, screens, artifacts, or areas of the product should I be touching while testing? RISK Why am I running this test? What could go wrong? What am I looking for? TIMEFRAME How long should it take to perform this testing? Are there setup tasks I should do before I officially start? What do I need to do when I’m done?
  • 14. When you look at a list of 10 charters, you should see 10 distinct testing missions, equating to around eight to 10 hours of heads-down testing. www.eurostarconferences.com @esconfs #esconfs
  • 15. BETTER CHARTERING www.eurostarconferences.com @esconfs #esconfs List specific risks and coverage targets Leverage mnemonics for risk and coverage ideas Develop a knowledge base for risk/coverage summaries Compare missions Try developing a couple of templates Charter for smaller sessions, then affinity map to create larger sessions Thumb vote for priority Use testing polarities Let charters emerge over time Track metrics
  • 16. LIST SPECIFIC RISK AND COVERAGE TARGETS Test the portal for reporting accuracy. Which reports? What do you mean by accuracy? Test reports X, Y, and Z for errors related to start and end time selection criteria, summing, totaling, and rounding. www.eurostarconferences.com @esconfs #esconfs
  • 17. MNEMONICS FOR RISK AND COVERAGE IDEAS • Application Touring: FFC CUTS VIDS • Coverage: SFDPOT • Coverage and Risk: FIBLOTS • Quality Criteria: CRUSSPICSTMPL • Test Techniques: FDSFSCURA • Oracles: HICCUPPS • Session Notes: MCOASTER www.eurostarconferences.com @esconfs #esconfs
  • 18. RISK AND COVERAGE KNOWLEDGE BASE As a team, build out a wiki detailing the nuance of various coverage areas and specific areas of risk. Commonly captured items will include: • Detailed feature listings/checklists • Integration points and data flow • Example and common test data / test beds • Known issues/gotchas • Application-specific or feature-specific test techniques www.eurostarconferences.com @esconfs #esconfs
  • 19. COMPARE MISSIONS • Test feature X • Stress test feature X • Performance test feature X What’s the difference? www.eurostarconferences.com @esconfs #esconfs
  • 20. My mission is to test <insert risk here> for <insert coverage here> Some examples: • My mission is to test for various boundary errors for Microsoft Word’s bullets and numbering feature. • My mission is to test for accurate error messaging pop-ups for Ford Motor Vehicle's Build and Price website. • My mission is to test for SQL injection vulnerabilities for application login and administration screens. www.eurostarconferences.com @esconfs #esconfs CHARTER TEMPLATE
  • 21. KiwanisInternationalApp (possible10to15mincharters) 1. Test for boundary related issues related to projects 2. Test for boundary related issues related to project reports 3. Test for deliverability related issues related to project emails 4. Test for data quality issues with clubs available on the app 5. Test for data accuracy issues related to calculations on project reports 6. Test for data accuracy issues related to calculations on service and fundraising summary stats 7. Test for data accuracy issues related to Piggy Bank commitment calculations 8. Perform basic link validation for items in the news feed 9. Perform a copy review of the various descriptive dialogs in the app looking for typos and grammar issues 10. Test for performance related issues related to mapping capabilities 11. Stress test the application to identify conditions that crash the app www.eurostarconferences.com @esconfs #esconfs CHARTER FOR SMALLER SESSIONS
  • 22. Walk your charters quickly as a team Use thumb voting to see if there is agreement around clarity and scope The particularly contentious charters will lead the team to discussion and debate (and new charters) Thumb Vote 101 High Priority “We need to run this charter.” Medium Priority “If we have time, we should run this charter.” Low Priority “It’s a test, and we could run it, but likely there are better uses of our time.” www.eurostarconferences.com @esconfs #esconfs THUMB VOTE
  • 23. TESTING POLARITIES Example Polarities: • Warming up vs. cruising vs. cooling down • Doing vs. describing • Careful vs. quick • Data gathering vs. data analysis • Working with the product vs. reading about the product • Working with the product vs. working with the developer • Training (or learning) vs. performing • Product focus vs. project focus • Solo work vs. team effort • Your ideas vs. other peoples’ ideas • Lab conditions vs. field conditions • Current version vs. old versions • Feature vs. feature • Requirement vs. requirement • Coverage vs. oracles • Testing vs. touring • Individual tests vs. general lab procedures and infrastructure • Testing vs. resting • Playful vs. serious The Process: • When you charter your tests, include polarities explicitly in your mission. • This practice is particularly effective for shorter test charters (20-30 minutes). • Or, when you charter, use the polarities to help you identify new charter ideas www.eurostarconferences.com @esconfs #esconfs Source: http://www.satisfice.com/blog/wp- content/uploads/2009/10/et-dynamics22.pdf IN YOUR CHARTER: • Testing vs. Touring • Feature vs. Feature • Requirement vs. Requirement WHILE CHARTERING: Individual Tests vs. General Lab Procedures Coverage vs. Oracles Lab Conditions vs. Field Conditions
  • 24. LET CHARTER EMERGE OVER TIME www.eurostarconferences.com @esconfs #esconfs 1. Pull down a charter for testing 2. Execute that charter 3. Debrief 4. After your debrief, add new charters and reprioritize existing charters 5. Repeat
  • 25. Time management is critical to chartering. Metrics help provide visibility into what’s really happening. Tagging allows for easier metrics gathering. I regularly tag the following: • bugs or issues for isolation (b) • possible new charters (c) • open questions (?) • tasks for automation (a) • tasks for documentation (doc) • setup, testing, investigation #... Some examples: • Actual execution time vs. estimated time • Confidence factor of tester on their coverage of the stated risks • Number of follow up charters resulting from lack of time • Percentage of session time spent setting up for testing • Percentage of session time spent testing • Percentage of session time spent investigating problems www.eurostarconferences.com @esconfs #esconfs TRACK METRICS
  • 26. BETTER CHARTERING www.eurostarconferences.com @esconfs #esconfs List specific risks and coverage targets Leverage mnemonics for risk and coverage ideas Develop a knowledge base for risk/coverage summaries Compare missions Try developing a couple of templates Charter for smaller sessions, then affinity map to create larger sessions Thumb vote for priority Use testing polarities Let charters emerge over time Track metrics
  • 27. EuroSTAR Webinar Archive Access almost 50 on-demand software testing webinars by the world's leading testing experts. Topics include…. www.eurostarconferences.com Agile, Automation, Test Management, Process Improvements, Risk-based Testing, Performance Testing, Test Documentation, Cloud Testing, Session-based Testing, Weekend Testers Visit the archive! http://www.eurostarconferences.com/community/member/webinar-archive @esconfs #esconfs
  • 28. The EuroSTAR Community. Are you a member? Did you know we have an online resource area with almost … www.eurostarconferences.com 600 software testing presentations, 60 topical webinars, over 40 videos and podcasts and 20 expert eBooks Do you know that membership is completely free and you can access these premium testing resources anytime? Join today! http://www.eurostarconferences.com/community/ member @esconfs #esconfs