SlideShare a Scribd company logo
Exploratory &
Automation Testing:
Create a Healthy Relationship
Dawn Jardine
2
Introduction
3
Toronto, Canada
4
1
2
3
4
Automation and its role in testing
Exploratory and its role in testing
The relationship
How we did it
Agenda
15
5
Automation and its
role in testing
6
Why use automation?
7
Automation can provide better coverage
• for a large code base
• when a live site is in maintenance mode
• when it supplements manual testing
8
Automation can find defects earlier
• Finds defects introduced during sprint development
• Supports process of continuous test and fix in lower life cycle
• Reduces the cost of rework
9
Automation speeds delivery to client
• Continuous integration and delivery during sprint
• Achieve 100% regression test case execution
• Release small; release often
10
When do you
implement
automation?
11
Implement automation when:
• Current code base is growing
• Newer features and functionality are being added to live site
• Time is increasing for manual test case execution
12
What do
you automate?
13
Automate:
• Repeated manual tests on code that is changed most often
during sprint development
• Manual regression tests executed pre-release
• Sanity tests that are performed manually during releases
14
How do you implement automation?
15
Basic steps to implementing
• Choose a valuable platform
• Involve your current QA team
16
Exploratory testing and its
role in testing
17
Where does exploratory testing fit?
18
“Insanity is doing the same
thing over and over again and
expecting different results”
~Albert Einstein
19 James Bach, Exploratory Testing Explained
“An interactive process of
simultaneous learning,
test design, and test
execution.”
20
Does a QA need certain special skills?
• Experience
• Ability to create test scenarios
• Autonomy, skill and creativity
21
Can a QA “explore” if they already know it?
• Yes!!!
• It is NOT user acceptance testing
• Web Application integration
22
The relationship
23
“Assumptions are the termites
of relationships”
~Henry Winkler
24
Build on the strengths
• Trust
• Communication and collaboration
• Empowerment
• Confidence
25
How we did it:
The challenges and
successes
26
The exploratory + automated approach
• Ecommerce web application was live
• Automation in place for regression
• Agile process in development
• QA was still the bottleneck
27
Management approval
for change
• Respect and value of the QA team
• Don’t be afraid to innovate!
• A plan of full-coverage approach
28
Shifting the mind-set
of the QA team
• Information sessions about exploratory testing
• Review of existing manual regression tests
• Created user stories for high level scenarios
29
Be creative, organized and brave!
• Mind-mapping sessions for the high level scenarios
• Everyone participated – automation and manual test team
• No test steps!!
• Traceability – record the results
• Organized a trial run for next regression cycle
30
Gain stakeholders
support
• Business wants return on
investment
• Transparency is key!
• Continually inform
• Provide metrics
31
Sample of metrics for status reports
Manual
Exploratory Tests
Desktop Browsers OS/Devices
Total
Desktop
Chrome
FF
IE11
MACSafari
IOS8Chrome
IOS9Safari
IOS8Safari
AndroidChrome
#of Test Cases 91 9 15 10 12 14 9 12 10
Pass 91 9 15 10 12 14 9 12 10
Fail 0
Blocked 0 0 0 0 0 0 0 0 0
Unexecuted 0 0 0 0 0 0 0 0 0
Executed 91 9 15 10 12 14 9 12 10
Percentage Execution
Complete 100% 100% 100% 100% 100% 100% 100% 100% 100%
32
Continued Collaboration
• Ongoing audits of release
work
• Review recorded results from
exploratory tests
• Repeat, Repeat, Repeat!
33
Advantages of this approach
• QA team confidence
• Project management can stick to release timelines
• Business benefits by staying competitive
34
May 2014
• 261 manually executed
step-by-step test cases
• No automation
• 6 days x 4 QAs
January 2016
• 49 Exploratory Test cases
• 1140 automated tests
covering 4 browsers
• 3 days x 2 QAs
vs
Rewards
35
What’s next?
• Automate the mobile tests
• Streamline current exploratory tests
• Change the format of our functional tests in sprint
36
Conclusion
Thank you
Q & A

More Related Content

What's hot

QTest
QTest QTest
5 incredibly useful next gen test automation tool features
5 incredibly useful next gen test automation tool features5 incredibly useful next gen test automation tool features
5 incredibly useful next gen test automation tool features
Neha Mutalik
 
Deliver Fast, Break Nothing Via Effective Building Developer and Tester Colla...
Deliver Fast, Break Nothing Via Effective Building Developer and Tester Colla...Deliver Fast, Break Nothing Via Effective Building Developer and Tester Colla...
Deliver Fast, Break Nothing Via Effective Building Developer and Tester Colla...
Ho Chi Minh City Software Testing Club
 
[HCMC STC Jan 2015] FATS: A Framework For Automated Testing Scenarios
[HCMC STC Jan 2015] FATS: A Framework For Automated Testing Scenarios[HCMC STC Jan 2015] FATS: A Framework For Automated Testing Scenarios
[HCMC STC Jan 2015] FATS: A Framework For Automated Testing Scenarios
Ho Chi Minh City Software Testing Club
 
Drive Faster Quality Insights through Customized Test Automation - Part 2
Drive Faster Quality Insights through Customized Test Automation - Part 2Drive Faster Quality Insights through Customized Test Automation - Part 2
Drive Faster Quality Insights through Customized Test Automation - Part 2
Perfecto by Perforce
 
Automation Tools Overview
Automation Tools OverviewAutomation Tools Overview
Automation Tools Overview
Murageppa-QA
 
Mobile Video Games Testing Principles - Benjamin Poirrier
Mobile Video Games Testing Principles - Benjamin PoirrierMobile Video Games Testing Principles - Benjamin Poirrier
Mobile Video Games Testing Principles - Benjamin Poirrier
Ho Chi Minh City Software Testing Club
 
Continuous testing for devops
Continuous testing for devopsContinuous testing for devops
Continuous testing for devops
Subrahmaniam S.R.V
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOps
Perfecto Mobile
 
Automate More with Selenium for your RWD
Automate More with Selenium for your RWDAutomate More with Selenium for your RWD
Automate More with Selenium for your RWD
Perfecto by Perforce
 
Key Factors To Ensure Test Automation Framework Success
Key Factors To Ensure Test Automation Framework SuccessKey Factors To Ensure Test Automation Framework Success
Key Factors To Ensure Test Automation Framework Success
Ho Chi Minh City Software Testing Club
 
Introduction to Back End Automation Testing - Nguyen Vu Hoang, Hoang Phi
Introduction to Back End Automation Testing - Nguyen Vu Hoang, Hoang PhiIntroduction to Back End Automation Testing - Nguyen Vu Hoang, Hoang Phi
Introduction to Back End Automation Testing - Nguyen Vu Hoang, Hoang Phi
Ho Chi Minh City Software Testing Club
 
How To Transform the Manual Testing Process to Incorporate Test Automation
How To Transform the Manual Testing Process to Incorporate Test AutomationHow To Transform the Manual Testing Process to Incorporate Test Automation
How To Transform the Manual Testing Process to Incorporate Test Automation
Ranorex
 
Testing automation in agile environment
Testing automation in agile environmentTesting automation in agile environment
Testing automation in agile environment
Perfecto Mobile
 
Automated vs manual testing
Automated vs manual testingAutomated vs manual testing
Automated vs manual testing
Kanoah
 
Automation in the world of project
Automation  in the world of projectAutomation  in the world of project
Automation in the world of project
Zbyszek Mockun
 
Does dev ops need continuous testing devops days des moines 2018 v1
Does dev ops need continuous testing devops days des moines 2018 v1Does dev ops need continuous testing devops days des moines 2018 v1
Does dev ops need continuous testing devops days des moines 2018 v1
GerieOwen
 
Crowd Sourced Testing - Real-World Experience Using Rainforest QA Crowd-Sourc...
Crowd Sourced Testing - Real-World Experience Using Rainforest QA Crowd-Sourc...Crowd Sourced Testing - Real-World Experience Using Rainforest QA Crowd-Sourc...
Crowd Sourced Testing - Real-World Experience Using Rainforest QA Crowd-Sourc...
Marc Fasel
 

What's hot (20)

QTest
QTest QTest
QTest
 
Introduction to qualitia software
Introduction to qualitia softwareIntroduction to qualitia software
Introduction to qualitia software
 
5 incredibly useful next gen test automation tool features
5 incredibly useful next gen test automation tool features5 incredibly useful next gen test automation tool features
5 incredibly useful next gen test automation tool features
 
Deliver Fast, Break Nothing Via Effective Building Developer and Tester Colla...
Deliver Fast, Break Nothing Via Effective Building Developer and Tester Colla...Deliver Fast, Break Nothing Via Effective Building Developer and Tester Colla...
Deliver Fast, Break Nothing Via Effective Building Developer and Tester Colla...
 
[HCMC STC Jan 2015] FATS: A Framework For Automated Testing Scenarios
[HCMC STC Jan 2015] FATS: A Framework For Automated Testing Scenarios[HCMC STC Jan 2015] FATS: A Framework For Automated Testing Scenarios
[HCMC STC Jan 2015] FATS: A Framework For Automated Testing Scenarios
 
Drive Faster Quality Insights through Customized Test Automation - Part 2
Drive Faster Quality Insights through Customized Test Automation - Part 2Drive Faster Quality Insights through Customized Test Automation - Part 2
Drive Faster Quality Insights through Customized Test Automation - Part 2
 
Automation Tools Overview
Automation Tools OverviewAutomation Tools Overview
Automation Tools Overview
 
Mobile Video Games Testing Principles - Benjamin Poirrier
Mobile Video Games Testing Principles - Benjamin PoirrierMobile Video Games Testing Principles - Benjamin Poirrier
Mobile Video Games Testing Principles - Benjamin Poirrier
 
Continuous testing for devops
Continuous testing for devopsContinuous testing for devops
Continuous testing for devops
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOps
 
Automate More with Selenium for your RWD
Automate More with Selenium for your RWDAutomate More with Selenium for your RWD
Automate More with Selenium for your RWD
 
Key Factors To Ensure Test Automation Framework Success
Key Factors To Ensure Test Automation Framework SuccessKey Factors To Ensure Test Automation Framework Success
Key Factors To Ensure Test Automation Framework Success
 
Introduction to Back End Automation Testing - Nguyen Vu Hoang, Hoang Phi
Introduction to Back End Automation Testing - Nguyen Vu Hoang, Hoang PhiIntroduction to Back End Automation Testing - Nguyen Vu Hoang, Hoang Phi
Introduction to Back End Automation Testing - Nguyen Vu Hoang, Hoang Phi
 
How To Transform the Manual Testing Process to Incorporate Test Automation
How To Transform the Manual Testing Process to Incorporate Test AutomationHow To Transform the Manual Testing Process to Incorporate Test Automation
How To Transform the Manual Testing Process to Incorporate Test Automation
 
Testing automation in agile environment
Testing automation in agile environmentTesting automation in agile environment
Testing automation in agile environment
 
Automated vs manual testing
Automated vs manual testingAutomated vs manual testing
Automated vs manual testing
 
Automation in the world of project
Automation  in the world of projectAutomation  in the world of project
Automation in the world of project
 
Does dev ops need continuous testing devops days des moines 2018 v1
Does dev ops need continuous testing devops days des moines 2018 v1Does dev ops need continuous testing devops days des moines 2018 v1
Does dev ops need continuous testing devops days des moines 2018 v1
 
XPDay 04
XPDay 04XPDay 04
XPDay 04
 
Crowd Sourced Testing - Real-World Experience Using Rainforest QA Crowd-Sourc...
Crowd Sourced Testing - Real-World Experience Using Rainforest QA Crowd-Sourc...Crowd Sourced Testing - Real-World Experience Using Rainforest QA Crowd-Sourc...
Crowd Sourced Testing - Real-World Experience Using Rainforest QA Crowd-Sourc...
 

Viewers also liked

CWong project sheets
CWong project sheetsCWong project sheets
CWong project sheetsCheryle Wong
 
Abshier Powerpoint2comp
Abshier Powerpoint2compAbshier Powerpoint2comp
Abshier Powerpoint2comp
babshier
 
Woodruff Career Docs_20170127
Woodruff Career Docs_20170127Woodruff Career Docs_20170127
Woodruff Career Docs_20170127Mike Woodruff
 
Sandra trabajo investigación
Sandra trabajo investigaciónSandra trabajo investigación
Sandra trabajo investigación
Sandra M.
 
Презентация авторской группы "Друзья Брауна"
Презентация авторской группы "Друзья Брауна"Презентация авторской группы "Друзья Брауна"
Презентация авторской группы "Друзья Брауна"
braunsfriends
 
4AEQUIPO12
4AEQUIPO124AEQUIPO12
Celiac Case study
Celiac Case studyCeliac Case study
Celiac Case studyYeyan Jin
 
Microsoft Technical Lead Resume (1)
Microsoft Technical Lead Resume (1)Microsoft Technical Lead Resume (1)
Microsoft Technical Lead Resume (1)Ritanshu Barnwal
 
Webinar ABAP 7.40 sp5/sp8 Releaseinformationen
Webinar ABAP 7.40 sp5/sp8 ReleaseinformationenWebinar ABAP 7.40 sp5/sp8 Releaseinformationen
Webinar ABAP 7.40 sp5/sp8 Releaseinformationen
Cadaxo GmbH
 

Viewers also liked (10)

CWong project sheets
CWong project sheetsCWong project sheets
CWong project sheets
 
Abshier Powerpoint2comp
Abshier Powerpoint2compAbshier Powerpoint2comp
Abshier Powerpoint2comp
 
Woodruff Career Docs_20170127
Woodruff Career Docs_20170127Woodruff Career Docs_20170127
Woodruff Career Docs_20170127
 
Sandra trabajo investigación
Sandra trabajo investigaciónSandra trabajo investigación
Sandra trabajo investigación
 
Презентация авторской группы "Друзья Брауна"
Презентация авторской группы "Друзья Брауна"Презентация авторской группы "Друзья Брауна"
Презентация авторской группы "Друзья Брауна"
 
Darwin d
Darwin dDarwin d
Darwin d
 
4AEQUIPO12
4AEQUIPO124AEQUIPO12
4AEQUIPO12
 
Celiac Case study
Celiac Case studyCeliac Case study
Celiac Case study
 
Microsoft Technical Lead Resume (1)
Microsoft Technical Lead Resume (1)Microsoft Technical Lead Resume (1)
Microsoft Technical Lead Resume (1)
 
Webinar ABAP 7.40 sp5/sp8 Releaseinformationen
Webinar ABAP 7.40 sp5/sp8 ReleaseinformationenWebinar ABAP 7.40 sp5/sp8 Releaseinformationen
Webinar ABAP 7.40 sp5/sp8 Releaseinformationen
 

Similar to StarWest2016_T24_ExploratoryAutomation_DawnJardine_Final

Exploratory Testing and Automated Testing: Create a Healthy Relationship
Exploratory Testing and Automated Testing: Create a Healthy RelationshipExploratory Testing and Automated Testing: Create a Healthy Relationship
Exploratory Testing and Automated Testing: Create a Healthy Relationship
TechWell
 
Agile testing - Principles and best practices
Agile testing  - Principles and best practicesAgile testing  - Principles and best practices
Agile testing - Principles and best practices
Dr Ganesh Iyer
 
7 steps to Software test automation success
7 steps to Software test automation success7 steps to Software test automation success
7 steps to Software test automation success
ngowda gowda (Looking For New job in Bangalore
 
5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation
Sauce Labs
 
7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation
Worksoft
 
Test automation and beyond developing an effective continuous test strategy d...
Test automation and beyond developing an effective continuous test strategy d...Test automation and beyond developing an effective continuous test strategy d...
Test automation and beyond developing an effective continuous test strategy d...
GerieOwen
 
Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle
Kari Kakkonen
 
Role of Test Automation in Modern Software Delivery Pipelines
Role of Test Automation in Modern Software Delivery PipelinesRole of Test Automation in Modern Software Delivery Pipelines
Role of Test Automation in Modern Software Delivery Pipelines
Kasun Kodagoda
 
Test Automation using UiPath Test Suite - Developer Circle Part-1.pdf
Test Automation using UiPath Test Suite - Developer Circle Part-1.pdfTest Automation using UiPath Test Suite - Developer Circle Part-1.pdf
Test Automation using UiPath Test Suite - Developer Circle Part-1.pdf
Diana Gray, MBA
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015
Fabricio Epaminondas
 
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
Andrew Williams
 
UiPath Test Automation Webinar Recap
UiPath Test Automation Webinar RecapUiPath Test Automation Webinar Recap
UiPath Test Automation Webinar Recap
Auxis Consulting & Outsourcing
 
QA Role in Agile Teams
QA Role in Agile Teams QA Role in Agile Teams
QA Role in Agile Teams
Synerzip
 
Forward5 Auxis VMware
Forward5 Auxis VMwareForward5 Auxis VMware
Forward5 Auxis VMware
Auxis Consulting & Outsourcing
 
Digital transformation through devops dod indianapolis
Digital transformation through devops dod indianapolisDigital transformation through devops dod indianapolis
Digital transformation through devops dod indianapolis
Peter Varhol
 
Continuous testing maximising velocity, quality and customer happiness
Continuous testing maximising velocity, quality and customer happinessContinuous testing maximising velocity, quality and customer happiness
Continuous testing maximising velocity, quality and customer happiness
PractiTest
 
When is a project ready for Software Automation_NEW
When is a project ready for Software Automation_NEWWhen is a project ready for Software Automation_NEW
When is a project ready for Software Automation_NEWMike Christesen
 
Mastering Automation Quality: Exploring UiPath's Test Suite for Seamless Test...
Mastering Automation Quality: Exploring UiPath's Test Suite for Seamless Test...Mastering Automation Quality: Exploring UiPath's Test Suite for Seamless Test...
Mastering Automation Quality: Exploring UiPath's Test Suite for Seamless Test...
DianaGray10
 
6 Top Tips to a Testing Strategy That Works
6 Top Tips to a Testing Strategy That Works6 Top Tips to a Testing Strategy That Works
6 Top Tips to a Testing Strategy That Works
Eggplant
 

Similar to StarWest2016_T24_ExploratoryAutomation_DawnJardine_Final (20)

Exploratory Testing and Automated Testing: Create a Healthy Relationship
Exploratory Testing and Automated Testing: Create a Healthy RelationshipExploratory Testing and Automated Testing: Create a Healthy Relationship
Exploratory Testing and Automated Testing: Create a Healthy Relationship
 
Agile testing - Principles and best practices
Agile testing  - Principles and best practicesAgile testing  - Principles and best practices
Agile testing - Principles and best practices
 
7 steps to Software test automation success
7 steps to Software test automation success7 steps to Software test automation success
7 steps to Software test automation success
 
5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation5 Steps to Jump Start Your Test Automation
5 Steps to Jump Start Your Test Automation
 
7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation7 Tips from Siemens Energy for Success with Automation
7 Tips from Siemens Energy for Success with Automation
 
Test automation and beyond developing an effective continuous test strategy d...
Test automation and beyond developing an effective continuous test strategy d...Test automation and beyond developing an effective continuous test strategy d...
Test automation and beyond developing an effective continuous test strategy d...
 
Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle Agile Testing – embedding testing into agile software development lifecycle
Agile Testing – embedding testing into agile software development lifecycle
 
Role of Test Automation in Modern Software Delivery Pipelines
Role of Test Automation in Modern Software Delivery PipelinesRole of Test Automation in Modern Software Delivery Pipelines
Role of Test Automation in Modern Software Delivery Pipelines
 
Test Automation using UiPath Test Suite - Developer Circle Part-1.pdf
Test Automation using UiPath Test Suite - Developer Circle Part-1.pdfTest Automation using UiPath Test Suite - Developer Circle Part-1.pdf
Test Automation using UiPath Test Suite - Developer Circle Part-1.pdf
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015
 
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
DevOps Workshop - Addressing Quality Challenges of Highly Complex and Integra...
 
UiPath Test Automation Webinar Recap
UiPath Test Automation Webinar RecapUiPath Test Automation Webinar Recap
UiPath Test Automation Webinar Recap
 
QA Role in Agile Teams
QA Role in Agile Teams QA Role in Agile Teams
QA Role in Agile Teams
 
Forward5 Auxis VMware
Forward5 Auxis VMwareForward5 Auxis VMware
Forward5 Auxis VMware
 
QA Best Practices in Agile World_new
QA Best Practices in Agile World_newQA Best Practices in Agile World_new
QA Best Practices in Agile World_new
 
Digital transformation through devops dod indianapolis
Digital transformation through devops dod indianapolisDigital transformation through devops dod indianapolis
Digital transformation through devops dod indianapolis
 
Continuous testing maximising velocity, quality and customer happiness
Continuous testing maximising velocity, quality and customer happinessContinuous testing maximising velocity, quality and customer happiness
Continuous testing maximising velocity, quality and customer happiness
 
When is a project ready for Software Automation_NEW
When is a project ready for Software Automation_NEWWhen is a project ready for Software Automation_NEW
When is a project ready for Software Automation_NEW
 
Mastering Automation Quality: Exploring UiPath's Test Suite for Seamless Test...
Mastering Automation Quality: Exploring UiPath's Test Suite for Seamless Test...Mastering Automation Quality: Exploring UiPath's Test Suite for Seamless Test...
Mastering Automation Quality: Exploring UiPath's Test Suite for Seamless Test...
 
6 Top Tips to a Testing Strategy That Works
6 Top Tips to a Testing Strategy That Works6 Top Tips to a Testing Strategy That Works
6 Top Tips to a Testing Strategy That Works
 

StarWest2016_T24_ExploratoryAutomation_DawnJardine_Final

Editor's Notes

  1. Title + Subtitle use. This is a guide only. You will have to adjust by eye depending on how long/short the titles are. Always centred on page.