SlideShare a Scribd company logo
1 of 18
@graemeRharvey

More Related Content

Viewers also liked

Exploratory testing
Exploratory testingExploratory testing
Exploratory testing
Huib Schoots
 
Exploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextExploratory Testing in an Agile Context
Exploratory Testing in an Agile Context
Elisabeth Hendrickson
 

Viewers also liked (12)

Dare to Explore: Discover ET!
Dare to Explore: Discover ET!Dare to Explore: Discover ET!
Dare to Explore: Discover ET!
 
ALM - Testes Exploratórios
ALM - Testes ExploratóriosALM - Testes Exploratórios
ALM - Testes Exploratórios
 
Henrik Andersson - Exploratory Testing Champions - EuroSTAR 2010
Henrik Andersson - Exploratory Testing Champions - EuroSTAR 2010Henrik Andersson - Exploratory Testing Champions - EuroSTAR 2010
Henrik Andersson - Exploratory Testing Champions - EuroSTAR 2010
 
Fredrik Rydberg - Can Exploratory Testing Save Lives - EuroSTAR 2010
Fredrik Rydberg - Can Exploratory Testing Save Lives - EuroSTAR 2010Fredrik Rydberg - Can Exploratory Testing Save Lives - EuroSTAR 2010
Fredrik Rydberg - Can Exploratory Testing Save Lives - EuroSTAR 2010
 
[GUTS-RS] Combatendo a síndrome do “testa ae” com sessões de exploratory testing
[GUTS-RS] Combatendo a síndrome do “testa ae” com sessões de exploratory testing[GUTS-RS] Combatendo a síndrome do “testa ae” com sessões de exploratory testing
[GUTS-RS] Combatendo a síndrome do “testa ae” com sessões de exploratory testing
 
Carsten Feildberg - Session Based Test Management in Practice - EuroSTAR 2010
Carsten Feildberg - Session Based Test Management in Practice - EuroSTAR 2010Carsten Feildberg - Session Based Test Management in Practice - EuroSTAR 2010
Carsten Feildberg - Session Based Test Management in Practice - EuroSTAR 2010
 
Exploratory testing
Exploratory testingExploratory testing
Exploratory testing
 
Apresentação inícial testes ágeis
Apresentação inícial testes ágeisApresentação inícial testes ágeis
Apresentação inícial testes ágeis
 
Exploratory Testing in an Agile Context
Exploratory Testing in an Agile ContextExploratory Testing in an Agile Context
Exploratory Testing in an Agile Context
 
Mind maps tutorial Agile Testing Days
Mind maps tutorial Agile Testing DaysMind maps tutorial Agile Testing Days
Mind maps tutorial Agile Testing Days
 
Michael mahlberg exploratory-testing-the_missing_half_of_bdd
Michael mahlberg exploratory-testing-the_missing_half_of_bddMichael mahlberg exploratory-testing-the_missing_half_of_bdd
Michael mahlberg exploratory-testing-the_missing_half_of_bdd
 
Exploratory test
Exploratory testExploratory test
Exploratory test
 

Recently uploaded

Recently uploaded (20)

What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Google I/O Extended 2024 Warsaw
Google I/O Extended 2024 WarsawGoogle I/O Extended 2024 Warsaw
Google I/O Extended 2024 Warsaw
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
The UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, OcadoThe UX of Automation by AJ King, Senior UX Researcher, Ocado
The UX of Automation by AJ King, Senior UX Researcher, Ocado
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
Demystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John StaveleyDemystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John Staveley
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024Top 10 Symfony Development Companies 2024
Top 10 Symfony Development Companies 2024
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 

Tracking Exploratory Test Sessions For Improved Reporting Capabilities

  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.

Editor's Notes

  1. Following a test team’s journey at an attempt to improve SBTM consistency across multiple agile teams. A reflection on an experience, not theory. Often it is easy to continue doing something the same way instead of rethinking how it could be done Why do we need improved reporting? - Knowing what we’ve done - Managers NEED to know Why use “exploratory methods”? - freedom to allow talented testers the freedom to explore the areas of importance, as they see them. - better coverage; repetitive testing can be done via automation
  2. Who Am I? -”Young” tester seeking to push the limits of myself and my company, and to grow and promote the industry of software testing
  3. I like to learn by doing Someone has to try new things…we were lucky enough to have a chance to
  4. What is SBTM? Creative test coverage with metrics
  5. Who recognizes this template? Why does it exist? What is it for? -Using text files is not wrong, but we can do better.
  6. There are tools to overcome some of these issues, but why create new tools when easy solutions exist?
  7. Nothing met our immediate needs better than SessionWeb Piloted the tool with 3 Agile teams (approx two testers on each team) Issues WE wanted to solve: -tracking sessions/test notes -categorizing sessions -reports -feedback! Issues we didn’t need to solve: -Time spent recording test steps -Capturing data/screenshots/etc
  8. Demo time!
  9. What can we infer from this? - Lots of sessions waiting to be debriefed. Are people not doing debriefs? If not…why not? 0 Not Executed means no pre-planning of sessions (at least currently) - 18 In-Progress. Do we have 18 testers currently testing? If not…why are these sessions in progress?
  10. What can we infer? We know testers prefer Chrome Is this a good representation of what our clients use? (The answer is no…) Why is IE9 more popular than IE10 or IE11?
  11. What can we infer? - Clearly all of our testers use Windows 7 machines. - Someone has access to OSX/iOS, but it’s probably not easy enough for people to use frequently
  12. What can we infer? There has been significantly more testing on Feature F1500 and F1461 than the other features shown. F1500 appears to have a large number of found defects associated with it – this makes sense F1461 doesn’t have nearly as many defects found – perhaps testing spent here is not valuable F1593 has lots of defects found and not much test time associated to it. Perhaps more test time should be spent here.
  13. Piloted with one small area of product development Realized we had to keep improving our process as we found areas for improvement Lots of change/flux was confusing to testers/managers
  14. I still say Success! Pilot stats: 6 months in, 650+ Sessions, 100+ Sessions/month, approx. 1300 hours of in-session time New department after reorganization: -dev managers specifically asked for SessionWeb to be kept alive -given multiple presentations about good SBTM practices to managers and their teams
  15. More insight into the test results Didn’t require individuals to roll up the information Data was available “on demand” to whoever needed it (testers, leads, managers, directors)
  16. Reports are clear and consistent No ramp up time understanding the data – presented in charts consistently across groups Raw data available to do additional manipulations/analysis
  17. Helped everyone (testers, devs, managers) understand exploratory/session based testing, and why we capture what we do Consistent process means anyone can test for any team and not have to ask questions about the how/what/where Helped maintain good charters/test notes Increased debriefs and the quality of them due to visibility
  18. Image References: 321.jpg - http://kuzbein.com/wordpress/wp-content/uploads/2013/04/321web.jpg   Sherlock - http://wowmoscow.net/sites/default/files/images/news/20080314_sherlock_holmes.jpg   speech bubble - http://www.woodcutcomponents.co.uk/shop/images/speech%20bubble.JPG   file icon - http://icons.iconarchive.com/icons/martz90/circle/512/files-icon.png   search icon - http://aux2.iconpedia.net/uploads/savesearch-folder-circle-icon-256.png   disk icon - http://thumbs.gograph.com/gg64765158.jpg   lock icon - http://static.freepik.com/free-photo/lock-in-a-circle_318-9883.jpg   eyes - http://glitternglued.files.wordpress.com/2011/10/picture-81.png   arrows - http://cdn5.fedobe.com/wp-content/uploads/2013/05/Arrow-Consistency.jpg   hands on keyboard - http://leftcoastcowboys.com/wp-content/uploads/2008/11/hands-on-keyboard.gif   explorer - http://www.src.edu/services/career/PublishingImages/Explore_2.jpg