Case: Exploratory Testingin an agile development organizationJohan Åtting, Sectra@johanattingIT Quality & Test Management,...
Developer, Tester,Project Manager,Test Manager,Support Manager,Operations Manager,Quality Manager,Line Manager, @johanatti...
WHY & HOWwe use exploratory testingin our context today
CONTEXT
13%87%Radiology IT - RIS/PACSOrthopedic ImagingRheumathologyOur mission is to increase effectiveness of healthcare, whi...
10 Agile development teams2 Testers + 4 Programmers per teamDevelopment & Testing - One department
Release TestDevelopment............Exploratory Testing- During Sprints- Between Sprints (Cross Team Testing)- During Relea...
What isExploratory Testing?
ExplorationWandering
Explore: The jungles of ...Using: Map, local guides, whip ...To: Find lost treasures
Explore: export functionTo: see how file size effects performanceUsing: The image data db, ...
Explore: export functionTo: collect info about design consistencyUsing: the import function as Oracle
Mission / Plan / ChartersTest...Adjust Observeuse feedback from the previoustest to inform the next
WHY
Benefitsof exploratory testing• It’s agile, flexible and responsive• Focus & time is spent on testing and noton writing te...
HOW
1 2 3 4 5 6 7 8 9 10Two week sprintSprintPlanningSprintDemoTestingInitial testplanningContinuous testre-planning
Initial test planning:1) Brainstorm test ideas based on:• Sprint objective• Quality characteristics (safety, security,perf...
Initial test planning:2) Organize test ideas (mind map) into charters:• Explore: Area, function, feature...• To: discover ...
TestingTest...Adjust ObserveAdjust: next step in the charter
Continuous test re-planningTest...Adjust ObserveAdjust: add charters & re-prioritize
When are we done?We use test levels!(to discuss test coverage)
0 Not tested1 Sanity Check2 Common & Critical3 Reasonable4 Complex5 Can’t think ofanything morethat could give usvalublein...
Test Owner Area StatusAnna ReportingBen ImportChris ExportDavid PrintingEllen User rolesFred Archiving (not started)G...
Output:(from a team sprint)• Exploratory test records• Updated (or new) regression tests• Test cases for the Release Test ...
Release TestDevelopment............Exploratory Testing- During Sprints- Between Sprints (Cross Team Testing)- During Relea...
Cross Team TestingGather all testers to test each others testobjects after every sprint.We need to get fresh, unbiased, in...
Sprint Sprint Sprint SprintCrossTeamTestingCrossTeamTestingCrossTeamTestingSprint Sprint Sprint SprintSprint Sprint Sprint...
1 2 3 4 5 6 7 8 9 10In the beginning of the next two week sprintSprintPlanningSprintDemoPlanning(1-2hours)Testing(6hours)C...
Planning (1-2 hours)1. High level presentation of test objects2. Planning sessions in smaller groups
Testing (6 hours)• Exploratory group testing sessions (*3)• Debrief to the Test Owner• Bug reporting
Time Slot Activity9:20 Test Session #1 70min10:30 Debriefing Session #1 15min10:45 FIKA11:00 Test Session #2 75min12:15 De...
TakeawaysCross Team Testing• Everyone is biased• There is a need for independent eyes• Cross Team Testing is one solutionM...
Release TestDevelopment............Exploratory Testing- During Sprints- Between Sprints (Cross Team Testing)- During Relea...
Release TestingBefore we start Relese Testing all areas needto have a ...
Test Owner Area StatusAnna ReportingBen ImportChris ExportDavid PrintingEllen User rolesFred ArchivingGreg Regressio...
Test Owner Area StatusAnna ReportingBen ImportChris ExportDavid PrintingEllen User rolesFred ArchivingGreg RegressionStart...
Release Testing• Planned by each test owner• Coordinated by a test project manager• Mix of: Retest of new features, workfl...
Test Owner Area StatusAnna ReportingBen ImportChris ExportDavid PrintingEllen User rolesFred ArchivingGreg Regressio...
Summary
Benefitsof exploratory testing• It’s agile, flexible and responsive• Focus & time is spent on testing and noton writing te...
We do Exploratory Testing- During Sprints- Between Sprints (Cross Team Testing)- During Release Test 
Twitter: @JohanAttingLinkedin: Johan ÅttingE-mail: johan.atting@sectra.com
Exploratory testing in an agile development organization  (it quality & test management conference   copenhagen 12 june 2013)
Upcoming SlideShare
Loading in …5
×

Exploratory testing in an agile development organization (it quality & test management conference copenhagen 12 june 2013)

807 views

Published on

A case about how a company (Sectra) is using Exploratory Testing in their agile development organization where testers and developers are sitting together in cross functional teams using Scrum.

Published in: Design
2 Comments
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total views
807
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
13
Comments
2
Likes
0
Embeds 0
No embeds

No notes for slide

Exploratory testing in an agile development organization (it quality & test management conference copenhagen 12 june 2013)

  1. 1. Case: Exploratory Testingin an agile development organizationJohan Åtting, Sectra@johanattingIT Quality & Test Management, Copenhagen 12 June 2013
  2. 2. Developer, Tester,Project Manager,Test Manager,Support Manager,Operations Manager,Quality Manager,Line Manager, @johanattingFounded EAST: Östergötlands Association for Software TestersConference speakerCurrently: Chief Quality Officer at SectraTransport, Bank& Finance,Telecom andMedTech.19 years in the software business
  3. 3. WHY & HOWwe use exploratory testingin our context today
  4. 4. CONTEXT
  5. 5. 13%87%Radiology IT - RIS/PACSOrthopedic ImagingRheumathologyOur mission is to increase effectiveness of healthcare, whilemaintaining or increasing quality in patient care.
  6. 6. 10 Agile development teams2 Testers + 4 Programmers per teamDevelopment & Testing - One department
  7. 7. Release TestDevelopment............Exploratory Testing- During Sprints- Between Sprints (Cross Team Testing)- During Release Test
  8. 8. What isExploratory Testing?
  9. 9. ExplorationWandering
  10. 10. Explore: The jungles of ...Using: Map, local guides, whip ...To: Find lost treasures
  11. 11. Explore: export functionTo: see how file size effects performanceUsing: The image data db, ...
  12. 12. Explore: export functionTo: collect info about design consistencyUsing: the import function as Oracle
  13. 13. Mission / Plan / ChartersTest...Adjust Observeuse feedback from the previoustest to inform the next
  14. 14. WHY
  15. 15. Benefitsof exploratory testing• It’s agile, flexible and responsive• Focus & time is spent on testing and noton writing test scripts• Learning about the system step by step• We find more bugs
  16. 16. HOW
  17. 17. 1 2 3 4 5 6 7 8 9 10Two week sprintSprintPlanningSprintDemoTestingInitial testplanningContinuous testre-planning
  18. 18. Initial test planning:1) Brainstorm test ideas based on:• Sprint objective• Quality characteristics (safety, security,performance, reliability, capability, usability…)• Risks (safety, business, technical)• ...or any other basis or source for test ideas=> Documented as e.g. mind maps
  19. 19. Initial test planning:2) Organize test ideas (mind map) into charters:• Explore: Area, function, feature...• To: discover information about...• Using: resources, oracles, heuristics
  20. 20. TestingTest...Adjust ObserveAdjust: next step in the charter
  21. 21. Continuous test re-planningTest...Adjust ObserveAdjust: add charters & re-prioritize
  22. 22. When are we done?We use test levels!(to discuss test coverage)
  23. 23. 0 Not tested1 Sanity Check2 Common & Critical3 Reasonable4 Complex5 Can’t think ofanything morethat could give usvalubleinformationTest LevelsTodays testing tasks progress boardTarget area
  24. 24. Test Owner Area StatusAnna ReportingBen ImportChris ExportDavid PrintingEllen User rolesFred Archiving (not started)Greg Regression (not started)Weekly test status meeting with all testersPossible problems* or we suspect there will beNo problems* found or suspicions of anyKnown problems**Problems = Preventing release or preventing further testing
  25. 25. Output:(from a team sprint)• Exploratory test records• Updated (or new) regression tests• Test cases for the Release Test phase• Bug reports (only unfixed bugs)But only if needed
  26. 26. Release TestDevelopment............Exploratory Testing- During Sprints- Between Sprints (Cross Team Testing)- During Release Test
  27. 27. Cross Team TestingGather all testers to test each others testobjects after every sprint.We need to get fresh, unbiased, independenteyes on what is being developed.
  28. 28. Sprint Sprint Sprint SprintCrossTeamTestingCrossTeamTestingCrossTeamTestingSprint Sprint Sprint SprintSprint Sprint Sprint SprintSprint Sprint Sprint Sprint
  29. 29. 1 2 3 4 5 6 7 8 9 10In the beginning of the next two week sprintSprintPlanningSprintDemoPlanning(1-2hours)Testing(6hours)Cross Team Testing
  30. 30. Planning (1-2 hours)1. High level presentation of test objects2. Planning sessions in smaller groups
  31. 31. Testing (6 hours)• Exploratory group testing sessions (*3)• Debrief to the Test Owner• Bug reporting
  32. 32. Time Slot Activity9:20 Test Session #1 70min10:30 Debriefing Session #1 15min10:45 FIKA11:00 Test Session #2 75min12:15 Debriefing Session #2 15min12:30 LUNCH13:30 Test Session #3 75min14:45 Debriefing Session #3 15min15:00 -> Bug Reporting / Optional additionaltesting
  33. 33. TakeawaysCross Team Testing• Everyone is biased• There is a need for independent eyes• Cross Team Testing is one solutionMore info:http://vimeo.com/53218576#at=0 (Video from ÖreDev 2012)http://www.slideshare.net/Johantting/cross-team-testing-at-dev-lin2013 (Slides from DevLin 2013)
  34. 34. Release TestDevelopment............Exploratory Testing- During Sprints- Between Sprints (Cross Team Testing)- During Release Test
  35. 35. Release TestingBefore we start Relese Testing all areas needto have a ...
  36. 36. Test Owner Area StatusAnna ReportingBen ImportChris ExportDavid PrintingEllen User rolesFred ArchivingGreg Regression... at the end of each row.
  37. 37. Test Owner Area StatusAnna ReportingBen ImportChris ExportDavid PrintingEllen User rolesFred ArchivingGreg RegressionStart of Release Test Phase
  38. 38. Release Testing• Planned by each test owner• Coordinated by a test project manager• Mix of: Retest of new features, workflowbased tests, regression tests...• Mix of exploratory tests & scripted tests• Test environment is freezed
  39. 39. Test Owner Area StatusAnna ReportingBen ImportChris ExportDavid PrintingEllen User rolesFred ArchivingGreg RegressionContinue Release Testing until
  40. 40. Summary
  41. 41. Benefitsof exploratory testing• It’s agile, flexible and responsive• Focus & time is spent on testing and noton writing test scripts• Learning about the system step by step• We find more bugs
  42. 42. We do Exploratory Testing- During Sprints- Between Sprints (Cross Team Testing)- During Release Test 
  43. 43. Twitter: @JohanAttingLinkedin: Johan ÅttingE-mail: johan.atting@sectra.com

×