A case about how a company (Sectra) is using Exploratory Testing in their agile development organization where testers and developers are sitting together in cross functional teams using Scrum.
Escorts Service Basapura ☎ 7737669865☎ Book Your One night Stand (Bangalore)
Exploratory testing in an agile development organization (it quality & test management conference copenhagen 12 june 2013)
1. Case: Exploratory Testing
in an agile development organization
Johan Åtting, Sectra
@johanatting
IT Quality & Test Management, Copenhagen 12 June 2013
2. Developer, Tester,
Project Manager,
Test Manager,
Support Manager,
Operations Manager,
Quality Manager,
Line Manager, @johanatting
Founded EAST: Östergötlands Association for Software Testers
Conference speaker
Currently: Chief Quality Officer at Sectra
Transport, Bank
& Finance,
Telecom and
MedTech.
19 years in the software business
3. WHY & HOW
we use exploratory testing
in our context today
5. 13%
87%Radiology IT - RIS/PACS
Orthopedic Imaging
Rheumathology
Our mission is to increase effectiveness of healthcare, while
maintaining or increasing quality in patient care.
6. 10 Agile development teams
2 Testers + 4 Programmers per team
Development & Testing - One department
16. Benefits
of exploratory testing
• It’s agile, flexible and responsive
• Focus & time is spent on testing and not
on writing test scripts
• Learning about the system step by step
• We find more bugs
18. 1 2 3 4 5 6 7 8 9 10
Two week sprintSprintPlanning
SprintDemo
Testing
Initial test
planning
Continuous test
re-planning
19. Initial test planning:
1) Brainstorm test ideas based on:
• Sprint objective
• Quality characteristics (safety, security,
performance, reliability, capability, usability…)
• Risks (safety, business, technical)
• ...or any other basis or source for test ideas
=> Documented as e.g. mind maps
20. Initial test planning:
2) Organize test ideas (mind map) into charters:
• Explore: Area, function, feature...
• To: discover information about...
• Using: resources, oracles, heuristics
23. When are we done?
We use test levels!
(to discuss test coverage)
24. 0 Not tested
1 Sanity Check
2 Common & Critical
3 Reasonable
4 Complex
5 Can’t think of
anything more
that could give us
valuble
information
Test Levels
Todays testing tasks progress board
Target area
25. Test Owner Area Status
Anna Reporting
Ben Import
Chris Export
David Printing
Ellen User roles
Fred Archiving (not started)
Greg Regression (not started)
Weekly test status meeting with all testers
Possible problems* or we suspect there will be
No problems* found or suspicions of any
Known problems*
*Problems = Preventing release or preventing further testing
26. Output:
(from a team sprint)
• Exploratory test records
• Updated (or new) regression tests
• Test cases for the Release Test phase
• Bug reports (only unfixed bugs)
But only if needed
28. Cross Team Testing
Gather all testers to test each others test
objects after every sprint.
We need to get fresh, unbiased, independent
eyes on what is being developed.
34. Takeaways
Cross Team Testing
• Everyone is biased
• There is a need for independent eyes
• Cross Team Testing is one solution
More info:
http://vimeo.com/53218576#at=0 (Video from ÖreDev 2012)
http://www.slideshare.net/Johantting/cross-team-testing-at-dev-lin2013 (Slides from DevLin 2013)
37. Test Owner Area Status
Anna Reporting
Ben Import
Chris Export
David Printing
Ellen User roles
Fred Archiving
Greg Regression
... at the end of each row.
38. Test Owner Area Status
Anna Reporting
Ben Import
Chris Export
David Printing
Ellen User roles
Fred Archiving
Greg Regression
Start of Release Test Phase
39. Release Testing
• Planned by each test owner
• Coordinated by a test project manager
• Mix of: Retest of new features, workflow
based tests, regression tests...
• Mix of exploratory tests & scripted tests
• Test environment is freezed
40. Test Owner Area Status
Anna Reporting
Ben Import
Chris Export
David Printing
Ellen User roles
Fred Archiving
Greg Regression
Continue Release Testing until
42. Benefits
of exploratory testing
• It’s agile, flexible and responsive
• Focus & time is spent on testing and not
on writing test scripts
• Learning about the system step by step
• We find more bugs
43. We do Exploratory Testing
- During Sprints
- Between Sprints (Cross Team Testing)
- During Release Test