EuroSTAR Software Testing Conference 2010 presentation on Can Exploratory Testing Save Lives by Fredrik Rydberg. See more at: http://conference.eurostarsoftwaretesting.com/past-presentations/

2. Presentation
Fredrik Rydberg AddQ Consulting
Started employment in 2001 at the Med Tech
company Maquet Critical Care
(Stockholm Sweden) as Test Manager and later Unit Manager for testing.
During the last 3 years Fredrik has been a Test Management consultant within sectors as Banking & Finance, Utilities (electricity) and Transport.

3. Can exploratory testing save lives?
Fredrik Rydberg
fredrik.rydberg@addq.se
+46 735 142 143Experiences from MAQUET

4. ProductDescriptions
Anesthesia System
Ventilator ( =Respirator)
http://www.youtube.com/watch?v=NbxxITf-N2Q

5. Description Ventilator Testning
Measured
value
Set value
P
Flow
V

6. Scripted requirement based testing at Maquet
•Authorities/ standards
•A ”Must” forced by security and quality
•Easy to evaluate when exit criteria is met
•Large amount of requirements to test –time consuming!
•Mostly manual test cases
•Old test cases result in few new defects!

7. Detailed requirements and scripted testing is absolutely needed but is not sufficient to fully verify the requirements !

8. Exploratory testing at Maquet
•Started spontaneously approx. 16 years ago
•Both requirements and parts not specified as req’s
•Also non functional tests
•User oriented
•Cost effective
•Intellectual
•Suitable method for clinical experts and other expertise

9. Comparison Scripted testing -ET
Scripted testing
1.Test Design with Test Steps
2.Execution
3.Logging in protocol

10. Exploratory Testing
•Parallel test design and execution
•Level of documentation is varying
Fully scripted
Less guidence
Fragmentic test cases
charters
Freestyle exploratory testing
Exploratory testing
Comparison Scripted testing -ET

11. Pitfalls/ experiences ET at Maquet
•Different testers have different ET skills
•Risk of decreased motivation without management and control…
•…But management and control may influence creativity negatively
•Difficult to monitor what has been tested without SBTM

12. Important capabilities for ET, Maquet
•Clinical competence
•Domain and technical knowledge
•Experience of previous testing/ project
•Test competence (testing techniques)
•Curious, patient, communication and networking skills

13. Variation in the project cycle (ET/ scripted)
ETScripted
Time (months) Test Time / month (hours)

14. Scripted testing
YIN
Slow, cold and calm
Exploratory
testing
YANG
Focused, warm and aggressive
Important to find the right mix!

15. Time distribution
ET execution18%
Automatic execution (scripted) 10%
Script execution27%
Script test reports5%
Designing Scripted Test cases25%
Other15% Efficient more times7 ET7)urdefects/ho(5.0)rdefect/hou(1) test timeET%(18) test timeScripted%(67:Efficiency Comparison  

16. Visit by James Bach (spring 2007)
•2 days visit
•Inspiration talk for employees
•Participation in testing
•”pep talk”
•Improvement advice
•Target of ET clearer

17. General Test tool support
•Maquet developed tool for session based testing (“Session Based Tester”)
•IBM Clear Quest, Requisite Pro
•Capture Replay tool
•Automatic loggings tool statistics
IBM Requisite Pro

18. SBTM-Session Based Test Management (Method)
•General method to manage, report and follow up ET
•Time box approx. 90 min (60-120 min)
•Supported by charters
•A result possible to evaluate
•Often combined with debriefing (approx. 15 min) Session-based testing is developed by Jonathan and James Bach (first presented in year 2000). See http://www.satisfice.com/sbtm for more info.

19. Session Based Testing Tool (Open Source)
•Maquet developed tool
•Started in use Aug. 2008
•Developed in Perl using SQL server
•Example field New session:
•Project
•Duration
•Which tests to be done?
•Which have been tested?

20. Example functionality:
•Add new session
•Report session
•Metrics: Fault intensity
•Archiving
•Test Reports
Session Based Testing Tool (Open Source)

21. Advantage SBTM Maquet
•Easier to manage and follow up ET
•Reuse of sessions
•Statistics
•Complementing the Test Report
•Participation from other departments
•Learn from other test sessions
•Common ET process

22. ET today at Maquet
•ET recognized and powerful test approach
•ET has resulted in an increased test focus at Maquet
•ET is the most efficient test approach
•ET contribution to Market Leading product

23. Future possibilities
•Continue to improve ET/ SBTM
•Improved debriefing/reporting
•ET + SBTM training for new employees
Suggestions future functionality (SBTM):
•Dynamic Metrics (parameter list)
•Validation / write protection of sessions
•iPhone/ Android version?
•Other tools?

24. Inspiration Exploratory testing
Inspiration links:
www.satisfice.com
www.developsense.com
www.kaner.com
http://testingeducation.org/wp/
Maquet SBT Open Source:
http://sites.google.com/site/sessionbasedtester/
Join the “Linked in” group “Session based testing”:
http://www.linkedin.com/groups?mostPopular=&gid=3316116

25. Fredrik Rydberg
fredrik.rydberg@addq.se

26. Summary
•ET needed to fully verify requirements
•Other defects with ET compared to ST
•Important to find the right mix
•Increased efficiency with ET
•SBTM (with tool) made ET more structured
•ET significant quality contribution