Test Plan Documentation Strategies - Mind Maps, Test Scripts, and More


Published on

Summary from the STP Con Testing Conference:

Documenting Tester instructions –
a Survey of Successful approaches

Track: Test Process

Justin hunter, CEO, Hexawise

This session provides an overview of several different approaches that are successfully used by testers to address the questions of: What type of testing documentation should you create; and how much detail is appropriate to include in order to guide testers and/or satisfy compliance requirements? This session will discuss pros and cons of several approaches, including:

• Checklists
• Mindmaps
• Test ideas
• Specification by example
• Detailed test scripts

In some software testing presentations, the speaker will explain why they think one particular approach is particularly attractive. This won’t be one of those talks.

Justin is a test design specialist who has enjoyed teaching testers how to improve the efficiency and effectiveness of their test case selection and documentation approaches.

Published in: Technology, Education
  • This is a presentation I gave recently at STP Con. A few quick comments:

    1) I misspoke in the mini interview. Hexawise is a test case design tool that allows testers to quickly generate unusually powerful sets of software tests. It is most definitely not a test case management tool. My bad.

    More info about Hexawise is at http://hexawise.com

    Sign up for a free account at: https://app.hexawise.com/signup-1-5

    2) This talk is a survey of successful approaches used by testers in the real world. It is interspersed with an unusually large number of references to blog posts, experience reports, thoughtful articles, and links to helpful tools, etc. For your convenience, links to all of those resources are made available here: http://bitly.com/bundles/hexawise/2
    Are you sure you want to  Yes  No
    Your message goes here

Test Plan Documentation Strategies - Mind Maps, Test Scripts, and More

  1. 1. Documenting Software Testing Instructions - A Survey of Successful Approaches STP Con October 25th, 2011 Justin Hunter
  2. 2. TopicsConsiderations Context Objectives TradeoffsDocuments “Straw Man” Mind Maps “Kan Ban Cards”Idea Triggers Personas Combinations Checklists
  3. 3. Topics3
  4. 4. Context is Critical Tester’sexperience level /Exploratory Testing skills Importance of keeping detailed records (e.g., for regulatory compliance or lawsuit defense) 4
  5. 5. Context is Critical Relatively little Enough detail to detail satisfy lawyers Tester’sexperience level /Exploratory Testing skills Enough detail to avoid tester Very confusion detailed Importance of keeping detailed records (e.g., for regulatory compliance or lawsuit defense) 5
  6. 6. Appropriate Amt. of Detail Sources: Key Terms (“Cloud Level” / “Clam Level”, etc.) 6 http://alistair.cockburn.us/Use+cases Image: accelerateddeliveryplatform.com
  7. 7. Objectives7
  8. 8. Objectives & Assumptions ? W o he h n? WPrescriptive vs. Exploratory? 8
  9. 9. Tradeoffs9
  10. 10. Practical Tradeoffs Reqs Docs and the SUTTest ideas from looking at... Reqs Docs Only Few Many Details in tests 10
  11. 11. Practical Tradeoffs Reqs Docs Exploratory / and the SUT Designed DuringTest ideas from looking at... Prescriptive / Reqs Docs Only Designed Before Few Details Many in tests 11
  12. 12. Documentation Strategy12
  13. 13. “Straw Man” Every test script MUST have: • Test Name • Test Number • Tester’s Name • Tester’s Dept. • Type of Test • Requirement Number • Pre-conditions • Configuration Details • Test Data • Specific Steps • Designer • Approver • Execution Time (est.) • Expected Results13
  14. 14. Time Management14
  15. 15. Overly-Detailed Documentation Actual Testing 15
  16. 16. Mind Maps16
  17. 17. Mind Maps - Test Plan Template http://www.softwaretestingclub.com/forum/topics/the-two- 17 page-test-plan?commentId=751045%3AComment%3A105353
  18. 18. Clear Communication Example http://www.softwaretestingclub.com/forum/topics/the-two-page- test-plan?commentId=751045%3AComment%3A105353 18
  19. 19. Evolution of a Mind Map “1. Identify Properties youre interested in”http://testerkiwi.blogspot.com/2011/04/building- exploratory-test-plan-redux.html 19
  20. 20. Evolution of a Mind Map“2. Start Collecting Intel” http://testerkiwi.blogspot.com/2011/04/building- exploratory-test-plan-redux.html 20
  21. 21. Evolution of a Mind Map3. After Talking to Developer... http://testerkiwi.blogspot.com/2011/04/building- exploratory-test-plan-redux.html 21
  22. 22. Mind MapsBenefits of mind maps so far...CreationMaintenance Quick and EasyCollaborationReportingBut would mind maps be detailed enough...... For junior testers?... For testers who will join team in six months? 22
  23. 23. “MindMapping 101 for Testers” http://www.bettertesting.co.uk/content/?p=956 23
  24. 24. Test Details into Test Mgmt Tool 24
  25. 25. Collaboration is Easy25
  26. 26. Planning & Prioritizing Riskshttp://www.bettertesting.co.uk/content/wp-content/uploads/2011/01/Test-Phase-3.4.jpg 26
  27. 27. Determining Scope - What’s Inhttp://www.bettertesting.co.uk/content/wp-content/uploads/2011/01/Test-Phase-3.4.jpg 27
  28. 28. Determining Scope - What’s Outhttp://www.bettertesting.co.uk/content/wp-content/uploads/2011/01/Test-Phase-3.4.jpg 28
  29. 29. Documentation29
  30. 30. Kan Ban Boards30
  31. 31. http://www.flickr.com/photos/roxocubes/5786672702/31
  32. 32. Paul Holland - (Testing-Focused) Paul Holland Alcatel-Lucent (personal photo of his testing board, used with permission) 32
  33. 33. Paul Holland - (Testing-Focused)Each note represents a half-day of testing activity (via Session-Based Testing) Highly recommended reading: For a detailed description of Paul’s experiences with this approach and his lessons learned, check out Michael Bolton’s “A Sticky Situation” in an upcoming version of STP Magazine. Also, Google “Session-based Test Management” 33
  34. 34. Kan Ban-ish Tool: Pivotal Tracker 34
  35. 35. More Details on Hover-Overs 35
  36. 36. More Detail Still by Clicking 36
  37. 37. Including Recent Test Results 37
  38. 38. Test Descriptions / Size Estimates 38
  39. 39. 39
  40. 40. Soap Opera TestingTests should be fun and aggressive.Write scenarios that are: • Reality-based • Exaggerated, and • Condensed http://www.logigear.com/logi_media_dir/Documents/ Soap_Opera_Testing.pdf 40
  41. 41. Soap Opera Testing http://www.logigear.com/logi_media_dir/Documents/ Soap_Opera_Testing.pdf41
  42. 42. Soap Opera Testing http://www.logigear.com/logi_media_dir/Documents/ Soap_Opera_Testing.pdf42
  43. 43. Soap Opera Examplehttp://www.logigear.com/logi_media_dir/ Documents/Soap_Opera_Testing.pdf 43
  44. 44. Personas and “Substruction” http://www.software-testing.com.au/blog/2006/07/30/ personas-substruction-and-other-trades-tricks/ 44
  45. 45. Behavior-Driven Development 45
  46. 46. Behavior-Driven Development 46
  47. 47. 47
  48. 48. How to Select?48
  49. 49. ... How to Handle More Details? 49
  50. 50. ... and still MORE Details?50
  51. 51. How to Document?Test inputs from all 3 mind mapsare entered. 51
  52. 52. jklhlkjhkjhkjhPairwise coverage in 19 highly-varied tests 52
  53. 53. Auto-populate Mad Libs-like tests 53
  54. 54. Detailed into Detailed Test Scripts Turned test scripts: quickly generated by Hexawise. 54
  55. 55. Checklists55
  56. 56. Checklists• Cem Kaner - The Value of Checklists and the Danger of Scripts: What Legal Training Suggests for Testers http:// www.kaner.com/pdfs/ValueOfChecklists.pdf• Justin Hunter - http://hexawise.wordpress.com/2009/11/04/ checklists-good-test-scripts-bad/• Thoughts from the Test Eye - Software Quality Characteristics• Michael Hunter, “You Are Not Done Yet”• Elizabeth Hendrickson - Testing Cheat Sheet 56
  57. 57. Software Quality Characteristics http://thetesteye.com/blog/2010/11/ software-quality-characteristics-1-0/ 57
  58. 58. Rapid Reporter (Free Tool)http://testing.gershon.info/reporter/ 58
  59. 59. Checklistswww.kaner.com/pdfs/ValueOfChecklists.pdf 59
  60. 60. “Easy-Access Executive Summary”An organized summary of all the links referred to in this presentation is available in the link below:http://bitly.com/bundles/hexawise/2 60
  61. 61. Invitation to SlideShare ViewersIf you have read this far, odds are very good that you’re the kind of tester who is interested in trying out promising new testing tools.Hexawise is a powerful test design tool that allows testers to zero in on a small set of unusually-powerful tests. We’d like you to try it and let us know what you think about it. We have an unusual - some have even said bizarre - pricing policy. While enterprise licenses of Hexawise start at $50,000 / year, we make fully- featured versions of Hexawise available at no cost for teams of 1 to 5 testers. It’s like having a rich uncle with a $50,0000 BMW who gave youthe keys to his car indefinitely. And he wouldn’t get upset even if you kept it or ruined it. Wouldn’t you at least want to take it out for a test drive?Sign up now https://app.hexawise.com/signup-1-5 61