Slides on Test Design for STP Webinar

1. Test Design for EveryoneAlan PageMicrosoft

2. Here, Test This!

3. “Test Design” is what Testers Do

4. What is test design?

5. Simple: Come up with some tests

6. Better(?): Come up with a well thought out and broad set of tests for the given context (based on application, schedule, etc.)

7. Another Take:Create an optimal set of tests that finds the maximum amount of customer facing issues possible with a minimum of over-testing and under-testing

8. “Over Testing” and “Under Testing”

9. Application

10. Application Testing Effort

11. Application Testing Effort Under Testing (stuff you forgot to test or didn’t know to test) (potential) Over Testing

18. bool Validate(string sideName) try { sideVal = Convert.ToUInt16(inputVal); if (sideVal < 1) { MessageBox.Show(sidename+ “must be greater than 0”, ...); return false; } } catch (OverflowException) { MessageBox.Show(sideName + "must be less than 65536", ...); return false; } catch (FormatException) { MessageBox.Show(sideName + "must be an integer value, ...); return false; } }

19. bool Validate(string sideName) try { sideVal = Convert.ToUInt16(inputVal); if (sideVal < 1) { MessageBox.Show(sidename+ “must be greater than 0”, ...); return false; } } catch (OverflowException) { MessageBox.Show(sideName + "must be less than 65536", ...); return false; } catch (FormatException) { MessageBox.Show(sideName + "must be an integer value, ...); return false; } } In English Three different types of errors: Zero or less Bigger than 65536 Non-numbers Any other input is testing the same thing

20. Application Testing Effort Reduce Over Testing (don’t test the same thing multiple times) Reduce Under Testing (miss less)

21. How do you become a better Test Designer?

22. You need ideas! Ideas you have used before New ideas to try Ideas suited for this productIdeas suited to your release schedule …and you need to know what to look for

23. Why Is it so Hard?

24. You Don’t Know What You Don’t Know Read Philip Armour’sThe Five Orders of Ignorance

25. All you can use is what you knowHammers are useful…

26. …but not everything is a nail

28. What you’ve already heard about test design

29. http://www.satisfice.com/glossary.shtml

30. The “what” has been covered

31. Let’s focus on the “how”

32. You need a Toolbox

33. Bigger is Better

35. What kinds of tests do you write

36. New Information False Positive False Negative No Information

37. Useful Tests?

38. Useful Test Provide High Levels of “New Information”

39. “Usefulness” depends on context

40. Some general guidelines: The first time you run a test it (almost) always provides new information If you run the same test the same way again, it may or may not provide new information

41. Some general guidelines (continued): Varying your tests gives them a higher chance of providing new information (randomization, data driven tests, “smart monkeys”, model-based testing, etc. all help here)

42. Examples / Practice

44. Test This: Add or Remove Contacts

45. Mind Mapping

47. What about adding a contact with a blank name?

51. 1 2 3 4

54. How to get better at Test Design

55. Practice, practice, practice Practice testing – be curious Collaborate http://testing-challenges.org Weekend Testing (http://weekendtesters.com) Learn How does your application work? Find new testing ideas Study critical thinking Read…anything

56. Make your tests useful…and notice when they’re not useful

57. The Best Test Tool of All Time

58. Your Brain (of course)

59. Be A Great Test Designer(and be a great tester too)

60. http://angryweasel.com/bloghttp://twitter.com/alanpage

61. Credits Opening slide pattern - some rights reserved by km6xo “Scrap metal” – some rights reserved by MAG (Mines Advisory Group) Ideas - Some rights reserved by orkboi Impossible triangle - some rights reserved by yui.kubo Hammer – some rights reserved by Noel C. Hankamer Nail - some rights reserved by crd! Hammer / screw - some rights reserved by justinbaeder Tomatoes - some rights reserved by lucianvenutian Toolbox - some rights reserved by Austin ampersand Zak Big Toolbox - some rights reserved by Fuschia Foot Brain - Some rights reserved by biologycorner