Test Design for EveryoneAlan PageMicrosoft
Here, Test This!
“Test Design” is what Testers Do
What is test design?
Simple: Come up with some tests
Better(?): Come up with a well thought out and broad set of tests for the given context (based on application, schedule, etc.)
Another Take:Create an optimal set of tests that finds the maximum amount of customer facing issues possible with a minimum of over-testing and under-testing
“Over Testing” and “Under Testing”
Application
ApplicationTesting Effort
ApplicationTesting EffortUnder Testing (stuff you forgot to testor didn’t know to test)(potential)Over Testing
bool Validate(string sideName)    try    {sideVal = Convert.ToUInt16(inputVal);        if (sideVal < 1)        {MessageBox.Show(sidename+ “must be greater than 0”, ...);            return false;}    }    catch (OverflowException)    {MessageBox.Show(sideName + "must be less than 65536", ...);        return false;    }    catch (FormatException)    {MessageBox.Show(sideName + "must be an integer value, ...);        return false;    }}
bool Validate(string sideName)       try    {sideVal = Convert.ToUInt16(inputVal);        if (sideVal < 1)        {MessageBox.Show(sidename+ “must be greater than 0”, ...);            return false;        }    }    catch (OverflowException)    {MessageBox.Show(sideName + "must be less than 65536", ...);        return false;    }    catch (FormatException)    {MessageBox.Show(sideName + "must be an integer value, ...);        return false;    }}In EnglishThree different types of errors:Zero or lessBigger than 65536Non-numbersAny other input is testing the same thing
ApplicationTesting EffortReduceOver Testing(don’t test the same thing multiple times)Reduce Under Testing(miss less)
How do you become a better Test Designer?
You need ideas!Ideas you have used beforeNew ideas to tryIdeas suited for this productIdeas suited to your release schedule…and you need to know what to look for
Why Is it so Hard?
You Don’t Know What You Don’t Know Read Philip Armour’sThe Five Orders of Ignorance
All you can use is what you knowHammers are useful…
…but not everything is a nail
What you’ve already heard about test design
http://www.satisfice.com/glossary.shtml
The “what” has been covered
Let’s focus on the “how”
You need a Toolbox
Bigger is Better
What kinds of tests do you write
New InformationFalse PositiveFalse NegativeNo Information
Useful Tests?
Useful Test Provide High Levels of “New Information”
“Usefulness” depends on context
Some general guidelines:The first time you run a test it (almost) always provides new informationIf you run the same test the same way again, it may or may not provide new information
Some general guidelines (continued):Varying your tests gives them a higher chance of providing new information (randomization, data driven tests, “smart monkeys”, model-based testing, etc. all help here)
Examples / Practice
Test This:Add or Remove Contacts
Mind Mapping
What about adding a contact with a blank name?
1234
How to get better at Test Design
Practice, practice, practicePractice testing – be curiousCollaboratehttp://testing-challenges.orgWeekend Testing (http://weekendtesters.com)LearnHow does your application work?Find new testing ideasStudy critical thinkingRead…anything
Make your tests useful…and notice when they’re not useful
The Best Test Tool of All Time
Your Brain (of course)
Be A Great Test Designer(and be a great tester too)
http://angryweasel.com/bloghttp://twitter.com/alanpage
CreditsOpening slide pattern - some rights reserved by km6xo“Scrap metal” – some rights reserved by MAG (Mines Advisory Group)Ideas - Some rights reserved by orkboiImpossible triangle - some rights reserved by yui.kuboHammer – some rights reserved by Noel C. HankamerNail - some rights reserved by crd!Hammer / screw - some rights reserved by justinbaederTomatoes - some rights reserved by lucianvenutianToolbox - some rights reserved by Austin ampersand ZakBig Toolbox - some rights reserved by Fuschia FootBrain - Some rights reserved by biologycorner

Test Design For Everyone

Editor's Notes

  • #4 analyze a product, evaluate risk, use tools, and think criticallyTest ideas – the more you have, the better you’ll do…but you need to apply them in contextSo you need knowledge of the product…And broad knowledge of testing ideas (design)…and what to look for (oracles)
  • #5 analyze a product, evaluate risk, use tools, and think criticallyTest ideas – the more you have, the better you’ll do…but you need to apply them in contextSo you need knowledge of the product…And broad knowledge of testing ideas (design)…and what to look for (oracles)
  • #22 analyze a product, evaluate risk, use tools, and think criticallyTest ideas – the more you have, the better you’ll do…but you need to apply them in contextSo you need knowledge of the product…And broad knowledge of testing ideas (design)…and what to look for (oracles)
  • #23 analyze a product, evaluate risk, use tools, and think criticallyTest ideas – the more you have, the better you’ll do…but you need to apply them in contextSo you need knowledge of the product…And broad knowledge of testing ideas (design)…and what to look for (oracles)
  • #25 Five Orders of Ignorance
  • #26 Five Orders of Ignorance
  • #37 What is a useful test?
  • #47 Mind map
  • #48 Mind map