Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Understanding
Combinations
(Pairs and
Triples)
Each of the end points on this mind map (like “Young” and
“Friendly”) are parameter values.

Parameters

Parameter Values
This is what they look like in Hexawise.

Parameters

Parameter Values
When you click on the “Create Tests” button...
1

Hexawise generates a set of 2-way tests that contains
every single pair ...
2-way coverage is useful because so many defects can be
quickly found by testing all pairs of parameter values.
Studies sh...
But what does 2-way coverage mean

EXACTLY?
It means this...
Every single pair of parameter values appears together
in at least one of these six test cases:
For example...

Stylish
and
Tax Collector?
That’s a pair of values. So at least one test includes it.

Stylish
and
Tax Collector?
Test # 2
Old
and
Shabby?
That’s a pair of values. So at least one test includes it.

Old
and
Shabby?
Test # 4
Hostile
and
Skydiving
Instructor?
That’s a pair of values. So at least one test includes it.

Hostile
and
Skydiving
Instructor?
Also test # 4
Young
and
Hostile?
That’s a pair of values. So at least one test includes it.

Young
and
Hostile?
Test # 6
We could do this for quite a while.
Choose any two parameter values.
Any pair of test inputs you choose will appear in at
...
For every set of Parameter Values entered into Hexawise
there is a finite set of pairs that can be tested.
There are exactl...
All of the pairs will be tested in at least one test.
All 24 of those possible pairs are tested in at least one
of the fol...
Again, the reason it is a good idea to test all possible pairs is
because...
It is an extremely efficient way to find defect...
But what about coverage of
combinations involving 3 or more
parameter values?
It depends!
Sometimes, a specific combination of 3
values will be included in your set of 2way tests.
Other times, combinat...
3-way combinations involve 3 specific parameter values.
Young
and
Stylish
and
Skydiving
Instructor?
3-way combinations involve 3 specific parameter values.
Let’s look at a few examples of 3-way combinations and
see if they ...
This 3-way combination does appear in the 2-way tests.
Young
and
Stylish
and
Skydiving
Instructor?
This 3-way
combination
...
This 3-way combination does not appear in the 2-way tests.
Old
and
Stylish
and
Skydiving
Instructor?
This 3-way combination does not appear in the 2-way tests.
Shabby
and
Hostile
and
Tax Collector?
This 3-way combination does not appear in the 2-way tests.
Young
and
Stylish
and
Hostile?
If you want to make sure that all possible
“triples” like these are covered by at least
one of your tests, you will need t...
These 3-way tests contain at least one test for every triple.
Some 3-way combinations are covered in 2-way tests.
All 3-way combinations are covered in 3-way tests.
Reviewing the same 3-way combinations as before,
we see that all of the...
For additional information, please see:

training.hexawise.com
or

help.hexawise.com
Upcoming SlideShare
Loading in …5
×

Pairwise Software Test Design - What exactly is pairwise coverage? What is 3-way coverage?

955 views

Published on

Pairwise testing (known as allpairs testing all pairs testing, pair-wise testing, and orthogonal array test design) is a simple but extremely powerful test design approach.

It helps testers create tests faster, create more powerful and thorough tests, and reduce the number of tests they need to execute. But what does "pairwise coverage" mean? Exactly? What is 3-way coverage? This pairwise testing tutorial clearly explains these fundamental test coverage concepts.

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Pairwise Software Test Design - What exactly is pairwise coverage? What is 3-way coverage?

  1. 1. Understanding Combinations (Pairs and Triples)
  2. 2. Each of the end points on this mind map (like “Young” and “Friendly”) are parameter values. Parameters Parameter Values
  3. 3. This is what they look like in Hexawise. Parameters Parameter Values
  4. 4. When you click on the “Create Tests” button... 1 Hexawise generates a set of 2-way tests that contains every single pair of those test parameter values in at least one of the tests. 2
  5. 5. 2-way coverage is useful because so many defects can be quickly found by testing all pairs of parameter values. Studies show that 84% of defects in production could have been found by testing for every pair of parameter values. 1 input or 2 inputs (“pairwise) 3 inputs 4, 5 or 6 inputs • • • • Medical Devices:  D.R. Wallace, D.R. Kuhn, Failure Modes in Medical Device Software: an Analysis of 15 Years of Recall Data, International Journal of Reliability, Quality, and Safety Engineering, Vol. 8, No. 4, 2001.     Browser, Server:  D.R. Kuhn, M.J. Reilly, An Investigation of the Applicability of Design of Experiments to Software Testing, 27th NASA/IEEE Software Engineering Workshop, NASA Goddard SFC 4-6 December, 2002 .   NASA database:  D.R. Kuhn, D.R. Wallace, A.J. Gallo, Jr., Software Fault Interactions and Implications for Software Testing, IEEE Trans. on Software Engineering, vol. 30, no. 6, June, 2004.   Network Security:  K.Z. Bell, Optimizing Effectiveness and Efficiency of Software Testing: a Hybrid Approach,  PhD Dissertation, North Carolina State University, 2006.  
  6. 6. But what does 2-way coverage mean EXACTLY?
  7. 7. It means this... Every single pair of parameter values appears together in at least one of these six test cases:
  8. 8. For example... Stylish and Tax Collector?
  9. 9. That’s a pair of values. So at least one test includes it. Stylish and Tax Collector? Test # 2
  10. 10. Old and Shabby?
  11. 11. That’s a pair of values. So at least one test includes it. Old and Shabby? Test # 4
  12. 12. Hostile and Skydiving Instructor?
  13. 13. That’s a pair of values. So at least one test includes it. Hostile and Skydiving Instructor? Also test # 4
  14. 14. Young and Hostile?
  15. 15. That’s a pair of values. So at least one test includes it. Young and Hostile? Test # 6
  16. 16. We could do this for quite a while. Choose any two parameter values. Any pair of test inputs you choose will appear in at least one of the 2-way tests.
  17. 17. For every set of Parameter Values entered into Hexawise there is a finite set of pairs that can be tested. There are exactly 24 possible pairs of parameter values in this simple example.
  18. 18. All of the pairs will be tested in at least one test. All 24 of those possible pairs are tested in at least one of the following 6 tests.
  19. 19. Again, the reason it is a good idea to test all possible pairs is because... It is an extremely efficient way to find defects given that: • Studies show that 84% of defects in production could have been found by testing for every pair of parameter values and • 2-way coverage can be achieved in a relatively small number of tests. 1 input or 2 inputs (“pairwise) 3 inputs 4, 5 or 6 inputs • • • • Medical Devices:  D.R. Wallace, D.R. Kuhn, Failure Modes in Medical Device Software: an Analysis of 15 Years of Recall Data, International Journal of Reliability, Quality, and Safety Engineering, Vol. 8, No. 4, 2001.     Browser, Server:  D.R. Kuhn, M.J. Reilly, An Investigation of the Applicability of Design of Experiments to Software Testing, 27th NASA/IEEE Software Engineering Workshop, NASA Goddard SFC 4-6 December, 2002 .   NASA database:  D.R. Kuhn, D.R. Wallace, A.J. Gallo, Jr., Software Fault Interactions and Implications for Software Testing, IEEE Trans. on Software Engineering, vol. 30, no. 6, June, 2004.   Network Security:  K.Z. Bell, Optimizing Effectiveness and Efficiency of Software Testing: a Hybrid Approach,  PhD Dissertation, North Carolina State University, 2006.  
  20. 20. But what about coverage of combinations involving 3 or more parameter values?
  21. 21. It depends! Sometimes, a specific combination of 3 values will be included in your set of 2way tests. Other times, combinations of 3 values will not be included in your set of tests. Let’s look at a few examples.
  22. 22. 3-way combinations involve 3 specific parameter values. Young and Stylish and Skydiving Instructor?
  23. 23. 3-way combinations involve 3 specific parameter values. Let’s look at a few examples of 3-way combinations and see if they are covered in our 2-way tests. Young and Stylish and Skydiving Instructor? Shabby and Hostile and Tax Collector? Old and Stylish and Skydiving Instructor? Young and Stylish and Hostile?
  24. 24. This 3-way combination does appear in the 2-way tests. Young and Stylish and Skydiving Instructor? This 3-way combination does appear in test # 3 We got lucky.
  25. 25. This 3-way combination does not appear in the 2-way tests. Old and Stylish and Skydiving Instructor?
  26. 26. This 3-way combination does not appear in the 2-way tests. Shabby and Hostile and Tax Collector?
  27. 27. This 3-way combination does not appear in the 2-way tests. Young and Stylish and Hostile?
  28. 28. If you want to make sure that all possible “triples” like these are covered by at least one of your tests, you will need to create 3-way tests. Let’s now take a look at the 3-way tests from Hexawise that includes at least one test for every possible combinations of “triples.”
  29. 29. These 3-way tests contain at least one test for every triple. Some 3-way combinations are covered in 2-way tests.
  30. 30. All 3-way combinations are covered in 3-way tests. Reviewing the same 3-way combinations as before, we see that all of them are covered in the 3-way tests. That’s because every single triple is included in the 3way tests. Young and Stylish and Skydiving Instructor? Old and Stylish and Skydiving Instructor? Test # 6 Test # 8 Shabby and Hostile and Tax Collector? Young and Stylish and Hostile? Test # 3 Test # 4
  31. 31. For additional information, please see: training.hexawise.com or help.hexawise.com

×