Upcoming SlideShare
×

# Pairwise testing - Strategic test case design

4,580 views

Published on

Using pairwise test case design to reduce the test cases needed for maximum test coverage.

Published in: Technology
2 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
4,580
On SlideShare
0
From Embeds
0
Number of Embeds
33
Actions
Shares
0
156
0
Likes
2
Embeds 0
No embeds

No notes for slide

### Pairwise testing - Strategic test case design

1. 1. Pairwise  Tes*ng       Strategic  Test  Case  Design   www.xboso6.com
2. 2. •  XBOSo&   –  Quality  and  Tes4ng  Process  Improvement  and  So&ware   Tes4ng  Experts   –  San  Francisco,  Beijing,  Oslo,  Amsterdam
3. 3. Pairwise  Tes*ng   The  Problem:   •  Many  defects  only  occur  when  a  combina4on  of  inputs  or   events  occur  that  interact  with  each  other.   •  But  to  test  every  combina4on  or  permuta4on  is  impossible.   The  Solu4on:   •  Pairwise  tes4ng  normally  begins  by  selec4ng  values  for  the   system’s  input  variables.     •  These  individual  values  are  o&en  selected  using  domain   equivalence  par44oning.       •  The  values  are  then  permuted  to  achieve  coverage  of  all  the   pairings.   3
4. 4. Pairwise  Tes*ng   •  The  reasoning  behind  all-­‐pairs  tes4ng:  the  simplest  bugs  in  a   program  are  generally  triggered  by  a  single  input  parameter.     •  The  next  simplest  category  of  bugs  consists  of  those   dependent  on  interac4ons  between  pairs  of  parameters,   which  can  be  caught  with  all-­‐pairs  tes4ng.     •  Bugs  involving  interac4ons  between  three  or  more   parameters  are  progressively  less  common,  while  at  the  same   4me  being  progressively  more  expensive  to  ﬁnd  by  exhaus4ve   tes4ng,  which  has  as  its  limit  the  exhaus4ve  tes4ng  of  all   possible  inputs.   4
5. 5. Pairwise  Tes*ng   Problem:  Test  System  S   Inputs    X  =  A,B    Y  =  C,D    Z  =  E,F       There  are  23  =  8  possible  combina4ons   5   System S X Y Z
6. 6. Pairwise  Tes*ng   6   8 Test Cases You can get rid of Test Case 2 (T2) •  {A, -, E} is covered in T1 •  { , -} is covered in T4 •  { -, } is covered in T6 Problem: •  How many test cases can you get rid of? •  List the Test cases you would keep. •  Time yourself. Test  ID   X   Y   Z   T1   A   C   E   T2   A   D   E   T3   A   C   F   T4   A   D   F   T5   B   C   E   T6   B   D   E   T7   B   C   F   T8   B   D   F   A, D D, E
7. 7. Pairwise  Tes*ng   Test  ID   X   Y   Z   T1   A   C   E   T2   A   D   E   T3   A   C   F   T4   A   D   F   T5   B   C   E   T6   B   D   E   T7   B   C   F   T8   B   D   F   7   Test  ID   X   Y   Z   T1   A   C   E   T4   A   D   F   T6   B   D   E   T7   B   C   F   8 Test Cases Pairwise testing 4 Test Cases
8. 8. Example   8   How many combinations are there for twelve inputs? - Hexawise.com
9. 9. Pairwise  Tes*ng         9   - Hexawise.com
10. 10. Pairwise  Tes*ng         10   - Hexawise.com
11. 11. Pairwise  Tes*ng         11
12. 12. Pairwise  Tes*ng  Tools   To  generate  the  20  test  cases  in  the  previous  slide  from  the   possible  3.7M  test  cases  would  be  extremely  4me  consuming.       Just  a  few  of  the  many  SW  tools  to  generate  the  tes4ng  pairs.   •  Hexawise   •  AllPairs  by  James  Bach,  free   •  PICT  (Pairwise  Independent  Combinatorial  Tes4ng)  tool   •  Automa4c  Eﬃcient  Test  Generator  or  AETG™   12
13. 13. Limita*ons  of  Pairwise  Tes*ng   Common  tes*ng  problems  are  made  worse:   •  Selec*ng  the  wrong  input:  The  problems  of  equivalence   par44oning  are  made  worse  with  a  pair  of  wrong  inputs.   •  The  ‘Oracle  Problem’:  Is  a  defect  properly  revealed  and  do  we   recognized  it  when  it  is  revealed?   •  The  input  values  to  the  system  under  test  are  not  discrete.     Problems  of  Pairwise  tes*ng:   •  High  probability  combina4ons  get  too  lihle  ahen4on.   •  Not  knowing  the  interdependencies  between  variables.   13
14. 14. Pairwise  Tes*ng   Works  well  when:   •  Number  of  possible  parameter  values  is  large     •  You  can  eﬀec4vely  separate  parameter  values   into  equivalence  classes.     14
15. 15. Pairwise  Tes*ng:  Cau*on   •  Pairwise  tes4ng  is  a  tool,  it  cannot  replace  a  skilled   experienced  tester.   •  As  a  tool,  it  is  eﬀec4ve  only  as  much  as  the  tester  is  eﬀec4ve.   •  Responsibility  for  good  tes4ng  remains  with  the  tester,  not   the  tool.   15
16. 16. Pairwise  Tes*ng   Conclusion:     •  When  used  properly,  pairwise  test  set  genera4on  is  an   important  technique  that  can  help  you  produce  beher   so&ware  systems.   16