Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Combinatorial
Software Test
   Design

                (Beyond
                Pairwise
                Testing)




     ...
Presented at STPCon
                                                              October, 2010

                         ...
L E M
      O B
P R
        3
The Problem


           You can’t test
            everything.




http://www.flickr.com/photos/maisonbisson/109211670/   2
The Problem



       “With so many
       possible tests to
       choose from...”




http://www.flickr.com/photos/madaba...
The Problem



                                    “Things might
                                        seem
            ...
The Problem




                                                     ... discouraging,
                                   ...
The Problem


                                                         So, how can you select
                            ...
The Problem




                        ... and still
                          achieve
                         adequate
...
The Problem




                     Wait. So this is a test design method to
                  achieve greater coverage i...
Not Just Fewer, Better Tests

     If you prioritize                                But for those testers who
     speed, ...
Not Just Fewer, Better Tests




                W  ay                                 The
                               ...
Not Just Fewer, Better Tests


                                                           How can you create “curiously
  ...
I O N
     L UT
S   O
       14
The Solution

                                                This presentation describes an
                             ...
Structured Variation




 To do so, use
combinatorial
  test design
methods that
     build
  structured
   variation
into...
Structured Variation?




                                                                 “Test One”




http://www.flickr...
Structured Variation?

      This is not what I mean by structured
                    variation.




                    ...
Structured Variation?

      This is not what I mean by structured
                    variation.




                    ...
Structured Variation?




 So if that’s not structured
     variation, what is?
20
Structured Variation



                                                  Structured
                                     ...
Structured Variation




                                                           ...each test
                         ...
Structured Variation



                                                   ...than the
                                   ...
Structured Variation




     ...testers
    will do their
     testing in
     different
        ways,




http://www.flic...
Structured Variation




                                                          ... and
                               ...
Structured Variation




          “OK. Maximum variation is better than redundantly
                repeated redundant re...
Coverage



                          No.
                        There’s
                         more.
                 ...
Coverage




    Accidentally
    leaving gaps
     in coverage
     can lead to
        painful
       results.




http:...
Coverage




                                                    Structured
                                              ...
Coverage




                          ... with as
                              few
                          resources
 ...
Coverage


                ...and in the
                    most
                  efficiently
                 structured...
F
        O &
      S
     U N
   C IO
 O T
F U        L E
   L ONA
 O I
S T
 R A  32
Single-Mode Faults




                                                             Most software defects
                ...
Dual-Mode Faults



     Software bugs caused
     by the interaction of
      two test inputs are
      very common too.
...
Triple-Mode Faults

                                      Defects that require three specific test
                        ...
Quadruple-Mode Faults


            Defects that require four more test inputs to
                  trigger them are extre...
Implication on Testing Strategy




                 Aha! So, to find bugs efficiently, we should
                      focu...
Implication on Testing Strategy

             That’s right, two.
            Focus on covering all
             possible t...
Implication on Testing Strategy

     To find bugs efficiently                                      we should focus
        ...
I C S
            AN
    E C   H
M
           40
Combinations of Test Inputs


                       The key
                     words here
                        are t...
Combinations of Test Inputs


      Test inputs
       will vary
     dramatically
     from project
      to project.



...
Combinatorial Software Test Design



                                                                           Well, duh...
Combinatorial Software Test Design


                Here’s how it works: First, people
                are required to id...
Combinatorial Software Test Design




 Second, computer algorithms will take those
   inputs and quickly calculate the sm...
Combinatorial Software Test Design

                                                     ... and again, the
              ...
Combinatorial Software Test Design



                 W  ay                                                The woman
    ...
Combinatorial Software Test Design



This guy, wanting more
coverage, should create
     more thorough
  combinatorial te...
OT
      Y N STS
    H
 W NT ?   E
   S  G ND
     I A
 E
D YH
   B  49
Advantages of Combinatorial Software Test Design

                              Test Design: Man vs. Machine
             ...
Coverage Advantages

            Whatever
            your test
              inputs,
         combinatorial
         test...
Advantages of Combinatorial Software Test Design




                                            OK, but wouldn’t I have t...
Advantages of Combinatorial Software Test Design



Perhaps, but maybe you...




               Forget        Run out of ...
Coverage Advantages




    Plus, using combinatorial testing, you’ll
also get additional coverage from combinations
 that...
Coverage Advantages


                                         ...specialized use cases,
                    No, its way
 ...
Coverage Advantages




  ... unexpected
  combinations,




http://www.flickr.com/photos/funky64/2875860978
              ...
Coverage Advantages




                                                                 ...notable,
                     ...
Coverage Advantages




                                                         ...even combinations that
               ...
Coverage Advantages

      ... and...
  combinations
  that will take
     you into
     different
   parts of the
    app...
Coverage Advantages




Quickly testing
unusual pairs is
     often
  surprisingly
  efficient and
effective. Many
  unexpe...
Constraint Handling

                                                                           Combinations
             ...
E T ?
         Y Y
  E AD
R
         62
Ready?




                                                           I’m ready
                                          ...
Hazards




                                                               Not so fast.
                                  ...
Limitations




                                                       And let’s
                                         ...
Prerequisites




                                                         It also requires
                              ...
Prerequisites




                                                    To thrive,
                                         ...
Practice Makes Perfect



         Training
       and practice
         will help
           you
         succeed.




ht...
Questions




                        ?

http://www.flickr.com/photos/nathaninsandiego/3647136579
                         ...
Ready?



                                                        Let’s get
                                              ...
D I X
                                         P E N
                   AP
http://www.flickr.com/photos/dannohung/332507807...
Thank you

      Special thanks to the folks at:
         and its talented contributing
      photographers, including: Mi...
Why 2-way Testing is Effective

Multiple thorough studies show approximately 85% of
defects in production could have been ...
Additional Information

Excellent introductory articles and instructional videos: www.CombinatorialTesting.com


Results o...
Practice Tips: 4-Step Process

     Four-Step Process to Design Efficient and Effective Tests

                           ...
Practice Tips: Warning Signs

“You might be headed for trouble if...”

                                                   ...
Upcoming SlideShare
Loading in …5
×

Combinatorial software test design beyond pairwise testing

47,574 views

Published on

Pairwise and combinatorial testing explained. Orthogonal array-based testing, pair-wise software testing, and other more thorough n-way combinatorial test design strategies are proven to be efficient and effective. Unfortunately, much of the material on the internet about these test design techniques are dense, impenetrable tomes filled with long Greek-letter-infused equations that only a mathematician could love. This presentation aims to explain the principles behind this powerful but under-appreciated software test design approach.

Published in: Technology

Combinatorial software test design beyond pairwise testing

  1. 1. Combinatorial Software Test Design (Beyond Pairwise Testing) 1
  2. 2. Presented at STPCon October, 2010 By Justin Hunter CEO of Hexawise Cover slide photo credit: "Mike" Michael L. Baird, flickr.bairdphotos.com http://www.flickr.com/photos/mikebaird/2127310513 2
  3. 3. L E M O B P R 3
  4. 4. The Problem You can’t test everything. http://www.flickr.com/photos/maisonbisson/109211670/ 2
  5. 5. The Problem “With so many possible tests to choose from...” http://www.flickr.com/photos/madabandon/344158061/ 2
  6. 6. The Problem “Things might seem uncontrollable...” http://www.flickr.com/photos/maisonbisson/110867288/sizes/l/in/photostream/ 2
  7. 7. The Problem ... discouraging, even. http://www.flickr.com/photos/cheo70/2702682262/ 2
  8. 8. The Problem So, how can you select just a few tests... http://www.flickr.com/photos/krawcowicz/3830467093/ 8
  9. 9. The Problem ... and still achieve adequate coverage? http://www.flickr.com/photos/editor/4861496460/ 9
  10. 10. The Problem Wait. So this is a test design method to achieve greater coverage in fewer tests. Right? http://www.flickr.com/photos/johncarleton/4890239248 10
  11. 11. Not Just Fewer, Better Tests If you prioritize But for those testers who speed, yes. You prioritize thoroughness, can achieve this method can also be greater coverage used to increase in fewer tests. thoroughness dramatically. It can help you prioritize around http://www.flickr.com/photos/legofenris/4212007962/ either goal. 11
  12. 12. Not Just Fewer, Better Tests W ay The choice is fewer yours to make. tes ts! etter ligh tly b ith s than we (W rage cove e now.) hav http://www.flickr.com/photos/thomashawk/2402194148 http://www.flickr.com/photos/thomashawk/2402190314 12
  13. 13. Not Just Fewer, Better Tests How can you create “curiously strong” test cases that will allow you to achieve either of these testing objectives? http://www.flickr.com/photos/photobunny_earl/2309612944/ 13
  14. 14. I O N L UT S O 14
  15. 15. The Solution This presentation describes an approach that will help you create powerful test cases. http://www.flickr.com/photos/maisonbisson/110867008/ 15
  16. 16. Structured Variation To do so, use combinatorial test design methods that build structured variation into your test cases. 16
  17. 17. Structured Variation? “Test One” http://www.flickr.com/photos/gelrodandjodie/148759732 17
  18. 18. Structured Variation? This is not what I mean by structured variation. “Test Two” http://www.flickr.com/photos/gelrodandjodie/148759732 18
  19. 19. Structured Variation? This is not what I mean by structured variation. “Test Three” http://www.flickr.com/photos/gelrodandjodie/148759732 19
  20. 20. Structured Variation? So if that’s not structured variation, what is? 20
  21. 21. Structured Variation Structured variation means http://www.flickr.com/photos/tk_five_0/2281768728 21
  22. 22. Structured Variation ...each test will be different http://www.flickr.com/photos/criben/2635618391 22
  23. 23. Structured Variation ...than the tests before it, and http://www.flickr.com/photos/horrgakx/3186982135 23
  24. 24. Structured Variation ...testers will do their testing in different ways, http://www.flickr.com/photos/pixelrobber/2626213081 24
  25. 25. Structured Variation ... and explore from more angles. http://www.flickr.com/photos/tk_five_0/2286693552 25
  26. 26. Structured Variation “OK. Maximum variation is better than redundantly repeated redundant repetition. Got it. That it? Am I good to go now?” http://www.flickr.com/photos/johncarleton/4890241530 26
  27. 27. Coverage No. There’s more. Much more. Remember coverage? http://www.flickr.com/photos/30431232@N03/3610846363 27
  28. 28. Coverage Accidentally leaving gaps in coverage can lead to painful results. http://www.flickr.com/photos/dwfromthepeg/240921271/ 28
  29. 29. Coverage Structured variation focuses on covering as much as possible... http://www.flickr.com/photos/localsurfer/38245658/ 29
  30. 30. Coverage ... with as few resources as possible... http://www.flickr.com/photos/mikebaird/4177570453/ 30
  31. 31. Coverage ...and in the most efficiently structured way possible. http://www.flickr.com/photos/dannohung/332507807 31
  32. 32. F O & S U N C IO O T F U L E L ONA O I S T R A 32
  33. 33. Single-Mode Faults Most software defects in production today can be triggered by just one test input. http://www.flickr.com/photos/vaguelyartistic/292083492 33
  34. 34. Dual-Mode Faults Software bugs caused by the interaction of two test inputs are very common too. http://www.flickr.com/photos/kanaka/1798327442 34
  35. 35. Triple-Mode Faults Defects that require three specific test inputs to trigger them are rare. http://www.flickr.com/photos/kanaka/1798327442 35
  36. 36. Quadruple-Mode Faults Defects that require four more test inputs to trigger them are extremely rare. http://www.flickr.com/photos/novembering/3816524907 36
  37. 37. Implication on Testing Strategy Aha! So, to find bugs efficiently, we should focus first on testing all possible combinations every set of TWO test inputs! http://www.flickr.com/photos/johncarleton/4890236762 37
  38. 38. Implication on Testing Strategy That’s right, two. Focus on covering all possible two-way defects first. In fact, that’s so important, can you say it a second time? http://www.flickr.com/photos/soulsurfingcrew/2273300213 38
  39. 39. Implication on Testing Strategy To find bugs efficiently we should focus first on testing all possible combinations of TWO test inputs! http://www.flickr.com/photos/johncarleton/4890236762 39
  40. 40. I C S AN E C H M 40
  41. 41. Combinations of Test Inputs The key words here are test inputs and combinations. http://www.flickr.com/photos/12tribu/2750040244 41
  42. 42. Combinations of Test Inputs Test inputs will vary dramatically from project to project. 42
  43. 43. Combinatorial Software Test Design Well, duh! They would, wouldn’t they? But how does it all work? http://www.flickr.com/photos/johncarleton/4377591743 43
  44. 44. Combinatorial Software Test Design Here’s how it works: First, people are required to identify what test inputs should be tested. http://www.flickr.com/photos/mikebaird/3939231212 44
  45. 45. Combinatorial Software Test Design Second, computer algorithms will take those inputs and quickly calculate the smallest possible number of tests required to meet the tester’s specified coverage objectives. 45
  46. 46. Combinatorial Software Test Design ... and again, the algorithms will W ay create different solutions fewer depending upon tes ts! your testing objectives. etter ligh tly b ith s than we (W rage cove e now.) hav http://www.flickr.com/photos/thomashawk/2402194148 http://www.flickr.com/photos/thomashawk/2402190314 46
  47. 47. Combinatorial Software Test Design W ay The woman wanting fewer fewer tests should tes ts! create 2-way tests r bette htly Wi ig th sl n we ha ve (AKA ( ha age t .) c over now “pairwise” or “AllPairs” tests) http://www.flickr.com/photos/thomashawk/2402194148 47
  48. 48. Combinatorial Software Test Design This guy, wanting more coverage, should create more thorough combinatorial tests. (e.g., 3-way, 4-way, or multi-strength tests). http://www.flickr.com/photos/thomashawk/2402190314 48
  49. 49. OT Y N STS H W NT ? E S G ND I A E D YH B 49
  50. 50. Advantages of Combinatorial Software Test Design Test Design: Man vs. Machine (Once test inputs are known) Speed: Coverage: Efficiency: Human Weeks Gaps Way More Tests Computer Seconds 100%* Way Fewer Tests *100% of what? 100% of the desired test input combinations (e.g., all pairs of values for 2-way tests solutions, all triplets of possible values for 3-way solutions, etc.).
  51. 51. Coverage Advantages Whatever your test inputs, combinatorial testing makes it easy and fast to cover all of the specific combinations of test inputs that you suspect are relatively important. http://www.flickr.com/photos/kanaka/1798327442 51
  52. 52. Advantages of Combinatorial Software Test Design OK, but wouldn’t I have tested those relatively important combinations on my own? http://www.flickr.com/photos/johncarleton/4890239248
  53. 53. Advantages of Combinatorial Software Test Design Perhaps, but maybe you... Forget Run out of time Use more tests Accidentally than needed repeat yourself 53
  54. 54. Coverage Advantages Plus, using combinatorial testing, you’ll also get additional coverage from combinations that wouldn’t have made your list, including... http://www.flickr.com/photos/tk_five_0/2706278432 54
  55. 55. Coverage Advantages ...specialized use cases, No, its way cooler than pareto. http://www.flickr.com/photos/subspace-eddy/2766810770/ 55
  56. 56. Coverage Advantages ... unexpected combinations, http://www.flickr.com/photos/funky64/2875860978 56
  57. 57. Coverage Advantages ...notable, combinations that capture your attention, http://www.flickr.com/photos/brb_photography/3221049399 57
  58. 58. Coverage Advantages ...even combinations that rarely, if ever, actually occur, http://www.flickr.com/photos/grassvalleylarry/5536758 58
  59. 59. Coverage Advantages ... and... combinations that will take you into different parts of the application (and different states) than you would have otherwise thought to explore. http://www.flickr.com/photos/28041049@N00/227352411 59
  60. 60. Coverage Advantages Quickly testing unusual pairs is often surprisingly efficient and effective. Many unexpected two-way interactions cause problems. 60
  61. 61. Constraint Handling Combinations that can’t ever appear in reality? Yes. No problem. You can prevent them from appearing in your plans with “Invalid Pairs.” http://www.flickr.com/photos/kaptainkobold/4741725788/lightbox/ 61
  62. 62. E T ? Y Y E AD R 62
  63. 63. Ready? I’m ready already! Let’s do this! http://www.flickr.com/photos/johncarleton/4890241530 63
  64. 64. Hazards Not so fast. Hazards exist. To succeed, you must use common sense. http://www.flickr.com/photos/christianhaugen/3604106558 64
  65. 65. Limitations And let’s not kid ourselves. Not just anyone can do http://www.flickr.com/photos/mikebaird/573368639 this stuff. 65
  66. 66. Prerequisites It also requires good critical thinking skills. http://www.flickr.com/photos/johncarleton/9511759/ 66
  67. 67. Prerequisites To thrive, you must be able to develop models about how the systems you’re exploring http://www.flickr.com/photos/10ch/3101275536/ 67 work.
  68. 68. Practice Makes Perfect Training and practice will help you succeed. http://www.flickr.com/photos/mikebaird/2128091146
  69. 69. Questions ? http://www.flickr.com/photos/nathaninsandiego/3647136579 69
  70. 70. Ready? Let’s get started! http://www.flickr.com/photos/mikebaird/2996548709 70
  71. 71. D I X P E N AP http://www.flickr.com/photos/dannohung/332507807 71
  72. 72. Thank you Special thanks to the folks at: and its talented contributing photographers, including: Mike Baird ... and the inimitable John Carleton (who, let the record show, had no input into the cartoonish statements we added next to his striking expressions). 72
  73. 73. Why 2-way Testing is Effective Multiple thorough studies show approximately 85% of defects in production could have been detected by simply testing all possible pairs of values. Text Source: “Combinatorial Testing” IEEE Computer, Aug, 2009. Rick Kuhn, Raghu Kacker, Jeff Lei, Justin Hunter
  74. 74. Additional Information Excellent introductory articles and instructional videos: www.CombinatorialTesting.com Results of a ten-project study of effectiveness: https://www.hexawise.com/Combinatorial-Softwar-Testing-Case-Studies-IEEE-Computer-Kuhn-Kacker-Lei-Hunter.pdf Free trials of our firm’s combinatorial test design tool: www.hexawise.com/users/new (which stay free indefinitely for companies using 1-4 licenses). Also, please feel free to contact me if you have any questions. I’d be happy to quickly review a test plan or two, answer your questions, give quick pointers to help you run a pilot, etc. Seriously.... I enjoy helping people get started with this test design approach. Please don’t hesitate to reach out. There’s no charge and no catch. 74
  75. 75. Practice Tips: 4-Step Process Four-Step Process to Design Efficient and Effective Tests Scope Level  of  Detail Passive  Field  Treatment 1.  Plan Be  clear  about  single  or  mul3ple: Acceptable  op3ons:   Dis3nguish  between  important  fields  (par3cularly  those  that   will  trigger  business  rules)  and  unimportant  fields  in  the   -­‐    Features  /  Func3ons  /  Capabili3es -­‐    High  level  “search  for  something” applica3on. -­‐    User  types -­‐    Medium  level  “search  for  a  book” Quickly  document  what  your  approach  will  be  towards   -­‐    Business  Units -­‐    Detailed  “search  for  ‘Catcher  in  the  Rye’   passive  fields.  You  might  consider:  ignore  them  (e.g.,  don’t   -­‐    H/W  or  S/W  Configura3ons            by  its  3tle” select  any  Values  in  your  plan)  or  a  3  Value  approach  such  as   “valid”  “Invalid  (then  fix)”  and  “Blank  (then  fix)” Configura3ons Users Ac3ons 2.  Create First  add  hardware  configura3ons Next,  add  mul3ple  types  of  users  (e.g.,   Start  with  Big  Common  Ac3ons  made  by  users administrator,  customer,  special  customer) AZer  comple3ng  Big  Common  Ac3ons,  circle  back  and  add   Next  add  soZware  configura3ons Consider  permission  /  authority  levels  of  admin   Small  Ac3ons  and  Excep3ons users  as  well  as  business  rules  that  different   Remember  some  ac3ons  may  be  system-­‐generated users  might  trigger Business  Rules Gap  Filling Itera3on 3.  Refine Select  Values  to  trigger  bus.  rules Iden3fy  gaps  by  analyzing  different  states,   Refine  longest  lists  of  Values;  reduce  their  numbers  by  using   equivalence  classes,  etc. orders  of  opera3ons,  decision-­‐tree  outcomes   Iden3fy  equivalence  classes sought  vs.  delivered  by  tests,  “gap  hun3ng”   Create  Tests  with  and  w/out  borderline  Values;  consider  cost/ conversa3ons  w/  SME’s,  etc. benefit  tradeoffs  of  addi3onal  test  design  refinements Test  for  boundary  values Fill  gaps  by  either  (i)  adding  Parameters  and/or   Consider  stopping  tes3ng  aZer  reaching  ~80%  coverage   Mark  constraints  /  invalid  pairs Values  or  (ii)  crea3ng  “one-­‐off”  tests.   Consider  2-­‐way,  3-­‐way,  and  Mixed-­‐Strength  op3ons Auto-­‐Scrip3ng Expected  Results Con3nuous  Improvement 4.  Execute       Add  auto-­‐scrip3ng  instruc3ons  once;   Export  the  tests  into  Excel  when  you’re  done   If  possible  measure  defects  found  per  tester  hour  “with  and   without  Hexawise”  and  share  the  results apply  those  instruc3ons  to  all  of  the  tests   itera3ng  the  plan in  your  plan  instantly Add  inputs  based  on  undetected  defects Add  complex  Expected  Results  in  Excel  post-­‐ Don’t  include  complex  Expected  Results   export Share  good,  proven,  plan  templates  with  others in  auto-­‐scripts   © Hexawise, 2010. All rights reserved. 26
  76. 76. Practice Tips: Warning Signs “You might be headed for trouble if...” Scope Level  of  Detail Passive  Field  Treatment 1.  Plan ...  You  cannot  clearly  describe  both  the   ...  Parameters  with  the  most  Values  have  more   ...  You  cannot  clearly  describe  your  strategy  to  deal  with   scope  of  your  test  plan  and  what  will  be   Values  than  they  require.    365  values  for  “days   unimportant  details.    If  Values  will  impact  business  rules,  focus   leZ  out  of  scope. of  the  year”  is  bad.    Instead,  use  equivalence   on  them.    If  Values  don’t  impact  business  rules,  consider   class  Values  like  “weekend”  &  “weekday.”    When   ignoring  them.   in  doubt,  choose  more  Parameters  and  fewer   Values. Configura3ons Users Ac3ons 2.  Create ...  You  have  ignored  hardware  and   ...  You  have  not  included  all  the  different  types   ...  You  start  entering  Small  Ac3ons  (e.g.,  “search  for  a   soZware  configura3ons    without  first   of  users  necessary  to  trigger  different  business   hardback  science  book  by  author  name”)  before  you  enter  Big   confirming  this  approach  with   rules.    What  user  types  might  create  different   Ac3ons  (e.g.,  “Put  Something  in  Cart.    Buy  it.”)    First  go  from   stakeholders. outcomes?    Authority  level?  Age?  Loca3on?     beginning  to  end  at  a  high  level.    AZer  you’ve  done  that,  feel   Income?    Customer  status? free  to  add  more  details.   Business  Rules Gap  Filling Itera3on 3.  Refine ...  You  forget  to  iden3fy  invalid  pairs.          -­‐   ...  You  assume  that  the  test  condi3ons  coming   ...  You  forget  to  look  at  the  Coverage  Analysis  charts.    If  you   or  -­‐    ...  You  rely  only  on  Func3onal   out  of  Hexawise  will  be  100%  of  the  tests  you   achieve  80%  coverage  in  the  first  quarter  of  the  tests,  you   Requirements  and  Tech  Specs  w/out   should  run.    There  might  well  be  addi3onal   should  measure  the  cost/benefit  implica3ons  of  execu3ng  the   thinking  hard  yourself  and  asking   “one-­‐off”  things  that  you  should  test  and/or  a   last  3/4  of  the  tests. ques3ons  to  SME’s  about  business  rules   few  nega3ve  tests  to  design  by  hand. and  outcomes  that  are  not  yet  triggered. Auto-­‐Scrip3ng Expected  Results Con3nuous  Improvement 4.  Execute ...  You  add  detailed  Expected  Results  in   ...  You  invest  a  lot  of  3me  in  calcula3ng  and   ...  You  don’t  ask  (when  defects  that  the  tests  missed  are  found   the  tests.                                                                                                          -­‐   documen3ng  Expected  Results  before  you  have   post-­‐tes3ng)  “What  input  could  have  been  added  to  the  test   or  -­‐  ...  You  forget  that  this  feature  exists   plan  to  detected  this?”  “Should  I  add  that  input  to  the   and  find  yourself  typing  out  test-­‐by-­‐test   determined  your  “final  version”  Parameters  and   instruc3ons  one-­‐by-­‐one. Values.    Last  minute  addi3ons  to  inputs  will   Hexawise  test  plan  now  to  improve  it  in  advance  of  the  next   jumble  up  test  condi3ons  for  most  test  cases. 3me  it  is  used?”   © Hexawise, 2010. All rights reserved. 27

×