Your SlideShare is downloading. ×
0
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Orthogonal array testing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Orthogonal array testing

18,237

Published on

2 Comments
1 Like
Statistics
Notes
  • Hi,

    What if each input has different number of variables?
    Say: Inputs - A , B & C
    A has 9 variables , B has 10 & C has 2

    - Thanks
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Prince,

    Nice presentation on an important topic. Don't forget Hexawise for OA / orthogonal array / pairwise testing. It is far easier to use than almost all the tools, is more powerful too, updated more frequently than any other tool, and it contains unparalleled help files at help.hexawise.com, sample instructional plans in the tool itself, and even test design training resources at training.hexawise.com.

    Hexawise is free for small teams to use and serious money for large companies to secure enterprise licenses.

    As of September, 2013, Hexawise is being used at more than 100 Fortune 500 firms and thousands of smaller firms.

    hexawise.com/free
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
18,237
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
403
Comments
2
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. ORTHOGONAL ARRAY TESTING Prince Bhanwra [801031024] Ravinder Dahiya [801031025] PRESENTED BY :
  • 2. Contents
    • Software Testing
    • Conventional Test Cases
      • Conventional Testing Issues
    • Software Faults
    • Why OATS..??
    • Terminology of OATS
    • Examples
    • OATS Advantages
    • OATS Limitations
    • Challenges
    • Mixed Mode OA
    • Available Test Automation Tools
    • References
    Thapar University, Patiala
  • 3. Testing
    • SOFTWARE TESTING
    • Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test.
    • CONVENTIONAL / EXHAUSTIVE TESTING
    • A test approach in which the test suite comprises all combinations of input values and preconditions.
    Thapar University, Patiala
  • 4. Conventional Test Cases
    • Example:
    • If we have three variables (A,B,C), each can have 3 values say (Red, Green, and Blue).
    • The possible combinations in conventional test cases would be 27 i.e. 3 3
    Thapar University, Patiala
  • 5. Conventional Testing Issues
    • Conventional Test Cases:
      • Variables:3
      • Input: 4
      • Possible cases: 81=3 4
      • Variables: 3
      • Input: 5
      • Possible Cases: 243 = 3 5
      • Variables: 5
      • Input: 5
      • Possible Cases: 3125 = 5 5
    Thapar University, Patiala
    • Orthogonal Test Cases
      • Variables:3
      • Input: 4
      • Possible cases: 9
      • Variables: 3
      • Input: 5
      • Possible Cases: 11
      • Variables: 5
      • Input: 5
      • Possible Cases: 21
  • 6. Software Faults Thapar University, Patiala
    • Region Faults.
    • Isolated Faults.
    Orthogonal Array based testing is highly effective for the detection of region faults with a relatively small number of tests .
  • 7. Why Orthogonal Array Testing (OATS)..??
    • Systematic, statistical way to test pair-wise interactions.
    • Interactions and integration points are a major source of defects.
    • Most defects arise from simple pair-wise interactions.
        • “ When the background is blue and the font is Arial and the layout has menus on the right and the images are large and it’s a Thursday then the tables don’t line up properly.”
    • Exhaustive testing is impossible.
    • Execute a well-defined, concise set of tests that are likely to uncover most (not all) bugs.
    • Orthogonal approach guarantees the pair-wise coverage of all variables.
    Thapar University, Patiala
  • 8. Terminology for working with OA’s
    • OA’s are commonly represented as :
    • L Runs (Levels Factors ) or OA(Runs, Factors, Levels, Strength) or
    • OA λ (Runs(N); Factors(k), Levels(v), Strength(t))
    • is an N × k array on v symbols such that every N × t sub-array contains all tuples of size t from v symbols exactly λ times.
    • Runs (N) – Number of rows in the array , which translates into the number of Test Cases that will be generated.
    • Factors (k) – Number of columns in the array , which translates into the maximum number of variables that can be handled by the array.
    • Levels (v) – Maximum number of values that can be taken on by any single factor.
    • Strength (t) – The number of columns it takes to see all the possibilities equal number of times.
    • ƒ λ -1 for software testing and is often omitted
    Thapar University, Patiala
  • 9. Example 1 Thapar University, Patiala A Web Page has three distinct sections (Top, Middle, Bottom) that can be individually shown or hidden from user No.of Factors=3 (Top,middle,Bottom) No.of Levels =2 (Hidden or shown) Array Type =L 4 (2 3 ) or OA(4,3,2,2) If we go for exhaustive testing we need : 2 x 2 x 2 = 8 Test Cases OA(Runs, Factors, Levels, Strength)
  • 10. Example 1 (contd..) Thapar University, Patiala Fixed Level Array: L 4 (2 3 ) L 4 2 3 – OA with 4 Runs 3 factors with 2 levels The Four Test Scenarios (4 Vs. 8) 1 - Display the home page and hide all sections. 2 - Display the home page and show all but the Top section. 3 - Display the home page and show all but the Middle section. 4 - Display the home page and show all but the Bottom section. F1 F2 F3 Run1 0 0 0 Run 2 0 1 1 Run 3 1 0 1 Run 4 1 1 0 Top Middle Bottom Test 1 Hidden Hidden Hidden Test 2 Hidden Visible Visible Test 3 Visible Hidden Visible Test 4 Visible Visible Hidden
  • 11. Example 2
    • Example:
    • If we have three variables (A,B,C), each can have 3 values say (Red, Green, and Blue).
    • The possible combinations in OATS test cases would be 9.
    Thapar University, Patiala
  • 12. OATS advantage to select a test set:
    • Guarantees testing the pair-wise combinations of all the selected variables.
    • Creates an efficient and concise test set with many fewer test cases than testing all combinations of all variables.
    • Creates a test set that has an even distribution of all pair-wise combinations.
    • Exercises some of the complex combinations of all the variables.
    • Is simpler to generate and less error prone than test sets created by hand.
    Thapar University, Patiala
  • 13. OATS basic fault model:
    • Interactions and integrations are a major source of defects.
    • Most of these defects are not a result of complex interactions such as
      • “ When the background is blue and the font is Arial and the layout has menus on the right and the images are large and it’s a Thursday then the tables don’t line up properly.”
    • Most of these defects arise from simple pair-wise interactions such as
      • “ When the font is Arial and the menus are on the right the tables don’t line up properly.”
    Thapar University, Patiala
  • 14. OATS basic fault model (contd..):
    • With so many possible combinations of components or settings, it is easy to miss one.
    • Randomly selecting values to create all of the pair-wise combinations is bound to create inefficient test sets and test sets with random, senseless distribution of values.
    • It covers 100% (9 of 9) of the pair-wise combinations, 33% (9 of 27) of the three-way combinations and 11% (9 of 81) of the four-way combinations.
    Thapar University, Patiala
  • 15. The Challenges
    • The number of tests may increase rapidly as the degree of interactions increases
    • ‰ Test generation algorithms must be more sensitive in terms of both time and space requirements
    • ‰ The need for test automation becomes even more serious.
      • Impractical to manually execute and inspect the results of a large number of test runs
      • However, on the Web there are pre-calculated orthogonal array tables for certain numbers of variables and values.
    Thapar University, Patiala
  • 16. Mixed Mode OA Thapar University, Patiala
  • 17. Available Test Automation Tools Thapar University, Patiala 1. CATS (Constrained Array Test System)   *) [Sherwood] Bell Labs. 2. OATS (Orthogonal Array Test System)   *) [Phadke] AT&T 3. AETG Telecordia Web-based, commercial 4. IPO (PairTest)   *) [Tai/Lei] 5. TConfig [Williams] Java-applet 6. TCG (Test Case Generator)   *) NASA 7. AllPairs Satisfice Perl script, free, GPL 8. Pro-Test SigmaZone GUI, commercial 9. CTS (Combinatorial Test Services) IBM Free for non-commercial use
  • 18. References
    • http://csrc.nist.gov/groups/SNS/acts/documents/ipo-nist.pdf
    • http://csrc.nist.gov/groups/SNS/acts/documents/aberdeen.ppt
    • http://ocw.mit.edu/courses/aeronautics-and-astronautics/16-881-robust-system-design-summer-1998/lecture-notes/l3_matrix_experiments4.pdf
    • http://ocw.mit.edu/courses/aeronautics-and-astronautics/16-881-robust-system-design-summer-1998/lecture-notes/l8_orth_arrays.pdf
    • http://delivery.acm.org/10.1145/1290000/1283384/p1-abam.pdf?key1=1283384&key2=5753427921&coll=DL&dl=ACM&CFID=9321178&CFTOKEN=33991028
    • http://www.developsense.com/pairwiseTesting.html
    • http://learnsoftwareprocesses.com/2007/08/21/orthogonal-array-testing-strategy-oats/comment-page-1/#comment-7394
    • http://www.vietnamesetestingboard.org/zbxe/?document_srl=219676
    • http://www.pairwise.org/tools.asp
    Thapar University, Patiala
  • 19.
    • Queries..?
    Thapar University, Patiala
  • 20.
    • Thank you !!
    Thapar University, Patiala

×