Real-world Testing of Object-oriented Software
Upcoming SlideShare
Loading in...5
×
 

Real-world Testing of Object-oriented Software

on

  • 572 views

 

Statistics

Views

Total Views
572
Views on SlideShare
572
Embed Views
0

Actions

Likes
0
Downloads
11
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Real-world Testing of Object-oriented Software Real-world Testing of Object-oriented Software Document Transcript

  • Real-world Testing of Object-oriented Software Steve Tockey stevet@construx.com http://www.construx.com © 2000 Construx Software Builders Inc. All Rights Reserved. Outline v Basics of Software Testing v Object-oriented Stuff v Testing in the Context of Object-orientation v Black-box Testing at the Operation Level v White-box Testing at the Method Level v Black-box Testing at the Class Level v Further Testing Construx Software (www.construx.com) 2 Basics of Software Testing
  • The Cost of Defects Cost to Correct Phase That a Defect Is Created Requirements Architecture Detailed design Construction Requirements Architecture Detailed Construction Maintenance design Phase That a Defect Is Corrected Construx Software (www.construx.com) 4 The Reality of Testing v “It only takes one failed test to show that the software doesn’t work, but even an infinite number of tests won’t prove that it does” [Beizer90] p5 Construx Software (www.construx.com) 5 Test Coverage, defined v “Coverage” is a measure of test set thoroughness u In other words, how much of some reference specification is exercised by a given set of test cases? Are there parts of the reference specification that would not be tested by those test cases? v Test sets that do not cover the reference specification are, according to that criterion, inadequate u But coverage in terms of one criterion usually does not guarantee adequacy in terms of any other coverage criteria One key to efficient testing is to achieve some desired One key to efficient testing is to achieve some desired flavor of coverage from the fewest possible test cases flavor of coverage from the fewest possible test cases Construx Software (www.construx.com) 6
  • Two Families of Testing v Functional (“Black-box”) Testing u The test designer is unaware of the internal structure of the software being tested v Structural (“White-box”) Testing u The test designer knows about the internal structure of the software and uses that knowledge to devise tests “Functional tests can, in principle, detect all [defects] but would take “Functional tests can, in principle, detect all [defects] but would take infinite time to do so. Structural tests are inherently finite but cannot infinite time to do so. Structural tests are inherently finite but cannot detect all [defects], even if completely executed” [Beizer90] p11 detect all [defects], even if completely executed” [Beizer90] p11 Construx Software (www.construx.com) 7 A Generic Software Lifecycle “System Acceptance Acceptance Objectives” Test Planning Test Execution System System Requirements Test Planning Test Execution Integration and Integration and Design Component Component Test Planning Test Execution Unit Test Unit Test Coding Planning Execution Construx Software (www.construx.com) 8 Object-oriented Stuff
  • Introducing Objects - the Client Perspective Operation 1 Identity Operation Operation 6 2 Persistent State Operation Operation 5 3 Operation 4 Construx Software (www.construx.com) 10 Software Objects - the Client Perspective // Create an account whose identity is Account_000295 and whose // persistent state is “opened with balance = $100.00” BankAccount Account_000295; Account_000295 = BankAccount ( 100.00f ); // Deposit $25.00 into the account Account_000295.deposit ( 25.00f ); // message is “deposit (25.00f)” // the balance is now $125.00 // this reply is implicit // Take out $75.00 from the account if ( Account_000295.withdraw ( 75.00f ) ) { system.out.println ( “Successful withdraw of $75.00” ); } else { system.out.println ( “Unsuccessful withdraw of $75.00” ); }; // In this case, the message is “withdraw ( 75.00f )” // the reply is a boolean (true = success, false = failure) // Of course, the reply should be true and the balance should be $50 // Close the account float refund = Account_000295.close (); // In this case, the message is “close ()” // the reply is a float ( =>0 means success and is the closing // balance, <0 means failure ) // Of course, the reply should be $50 and the persistent state of // Account_000295 should be closed with balance = $0.00 ... Construx Software (www.construx.com) 11 Introducing Objects - the Supplier Perspective Operation 1 Identity Oper. Method Oper. 6 A 2 Method Attribute P Method D Attribute Q B ... Oper. Method Oper. 5 3 C Operation 4 Construx Software (www.construx.com) 12
  • Software Objects - the Supplier Perspective public class BankAccount { public boolean withdraw ( float anAmount ) { if ( accountStatus && currentBalance > anAmount ) { boolean accountStatus; // false = closed, true = opened currentBalance = currentBalance - anAmount; float currentBalance; return true; } else { public BankAccount ( float initialBalance ) { return false; // User error, not enough money accountStatus = true; } currentBalance = initialBalance; } } public float examineBalance ( ) { public void open ( float initialBalance ) { if ( accountStatus ) { if ( !accountStatus ) { return currentBalance; accountStatus = true; } else { currentBalance = initialBalance; return -1.0f; // User error, the account is closed } } } } public void deposit ( float anAmount ) { public float close ( ) { if ( accountStatus ) { if ( accountStatus ) { currentBalance = currentBalance + anAmount; accountStatus = false; } return currentBalance; } } else { return -1.0f; // User error, the account is already closed } } } Construx Software (www.construx.com) 13 Language Pragmatics v Encapsulation v Classes and instantiation v Inheritance v Polymorphism and run-time binding Construx Software (www.construx.com) 14 Inheritance public class CheckingAccount extends BankAccount { float perCheckFee; public CheckingAccount ( float initialBalance, float fee ) { super ( initialBalance ); perCheckFee = fee; } public void open ( float initialBalance, float fee ) { if ( !accountStatus ) { accountStatus = true; currentBalance = initialBalance; perCheckFee = fee; } } public boolean withdraw ( float anAmount ) { if ( accountStatus && currentBalance > (anAmount + perCheckFee) ) { currentBalance = currentBalance - (anAmount + perCheckFee); return true; } else { return false; } } } Construx Software (www.construx.com) 15
  • Testing in the Context of Object-orientation A New Level for Testing “System Acceptance Acceptance Objectives” Test Planning Test Execution System System Requirements Test Planning Test Execution Integration and Integration and Design Component Component Test Planning Test Execution Unit Test Unit Test Coding Planning Execution Construx Software (www.construx.com) 17 Inheritance - “Flattened” Classes public boolean withdraw ( float anAmount ) { public class CheckingAccount { if ( accountStatus && currentBalance > (anAmount + perCheckFee) ) { boolean accountStatus; // false = closed, true = opened currentBalance = currentBalance - (anAmount + float currentBalance; perCheckFee); float perCheckFee; return true; } else { public CheckingAccount ( float initialBalance, float fee ) { return false; accountStatus = true; } currentBalance = initialBalance; } perCheckFee = fee; } public float examineBalance ( ) { if ( accountStatus ) { public void open ( float initialBalance, float fee ) { return currentBalance; if ( !accountStatus ) { } else { accountStatus = true; return -1.0f; // User error, the account is closed currentBalance = initialBalance; } perCheckFee = fee; } } } public float close ( ) { if ( accountStatus ) { public void deposit ( float anAmount ) { accountStatus = false; if ( accountStatus ) { return currentBalance; currentBalance = currentBalance + anAmount; } else { } return -1.0f; // User error, the account is already } closed } } } Construx Software (www.construx.com) 18
  • Inheritance - Avoiding Encapsulation public void testBankAccount ( ) { // create and initialize the account BankAccount accountToTest = BankAccountScaffold( ); public class BankAccountScaffold extends BankAccount { // Test case 1 - Open a closed account with $100.00 accountToTest.setStatus ( false ); public void setStatus ( boolean newStatus ) { accountToTest.setBalance ( 0.00f ); accountStatus = newStatus; accountToTest.open ( 100.00f ); } if ( !accountToTest.getStatus ) { system.out.println ( “Case 1: error in status” ) } public boolean getStatus ( ) { if ( accountToTest.examineBalance <> 100.00f ) { return accountStatus; system.out.println ( “Case 1: error in balance” ) } } // Test case 2 - Deposit $50 with $200 already in public void setBalance ( float newBalance ) { accountToTest.setStatus ( true ); currentBalance = newBalance; accountToTest.setBalance ( 200.00f ); } accountToTest.deposit ( 50.00f ); if ( !accountToTest.getStatus ) { // note: getBalance already exists as examineBalance system.out.println ( “Case 2: error in status” ) } if ( accountToTest.examineBalance <> 250.00f ) { } system.out.println ( “Case 2: error in balance” ) } // Test case 3 - Withdraw $50 with $200 already in ... // Test case 4 - ... ... } Construx Software (www.construx.com) 19 Black-box Testing at the Operation level Domain Coverage v A variable’s domain is the set of values that it can take on v Domain Coverage means that there is a sufficient set of test cases to test the domain of each parameter (on each software function) u Rather than testing each possible value in a variable’s domain, we will use “Equivalence Classes” Construx Software (www.construx.com) 21
  • Equivalence Classes v Suppose some weather software has two input variables u Temperature Minimum Maximum Allowed Allowed Temperature 0F 32 F Temperature Equivalence Equivalence Equivalence Equivalence Equivalence Class A Class B Class C Class D Class E (Invalid) (Invalid) u Cloudiness ↓ Valid equivalence class (Clear, Partly Cloudy, Mostly Cloudy, Overcast) ↓ Invalid equivalence class - anything else Construx Software (www.construx.com) 22 Boundary Value Analysis v “Boundary condition” defects are so common that special test cases should be designed just for them u Be sure to have (or create one) a positive test case for the lowest valid value in each equivalence class and another for the highest valid value u Be sure to have (or create one) a negative test case for the predecessor of the lowest valid value and for the successor of the highest valid value v Zero is another common trouble spot on numeric inputs u Have test cases for -1, 0, and +1 Construx Software (www.construx.com) 23 Example Black-box Test Cases Temperature Cloudiness Expected Result -120 Clear … 1 Partly Cloudy … 180 Overcast … -121 Clear Temp too low 181 Overcast Temp too high 90 "Sunny" Invalid Cloudiness Also test -1, 0, 31, 32, and 33 Construx Software (www.construx.com) 24
  • White-box Testing at the Method level Control-flow Testing v The software structure is viewed from a flow-of- control (i.e., instruction sequencing) perspective u Control-flow testing assumes that there are defects in the software’s flow-of-control logic Construx Software (www.construx.com) 26 Control-flow Coverage Case Study, Input and Output 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Input Output Construx Software (www.construx.com) 27
  • Control-flow Coverage Case Study, Code procedure Number_Puzzle; var Across: Across_Index; Down: Down_Index; Clue_Number: integer; begin Clue_Number := 1; for Down := 1 to Down_Size do for Across := 1 to Across_Size do if ( Puzzle[ Across, Down ].Color = White ) and ( ( ( Puzzle[ Across-1, Down ].Color = Black ) and ( Puzzle[ Across+1, Down ].Color = White ) ) or ( ( Puzzle[ Across, Down-1 ].Color = Black ) and ( Puzzle[ Across, Down+1 ].Color = White ) ) ) then begin Puzzle[ Across, Down ].Number := Clue_Number; Clue_Number := Clue_Number + 1 end; end; Construx Software (www.construx.com) 28 Example Control-flow Graph En a 1 b f 2 d 3 h (else) i c e 4 Ex g 5 (then) Construx Software (www.construx.com) 29 Control-flow Coverage Criteria v Statement Coverage v Decision (Branch) Coverage v Condition/Decision Coverage v Modified Condition/Decision Coverage v Multiple Condition Coverage v Path Coverage Construx Software (www.construx.com) 30
  • Data-flow Testing v “It is our belief that just as one would not feel confident about a program without executing every statement in it as part of some test, one should not feel confident about a program without having seen the effect of using the value produced by each and every computation” [Rapps82] “The data-flow criteria were developed to detect errors in “The data-flow criteria were developed to detect errors in data usage and concentrate on the interactions between data usage and concentrate on the interactions between variable definition and reference” [Chilenski94] variable definition and reference” [Chilenski94] Construx Software (www.construx.com) 31 An Example Data-flow Graph d ud 1 2 3 4 5 6 7 u 11 8 (k?) u 14 13 10 9 u 12 Construx Software (www.construx.com) 32 Data-flow Coverage Criteria v All Definitions (Reach) Coverage v All Uses Coverage v All DU Paths Coverage Construx Software (www.construx.com) 33
  • Black-box Testing at the Class level State-transition Models Closed Delete Open Withdraw(Amt) Close [Amt<=Balance] Deposit(Amt) Opened Withdraw(Amt) Deposit(Amt) [Amt>Balance] [Balance+Amt>=0.00] Deposit(Amt) Overdrawn [Balance+Amt<0.00] Construx Software (www.construx.com) 35 Further Testing
  • Further Testing v White-box testing at the class level v Integration testing of multiple classes v System testing v Acceptance testing Construx Software (www.construx.com) 37 Summary References v [Beizer90] Boris Beizer, Software Testing Techniques, 2nd Ed., Van Nostrand Reinhold, 1990 v [Chilenski94] John Chilenski, Steven Miller, "Applicability of Modified Condition/Decision Coverage to Software Testing", Software Engineering Journal, September, 1994 v [Rapps82] S Rapps, E J Weyuker, "Data Flow Analysis Techniques for Test Data Selection", Sixth International Conference on Software Engineering, Tokyo, Japan, September, 1982 v see also http://www.rbsc.com Construx Software (www.construx.com) 39