WHITE BOX TESTING
Ms. Neethu Tressa
Assistant Professor
WHITE BOX TESTING
• Way of testing the external functionality of the code
• Also known as clear box, glass box or open box
• Consider program code, code structure, internal design flow
• Defects come about because of:
• Incorrect translation of requirements and design into program code
• Programming errors and programming language idiosyncrasies
• Classification:
1. Static Testing
2. Structural Testing
Classification of White Box Testing
STATIC TESTING
• Require only source code, not binaries or executables
• Two ways to perform:
• By humans
• With the help of specialized tools
• Involves testing to find out whether
• The code works according to the functional requirements
• The code has been written in accordance with the design developed earlier in
the project life cycle
• The code for any functionality has been missed out
• The code handles errors properly
Static Testing by Humans
• Humans read program code to detect errors
• Advantages:
• Humans can find errors that computers cannot
• By making multiple humans read and evaluate the program, we can get
multiple perspectives
• A human evaluation of the code can compare it against the specifications or
design to ensure that it does what is intended to do
• A human evaluation can detect many problems at one go and can identify the
root causes of the problems
• By making humans test the code before execution, computer resources can
be saved
• This proactive testing method minimizes the delay in identification of the
problems
• Finding defects later can create pressure on programmers resulting in higher
chances of other defects creeping in
Static Testing by Humans (Contd..)
• Different methods:
1. Desk Checking of the code
2. Code Walkthrough
3. Code Review
4. Code Inspection
Desk Checking
• Done manually by the author of the code before compiling and
executing
• Verifies the portions of the code for correctness on the spot by
comparing with design or specifications
• Completely relies on author’s thoroughness, diligence and skills
• This method is characterized by:
• No structured method or formalism to ensure completeness
• No maintaining of a log or checklist
• Advantages:
• An expert programmer is well equipped to read and understand own code
• Since this is done individually, there are fewer scheduling and logistics
overheads
• Defects are detected and corrected with minimum time delay
Desk Checking (Contd…)
• Disadvantages:
• While detecting problems, the developer may be tunnel visioned and have
blind spots to certain types of problems
• Developers generally prefer to write new code rather than testing
• Since its person-dependent and informal, it may not work consistently across
all developers
Code Walkthrough
• Group oriented method bringing multiple perspectives
• Set of people look at the program code and raise questions for author
for which he has to answer or find answer
Formal Inspection
• Code inspection/Fagan inspection
• Focus is to detect all faults, violations and other side-effects
• Defects are detected by:
• Demanding thorough preparation before inspection/review
• Enlisting multiple diverse views
• Assigning specific roles to multiple participants
• Going sequentially through the code in a structured manner
Formal Inspection (Contd..)
• Should take place after desk checking and walkthroughs
• When the code is ready, arrange an inspection meeting
• Roles:
1. Author
2. Moderator – formally run the inspection
3. Inspectors – provide review comments for the code
4. Scribe – take detailed notes during meeting and circulate to inspection team
• Author or moderator selects review team
• Inspectors get hard/soft copies of code with other supporting
documents
• Moderator informs about date, time and venue of inspection
meeting(defect logging meeting)
Formal Inspection (Contd..)
• Inspectors raise defects classified in two dimensions – minor/major
and systemic/misexecution
• Misexecution – happens due to error or slip on the part of the author
• Systemic – machine specific errors
• Minor – may not substantially affect a program
• Major – need immediate attention
• In case defects are severe, the team may optionally call for a review
meeting to ensure that they address the problems
Combining various methods
• Challenges in formal inspections:
• Time consuming
• Logistics and scheduling can become an issue
• Not always possible to go through every line of code
• Portions of code classified into:
• High, medium and low
• “High or Medium” should be subject to formal inspections
• “Low” can be subject to either walkthroughs or desk checking
Static Analysis Tools
• Extension of compilers
• Reduce manual work and analyze the code to find errors such as:
• Whether there are unreachable codes
• Variables declared but not used
• Mismatch in definition and assignment of values to variables
• Illegal or error prone type casting of variables
• Use of non-portable or architecture-dependent programming constructs
• Memory allocated but not having corresponding statements for freeing them
up memory
• Calculation of cyclomatic complexity
STRUCTURAL TESTING
• Consider code, code structure, internal design and how they are
coded
• Tests are actually run by the computer on the built product
Unit/Code Functional Testing
• Initial part of structural testing before subjecting to code coverage
and code complexity testing
• Performs quick checks in several methods:
• Developer can perform obvious tests, knowing inputs and corresponding
outputs. By repeating this test, confidence level of the developer increases
• For modules with complex logic or conditions, developer can build a “debug
version” by putting intermediate print statements
• Run the product under a debugger or IDE
Code Coverage Testing
• Involves designing and executing test cases and finding out the
percentage of code covered by testing using instrumentation of code
• Instrumentation does the following:
• Rebuilds the product, linking the product with a set of libraries
• Instrumented code can monitor and find the portions of code covered
• Allow reporting on the portions of the code covered frequently
• Types of coverage:
• Statement Coverage
• Path Coverage
• Condition Coverage
• Function Coverage
Statement Coverage
• Writing test cases that execute each of the program statements
• applies on the following program constructs:
• Sequential control flow – test cases to run through top to bottom
• Two-way decision statements(if then else) – separate test cases for if and then parts
• Multi-way decision statements(switch) – multiple test cases for different switch
cases
• Loops – design test cases to:
• Skip the loop completely
• Check “normal” operations
• Cover the loop around the “boundary”
• Statement Coverage = Total statements exercised X 100
Total number of executable statements in program
Path Coverage
• Split a program into a number of distinct paths
• Paths:
• A
• B-D-G
• B-D-H
• B-C-E-G
• B-C-E-H
• B-C-F-G
• B-C-F-H
• Path Coverage = Total paths exercised X 100
Total number of paths in program
Condition Coverage
• Design test cases that exercise each Boolean expression
• Percentage of conditions covered by a set of test cases
• Condition Coverage = Total decisions exercised X 100
Total number of decisions in program
Function Coverage
• Identify how many program functions are covered by test cases
• Improve performance and quality of the product
• Advantages:
• Functions are easier to identify and hence easier to write test cases
• Due to higher level of abstraction, easier to achieve 100 percent function coverage
• By having more logical mapping to requirements, more direct correlation to the test
coverage of the product
• Easier to prioritize the functions based on the importance of the requirements they
realize
• Provides natural transition to black box testing
• Function Coverage = Total functions exercised X 100
Total number of functions in program
Code Complexity Testing
• Cyclomatic Complexity – metric that quantifies the complexity of a program
• Identify independent paths
• Find out the upper bound on number of tests that cover all the statements
• Represent a program as a flow graph consisting of nodes and edges as
follows:
• Identify the predicates or decision points
• Make the predicates simple
• Combine all sequential statements to single node
• When sequential statements are followed by a simple predicate, create a predicate
node by combining them
• Add a node to represent all sequential statements at the end of the program
• Cyclomatic Complexity = Number of predicate nodes + 1
OR
• Cyclomatic Complexity = E-N+2
• Independent Path – path that has at least one edge not traversed before in
other paths
• Basis Set – set of independent paths that cover all the edges
Calculating and Using Cyclomatic Complexity
CHALLENGES IN WHITE BOX TESTING
• Human tendency of a developer being unable to find the defects in
his/her code
• Fully tested code may not correspond to realistic scenarios
White box testing

White box testing

  • 1.
    WHITE BOX TESTING Ms.Neethu Tressa Assistant Professor
  • 2.
    WHITE BOX TESTING •Way of testing the external functionality of the code • Also known as clear box, glass box or open box • Consider program code, code structure, internal design flow • Defects come about because of: • Incorrect translation of requirements and design into program code • Programming errors and programming language idiosyncrasies • Classification: 1. Static Testing 2. Structural Testing
  • 3.
  • 4.
    STATIC TESTING • Requireonly source code, not binaries or executables • Two ways to perform: • By humans • With the help of specialized tools • Involves testing to find out whether • The code works according to the functional requirements • The code has been written in accordance with the design developed earlier in the project life cycle • The code for any functionality has been missed out • The code handles errors properly
  • 5.
    Static Testing byHumans • Humans read program code to detect errors • Advantages: • Humans can find errors that computers cannot • By making multiple humans read and evaluate the program, we can get multiple perspectives • A human evaluation of the code can compare it against the specifications or design to ensure that it does what is intended to do • A human evaluation can detect many problems at one go and can identify the root causes of the problems • By making humans test the code before execution, computer resources can be saved • This proactive testing method minimizes the delay in identification of the problems • Finding defects later can create pressure on programmers resulting in higher chances of other defects creeping in
  • 6.
    Static Testing byHumans (Contd..) • Different methods: 1. Desk Checking of the code 2. Code Walkthrough 3. Code Review 4. Code Inspection
  • 7.
    Desk Checking • Donemanually by the author of the code before compiling and executing • Verifies the portions of the code for correctness on the spot by comparing with design or specifications • Completely relies on author’s thoroughness, diligence and skills • This method is characterized by: • No structured method or formalism to ensure completeness • No maintaining of a log or checklist • Advantages: • An expert programmer is well equipped to read and understand own code • Since this is done individually, there are fewer scheduling and logistics overheads • Defects are detected and corrected with minimum time delay
  • 8.
    Desk Checking (Contd…) •Disadvantages: • While detecting problems, the developer may be tunnel visioned and have blind spots to certain types of problems • Developers generally prefer to write new code rather than testing • Since its person-dependent and informal, it may not work consistently across all developers
  • 9.
    Code Walkthrough • Grouporiented method bringing multiple perspectives • Set of people look at the program code and raise questions for author for which he has to answer or find answer Formal Inspection • Code inspection/Fagan inspection • Focus is to detect all faults, violations and other side-effects • Defects are detected by: • Demanding thorough preparation before inspection/review • Enlisting multiple diverse views • Assigning specific roles to multiple participants • Going sequentially through the code in a structured manner
  • 10.
    Formal Inspection (Contd..) •Should take place after desk checking and walkthroughs • When the code is ready, arrange an inspection meeting • Roles: 1. Author 2. Moderator – formally run the inspection 3. Inspectors – provide review comments for the code 4. Scribe – take detailed notes during meeting and circulate to inspection team • Author or moderator selects review team • Inspectors get hard/soft copies of code with other supporting documents • Moderator informs about date, time and venue of inspection meeting(defect logging meeting)
  • 11.
    Formal Inspection (Contd..) •Inspectors raise defects classified in two dimensions – minor/major and systemic/misexecution • Misexecution – happens due to error or slip on the part of the author • Systemic – machine specific errors • Minor – may not substantially affect a program • Major – need immediate attention • In case defects are severe, the team may optionally call for a review meeting to ensure that they address the problems
  • 12.
    Combining various methods •Challenges in formal inspections: • Time consuming • Logistics and scheduling can become an issue • Not always possible to go through every line of code • Portions of code classified into: • High, medium and low • “High or Medium” should be subject to formal inspections • “Low” can be subject to either walkthroughs or desk checking
  • 13.
    Static Analysis Tools •Extension of compilers • Reduce manual work and analyze the code to find errors such as: • Whether there are unreachable codes • Variables declared but not used • Mismatch in definition and assignment of values to variables • Illegal or error prone type casting of variables • Use of non-portable or architecture-dependent programming constructs • Memory allocated but not having corresponding statements for freeing them up memory • Calculation of cyclomatic complexity
  • 14.
    STRUCTURAL TESTING • Considercode, code structure, internal design and how they are coded • Tests are actually run by the computer on the built product
  • 15.
    Unit/Code Functional Testing •Initial part of structural testing before subjecting to code coverage and code complexity testing • Performs quick checks in several methods: • Developer can perform obvious tests, knowing inputs and corresponding outputs. By repeating this test, confidence level of the developer increases • For modules with complex logic or conditions, developer can build a “debug version” by putting intermediate print statements • Run the product under a debugger or IDE
  • 16.
    Code Coverage Testing •Involves designing and executing test cases and finding out the percentage of code covered by testing using instrumentation of code • Instrumentation does the following: • Rebuilds the product, linking the product with a set of libraries • Instrumented code can monitor and find the portions of code covered • Allow reporting on the portions of the code covered frequently • Types of coverage: • Statement Coverage • Path Coverage • Condition Coverage • Function Coverage
  • 17.
    Statement Coverage • Writingtest cases that execute each of the program statements • applies on the following program constructs: • Sequential control flow – test cases to run through top to bottom • Two-way decision statements(if then else) – separate test cases for if and then parts • Multi-way decision statements(switch) – multiple test cases for different switch cases • Loops – design test cases to: • Skip the loop completely • Check “normal” operations • Cover the loop around the “boundary” • Statement Coverage = Total statements exercised X 100 Total number of executable statements in program
  • 18.
    Path Coverage • Splita program into a number of distinct paths • Paths: • A • B-D-G • B-D-H • B-C-E-G • B-C-E-H • B-C-F-G • B-C-F-H
  • 19.
    • Path Coverage= Total paths exercised X 100 Total number of paths in program Condition Coverage • Design test cases that exercise each Boolean expression • Percentage of conditions covered by a set of test cases • Condition Coverage = Total decisions exercised X 100 Total number of decisions in program Function Coverage • Identify how many program functions are covered by test cases • Improve performance and quality of the product • Advantages: • Functions are easier to identify and hence easier to write test cases • Due to higher level of abstraction, easier to achieve 100 percent function coverage
  • 20.
    • By havingmore logical mapping to requirements, more direct correlation to the test coverage of the product • Easier to prioritize the functions based on the importance of the requirements they realize • Provides natural transition to black box testing • Function Coverage = Total functions exercised X 100 Total number of functions in program
  • 21.
    Code Complexity Testing •Cyclomatic Complexity – metric that quantifies the complexity of a program • Identify independent paths • Find out the upper bound on number of tests that cover all the statements • Represent a program as a flow graph consisting of nodes and edges as follows: • Identify the predicates or decision points • Make the predicates simple
  • 22.
    • Combine allsequential statements to single node • When sequential statements are followed by a simple predicate, create a predicate node by combining them • Add a node to represent all sequential statements at the end of the program
  • 23.
    • Cyclomatic Complexity= Number of predicate nodes + 1 OR • Cyclomatic Complexity = E-N+2 • Independent Path – path that has at least one edge not traversed before in other paths • Basis Set – set of independent paths that cover all the edges Calculating and Using Cyclomatic Complexity
  • 24.
    CHALLENGES IN WHITEBOX TESTING • Human tendency of a developer being unable to find the defects in his/her code • Fully tested code may not correspond to realistic scenarios