White Box testing by Pankaj Thakur, NITTTR Chandigarh

Presentation
On
White Box Testing
Submitted to:
Dr. Rakesh Kumar
Assistant Professor
Department of CSE
NITTTR Chandigarh
Submitted by:
Pankaj Thakur
ME(CSE) Modular
RN 171408
1

Sub topics
• Introduction to White Box Testing
• Various Form
• Comparison with Black Box Testing
• Cyclomatic Complexity
• Control Flow Testing
• Data Flow Testing
• Slice Based Testing
• Mutation Testing
• Advantages and Disadvantages of White Box Testing
• Latest research papers based on White Box Testing
• Summary
• References
• Questions & Answers
2

Introduction
• White Box Testing is a method of testing software that tests
internal structures or workings of an application, as opposed to its
functionality (i.e. black-box testing)
• For designing test cases it uses internal perspective of the
system, as well as programming skills,.
White Box Testing is also known as:
1) Glass box testing
2) Clear box testing
3) Open box testing
4) Transparent box testing
5) Structural testing
6) Logic driven testing
7) Design based testing
3

Forms of White Box Testing Techniques
White
Box
Testing
Data
Flow
Testing
Control
Flow
Testing
Basis
Path
Testing
Loop
Testing
Branch
Testing
4

Criteria Black Box Testing White Box Testing
Definition
Black Box Testing is a software testing method
in which the internal structure/ design/
implementation of the item being tested is
NOT known to the tester
White Box Testing is a software
testing method in which the
internal structure/ design/
implementation of the item being
tested is known to the tester.
Levels Applicable To
Mainly applicable to higher levels of testing:
Acceptance Testing
System Testing
Mainly applicable to lower levels
of testing:
Unit Testing
Integration Testing
Responsibility Generally, independent Software Testers Generally, Software Developers
Programming
Knowledge
Not Required Required
Implementation
Knowledge
Not Required Required
Basis for Test Cases Requirement Specifications Detail Design
White Box Vs Black Box Testing
5

Cyclomatic Complexity
•There are many paths in any program.
•But we try to focus on independent paths in that program.
•Independent paths should have at least one new node or edge which is to be
traversed.
•Number of independent paths through a program is called cyclomatic number in
graph theory which has been redefined as cyclomatic complexity.
•McCabe introduced this concept and gave three methods to calculate cyclomatic
complexity.
1. V(G) = e – n + 2P
where V(G) = Cyclomatic complexity
G = program graph
e = number of edges
n = number of nodes
P = number of connected components
1. V(G) = Number of region of the program graph.
2. V(G) = π + 1
where π = number of predicate nodes in G. (each should have two outgoing
edges)
6

Properties of Cyclomatic Complexity
• V(G) is the maximum number of independent paths in program graph G.
• V(G) ≥ 1
• G has only one path if V(G) = 1
• Addition or deletion of functional statements to program graph G does not affect V(G).
• V(G) depends only on the decision structure of G.
a
c d
e
b
Program Graph
Cyclomatic Complexity (Three ways):
1. V(G) = e – n + 2P
= 7 – 5 + 2X1
= 4
2. V(G) = No. of regions of the graphs
= 4
3. V(G) = π + 1
= 3 + 1
= 4
( node a, c, and d are predicate nodes )
1 2
34
Four
Independent
paths are:
1. ace,
2. ade,
3. adce,
4. acbe 7

Control Flow Testing
• Testing technique based on program coverage criteria.
• Very popular technique due to its simplicity and effectiveness.
• Here we try to find various paths of a program and design test cases to execute those
paths.
• We may write test cases that ensure the execution of every statement, or branch, or
condition, or path in a program
Control Flow
Testing
Statement
Coverage
Branch
Coverage
Condition
Coverage
Path
Coverage
8

Statement Coverage
#include<stdio.h>
#include<conio.h>
1 void main()
2 {
3 int a,b,c,x=0,y=0;
4 clrscr();
5 printf("Enter three numbers : ");
6 scanf("%d%d%d", &a,&b,&c);
7 if((a>b)&&(a>c)){
8 x=a*a+b*b;
9 }
10 if(b>c){
11 y=a*a-b*b;
12 }
13 printf("x=%d y=%d",x,y);
14 getch();
15 }
We want to execute
every statement of the
program.
Cyclomatic Complexity:
1. V(G) = e –n + 2P
= 16-15+2 = 3
2. V(G) = no. of regions
= 3
3. V(G) = π + 1
= 2+1 = 3
Program
1 2 3 54 6
7
8
9
10
11
12
13 14 15Program
Graph
Total Paths = 4
1. 1-7, 10, 13-15
2. 1-7, 10-15
3. 1-10, 13-15
4. 1-15
Independent Paths = 3
1. 1-7, 10, 13-15
2. 1-7, 10-15
3. 1-10, 13-15
If, we input a=9,
b=8, c=7, all
statements are
executed!
9

Branch Coverage
We want to test every
branch of the program.
#include<stdio.h>
#include<conio.h>
1 void main()
2 {
4 clrscr();
6 scanf("%d%d%d", &a,&b,&c);
7 if((a>b)&&(a>c)){
8 x=a*a+b*b;
9 }
10 if(b>c){
11 y=a*a-b*b;
12 }
14 getch();
15 }
Program
1 2 3 54 6
7
8
9
10
11
12
13 14 15Program
Graph
If, we input a=9,
b=8, c=7, all true
conditions are
executed!
Path: 1-15
If, we input a=7,
b=8, c=9, all false
conditions are
executed!
Path: 1-7, 10, 13-15
These two test
cases out of four
are sufficient to
guarantee 100%
branch coverage. 10

Condition Coverage
We want to test every condition at least once.
Better than branch coverage because we want to test every condition at least once.
However, branch coverage can be achieved without testing every condition.
#include<stdio.h>
#include<conio.h>
1 void main()
2 {
4 clrscr();
6 scanf("%d%d%d", &a,&b,&c);
7 if((a>b)&&(a>c)){
8 x=a*a+b*b;
9 }
10 if(b>c){
11 y=a*a-b*b;
12 }
14 getch();
15 }
Program
Consider the 7th statement having two conditions.
Four possibilities:
1. Both are true. (a>b and a>c)
2. First is true, second is false. (a>b and a<c)
3. First is false, second is true. (a<b and a>c)
4. Both are false (a<b and a<c))
We should write test cases for every true and false
condition.
1. a=9, b=8, c=7 (first possibility)
2. a=9, b=8, c=10(second possibility)
3. a=7, b=8, c=9 (third and fourth possibility)
These three test cases out of four are
sufficient to ensure execution of every
condition of the program!
11

Path Coverage
We want to test every path of the program.
There may be too many paths due to loops and feedback connections.
It may not be possible to achieve this goal of executing all paths in many programs.
At least all independent paths should be executed.
Guarantee statement coverage, branch coverage, and condition coverage.
However, there are many paths in the program and it may not be possible to executed all
the paths.
As we go statement coverage to path coverage more resources and time may be required!
12

Path Coverage (cont…)
#include<stdio.h>
#include<conio.h>
1 void main()
2 {
4 clrscr();
6 scanf("%d%d%d", &a,&b,&c);
7 if((a>b)&&(a>c)){
8 x=a*a+b*b;
9 }
10 if(b>c){
11 y=a*a-b*b;
12 }
14 getch();
15 }
Program
Four paths:
1. 1-7, 10, 13-15
2. 1-7, 10-15
3. 1-10, 13-15
4. 1-15
1 2 3 54 6
7
8
9
10
11
12
13 14 15
Program
Graph
S. No. Paths Id. Paths
Inputs
Expected Output
a b c
1 Path1 1-7, 10, 13-15 7 8 9 x=0, y=0
2 Path2 1-7, 10-15 7 8 6 x=0, y=-15
3 Path3 1-10, 13-15 9 7 8 x=130, y=0
4 Path4 1-15 9 8 7 x=145, y=17
13

Loop Testing
Classes of Loops
Focus on the verity of loop constructs.
Simple Loop
1. Skip the loop entirely.
2. Only one pass through the loop.
3. Two passes through the loop.
4. m passes through the loop (m<n)
5. n-1, n, n+1 passes through the loop.
(n is the max. no. of allowable passes through the loop)
Nested Loop
1. Start the innermost loop. Set all other loops to
minimum value.
2. Conduct simple loop tests for the innermost
loop while holding the outer loop at their
minimum iteration parameter (e.g. loop
counter) values. Add other tests for out-of-
range or excluded values.
3. Work outward, conducting tests for the next
loop, but keeping all other outer loops at
minimum values and other nested loops to
“typical” values.
4. Continue until all loops have been tested. 14

Loop Testing (Cont…)
Concatenated Loops
1. Use approach defined for simple loops if each loop is independent of the other.
2. Use approach defined for nested loops if each loop is independent of the other
Unstructured Loops
1. Redesign, whenever possible!
15

Data Flow Testing
#include<stdio.h>
1 void main()
2 {
3 int a,b,c;
4 a = b + c;
5 printf("%d", a);
6 }
What will be the output?
Data Flow Testing may help us to minimize such
mistakes.
Data Flow Testing concentrates on how a variable is
defined and used (referenced) at different places of
a program.
• Nothing to do with data flow diagrams.
• Based on variables, their usage and their definition(s)/assignment in the program.
• Main points of concern are:
1. Variable definition (statements where variables receive values)
2. Variable reference ( statements where these values are used)
16

Data Flow Testing (Cont…)
Define/Reference Anomalies
1. A variable is defined but never used.
2. A variable is used but never defined.
3. A variable is defined twice before it is used.
4. A variable is used before even first-definition.
May be identified by static
analysis of the program
(i.e. without executing it)
Some Definitions
1. Defining node, DEF(v, n): A node in the program graph is a defining node for a variable
‘v’, if and only if, the value of the variable ‘v’ is defined in the statement corresponding
to that node.
2. Usage node, USE( v, n) : A node in the program graph is a usage node for a variable ‘v’, if
and only if, the value of the variable ‘v’ is used in the statement corresponding to that
node.
3. Definition use Path (du path): A definition use path for a variable ‘v’ is a path between
two nodes ‘m’ and ‘n’ where ‘m’ is the initial node in the path and ‘n’ is the final node
in the path.
4. Definition clear Path (dc path): A definition clear path for a variable ‘v’ is a definition use
path with initial and final nodes DEF( v, n) and USE( v, n) such that no other node in the
path is a defining node of variable ‘v’.
The du-paths that are not dc paths are potential troublesome paths. They should
be identified and tested on topmost priority.
17

Identification of du and dc paths
1. Draw the program graph of the program.
2. Find all variables of the program and prepare a table for define / use status of
all the variables using the following format:
3. Generate all du-paths from define / use variable table using the following format:
4. Identify those du-paths which are not dc-paths.
S. No. Variable(s) Defined at node Used at node
S. No. Variable du-path(begin, end)
18

Testing strategies using du-paths:
Here we want to generate test cases which traces every definition to each of its use and every
use is traced to each of its definition.
1. Test all du-paths (Strongest among all)
All du-paths generated for all variables are tested
2. Test all uses
Here we find at least one path from every definition of every variable to every use of that
variable which can be reached by that definition.
3. Test all definition
Here we find paths from every definition of every variable to at least one use of that
variable………………
19

Generation of test
cases
S. No. Variable(s) Defined at node Used at node
1 a 6 11, 12, 13
2 b 8 11, 20, 24
3 c 10 12, 16, 20, 21
S. No. Variable du-path(begin, end)
1 a
6, 11
6, 12
6, 13
2 b
8, 11
8, 20
8, 24
3 c
10, 12
10, 16
10, 20
10, 21
#include<stdio.h>
#include<conio.h>
1 void main()
2 {
3 float a,b,c;
4 clrscr();
5 printf("Enter number 1 : n");
6 scanf("%f",&a);
8 scanf("%f",&b);
10 scanf("%f",&c);
11 if(a>b){
12 if(a>c){
13 printf("The largest number is %fn",a);
14 }
15 else {
16 printf("The largest number is %fn",c);
17 }
18 }
19 else {
20 if(c>b) {
21 printf("The largest number is : %fn",c);
22 }
23 else {
24 printf("The largest number is : %fn",b);
25 }
26 }
27 getch();
28 }
Program
Graph
Program to find largest of 3
numbers.
20

SN
Inputs
Expected O/P Remarksa b c
1 9 8 7 9 6-11
2 9 8 7 9 6-12
3 9 8 7 9 6-13
4 7 9 8 9 8-11
5 7 9 8 9 8-11, 19, 20
6
7 9 8 9
8-11, 19, 20, 23, 24
7 8 7 9 9 10-12
8 8 7 9 9 10-12, 15, 16
9 7 8 9 9 10, 11, 19, 20
10 7 8 9 9 10, 11, 19-21
21

Slice Based Testing
• Here we prepare various subsets ( called slices ) of a program with respect to its
variables and their selected locations in the program.
•Each variable with one of its location will give us a program slice.
•A large program may have many smaller programs ( its slices ), each constructed for
different variable subsets.
•Slices are typically simpler than the original program, thereby simplifying the
process of testing of the program.
How to create program slice?
Guidelines for slicing
1. All statements where variables are defined and redefined should be consider.
2. All statements where variables receive values externally should be considered.
3. All statements where output of a variable is printed should be considered.
4. All statements where some relevant output is printed should be considered.
5. The status of all variables may be considered at the last statement of the
program.
22

Slice Based Testing (Cont…)
1 void main()
2 {
3 int a,b,c,d,e;
4 printf("Enter the values for a, b, and c n");
5 scanf("%d%d%d", &a,&b,&c);
6 d=a+b;
7 e=b+c;
8 printf("%d", d);
9 printf("%d", e);
10 }
Example program
• Many slices of the given program may be created.
• A statement may have many variables.
• Only one variable should be used to generate a slice at
a time.
• Different variables in the same statement will generate
a different program slice.
•Hence there may be number of slices of a program depending upon the slicing criteria.
1 void main()
2 {
3 int a,b,c,d,e;
5 scanf("%d%d%d", &a,&b,&c);
7 e=b+c;
9 printf("%d", e);
10 }
Slice on criteria S(e, 10) = (1,2,3,4,5,7,9,10)
1 void main()
2 {
3 int a,b,c,d,e;
5 scanf("%d%d%d", &a,&b,&c);
6 d=a+b;
8 printf("%d", d);
10 }
Slice on criteria S(d, 10) = (1,2,3,4,5,6,8,10)
1 void main()
2 {
3 int a,b,c,d,e;
5 scanf("%d%d%d", &a,&b,&c);
7 e=b+c;
10 }
Slice on criteria S(e, 7) = (1,2,3,4,5,7,10)
1 void main()
2 {
3 int a,b,c,d,e;
5 scanf("%d%d%d", &a,&b,&c);
6 d=a+b;
10 }
Slice on criteria S(d, 6) = (1,2,3,4,5,6,10)
1 void main()
2 {
3 int a,b,c,d,e;
5 scanf("%d%d%d", &a,&b,&c);
10 }
Slice on criteria S(a, 5) = (1,2,3,4,5,10)
23

Mutation Testing
We may have large number of test cases for any program.
We neither have time nor resources to execute all of them.
We may select a few test cases using any technique and prepare a test suit.
Mutation testing may help us to assess the effectiveness and quality of a test suit and
may also enhance the test suit, if it is not adequate for a program.
Mutation and Mutants
The process to change a program is called mutation.
We prepare a copy of the program under test and make a change in the statement of
the program.
This changed version of the program is known as a mutant of the original program.
The behavior of the mutant may be different from the original program.
Original program and the mutant are syntactically correct and should compile
correctly.
We may make many mutants of a program.
24

Mutation Testing
#include<stdio.h>
#include<conio.h>
1 void main()
2 {
3 float a,b,c;
4 clrscr();
6 scanf("%f",&a);
8 scanf("%f",&b);
10 scanf("%f",&c);
11 if(a>b){
12 if(a>c){
14 }
15 else {
17 }
18 }
19 else {
20 if(c>b) {
22 }
23 else {
25 }
26 }
27 getch();
28 }
#include<stdio.h>
#include<conio.h>
1 void main()
2 {
3 float a,b,c;
4 clrscr();
6 scanf("%f",&a);
8 scanf("%f",&b);
10 scanf("%f",&c);
11 if(a>b){ ←if(a=b){ //mutated statement '>' replaced by '='
12 if(a>c){
14 }
15 else {
17 }
18 }
19 else {
20 if(c>b) {
22 }
23 else {
25 }
26 }
27 getch();
28 }
#include<stdio.h>
#include<conio.h>
1 void main()
2 {
3 float a,b,c;
4 clrscr();
6 scanf("%f",&a);
8 scanf("%f",&b);
10 scanf("%f",&c);
11 if(a>b){
12 if(a>c){
14 }
15 else {
17 }
18 }
19 else {
20 if(c>b) { ←if(c<b){ //mutated statement '>' replaced by '='
22 }
23 else {
25 }
26 }
27 getch();
28 }
Original program Mutant M1 of Original program Mutant M2 of Original program 25

Mutation Testing
Mutation Operators
Mutants are produced by applying mutant operators.
An operator is essentially a grammatical rule that changes a single expression to
another expression.
The changed expression should be grammatically correct as per the used language.
Some mutant operators for c++/Java are:
1. Changing the access modifier.
2. Static modifier changed.
3. Argument order changed.
4. Super keyword changed
5. Operator changed.
6. Any operand changed.
Mutation Score
Always between 0 and 1.
A higher value of mutation score indicates the effectiveness of the test suit
Mutation Score =
No. of mutant killed
Total no. of mutants
26

Mutation Testing
SN a b c Expected o/p
1 6 10 2 10
2 10 6 2 10
3 6 2 10 10
4 6 10 20 20
A program along with the test suit is given. Generate five mutants (M1 to M5) and calculate
mutation score of this test suit.
Solution
#include<stdio.h>
#include<conio.h>
1 void main()
2 {
3 float a,b,c;
4 clrscr();
6 scanf("%f",&a);
8 scanf("%f",&b);
10 scanf("%f",&c);
11 if(a>b){
12 if(a>c){
14 }
15 else {
17 }
18 }
19 else {
20 if(c>b) {
22 }
23 else {
25 }
26 }
27 getch();
28 }
Mutant No. Line No. Original line Modified line
M1 11
if(a>b)
if(a<b)
M2 11
if(a>b)
if(a>(b+c))
M3 12
if(a>c)
if(a<c)
M4 20
if(c>b)
if(c=b)
M5 16 printf("The largest number is :%f", c);
printf("The largest number is :%f", b);
Actual o/p of mutant M1
SN a b c Expected o/p Actual o/p
1 6 10 2 10 6
2 10 6 2 10 6
3 6 2 10 10 10
4 6 10 20 20 20
27

1 6 10 2 10 10
2 10 6 2 10 2
3 6 2 10 10 6
4 6 10 20 20 20
1 6 10 2 10 10
2 10 6 2 10 10
3 6 2 10 10 10
4 6 10 20 20 20
1 6 10 2 10 10
2 10 6 2 10 10
3 6 2 10 10 10
4 6 10 20 20 10
1 6 10 2 10 10
2 10 6 2 10 10
3 6 2 10 10 2
4 6 10 20 20 20
Mutation Score = No. of mutant killed
Total no. of mutants
=4/5
=0.8
Mutation Testing (Cont…)
Mutatnt M2 is live in this
example.
We have to write a
specific test case to kill
this mutant.
a=10,b=5,c=6
Expected o/p= 10
Actual o/p= 6.
This test case is very
important and should be
added to the given test
suit.
Revised test suit
1 6 10 2 10 10
2 10 6 2 10 10
3 6 2 10 10 10
4 6 10 20 20 20
5 10 5 6 10 6
28

Advantages of White Box Testing
• Side effects of having the knowledge of the source code is beneficial to thorough testing.
•Optimization of code becomes easy as inconspicuous bottlenecks are exposed.
•Gives the programmer introspection because developers carefully describe any new
implementation.
•Provides traceability of tests from the source, thereby allowing future changes to the
source to be easily captured in the newly added or modified tests.
• Easy to automate.
•Provides clear, engineering-based rules for when to stop testing.
Disadvantages of White Box Testing
•White-box testing brings complexity to testing and requires a programmer with a high
level of knowledge.
•On some occasions, it is not realistic to be able to test every single existing condition of
the application and some conditions will be untested.
•The tests focus on the software as it exists, and missing functionality may not be
discovered.
• The resulting test can be fragile because they are tightly coupled to the specific
implementation of the thing being tested. The code under test could be rewritten to
implement the same functionality in a different way that invalidates the assumptions
baked into the test. This could result in tests that fail unnecessarily or, in the worst case,
tests that now give false positives and mask errors in the code.
Advantages and Dissadvantages
29

Latest Research papers based on White Box Testing
1. RESTful API Automated Test Case Generation
(Andrea Arcuri, 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS) )
2. Automated Test Generation and Mutation Testing for Alloy
(Allison Sullivan; Kaiyuan Wang; Razieh Nokhbeh Zaeem; Sarfraz Khurshid, 2017 IEEE International
Conference on Software Testing, Verification and Validation (ICST) )
3. Safety - Critical Software Quality Improvement Using Requirement Analysis
(Hijung Hwang; Yong B. Park, 2017 International Conference on Platform Technology and Service
(PlatCon) )
4. White-Box vs. Black-Box Complexity of Search Problems: Ramsey and Graph Property Testing
( Ilan Komargodski; Moni Naor; Eylon Yogev, 2017 IEEE 58th Annual Symposium on Foundations of
Computer Science (FOCS) )
5. Vehicle network security testing
( Jihas Khan, 2017 Third International Conference on Sensing, Signal Processing and Security (ICSSS) )
6. Study on the online test of spindle's dynamic radial error of machine tool
(Jikun Zhou; Rong Zhang; Mingxiang Ling, 2017 IEEE International Conference on Mechatronics and
Automation (ICMA))
7. Fuzzy rule based systems for interpretable sentiment analysis
( Han Liu; Mihaela Cocea, 2017 Ninth International Conference on Advanced Computational
Intelligence (ICACI) )
30

White box testing is verification and
validation technique which software
engineers can use to examine their
code works as expected.
Summary
31

References
1. Software Engineering by Pressman.
2. Software Engineering by KK Agarwal
3. IEEE Xplore
32

White Box testing by Pankaj Thakur, NITTTR Chandigarh

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to White Box testing by Pankaj Thakur, NITTTR Chandigarh

Similar to White Box testing by Pankaj Thakur, NITTTR Chandigarh (20)

Recently uploaded

Recently uploaded (20)

White Box testing by Pankaj Thakur, NITTTR Chandigarh