DejaVOO: A Regression Testing Tool for Java Software *
Alessandro Orso, Donglin Liang, Mary Jean Harrold and Manas Tungare — Georgia Institute of Technology
http://www.cc.gatech.edu/aristotle/
1. Regression Testing 5. Our Safe Regression Test Selection Algorithm: Characteristics
• Performed on new versions of the software to provide confidence that: • It handles Java object-oriented features resulting in simple components and complex
– the changed parts behave as intended interactions
– the unchanged parts are not adversely affected by the modifications – inheritance
• Can be very expensive, especially in some contexts – polymorphism and dynamic binding
– daily re-testing after nightly builds – exception handling
– frequent re-testing during development • It handles incomplete programs that play an important role in object-oriented software
– for the extreme programming paradigm – components
– applications using libraries (e.g., AWT)
– subsystems
2. Regression Test Selection
6. DejaVOO: An Implementation of our Regression Test Selection
A safe algorithm
T-T’
Algorithm for Java Software
• requires certain assumptions for safety
Program
T
• ensures that T’ contains all test cases in T that
P
T’
can expose a fault in P’ Architecture of DejaVOO
– same execution environment Execute P / Record coverage
– deterministic test runs
P’ Version
–…
P
of P Coverage
Events
T’ JVM Profiler
T
Matrix
Test T’
Selection
JVM: Java Virtual Machine
3. Potential Savings Tool
ous
Danger
P
12 12 JIG for P
Profiler: JVM Profiling Interface
Graph
Entities
JABA
P’ JIG for P'
Traversal
JABA: Java Architecture for
Time to
Bytecode Analysis
Identify dangerous entities
select T’
Time to select T’ + Time to run T’ Time to run 3
3
9 9
< Time to run T T
Savings
Time to
run T’
Prototype GUI for DejaVOO
6 6
• It shows a SeeSoft view of the source
code of P
4. Our Safe Regression Test Selection Algorithm: Overview
• It highlights the lines of code that have
been modified
• It displays the test cases to be rerun
c. Differentiate G, G’ to find dangerous entities
a. Construct graph representations for P and P’
• It lets the user selectively shows which
test cases traverse which parts of the
G G’ Dangerous Edge
G G’ code and vice-versa
if() if() if() if()
1 1
2 2 1 1
2 2
doA doB doA doC doA doB doA doC
b. Associate test cases in T with entities in P d. Select test cases based on dangerous entities 7. Empirical Studies: Sizes of Selected Subsets of T
Subjects
% Test Cases Selected
100
• Siena: Event notification system (185 methods),
90
TC TC 80
tc1 tc2 tc3 tc1 tc2 tc3 70
En En 7 versions, 138 test cases, 70% method coverage 60
• Jedit: Text editor (3495 methods), 50
40
en1 X en1 X 30
11 versions, 189 test cases, 75% method coverage 20
• Jmeter: Web-application test tool (109 methods), 10
0
Siena Jedit Jmeter RegExp
en2 X X en2 X X 5 versions, 50 test cases, 67% method coverage V1-V7 V1-V11 V1-V5 V1-V9
• RegExp: Regular-expression library (168 methods), Program Name
10 versions, 66 test cases, 46% method coverage
* This work was supported in part by a grant from Boeing Aerospace Corporation to Georgia Tech, by National Science Foundation award CCR-9707792 to Ohio State
University, and awards CCR-9988294, CCR-0096321, and EIA-0196145 to Georgia Tech, and by the State of Georgia to Georgia Tech under the Yamacraw Mission.