Test Blueprints

Test Blueprints
Exposing Side Effects in Execution
Traces to Suport Writing Unit Tests

Adrian Lienhard, Tudor Gîrba,
Orla Greevy and Oscar Nierstrasz
Software Composition Group
University of Bern, Switzerland

Thanks for the support:

Unit Testing

Everybody knows
...unit tests are important!
...write tests first!

CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch

Unit Testing

Everybody knows
...unit tests are important!
...write tests first!

Fair enough, but what if
...the code exists already?
...and you don’t understand it well?


Unit Testing

Yet, writing unit tests for
legacy systems is important.

“Tests are your life insurance”


Challenges of Testing
Legacy Code
1. Selecting a unit to test
choose unit of appropriate complexity

2. Creating a fixture
instantiate and set up graph of objects

3. Executing the unit under test
stimulate the fixture

4. Verifying the expected behavior
assert side effects and return value


Idea
Use example runs of the program to extract
test scenarios.

Perform a dynamic analysis to extract a test
blueprint that shows:
- how to create a fixture for a selected unit
- how to execute the unit
- how to verify the expected behavior


Program Instrumentation
Example program execution Execution of existing tests
object ﬂow +
coverage data
execution trace data
Analysis

Tool

Execution Trace Test Blueprint


Test Blueprint

Execution Trace Test Blueprint

ASTChecker>>declareVariableNode:
...
FunctionScope>>addTemp:

Execution
TempVar class>>new
...initialization...
Unit
TempVar>>name:
TempVar>>scope:
KeyedSet>>add:
...library code...

...


Test Blueprint

Legend :Class existing instance
existing reference

:Class new instance
new reference (side effect)


Creating the Fixture

Unit Test for addTemp: method
1 fscope := FunctionScope new.
2 varname := ’x’.

existing reference

:Class new instance


Executing the Unit under Test


3 var := fscope addTemp: name.

existing reference

:Class new instance


Verifying Expected Behavior


3 var := fscope addTemp: name.

4 self assert: var class = TempVar.
5 self assert: var name = varname.
6 self assert: var scope = fscope.
7 self assert: ( Legend :Class existing instance
fscope tempVars includes: var). existing reference

:Class new instance


Blueprint Examples
IRBuilder>>add: FunctionScope>>lookupVar:

NonClosureScopeFixer>>acceptVarNode:
InstanceScope>>newMethodScope

ace Test Blueprint


Implementation
dynamic model static model Aliases are created
Instance * 1
Class
when an object is:
1 ‣ instantiated
children
* 1 activations
0..1 *
1
* ‣ stored into or read
Alias Activation * Method from a field
receiver *
0..1 * creator 1 root
parent ‣ stored into or read
createdAliases from an array
Execution-
Unit ‣ passed as argument/
return value
Object Flow meta-model

The flow of an object in the system is tracked by
the parent-child relationship of its Aliases


Implementation
Execution Unit
w r r
object_1
w
object_2
w r w
object_3

Imported Exported
references references
t

Imported references = used state (->fixture)
Exported references = side effects (->assertions)


Initial Case Study I
How straightforward is it to use the
proposed tool?

Target system: Insurance broker
application (520 classes, 6 years old)

Observations
- One developer produced 12 tests in 2h
- Average fixture size: 5
- Average side effects: 4
- Test Blueprint worked surprisingly well
...yet, sporadically resorted to consulting source code
- Insufficient support for selecting execution units

Initial Case Study II
How do tests written using our approach differ compared
to conventional tests written by an expert?

Content Management System (377 classes)

Study: selected 14 unit tests, removed all 84 assertions
and then systematically rewrote them using our tool

Resulting assertions
- 72 identical
- 5 additional
- 12 missed ==> verification that special objects are left unmodified
our approach does not detect unmodified objects
that are worthwhile to verify!


Conclusions
Yes, we can write unit test for not well known code
...by exploiting runtime information

Test Blueprints tell us
...how the fixture and assertions have to look like

Difficulties are how to
...select good execution units in the trace
...initialize objects to bring them into the desired state


Conclusions
Yes, we can write unit test for not well known code
...by exploiting runtime information

Test Blueprints tell us
...how the fixture and assertions have to look like

Difficulties are how to
...select good execution units in the trace
...initialize objects to bring them into the desired state

Questions?

Test Blueprints

More Related Content

What's hot

Viewers also liked

Similar to Test Blueprints

More from lienhard

Recently uploaded

Test Blueprints