Test Blueprints

4,361 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
4,361
On SlideShare
0
From Embeds
0
Number of Embeds
36
Actions
Shares
0
Downloads
41
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Test Blueprints

  1. 1. Test Blueprints Exposing Side Effects in Execution Traces to Suport Writing Unit Tests Adrian Lienhard, Tudor Gîrba, Orla Greevy and Oscar Nierstrasz Software Composition Group University of Bern, Switzerland Thanks for the support:
  2. 2. Unit Testing Everybody knows ...unit tests are important! ...write tests first! CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  3. 3. Unit Testing Everybody knows ...unit tests are important! ...write tests first! Fair enough, but what if ...the code exists already? ...and you don’t understand it well? CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  4. 4. Unit Testing Yet, writing unit tests for legacy systems is important. “Tests are your life insurance” CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  5. 5. Challenges of Testing Legacy Code 1. Selecting a unit to test choose unit of appropriate complexity 2. Creating a fixture instantiate and set up graph of objects 3. Executing the unit under test stimulate the fixture 4. Verifying the expected behavior assert side effects and return value CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  6. 6. Idea Use example runs of the program to extract test scenarios. Perform a dynamic analysis to extract a test blueprint that shows: - how to create a fixture for a selected unit - how to execute the unit - how to verify the expected behavior CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  7. 7. Program Instrumentation Example program execution Execution of existing tests object flow + coverage data execution trace data Analysis Tool Execution Trace Test Blueprint CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  8. 8. Test Blueprint Execution Trace Test Blueprint ASTChecker>>declareVariableNode: ... FunctionScope>>addTemp: Execution TempVar class>>new ...initialization... Unit TempVar>>name: TempVar>>scope: KeyedSet>>add: ...library code... ... CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  9. 9. Test Blueprint Legend :Class existing instance existing reference :Class new instance new reference (side effect) CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  10. 10. Creating the Fixture Unit Test for addTemp: method 1 fscope := FunctionScope new. 2 varname := ’x’. Legend :Class existing instance existing reference :Class new instance new reference (side effect) CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  11. 11. Executing the Unit under Test Unit Test for addTemp: method 1 fscope := FunctionScope new. 2 varname := ’x’. 3 var := fscope addTemp: name. Legend :Class existing instance existing reference :Class new instance new reference (side effect) CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  12. 12. Verifying Expected Behavior Unit Test for addTemp: method 1 fscope := FunctionScope new. 2 varname := ’x’. 3 var := fscope addTemp: name. 4 self assert: var class = TempVar. 5 self assert: var name = varname. 6 self assert: var scope = fscope. 7 self assert: ( Legend :Class existing instance fscope tempVars includes: var). existing reference :Class new instance new reference (side effect) CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  13. 13. Blueprint Examples IRBuilder>>add: FunctionScope>>lookupVar: NonClosureScopeFixer>>acceptVarNode: InstanceScope>>newMethodScope ace Test Blueprint CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  14. 14. Implementation dynamic model static model Aliases are created Instance * 1 Class when an object is: 1 ‣ instantiated children * 1 activations 0..1 * 1 * ‣ stored into or read Alias Activation * Method from a field receiver * 0..1 * creator 1 root parent ‣ stored into or read createdAliases from an array Execution- Unit ‣ passed as argument/ return value Object Flow meta-model The flow of an object in the system is tracked by the parent-child relationship of its Aliases CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  15. 15. Implementation Execution Unit w r r object_1 w object_2 w r w object_3 Imported Exported references references t Imported references = used state (->fixture) Exported references = side effects (->assertions) CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  16. 16. Initial Case Study I How straightforward is it to use the proposed tool? Target system: Insurance broker application (520 classes, 6 years old) Observations - One developer produced 12 tests in 2h - Average fixture size: 5 - Average side effects: 4 - Test Blueprint worked surprisingly well ...yet, sporadically resorted to consulting source code - Insufficient support for selecting execution units CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  17. 17. Initial Case Study II How do tests written using our approach differ compared to conventional tests written by an expert? Content Management System (377 classes) Study: selected 14 unit tests, removed all 84 assertions and then systematically rewrote them using our tool Resulting assertions - 72 identical - 5 additional - 12 missed ==> verification that special objects are left unmodified our approach does not detect unmodified objects that are worthwhile to verify! CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  18. 18. Conclusions Yes, we can write unit test for not well known code ...by exploiting runtime information Test Blueprints tell us ...how the fixture and assertions have to look like Difficulties are how to ...select good execution units in the trace ...initialize objects to bring them into the desired state CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch
  19. 19. Conclusions Yes, we can write unit test for not well known code ...by exploiting runtime information Test Blueprints tell us ...how the fixture and assertions have to look like Difficulties are how to ...select good execution units in the trace ...initialize objects to bring them into the desired state Questions? CSMR’08 : : Test Blueprints : : Adrian Lienhard : : www.adrian-lienhard.ch : : lienhard@iam.unibe.ch

×