Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Test-Driven Development
Effectiveness
beyond anecdotal evidence
Krzysztof Jelski
About me
Training
Software
TDD
TDD benefits
Dev safety Better designLess defects
Less debugging Less „will it work?”
Why bother?
Proofs
Insights
Metrics
Tools
Internal quality External quality
Test qualities
Productivity Cost
Internal quality External quality
Test qualities
Cost
- no effect observed
Productivity
Internal quality External quality
Test qualities
Cost
- no effect observed
- significant result
Productivity
Common experiment design
themes
• TDD vs Test-Last
• Between-subjects or repeated measures design
• TDD training at the be...
Internal quality metrics
• CBO - Coupling Between Object Classes
• WMC - Weighted Methods per Class
• RFC - Response For a...
Internal quality
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010
Internal quality
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
Internal quality
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
Coupling Be...
Internal quality
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
Coupling Be...
3 experiments
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
Coupling Betwe...
5-7 weeks preparation
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
Coupli...
8-9 weeks duration
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
Coupling ...
Custom tools
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
Coupling Betwee...
External quality metrics
• Number of defects found
• Defect density (defects/KLOC)
• Percentage of acceptance tests passed...
External quality - studies
Y. Rafique et al., „The Effects of Test-Driven Development
on External Quality and Productivity:...
External quality - studies
Y. Rafique et al., „The Effects of Test-Driven Development
on External Quality and Productivity:...
External quality - studies
Y. Rafique et al., „The Effects of Test-Driven Development
on External Quality and Productivity:...
Productivity metrics
• Development time per task
• Number of acceptance tests passed per hour
• (sadly LOC was used in sev...
Productivity - studies
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
Numbe...
Test qualities
• Code coverage
• Mutation Score Indicator
• Number of negative test cases
Test qualities - studies
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
Cod...
Test qualities - studies
L. Madeyski, „Test-Driven Development.

An Empirical Evaluation of Agile Practice”, 2010 >100
A. ...
Cost - metrics
• Fault cost
• Total project cost
• Maintenance cost
Cost - studies
Damm, L.O., Lundberg, L.: „Results from introducing
component-level test automation and Test-Driven
Develop...
Do I do TDD?
conformance =
TDD_changes + refactorings
all _changes
Roberto Latorre, „Effects of developer experience on
le...
Industrial Logic, e-learning
Industrial Logic, TDD Dashboard
Micheal Hilton, Danny Dig, http://cope.eecs.oregonstate.edu/visualization.html
In your team
Internal quality
External quality
TDD conformance*
Measure:
* - and tell me when you found out how
Internal quality
External quality
Productivity
Icons made by Freepik from www.flaticon.com CC 3.0 BY
krzysztof.jelski@pragmatists.pl
@krzysztofjelski
Thank you!
Upcoming SlideShare
Loading in …5
×

Test-Driven Development Effectiveness - AgileEE 2016

480 views

Published on

Talk presented at Agile Eastern Europe 2016 conference in Kiev.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Test-Driven Development Effectiveness - AgileEE 2016

  1. 1. Test-Driven Development Effectiveness beyond anecdotal evidence Krzysztof Jelski
  2. 2. About me Training Software
  3. 3. TDD
  4. 4. TDD benefits Dev safety Better designLess defects Less debugging Less „will it work?”
  5. 5. Why bother? Proofs Insights Metrics Tools
  6. 6. Internal quality External quality Test qualities Productivity Cost
  7. 7. Internal quality External quality Test qualities Cost - no effect observed Productivity
  8. 8. Internal quality External quality Test qualities Cost - no effect observed - significant result Productivity
  9. 9. Common experiment design themes • TDD vs Test-Last • Between-subjects or repeated measures design • TDD training at the beginning • Measuring conformance to TDD
  10. 10. Internal quality metrics • CBO - Coupling Between Object Classes • WMC - Weighted Methods per Class • RFC - Response For a Class
  11. 11. Internal quality L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010
  12. 12. Internal quality L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100
  13. 13. Internal quality L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100 Coupling Between Objects
  14. 14. Internal quality L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100 Coupling Between Objects Weighted Methods per Class Response For a Class
  15. 15. 3 experiments L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100 Coupling Between Objects Accounting Submissions Smells&Library
  16. 16. 5-7 weeks preparation L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100 Coupling Between Objects
  17. 17. 8-9 weeks duration L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100 Coupling Between Objects 23-27 User Stories
  18. 18. Custom tools L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100 Coupling Between Objects Activity tracking OO metrics Mutation testing
  19. 19. External quality metrics • Number of defects found • Defect density (defects/KLOC) • Percentage of acceptance tests passed • Quality mark given by client
  20. 20. External quality - studies Y. Rafique et al., „The Effects of Test-Driven Development on External Quality and Productivity: A Meta-Analysis”, 2013 External quality (industrial studies) 27
  21. 21. External quality - studies Y. Rafique et al., „The Effects of Test-Driven Development on External Quality and Productivity: A Meta-Analysis”, 2013 External quality (industrial studies) 27 O. Slyngstad et al., „The Impact of Test Driven Development on the Evolution of a Reusable Framework of Components: An Industrial Case Study”, 2008. 100 Defect density -33%
  22. 22. External quality - studies Y. Rafique et al., „The Effects of Test-Driven Development on External Quality and Productivity: A Meta-Analysis”, 2013 External quality (industrial studies) 27 O. Slyngstad et al., „The Impact of Test Driven Development on the Evolution of a Reusable Framework of Components: An Industrial Case Study”, 2008. 100 Defect density -33% N. Nagappan et al., “Realizing Quality Improvement through Test Driven Development: Results and Experiences of Four Industrial Teams,” 2008. 40 Defect density -62% .. -90%
  23. 23. Productivity metrics • Development time per task • Number of acceptance tests passed per hour • (sadly LOC was used in several studies)
  24. 24. Productivity - studies L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100 Number of Acceptance Tests / hour Y. Rafique et al., „The Effects of Test-Driven Development on External Quality and Productivity: A Meta-Analysis”, 2013 27 Productivity (industrial) -22%
  25. 25. Test qualities • Code coverage • Mutation Score Indicator • Number of negative test cases
  26. 26. Test qualities - studies L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100 Code coverage Mutation Score Indicator No effects observed
  27. 27. Test qualities - studies L. Madeyski, „Test-Driven Development.
 An Empirical Evaluation of Agile Practice”, 2010 >100 A. Čaušević et al., „Industrial Study on Test Driven Development: Challenges and Experience”, 2013 60 Number of negative test cases Code coverage Mutation Score Indicator No effects observed
  28. 28. Cost - metrics • Fault cost • Total project cost • Maintenance cost
  29. 29. Cost - studies Damm, L.O., Lundberg, L.: „Results from introducing component-level test automation and Test-Driven Development.”, 2006

  30. 30. Do I do TDD? conformance = TDD_changes + refactorings all _changes Roberto Latorre, „Effects of developer experience on learning and applying Unit Test-Driven Development”, 2013
  31. 31. Industrial Logic, e-learning
  32. 32. Industrial Logic, TDD Dashboard
  33. 33. Micheal Hilton, Danny Dig, http://cope.eecs.oregonstate.edu/visualization.html
  34. 34. In your team Internal quality External quality TDD conformance* Measure: * - and tell me when you found out how
  35. 35. Internal quality
  36. 36. External quality
  37. 37. Productivity
  38. 38. Icons made by Freepik from www.flaticon.com CC 3.0 BY krzysztof.jelski@pragmatists.pl @krzysztofjelski Thank you!

×