Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Test Pyramid Heresy - a fresh look at test automation strategies

343 views

Published on

The Test Pyramid is a staple in Test Automation theory, and is used by many teams as a cornerstone of their test automation strategy. But is it still the best model for modern development practices? Are there better and more efficient ways of thinking about test automation today?

Test automation is nowadays essential in any project, but inefficient test automation is increasingly becoming a major cause of waste and, ironically, delay. Test suites need to be lean, slick and efficient, with just enough tests (and the right kind of tests!) at each level to provide confidence, but not so many as to slow down the test suite or increase maintenance costs. And automation requirements vary wildly from one kind of project to another. The way we think about test automation strategy clearly needs a rehaul.

Drawn from approaches such as Lean, Behaviour Driven Development, and Outside-in Development, this talk presents a different way of looking at test automation, one that will help you focus your test automation efforts where they will add real value.

You will never look at a pyramid in the same way again!

Published in: Technology
  • Be the first to comment

A Test Pyramid Heresy - a fresh look at test automation strategies

  1. 1. @wakaleo 1 A fresh look at test automation strategies The Testing Pyramid Heresy John Ferguson Smart http://johnfergusonsmart.com
  2. 2. @wakaleo John Ferguson Smart John Ferguson Smart “I help teams of smart people 
 learn to work together more efficiently, 
 to deliver better software sooner” 2 https://johnfergusonsmart.com
  3. 3. https://johnfergusonsmart.com “Test Automation is like any other tool It’s either a benefit or a hazard” 3
  4. 4. @wakaleo https://johnfergusonsmart.com How much are your tests worth? 4 Weekly cost vs weekly savings $0 $20,000 $40,000 $60,000 $80,000 $100,000 $120,000 $140,000 $160,000 $180,000 1/09/2015 1/10/2015 1/11/2015 1/12/2015 Test0suite0cost0vs0savings0(per0week) Cost0saved0par0week Test0suite0cost0per0week Team cost is constant Time saved increases with the number of executed tests
  5. 5. @wakaleo https://johnfergusonsmart.com How much are your tests worth? 5 $0 $200,000 $400,000 $600,000 $800,000 $1,000,000 $1,200,000 1/09/2015 1/10/2015 1/11/2015 1/12/2015 What0is0your0test0suite0worth? Test0suite0ROI Test0suite0cost Cumulated cost vs cumulated savings Cumulated value should increase faster than cumulated cost
  6. 6. @wakaleo https://johnfergusonsmart.com Now factor in high maintenance 6 20-40% Typical maintenance overhead for poorly-written test suites
  7. 7. @wakaleo https://johnfergusonsmart.com Now factor in high maintenance 7 Number of tests Maintenanceoverhead
  8. 8. @wakaleo https://johnfergusonsmart.com $0 $50,000 $100,000 $150,000 $200,000 $250,000 $300,000 $350,000 $400,000 $450,000 $500,000 1/09/2015 1/10/2015 1/11/2015 1/12/2015 1/01/2016 What0is0your0test0suite0worth? Test0suite0cost Test0suite0ROI Now factor in high maintenance 8 High-maintenance costs reduce the number of tests a team can write High maintenance costs can cause your test suite to lose value
  9. 9. @wakaleo https://johnfergusonsmart.com Enter the Testing Pyramid 9
  10. 10. @wakaleo https://johnfergusonsmart.com …or the Testing Ice-cream cone 10
  11. 11. @wakaleo https://johnfergusonsmart.com In other words… 11 Don’t use a web test where a service or unit test could do the same job.
  12. 12. 12
  13. 13. 13 “All models are wrong, but some are useful.” - George Box
  14. 14. 14
  15. 15. 15 The Testing Pyramid is about how a test is implemented. But is this the best way to classify our tests?
  16. 16. @wakaleo Developer TestsQA TestsBusiness Tests Tests come from different sources 16
  17. 17. @wakaleo Developer TestsQA TestsBusiness Tests Tests come from different sources 17
  18. 18. @wakaleo Developer TestsQA TestsBusiness Tests Tests come from different sources 18
  19. 19. @wakaleo Developer TestsQA TestsBusiness Tests Tests come from different sources 19
  20. 20. @wakaleo DemonstrateDescribeDiscover Tests have different roles 20
  21. 21. @wakaleo DemonstrateDescribeDiscover Tests have different roles 21
  22. 22. @wakaleo https://johnfergusonsmart.com Test (activities) that discover 22 Example Mapping Impact Mapping Feature Mapping
  23. 23. @wakaleo DemonstrateDescribeDiscover Tests have different roles 23
  24. 24. @wakaleo https://johnfergusonsmart.com Test (activities) that describe 24
  25. 25. @wakaleo https://johnfergusonsmart.com Test (activities) that describe 25
  26. 26. @wakaleo DemonstrateDescribeDiscover Tests have different roles 26
  27. 27. @wakaleo https://johnfergusonsmart.com Example and Feature Mapping Tests have different roles 27 Discover D escribe Dem onstrate Impact Mapping Executable specifications Performance tests Functional tests Unit tests Regression tests
  28. 28. 28 A new testing quadrant
  29. 29. 29 A new testing quadrant
  30. 30. 30 A new testing quadrant
  31. 31. 31 A new testing quadrant
  32. 32. 32 A new testing quadrant
  33. 33. 33 Goldilocks Test Suites
  34. 34. @wakaleo 34 Questions? John Ferguson Smart http://johnfergusonsmart.com

×