Your SlideShare is downloading. ×
Test automation: the ROI myth
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Test automation: the ROI myth

3,339

Published on

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,339
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
137
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. Test automationThe myth of ROI Gilles Mantel ALE2011
    • 2. 2
    • 3. What are we talking about ? Unit tests Only
automated Integration tests Only
automated Functional tests ▶ End to end ▶ GUI Cannot
be
 ▶ Exploratory automated Only
automated Stress, performance, reliability etc Cannot
be
 automated Usability, AB tests etc
    • 4. What are we talking about ? Unit tests Integration tests Functional tests Automatable
 with
s/w
robots ▶ End to end ▶ GUI ▶ Exploratory Stress, performance, reliability etc Usability, AB tests etc
    • 5. ROI formula ROI = manual cost – automation cost Manual cost: execution Automation cost: software licenses, automation scripts, results analysis, script maintenance
    • 6. ROI calculators
    • 7. Cumulated cost over time 3 2 1
    • 8. Main points Complex / Cumbersome Many unknowns ▶ Make assumptions Results not encouraging What’s missing ?
    • 9. Cost of a defect
    • 10. Cost of a defect
    • 11. Financial option Right to buy a financial product (underlying) at a given price (strike) at a given date in the future (maturity date) Pay a prime to buy this right Ex: I pay 10$ the right to buy a Google share in December 2011 at the price of 530$
    • 12. Financial model Benefits at maturity In the Money Underlying price at maturity Out of Money
    • 13. Financial model of test automation ROI
    • 14. Financial model of test automation Saved cost of identified defects
    • 15. Modèle de gain ROI Cost of existing defects Saved cost of Investment in identified defects automation
    • 16. Investment in automation ? In a Financial Bank: the mean cost of a defect is around 300 000 € At a web travel agency: 1mn website downtime costs 20 000€in terms of lack of revenue Measure !
    • 17. Investment in automation ? In your organisation, how much costs: ▶ A late delivery? Check with marketing dpt ▶ Maintenance teams fixing defects ? Check with the CIO ▶ Operational loss caused by defects ? Check with the financial dpt ▶ Negative perceptions of brand ? Check with the sales dpt www.xebia.fr / blog.xebia.fr 17
    • 18. Investment in automation ?
    • 19. Investment in automation ?
    • 20. Testing pyramid (Mike Cohn)
    • 21. What ROI for an agile team ?
    • 22. What ROI for an agile team ?
    • 23. Where to invest ? and last, a bit here Then here First here
    • 24. Investment in automation ?
    • 25. Waterfall Pyramid
    • 26. What ROI for a legacy ?
    • 27. What ROI for a legacy ?
    • 28. Where to invest ? First here Then here And last here
    • 29. Conclusions  The ROI is not directly linked with time  The ROI is linked with non-quality  Agile practices lead to reducing the needed investment by iteratively automating  Measure how much you can invest based on your non-quality predictions
    • 30. Twitter: @gmantelE-mail: gmantel@xebia.frMERCI

    ×