• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Test automation: the ROI myth
 

Test automation: the ROI myth

on

  • 3,332 views

 

Statistics

Views

Total Views
3,332
Views on SlideShare
3,066
Embed Views
266

Actions

Likes
3
Downloads
108
Comments
0

5 Embeds 266

http://agileinthemix.com 262
https://twitter.com 1
http://www.slideshare.net 1
http://paper.li 1
http://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Test automation: the ROI myth Test automation: the ROI myth Presentation Transcript

  • Test automationThe myth of ROI Gilles Mantel ALE2011
  • 2
  • What are we talking about ? Unit tests Only
automated Integration tests Only
automated Functional tests ▶ End to end ▶ GUI Cannot
be
 ▶ Exploratory automated Only
automated Stress, performance, reliability etc Cannot
be
 automated Usability, AB tests etc
  • What are we talking about ? Unit tests Integration tests Functional tests Automatable
 with
s/w
robots ▶ End to end ▶ GUI ▶ Exploratory Stress, performance, reliability etc Usability, AB tests etc
  • ROI formula ROI = manual cost – automation cost Manual cost: execution Automation cost: software licenses, automation scripts, results analysis, script maintenance
  • ROI calculators
  • Cumulated cost over time 3 2 1
  • Main points Complex / Cumbersome Many unknowns ▶ Make assumptions Results not encouraging What’s missing ?
  • Cost of a defect
  • Cost of a defect
  • Financial option Right to buy a financial product (underlying) at a given price (strike) at a given date in the future (maturity date) Pay a prime to buy this right Ex: I pay 10$ the right to buy a Google share in December 2011 at the price of 530$
  • Financial model Benefits at maturity In the Money Underlying price at maturity Out of Money
  • Financial model of test automation ROI
  • Financial model of test automation Saved cost of identified defects
  • Modèle de gain ROI Cost of existing defects Saved cost of Investment in identified defects automation
  • Investment in automation ? In a Financial Bank: the mean cost of a defect is around 300 000 € At a web travel agency: 1mn website downtime costs 20 000€in terms of lack of revenue Measure !
  • Investment in automation ? In your organisation, how much costs: ▶ A late delivery? Check with marketing dpt ▶ Maintenance teams fixing defects ? Check with the CIO ▶ Operational loss caused by defects ? Check with the financial dpt ▶ Negative perceptions of brand ? Check with the sales dpt www.xebia.fr / blog.xebia.fr 17
  • Investment in automation ?
  • Investment in automation ?
  • Testing pyramid (Mike Cohn)
  • What ROI for an agile team ?
  • What ROI for an agile team ?
  • Where to invest ? and last, a bit here Then here First here
  • Investment in automation ?
  • Waterfall Pyramid
  • What ROI for a legacy ?
  • What ROI for a legacy ?
  • Where to invest ? First here Then here And last here
  • Conclusions  The ROI is not directly linked with time  The ROI is linked with non-quality  Agile practices lead to reducing the needed investment by iteratively automating  Measure how much you can invest based on your non-quality predictions
  • Twitter: @gmantelE-mail: gmantel@xebia.frMERCI