The document discusses several methods for calculating the return on investment of test automation: 1. The Linz/Daigl Cost Benefit Ratio Method calculates a cost savings ratio of 33% for automated testing versus manual testing based on the costs of design, implementation, execution and analysis. 2. The Cost Savings Ratio Method shows ROI increases from 21% after one test execution to 5039% after 76 test executions. 3. The Test Effectiveness Method shows automated tests find 13% of total defects found. While ROI models provide limited analysis, test automation provides additional intangible benefits like supporting management goals, test tool reuse, and enhancing employee skills. Automated and manual testing also find different types

1. Return-On-Investment Analysis on Test Automation

2. Some Basic ROI Methods Standard Cost Benefit Ratio Method: E = Benefits / Cost Linz/Daigl Cost Benefit Ratio Method : E n = cost of automated testing / cost of manual testing E n = A a / A m E n = (V a + n a *D a ) / (V m + n m *D m ) Cost Savings Ratio Method: ROI a = $ value of time savings due automation / cost of automation ROI a = ∑B a / ∑C a Test Effectivenss Method: T eff = # of Defects found by automation / All defects found T eff = D a / D m

4. Cost Benefit Ratio Method Cost of Manual Testing: A m = (V m + n m *D m ) V m = Cost of manual test case design V m = Staff Engineers*Design Days*Staff Salary V m = (2)*((12 months*20 days/month)*75%)*($50/hr*8 hr/day) V m = $144,000 n m = Number of manual test execution cycles n m = 2 Note: 4 month test duration D m = Cost of manual test execution & results analysis D m = Staff Engineers*Test Days*Staff Salary D m = (2)*(8 hours/day*20 days/month*2 month*.80)*($50/hr) D m = $25,600 Note: Cost per test cycle A m = (V m + n m *D m ) = $144,000 + 2*$25,600 A m = $195,200

5. Cost Benefit Ratio Method Cost of Automated Testing: A a = (V a + n a *D a ) V a = Cost of automation design and implementation V a = Staff Engineers*Design Days*Staff Salary+Equipment Costs V a = (2)*((9 months*20 days/month)*75%)*($50/hr*8 hr/day) + $12,000 V a = $120,000 n a = Number of automated test executions n a = Automated Test execution every work day for 95% of the test cycle n a = (20 work days/month)*(4 months)*95% n a = 76 D a = Cost of automated test results analysis & maintenance D a = Staff Engineers*Design Days*Staff Salary D a = (1)*(8 hours)*($50/hr) D a = $400 A a = (Va + na*Da) = $120,000 + 76*$400 A a = $150,400

6. Cost Benefit Ratio Method Cost Benefit Ratio : E n = A a / A m = $150,400 / $195,200 E n = 0.77 So there is a 33% Cost Savings of Automated Testing versus Manual Testing.

7. Cost Savings Ratio Method ROI a = ∑ Benefits of automation / ∑ Costs of automation ROI a = ∑Ba / ∑Ca B a = Benefits of automation = Cost of manual test execution - Cost of automated test execution = $25,600 - $400 B a = $25,200 C a = Cost of automation design and implementation = Staff Engineers*Design Days*Staff Salary + Equipment Costs = (2)*((9 months*20 days/month)*75%)*($50/hr*8 hr/day) C a = $120,000 So: ROI a = B a / C a ROI a = 21% … after (1) test execution ROI a = 42% …after (2) test executions ROI a = 105% …after (5) test executions ROI a = 5039% …after (76) test executions

8. Test Effectiveness Method T eff = # of Defects found by automation / All defects found T eff = D a / D m T eff = 1 / 8 T eff = 13%

9. Summary of ROI Analysis Linz&Daigl Cost Benefit Ratio: E n = 33% Cost Savings Cost Savings Ratio: ROI a = 21% … after (1) test execution ROI a = 42% …after (2) test executions ROI a = 105% …after (5) test executions ROI a = 5039% …after (76) test executions Tests Effectiveness: T eff = 13%