1. Defining the Value of
GUI Test Automation
Copyright © 2013 Yury Makedonov
Phone: (647) 342-6675
E-mail: yury.makedonov@gmail.com
Blog: http://yurymak.blogspot.ca/
Legacy site: http://www.softwaretestconsulting.com
YouDo app: http://udlst.com/
Yury Makedonov

2. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov2
 This workshop is for everybody who is involved in
making decisions about GUI test automation
performed by an independent testing team:
 Managers of functional test teams
 Managers of GUI test automation teams
 GUI test automation specialists
 Functional testers
Target audience

3. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov3
1. Introduction
2. Relationship between functional testing and GUI Test
Automation
3. The challenges of calculating Return on Investment (ROI)
4. The Cost of GUI Test Automation
5. The Value of GUI Test Automation
6. Defining the goals of:
• Software Testing Team
• Software Test Automation team
7. Defining Software Test Automation metrics
Outline

4. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov4
 Independent Functional/System Testing:
 Typically GUI based tests
 Testing is performed typically by an independent
testing team
 Automation is performed by functional testers or
devoted test automation specialists
 This presentation is mainly about:
“Regression GUI test automation”
“Regression GUI test automation” = automation of the
execution of manual functional test cases which were
developed and documented by a human
 Functional test cases typically have to be created
before the start of GUI test automation.
Independent Testing Team and
GUI Regression Test Automation

5. Defining the Value of GUI Test Automation © Copyright 2009 Yury Makedonov5
Scope of Test Automation

6. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov6
Return On Investment (ROI) is one of the most popular
metrics of efficiency of everything,
including Test Automation
Return On Investment (ROI) definition:
ROI = Benefits $$$ / Cost $$$
Measuring Benefits of Test Automation
An example of ROI calculation:
 Cost = $1,000
 Benefits = $5,000
 ROI = $5,000 / $1,000 = 5 = 500%

7. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov7
 To calculate ROI we need to know:
 Cost $$$
 Benefits $$$
 Calculating $$$ cost typically is not an issue
 Calculating $$$ benefits often is a challenge
Challenges of defining
the Value of Test Automation
 A typical question on the Internet:
“How do I convince senior management of the
importance of Test Automation?”.

8. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov8
 To calculate ROI we need to know:
 Cost $$$
 Benefits $$$
 Calculating $$$ cost typically is not an issue
 Calculating $$$ benefits often is a challenge

Challenges of defining
the Value of Test Automation
Right questions:
“How do I define and measure the value
of Test Automation?
How do I communicate the value of Test
Automation to senior management?

9. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov9
GUI Test Automation Process
 Real test automation process (to calculate $$$):
 Choose a test automation approach and develop
a test automation framework
 Design test automation scripts
(efficient, maintainable, portable, etc.)
 Record a script.
 Enhance a script (add functions and/or data
driving)
 Execute scripts
 Analyze causes of scripts’ failures
 Report defects
 Repair failed scripts

10. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov10
Cost Breakdown
Between Phases of GUI test automation project
 Once, at the beginning of a GUI test automation project:
 Tool selection / Proof of Concept
(Integrated vs. standalone? Can this tool drive your GUI?)
 Development of a basic framework
 For each version/release:
 Design of test automation scripts
(efficient, maintainable, portable, etc.)
 Development (and debugging) of automated test scripts
 For each build:
 Execution of automated scripts / analysis of results /
reporting of anomalies/defects
 Maintenance of automated scripts - fixing of defects in
scripts or test data.

11. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov11
Priorities of a Software Development Organization
Scope
CostTime
Original “PMI triangle”

12. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov12
Priorities of a Software Development Organization
Quality
CostTime
Modified “PMI triangle”

13. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov13
Priorities of a Software Development Organization
Priorities exercise

14. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov14
Customers of Test Automation Team
Software Development Team
Software Testing Team
GUI Test Automation Team
Product Management Team
Client

15. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov15
Customers: “This is not what we wanted!”

16. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov16
 Main “Clients” of a Test automation team:
 Developers
 Managers
Clients of Test Automation Team
Developers
To fix or not to fix?
That is a question.
Test Team
Test
Automation
Managers
To ship or not to ship?
That is a question.

17. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov17
Priorities of a Software Development Organization
Customers of test
automation team exercise

18. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov18
Quality
CostTime
 Priorities of a Software Development Organization, e.g.:
 Accelerating software development
 Increasing the quality of developed software
 Decreasing the cost of software development.
Priorities of a Software Development Organization

19. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov19
Quality
CostTime
 Define your test automation goals, e.g.:
 Decreasing testing time
 Increasing test coverage
 Saving testing man-hours
Priorities of Test Automation Team

20. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov20
 Developing metrics for the following test
automation goals:
1. Decreasing the time of a testing cycle
Accelerating time to market
2. Increasing test coverage / application quality
3. Saving testers' efforts
Test Automation metrics exercise
Test Automation metrics
exercise

21. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov21
Priorities of a Software Development Organization
ROI exercise (optional)

22. Defining the Value of GUI Test Automation © Copyright 2013 Yury Makedonov22
Contact information
Yury Makedonov
Phone: (647) 342-6675
E-mail: yury.makedonov@gmail.com
Blog: http://yurymak.blogspot.ca/
Legacy site: http://www.softwaretestconsulting.com
YouDo app: http://udlst.com/
Facebook: https://www.facebook.com/yury.makedonov
Twitter: @yurymakedonov
LinkedIn:
http://www.linkedin.com/profile/view?id=yury.makedonov