The webinar discusses sustainable test automation strategies to enhance efficiency and ROI, highlighting the historical evolution and challenges faced in the field. Key considerations include identifying business problems, measuring benefits, and utilizing various automation tools and models to improve test coverage and speed. The document also shares a customer case study demonstrating significant improvements in test coverage and productivity through effective automation practices.

1. Sustainable test automation strategies to maximise ROI
27th September 2022

2. What you will get out of this Webinar
Understand sustainable test automation that grows more efficient
sprint-over-sprint, while targeting at-risk system logic.
1. A brief history of test automation
2. Speed and coverage: Why is automation underachieving?
3. Key considerations for your test automation strategy
4. Solutions for scalable test automation (with demos!)
5. Questions and discussion

3. A brief history of test
automation
Nalin Parbhu, Infuse Consulting

4. Software Test Automation - A brief history
Test
Automation
has existed
over 20-years
*Horizon Capital
1990s
Commercial test
automation tools –
Mercury, Segue, SQA
2022 2027
18% test automation
market growth per
year
Forecast until 2027
Vs 7% for software
testing overall*
First Selenium
release
2004
No longer dominated by 3
vendors
Plethora of tools confuse
market and users
Historically used for regression testing Used throughout the lifecycle in iterative approaches
2001
The Agile
Manifesto

5. Test Automation today
20 year’s later - Test Automation is under-achieving
• Speed: Just 20% of tests today are automated (including
regression) [1]
• Quality: No one asks about Test Automation Coverage
Without measuring coverage, how do you measure value and
ROI ?
[1] The World Quality Report

6. Speed and coverage: Why is
automation underachieving?
George Blundell, Curiosity Software

7. Test automation coverage
Components
Many logical
combinations
Combine
Customer Apps
Business Apps (CRM,
ERP…)
APIs
Database
s
Third party systems
Systems
More tests than you can run in-sprint
Elements/objects/
endpoints/data
Combinatorial
explosion
10x10x10x10 = 10,000 possible test scenarios to script and maintain
Modules
E2E scenarios
Combine Combine

8. Test creation
remains manual
Scripting is too
repetitive
Automating does
not help coverage
Test data breaks
automation
Brittle tests create
impossible maintenance
Automation becomes a
“black box” and silo
The broken promise of Test Automation
Has automation introduced more time than it’s saved?
10001110101
10101110101
10001010101
10011110101
10001110111
≠

9. Key considerations for your
test automation strategy
Nalin Parbhu, Infuse Consulting

10. How to approach test automation ?
1
Identify Business Problem
• Time to Market
• Speed
• Quality
• Disruption

11. How to approach test automation ?
1
Identify Business Problem
Identify how much of the
stack you control ?
2
• Time to Market
• Speed
• Quality
• Disruption • New platform from
scratch
• Cloud/Native
• Legacy
• Packaged Application

12. How to approach test automation ?
1
Identify Business Problem
Identify how much of the
stack you control ?
2
Test Approach
3
• Time to Market
• Speed
• Quality
• Disruption • New platform from
scratch
• Cloud/Native
• Legacy
• Packaged Application
• Developer-centric
• QA-centric
• Customer journey
• Integration of products

13. How to approach test automation ?
1
Identify Business Problem
Identify how much of the
stack you control ?
2
Test Approach
3
Choices
4
• Time to Market
• Speed
• Quality
• Disruption • New platform from
scratch
• Cloud/Native
• Legacy
• Packaged Application
• Developer-centric
• QA-centric
• Customer journey
• Integration of products • Build from scratch
• Get a tool/solution for QA
1) Note for success
a) You still need to define your business, functional and
non-functional requirements properly for your
engineers
b) You need good engineers to build and test the code
against those requirements correctly
c) You need release teams to deploy it correctly into the
test environment to notes #1-#4 to work

14. How to approach test automation ?
1
Identify Business Problem
Identify how much of the
stack you control ?
2
Test Approach
3
Choices
4
• Time to Market
• Speed
• Quality
• Disruption • New platform from
scratch
• Cloud/Native
• Legacy
• Packaged Application
• Developer-centric
• QA-centric
• Customer journey
• Integration of products • Build from scratch
• Get a tool/solution for QA
Measure
Benefits
1) Note for success
a) You still need to define your business, functional and
non-functional requirements properly for your
engineers
b) You need good engineers to build and test the code
against those requirements correctly
c) You need release teams to deploy it correctly into the
test environment to notes #1-#4 to work

15. And then reality bites back…….
• Businesses are risk averse yet spends a lot of expensive management
consulting fees building requirements
• Procurement enables the business to buy M.E.A.T. (Most Economically
Advantageous Tender) and most the time its wrong
• And you will find the project will be late and testing under pressure
because the quality measures were applied too late (clue: if you find
issues in testing then the issues were earlier in the SDLC)
• As for the business requirements, what can go wrong ?

16. As for requirements, what can go wrong ?
• If you find the project will be late
and testing under pressure; then
the quality measures were applied
too late (clue: if you find issues in
testing then the issues were
earlier in the SDLC)

17. How to approach test automation ?
1
Identify Business Problem
Identify how much of the
stack you control ?
2
Test Approach
3
Choices
4
• Time to Market
• Speed
• Quality
• Disruption • New platform from
scratch
• Cloud/Native
• Legacy
• Packaged Application
• Developer-centric
• QA-centric
• Customer journey
• Integration of products • Build from scratch
• Get a tool/solution for QA
Measure
Benefits
1) Note for success
a) You still need to define your business, functional and
non-functional requirements properly for your
engineers
b) You need good engineers to build and test the code
against those requirements correctly
c) You need release teams to deploy it correctly into the
test environment to notes #1-#4 to work

18. Test Automation tools and strategies over the last 20-years
1st Generation:
Record and
Replay
2nd Generation:
Scripted Tests
3rd Generation: Data-
driven and keyword
frameworks
4th Generation: Record and
Replay 2.0/3.0
5th Generation: Model-
based/AI/Scriptless

19. Solutions for scalable test
automation
George Blundell, Curiosity Software

20. Optimised Test Suites
Test Automation Scripts
Detailed Requirements
Model of an Application
Living
Documentation
Cross-functional teams
RPA Engine
How does it work?

21. Risk-based regeneration following change
Generate accurate user
stories to build better code
Generate tests to
catch bugs in-sprint
Deliver quality software in
short sprints
Cross-
functional
teams
Collaborative
“Living
Documentation”
Stories Tests Models

22. Step 1: Analyse
business process
and scan the
application
Step 4: Automated
the test case with
useMango/Jenkins
Step 5: Automated
reporting from
Jira/Confluence
Summary of Demonstration
Step 2: Design
representative
model of the
system under test
Step 3: Generate
optimised test
cases and export
to Jira Xray and
useMango

23. Demo
© Curiosity Software Ireland Ltd. 2022
© Infuse Consulting Ltd. 2022

24. Solving challenges in automation
Modelling does the hard work upfront, helping break barriers to test automation:
Accelerate and
automate test
and data
creation.
1
Minimize
rework and
manual
maintenance.
2
Optimize to
find bugs
earlier and at
less cost to fix.
3
Maximise
reusability to
focus on value-
added work.
5
Avoid test data
bottlenecks and
automated test
failures.
6
4
Reduces silos,
miscommunication
and “information
hops”.

25. Customer Case Study
Test Coverage
increased by 50%
Speed increased
by 80%
15x increase in
test automation
productivity
$300m Start-up 5g
telco business built
from the ground up
Constantly evolving
business
requirements
Complex system
architecture; 163 items
in the Master
integration list
10 major products to
integrate
Demand from
investors and
shareholders to
launch on time
Customer Situation
Benefits

26. © Curiosity Software Ireland Ltd. 2022
© Infuse Consulting Ltd. 2022
Questions and Discussion

27. Carry on your learning
• Try what you’ve seen today – Head to
TestModeller.io and useMango.co.uk for a free trial
• For a joined-up trial, book a meeting with a
Curiosity expert or email Infuse on info@infuse.it
• We’ll email the webinar recording and slides –
please share!

28. Thank you
George.blundell@curiosity.software
Nalin.parbhu@infuse.it