SlideShare a Scribd company logo
© Curiosity Software Ireland Ltd. 2022
Hitting the right test
coverage for CI/CD
November 10th 2022
James Walker
Director of Technology, Curiosity.
James.Walker@Curiosity.Software
@CuriositySoft
Chris Harbert
Founder and CEO, Testery.
Chris.Harbert@Testery.io
@ChrisHarbert
© Curiosity Software Ireland Ltd. 2022
Hitting the right test coverage for
CI/CD
1. Why is automation lagging behind CI/CD?
2. Test automation and orchestration for CI/CD
3. Demo
4. Q&A and next steps
© Curiosity Software Ireland Ltd. 2022
A team of experts here
to support you
An ethos of partnering with
our customers
¼ Fortune 100 &
plenty outside
Inventors of multiple tools
on the market
Inventors in modelling, data,
automation, and RQMTs
30+ years of
experience
© Curiosity Software Ireland Ltd. 2022
Experts that enable
your test strategy
Interact with your tests via the
Web, command line, and API
Run the right tests at the
right time in your pipeline
Run millions of tests per
month
Support most open-source
testing frameworks
Test runs that span environments,
frameworks, or teams
© Curiosity Software Ireland Ltd. 2022
Why is automation lagging
behind CI/CD?
Challenges in creating, maintaining and executing tests
© Curiosity Software Ireland Ltd. 2022
Why CI/CD requires test automation
• CI/CD and Devops practices result in more versions of the code
are being deployed to more places more often.
• Manual approaches struggle to keep up and produce cost
effective, timely, and consistent results.
1.How can you create tests?
2.How can you maintain tests?
3.How can you execute the tests?
4.What about coverage?
© Curiosity Software Ireland Ltd. 2022
When to move to automation
Cost
of
testing
Features
Point at which operating costs
recoup implementation costs
© Curiosity Software Ireland Ltd. 2022
Options for creating tests
1.Writing the code for the tests
2.Record-and-playback
3.Model-based testing
© Curiosity Software Ireland Ltd. 2022
Considerations for maintaining tests
1.Tests and code must both be updated as functionality changes
2.How many places will you have to update
3.How will you know what needs to be updated
4.When will you know it needs to change
5.How can you lower the cost of making that change
6.How will you keep track of different versions of changes
© Curiosity Software Ireland Ltd. 2022
Challenges executing tests
Running the right tests at
the right time
Running reliably at high
levels of parallelism
Matching code and
test versions
Running tests that span
projects, environments,
and frameworks
© Curiosity Software Ireland Ltd. 2022
Challenges creating and maintaining
tests
10001110101
10101110101
10001010101
10011110101
10001110111
≠
Test creation
remains manual
Scripting is too
repetitive
Automating does
not boost coverage
Test data breaks automation
Brittle tests create
impossible maintenance
Automation becomes a
“black box” and silo
© Curiosity Software Ireland Ltd. 2022
What about coverage?
Components
Many logical combinations.
Modules
E2E scenarios
Combine
Customer Apps
Business Apps (CRM, ERP…)
APIs Databases
Third party systems
Systems
More tests than you can run in-sprint.
Elements/objects/
endpoints/data
Combinatorial
explosion
10x10x10x10 = 10,000 possible test scenarios to script and maintain
Combine Combine
© Curiosity Software Ireland Ltd. 2022
Alternatives to scripted frameworks?
Record/Playback No Code Keyword-driven
Brittle tests need re-recording
SUT is limited by tooling
SUT is limited by tooling
Locked into limited actions
Fiddley keyword config
Scripting to expand keywords
Designing/recording tests remains linear, time-consuming and unsystematic
What about coverage?!
© Curiosity Software Ireland Ltd. 2022
Test automation and
orchestration for CI/CD
Auto-generate and execute targeted tests
© Curiosity Software Ireland Ltd. 2022
Test generation for CI/CD
Synchronise data from across
the SDLC to generate targeted
tests in-sprint
Find / Make / Allocate Test Data
Run Results
Tests
New
Iteration
 Seamless collaboration
 Better RQMTs  Better code
 Rigorous testing in-sprint
 Quality with every release
Test Automation
Test Automation
Test Cases
User stories
BDD/Gherkin
Scans and
Recordings
Requirements
Diagrams
Living Documentation
Cross-functional teams
BAs, POs,
testers, Dev,
SDETs, DBAs.
Find / Make / Allocate Test Data
© Curiosity Software Ireland Ltd. 2022
Test orchestration for CI/CD
Dev QA RC Prod
Feature A
Feature B
Functional Tests
Smoke Tests
Regression Tests
Monitoring Tests
Determine which tests to run
Decide when and where to run them
Automatically scale infrastructure
Respond to test failures
Demo
© Curiosity Software Ireland Ltd. 2022
© Curiosity Software Ireland Ltd. 2022
Rapid releases with in-built quality
Change
User stories &
Code changes
Central source
of truth
Targeted tests and data, ready as soon as code is
</>
Execute tests
Run
Results
⁻ Across environments
⁻ Across the testing pyramid
⁻ Across repositories
⁻ Across systems
Faster, less-buggy dev
© Curiosity Software Ireland Ltd. 2022
Rapid releases with in-built quality
Drive faster, less buggy development
Maintain clear and complete specifications that reduce rework in development
Shorten release cycles with bottleneck-free testing
Generate and maintain tests in parallel to development, running them as soon
as code checked-in
Find bugs earlier, at less cost and impact to fix
Target rigorous testing based on system logic and changes, testing across
complex hybrid architectures
© Curiosity Software Ireland Ltd. 2022
Questions and Discussion
© Curiosity Software Ireland Ltd. 2022
Carry on the conversation?
• Book a demo and discussion with a Curiosity expert!
• Learn how to get started with Team Testery!
• Try Test Modeller. Visit Testmodeller.io for a free trial and online tutorials.
• Follow Curiosity Software on LinkedIn and @CuriositySoft on Twitter for
resources and news.
• Follow Testery on LinkedIn and @Testery_io on Twitter.
© Curiosity Software Ireland Ltd. 2022
Thank you
James.Walker@Curiosity.Software
Chris.harbert@testery.io

More Related Content

Similar to Curiosity and Testery Present: Hitting the right test coverage for CI/CD

Curiosity and Lemontree present - Data Breaks DevOps: Why you need automated ...
Curiosity and Lemontree present - Data Breaks DevOps: Why you need automated ...Curiosity and Lemontree present - Data Breaks DevOps: Why you need automated ...
Curiosity and Lemontree present - Data Breaks DevOps: Why you need automated ...
Curiosity Software Ireland
 
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Applitools
 
"Software Quality in the Service of Innovation in the Insurance Industry"
"Software Quality in the Service of Innovation in the Insurance Industry""Software Quality in the Service of Innovation in the Insurance Industry"
"Software Quality in the Service of Innovation in the Insurance Industry"
Applitools
 
What is Low Code Test Automation and its importnce?
What is Low Code Test Automation and its importnce?What is Low Code Test Automation and its importnce?
What is Low Code Test Automation and its importnce?
kalichargn70th171
 
Let's banish "it works on my machine"
Let's banish "it works on my machine"Let's banish "it works on my machine"
Let's banish "it works on my machine"
Stephanie Locke
 
Curiosity and Sauce Labs present - When to stop testing: 3 dimensions of test...
Curiosity and Sauce Labs present - When to stop testing: 3 dimensions of test...Curiosity and Sauce Labs present - When to stop testing: 3 dimensions of test...
Curiosity and Sauce Labs present - When to stop testing: 3 dimensions of test...
Curiosity Software Ireland
 
Quality Assurance and its Importance in Software Industry by Aman Shukla
Quality Assurance and its Importance in Software Industry by Aman ShuklaQuality Assurance and its Importance in Software Industry by Aman Shukla
Quality Assurance and its Importance in Software Industry by Aman Shukla
AbhishekKumar773294
 
From 0 to DevOps in 80 Days [Webinar Replay]
From 0 to DevOps in 80 Days [Webinar Replay]From 0 to DevOps in 80 Days [Webinar Replay]
From 0 to DevOps in 80 Days [Webinar Replay]
Dynatrace
 
How to build confidence in your release cycle
How to build confidence in your release cycleHow to build confidence in your release cycle
How to build confidence in your release cycle
DiUS
 
Technology and Digital Platform | 2019 partner summit
Technology and Digital Platform | 2019 partner summitTechnology and Digital Platform | 2019 partner summit
Technology and Digital Platform | 2019 partner summit
Andrew Kumar
 
The quality assurance checklist for progressive testing
The quality assurance checklist for progressive testingThe quality assurance checklist for progressive testing
The quality assurance checklist for progressive testing
Maitrikpaida
 
The Quality Assurance Checklist for Progressive Testing
The Quality Assurance Checklist for Progressive TestingThe Quality Assurance Checklist for Progressive Testing
The Quality Assurance Checklist for Progressive Testing
Cygnet Infotech
 
DockerCon SF 2015: Faster, Cheaper, Safer
DockerCon SF 2015: Faster, Cheaper, SaferDockerCon SF 2015: Faster, Cheaper, Safer
DockerCon SF 2015: Faster, Cheaper, Safer
Docker, Inc.
 
Deployment Automation for Hybrid Cloud and Multi-Platform Environments
Deployment Automation for Hybrid Cloud and Multi-Platform EnvironmentsDeployment Automation for Hybrid Cloud and Multi-Platform Environments
Deployment Automation for Hybrid Cloud and Multi-Platform Environments
IBM UrbanCode Products
 
Service Virtualization: What Testers Need to Know
Service Virtualization: What Testers Need to KnowService Virtualization: What Testers Need to Know
Service Virtualization: What Testers Need to Know
TechWell
 
May 2023 EMEA New ThousandEyes Product Features and Release Highlights.pptx
May 2023 EMEA New ThousandEyes Product Features and Release Highlights.pptxMay 2023 EMEA New ThousandEyes Product Features and Release Highlights.pptx
May 2023 EMEA New ThousandEyes Product Features and Release Highlights.pptx
ThousandEyes
 
Forward5 Auxis VMware
Forward5 Auxis VMwareForward5 Auxis VMware
Forward5 Auxis VMware
Auxis Consulting & Outsourcing
 
Add AI to Your SDLC, presented by Applitools and Curiosity
Add AI to Your SDLC, presented by Applitools and CuriosityAdd AI to Your SDLC, presented by Applitools and Curiosity
Add AI to Your SDLC, presented by Applitools and Curiosity
Applitools
 
Improving Quality through Continuous Integration - A case study of CollabNet
Improving Quality through Continuous Integration - A case study of CollabNetImproving Quality through Continuous Integration - A case study of CollabNet
Improving Quality through Continuous Integration - A case study of CollabNet
Venkat Janardhanam, MS, MBA
 
Devops transformation in the Rational Collaborative Lifecycle Organization
Devops transformation in the Rational Collaborative Lifecycle OrganizationDevops transformation in the Rational Collaborative Lifecycle Organization
Devops transformation in the Rational Collaborative Lifecycle Organization
Robbie Minshall
 

Similar to Curiosity and Testery Present: Hitting the right test coverage for CI/CD (20)

Curiosity and Lemontree present - Data Breaks DevOps: Why you need automated ...
Curiosity and Lemontree present - Data Breaks DevOps: Why you need automated ...Curiosity and Lemontree present - Data Breaks DevOps: Why you need automated ...
Curiosity and Lemontree present - Data Breaks DevOps: Why you need automated ...
 
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
Top 5 Features To Look for in a Codeless Automation Solution -- Presentation ...
 
"Software Quality in the Service of Innovation in the Insurance Industry"
"Software Quality in the Service of Innovation in the Insurance Industry""Software Quality in the Service of Innovation in the Insurance Industry"
"Software Quality in the Service of Innovation in the Insurance Industry"
 
What is Low Code Test Automation and its importnce?
What is Low Code Test Automation and its importnce?What is Low Code Test Automation and its importnce?
What is Low Code Test Automation and its importnce?
 
Let's banish "it works on my machine"
Let's banish "it works on my machine"Let's banish "it works on my machine"
Let's banish "it works on my machine"
 
Curiosity and Sauce Labs present - When to stop testing: 3 dimensions of test...
Curiosity and Sauce Labs present - When to stop testing: 3 dimensions of test...Curiosity and Sauce Labs present - When to stop testing: 3 dimensions of test...
Curiosity and Sauce Labs present - When to stop testing: 3 dimensions of test...
 
Quality Assurance and its Importance in Software Industry by Aman Shukla
Quality Assurance and its Importance in Software Industry by Aman ShuklaQuality Assurance and its Importance in Software Industry by Aman Shukla
Quality Assurance and its Importance in Software Industry by Aman Shukla
 
From 0 to DevOps in 80 Days [Webinar Replay]
From 0 to DevOps in 80 Days [Webinar Replay]From 0 to DevOps in 80 Days [Webinar Replay]
From 0 to DevOps in 80 Days [Webinar Replay]
 
How to build confidence in your release cycle
How to build confidence in your release cycleHow to build confidence in your release cycle
How to build confidence in your release cycle
 
Technology and Digital Platform | 2019 partner summit
Technology and Digital Platform | 2019 partner summitTechnology and Digital Platform | 2019 partner summit
Technology and Digital Platform | 2019 partner summit
 
The quality assurance checklist for progressive testing
The quality assurance checklist for progressive testingThe quality assurance checklist for progressive testing
The quality assurance checklist for progressive testing
 
The Quality Assurance Checklist for Progressive Testing
The Quality Assurance Checklist for Progressive TestingThe Quality Assurance Checklist for Progressive Testing
The Quality Assurance Checklist for Progressive Testing
 
DockerCon SF 2015: Faster, Cheaper, Safer
DockerCon SF 2015: Faster, Cheaper, SaferDockerCon SF 2015: Faster, Cheaper, Safer
DockerCon SF 2015: Faster, Cheaper, Safer
 
Deployment Automation for Hybrid Cloud and Multi-Platform Environments
Deployment Automation for Hybrid Cloud and Multi-Platform EnvironmentsDeployment Automation for Hybrid Cloud and Multi-Platform Environments
Deployment Automation for Hybrid Cloud and Multi-Platform Environments
 
Service Virtualization: What Testers Need to Know
Service Virtualization: What Testers Need to KnowService Virtualization: What Testers Need to Know
Service Virtualization: What Testers Need to Know
 
May 2023 EMEA New ThousandEyes Product Features and Release Highlights.pptx
May 2023 EMEA New ThousandEyes Product Features and Release Highlights.pptxMay 2023 EMEA New ThousandEyes Product Features and Release Highlights.pptx
May 2023 EMEA New ThousandEyes Product Features and Release Highlights.pptx
 
Forward5 Auxis VMware
Forward5 Auxis VMwareForward5 Auxis VMware
Forward5 Auxis VMware
 
Add AI to Your SDLC, presented by Applitools and Curiosity
Add AI to Your SDLC, presented by Applitools and CuriosityAdd AI to Your SDLC, presented by Applitools and Curiosity
Add AI to Your SDLC, presented by Applitools and Curiosity
 
Improving Quality through Continuous Integration - A case study of CollabNet
Improving Quality through Continuous Integration - A case study of CollabNetImproving Quality through Continuous Integration - A case study of CollabNet
Improving Quality through Continuous Integration - A case study of CollabNet
 
Devops transformation in the Rational Collaborative Lifecycle Organization
Devops transformation in the Rational Collaborative Lifecycle OrganizationDevops transformation in the Rational Collaborative Lifecycle Organization
Devops transformation in the Rational Collaborative Lifecycle Organization
 

More from Curiosity Software Ireland

Curiosity Software Presents: Isolating blast radiuses for testing - How to no...
Curiosity Software Presents: Isolating blast radiuses for testing - How to no...Curiosity Software Presents: Isolating blast radiuses for testing - How to no...
Curiosity Software Presents: Isolating blast radiuses for testing - How to no...
Curiosity Software Ireland
 
Curiosity Software Presents: Modelling for Continuous Testing
Curiosity Software Presents: Modelling for Continuous TestingCuriosity Software Presents: Modelling for Continuous Testing
Curiosity Software Presents: Modelling for Continuous Testing
Curiosity Software Ireland
 
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity Software Ireland
 
Curiosity and Windocks Present - Turn your production systems into test-ready...
Curiosity and Windocks Present - Turn your production systems into test-ready...Curiosity and Windocks Present - Turn your production systems into test-ready...
Curiosity and Windocks Present - Turn your production systems into test-ready...
Curiosity Software Ireland
 
Curiosity and Sogeti Present - The state of test data in 2022: New challenges...
Curiosity and Sogeti Present - The state of test data in 2022: New challenges...Curiosity and Sogeti Present - The state of test data in 2022: New challenges...
Curiosity and Sogeti Present - The state of test data in 2022: New challenges...
Curiosity Software Ireland
 
Curiosity and Lemontree present - Test Data Automation: Move from slow and ma...
Curiosity and Lemontree present - Test Data Automation: Move from slow and ma...Curiosity and Lemontree present - Test Data Automation: Move from slow and ma...
Curiosity and Lemontree present - Test Data Automation: Move from slow and ma...
Curiosity Software Ireland
 
Curiosity Software and RCG Global Services Present - Solving Test Data: the g...
Curiosity Software and RCG Global Services Present - Solving Test Data: the g...Curiosity Software and RCG Global Services Present - Solving Test Data: the g...
Curiosity Software and RCG Global Services Present - Solving Test Data: the g...
Curiosity Software Ireland
 
Curiosity and EPI-USE Labs present - Testing across SAP and non-SAP systems: ...
Curiosity and EPI-USE Labs present - Testing across SAP and non-SAP systems: ...Curiosity and EPI-USE Labs present - Testing across SAP and non-SAP systems: ...
Curiosity and EPI-USE Labs present - Testing across SAP and non-SAP systems: ...
Curiosity Software Ireland
 
Curiosity and Coforge present - Oracle FLEXCUBE: De-risk upgrades and migrati...
Curiosity and Coforge present - Oracle FLEXCUBE: De-risk upgrades and migrati...Curiosity and Coforge present - Oracle FLEXCUBE: De-risk upgrades and migrati...
Curiosity and Coforge present - Oracle FLEXCUBE: De-risk upgrades and migrati...
Curiosity Software Ireland
 
Curiosity and SelectorsHub present: Generate Rigorous Automated Tests - and L...
Curiosity and SelectorsHub present: Generate Rigorous Automated Tests - and L...Curiosity and SelectorsHub present: Generate Rigorous Automated Tests - and L...
Curiosity and SelectorsHub present: Generate Rigorous Automated Tests - and L...
Curiosity Software Ireland
 
Curiosity Software, Infuse and Kumoco present: The Democratisation of Testing
Curiosity Software, Infuse and Kumoco present: The Democratisation of TestingCuriosity Software, Infuse and Kumoco present: The Democratisation of Testing
Curiosity Software, Infuse and Kumoco present: The Democratisation of Testing
Curiosity Software Ireland
 
Development Has Moved On: Test data needs to catch up with containers
Development Has Moved On: Test data needs to catch up with containersDevelopment Has Moved On: Test data needs to catch up with containers
Development Has Moved On: Test data needs to catch up with containers
Curiosity Software Ireland
 
The Broken Promise of Test Automation: Why are we still hand-cranking tests?
The Broken Promise of Test Automation: Why are we still hand-cranking tests?The Broken Promise of Test Automation: Why are we still hand-cranking tests?
The Broken Promise of Test Automation: Why are we still hand-cranking tests?
Curiosity Software Ireland
 
In the beginning there was a model: Using requirements models to drive rigoro...
In the beginning there wasa model: Using requirements models to drive rigoro...In the beginning there wasa model: Using requirements models to drive rigoro...
In the beginning there was a model: Using requirements models to drive rigoro...
Curiosity Software Ireland
 
Test data automation: delivering quality data at speed
Test data automation: delivering quality data at speedTest data automation: delivering quality data at speed
Test data automation: delivering quality data at speed
Curiosity Software Ireland
 
To Open Banking and Beyond: Developing APIs that are Resilient to every new I...
To Open Banking and Beyond: Developing APIs that are Resilient to every new I...To Open Banking and Beyond: Developing APIs that are Resilient to every new I...
To Open Banking and Beyond: Developing APIs that are Resilient to every new I...
Curiosity Software Ireland
 
Five reasons to model during QA
Five reasons to model during QAFive reasons to model during QA
Five reasons to model during QA
Curiosity Software Ireland
 
Model-Based Test Automation: Automate more, automate faster, automate accurately
Model-Based Test Automation: Automate more, automate faster, automate accuratelyModel-Based Test Automation: Automate more, automate faster, automate accurately
Model-Based Test Automation: Automate more, automate faster, automate accurately
Curiosity Software Ireland
 

More from Curiosity Software Ireland (18)

Curiosity Software Presents: Isolating blast radiuses for testing - How to no...
Curiosity Software Presents: Isolating blast radiuses for testing - How to no...Curiosity Software Presents: Isolating blast radiuses for testing - How to no...
Curiosity Software Presents: Isolating blast radiuses for testing - How to no...
 
Curiosity Software Presents: Modelling for Continuous Testing
Curiosity Software Presents: Modelling for Continuous TestingCuriosity Software Presents: Modelling for Continuous Testing
Curiosity Software Presents: Modelling for Continuous Testing
 
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
Curiosity and Infuse Consulting Present: Sustainable Test Automation Strategi...
 
Curiosity and Windocks Present - Turn your production systems into test-ready...
Curiosity and Windocks Present - Turn your production systems into test-ready...Curiosity and Windocks Present - Turn your production systems into test-ready...
Curiosity and Windocks Present - Turn your production systems into test-ready...
 
Curiosity and Sogeti Present - The state of test data in 2022: New challenges...
Curiosity and Sogeti Present - The state of test data in 2022: New challenges...Curiosity and Sogeti Present - The state of test data in 2022: New challenges...
Curiosity and Sogeti Present - The state of test data in 2022: New challenges...
 
Curiosity and Lemontree present - Test Data Automation: Move from slow and ma...
Curiosity and Lemontree present - Test Data Automation: Move from slow and ma...Curiosity and Lemontree present - Test Data Automation: Move from slow and ma...
Curiosity and Lemontree present - Test Data Automation: Move from slow and ma...
 
Curiosity Software and RCG Global Services Present - Solving Test Data: the g...
Curiosity Software and RCG Global Services Present - Solving Test Data: the g...Curiosity Software and RCG Global Services Present - Solving Test Data: the g...
Curiosity Software and RCG Global Services Present - Solving Test Data: the g...
 
Curiosity and EPI-USE Labs present - Testing across SAP and non-SAP systems: ...
Curiosity and EPI-USE Labs present - Testing across SAP and non-SAP systems: ...Curiosity and EPI-USE Labs present - Testing across SAP and non-SAP systems: ...
Curiosity and EPI-USE Labs present - Testing across SAP and non-SAP systems: ...
 
Curiosity and Coforge present - Oracle FLEXCUBE: De-risk upgrades and migrati...
Curiosity and Coforge present - Oracle FLEXCUBE: De-risk upgrades and migrati...Curiosity and Coforge present - Oracle FLEXCUBE: De-risk upgrades and migrati...
Curiosity and Coforge present - Oracle FLEXCUBE: De-risk upgrades and migrati...
 
Curiosity and SelectorsHub present: Generate Rigorous Automated Tests - and L...
Curiosity and SelectorsHub present: Generate Rigorous Automated Tests - and L...Curiosity and SelectorsHub present: Generate Rigorous Automated Tests - and L...
Curiosity and SelectorsHub present: Generate Rigorous Automated Tests - and L...
 
Curiosity Software, Infuse and Kumoco present: The Democratisation of Testing
Curiosity Software, Infuse and Kumoco present: The Democratisation of TestingCuriosity Software, Infuse and Kumoco present: The Democratisation of Testing
Curiosity Software, Infuse and Kumoco present: The Democratisation of Testing
 
Development Has Moved On: Test data needs to catch up with containers
Development Has Moved On: Test data needs to catch up with containersDevelopment Has Moved On: Test data needs to catch up with containers
Development Has Moved On: Test data needs to catch up with containers
 
The Broken Promise of Test Automation: Why are we still hand-cranking tests?
The Broken Promise of Test Automation: Why are we still hand-cranking tests?The Broken Promise of Test Automation: Why are we still hand-cranking tests?
The Broken Promise of Test Automation: Why are we still hand-cranking tests?
 
In the beginning there was a model: Using requirements models to drive rigoro...
In the beginning there wasa model: Using requirements models to drive rigoro...In the beginning there wasa model: Using requirements models to drive rigoro...
In the beginning there was a model: Using requirements models to drive rigoro...
 
Test data automation: delivering quality data at speed
Test data automation: delivering quality data at speedTest data automation: delivering quality data at speed
Test data automation: delivering quality data at speed
 
To Open Banking and Beyond: Developing APIs that are Resilient to every new I...
To Open Banking and Beyond: Developing APIs that are Resilient to every new I...To Open Banking and Beyond: Developing APIs that are Resilient to every new I...
To Open Banking and Beyond: Developing APIs that are Resilient to every new I...
 
Five reasons to model during QA
Five reasons to model during QAFive reasons to model during QA
Five reasons to model during QA
 
Model-Based Test Automation: Automate more, automate faster, automate accurately
Model-Based Test Automation: Automate more, automate faster, automate accuratelyModel-Based Test Automation: Automate more, automate faster, automate accurately
Model-Based Test Automation: Automate more, automate faster, automate accurately
 

Recently uploaded

Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
Ayan Halder
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
Łukasz Chruściel
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
Revolutionizing Visual Effects Mastering AI Face Swaps.pdf
Revolutionizing Visual Effects Mastering AI Face Swaps.pdfRevolutionizing Visual Effects Mastering AI Face Swaps.pdf
Revolutionizing Visual Effects Mastering AI Face Swaps.pdf
Undress Baby
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
TheSMSPoint
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
Rakesh Kumar R
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
Deuglo Infosystem Pvt Ltd
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Crescat
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
timtebeek1
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
ICS
 
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppAI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
Google
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Neo4j
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Aftab Hussain
 

Recently uploaded (20)

Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
Revolutionizing Visual Effects Mastering AI Face Swaps.pdf
Revolutionizing Visual Effects Mastering AI Face Swaps.pdfRevolutionizing Visual Effects Mastering AI Face Swaps.pdf
Revolutionizing Visual Effects Mastering AI Face Swaps.pdf
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 
Transform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR SolutionsTransform Your Communication with Cloud-Based IVR Solutions
Transform Your Communication with Cloud-Based IVR Solutions
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
 
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI AppAI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
AI Fusion Buddy Review: Brand New, Groundbreaking Gemini-Powered AI App
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
 

Curiosity and Testery Present: Hitting the right test coverage for CI/CD

  • 1. © Curiosity Software Ireland Ltd. 2022 Hitting the right test coverage for CI/CD November 10th 2022 James Walker Director of Technology, Curiosity. James.Walker@Curiosity.Software @CuriositySoft Chris Harbert Founder and CEO, Testery. Chris.Harbert@Testery.io @ChrisHarbert
  • 2. © Curiosity Software Ireland Ltd. 2022 Hitting the right test coverage for CI/CD 1. Why is automation lagging behind CI/CD? 2. Test automation and orchestration for CI/CD 3. Demo 4. Q&A and next steps
  • 3. © Curiosity Software Ireland Ltd. 2022 A team of experts here to support you An ethos of partnering with our customers ¼ Fortune 100 & plenty outside Inventors of multiple tools on the market Inventors in modelling, data, automation, and RQMTs 30+ years of experience
  • 4. © Curiosity Software Ireland Ltd. 2022 Experts that enable your test strategy Interact with your tests via the Web, command line, and API Run the right tests at the right time in your pipeline Run millions of tests per month Support most open-source testing frameworks Test runs that span environments, frameworks, or teams
  • 5. © Curiosity Software Ireland Ltd. 2022 Why is automation lagging behind CI/CD? Challenges in creating, maintaining and executing tests
  • 6. © Curiosity Software Ireland Ltd. 2022 Why CI/CD requires test automation • CI/CD and Devops practices result in more versions of the code are being deployed to more places more often. • Manual approaches struggle to keep up and produce cost effective, timely, and consistent results. 1.How can you create tests? 2.How can you maintain tests? 3.How can you execute the tests? 4.What about coverage?
  • 7. © Curiosity Software Ireland Ltd. 2022 When to move to automation Cost of testing Features Point at which operating costs recoup implementation costs
  • 8. © Curiosity Software Ireland Ltd. 2022 Options for creating tests 1.Writing the code for the tests 2.Record-and-playback 3.Model-based testing
  • 9. © Curiosity Software Ireland Ltd. 2022 Considerations for maintaining tests 1.Tests and code must both be updated as functionality changes 2.How many places will you have to update 3.How will you know what needs to be updated 4.When will you know it needs to change 5.How can you lower the cost of making that change 6.How will you keep track of different versions of changes
  • 10. © Curiosity Software Ireland Ltd. 2022 Challenges executing tests Running the right tests at the right time Running reliably at high levels of parallelism Matching code and test versions Running tests that span projects, environments, and frameworks
  • 11. © Curiosity Software Ireland Ltd. 2022 Challenges creating and maintaining tests 10001110101 10101110101 10001010101 10011110101 10001110111 ≠ Test creation remains manual Scripting is too repetitive Automating does not boost coverage Test data breaks automation Brittle tests create impossible maintenance Automation becomes a “black box” and silo
  • 12. © Curiosity Software Ireland Ltd. 2022 What about coverage? Components Many logical combinations. Modules E2E scenarios Combine Customer Apps Business Apps (CRM, ERP…) APIs Databases Third party systems Systems More tests than you can run in-sprint. Elements/objects/ endpoints/data Combinatorial explosion 10x10x10x10 = 10,000 possible test scenarios to script and maintain Combine Combine
  • 13. © Curiosity Software Ireland Ltd. 2022 Alternatives to scripted frameworks? Record/Playback No Code Keyword-driven Brittle tests need re-recording SUT is limited by tooling SUT is limited by tooling Locked into limited actions Fiddley keyword config Scripting to expand keywords Designing/recording tests remains linear, time-consuming and unsystematic What about coverage?!
  • 14. © Curiosity Software Ireland Ltd. 2022 Test automation and orchestration for CI/CD Auto-generate and execute targeted tests
  • 15. © Curiosity Software Ireland Ltd. 2022 Test generation for CI/CD Synchronise data from across the SDLC to generate targeted tests in-sprint Find / Make / Allocate Test Data Run Results Tests New Iteration  Seamless collaboration  Better RQMTs  Better code  Rigorous testing in-sprint  Quality with every release Test Automation Test Automation Test Cases User stories BDD/Gherkin Scans and Recordings Requirements Diagrams Living Documentation Cross-functional teams BAs, POs, testers, Dev, SDETs, DBAs. Find / Make / Allocate Test Data
  • 16. © Curiosity Software Ireland Ltd. 2022 Test orchestration for CI/CD Dev QA RC Prod Feature A Feature B Functional Tests Smoke Tests Regression Tests Monitoring Tests Determine which tests to run Decide when and where to run them Automatically scale infrastructure Respond to test failures
  • 17. Demo © Curiosity Software Ireland Ltd. 2022
  • 18. © Curiosity Software Ireland Ltd. 2022 Rapid releases with in-built quality Change User stories & Code changes Central source of truth Targeted tests and data, ready as soon as code is </> Execute tests Run Results ⁻ Across environments ⁻ Across the testing pyramid ⁻ Across repositories ⁻ Across systems Faster, less-buggy dev
  • 19. © Curiosity Software Ireland Ltd. 2022 Rapid releases with in-built quality Drive faster, less buggy development Maintain clear and complete specifications that reduce rework in development Shorten release cycles with bottleneck-free testing Generate and maintain tests in parallel to development, running them as soon as code checked-in Find bugs earlier, at less cost and impact to fix Target rigorous testing based on system logic and changes, testing across complex hybrid architectures
  • 20. © Curiosity Software Ireland Ltd. 2022 Questions and Discussion
  • 21. © Curiosity Software Ireland Ltd. 2022 Carry on the conversation? • Book a demo and discussion with a Curiosity expert! • Learn how to get started with Team Testery! • Try Test Modeller. Visit Testmodeller.io for a free trial and online tutorials. • Follow Curiosity Software on LinkedIn and @CuriositySoft on Twitter for resources and news. • Follow Testery on LinkedIn and @Testery_io on Twitter.
  • 22. © Curiosity Software Ireland Ltd. 2022 Thank you James.Walker@Curiosity.Software Chris.harbert@testery.io