This webinar was hosted live on 28th January 2023. Watch the on demand recording here: https://www.curiositysoftware.ie/modelling-continuous-testing-webinar Teams striving to “do” DevOps and test continuously often reach for nearby tools, without first fixing their mindset and techniques. This often backfires long-term, as sticking-plaster solutions introduce anti-patterns, uncertainty, and repetition to tight sprints. Monolithic automation frameworks break and need fixing as developers make changes to systems coded using more modern techniques. Further maintenance emerges as tests, data and environments become misaligned. And has anyone stopped to consider how well you understand the system being developed, given its complexity exceeds human comprehension and has been reflected in the bewildering pipelines designed to test it? This webinar will consider how you can apply a DevOps mindset to Continuous Testing, before exploring tools and techniques to support it. It will discuss how you can componentise vastly complex systems into understandable chunks, while mapping each component to the automation, data and environments needed to test it. You will see how cross-functional teams can combine reusable models to map complex systems, while generating tests, data and environments for in-sprint testing. As complex systems change, updating individual models and automation libraries regenerates rigorous tests, maintaining both the living documentation and aligned artifacts needed for Continuous Testing. Watch the webinar on demand - https://www.curiositysoftware.ie/modelling-continuous-testing-webinar

1. © Curiosity Software Ireland Ltd. 2023
Modelling for Continuous
Testing
February 28th 2023
Ben Riley
CIO, Curiosity
Ben.Riley@Curiosity.Software
@CuriositySoft
Rich Jordan
Enterprise Solutions Architect, Curiosity
Rich.Jordan@Curiosity.Software
@RichJordan81

2. © Curiosity Software 2023
You have a DevOps Transformation, but your Test
approach hasn’t really changed
• You have a large team of testers doing mainly e2e tests (even if they call it something else).
• You have a test pack largely made up of UI based e2e tests which always grows after each release – you suspect
there’s lots of duplicated effort in there.
• Testing is delayed because you are waiting on large, complicated e2e environments to be built, or waiting for your
slot in the few e2e environments that persist in your organization.
• Test coverage relies on traceability back to requirements that you know are incomplete or out of date.
• Test design is largely an activity that involves testers creating test cases without input from the wider development
team. Alternatively, you don’t design test at all; you just write test cases/automation.
• A large % of your defects are environment/config related, causing large delays in test phases.
• You are finding fundamental design issues in late, large, slow e2e test phases.
• Despite trying many different tools, you are struggling to increase the rate of tests executed through automation as
the tests are just too flaky!
Original Image:
Khanargy, Wikimedia
Commons, published
under CC BY-SA 4.0
license.

3. © Curiosity Software 2023
Your Test approach and the Testing
Pyramid
• Traditional UI testing is too slow
• Doesn’t lend itself to stable,
repeatable automation at scale
• Quickly requires lots of real e2e
environments
• Doesn’t get to the real route
cause of problems

4. © Curiosity Software 2023
DevOps: what are the 3 ways of
DevOps Principle
The First Way: Flow/Systems Thinking.
The Second Way: Amplify Feedback Loops.
The Third Way: Culture of Continual Experimentation and
Learning.
The Three Ways: The Principles Underpinning DevOps
by Gene Kim / IT Revolution
https://itrevolution.com/articles/the-three-ways-principles-
underpinning-devops/
Original Image: Khanargy, Wikimedia Commons,
published under CC BY-SA 4.0 license.

5. © Curiosity Software 2023
Flow
• Never passing a known defect to downstream work centers
• Never allowing local optimization to create global degradation
• Always seeking to increase flow
• Always seeking to achieve profound understanding of the system
What do testers actually spend their time doing?
Typical Test team effort
Comprehension of system /
change
30%
(Automation) Test script
creation/maintenance
20%
Test execution 20%
Waiting to test 5%
Evidencing / Reporting 10%
Other (Meetings / Self-
Development)
15%
100%
Sample represents 50 testers
Software entropy is the idea
that software eventually rots as it is
changed if sufficient care is not taken to
maintain coherence with product design
and established design principles.
Managing Software entropy
1.Improve the communication in your team.
2.Maintain coherence with product design
and established design principles

6. © Curiosity Software 2023
Flow
• Never passing a known defect to downstream work centers
• Never allowing local optimization to create global degradation
• Always seeking to increase flow
• Always seeking to achieve profound understanding of the system
Diagram adapted from John Ferguson Smart’s “BDD
at the heart of any DevOps Transformation”
3) We relentlessly automate
testing and checking of the
models in test execution!
If we don’t know the
outcomes, we find out
through the process of
building the models…
we ask critical
questions and do
critical thinking
1), 2) We build a common understanding of the
stack and how its changing through collaborative
teams, building models with engineered,
predictable & repeatable outcomes!
4) We execute all checks as
part of the standard CI/CD
process!
4a) We are driven by the metric
we gather to provide quality and
lean improvements wherever they
exist
5) We play an active role in
promoting best practices and
celebrating continual
improvements
Illustrate
Formulae
Automate
Validate
Demonstrate
Feedback
Visibility
Valuable Software
Testers
Business Rules
Examples
A Shared Understanding
Executable Specifications,
Wireframes…
Automated Acceptance Tests
Living Documentation
1
2
3
4
5
BA
Business
Developers

7. © Curiosity Software 2023
Amplify Feedback Loops • Shortening and amplifying all feedback loops
• Embedding knowledge where we need it.
• Shifting Test left in engaging design – robust test design of rules and flows – we build models
• Quick to build, stable and quick to run automated tests
• Fail fast and fail early – once built, API test runs in line with DevOps cadence for delivery
orchestrated inline with build deployment
• Iterative building block approach enables Continuous Integration delivery
Manual Tests
Integration Tests
Unit Tests
E2E Tests
(UI Testing)
What the automation pyramid
says good looks like
UI / UX / Look and Feel /GUI
Infrastructure
Device Client / Functionality / API
Interface / SOA Rules / Services
System of Record / ETL Data Transition Rules /
Batch Services Comparison Rules Engines
E2E
Acceptance
UI Channel
Change
Going Beyond The Test Pyramid
(everything is a model)
App / Client Functional
Change / Contracts
Platform /
Infra Build
Interface Change /
Contracts
ETL Validation / Big
Data
Stubs mapped against understood coverage and version-
controlled interface spec
Stubs mapped against understood coverage and version-
controlled interface spec

8. © Curiosity Software 2023
Shortening and amplifying feedback loops –
modelling for isolation
1) Reinforcing / proving the external boundaries
2) Reinforcing / proving the internal boundaries
3) Proving the systems elements in isolation
4) Proving batch / reporting
5) Proving the golden thread of the platform
6) Stakeholder demonstration
The more this last stage deemed to be testing,
the more monolithic the platform and change
will become

9. © Curiosity Software 2023
Culture of Continual Experimentation
and Learning. • Continual experimentation
• Taking risks and learning from failure
• Understanding that repetition and practice is the prerequisite to mastery
Models help us to understand our systems & change impact
Refactor and optimize based on isolated risk
Minimized the blast radius (the impact of failure)

10. Demo
© Curiosity Software Ireland Ltd. 2023

11. © Curiosity Software 2023
Flow
• Never passing a known defect to downstream work centers
• Never allowing local optimization to create global degradation
• Always seeking to increase flow
• Always seeking to achieve profound understanding of the system
Diagram adapted from John Ferguson Smart’s “BDD
at the heart of any DevOps Transformation”
3) We relentlessly automate
testing and checking of the
models in test execution!
If we don’t know the
outcomes, we find out
through the process of
building the models…
we ask critical
questions and do
critical thinking
1), 2) We build a common understanding of the
stack and how its changing through collaborative
teams, building models with engineered,
predictable & repeatable outcomes!
4) We execute all checks as
part of the standard CI/CD
process!
4a) We are driven by the metric
we gather to provide quality and
lean improvements wherever they
exist
5) We play an active role in
promoting best practices and
celebrating continual
improvements
Illustrate
Formulae
Automate
Validate
Demonstrate
Feedback
Visibility
Valuable Software
BA Testers
Business
Developers
Business Rules
Examples
A Shared Understanding
Executable Specifications,
Wireframes…
Automated Acceptance Tests
Living Documentation
1
2
3
4
5

12. © Curiosity Software Ireland Ltd. 2023
Questions and Discussion

13. © Curiosity Software 2023
Carry on the conversation?
• Book a demo and discussion with a Curiosity expert!
• Try Test Modeller. Visit Testmodeller.io for a free trial and online tutorials.
• Follow Curiosity Software on LinkedIn and @CuriositySoft on Twitter for
resources and news.
• Connect with Ben and Rich on LinkedIn.

14. © Curiosity Software Ireland Ltd. 2023
Thank you
Ben.Riley@Curiosity.Software
Rich.Jordan@Curiosity.Software