Curiosity software Ireland and Perfecto present: achieving in-sprint regression- convert continuous feedback into rigorous automated testing

© Curiosity Software Ireland Ltd. 2020
Achieving In-Sprint Regression
Convert Continuous Feedback Into Rigorous Automated Testing
September 9th 2020
Julius Mong
Senior Solutions Engineer, Perfecto
Jmong@perforce.com
James Walker, PhD
Director, Curiosity Software Ireland
James.Walker@Curiosity.Software

Achieving In-Sprint Regression
1. Fireside chat: why isn’t in-sprint regression widespread?
3. Demo: Testing continuously with model-based testing
4. Q&A and discussion
2. Why model-based test automation?

Global Reach
• Customers in 80 countries
• Over 20,000 customers worldwide
• 400,000 users strong
• More than 250 of the Fortune 500
• 14 brands covering full SDLC
• 20+ offices and 11 data centers which
give us global vision and reach
• Over 800 employees in 25 countries

Automation At Scale
TESTING AT
ENTERPRISE SCALE
FIND BUGS
FASTER
FIX BUGS
FASTER
SHIP STABLE CODE
FASTER
ACHIEVE AUTOMATION
GOALS QUICKLY
OPEN & INTEGRATED
AUTOMATION
TRUSTED DEVOPS
LEADER
REDUCED RISK OF
ESCAPED DEFECTS

Curiosity Software Ireland
• In 2017, Curiosity launched with technologies and
expertise built on decades of experience.
• We’ve built test data solutions since 1995, when
our MD founded the first of 5 pioneering vendors.
• Today we’re your specialist vendor in
requirements modelling, test data automation,
and model-based test automation
-Our global presence-
-And trusted partners in every continent-
Curiosity locations

1. What Challenges
Are You Seeing?
What’s blocking you from achieving in-sprint regression testing?

What Continuous Testing looks like
CONTINUOUS

What most DevOps processes look like
2-3 WEEKS 1-3 WEEKS
❌ Unstable
❌ Labor-intensive
❌ Cluttered
❌ Slow

Poll: What are the barriers to in-sprint
regression testing at your organisation?

Insufficient Test Coverage

Insufficient Test Coverage
Are your users testing your app before you do?
Manual testing is boring and costly
What’s stopping you from automating?
Can you test real user journeys?

Lab/Environment
To cloud or not to cloud? Is it a real cloud?
What about security?
Can you scale easily?
Public vs private lab
Is it the team that will support you or just a lab?

Feedback and Analysis Is Too Slow
• Still crunching through test reports?
• Make quick informed decisions.
• Top failure reasons for your tests?
• False negatives / positives?
• Can you do RCA efficiently?

Pre-execution blockers to in-sprint
testing
What’s going wrong pre-execution? 5 common blockers:
1. Test creation remains wholly manual (and slow!)
2. Manual test case design or scripting cannot achieve sufficient
coverage for complex systems, which are released at the risk of
bugs.
3. Test data destabilises automation and creates bottlenecks.
4. Brittle scripts need maintaining manually after every change.
5. Automation code is not easily re-usable, over-working skilled
engineers.

Change is a real headache…
Change requests/user
stories flow in
A code change made to a
maze of components
Which tests are
affected? Which need
updating?
What has been
impacted by the
change? What
needs re-testing?
Test Automation Engineer
Automated test maintenance kills the ROI of automation
No formal mapping: code requirements
Code
Commit

“Information hops” leak time & quality
Silo
‘Agile’, Continuous Testing, “DevOps”… Or mini-waterfalls?
Wall Information Hop Delay
No feedback
or traceability
Dev Automation EngineersQA/TestBAs
Effort: Repetitious.
“Flat” formats prevent
automation in QA & Dev.
Quality:Most bugs
originate in incomplete,
ambiguous requirements.
Effort: Re-coding after
misunderstood designs
and uncaught bugs.
Quality:Bugs are more
damaging the later they
are discovered.
Effort: Slowly create
and maintain 1000s of
test cases. Wait for data.
Quality:Manually
derived tests hit a
fraction of system logic.
Effort: Repetitious
scripting, growing
maintenance.
Quality: Data clashes,
false positives, false
negatives, low coverage.
Formulate complex requirements
and change requests in
disparate, unconnected formats.
Convert the incomplete designs
into code, trying to fix the impact
of changes on complex systems.
Unsystematically create high
volume, low coverage test cases,
mismatched with system designs.
Choose between creating new
scripts for new logic or
maintaining existing code.

2. Testing continuously with
model-based testing
Convert Continuous Feedback Into Rigorous Automated Testing

Model-Based Test
Automation
Recorded
tests activity
Test Cases
BPMN
diagrams
BDD
Scenarios
Business
Requirements
Application Scans
Reactive Model
of the system
Find / Make / Allocate Test Data
Run Results
Tests
New
Iteration
 Perfect sets of tests
 Fast Maintenance
 Reusable Components
 Move faster
 Deliver with confidence
Test AutomationTest Automation

MBTA busts pre-execution
barriers to in-sprint regression
MBTA works as an accelerator, not a replacement. It allows you to:
• Minimize duplicate effort and avoid repetitious test scripting – engineers focus on
creating what’s needed. Cross-functional can re-use the code!
• Automate maintenance of existing code – focus on newly added/updated logic.
• No test data frustrations – “Just in time” data avoids frustrating data clashes, false
positives, and false negatives.
• Optimise tests for quality - automation finds bugs before they hit production.

Create Models of
Requirements
Find / Make / Allocate Test Data
Scan web applications to
create robot resources
Export automated test
scripts
Make / Find Test data
Scripts are
automatically pushed
to source control
management system
Automation Tests
Run results
Embedded Data
Automation Execution
Continuous Delivery Tooling
Product Owner
Tester
Tester
Test execution
Perfecto handles
the rest

Demo
1. Test Modeller: In-sprint test design and maintenance
2. Perfecto Analytics: Cross-platform execution, CI dashboards, & heatmaps

Fast Feedback Loop
• CI Dashboard
• Heatmaps
• RCA breadcrumbs
• Unique test report URLs
• Single pane of glass for all stakeholders

Breaking the barriers to In-
Sprint Regression
Barriers to In-Sprint Regression Model Based Test Automation
✘ Slow and repetitious test creation. ✓ Auto-generate tests from quick-to-build flowcharts.
✘ Low Test Coverage ✓ Apply coverage algorithms to create the smallest set of tests needed for rigorous testing.
✘ Test data creates bottlenecks and
destabilises automation.
✓ Define dynamic data functions that resolve during execution for each automated test, or
use high-speed data allocation to automatically find or make data for every possible test!
✘ Manual test maintenance. ✓ Simply update the easy-to-maintain flowchart models and auto-update tests.
✘ The central team of engineers are
over-burdened.
✓ Custom-created automation recipes are available in a flexible “low code” approach,
allowing anybody to automate their tests. Subflows are also fully re-usable.
✘ Poor device/platform coverage ✓ Cloud-based execution across a complete range of devices, platforms and systems
✘ Hard-to-simulate prod. behaviour ✓ Cloud environments simulate complex production configurations seamlessly
✘ Time lost to run result analysis ✓ Test execution automatically populates granular dashboards

Questions and Discussion

Carry on the conversation?
Fill out the post-webinar survey – we’d love to hear from
you!
• Visit Testmodeller.io for a free trial. Use our free Perfecto tutorial
to get started with in-sprint regression.
• We’ll email out the slides, a webinar recording and ways you
can get in touch.

Thank you for listening
Email us if you have any questions or feedback, or
would like a demo of any of the technology shown
Julius Mong
Senior Solutions Engineer, Perfecto
Jmong@perforce.com
James Walker, PhD
Director, Curiosity Software Ireland
James.Walker@Curiosity.Software

Curiosity software Ireland and Perfecto present: achieving in-sprint regression- convert continuous feedback into rigorous automated testing

More Related Content

What's hot

Similar to Curiosity software Ireland and Perfecto present: achieving in-sprint regression- convert continuous feedback into rigorous automated testing

More from Curiosity Software Ireland

Recently uploaded

Curiosity software Ireland and Perfecto present: achieving in-sprint regression- convert continuous feedback into rigorous automated testing