SlideShare a Scribd company logo
1 of 38
Download to read offline
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Moral of the Story
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
What exactly is a
brittle test?
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
What exactly is a brittle test?
Fragile, flaky, easy to break
Not deterministic result
Run 1
Input
Run 2
Run 3
Run 4
Not due to only the test code
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Raise your hand if
you ever had to deal
with brittle tests!
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
What are the
consequences
of brittle tests?
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
High Maintenance Cost
Keyword
test 1
test 2
test 3
test 4
75% Fail
Rollercoaster result trend
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
High Maintenance Cost
Debug and manual test
Fix broken tests
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Real Example
Test 1...50 Precondition Step Assert
Nightly regression tests (1000+)
Most of the time working
This time many tests failed
RELEASE DROP
What would you do?
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Hidden Fail
Test 1 Precondition Step Assert
Test 50 Precondition Step Assert
Cognitive bias: test fails only because of the precondition
What if a real fail has been hidden?
Test 1 Precondition Step Assert
Test 50 Precondition Step Assert BUG!
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Useless feedback
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Low Trust - Low Morale
Worthless feedback ignored
Maintenance avoided
Tests get even more broken
Automation is not trusted
V
i
c
i
o
u
s
L
o
o
p
Effort with no
results
No result ==
No commitment
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
What makes our
tests brittle?
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
How can we defeat
the Dragon?
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Ice Cream
Testing Pyramid Real world: Ice Cream
Practical Test Pyramid Ham Vocke - Martin Fowler’s blog
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Ice Cream
Move
Down to the Pyramid
Tests
Part of tests
Setup & Teardown
Diamo una chance alle codebase legacy Nicola Mincuzzi
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Multipurpose / Holistic
Test 1 Setup Register Assert
Registration test:
Login Assert ... Assert
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Multipurpose / Holistic
“There should never be more than
one reason for a class to change”
Robert C. Martin (Uncle Bob)
“There should never be more than
one reason for a test to fail”
Single Responsibility Principle
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Multipurpose / Holistic
Test 1 Registration Check User
Single assert
Single Responsibility
Setup 1 Checkout
Setup 2 Streaming
Test 1 Registration
Test 2 Login
Test one thing
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Poor Design
Sleep
Best effort automation
Quick and Dirty
Longer learning curve
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Poor Design
Invest in automation
Agree on development guidelines
(Pekka’s how to write good test cases)
How to write good test cases using Robot Framework Pekka Clark
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Poor Design
Reduce technical debt
Pair development
Boy Scout Rule:
“Always leave the code you are editing better
than you found it” Robert C. Martin (Uncle Bob)
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Good and Bad Technical Debt
“Anything about your code that slows you down over the long term.
Hard-to-read code, duplication, tangled dependencies, etc.”
Good And Bad Technical Debt Henrik Kniberg
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Unreliable and slow environment
Undersized
Environment
Front End Developer
Dev - Ops experiment
Business test
Backend Mad Developer
(internal server error hero)
Super complicated auto implosion mega query
Third part never-working services
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Unreliable and slow environment
Dedicated test environment
Pay for limitless resources
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Unreliable and slow environment
Mock third party systems
Strive for performances
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Hard coded data
Limited products
Test product: 3E24B278-D1EC-358C-ADE5-F51703BBB342
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Hard coded Data
Data Preparation
Data Targeting
Teardown Matters
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
No continuous Testing
Test suite needs to be executed
People is lazy forget to run
Not comparable results
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
No continuous Testing
Continuous testing to collect feedback
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Remove/reduce barriers to test execution
No continuous Testing
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Daily Standup
No continuous Testing
Include test feedback
in release process
Discuss reports together
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Happy Ending
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Automate everything
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Quality assistance
Whole team in charge for quality
More TDD and pair programming
Quality Assistance over
Quality Assurance
BDD and TDD Outside in
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Robot Framework BDD
*** Settings ***
Library SeleniumLibrary
Resource settings.resource
*** Test Cases ***
App Game Available
Given The Browser
When Navigate To The App Game
Then Title Should Be Tic Tac Toe
*** Keywords ***
Given The Browser
Open Browser ${EMPTY} ${BROWSER}
When Navigate To The App Game
Go To ${APP_URL}
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Quality Assistance Team
Angelo Caovilla
Roberta Massimino
Fabrizio Premoli
Emanuele Bonanno
Art Director
Matteo Villa
All CHILI ex-colleagues
Special Thanks to
🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
Contacts
Angelo Caovilla
acaovilla
Robot Framework Milano
lucagiove
Luca Giovenzana
@luboxit

More Related Content

Similar to Beware The Brittle Dragon - A Test Automation Epic Tale

Scaling Your Tests: Continued Change Without Fear
Scaling Your Tests: Continued Change Without FearScaling Your Tests: Continued Change Without Fear
Scaling Your Tests: Continued Change Without FearTechWell
 
From Gatekeeper to Partner by Kelsey Shannahan
From Gatekeeper to Partner by Kelsey ShannahanFrom Gatekeeper to Partner by Kelsey Shannahan
From Gatekeeper to Partner by Kelsey ShannahanQA or the Highway
 
The Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can StealThe Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can Stealmozilla.presentations
 
An Agilist's Guide to Excelling on a Scrum Team as a Tester
An Agilist's Guide to Excelling on a Scrum Team as a TesterAn Agilist's Guide to Excelling on a Scrum Team as a Tester
An Agilist's Guide to Excelling on a Scrum Team as a TesterAndy Melichar
 
Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017
Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017
Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017Codemotion
 
The Missing Piece between Discovery and Execution
The Missing Piece between Discovery and ExecutionThe Missing Piece between Discovery and Execution
The Missing Piece between Discovery and ExecutionSteffen Kastner
 
Solving Flaky Automated Tests Using Machine Learning
Solving Flaky Automated Tests Using Machine LearningSolving Flaky Automated Tests Using Machine Learning
Solving Flaky Automated Tests Using Machine LearningJames Farrier
 
UX Without the U Is Your X
UX Without the U Is Your XUX Without the U Is Your X
UX Without the U Is Your XBrandon Ward
 
Building for resilience (with speaking notes)
Building for resilience (with speaking notes)Building for resilience (with speaking notes)
Building for resilience (with speaking notes)Abe Gong
 
Real World Lessons On The Anti-Patterns of Node.JS
Real World Lessons On The Anti-Patterns of Node.JSReal World Lessons On The Anti-Patterns of Node.JS
Real World Lessons On The Anti-Patterns of Node.JSBen Hall
 
What to Avoid When Writing Unit Tests - Robert Fornal
What to Avoid When Writing Unit Tests - Robert FornalWhat to Avoid When Writing Unit Tests - Robert Fornal
What to Avoid When Writing Unit Tests - Robert FornalQA or the Highway
 
Node.js Anti Patterns
Node.js Anti PatternsNode.js Anti Patterns
Node.js Anti PatternsBen Hall
 
Tips on Starting a Compelling Vlog
Tips on Starting a Compelling VlogTips on Starting a Compelling Vlog
Tips on Starting a Compelling VlogFrank La Vigne
 
Slaying The Legacy Dragon: Practical Lessons in Replacing Old Software
Slaying The Legacy Dragon: Practical Lessons in Replacing Old SoftwareSlaying The Legacy Dragon: Practical Lessons in Replacing Old Software
Slaying The Legacy Dragon: Practical Lessons in Replacing Old SoftwareTim Berglund
 
Performance - a challenging craft
Performance  - a challenging craftPerformance  - a challenging craft
Performance - a challenging craftFabian Lange
 
Leading with Tech Safety: An Unexpected Pathway to Excellence
Leading with Tech Safety: An Unexpected Pathway to ExcellenceLeading with Tech Safety: An Unexpected Pathway to Excellence
Leading with Tech Safety: An Unexpected Pathway to ExcellenceJoshua Kerievsky
 
Steve Krug Explains It All for You - SxSW 2011
Steve Krug Explains It All for You - SxSW 2011Steve Krug Explains It All for You - SxSW 2011
Steve Krug Explains It All for You - SxSW 2011Steve Krug
 
Akka in 100 slides or less
Akka in 100 slides or lessAkka in 100 slides or less
Akka in 100 slides or lessDerek Wyatt
 

Similar to Beware The Brittle Dragon - A Test Automation Epic Tale (20)

Scaling Your Tests: Continued Change Without Fear
Scaling Your Tests: Continued Change Without FearScaling Your Tests: Continued Change Without Fear
Scaling Your Tests: Continued Change Without Fear
 
TDD and Getting Paid
TDD and Getting PaidTDD and Getting Paid
TDD and Getting Paid
 
From Gatekeeper to Partner by Kelsey Shannahan
From Gatekeeper to Partner by Kelsey ShannahanFrom Gatekeeper to Partner by Kelsey Shannahan
From Gatekeeper to Partner by Kelsey Shannahan
 
The Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can StealThe Most Important Thing: How Mozilla Does Security and What You Can Steal
The Most Important Thing: How Mozilla Does Security and What You Can Steal
 
An Agilist's Guide to Excelling on a Scrum Team as a Tester
An Agilist's Guide to Excelling on a Scrum Team as a TesterAn Agilist's Guide to Excelling on a Scrum Team as a Tester
An Agilist's Guide to Excelling on a Scrum Team as a Tester
 
Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017
Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017
Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017
 
The Missing Piece between Discovery and Execution
The Missing Piece between Discovery and ExecutionThe Missing Piece between Discovery and Execution
The Missing Piece between Discovery and Execution
 
Solving Flaky Automated Tests Using Machine Learning
Solving Flaky Automated Tests Using Machine LearningSolving Flaky Automated Tests Using Machine Learning
Solving Flaky Automated Tests Using Machine Learning
 
UX Without the U Is Your X
UX Without the U Is Your XUX Without the U Is Your X
UX Without the U Is Your X
 
Coaching Testing
Coaching Testing Coaching Testing
Coaching Testing
 
Building for resilience (with speaking notes)
Building for resilience (with speaking notes)Building for resilience (with speaking notes)
Building for resilience (with speaking notes)
 
Real World Lessons On The Anti-Patterns of Node.JS
Real World Lessons On The Anti-Patterns of Node.JSReal World Lessons On The Anti-Patterns of Node.JS
Real World Lessons On The Anti-Patterns of Node.JS
 
What to Avoid When Writing Unit Tests - Robert Fornal
What to Avoid When Writing Unit Tests - Robert FornalWhat to Avoid When Writing Unit Tests - Robert Fornal
What to Avoid When Writing Unit Tests - Robert Fornal
 
Node.js Anti Patterns
Node.js Anti PatternsNode.js Anti Patterns
Node.js Anti Patterns
 
Tips on Starting a Compelling Vlog
Tips on Starting a Compelling VlogTips on Starting a Compelling Vlog
Tips on Starting a Compelling Vlog
 
Slaying The Legacy Dragon: Practical Lessons in Replacing Old Software
Slaying The Legacy Dragon: Practical Lessons in Replacing Old SoftwareSlaying The Legacy Dragon: Practical Lessons in Replacing Old Software
Slaying The Legacy Dragon: Practical Lessons in Replacing Old Software
 
Performance - a challenging craft
Performance  - a challenging craftPerformance  - a challenging craft
Performance - a challenging craft
 
Leading with Tech Safety: An Unexpected Pathway to Excellence
Leading with Tech Safety: An Unexpected Pathway to ExcellenceLeading with Tech Safety: An Unexpected Pathway to Excellence
Leading with Tech Safety: An Unexpected Pathway to Excellence
 
Steve Krug Explains It All for You - SxSW 2011
Steve Krug Explains It All for You - SxSW 2011Steve Krug Explains It All for You - SxSW 2011
Steve Krug Explains It All for You - SxSW 2011
 
Akka in 100 slides or less
Akka in 100 slides or lessAkka in 100 slides or less
Akka in 100 slides or less
 

Recently uploaded

Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Andreas Granig
 
architecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfarchitecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfWSO2
 
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Lisi Hocke
 
Salesforce Introduced Zero Copy Partner Network to Simplify the Process of In...
Salesforce Introduced Zero Copy Partner Network to Simplify the Process of In...Salesforce Introduced Zero Copy Partner Network to Simplify the Process of In...
Salesforce Introduced Zero Copy Partner Network to Simplify the Process of In...CloudMetic
 
Food Delivery Business App Development Guide 2024
Food Delivery Business App Development Guide 2024Food Delivery Business App Development Guide 2024
Food Delivery Business App Development Guide 2024Chirag Panchal
 
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypseTomasz Kowalczewski
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMarkus Moeller
 
Effective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeConEffective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeConNatan Silnitsky
 
^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto
^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto
^Clinic ^%[+27788225528*Abortion Pills For Sale In sowetokasambamuno
 
What is a Recruitment Management Software?
What is a Recruitment Management Software?What is a Recruitment Management Software?
What is a Recruitment Management Software?NYGGS Automation Suite
 
Community is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletCommunity is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletAndrea Goulet
 
^Clinic ^%[+27788225528*Abortion Pills For Sale In birch acres
^Clinic ^%[+27788225528*Abortion Pills For Sale In birch acres^Clinic ^%[+27788225528*Abortion Pills For Sale In birch acres
^Clinic ^%[+27788225528*Abortion Pills For Sale In birch acreskasambamuno
 
Lessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfLessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfSrushith Repakula
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024MulesoftMunichMeetup
 
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...OnePlan Solutions
 
A Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdfA Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdfICS
 
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit MilanWorkshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit MilanNeo4j
 

Recently uploaded (20)

Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
 
Abortion Clinic Pretoria ](+27832195400*)[ Abortion Clinic Near Me ● Abortion...
Abortion Clinic Pretoria ](+27832195400*)[ Abortion Clinic Near Me ● Abortion...Abortion Clinic Pretoria ](+27832195400*)[ Abortion Clinic Near Me ● Abortion...
Abortion Clinic Pretoria ](+27832195400*)[ Abortion Clinic Near Me ● Abortion...
 
architecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfarchitecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdf
 
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
 
Salesforce Introduced Zero Copy Partner Network to Simplify the Process of In...
Salesforce Introduced Zero Copy Partner Network to Simplify the Process of In...Salesforce Introduced Zero Copy Partner Network to Simplify the Process of In...
Salesforce Introduced Zero Copy Partner Network to Simplify the Process of In...
 
Food Delivery Business App Development Guide 2024
Food Delivery Business App Development Guide 2024Food Delivery Business App Development Guide 2024
Food Delivery Business App Development Guide 2024
 
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
[GeeCON2024] How I learned to stop worrying and love the dark silicon apocalypse
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdf
 
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
 
Effective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeConEffective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeCon
 
Abortion Clinic In Springs ](+27832195400*)[ 🏥 Safe Abortion Pills in Springs...
Abortion Clinic In Springs ](+27832195400*)[ 🏥 Safe Abortion Pills in Springs...Abortion Clinic In Springs ](+27832195400*)[ 🏥 Safe Abortion Pills in Springs...
Abortion Clinic In Springs ](+27832195400*)[ 🏥 Safe Abortion Pills in Springs...
 
^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto
^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto
^Clinic ^%[+27788225528*Abortion Pills For Sale In soweto
 
What is a Recruitment Management Software?
What is a Recruitment Management Software?What is a Recruitment Management Software?
What is a Recruitment Management Software?
 
Community is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletCommunity is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea Goulet
 
^Clinic ^%[+27788225528*Abortion Pills For Sale In birch acres
^Clinic ^%[+27788225528*Abortion Pills For Sale In birch acres^Clinic ^%[+27788225528*Abortion Pills For Sale In birch acres
^Clinic ^%[+27788225528*Abortion Pills For Sale In birch acres
 
Lessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfLessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdf
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
 
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
Optimizing Operations by Aligning Resources with Strategic Objectives Using O...
 
A Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdfA Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdf
 
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit MilanWorkshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
Workshop: Enabling GenAI Breakthroughs with Knowledge Graphs - GraphSummit Milan
 

Beware The Brittle Dragon - A Test Automation Epic Tale

  • 1. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla
  • 2. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Moral of the Story
  • 3. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla What exactly is a brittle test?
  • 4. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla What exactly is a brittle test? Fragile, flaky, easy to break Not deterministic result Run 1 Input Run 2 Run 3 Run 4 Not due to only the test code
  • 5. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Raise your hand if you ever had to deal with brittle tests!
  • 6. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla What are the consequences of brittle tests?
  • 7. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla High Maintenance Cost Keyword test 1 test 2 test 3 test 4 75% Fail Rollercoaster result trend
  • 8. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla High Maintenance Cost Debug and manual test Fix broken tests
  • 9. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Real Example Test 1...50 Precondition Step Assert Nightly regression tests (1000+) Most of the time working This time many tests failed RELEASE DROP What would you do?
  • 10. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Hidden Fail Test 1 Precondition Step Assert Test 50 Precondition Step Assert Cognitive bias: test fails only because of the precondition What if a real fail has been hidden? Test 1 Precondition Step Assert Test 50 Precondition Step Assert BUG!
  • 11. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Useless feedback
  • 12. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Low Trust - Low Morale Worthless feedback ignored Maintenance avoided Tests get even more broken Automation is not trusted V i c i o u s L o o p Effort with no results No result == No commitment
  • 13. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla What makes our tests brittle?
  • 14. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla How can we defeat the Dragon?
  • 15. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Ice Cream Testing Pyramid Real world: Ice Cream Practical Test Pyramid Ham Vocke - Martin Fowler’s blog
  • 16. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Ice Cream Move Down to the Pyramid Tests Part of tests Setup & Teardown Diamo una chance alle codebase legacy Nicola Mincuzzi
  • 17. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Multipurpose / Holistic Test 1 Setup Register Assert Registration test: Login Assert ... Assert
  • 18. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Multipurpose / Holistic “There should never be more than one reason for a class to change” Robert C. Martin (Uncle Bob) “There should never be more than one reason for a test to fail” Single Responsibility Principle
  • 19. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Multipurpose / Holistic Test 1 Registration Check User Single assert Single Responsibility Setup 1 Checkout Setup 2 Streaming Test 1 Registration Test 2 Login Test one thing
  • 20. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Poor Design Sleep Best effort automation Quick and Dirty Longer learning curve
  • 21. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Poor Design Invest in automation Agree on development guidelines (Pekka’s how to write good test cases) How to write good test cases using Robot Framework Pekka Clark
  • 22. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Poor Design Reduce technical debt Pair development Boy Scout Rule: “Always leave the code you are editing better than you found it” Robert C. Martin (Uncle Bob)
  • 23. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Good and Bad Technical Debt “Anything about your code that slows you down over the long term. Hard-to-read code, duplication, tangled dependencies, etc.” Good And Bad Technical Debt Henrik Kniberg
  • 24. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Unreliable and slow environment Undersized Environment Front End Developer Dev - Ops experiment Business test Backend Mad Developer (internal server error hero) Super complicated auto implosion mega query Third part never-working services
  • 25. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Unreliable and slow environment Dedicated test environment Pay for limitless resources
  • 26. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Unreliable and slow environment Mock third party systems Strive for performances
  • 27. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Hard coded data Limited products Test product: 3E24B278-D1EC-358C-ADE5-F51703BBB342
  • 28. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Hard coded Data Data Preparation Data Targeting Teardown Matters
  • 29. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla No continuous Testing Test suite needs to be executed People is lazy forget to run Not comparable results
  • 30. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla No continuous Testing Continuous testing to collect feedback
  • 31. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Remove/reduce barriers to test execution No continuous Testing
  • 32. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Daily Standup No continuous Testing Include test feedback in release process Discuss reports together
  • 33. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Happy Ending
  • 34. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Automate everything
  • 35. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Quality assistance Whole team in charge for quality More TDD and pair programming Quality Assistance over Quality Assurance BDD and TDD Outside in
  • 36. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Robot Framework BDD *** Settings *** Library SeleniumLibrary Resource settings.resource *** Test Cases *** App Game Available Given The Browser When Navigate To The App Game Then Title Should Be Tic Tac Toe *** Keywords *** Given The Browser Open Browser ${EMPTY} ${BROWSER} When Navigate To The App Game Go To ${APP_URL}
  • 37. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Quality Assistance Team Angelo Caovilla Roberta Massimino Fabrizio Premoli Emanuele Bonanno Art Director Matteo Villa All CHILI ex-colleagues Special Thanks to
  • 38. 🐉 Beware the Brittle Dragon - Luca Giovenzana & Angelo Caovilla Contacts Angelo Caovilla acaovilla Robot Framework Milano lucagiove Luca Giovenzana @luboxit