With automation rapidly becoming more a important part of any testing process, teams rely ever more on the result of their automated tests as an indicator of product quality. In moving towards a Continuous Delivery or Continuous Deployment model, automated tests are often the only gatekeeper before software is put into production. But can automation be trusted enough to play this important role in the software development and delivery process? And what can you do to make sure your automation can be trusted enough to be relied on?
In this talk, we'll take a look at the relationship between trust, confidence and test automation. We'll see a number of ways in which test automation can deceive you, how trust in test automation is lost and how you can build tests you can trust on.
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Who will guard the guards themselves? How to trust your automation and avoid deceit
1. Who will guard the
guards themselves?
How to trust your automation and avoid deceit
Bas Dijkstra
www.ontestautomation.com
bas@ontestautomation.com
@_basdijkstra
2. _I build test automation solutions
_I teach test automation, service virtualization
_I write www.ontestautomation.com
TechBeacon, StickyMinds, …
6. Confidence
“Our product is
able to perform
function X”
Control Trust
“I control the
execution of X”
“I trust our
product to be able
to perform X”
7. I want to be confident about the ability of our
product to perform function X
Automated tests are used to build trust in our
product performing function X correctly
I need to be able to trust my automated tests
around function X
8. “I’m not upset that
you lied to me,
I’m upset that from
now on I can’t
believe you”
--- Friedrich Nietzsche ---
9. False positives
Persistent > fix it or fire it
Intermittent > ‘flaky tests’
https://youtu.be/XnkWkrbzMh0
Richard Bradshaw – Selenium Conf Berlin
19. If you want to trust your tests,
start building tests you can trust on
20.
21. Keep your tests fresh
Do code review on your test code
Test your tests periodically
Review your suite from time to time
Don’t be afraid to throw away tests
CD requires the ability to test continuously, every deployment should be thoroughly tested before being put into a production environment
Test automation plays a big role in this
Test automation is a first step towards CT, but just doing test automation isn’t enough
And what’s a well known way to build trust in an application’s abilities? Yes, testing.
If you're in doubt whether something should be an automated end-to-end test, chances are high the answer is 'no'
Avoid E2E test automation as much as possible
Vervelend, maar niet per se gevaarlijk
Wel risico dat in de loop van de tijd tests bekend staan dat ze falen (oh vergeet die maar) > waardeloos!
Vervelend, maar niet per se gevaarlijk
Wel risico dat in de loop van de tijd tests bekend staan dat ze falen (oh vergeet die maar) > waardeloos!
Vervelend, maar niet per se gevaarlijk
Wel risico dat in de loop van de tijd tests bekend staan dat ze falen (oh vergeet die maar) > waardeloos!
Vervelend, maar niet per se gevaarlijk
Wel risico dat in de loop van de tijd tests bekend staan dat ze falen (oh vergeet die maar) > waardeloos!
Vervelend, maar niet per se gevaarlijk
Wel risico dat in de loop van de tijd tests bekend staan dat ze falen (oh vergeet die maar) > waardeloos!
Vervelend, maar niet per se gevaarlijk
Wel risico dat in de loop van de tijd tests bekend staan dat ze falen (oh vergeet die maar) > waardeloos!
Vervelend, maar niet per se gevaarlijk
Wel risico dat in de loop van de tijd tests bekend staan dat ze falen (oh vergeet die maar) > waardeloos!
Code review op testcode
Test de tests tijdens het schrijven ervan
Test de tests periodiek
Code review op testcode
Test de tests tijdens het schrijven ervan
Test de tests periodiek
Property-based tests make statements about the output of your code based on the input, and these statements are verified for many different possible inputs.
Code review op testcode
Test de tests tijdens het schrijven ervan
Test de tests periodiek