The Death of Flaky Tests by Dave Haeffner
Dave Haeffner is here to bury flaky tests, not to praise them. For far too long we have allowed flaky tests to live in our world, creeping in and eroding the trust in our Selenium tests. They have wrought unreliable test results, noise, and ultimately an infinite amount of time wasted throughout our industry. Some blame Selenium for this. Others point a finger at the browser vendors. Some call out either instabilities in their Application Under Test or poor design practices in their test code. Flaky Tests are often talked about in whispers, and ultimately accepted as a part of reality that we cannot control. Dave says it’s time for a revolution! Join him in this SauceCon 2018 presentation as he shows you how to take the power back from Flaky Tests and end their reign once and for all.
Recommended
More Related Content
What's hot
What's hot (11)
Similar to The Death of Flaky Tests by Dave Haeffner
Similar to The Death of Flaky Tests by Dave Haeffner (20)
More from Sauce Labs
More from Sauce Labs (20)
Recently uploaded
Recently uploaded (20)
The Death of Flaky Tests by Dave Haeffner
- 1. The Death of Flaky Tests By Dave Haeffer, @TourDeDave
- 3. http://bit.ly/saucecon18-1 ~1.5% of runs report a flaky result ~16% of tests have some level of flakiness Re-runs cited as a common mitigation strategy
- 5. “a re-run culture is toxic” “Flaky” tests that caught a bug that was missed Your application is flaky
- 7. “take flake back” - #hooray4flake Tracked stats for individual test runs using standard deviation At a glance way to spot tests which require attention http://testarmada.io/
- 9. You can avoid flaky tests with good test design practices.
- 11. Flakiness is something we design into our tests. Flakiness is an opportunity to improve, don’t ignore it. Acronym and questions.
- 12. http://bit.ly/saucecon18-6 If you do not treat test automation well, why are you expecting miracles?
- 17. 100 0 100 Code Score (%) RunResult(Pass%) Individual Test Matrix Reliable External Issues Likely To Fail Broken
- 19. Available tooling doesn’t cater to people with existing Selenium code. Best practices are all around us, but we’re not all using them. Let’s codify these practices into automated feedback loops to make the path to successful automated testing simpler for everyone.
- 20. Image sources: https://thenounproject.com/icon/1532282/ https://thenounproject.com/icon/947849/ Created by dDara from the Noun Project Created by Adrien Coquet from the Noun Project Test Code In Flight Test Code At Rest
- 21. NOTE: You Need Test Run Data Too Created by dDara from the Noun Project Created by Adrien Coquet from the Noun Project Abstract Event Listener Tapping The Wire Protocol Log Parsing Static Code Analysis
- 22. 100 0 100 Code Score (%) RunResult(Pass%) Individual Test Matrix Reliable External Issues Likely To Fail Broken
- 27. Abstract Event Listener C# - http://bit.ly/saucecon2018-12 Java - http://bit.ly/saucecon2018-13 Python- http://bit.ly/saucecon2018-14 Ruby - http://bit.ly/saucecon2018-15 JS - http://bit.ly/saucecon2018-16
- 28. Static code analysis a.k.a. “linting” “A linter or lint refers to tools that analyze source code to flag programming errors, bugs, stylistic errors, and suspicious constructs. The term is originated from a Unix utility that examined C language source code.” https://en.wikipedia.org/wiki/Lint_(software)
- 29. Static code analysis a.k.a. “linting” “A linter or lint refers to tools that analyze source code to flag programming errors, bugs, stylistic errors, and suspicious constructs. The term is originated from a Unix utility that examined C language source code.” https://en.wikipedia.org/wiki/Lint_(software)
- 30. Outro ● Zoom if (just a little bit) ● Remove “faayy from our vocabulary ● Let’s truly treat test code liae app code by oferif equivaleft toolif http://davehaeffner.com/contact/