Who will test your tests?
•
1 like•808 views
This talk was given by Priti Biyani and me at XP Conference, Bangalore 2015. It basically talks about non-deterministic tests and the problems due to these tests. It also outlines the causes and solutions to solve this problem.
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (19)
Similar to Who will test your tests?
Similar to Who will test your tests? (20)
Recently uploaded
Recently uploaded (20)
Who will test your tests?
- 1. WHO WILL TEST YOUR TESTS ? Yahya Poonawala @meetykp Priti Biyani @pritibiyani
- 3. TESTSCode works Reduce Bugs Reduce cost of change Allow Refactoring Faster development Reduce fear 3
- 4. NOW WHEN YOU HEAR …. 4
- 5. Oh, it failed? Don’t worry. Just re-trigger it! 5
- 6. Oh, it failed? Strange; it passed on my machine! 6
- 7. Oh, it failed? Run it on agent 007! It’s an agent issue! 7
- 8. Oh, it failed? I know. It fails at 12AM! 8
- 9. 9
- 10. Now, again, WHY DO WE WRITE TESTS? 1 0
- 11. Tests prove that the code actually works Tests prove that the code sometimes works 1 1
- 12. Tests reduce bugs Tests itself might have bugs 1 2
- 13. Tests reduce the cost of change Change? Are you kidding? 1 3
- 14. Tests helps in faster development. Testing is slowing me down L 1 4
- 17. 1 7
- 19. 1 9
- 22. 2 2
- 23. 2 3
- 24. DO NOT DELETE YOUR FLAKY TESTS!! 2 4
- 26. Thread 1::! " " if( name is not Yahya ) {! " " " " " "…! " " " " " "fputs( name, file )! " " " " " "…! " " " " "}! ! Thread 2::! " " Initialize name to Yahya! !
- 27. CONCURRENCY 2 7
- 28. Initialize lock;! ! Thread 1::! " " Acquire lock! " " " " "if( name is not Yahya) {! " " " " " ". . . ! " " " " " "fputs( name, file )! " " " " " ". . .! " " " " "}! " " " " "Release lock! ! Thread 2::! " " Acquire lock! " " " " " Initialize name to"Yahya! " " " " " Release lock! ! !
- 33. ! !# pseudo-code ! !! ! !makeAsyncCall; ! !sleep(2000); ! !readResponse; !
- 35. ! ! "make asynchronous call;! ! "While (response received?)! " "if timeout then ! " " " "throw Test Timeout Exception ! "! " else! " " "sleep(10)! . . . ! ! "read response! ! !
- 37. # //pseudo-code! FILE_NAME = “content.txt”! ! // Test 1:! ! "FileReader.read(FILE_NAME).to be empty! ! // Test2: Write to the file! ! "data = “Priti”! "FileWriter.write(FILE_NAME, data)! "FileReader.read(FILE_NAME) == data! ! // Test 3: Updates the file! ! "append_data = “Biyani”! "complete_data = “Priti Biyani”! FileWrite.append(FILE_NAME, append_data)! FileReader.read(FILE_NAME) == complete_data! !
- 40. 4 0
- 42. STOP CALLING YOUR BUILD FLAKY!! 4 2
- 44. IT’S NOT ALWAYS YOUR TESTS 4 4
- 45. DO NOT GET OVERWHELMED BY FAILING TESTS 4 5
- 46. REMEMBER! 4 6
- 47. LOVE YOUR TEST CODE ! 4 7
- 48. There is no such thing as FLAKY TEST! If its flaky, its not a test ! 4 8
- 50. REFERENCES • An Empirical Analysis of Flaky tests • Eradicating Non-Determinism in Tests – Martin Fowler • No more flaky tests on the GO team - Pavan Sudarshan • Top 12 reasons to write unit tests • http://tech.just-eat.com/2015/03/30/how-to- fix-flaky-tests/ 5 0