The document discusses best practices for continuous integration and automated testing including keeping the build green, advantages of a green build, test-driven development, continuous integration, continuous deployment, making the build fast, running tests in parallel, using headless browsers, dealing with flaky tests, ensuring tests are accurate and readable, and rejecting broken builds.

1. Evergreen Build Dima Kovalenko

2. About me Likes cheese @dimacus www.agilesoftwaretesting.com

3. Where I worked

4. Where I work www.groupon.ru

5. Why am I here

6. Why am I here

7. Why am I here

8. Why am I here Green build Auto merge Auto revert

9. Why have automated tests? Expensive

10. Expensive

11. Why have automated tests? Expensive A lot of work

12. Why have automated tests? Expensive A lot of work Always breaking

13. Always breaking

14. Why have automated tests? Expensive A lot of work Always breaking Cannot make money from them

15. Advantages of green build A healthy way of life

17. Advantages of green build A healthy way of life Confidence in the code base

18. Advantages of green build A healthy way of life Confidence in the code base Release new features faster

19. Advantages of green build EASY TO SEE WHEN THINGS GO WRONG!!

21. Quick Overview Test Driven Development (TDD) Continuos Integration (CI) Trunk Based Development (TBD) Continuos Deployment

22. Continuos Deployment http://bit.ly/OxJv

23. “YES!” 1 Developer on the team is not enough Manager says “we have to” is not enough Majority of the team needs to agree to improve

24. Let’s make the build green Speed Always running Accurate Readable Reject broken build

25. Speed 10 mins or less Parallel Tests Split Build Ramdisk Headless Browser

26. Parallel Tests Most modern computers can run more then 1 test at the time Many tools parallel_tests (github)

27. Simple Parallel test_groups = [ "group 1" , "group 2" , "group 3" ] Parallel .map(test_groups) do | parallel_build | run_test parallel_build end

28. Our Tests Cucumber: 300 Selenium: 170 Rspec: 14,110

29. Cucumber: 8 Parallel RSpec: 21 Parallel Selenium: 10 Parallel

30. Use more computers Run each group, on a different computer in parallel (Double parallel) Split into smaller groups if computers are slow

32. Jenkins Conductor

34. Jenkins Conductor http://bit.ly/tF8E5c

35. Google’s Idea Run the tests that only apply to current module If test fail, they fail super fast Always run the whole suite right after http://bit.ly/jUBirG

38. RAMdisk Faster then Hard Disk Faster then SSD (Wikipedia)

40. Headless Browser Webrat Mechanize Firefox Capybara-Webkit

42. Capybara-Webkit http://bit.ly/o0tHp1 Biggest boost if only testing Javascript Database prevented us from using it

43. Reliability Flaky tests Test dependencies Test pollution

44. Flaky Test Worse then no test at all Make you lose confidence in build Most noticeable on Selenium Run multiple times in row to debug

45. Selenium Tips Use element IDs to click on Check element exists before clicking Wait for AJAX to load

46. Test the test for i in {1..20}; do run_test; done

47. Accurate Test fails when it should Test passes when it should Test actually tests something

48. Readable Cucumber Rspec Keep them clean

49. Reject broken build Groupon setup Master Release Any-Branch

50. Any-Branch Test Merge/Rebase

51. Test New Branch Has to start with test/..... Can choose to run only a single item

57. Merge/Rebase Starts with merge/... or rebase/... If tests fail, ignore If tests pass, merge into master

65. Questions?