Stabilizing Selenium BuildDima Kovalenkodima@groupon.com@dimacus
We Are Hiring• dima@groupon.com• @dimacus
Love My Job• Love testing• Love hacking• Love talking about above
Partner In CrimeDarko MarinovAssociate ProfessorUniversity of Illinois
Stability is a Journey not destination
Culture
Culture Changes• Convince Management that flakey tests wastea lot of money• Discourage developers from relying on tests• P...
Fix Ajax Waiting• jQuery– evaluate_script("jQuery.active") == 0– evaluate_script($(":animated").length) == 0
Flakey Tests
Self contained tests• Separate Databases• Self reliant tests– Setup/Teardown
Faster Feedback Loop• Fight to get build to 10 mins• Notify of failures before build is finished– Rspec/Cucumber Early Not...
Flakey Detector• Check Jenkins or Git diff• If a test was modified, kick of flake detectorbuild– Run changed test 5k times...
Fix Environment Problems!!• Tests get blamed if environment fails• Standardize test boxes– Chef, Puppet, w/e
Flakey Environment
Windows Tips• Don’t use IE• Move mouse out of browser in IE• Maximize test window• Don’t use RPD/RDC– Use VNC
Linux Tips• Run Firefox in separate Desktop
Speed up tests• Parallel more• Does it have to be Selenium?– Can this be a unit test?– Can it be headless?– Can you run Ja...
Black Hole Proxy• Capture any external requests, return 200• Speeds tests up• Reliability goes up• em-proxy– https://gist....
Gem Cacher• Cache any external dependencies locally• Don’t rely on rubygems.org to always be up!– github.com/dimacus/gem-c...
Credit on Grid idea• David Burns• @AutomatedTester
Selenium Grid Extras• github.com/groupon/Selenium-Grid-Extras
Basic Idea• Help maintain the Environment to producestable tests• Turn Selenium Grid Nodes into dumbbrowsers
Control OS from tests• Clean up old sessions• Kill browsers• Take full OS Screenshots• Move mouse• Reboot windows
Future plan• Record videos• Share files with grid nodes• Spin up/down VMs on the fly• Image comparison• Auto update Seleni...
Need your help• https://github.com/groupon/Selenium-Grid-Extras
Questions?42!
Stabilizing SE Build - Selenium conf 2013
Stabilizing SE Build - Selenium conf 2013
Upcoming SlideShare
Loading in …5
×

Stabilizing SE Build - Selenium conf 2013

3,137 views

Published on

Talk at 2013 Selenium Conference on the topic of stabilizing a build and making it green over all

Published in: Technology
1 Comment
4 Likes
Statistics
Notes
  • For brevity in the slides here, the following blog post elaborates on some merits to use of VNC over RDP for GUI test automation: http://autumnator.wordpress.com/2011/11/18/how-to-run-remote-headless-gui-automation/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
3,137
On SlideShare
0
From Embeds
0
Number of Embeds
33
Actions
Shares
0
Downloads
10
Comments
1
Likes
4
Embeds 0
No embeds

No notes for slide

Stabilizing SE Build - Selenium conf 2013

  1. 1. Stabilizing Selenium BuildDima Kovalenkodima@groupon.com@dimacus
  2. 2. We Are Hiring• dima@groupon.com• @dimacus
  3. 3. Love My Job• Love testing• Love hacking• Love talking about above
  4. 4. Partner In CrimeDarko MarinovAssociate ProfessorUniversity of Illinois
  5. 5. Stability is a Journey not destination
  6. 6. Culture
  7. 7. Culture Changes• Convince Management that flakey tests wastea lot of money• Discourage developers from relying on tests• Prevent red build from merging intomaster/trunk• Red master is NOT OK!
  8. 8. Fix Ajax Waiting• jQuery– evaluate_script("jQuery.active") == 0– evaluate_script($(":animated").length) == 0
  9. 9. Flakey Tests
  10. 10. Self contained tests• Separate Databases• Self reliant tests– Setup/Teardown
  11. 11. Faster Feedback Loop• Fight to get build to 10 mins• Notify of failures before build is finished– Rspec/Cucumber Early Notification Formatters– Blog post to follow– engineering.groupon.com
  12. 12. Flakey Detector• Check Jenkins or Git diff• If a test was modified, kick of flake detectorbuild– Run changed test 5k times in 10 mins– In random order– They should always be green!• engineering.groupon.com
  13. 13. Fix Environment Problems!!• Tests get blamed if environment fails• Standardize test boxes– Chef, Puppet, w/e
  14. 14. Flakey Environment
  15. 15. Windows Tips• Don’t use IE• Move mouse out of browser in IE• Maximize test window• Don’t use RPD/RDC– Use VNC
  16. 16. Linux Tips• Run Firefox in separate Desktop
  17. 17. Speed up tests• Parallel more• Does it have to be Selenium?– Can this be a unit test?– Can it be headless?– Can you run Jasmine instead?
  18. 18. Black Hole Proxy• Capture any external requests, return 200• Speeds tests up• Reliability goes up• em-proxy– https://gist.github.com/dimacus/5757573
  19. 19. Gem Cacher• Cache any external dependencies locally• Don’t rely on rubygems.org to always be up!– github.com/dimacus/gem-cacher• Local Maven repo
  20. 20. Credit on Grid idea• David Burns• @AutomatedTester
  21. 21. Selenium Grid Extras• github.com/groupon/Selenium-Grid-Extras
  22. 22. Basic Idea• Help maintain the Environment to producestable tests• Turn Selenium Grid Nodes into dumbbrowsers
  23. 23. Control OS from tests• Clean up old sessions• Kill browsers• Take full OS Screenshots• Move mouse• Reboot windows
  24. 24. Future plan• Record videos• Share files with grid nodes• Spin up/down VMs on the fly• Image comparison• Auto update Selenium WebDriver• Black-hole proxy integration• Improve Grid API• Restart SE Node• Idle machine utilization
  25. 25. Need your help• https://github.com/groupon/Selenium-Grid-Extras
  26. 26. Questions?42!

×