Selenium at Mozilla: An Essential Element to our Success

984 views

Published on

Published in: Technology
  • Be the first to comment

Selenium at Mozilla: An Essential Element to our Success

  1. 1. Selenium at Mozilla: AnEssential Element to our Success Stephen Donner Web QA March 25, 2013
  2. 2. Overview:• History• 2013 Scale• Infrastructure• Custom Plugins/Configs• Lessons Learned from the Firefox 4 Era• Along Came Continuous Deployment• Community Collaboration• Our Key Takeaways
  3. 3. History, part 1• 1st IDE tests in 2007; written by me for the Rock Your Firefox Facebook application• Started in earnest in May 2010 • Custom test-runner (w/concurrency) •1 Mac Mini• In November, same year, still only 2 suites (addons.mozilla.org and support.mozilla.org); same stack and setup• And, still: • No POM • No screenshots • No Selenium 2 / WebDriver • Which meant no mobile...
  4. 4. History, part 2• October2010, we hired David Burns,and really started maturing our page-object model• AroundJune 2011 we startedmigrating RC → WebDriver• August 15, 2011, got full HTML reports
  5. 5. 2013 scale• 13 projects w/WebDriver automation• August 16, 2012, we officially dropped RC support •2 with mobile support• Environments: • Dev/trunk/master (synonymous) • Staging • Production
  6. 6. Infrastructure• Hardware: •7 Mac Minis •4 VMs• Browsers/OSs: • Latest Firefox + Beta + Aurora, Opera, Chrome, Safari, IE 6 – 10 • Mac OS X + Windows XP/7 • Mobile (through Sauce Labs)
  7. 7. Custom Plugins/Configs• pytest-mozwebqa • Our mainstay – integrates with py.test (test runner), creates HTML reports, Sauce Labs integration, flexibility, test-data safety• moz-grid-config • Quickly deploy new Selenium and browser versions to our local Grid• pytest-rerunfailures • Comes close to solving “staging-server/WebDriver intermittent test failures”
  8. 8. (Quick) Infra Demo
  9. 9. Lessons learned from the Firefox 4era• Page object model really helped us scale and maintain• “Selenium+” - augmented what we already did well with WebDriver• Re-usable templates (for style-guide/community)• Destructive vs. non-destructive • Otherwise known as “dont spam addons.mozilla.org or any other site in production, ever again”
  10. 10. And Along Came ContinuousDeployment...• Shock + fear, which segued into... • Risk-assessment • Shift in where/how; not “what” coverage • Healthy,reset view of Seleniums role in Web testing @ Mozilla
  11. 11. Community Collaboration • Always striving to anticipate and provide for an answer to “what can I work on?” • Open GitHub Issues for both paid and community contributors • Heavy community participation, and wed love to see more! • RedHat and some startups have used pytest-mozwebqa • Fork our repositories
  12. 12. Our key takeaways• Use POM if it makes sense for your apps/websites• Strive for consistency wherever and whenever possible/practical• Be tenacious in evangelizing + improving your tests value• Look at readying for and/or doing Continuous Delivery, if you can• Focus on your core strengths• Dont test everything “just because you can”
  13. 13. Reach out to us!• Please, ask lots of questions; likewise, make suggestions, compare notes, etc.• mozwebqa@mozilla.org (mailing list)• #mozwebqa on irc.mozilla.org• Homepage: https://quality.mozilla.org/teams/web-qa/

×