• Like
Selenium at Mozilla: An Essential Element to our Success
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Selenium at Mozilla: An Essential Element to our Success



Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Selenium at Mozilla: AnEssential Element to our Success Stephen Donner Web QA March 25, 2013
  • 2. Overview:• History• 2013 Scale• Infrastructure• Custom Plugins/Configs• Lessons Learned from the Firefox 4 Era• Along Came Continuous Deployment• Community Collaboration• Our Key Takeaways
  • 3. History, part 1• 1st IDE tests in 2007; written by me for the Rock Your Firefox Facebook application• Started in earnest in May 2010 • Custom test-runner (w/concurrency) •1 Mac Mini• In November, same year, still only 2 suites (addons.mozilla.org and support.mozilla.org); same stack and setup• And, still: • No POM • No screenshots • No Selenium 2 / WebDriver • Which meant no mobile...
  • 4. History, part 2• October2010, we hired David Burns,and really started maturing our page-object model• AroundJune 2011 we startedmigrating RC → WebDriver• August 15, 2011, got full HTML reports
  • 5. 2013 scale• 13 projects w/WebDriver automation• August 16, 2012, we officially dropped RC support •2 with mobile support• Environments: • Dev/trunk/master (synonymous) • Staging • Production
  • 6. Infrastructure• Hardware: •7 Mac Minis •4 VMs• Browsers/OSs: • Latest Firefox + Beta + Aurora, Opera, Chrome, Safari, IE 6 – 10 • Mac OS X + Windows XP/7 • Mobile (through Sauce Labs)
  • 7. Custom Plugins/Configs• pytest-mozwebqa • Our mainstay – integrates with py.test (test runner), creates HTML reports, Sauce Labs integration, flexibility, test-data safety• moz-grid-config • Quickly deploy new Selenium and browser versions to our local Grid• pytest-rerunfailures • Comes close to solving “staging-server/WebDriver intermittent test failures”
  • 8. (Quick) Infra Demo
  • 9. Lessons learned from the Firefox 4era• Page object model really helped us scale and maintain• “Selenium+” - augmented what we already did well with WebDriver• Re-usable templates (for style-guide/community)• Destructive vs. non-destructive • Otherwise known as “dont spam addons.mozilla.org or any other site in production, ever again”
  • 10. And Along Came ContinuousDeployment...• Shock + fear, which segued into... • Risk-assessment • Shift in where/how; not “what” coverage • Healthy,reset view of Seleniums role in Web testing @ Mozilla
  • 11. Community Collaboration • Always striving to anticipate and provide for an answer to “what can I work on?” • Open GitHub Issues for both paid and community contributors • Heavy community participation, and wed love to see more! • RedHat and some startups have used pytest-mozwebqa • Fork our repositories
  • 12. Our key takeaways• Use POM if it makes sense for your apps/websites• Strive for consistency wherever and whenever possible/practical• Be tenacious in evangelizing + improving your tests value• Look at readying for and/or doing Continuous Delivery, if you can• Focus on your core strengths• Dont test everything “just because you can”
  • 13. Reach out to us!• Please, ask lots of questions; likewise, make suggestions, compare notes, etc.• mozwebqa@mozilla.org (mailing list)• #mozwebqa on irc.mozilla.org• Homepage: https://quality.mozilla.org/teams/web-qa/