Continuous Integration with Mozmill

2,016 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,016
On SlideShare
0
From Embeds
0
Number of Embeds
316
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Continuous Integration with Mozmill

  1. 1. quality.mozilla.org qa automation services Mozmill CIContinuous Integration with Mozmill Henrik Skupin <hskupin@mozilla.com> Co-Lead of QA Automation Services 05.02.2012 - FOSDEM
  2. 2. OverviewBenefits of MozmillCurrent situation in testing Firefox releases / buildsGoals with Continuous IntegrationCurrent StatusRoadmapQuestions
  3. 3. BENEFITS OF MOZMILL Test harness for functional UI driven tests Support for any Gecko / XULRunner based application Test Execution independent of the used locale No need to create custom builds with tests enabled Easy to setup due to our preconfigured test environment Centralized reporting mechanism for test results
  4. 4. CURRENT SITUATION IN TESTING Firefox Releases Test cycles for candidate builds and application updates Test execution across platforms and locales based on configuration file Manual triggered download and execution via Pulse Pulse triggers test machines to start wanted tests
  5. 5. CURRENT SITUATION IN TESTING Daily Firefox builds Currently run for Nightly, Aurora, and Namoroka Tests triggered by crontab or Windows tasks Tests run against pre-installed builds Executed tests for en-US locale only: Update, Functional, L10n, Endurance, Remote, Add-ons
  6. 6. GOALS WITH CONTINUOUS INTEGRATIONGoals Free-up human resources by killing the last manual tasks Better hardware utilization with distributed test-runs Flexible configuration of builds to test and machines Immediate test execution when builds have been made available Increase the number of builds and locales to test
  7. 7. CURRENT STATUS Status for Firefox Releases
  8. 8. CURRENT STATUS Status for Daily Firefox Builds Note: Not everything has been implemented yet!
  9. 9. ROADMAP Increase # of executors to run multiple tests concurrently on the same node Extend hardware to execute tests for all localized daily builds like Nightly and Aurora Switch the on-demand test framework for releases to use a Pulse consumer instead of configuration files Tight integration with Case Conductor to combine automated and manual test results
  10. 10. ROADMAP Setup a CI system for localized builds Test tinderbox builds triggered by checkins from localizers Execute L10n tests to prove quality of the localization (e.g. cut-off elements, access keys, ...) Upload created reports and screenshots to a public dashboard for further processing We need your ideas and your enthusiasm to create a good suite of L10n tests!
  11. 11. REFERENCES QA Automation Services: https://wiki.mozilla.org/QA/Automation_Services Mozmill CI for Release Builds: https://wiki.mozilla.org/QA/Automation_Services/Projects/Mozmill_Automation/ On_Demand_Test_Framework Mozmill CI for Daily Builds: https://wiki.mozilla.org/QA/Automation_Services/Projects/Mozmill_Automation/ Triggered_Daily_Test-runs Mozmill CI for L10n Builds: https://wiki.mozilla.org/QA/Automation_Services/Projects/Mozmill_Automation/ Triggered_l10n_Test-runs
  12. 12. QUESTIONS & ANSWERS Any Questions?
  13. 13. Thanks

×