MozMill Crowd - The birth of global automated testing

quality.mozilla.org qa automation MOZMILL CROWD The birth of global automated testing Anthony Hughes <ashughes@mozilla.com> Henrik Skupin <hskupin@mozilla.com>

MOZMILL CROWD

MOZMILL CROWD • What is MozMill?

MOZMILL CROWD • What is MozMill? • Automated test-runs

MOZMILL CROWD • What is MozMill? • Automated test-runs • Extended testing with MozMill

MOZMILL CROWD • What is MozMill? • Automated test-runs • Extended testing with MozMill • MozMill Crowd extension

MOZMILL CROWD • What is MozMill? • Automated test-runs • Extended testing with MozMill • MozMill Crowd extension • Demo

MOZMILL CROWD • What is MozMill? • Automated test-runs • Extended testing with MozMill • MozMill Crowd extension • Demo • Questions & Answers

WHAT IS MOZMILL • Automation • Framework & API • Reporting • IDE • Documentation • Community

AUTOMATED TEST-RUNS • BFT • Selection of automatable scripted manual tests from Litmus (including Smoketests and BFT) • Creation of new tests from too complex test scenarios • Breakup in restart-less and restart tests • Support for all localized builds • Executable on all platforms

AUTOMATED TEST-RUNS • Localization (L10n) • Multiple usage of the same access keys • Checking existing windows and dialogs for cut-off elements • Accessibility • Testsof proper XUL markup for exposing content to screen readers

AUTOMATED TEST-RUNS • Add-ons • Tests for add-ons provided by add-on authors • Add-ons hosted on AMO or on other servers • Breakup in restart-less and restart tests • (Software Update) • Release testing of feature and security releases • Will not be promoted in MozMill Crowd

EXTENDED TESTING WITH MOZMILL • In-house testing isn’t enough, we have to do more! • Outsourcing tasks to a large group of people • Increased coverage for daily test-runs • Support localization teams by executing test-runs for localized builds • Help add-on authors to get their add-ons tested • Performance tests on the testers system (planned)

MOZMILL CROWD EXTENSION • Tool to allow crowd sourcing of Mozmill tests • Which requirements have we set? • Support for all platforms and branches • Automatic setup of the whole MozMill environment • Execution of available test-runs with a single click • Sending of test results to a report server • Customizable for more focused testing

MOZMILL CROWD EXTENSION • Separate dialog with controls for test parameters • Area for immediate output of test results

MOZMILL CROWD EXTENSION • Preferences • Selection of the test repository • Sending reports • Security options for add-ons test-run

DEMO • What will be shown? • Add-on test-run for the Google Toolbar extension • BFT test-run in Firefox (fa) • Test results

QUESTIONS & ANSWERS Questions? We will collected ideas and proposals in the following Etherpad document: http://etherpad.mozilla.com:9000/h03KB01vcS

Thanks

http://www.hskupin.info	106
http://quality.mozilla.org	3
http://quality-new.stage.mozilla.com	1
http://translate.googleusercontent.com	1
https://www.hskupin.info	1

MozMill Crowd - The birth of global automated testing

by Henrik Skupin on Jul 13, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 112

Statistics

MozMill Crowd - The birth of global automated testing — Presentation Transcript