Mozilla
Under the Hood

Presented by
Marcia Knous
Team Mozilla QA

Brief History of Mozilla

Mozilla is...
A global effort to
promote choice and
innovation on the
Internet
An open source project

Software for
the people,
by the people

NY Times two page ad
December 16, 2004

What fuels us?

Community
‣ 2008 Mozilla Summit

We build on top of each other

A Good Day at Mozilla :)

The tree is green...

...So what do we test?

Everything!
• Firefox Release builds
• Including Firefox Localizations
• Firefox Nightly Builds
• Firefox Mobile (Fennec)
• Firefox web properties
• Thunderbird

What do we focus on?
•Basic functionality
•Standards Compliance
•Performance

How do we
do this?
•Automated Tests
•Manual Tests

The Mozilla Platform

Automated
Tests

Unit Tests
xpcshell
Compiled code

XPC Shell
‣Anything available to the XPCOM layer (through scriptable interfaces) can be tested with xpcshell.
‣Limitations: Can’t easily open windows, test chrome, work with HTML parser or the DOM
‣

Graphical & Interactive Tests
Mochitest
Chrome Tests
Reftest
Crash Tests

Content Mochitest
Highlights
Runs tests in a full browser environment with chrome
privileges
Can simulate user input (keyboard/mouse)
Good for testing things such as password manager and
popup blocking

Mochichrome
Highlights
Elevated privileges allow access to XPCOM
components and services
Uses simple JS function calls to aid in correctness
Useful for testing UI components such as:
Download Manager/Addons Manager

Browser Chrome Tests
Tests are written as JS files
Uses simple JS function calls to aid in correctness
Useful for testing Browser UI features

Crash Tests
Based on the reftest
framework
The test works by
loading the file and
completing without
crashing or asserting

Reftests
➡Layout Engine Visual Tests
➡Results are reported as Pass, Fail or Unexpected

Desired Result of the test

Talos
http://graphs.mozilla.org

Leak Tests

Topsite Testing

Other Tests
➡Fuzz Tests
➡Accessibility
➡JS Tests

Mozmill
Functional UI
testing for Mozilla
Applications.

Manual Tests

Litmus
‣Open 24/7 for your
testing enjoyment
‣Test cases written by
Mozilla QA & community

Litmus Full Functional Tests

Litmus Basic Functional Tests

Litmus Smoketests

Manual Debugging Tools
The Error Console is your friend!

Even with all these tests...

There is no magic bullet

The bug that forced FF 3 RC2

There is humor in testing
software...

Keynote software error
Common sight while preparing this presentation, but
at least I can send feedback...

Who tests?
Fosdem, 2009

The Community
Nightly Testers - ~ 10K
Betatesters - ~ 3 Million
Web Developers
Mozilla QA Team - 18

The QA
Companion
Useful add on that
allows the community
to test

An Example of Code Coverage

How do we get feedback?

Bugzilla
Community Members file
bugs and help triage
The Mozilla “kitchen”

Hendrix
Feedback link is postioned prominently on Release Notes
and first run page
Blocker bug for Firefox 3 was found through this feedback
mechanism

First run page before we changed the version to Firefox 3.5...
But nonetheless a good example of a the direct
feedback links (Bugzilla, Hendrix) on first run page

Twitter
Tweets regarding issues
with Firefox are reported
and monitored

Breakpad
Crash reporting tool

MozillaZine
Firefox Forum
Discussions
At one time was the
premier Mozilla news
site

Newsgroups - moz.dev.quality

QMO - The QA Portal
Central repository for project info and documents

reporter.mozilla.org
Linked directly from the
Help menu
Allows you to report a
broken website right
from the UI

So who reviews all this
feedback?
Sometimes tough to
separate the “noise”
from the real issues

We give back...

Songbird Miro
Blackbird Flock
Instantbird Celtx
Seamonkey Komodo
KompoZer
Nokia’s Internet Tablet
Wyzo

Projects/Technologies
Projects Technologies
Firefox Gecko
Camino XULRunner
Bugzilla Mozilla Labs
Fennec http://labs.mozilla.com/projects/
Lightning and Sunbird
Seamonkey
Thunderbird

The end...questions?

Resources/Background
material
The History of Mozilla: https://library.mozilla.org/The_History_of_Mozilla
QMO: http://quality.mozilla.org/
Litmus: https://litmus.mozilla.org/
Mozmill: http://code.google.com/p/mozmill/
Developer info: https://developer.mozilla.org/
Mozilla Labs: http://labs.mozilla.com/
Tinderbox for Firefox 3.5: http://tinderbox.mozilla.org/showbuilds.cgi?
tree=Firefox3.5

Resources/Background
material
Get the Firefox Beta: http://www.mozilla.com/en-US/firefox/all-beta.html
QA Companion: https://addons.mozilla.org/en-US/firefox/addon/5428
Hendrix: http://hendrix.mozilla.org/
Mozilla support: http://support.mozilla.com/en-US/kb/
Mozilla Digital Memory Bank: http://chnm.gmu.edu/mozilla-digital-memory-bank/
Marcia’s Firefox Robot Wars Video: http://www.youtube.com/watch?
v=OnZClPMoYEY

Photo credits
Jaguar Inline 6 Car Engine Photo courtesy of dave_7.
Licensed under a Creative Commons license - http://
www.flickr.com/photos/daveseven/250993044/
Bullet photo courtesy of JinglyJon. Licensed under a
Creative Commons license : http://www.flickr.com/
photos/jinglyjon/180600454/