Mozilla: Under the Hood The browser has in many ways become a mini-operating system, attempting to harness all the internet’s power and complexity while making it simple enough so that anyone with minimal computer skills can visit websites and communicate with people all over the world. The technical challenges of developing and testing software in this kind of environment are immense, given the rapid changes going on in the internet space and the fact that the browser is called upon to parse as many as 1 trillion pages full of dynamic content. As time has gone on, many websites have gone from looking like static pages to resembling applications, which presents it own set of difficulties. Given these challenges, how does Mozilla ensure high quality and usability of every browser release? I will explore Mozilla under the hood and illuminate how Mozilla works to meet these challenges by introducing some of the tools that Mozilla uses to test its products. I will also demonstrate how Mozilla leverages help from the community as well as gives back to the community as part of the open development process.

1. Mozilla
Under the Hood

2. Presented by
Marcia Knous
Team Mozilla QA

3. Brief History of Mozilla

4. Mozilla is...
A global effort to
promote choice and
innovation on the
Internet
An open source project

5. Software for
the people,
by the people

6. NY Times two page ad
December 16, 2004

7. What fuels us?

8. Community
‣ 2008 Mozilla Summit

9. We build on top of each other

10. A Good Day at Mozilla :)

11. The tree is green...

12. ...So what do we test?

13. Everything!
• Firefox Release builds
• Including Firefox Localizations
• Firefox Nightly Builds
• Firefox Mobile (Fennec)
• Firefox web properties
• Thunderbird

14. What do we focus on?
•Basic functionality
•Standards Compliance
•Performance

15. How do we
do this?
•Automated Tests
•Manual Tests

16. The Mozilla Platform

17. Automated
Tests

18. Unit Tests
xpcshell
Compiled code

19. XPC Shell
‣Anything available to the XPCOM layer (through scriptable interfaces) can be tested with xpcshell.
‣Limitations: Can’t easily open windows, test chrome, work with HTML parser or the DOM
‣

20. Graphical & Interactive Tests
Mochitest
Chrome Tests
Reftest
Crash Tests

21. Content Mochitest
Highlights
Runs tests in a full browser environment with chrome
privileges
Can simulate user input (keyboard/mouse)
Good for testing things such as password manager and
popup blocking

22. Mochichrome
Highlights
Elevated privileges allow access to XPCOM
components and services
Uses simple JS function calls to aid in correctness
Useful for testing UI components such as:
Download Manager/Addons Manager

23. Browser Chrome Tests
Tests are written as JS files
Uses simple JS function calls to aid in correctness
Useful for testing Browser UI features

24. Crash Tests
Based on the reftest
framework
The test works by
loading the file and
completing without
crashing or asserting

25. Reftests
➡Layout Engine Visual Tests
➡Results are reported as Pass, Fail or Unexpected

26. Desired Result of the test

27. Talos
http://graphs.mozilla.org

28. Leak Tests

29. Topsite Testing

30. Other Tests
➡Fuzz Tests
➡Accessibility
➡JS Tests

31. Mozmill
Functional UI
testing for Mozilla
Applications.

33. Manual Tests

34. Litmus
‣Open 24/7 for your
testing enjoyment
‣Test cases written by
Mozilla QA & community

35. Litmus Full Functional Tests

36. Litmus Basic Functional Tests

37. Litmus Smoketests

38. Manual Debugging Tools
The Error Console is your friend!

39. Even with all these tests...

40. There is no magic bullet

41. The bug that forced FF 3 RC2

42. There is humor in testing
software...

43. Keynote software error
Common sight while preparing this presentation, but
at least I can send feedback...

44. Who tests?
Fosdem, 2009

45. The Community
Nightly Testers - ~ 10K
Betatesters - ~ 3 Million
Web Developers
Mozilla QA Team - 18

46. The QA
Companion
Useful add on that
allows the community
to test

48. An Example of Code Coverage

49. How do we get feedback?

50. Bugzilla
Community Members file
bugs and help triage
The Mozilla “kitchen”

51. Hendrix
Feedback link is postioned prominently on Release Notes
and first run page
Blocker bug for Firefox 3 was found through this feedback
mechanism

52. First run page before we changed the version to Firefox 3.5...
But nonetheless a good example of a the direct
feedback links (Bugzilla, Hendrix) on first run page

53. Twitter
Tweets regarding issues
with Firefox are reported
and monitored

54. Breakpad
Crash reporting tool

55. MozillaZine
Firefox Forum
Discussions
At one time was the
premier Mozilla news
site

56. Newsgroups - moz.dev.quality

57. QMO - The QA Portal
Central repository for project info and documents

58. reporter.mozilla.org
Linked directly from the
Help menu
Allows you to report a
broken website right
from the UI

59. So who reviews all this
feedback?
Sometimes tough to
separate the “noise”
from the real issues

60. We give back...

61. Songbird Miro
Blackbird Flock
Instantbird Celtx
Seamonkey Komodo
KompoZer
Nokia’s Internet Tablet
Wyzo

62. Projects/Technologies
Projects Technologies
Firefox Gecko
Camino XULRunner
Bugzilla Mozilla Labs
Fennec http://labs.mozilla.com/projects/
Lightning and Sunbird
Seamonkey
Thunderbird

63. The end...questions?

64. Resources/Background
material
The History of Mozilla: https://library.mozilla.org/The_History_of_Mozilla
QMO: http://quality.mozilla.org/
Litmus: https://litmus.mozilla.org/
Mozmill: http://code.google.com/p/mozmill/
Developer info: https://developer.mozilla.org/
Mozilla Labs: http://labs.mozilla.com/
Tinderbox for Firefox 3.5: http://tinderbox.mozilla.org/showbuilds.cgi?
tree=Firefox3.5

65. Resources/Background
material
Get the Firefox Beta: http://www.mozilla.com/en-US/firefox/all-beta.html
QA Companion: https://addons.mozilla.org/en-US/firefox/addon/5428
Hendrix: http://hendrix.mozilla.org/
Mozilla support: http://support.mozilla.com/en-US/kb/
Mozilla Digital Memory Bank: http://chnm.gmu.edu/mozilla-digital-memory-bank/
Marcia’s Firefox Robot Wars Video: http://www.youtube.com/watch?
v=OnZClPMoYEY

66. Photo credits
Jaguar Inline 6 Car Engine Photo courtesy of dave_7.
Licensed under a Creative Commons license - http://
www.flickr.com/photos/daveseven/250993044/
Bullet photo courtesy of JinglyJon. Licensed under a
Creative Commons license : http://www.flickr.com/
photos/jinglyjon/180600454/