Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

2010 GTAC Crowd Source Testing Mozilla Style

on

  • 2,071 views

Presented at GTAC 2010 conference, Hyderabad, India

Presented at GTAC 2010 conference, Hyderabad, India

Statistics

Views

Total Views
2,071
Views on SlideShare
2,058
Embed Views
13

Actions

Likes
0
Downloads
22
Comments
0

1 Embed 13

http://f74.faxo.com:3000 13

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

2010 GTAC Crowd Source Testing Mozilla Style 2010 GTAC Crowd Source Testing Mozilla Style Presentation Transcript

  • Crowd Source Testing Mozilla Community Style Matt Evans Mozilla QA Director Wednesday, November 10, 2010
  • about:talk • Background and Scale of Mozilla • Crowdsourcing: Crowd vs. Community • History of Mozilla Community testing • How Mozilla succeeds with Community testing Wednesday, November 10, 2010
  • about:me • I have been in the testing biz for most my career • 9 mos as QA Director @ Mozilla • Manage a team of ~30 Global Test Engineers Wednesday, November 10, 2010
  • about:mozilla Wednesday, November 10, 2010
  • about:mozilla Mozilla’s Mission: is to promote openness, innovation, and opportunity on the web Wednesday, November 10, 2010
  • I’m still getting my head around scale at Mozilla • The scale of information flow • The scale of the user base Wednesday, November 10, 2010
  • Mozilla Firehose Wednesday, November 10, 2010
  • Mozilla Firehose • Many sources of information • bugzilla (200+bugs/day) • forums (50+) • blogs (50+) • tweets (100s/day) • wikis (1000s pages) • irc (50+ channels) Wednesday, November 10, 2010
  • 400,000,000 Mozilla Scale 400,000,000 Users 350,000,000 300,000,000 250,000,000 200,000,000 400+ Developers 10,000+ 150,000,000 Nightly Testers 2,500,000 100,000,000 Beta Testers 50,000,0000 Wednesday, November 10, 2010
  • Story Time Wednesday, November 10, 2010
  • A Recent Firefox product team meeting Big decision: Ship or No-Ship to Beta Wednesday, November 10, 2010
  • Protector of the tester community Wednesday, November 10, 2010
  • Wednesday, November 10, 2010
  • Wednesday, November 10, 2010
  • Get feedback from tester community Wednesday, November 10, 2010
  • Wednesday, November 10, 2010
  • We were both wrong right! Wednesday, November 10, 2010
  • It was a big change for my perspective Wednesday, November 10, 2010
  • Respecter of the Tester Community as QA Team Partner Wednesday, November 10, 2010
  • We now work much better together Wednesday, November 10, 2010
  • Crowd Sourcing: It involves a crowd working towards a common goal Wednesday, November 10, 2010
  • What is new, the virtual crowd is enabled by internet technologies Wednesday, November 10, 2010
  • Wednesday, November 10, 2010
  • Community approach to software testing • Building a global community of testers ~30k+ • Stewardship of the test community is their secret sauce Wednesday, November 10, 2010
  • Mozilla Labs Crowdsource Crowdsourcing Project Wednesday, November 10, 2010
  • Mozilla Labs Crowdsource Crowdsourcing Project Key Findings • there are two types users: heavy users and casual users. • 90-9-1 Principle • crowd motivation is separated into two forms: intrinsic and extrinsic • crowdsourcing relies on people, and thus works when participants are treated as people. Wednesday, November 10, 2010
  • Community Crowd VS Wednesday, November 10, 2010
  • Mozilla uses a hybrid of crowd based feedback mechanisms and community management “Crowd sourced testing Mozilla Style” Wednesday, November 10, 2010
  • The History of Mozilla Testing Community Wednesday, November 10, 2010
  • Firefox 4.0 expected Firefox release 1.5-3.5 Firefox 1.0 released released Mozilla 1.0 Phoenix 0.1 Mozilla.org is is released released launched by Netscape 1998 2002 2004 2005-2009 2010 Pioneer Community Testers Community Testers, The Next Generation 1 2 3 Wednesday, November 10, 2010
  • 1998 2002 2004 2005-2009 2010 Pioneer Community Testers Community Testers, The Next Generation 1 2 3 1998 Netscape open sourced the Netscape browser source code. Wednesday, November 10, 2010
  • History of Mozilla’s Tester Community Asa Dotzler First Mozilla Community Tester • A lot of passionate people without a development background wanted to get involved. Wednesday, November 10, 2010
  • History of Mozilla’s Tester Community The Rise of the Nightly Build Wednesday, November 10, 2010
  • History of Mozilla’s Tester Community • A great success • The community of testers grew to the 1000s Wednesday, November 10, 2010
  • History of Mozilla’s Tester Community Too much of good thing Wednesday, November 10, 2010
  • History of Mozilla’s Tester Community • As tester community members increased so did the noise Wednesday, November 10, 2010
  • 1998 2002 2004 2005-2009 2010 Pioneer Community Testers Community Testers, The Next Generation 1 2 3 • 2002+ • The rise of Firefox code base • Split from AOL/Netscape • 2003 Mozilla Foundation is formed Wednesday, November 10, 2010
  • 1998 2002 2004 2005-2009 2010 Pioneer Community Testers Community Testers, The Next Generation 2 Wednesday, November 10, 2010
  • 1998 2002 2004 2005-2009 2010 Pioneer Community Testers Community Testers, The Next Generation 1 2 3 • 2007+ • The rise of Continuous Integration and unit tests Wednesday, November 10, 2010
  • Continuous Integration • The largest CI system built on Buildbot and Tinderbox technologies • For every code change • 11 full builds of the browser across supported platforms • ~300k unit/perf tests per build ~3.3 million automated tests executed for every committed code change Wednesday, November 10, 2010
  • Minefield is the result of Continuous Integration Continuous Integration and Unit Testing is the Foundation of Today’s Mozilla Community Testing Success Wednesday, November 10, 2010
  • Community Feedback Wednesday, November 10, 2010
  • Mozilla Runs On Bugzilla Wednesday, November 10, 2010
  • Crash Data Analysis Wednesday, November 10, 2010
  • Crash Data Analysis Wednesday, November 10, 2010
  • Crash Data Analysis 5 years of crash data Filed Fixed Dups FF1.5 FF2.0 FF3.0 FF3.5 Wednesday, November 10, 2010
  • Firefox 4 Beta Feedback Wednesday, November 10, 2010
  • Firefox 4 Beta Feedback LCD Screens of Feedback and Twitter mentions are displayed throughout the Mozilla Office Wednesday, November 10, 2010
  • Engaging the Mozilla Tester Community Wednesday, November 10, 2010
  • Manual Testing Litmus - Test Case Management Wednesday, November 10, 2010
  • Test and Bug Days Wednesday, November 10, 2010
  • Community Focused Automated Testing Mozmill Test Automation Framework • Javascript framework for testing gecko based applications • Includes an IDE • Installs as an Firefox extension • Better suited for community contribution Wednesday, November 10, 2010
  • Mozilla Community Based Automated Testing Wednesday, November 10, 2010
  • Mozilla Community Based Automated Testing Wednesday, November 10, 2010
  • Overview of How we succeed with Community Testing • Continuous Integration is the foundation of today’s community testing • Utilizing Community Feedback • Engage in Manual Testing efforts • Engage Community focused automated testing in the near future Wednesday, November 10, 2010
  • Community Stewardship Wednesday, November 10, 2010
  • Community Stewardship Wednesday, November 10, 2010
  • Community Stewardship quality.mozilla.org Wednesday, November 10, 2010
  • Reaching Out to the Community quality.mozilla.org quality.mozilla.org Wednesday, November 10, 2010
  • A call out to you! Wednesday, November 10, 2010
  • Thank you Questions? Wednesday, November 10, 2010