Testing Mozilla Web Properties Stephen Donner 6/24/2009

Meet the Team Stephen Donner - [email_address] Krupa Raj - [email_address] Raymond Etornam Agbeame - [email_address]

Stephen Donner - [email_address]

Krupa Raj - [email_address]

Raymond Etornam Agbeame - [email_address]

Technologies we test AJAX Drupal Spread Firefox QMO Apache .htaccess - bug 478952 Open-source search engines Sphinx (SUMO) TikiWiki (SUMO is its biggest consumer) Media types Ogg Flash .swf vs. .flv

AJAX

Drupal

Spread Firefox

QMO

Apache

.htaccess - bug 478952

Open-source search engines

Sphinx (SUMO)

TikiWiki (SUMO is its biggest consumer)

Media types

Ogg

Flash

.swf vs. .flv

Tools We Use Firebug Web Developer toolbar Live HTTP Headers SQL Injection Selenium IDE RC VMWare Fusion Acunetix W3C

Firebug

Web Developer toolbar

Live HTTP Headers

SQL Injection

Selenium

IDE

RC

VMWare Fusion

Acunetix

W3C

Gotchas Staging server vs. production Different environments Configs (permissions, etc.) Application versions Caching Memcache NetScaler / Zeus DB setups Slave vs. master Replication lag Legacy browsers / non-standard configs IE 6, anyone? JavaScript disabled

Staging server vs. production

Different environments

Configs (permissions, etc.)

Application versions

Caching

Memcache

NetScaler / Zeus

DB setups

Slave vs. master

Replication lag

Legacy browsers / non-standard configs

IE 6, anyone?

JavaScript disabled

Gotchas, part 2 Partner integration GetSatisfaction reCaptcha Mozilla Service Week Add-on promotions Fashion Your Firefox

Partner integration

GetSatisfaction

reCaptcha

Mozilla Service Week

Add-on promotions

Fashion Your Firefox

Pain Points Tight schedules Quarterly goals from other parts of the org Lack of clear specs Wireframes vs. mockups (vs. full user-interaction testing) Project direction / ownership Out-of-band testing requests Ops/IT - hardware failure, new-technology evals, regression-testing of new hardware

Tight schedules

Quarterly goals from other parts of the org

Lack of clear specs

Wireframes vs. mockups

(vs. full user-interaction testing)

Project direction / ownership

Out-of-band testing requests

Ops/IT - hardware failure, new-technology evals, regression-testing of new hardware

Automated vs. Manual Testing Balance need to rapidly execute/ship vs. long-term stability Ramp-up cost Can't automate everything Quite a few projects are one-off (time investment) Impact Mozilla Fashion Your Firefox Operation Firefox Good automation candidates Search APIs (services.addons.mozilla.org) Static content Headers/footers/legal/privacy policies/EULAs Login / logout / registration/ forgotten password Exception of Captcha systems

Balance need to rapidly execute/ship vs. long-term stability

Ramp-up cost

Can't automate everything

Quite a few projects are one-off (time investment)

Impact Mozilla

Fashion Your Firefox

Operation Firefox

Good automation candidates

Search

APIs (services.addons.mozilla.org)

Static content

Headers/footers/legal/privacy policies/EULAs

Login / logout / registration/ forgotten password

Exception of Captcha systems

Grades / Levels of Testing Acceptance check Full Functionality Test Regression testing Basic Functionality Test (Exhaustive list: http://www.softwaretestinghelp.com/types-of-software-testing/ )

Acceptance check

Full Functionality Test

Regression testing

Basic Functionality Test

(Exhaustive list: http://www.softwaretestinghelp.com/types-of-software-testing/ )

What we use Selenium for Verify page titles Verify content pieces (text, links, images) Pagination Search terms persist in textfields ~ 15 commands

Verify page titles

Verify content pieces (text, links, images)

Pagination

Search terms persist in textfields

~ 15 commands

Example of manual testing Mozilla Service Week (http://mozillaservice.org)

Mozilla Service Week (http://mozillaservice.org)

Selenium demo AMO Basic Functionality Test AMO Search

AMO Basic Functionality Test

AMO Search

How can you help? Vet content Test browser compatibility Sanity check our testing Write unit tests

Vet content

Test browser compatibility

Sanity check our testing

Write unit tests

Resources Public mailing list: [email_address] Homepage: https://wiki.mozilla.org/QA/Execution/Web_Testing IRC channel: irc.mozilla.org/webqa

Public mailing list: [email_address]

Homepage: https://wiki.mozilla.org/QA/Execution/Web_Testing

IRC channel: irc.mozilla.org/webqa

Questions? Comments? Suggestions?

Thanks for your interest!