Test Pilot is a opt-in platform aims to
improve the Web by collecting structured
user feedback from Firefox and other
Mozilla Labs experiments.

The Test Pilot platform
1.We run Test Pilot as a standalone opt-in program and not
as part of Firefox.
2.We will never collect user information without first having the
participant’s explicit permission.
3.We only transmit and store anonymous and
sanitized information in order to protect individual
privacy.*
4.We respect participants and their privacy by making test
data knowable and manageable for individuals.

Design question 1:
"What should the default action be when the user clicks in
the location bar for the first time?"
Design question 2:
"When user clicks a link, where this link should be opened as
a tab, at the end of the tab bar, or near the current tab?"

Firefox is a vehicle for 300 million people carrying their
work and lives on the Internet.
How might we help product teams make better decisions to
enhance user experience?

A good statistic aids a decision or shapes an
opinion.
How should we protect our user privacy while
we collect structured data for statistics?

A insightful statistic does not necessarily require
user private data.
Examples:
1. What are top-5 ubiquity commands people use the most?
2. How often people use the down-arrow near awesome bar and when?
3. How many tabs do people open?

Test Pilot Mission:
1. Improve the open Web experience by making anonymous,
aggregate usability & usage data available to developers as a
shared public resource.
2. Respect the user & their privacy by making test data
knowable and manageable for individuals.

Test Pilot Platform:
Collecting structured user data to improve product design.
https://testpilot.mozillalabs.com

Privacy and security with the utmost importance
Two focuses:
1. Establishing data privacy policy
2. Enabling participants’ controls on their data

Respect participants’ privacy during data collection,
storage and publishing
https://testpilot.mozillalabs.com

Give participants’ controls in the entire process
User controls on their data:
1 2
People opt in to the People participate in a specific Test Pilot study
Test Pilot program
1. Introduce Test Pilot
and its purpose. Before a study starts During the study Study complets
2. Introduce privacy policies: 2-1 2-2 2-3
2.1 User control
1. Introduce what this 1. Reinforce that no
2.2 Data collection and storage
study is and what data PII will be shared
2.3 Data sharing
need to be collected.
People are able to People are able to
3. Clarify how this program may
2. Clarify if this program review what study review their data.
change current product
may change current they are participating.
performance:
Firefox UI. People submit the
3.1 Won't lose any existing data,
study data or leave.
e.g bookmarks.
People opt-in a study,
3.2 UI may or may not change.
or leave.
People opt-in to Test Pilot.
People are able to opt out at any time.
No data will be reported until people submit.
People can opt out of the Test Pilot program at any time.

Let participants see the data as it is being
collected and before it gets sent to the server.
https://testpilot.mozillalabs.com

First large scale privacy-sensitive usability study with
the Test Pilot framework around tab usage completed
with over 5,000 participants, results and raw data
posted under Creative Commons license

How Math Teachers Can Help Improve the Web?
https://testpilot.mozillalabs.com

Try it out and improve the
Web together!
https://testpilot.mozillalabs.com
Twitter: @MozTestPilot