This document discusses user research and conducting experiments to test assumptions. It provides: 1. A framework for identifying risky assumptions about users, crafting experiments to test those assumptions, carrying out the experiments, and using the results. 2. Tips for crafting effective experiments, including starting with the riskiest assumption, tracking results, redesigning experiments if needed, defining clear measures of success, and making decisions as a team. 3. Best practices for carrying out experiments, such as being brief, asking open questions, active listening, and leaving personal questions to the end. The document provides examples of case studies to test assumptions about passwords and online dating.

1. 1
@jewl
with

2. The Approach
2Julia Shalet, Product Doctor
1. Identify Risky Assumptions about People
2. Craft Experiments to Reduce those Risks
3. Carry out the Experiment
4. Use the Results effectively

3. What are you working on?
Be honest about assumptions you
have made about how people
feel / behave / want….
33
1. Identifying Risky Assumptions
Julia Shalet, Product Doctor

4. 1. Goal
What is it that we’re
trying to learn /
prove? Have we
assumed a customer
problem or behaviour?
2. People
Who is our customer
/ user? What are our
recruitment criteria?
Where will we find
them?
3. Logistics
How are we going to
conduct our
experiment? When?
Who will carry out the
experiment? Where?
4. Measurement
How will we know
if our experiment
is a success?
5. Outcome
What did we learn? What will
you do if this experiment is a
success? What will you do if
this experiment is a failure?
2. Crafting Experiments
Extract from Pearson Product Lifecycle training @Leanplc

5. 5Julia Shalet, Product Doctor
1. Always start with your riskiest assumption
2. Track every experiment and its results
3. If your first experiment doesn’t work, redesign it and try again
4. Rather than worry about sample sizes, start with a number &
when you begin to hear the same things, you can stop!
5. Design experiments together as a team
6. Define clear measures of success for each experiment
7. Share learnings and results with the team
8. Make decisions together after each experiment.
Tips for Crafting Experiments

6. 66
3. Carry out the Experiment
Volunteer Please!
Watch for good / bad practice…
Julia Shalet, Product Doctor

7. Good practice?
7

8. Good practice: Any to Add?
• Openly time it
• Be brief – get them talking
• Open, simple, not leading questions
• Less is more e.g. don’t mention brand names
unless that is what you are researching
• Active Listening: repeat & summarise
• Probe – don’t miss any clues
• Record, listen back & review for bias
• Leave more personal information to the end
Introduction:
• State what you need & the incentive
• Check Profile Fit
• Set out your neutrality
• Permission to Record

9. 9
IN TEAMS OF 3:
Respondent, Interviewer, Observer
Practice
Julia Shalet, Product Doctor
EXERCISE

10. Thanks!
@LeanPlc@Moblacad@jewl

11. Back Up Case Studies
(borrowed from The Mobile Academy
alumni!)
1. Passwords
We know that many people struggle to remember their passwords.
We think that people are more likely to remember visuals rather
than alphanumerics. Our idea is to let people upload a picture and
pick 4 points on that picture instead of an alpha-numeric
password.
2. Dating
Many people are nervous of going on dates with people that they
have found through dating sites and Apps. Our idea is to offer the
opportunity to date in pairs with our “double dating” App.