Validating hypotheses with user research

Validating Hypothesis
with User Research

UX Agency for
next generation companies

What jobs do they want to accomplish?
In what context?
What pains do they have?
What gains do they want?
How frequent do they want to preform the job?
What did they do before? What happened after?
What did they switch from? Why?
Understanding your users…

…and validating our
hypothesis

How do you validate your
hypothesis?

01 Deﬁne your Hypothesis
02 Deﬁne your Experiment
03 Conduct your Experiment
04 Analyse, Share and Act on your Result
The Validation Process

What do you want to validate?
Desirability - “Do they want this?”
Feasibility - “Can we do this?”
Viability - “Should we do this?”
Desirability Feasibility
Viability

“We believe that …”
✅ Testable
✅ Precise
✅ Focused
Define your hypothesis
We believe that our users will spend
more time on a video if they can
engage with the creator in a chat

✅ Hypothesis - “We believe that …”
✅ Experiment method - “To verify that, we will …”
✅ Measurement method - “And measure …”
✅ Success Criteria - “We are right if …”
Define your experiment
Credit: Ryan Yosua

Picking the right
experiment method
02 Deﬁne your Experiment (Experiment Method)

What Users Do
Observational Research
What Users Say
Attitudinal Research
Direct Contact with Users
Qualitative Research
Indirect Contact with Users
Quantitative Research

What Users Do
What Users Say
👍 Great if you want to ﬁnd out if you
product is ﬁndable, understandable,
usable.
👍 Great to see if your users are actually
committing to what they tell you
👎 Might miss out on the whole picture

What Users Do
What Users Say
👍 Great for uncovering the needs, pains,
gains your users have
👍 Great for understanding why your users
preform a job and ﬁnding their
motivations
👎 Often more time consuming

What Users Do
What Users Say
👍 Great for going deeper into areas of
interest
👍 Gives you the possibility to be ﬂexible
and adjust to get better insights
👎 Time consuming before/during/after

What Users Do
What Users Say
👍 Can be automated and scalable
👍 Can be used for more reliable
comparisons over time
👎 Leaving out the ‘why’ and underlaying
needs

What Users Do
What Users Say
• Field Studies
• Usability Test
• Benchmark
• Recordings
• A/B Testing
• Analytics
• Heatmap
• Click Analysis
• Interviews
• Focus Groups
• Participatory design
• Customer Feedback
• Surveys
• Diary Studies
• Card Sorting
• Concept Testing
• Surveys
• NPS
• Desirability Studies

What Users Do
What Users Say
• Field Studies
• Usability Test
• Benchmark
• Recordings
• A/B Testing
• Analytics
• Heatmap
• Click Analysis
• Interviews
• Focus Groups
• Surveys
• Diary Studies
• Card Sorting
• Concept Testing
• Surveys
• NPS
Use if you want to understand what is
most important to your users and why
that is in a certain experience.

What Users Do
What Users Say
Why & What to fix
• Field Studies
• Usability Test
• Benchmark
• Recordings
• A/B Testing
• Analytics
• Heatmap
• Click Analysis
• Interviews
• Focus Groups
• Surveys
• Diary Studies
• Card Sorting
• Concept Testing
• Surveys
• NPS
Use if you want to gain understanding in
what users do when performing a task or
job

What Users Do
What Users Say
• Field Studies
• Usability Test
• Benchmark
• Recordings
• A/B Testing
• Analytics
• Heatmap
• Click Analysis
• Interviews
• Focus Groups
• Surveys
• Diary Studies
• Card Sorting
• Concept Testing
• Surveys
• NPS
Use if you want to understand how users
group information and what mental
models they have

Deep-dive in
Experiment Methods

Why should I do Interviews?
When you need qualitative insights into your users needs, pains, gains,
and what jobs that they want to accomplish. Conducting interviews are
great ways to validate desirability and viability hypothesis.
What do I need?
1. Vetted Interviewees (your target users), booked times and location
2. Written script with focused questions
3. Assigned Interviewers and notetakers
Interview

• Find and vet Interviewees
• Book time slots and send
invitations
• Write a script
• Invite a co-worker as a
notetaker
Before During After
• Debrief after each interview
• Synthesise the notes and
group insights
• Share results throughout the
organisation
• Act on the result (validated or
invalidated)
• Ask questions and be flexible
with the script
• Capture what the user is
saying and their emotions
Interview

Tools to use
PingPong - Focuses speciﬁcally on moderated remote user research. Recruit from their pool of 10k+
testers or use your own users.
Lookback - Lets you see how your users are navigating your test in real time or in recorded sessions.
Zoom & Google Meet - With the ability to record these conferencing softwares are also great for user
interviews.
Interview

Have in mind
01 Everyone’s time is at a premium right now — don’t waste your users’ time
02 If you have one hour with the participant, plan for research for just 45 minutes
03
If you can’t bring a colleague that can take notes then use a software where
you can record the session
Interview

Why should I do Surveys?
When you need attitudinal data from a big number of users,
for example how they rank different features in your service.
Keep in mind that the data can be kind of shallow, and that
survey is a method you should combine with other methods.
What do I need?
1. Vetted users (your target users)
2. Simple and focused questions
3. Survey tool
Survey

• Consider the type of
participants and how they will
be invited to the survey
• Consider types of questions
(open or closed)
• For longer surveys incentives can
be needed (like the possibility to
win a gift card or similar)
Before During After
• Keep track of responses
• Thank the participants (and give
them the incentive if applied)
• Evaluate the survey results
• Summarise the results and share
within your organisation
• Act on the result (validated or
invalidated)
Survey

Recommended tools
• Google Forms
• Typeform
• Surveymonkey
• Hotjar
Survey

Have in mind
01 Keep the questions simple and direct
02 Make the survey short, or else there will be a drop off in response rate
03 Be careful when you analyze the data - people might interpret the same question differently
Survey

Why should I do Usability Testing?
Identify usability improvements before the design gets coded, to save
valuable time and money. You can also evaluate current features to see
how they work - too many companies have usability problems they are
unaware of.
What do I need?
1. Vetted users (your target users)
2. Testing plan
3. Tool for remote usability testing
Usability Testing

• Recruit participants
• Make a testing plan
• Set up and invite
participants to online tool
Before During
• Record the session
• Make the user feel comfortable
• Make sure you’re testing the
product, not the user
After
• Debrief with your colleagues
working in the same project
• Look through the recordings
and highlight patterns
Usability Testing

Tools to use
Usabilityhub - A range of different unmoderated methods such as
click analysis and preference tests.
Usertesting - One of the most popular tools that let you both recruit
and conduct research.
PingPong - Focuses speciﬁcally on moderated remote user research.
Recruit from their pool of 10k+ testers or use your own users.
Lookback - Lets you see how your users are navigating your test in
real time or in recorded sessions.
Usability Testing

Have in mind
01 Five participants are enough to ﬁnd 85% of all usability problems
02
Test with your target user group, people with true incentives to use the product
probably value different things than you close family and colleagues do
03
Remind the users to “think aloud” when using the product, and help them doing so
by asking questions - “Why did you do that?” “What did you think of this button?”
Usability Testing

Validating hypotheses with user research

Validating hypotheses with user research

More Related Content

Similar to Validating hypotheses with user research

Recently uploaded

Validating hypotheses with user research