Discovering

WHY

from numbers

1234567890123456789012345678901234567890123456789012345678901234567890

©2013 Webnographer...
Why Numbers?
Really, the question is: Why not?

©2013 Webnographer Limited

2
20th Anniversary
of Nielson’s Law.
20 years ago it
was expensive.
There was a
diminishing
return.
N (1-(1- .31 ) n )
©2013...
Discoverability Gap based on the Theory

Likelihood of discovering an issue

100%

Lab Testing with 10 users
80%

Remote T...
56%

18%

30%

Issues that can be reliably found in the Lab
(5 respondents)

27%
25%

5 respondents find 80% of issues wit...
45%

28%

30%

Issues that can be reliably found in the Lab
(10 respondents)

27%
25%

10 respondents find 85% of issues w...
Benefits: Prioritise findings
Margin of error
45%

Because Remote Un-moderated
40%

Testing evaluates designs with

40%

l...
Why, why?

Why care about WHY in
quantitative research?

©2013 Webnographer Limited
Quant research is powerless without why
• If you know 50% of customers can’t find what they
need. What can you change?
• I...
The 2 Cultures
Science vs. humanities
One without the other is dangerous

C P Snow
©2013 Webnographer Limited

10
How to get WHY
from numbers?

©2013 Webnographer Limited

11
WISDOM

KNOWLEDGE

HOW

INFORMATION

WHAT

DATA

©2013 Webnographer Limited

WHY

THAT

12
Theory first

Socrates

Karl Popper

The Socratic method influenced scientific
method where you start with a hypothesis.

...
Finding out
WHY
customers
want your
product

©2013 Webnographer Limited

14
Nobel Prize

Winners of the Nobel prize in science were able to explain
phenomenons in the world through theory first
©201...
4 types of hypothesis for UX research

Design
as hypothesis

Hypothesis for
why you are RIGHT

©2013 Webnographer Limited
...
Design as hypothesis

Optimum Path:
• Is the shortest possible path to the target page.
• Is the hypothesis of the designe...
Design as hypothesis
Lostness
There may be other alternative routes to the target
page which are longer.

x
x

However, th...
Other Data as Hypothesis

Survey data

©2013 Webnographer Limited

Lab test findings

Web analytics data

19
Stakeholder Hypothesis

CONFUSION

©2013 Webnographer Limited

UNDERSTANDING

20
Review Methods as Hypothesis

Heuristic Evaluation, Cognitive Walkthrough,
or GOMS for each step of the design hypothesis
...
Create a UX Hypothesis

The hypothesis is the theory
WHY something happens.
•

Write down a hypothesis for
how customers b...
Test your hypothesis through un-moderated remote usability testing

1

2

To evaluate the research
questions, participants...
©2013 Webnographer Limited

24
Remote Usability Testing Can
When using the right method

©2013 Webnographer Limited

25
Thank you.
sabrina@webnographer.com
@sabrinamach

james@webnographer.com
@jamespage

©2013 Webnographer Limited

26
Upcoming SlideShare
Loading in...5
×

UX by the numbers: Discovering the why from numbers

267

Published on

Published in: Technology, Design
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
267
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
1
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

UX by the numbers: Discovering the why from numbers

  1. 1. Discovering WHY from numbers 1234567890123456789012345678901234567890123456789012345678901234567890 ©2013 Webnographer Limited 1
  2. 2. Why Numbers? Really, the question is: Why not? ©2013 Webnographer Limited 2
  3. 3. 20th Anniversary of Nielson’s Law. 20 years ago it was expensive. There was a diminishing return. N (1-(1- .31 ) n ) ©2013 Webnographer Limited 3
  4. 4. Discoverability Gap based on the Theory Likelihood of discovering an issue 100% Lab Testing with 10 users 80% Remote Testing with 80 users 60% 40% N (1-(1- .31 ) n ) 20% 0% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Visibility of the issue The above Lab testing numbers mean that 10 users will find 85% of the issues with a visibility of 17%. The these numbers use the formula from Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of the finding of usability problems,“. ©2013 Webnographer Limited 4
  5. 5. 56% 18% 30% Issues that can be reliably found in the Lab (5 respondents) 27% 25% 5 respondents find 80% of issues with 30% visibility BUT that is only 18% of ALL issues 21% Issue Distribution Issue distribution is the % of issues that fall into each category of issue visibility Discoverability Gap based on data 20% 18% of interactions with issues identified 27% of interactions have no issue = 45.5% coverage 15% 14% 56% of interactions that have issues get missed. 10% 8% 5% 6% 5% 2% 3% 3% 2% 1% 1% 2% 1% 0% 1% 0% 1% 1% 1% 1% 0% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% ©2013 Webnographer Limited Likelihood of an issue occurring 5
  6. 6. 45% 28% 30% Issues that can be reliably found in the Lab (10 respondents) 27% 25% 10 respondents find 85% of issues with 17% visibility BUT that is only 28% of ALL issues 21% Issue Distribution Issue distribution is the % of issues that fall into each category of issue visibility Discoverability Gap based on data 20% 28% of interactions with issues identified 27% of interactions have no issue = 55% coverage 15% 14% 45% of interactions that have issues get missed. 10% 8% 5% 6% 5% 2% 3% 3% 2% 1% 1% 2% 1% 0% 1% 0% 1% 1% 1% 1% 0% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% ©2013 Webnographer Limited Likelihood of an issue occurring 6
  7. 7. Benefits: Prioritise findings Margin of error 45% Because Remote Un-moderated 40% Testing evaluates designs with 40% large numbers of users, actionable 35% metrics have a small margin of 30% error. This allows the prioritisation of insights and high level 25% decision making. 20% 15% The findings from the lab have a large margin of error because they 9% 10% are evaluated with only a few 5% users. This makes it hard to Number of participants 0% 0 50 Lab Testing ©2013 Webnographer Limited 100 150 200 250 Remote Un-moderated Testing 300 350 prioritise the findings. The chart uses Sauro’s 20/20 rule. The rule states that Testing with 20 users will produce a margin of error of +/-20%. Also see http://www.measuringusability.com/test-margin.php for an explanation of the 20/20 rule. 7
  8. 8. Why, why? Why care about WHY in quantitative research? ©2013 Webnographer Limited
  9. 9. Quant research is powerless without why • If you know 50% of customers can’t find what they need. What can you change? • If you cannot explain a finding, it is powerless. ©2013 Webnographer Limited
  10. 10. The 2 Cultures Science vs. humanities One without the other is dangerous C P Snow ©2013 Webnographer Limited 10
  11. 11. How to get WHY from numbers? ©2013 Webnographer Limited 11
  12. 12. WISDOM KNOWLEDGE HOW INFORMATION WHAT DATA ©2013 Webnographer Limited WHY THAT 12
  13. 13. Theory first Socrates Karl Popper The Socratic method influenced scientific method where you start with a hypothesis. A hypothesis is proven in the negative. ©2013 Webnographer Limited 13
  14. 14. Finding out WHY customers want your product ©2013 Webnographer Limited 14
  15. 15. Nobel Prize Winners of the Nobel prize in science were able to explain phenomenons in the world through theory first ©2013 Webnographer Limited
  16. 16. 4 types of hypothesis for UX research Design as hypothesis Hypothesis for why you are RIGHT ©2013 Webnographer Limited Other data as hypothesis Stakeholder Views as hypothesis Review methods as hypothesis Hypothesis for why you are WRONG 16
  17. 17. Design as hypothesis Optimum Path: • Is the shortest possible path to the target page. • Is the hypothesis of the designer of how customers behave. 1 Click on Broadband & Internet in Help and Support ©2013 Webnographer Limited 2 Reached the Help hub page 3 Click on My connection is slow 4 Reach the “Slow connection” page 17
  18. 18. Design as hypothesis Lostness There may be other alternative routes to the target page which are longer. x x However, the interface should help people find the shortest path to the information. Customers do not want the scenic route. They want to get stuff done. x 1 2 Click on Broadband & Internet in Help and Support 3 Reached the Help hub page x x ©2013 Webnographer Limited 4 Reach the “Slow connection” page Click on My connection is slow x x x Longer routes can be an indication for the interface facilitating error recovery. 18
  19. 19. Other Data as Hypothesis Survey data ©2013 Webnographer Limited Lab test findings Web analytics data 19
  20. 20. Stakeholder Hypothesis CONFUSION ©2013 Webnographer Limited UNDERSTANDING 20
  21. 21. Review Methods as Hypothesis Heuristic Evaluation, Cognitive Walkthrough, or GOMS for each step of the design hypothesis ©2013 Webnographer Limited 21
  22. 22. Create a UX Hypothesis The hypothesis is the theory WHY something happens. • Write down a hypothesis for how customers behave • What evidence do you need to support your hypothesis? • What evidence do you need to disprove your hypothesis? ©2013 Webnographer Limited 22
  23. 23. Test your hypothesis through un-moderated remote usability testing 1 2 To evaluate the research questions, participants are asked to complete a Webtask on the website. The Webtask presents participants with a task to perform using the website. While they navigate through the site their interactions are recorded with our remote usability testing tool. 3 4 Remote usability testing records all interactions with the website, such as clicks, hovering, keystrokes, back press, page scrolling, time in each form item, errors per item, time on page, pages visited, task completion rate, time on task. Participant’s interaction with the website allows us to collect comprehensive information regarding website usability, and customer behaviours. Webtasks provide thus valuable insights on where and why customers are experiencing problems, and casts light on ways to improve website usability. ©2013 Webnographer Limited 23
  24. 24. ©2013 Webnographer Limited 24
  25. 25. Remote Usability Testing Can When using the right method ©2013 Webnographer Limited 25
  26. 26. Thank you. sabrina@webnographer.com @sabrinamach james@webnographer.com @jamespage ©2013 Webnographer Limited 26
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×