UX by the numbers: Discovering the why from numbers

  • 168 views
Uploaded on

 

More in: Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
168
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Discovering WHY from numbers 1234567890123456789012345678901234567890123456789012345678901234567890 ©2013 Webnographer Limited 1
  • 2. Why Numbers? Really, the question is: Why not? ©2013 Webnographer Limited 2
  • 3. 20th Anniversary of Nielson’s Law. 20 years ago it was expensive. There was a diminishing return. N (1-(1- .31 ) n ) ©2013 Webnographer Limited 3
  • 4. Discoverability Gap based on the Theory Likelihood of discovering an issue 100% Lab Testing with 10 users 80% Remote Testing with 80 users 60% 40% N (1-(1- .31 ) n ) 20% 0% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% Visibility of the issue The above Lab testing numbers mean that 10 users will find 85% of the issues with a visibility of 17%. The these numbers use the formula from Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of the finding of usability problems,“. ©2013 Webnographer Limited 4
  • 5. 56% 18% 30% Issues that can be reliably found in the Lab (5 respondents) 27% 25% 5 respondents find 80% of issues with 30% visibility BUT that is only 18% of ALL issues 21% Issue Distribution Issue distribution is the % of issues that fall into each category of issue visibility Discoverability Gap based on data 20% 18% of interactions with issues identified 27% of interactions have no issue = 45.5% coverage 15% 14% 56% of interactions that have issues get missed. 10% 8% 5% 6% 5% 2% 3% 3% 2% 1% 1% 2% 1% 0% 1% 0% 1% 1% 1% 1% 0% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% ©2013 Webnographer Limited Likelihood of an issue occurring 5
  • 6. 45% 28% 30% Issues that can be reliably found in the Lab (10 respondents) 27% 25% 10 respondents find 85% of issues with 17% visibility BUT that is only 28% of ALL issues 21% Issue Distribution Issue distribution is the % of issues that fall into each category of issue visibility Discoverability Gap based on data 20% 28% of interactions with issues identified 27% of interactions have no issue = 55% coverage 15% 14% 45% of interactions that have issues get missed. 10% 8% 5% 6% 5% 2% 3% 3% 2% 1% 1% 2% 1% 0% 1% 0% 1% 1% 1% 1% 0% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100% ©2013 Webnographer Limited Likelihood of an issue occurring 6
  • 7. Benefits: Prioritise findings Margin of error 45% Because Remote Un-moderated 40% Testing evaluates designs with 40% large numbers of users, actionable 35% metrics have a small margin of 30% error. This allows the prioritisation of insights and high level 25% decision making. 20% 15% The findings from the lab have a large margin of error because they 9% 10% are evaluated with only a few 5% users. This makes it hard to Number of participants 0% 0 50 Lab Testing ©2013 Webnographer Limited 100 150 200 250 Remote Un-moderated Testing 300 350 prioritise the findings. The chart uses Sauro’s 20/20 rule. The rule states that Testing with 20 users will produce a margin of error of +/-20%. Also see http://www.measuringusability.com/test-margin.php for an explanation of the 20/20 rule. 7
  • 8. Why, why? Why care about WHY in quantitative research? ©2013 Webnographer Limited
  • 9. Quant research is powerless without why • If you know 50% of customers can’t find what they need. What can you change? • If you cannot explain a finding, it is powerless. ©2013 Webnographer Limited
  • 10. The 2 Cultures Science vs. humanities One without the other is dangerous C P Snow ©2013 Webnographer Limited 10
  • 11. How to get WHY from numbers? ©2013 Webnographer Limited 11
  • 12. WISDOM KNOWLEDGE HOW INFORMATION WHAT DATA ©2013 Webnographer Limited WHY THAT 12
  • 13. Theory first Socrates Karl Popper The Socratic method influenced scientific method where you start with a hypothesis. A hypothesis is proven in the negative. ©2013 Webnographer Limited 13
  • 14. Finding out WHY customers want your product ©2013 Webnographer Limited 14
  • 15. Nobel Prize Winners of the Nobel prize in science were able to explain phenomenons in the world through theory first ©2013 Webnographer Limited
  • 16. 4 types of hypothesis for UX research Design as hypothesis Hypothesis for why you are RIGHT ©2013 Webnographer Limited Other data as hypothesis Stakeholder Views as hypothesis Review methods as hypothesis Hypothesis for why you are WRONG 16
  • 17. Design as hypothesis Optimum Path: • Is the shortest possible path to the target page. • Is the hypothesis of the designer of how customers behave. 1 Click on Broadband & Internet in Help and Support ©2013 Webnographer Limited 2 Reached the Help hub page 3 Click on My connection is slow 4 Reach the “Slow connection” page 17
  • 18. Design as hypothesis Lostness There may be other alternative routes to the target page which are longer. x x However, the interface should help people find the shortest path to the information. Customers do not want the scenic route. They want to get stuff done. x 1 2 Click on Broadband & Internet in Help and Support 3 Reached the Help hub page x x ©2013 Webnographer Limited 4 Reach the “Slow connection” page Click on My connection is slow x x x Longer routes can be an indication for the interface facilitating error recovery. 18
  • 19. Other Data as Hypothesis Survey data ©2013 Webnographer Limited Lab test findings Web analytics data 19
  • 20. Stakeholder Hypothesis CONFUSION ©2013 Webnographer Limited UNDERSTANDING 20
  • 21. Review Methods as Hypothesis Heuristic Evaluation, Cognitive Walkthrough, or GOMS for each step of the design hypothesis ©2013 Webnographer Limited 21
  • 22. Create a UX Hypothesis The hypothesis is the theory WHY something happens. • Write down a hypothesis for how customers behave • What evidence do you need to support your hypothesis? • What evidence do you need to disprove your hypothesis? ©2013 Webnographer Limited 22
  • 23. Test your hypothesis through un-moderated remote usability testing 1 2 To evaluate the research questions, participants are asked to complete a Webtask on the website. The Webtask presents participants with a task to perform using the website. While they navigate through the site their interactions are recorded with our remote usability testing tool. 3 4 Remote usability testing records all interactions with the website, such as clicks, hovering, keystrokes, back press, page scrolling, time in each form item, errors per item, time on page, pages visited, task completion rate, time on task. Participant’s interaction with the website allows us to collect comprehensive information regarding website usability, and customer behaviours. Webtasks provide thus valuable insights on where and why customers are experiencing problems, and casts light on ways to improve website usability. ©2013 Webnographer Limited 23
  • 24. ©2013 Webnographer Limited 24
  • 25. Remote Usability Testing Can When using the right method ©2013 Webnographer Limited 25
  • 26. Thank you. sabrina@webnographer.com @sabrinamach james@webnographer.com @jamespage ©2013 Webnographer Limited 26