Unmoderated remote user        testing:    Do’s and don’ts       2012 © Humix, all rights reserved
Remote User Testing     2012 © Humix, all rights reserved   2
Synchronous vs asynchronous         2012 © Humix, all rights reserved   3
Moderated vs unmoderated        2012 © Humix, all rights reserved   4
Moderated usability testing         2012 © Humix, all rights reserved   5
Unmoderated remote  usability testing     2012 © Humix, all rights reserved   6
FiguresSource: www.measuringusability.com/blog/ux-methods.php 2011 UPA Salary Survey (biennal)          2012 © Humix, all ...
ADVANTAGES       2012 © Humix, all rights reserved   8
Test large groups    2012 © Humix, all rights reserved   9
Real environments    2012 © Humix, all rights reserved   10
Reduce costs  2012 © Humix, all rights reserved   11
Recruitment 2012 © Humix, all rights reserved   12
Organisation  2012 © Humix, all rights reserved   13
Fast results 2012 © Humix, all rights reserved   14
Quantitative & Qualitative        2012 © Humix, all rights reserved   15
Intercept users   2012 © Humix, all rights reserved   16
Self-reported feedback      2012 © Humix, all rights reserved   17
DRAWBACKS       2012 © Humix, all rights reserved   18
Gold diggers  2012 © Humix, all rights reserved   19
No interaction  2012 © Humix, all rights reserved   20
Only survey answers     2012 © Humix, all rights reserved   21
False positives   2012 © Humix, all rights reserved   22
Task time… 2012 © Humix, all rights reserved   23
TOOLS        2012 © Humix, all rights reserved   24
Choose the right toolFunctionality of tools has ahuge impact on the type ofinfo you’re able to gather                2012 ...
Choose the right tool• What is your budget?• What is your goal?• How much support do you need?• Will you be able to conduc...
Information Architecture toolsCard sorting               2012 © Humix, all rights reserved   27
Information Architecture toolsTree testing               2012 © Humix, all rights reserved   28
Click based feedbackTask & Interface testing                2012 © Humix, all rights reserved   29
Feedback gathering     2012 © Humix, all rights reserved   30
Unmoderated remoteusability testing tools      2012 © Humix, all rights reserved   31
When to use URUT?• Not a replacement for moderated testing  or research• Best used to complement  your qualitative researc...
Interesting reading• Remote Research: Real Users,  Real Time, Real Research by Nate  Bolt, Tony Tulathimutte, 2010• Beyond...
www.humix.be                   Lonneke.Spinhof@humix.be                   @humix @lonnekespinhof2012 © Humix, all rights r...
Upcoming SlideShare
Loading in …5
×

Unmoderated Remote Usability Testing

896 views

Published on

Do's and don'ts of unmoderated remote user testing. How we put URUT into practice.

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
896
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
9
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Remote usability testingdenotes a situationwhere “the evaluators are separated in spaceand/or time from users”Thisdefinitionemphasizestwogeneral types of methods: synchronousandasynchronous.
  • Reducecosts, depending on the tool youuse
  • It is easierto get participantstovolunteerbecausethey do not have totravel.
  • Fewerlogisticsto manage and no needtoworryabout no-shows andgetting last minute replacements
  • Partial automated results  faster resultsEasier to conduct multiple small tests
  • Both quantitativeand (some) qualitative dataQuantitative:Taks-completionrateTime on taskTime on pageClickstreampathsSatisfaction ratings or opinion rankingsQualitative:Most of these tools canalsocapturequalitative feedback as users complete theirtasks - such as users’ suggestionsandcomments
  • Someparticipantsmayonlybeinterested in earning the honorarium you’veprovided as an incentive.You have little or no contact and thus control over the motivation of participants
  • Nothing beats watchingparticipants in real time andbeingabletoaskprobingquestionsaboutwhatthey are doing as it’s happening.
  • Whatparticipants report on surveyscanbevery different in comparisontowhattheyactually do.
  • It’s possibleforparticipantstothinkthey’vesuccessfullycompleted a taskwhentheyhaven’t
  • Does it matter ifittooksomeonelongerto complete a task?
  • Unmoderated Remote Usability Testing

    1. 1. Unmoderated remote user testing: Do’s and don’ts 2012 © Humix, all rights reserved
    2. 2. Remote User Testing 2012 © Humix, all rights reserved 2
    3. 3. Synchronous vs asynchronous 2012 © Humix, all rights reserved 3
    4. 4. Moderated vs unmoderated 2012 © Humix, all rights reserved 4
    5. 5. Moderated usability testing 2012 © Humix, all rights reserved 5
    6. 6. Unmoderated remote usability testing 2012 © Humix, all rights reserved 6
    7. 7. FiguresSource: www.measuringusability.com/blog/ux-methods.php 2011 UPA Salary Survey (biennal) 2012 © Humix, all rights reserved 7
    8. 8. ADVANTAGES 2012 © Humix, all rights reserved 8
    9. 9. Test large groups 2012 © Humix, all rights reserved 9
    10. 10. Real environments 2012 © Humix, all rights reserved 10
    11. 11. Reduce costs 2012 © Humix, all rights reserved 11
    12. 12. Recruitment 2012 © Humix, all rights reserved 12
    13. 13. Organisation 2012 © Humix, all rights reserved 13
    14. 14. Fast results 2012 © Humix, all rights reserved 14
    15. 15. Quantitative & Qualitative 2012 © Humix, all rights reserved 15
    16. 16. Intercept users 2012 © Humix, all rights reserved 16
    17. 17. Self-reported feedback 2012 © Humix, all rights reserved 17
    18. 18. DRAWBACKS 2012 © Humix, all rights reserved 18
    19. 19. Gold diggers 2012 © Humix, all rights reserved 19
    20. 20. No interaction 2012 © Humix, all rights reserved 20
    21. 21. Only survey answers 2012 © Humix, all rights reserved 21
    22. 22. False positives 2012 © Humix, all rights reserved 22
    23. 23. Task time… 2012 © Humix, all rights reserved 23
    24. 24. TOOLS 2012 © Humix, all rights reserved 24
    25. 25. Choose the right toolFunctionality of tools has ahuge impact on the type ofinfo you’re able to gather 2012 © Humix, all rights reserved 25
    26. 26. Choose the right tool• What is your budget?• What is your goal?• How much support do you need?• Will you be able to conduct competitive research?• Can you test prototypes?• Can you export findings?• How will the results be visualized?• Does it come in different languages?• Can you combine different methods using the same tool?• Are there integrated recruiting tools? 2012 © Humix, all rights reserved 26
    27. 27. Information Architecture toolsCard sorting 2012 © Humix, all rights reserved 27
    28. 28. Information Architecture toolsTree testing 2012 © Humix, all rights reserved 28
    29. 29. Click based feedbackTask & Interface testing 2012 © Humix, all rights reserved 29
    30. 30. Feedback gathering 2012 © Humix, all rights reserved 30
    31. 31. Unmoderated remoteusability testing tools 2012 © Humix, all rights reserved 31
    32. 32. When to use URUT?• Not a replacement for moderated testing or research• Best used to complement your qualitative research 2012 © Humix, all rights reserved 32
    33. 33. Interesting reading• Remote Research: Real Users, Real Time, Real Research by Nate Bolt, Tony Tulathimutte, 2010• Beyond the usability lab: Conducting Large-scale Online User Experience Studies, Bill Albert, Tom Tullis, Donna Tedesco, 2010 2012 © Humix, all rights reserved 33
    34. 34. www.humix.be Lonneke.Spinhof@humix.be @humix @lonnekespinhof2012 © Humix, all rights reserved

    ×