UserZoom Big Design Conf May09


Published on

Unmoderated remote usability testing

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

UserZoom Big Design Conf May09

  1. 1. Tips and Tricks for Remote Unmoderated Usability Testing Alfonso de la Nuez Founder at UserZoom May 2009 1
  2. 2. Overview • Introduction to Remote Unmoderating User Testing (RUUT) • What is the value and how does it work? • When and how to use it as a research method • Best practices and examples
  3. 3. What does the market say about web research & analytics? Usability testing in the lab is great, but it has its limitations Web analytics tells you a lot about the WHAT, not the WHY Conducting international testing is expensive Benchmarking and measuring UX is more important than ever
  4. 4. Overall conclusion: Today’s Web and E-business Managers seek more sophisticated and innovative research and analytics products to help them compete
  5. 5. Comparing various research methods for usability research Test REAL Large Behavioural Natural tasks Samples Reasons WHY Data Setting Focus Groups Usability Labs Online surveys Web analytics RUUT
  6. 6. RUUT: What is the value and how does it work? 1. No face-to-face, human moderation (moderation through software) 2. Hundreds of users participate simultaneously… 3. …from geographically spread locations (local & int’l) and… 4. …in their natural context (using their own PCs) 5. Software captures both usability data and users’ behavior Very cost-effective and faster
  7. 7. Define your study Launch study, collect data Analyze results & & monitor progress generate a report Choose the type of study Invite users to participate Perform advanced data you want to conduct (or ask your visitors to do it) analysis Write your study script Collect data automatically Use filtering capabilities Choose the recruitment Monitor the progress real- Copy & paste results method & audience time Export automatically into Pre-test before launch Word or Excel
  8. 8. What about user recruitment? Option 1: Panel companies* • Million users, spread across the world • Wide variety of profiles, great segmentation capabilities • Strict, professional control and panel mgmt. Option 2: Intercept method • Ask users who visit the website to participate in a study. • Insert a javascript in the page * Use a panel company Option 3: Private panels, mailing lists • We can also work with private databases
  9. 9. When to use this research method Whenever you need to quantify usability problems, not just identify them If you need to test users in various locations (local or international) For benchmarking and UX measurement projects To obtain user behavior data (search strategy, click-streams and click-maps) When you want to combine usability and market research For quick pre-testing during the development cycle (prototype testing)
  10. 10. Task based studies: Real users attempting real tasks Websites
  11. 11. Task based studies: Test any web-based interface Prototypes
  12. 12. Task based studies: Test any web-based interface Mobile interfaces
  13. 13. Also, you can conduct… • Online Card sorting exercises • True Intent studies • Free search studies Let’s take a look… (all videos are taken from pilot studies)
  14. 14. Card sorting: Open and closed categories
  15. 15. True intent studies (through intercept method)
  16. 16. Online user behavior research (through free search studies)
  17. 17. Important! Remote does not mean you’re not in control About scripting capabilities: • High-level of flexibility • Advanced task validation criteria (by question, by page interception, by time...) • Advanced questionnaires tool (all type of questions: Choice, scales, open...) • Insert multimedia (images, videos, flash, applets, sounds...) • Manage advanced logic/branching & conditions • Behavioral questionnaires • Task randomization • Time-out options • Sample quality control
  18. 18. Ok… What about the results you get?
  19. 19. Effectiveness & efficiency ratios, plus answers to the questionnaire
  20. 20. You have filtering capabilities
  21. 21.
  22. 22.
  23. 23. Text clouds to analyze open-ended questions
  24. 24. Clickstreams per task
  25. 25. Clickmaps/heatmaps per page
  26. 26. Finally, about reporting & exporting capabilities Generate automatic reports (.doc, .excel, .ppt) Copy & paste capabilities Export raw-data for further advanced analysis Share results online
  27. 27. Any examples?
  28. 28. Testing users in various locations (local or international) Monster’s UX Director: “Need to test various local sites, in 5 countries, but I have a limited budget and time plan” They tested 200 users online, in 5 markets Data was collected in approximately 1 month A year later the study was repeated
  29. 29. Need to quantify usability issues, not just identify them Spanair’s Marketing Director: “Loved the results from the lab, but I simply can’t Spanair is Spain’s 2nd largest airline make a decision based on the feedback from 10 users” The airline ended up testing 15 users in the lab (included eye-tracking), then added 229 users online Tasks were identical, results were not…
  30. 30. Nº Nº Success Success of users of users rate rate Task (Lab) (remote) (Labs) (UZ) Find a flight of a special class 3/15 185/229 20 % 71 % Register on the web 4/5 5/12 70% 42 % Check your current balance (loyalty program) 8/15 65/73 53.3% 88 % Spanair is Spain’s 2nd largest airline
  31. 31. For UX benchmarking & brand proposition research Arsys’s Marketing Director: Arsys is Spain’s “We’d like to measure both user experience and leading ISP branding metrics over a period of 3 years” They test 200 users each semester UK’s Training & Upload results in a UX Dashboard Development Agency
  32. 32. When to use unmoderated RUT Examples Testing users in their natural context of use Badoo’s Management had 3 goals and issues in mind: Badoo is a UK- 1. Were concerned about testing in the lab based, fast-growing online dating and 2. Wanted to check differences between cultures social network site 3. Wanted to combine usability testing with brand-value proposition testing RUUT was the perfect solution for them: Were able to ask private matters openly Were able to test in 5 markets at once and compare results Asked users about their brand-value perceptions
  33. 33. Thanks and have a great day! Would you like a demo? Contact us and we’ll offer you a chance to see and try out our technology Do you have any questions? We’re sure you have many questions. Don’t hesitate to contact us and we’ll be glad to answer them. Contact information Alfonso de la Nuez Virginie Glaenzer Xavier Mestres Founder & Int’l Business Dev Sales Manager, USA Founder & CTO / R&D USA Office Members of 440 N. Wolfe Rd. - Sunnyvale, CA 94085 (USA) · [t] +1 (408) 524 7445 Spain offices Av. Diagonal 419, 3º 2ª - 08008 Barcelona - T. +34 93 414 75 54 - F. +34 93 368 42 95 Gran Vía 62, 7º Izda. - 28013 Madrid - T. +34 91 542 62 52 - F. +34 91 298 08 92
  34. 34. Introduction: Our ‘elevator pitch’ 1. What is UserZoom and what do we do? • International market research firm, specializing in online user experience and usability research. • We help brands boost website performance and improve online business revenues. 2. How do we do it? • We create a unique web-based software application that empowers researchers to conduct highly sophisticated online research. • We offer both a self-service solution and professional services. 3. Bottom line… Why should you work with us? • You’ll be able to capture & measure the critical and actionable data about your users’ online experience that you need to manage your business, and no other tool can provide!