UserZoom Big Design Conf May09

  • 1,051 views
Uploaded on

Unmoderated remote usability testing

Unmoderated remote usability testing

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,051
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
12
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. www.userzoom.com Tips and Tricks for Remote Unmoderated Usability Testing Alfonso de la Nuez Founder at UserZoom May 2009 1
  • 2. www.userzoom.com Overview • Introduction to Remote Unmoderating User Testing (RUUT) • What is the value and how does it work? • When and how to use it as a research method • Best practices and examples
  • 3. www.userzoom.com What does the market say about web research & analytics? Usability testing in the lab is great, but it has its limitations Web analytics tells you a lot about the WHAT, not the WHY Conducting international testing is expensive Benchmarking and measuring UX is more important than ever
  • 4. www.userzoom.com Overall conclusion: Today’s Web and E-business Managers seek more sophisticated and innovative research and analytics products to help them compete
  • 5. www.userzoom.com Comparing various research methods for usability research Test REAL Large Behavioural Natural tasks Samples Reasons WHY Data Setting Focus Groups Usability Labs Online surveys Web analytics RUUT
  • 6. www.userzoom.com RUUT: What is the value and how does it work? 1. No face-to-face, human moderation (moderation through software) 2. Hundreds of users participate simultaneously… 3. …from geographically spread locations (local & int’l) and… 4. …in their natural context (using their own PCs) 5. Software captures both usability data and users’ behavior Very cost-effective and faster
  • 7. www.userzoom.com Define your study Launch study, collect data Analyze results & & monitor progress generate a report Choose the type of study Invite users to participate Perform advanced data you want to conduct (or ask your visitors to do it) analysis Write your study script Collect data automatically Use filtering capabilities Choose the recruitment Monitor the progress real- Copy & paste results method & audience time Export automatically into Pre-test before launch Word or Excel
  • 8. www.userzoom.com What about user recruitment? Option 1: Panel companies* • Million users, spread across the world • Wide variety of profiles, great segmentation capabilities • Strict, professional control and panel mgmt. Option 2: Intercept method • Ask users who visit the website to participate in a study. • Insert a javascript in the page * Use a panel company Option 3: Private panels, mailing lists • We can also work with private databases
  • 9. www.userzoom.com When to use this research method Whenever you need to quantify usability problems, not just identify them If you need to test users in various locations (local or international) For benchmarking and UX measurement projects To obtain user behavior data (search strategy, click-streams and click-maps) When you want to combine usability and market research For quick pre-testing during the development cycle (prototype testing)
  • 10. www.userzoom.com Task based studies: Real users attempting real tasks Websites
  • 11. www.userzoom.com Task based studies: Test any web-based interface Prototypes
  • 12. www.userzoom.com Task based studies: Test any web-based interface Mobile interfaces
  • 13. www.userzoom.com Also, you can conduct… • Online Card sorting exercises • True Intent studies • Free search studies Let’s take a look… (all videos are taken from pilot studies)
  • 14. www.userzoom.com Card sorting: Open and closed categories
  • 15. www.userzoom.com True intent studies (through intercept method)
  • 16. www.userzoom.com Online user behavior research (through free search studies)
  • 17. www.userzoom.com Important! Remote does not mean you’re not in control About scripting capabilities: • High-level of flexibility • Advanced task validation criteria (by question, by page interception, by time...) • Advanced questionnaires tool (all type of questions: Choice, scales, open...) • Insert multimedia (images, videos, flash, applets, sounds...) • Manage advanced logic/branching & conditions • Behavioral questionnaires • Task randomization • Time-out options • Sample quality control
  • 18. www.userzoom.com Ok… What about the results you get?
  • 19. www.userzoom.com Effectiveness & efficiency ratios, plus answers to the questionnaire
  • 20. www.userzoom.com You have filtering capabilities
  • 21. www.userzoom.com
  • 22. www.userzoom.com
  • 23. www.userzoom.com Text clouds to analyze open-ended questions
  • 24. www.userzoom.com Clickstreams per task
  • 25. www.userzoom.com Clickmaps/heatmaps per page
  • 26. www.userzoom.com Finally, about reporting & exporting capabilities Generate automatic reports (.doc, .excel, .ppt) Copy & paste capabilities Export raw-data for further advanced analysis Share results online
  • 27. www.userzoom.com Any examples?
  • 28. www.userzoom.com Testing users in various locations (local or international) Monster’s UX Director: “Need to test various local sites, in 5 countries, but I have a limited budget and time plan” They tested 200 users online, in 5 markets Data was collected in approximately 1 month Visitbritain.co.uk A year later the study was repeated
  • 29. www.userzoom.com Need to quantify usability issues, not just identify them Spanair’s Marketing Director: “Loved the results from the lab, but I simply can’t Spanair is Spain’s 2nd largest airline make a decision based on the feedback from 10 users” The airline ended up testing 15 users in the lab (included eye-tracking), then added 229 users online Tasks were identical, results were not…
  • 30. www.userzoom.com Nº Nº Success Success of users of users rate rate Task (Lab) (remote) (Labs) (UZ) Find a flight of a special class 3/15 185/229 20 % 71 % Register on the web 4/5 5/12 70% 42 % Check your current balance (loyalty program) 8/15 65/73 53.3% 88 % Spanair is Spain’s 2nd largest airline
  • 31. www.userzoom.com For UX benchmarking & brand proposition research Arsys’s Marketing Director: Arsys is Spain’s “We’d like to measure both user experience and leading ISP branding metrics over a period of 3 years” They test 200 users each semester UK’s Training & Upload results in a UX Dashboard Development Agency
  • 32. www.userzoom.com When to use unmoderated RUT Examples Testing users in their natural context of use Badoo’s Management had 3 goals and issues in mind: Badoo is a UK- 1. Were concerned about testing in the lab based, fast-growing online dating and 2. Wanted to check differences between cultures social network site 3. Wanted to combine usability testing with brand-value proposition testing RUUT was the perfect solution for them: Were able to ask private matters openly Were able to test in 5 markets at once and compare results Asked users about their brand-value perceptions
  • 33. Thanks and have a great day! Would you like a demo? Contact us and we’ll offer you a chance to see and try out our technology Do you have any questions? We’re sure you have many questions. Don’t hesitate to contact us and we’ll be glad to answer them. Contact information Alfonso de la Nuez Virginie Glaenzer Xavier Mestres Founder & Int’l Business Dev Sales Manager, USA Founder & CTO / R&D alfonso@userzoom.com vglaenzer@userzoom.com xmestres@userzoom.com USA Office Members of 440 N. Wolfe Rd. - Sunnyvale, CA 94085 (USA) · [t] +1 (408) 524 7445 Spain offices Av. Diagonal 419, 3º 2ª - 08008 Barcelona - T. +34 93 414 75 54 - F. +34 93 368 42 95 Gran Vía 62, 7º Izda. - 28013 Madrid - T. +34 91 542 62 52 - F. +34 91 298 08 92 www.userzoom.com
  • 34. www.userzoom.com Introduction: Our ‘elevator pitch’ 1. What is UserZoom and what do we do? • International market research firm, specializing in online user experience and usability research. • We help brands boost website performance and improve online business revenues. 2. How do we do it? • We create a unique web-based software application that empowers researchers to conduct highly sophisticated online research. • We offer both a self-service solution and professional services. 3. Bottom line… Why should you work with us? • You’ll be able to capture & measure the critical and actionable data about your users’ online experience that you need to manage your business, and no other tool can provide!