Models and Methods for Global User Research


Published on

With the increasing focus on globalization of products and services, the need to understand user experience in distant markets is more urgent. However, conducting global user research can be confusing and is a significant risk of time and resources. In this webinar, we talk through several methods for global research. We’ll present these various methods and the tradeoffs and considerations for choosing one method over another. We’ll also step through the elements of success in conducting a global study – from planning to results.

This webinar is not about localization or globalization of user interfaces, it will instead focus on methods and practices for how one conducts successful global user research.

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Compression of time Irrelevance of place User research at a distanceConnection of Cultures User research close up
  • Thousands of users can be tested in multiple languages, locations and time zones in a short period of time. Two approaches:Task – based with a plug in“True Intent” based Quantitative and qualitative data can be collectedCan be extended easily to competitive or longitudinal testingExamples: LEOtrace, Vividence, User ZoomAugments traditional lab testing
  • <provide examples><because budgets are always an issue, the idea is to meet business objectives while optimizing time / resources><introduce a real case that can flow through balance of talk>
  • of
  • Japan – picking participantsFrance – insult to pay money, pick wineMexico – test in hotelsBrazil – power Translations  English to French increase time for testing by 10-20%Moderators have to slow downFall on their sword
  • Models and Methods for Global User Research

    1. 1. Robert Schumacher, PhD.Managing DirectorUser Centric, Inc
    2. 2. RobertM. Robert M.SchumacherSchumacher Managing Managing Director Director
    3. 3. User experience research in many places…
    4. 4. What is this talk not about• Globalization or localization of test artifacts
    5. 5. Increasing demand for global user research“The World is Flat” – Thomas Friedman Pace of technology introduction is increasing Global companies introducing products in unfamiliar markets and market segments Products / services are getting more complex Internet and mobility change what we work on and the way we work
    6. 6. Compression of Time Irrelevance of Place Must find ways to do user research better, faster, and Connection of (often) cheaper than we do today Cultures
    7. 7. • The Internet culture creates expectations – Partners / clients / collaborators / Compression of developers around the Time globe create work schedules that are on “world time”• Speed is a feature of user research – Agile development meets user research  must adapt methods – Example: Six country web site review done in 24 hours
    8. 8. • Outsourcing of core and supporting activities – Some kinds of user Compression of research can be Time outsourced to different time zones to achieve rapid turn around More research, more data… – Example: Overnight transcription of usability tests more faster• New tools enable faster production of artifacts – Automated data collection / summarization tools – Example: Digital video editing
    9. 9. RemoteMethodsFace to face
    10. 10. New technologies and techniques allow for Remote: – Moderated testing – Unmoderated testing – ObservationIrrelevance of Place
    11. 11. Remote Moderated Testing Products like GotoMeeting allow connections to the test (or observation) computer to the Internet. VoIP can carry voice cheaply. Translator ObserversParticipant Moderator LiveMeeting WebEx GoToMeeting For screen VoIP Audio Skype GoogleTalk
    12. 12. Remote Unmoderated Testing Find the umbrella. • Hundreds of users agree to participate in a study • In their natural context • From geographically spread locations • Users try to complete tasks + answer questions • No human moderation needed • Browser bar connects users with secure servers
    13. 13. Remote Unmoderated Testing ‘Task-based’ Surveys Online Card Sorting > Open or closed > Online/remote Usability Studies (unmoderated) > Stand alone or > Benchmarking (competitive /comparison) > Integrated with task-based studies & surveys > UX Dash`boards (measure ROI) Robust Set of Services Online Surveys User Recruiting Tool > Ad hoc research > Intercept real visitors (tab or layer) > Voice of Customer studies > Create your own private panel > Integrated with Web Analytics data > Use a panel provider*
    14. 14. RemoteMethodsFace to face
    15. 15. • Sometimes ‘place’ matters and it seems like you must test with users F2F• Some guidelines…
    16. 16. When to do… Remote Testing In Person• Web/software UI • Physical artifact• More quantitative based • Need rich qualitative feedback• Large, distributed sample • Need to have the or low incidence human connection• Low(er) budget • Ensure high-level of• High penetration of consistency Internet access • Uncertain of quality or environment
    17. 17. Recent Research Questions that suggested face to face research was needed• Why are conversions are lower in Brazil than in the rest of the world?• What is the customer journey for our product suite in Japan?• Is our site perceived to be ‘Chinese’ (enough)?• Do my translations into Spanish affect performance?• We have a new navigation system for our HDTV – how usable is it for American consumers?• Development wants to be certain that the site ‘works’ for our global users before it is launched…in 18 countries
    18. 18. You cannot and should not… conduct user research unless youknow what the purpose of the study is. DUH! However…
    19. 19. A lot of the time we spend is NOT on the technical. It’s on planning and logistics. Planning the conditions to collect good data.
    20. 20.  Increasing demand for user testing across borders – Usability practices can vary widely country to country – Many issues to consider for fieldwork… • Moderators • Interpreters and translations • Recruiting and incentives over-prepare Impossible to • Facilities Connection of Cultures
    21. 21. On-site  travelTwo Models of F2F Global User Research “Over the transom” Work remote with local partners
    22. 22. Images person in easy chair, or airline seatTravel to destinations? Stay put…work remote?- Cost: Time & money! - Cost:- Benefit: - Worry - Richer data - Possible quality issues - Higher consistency - Benefit: - Lower risk of failure - Lower cost - Faster turn around
    23. 23. Four Elements ofUser Research Projects Preparation Fieldwork Analysis & Reporting Project Management
    24. 24. Preparation: Recruiting• Recruiting practices differ widely – Privacy: Lists…sigh
    25. 25. So you need to test in… Be prepared for (very) high recruitment costs
    26. 26. Recruiting• Recruiting practices differ widely – Privacy: Sometimes a list is bad, really bad – Selecting participants…or not• Incidence and participation differs – How many “extras” or “floaters” do you need? What is the “show” rate? – What did you pay your participants?
    27. 27. Vive La France!
    28. 28. So you need to test in…Find creative ways to incent participants
    29. 29. Interpreters• “Think aloud” data is simply more difficult to collect through interpreters…• Professional simultaneous interpreters – Translate everything; maybe hire 2 interpreters? – Do not save money by hiring “cheap” interpreters• Language Expansion – Interpreters have to keep up… 32
    30. 30. So you need to test in…Make sure moderator speaks at an appropriate rate for interpreters
    31. 31. Interpreters & Translations• Observation Room: • Interpreters should see participant’s face • What’s the ambient language in obs room? • Observers listening live in different languages?• Translating test documents – Consider double translation to check accuracy (i.e., translate it back to native tongue)
    32. 32. Planning Fieldwork• Who should moderate? – Native speakers only? Fluent non-native OK?• What time of day can sessions be scheduled?• How many sessions can the moderator do per day?• National, cultural, religious issues…bugger
    33. 33.  Finding vendors can be hard and can be easy, some strategies… – Ask colleagues and clients – Resource networks • UPA • HCIRN • STC – Search engines (google, yahoo, ask) Vendor quality (over long-distance) is hard, but necessary; some ideas… – Check References Connection of – Ask for CVs of consultants Cultures – Ask about experience with foreign clients – Recent clients
    34. 34.  Business practices – Confidentiality and propriety not viewed similarly – Project timelines and urgencies may be more “elastic” – Budget, currency and payment terms can be Connection of challenging – always Cultures hidden costs (e.g., visas)
    35. 35. So you need to test in…Plan for a lot of time between sessions
    36. 36. Observers are HereRemote Observation Project team wants to watch Testing is Here testing, but can’t travel? Stream sessions…requires knowledge of network infrastructure and special tools – Encoded video pushed to streaming server and viewed anywhere
    37. 37. Technical Requirements• Ship or Carry Equipment? – Customs = Gotcha!
    38. 38. So you need to test in…Careful what you pack for the lab
    39. 39. Technical Requirements• Ship or Carry Equipment? – Customs = Gotcha!• Even keyboards! AZERTY v QWERTY• Powering equipment 220v or 110v or Both? – E.g., Sao Paulo has either 220 v or 110 v plugs• Local power…hopefully
    40. 40. So you need to test in… Have a back up plan in caseof power loss…or Internet…or monsoons
    41. 41. Facilities• Often unfamiliar and out of our control – Facility quality is hard to judge
    42. 42. So you need to test in… Some respondents won’t go to test facilities; test in hotels
    43. 43. Budgeting• Budgets are always up in the air with global user research – Typically more expensive• Then there are the extraordinary expenses…
    44. 44. So you need to test in… Budget for security guards
    45. 45. Test Planning• A Test Plan identifies all of the points of the testing: – Objectives – Research design – Participant screening criteria – blah… blah… blah…• And Intercultural UX Research elements
    46. 46. Robert Schumacher, PhD.Managing DirectorUser Centric, a Better User Experience in call centers, Press 1
    47. 47. There are many ways to succeedThere are many more ways to fail