Presentation by TecEd's Stephanie Rosenbaum and Larry Rusinsky at IUE 2015 (Internet User Experience) Ann Arbor, Michigan Case study of conducting usability testing in multiple international locations to provide design guidance that would ensure a smartphone app would succeed in multiple languages.

1. 24-hour Days, 4 Languages:
Challenges of International
Usability Testing

2. •Founded 1967: Ann Arbor, Rochester NY, Palo Alto
•Early practitioners of user research
Contributed to the development of early usability
testing methodologies that are standard today
•More than 150 clients; close to 2,000 projects
•Apps, websites, devices, other products for B2C/B2B

4. What makes a good localized
app?
•When do salespeople want materials in English,
and when in their local language?
•When do they search using English,
and when in their local language?
6
1 1
6
1
4
2 10
2
4
6
8
10
12
14
16
18
20
22
English Local
language
Other
response
NumberofParticipants
Favored Language
Favored Search Language
Do you prefer to search using English or
[language]?
French Speakers
Mandarin Speakers
Spanish Speakers

5. •Engaged bilingual partner researchers in France,
Chile, and China
•Coordinated with our client’s participant recruiters
•Created translation-ready research protocol in English
•Had the protocol translated into French, Spanish,
and Chinese
•Created data-logging template
•Had our partner researchers conduct pilot-test sessions in
the local language
Here’s What We Did

6. •Reviewed pilot session results with our client,
then adjusted the protocol
•Had our partner researchers conduct 7 or 8 sessions in the
local language
•Collected data logs in English from our partner researchers
•Arranged for translation of the session recordings into
English
•Wrote results report for presentation to our client, with
partner researcher input
More Details

7. Engaging Research Partners
• User research skills and experience
• Bilingual communication facility
• Availability for desired schedule

8. Coordinating Participant
Recruiting
 Salespeople: sensitive
recruiting
 Indirect access and
communication
 Unexpected outcomes

9.  Content similar to U.S. research projects
 Needed to be foolproof
 No U.S. slang, no ambiguities

10. Data-Logging Templates
 Consistent reporting from
researchers using 3 different
languages
 Sessions in native language, but
English data logs
 Structured logs facilitated data
analysis

11. Pilot-Test Sessions
 One in each language
 Observed by client stakeholders
over WebEx
 Each followed by debrief in English
with the partner researcher

12. At Last, the Sessions
 7 – 8 sessions in each language
 Over 15 time zones
 Just-in-time recruiting

13.  Arranged to translate all session
recordings into English
 5 cycles of report deliverables
 15 graphs of participant preferences
 Hundreds of participant quotations

14.  Project management for a multiple-
language usability test is hugely complex
 Maintaining consistency of methodology in
multiple countries is another challenge
 International holidays differ and can impact
schedules
 A project in 5 time zones spanning 15 hours
means you are never off-duty

15. Thank you!
Stephanie Rosenbaum
stephanie@teced.com
Larry Rusinsky
larry@teced.com