UXPA 2013 Annual Conference July 11, 2013 1:30 - 2:30pm by Helena Roeber, Jhilmil Jain UX research professionals often desire most that their work has impact, which can be tricky in fast-paced industry environments with short development cycles. This talk describes C-RITE , a method that maximizes impact through cross-disciplinary collaboration on research observation, analysis, and design exploration within an agile, user-centered development framework. We showcase real-world tested techniques that Google’s Android and TV teams have developed and share tactics for reducing logistical overhead in a lean, iterative user-centered design process.

1. C(ollab) RITE
How to run impactful iterative studies in a fast paced environment
Helena Roeber
Jhilmil Jain
Google
#uxpa2013
#crite
@jhilmiljain
@helenaroeber
#androidux

2. C-RITE Impact
• User research insights are adopted quickly
• Product team has clear sense of progress
• Researchers and designers earn
understanding and respect for their work
• Advocates for the user centered design
process are born
#uxpa2013 #crite #androidux

3. C-RITE
• Iterative UX research method suitable for agile
environments
• From research to new design solution in 1 day
• 7 research sessions in 2 days
• Collaborative design workshop each day
• Prototype changes between day 1 and 2
• Can be repeated weekly
• Stakeholders collaborate in all UX phases
#uxpa2013 #crite #androidux

4. User Research Stages
Research Analysis
Reporting
&
Prioritization
Design Prototyping
Planning
& Recruiting
#uxpa2013 #crite #androidux

5. The
Players
The Players and Their Roles

6. The
Researcher
#uxpa2013 #crite #androidux

7. Traditional Researcher Role
Research Analysis Design Prototyping
Reporting
&
Prioritization
Planning
& Recruiting
Researcher

8. The
Designer
#uxpa2013 #crite #androidux

9. Reporting
&
Prioritization
Planning
& Recruiting
Engineers
Traditional Designer Role
Engineer
Research Analysis Design Prototyping
Reporting
&
Prioritization
Planning
& Recruiting
Designer Designer
Researcher

10. The
Product
Manager
#uxpa2013 #crite #androidux

11. Research Analysis
Reporting
&
Prioritization
Design Prototyping
Planning
& Recruiting
Traditional PM Role
Reporting
&
Prioritization
Planning
& Recruiting
Research Analysis Design Prototyping
Reporting
&
Prioritization
Planning
& Recruiting
Designer Designer
Researcher
PM PM

12. The
Engineer
#uxpa2013 #crite #androidux

13. Research Analysis
Reporting
&
Prioritization
Design Prototyping
Planning
& Recruiting
Traditional Engineering Role
Reporting
&
Prioritization
Planning
& Recruiting
Research Analysis Design Prototyping
Reporting
&
Prioritization
Planning
& Recruiting
Designer Designer
Researcher
PM PM
Engineer

14. Collaboration in C-RITE
Research Analysis Design Prototyping
Reporting
&
Prioritization
Planning
& Recruiting
Designer
PM
Engineer
Researcher

15. Time Line - Traditional Usability
Research Analysis
Reporting
&
Prioritization
Design Prototyping
Planning
& Recruiting
1-2 weeks 1-2 weeks 1/2 - 1week 1/2 -1week
3 - 6 weeks
#uxpa2013 #crite #androidux

16. Time Line - Traditional Usability
Research Analysis
Reporting
&
Prioritization
Design Prototyping
Planning
& Recruiting
1- 2 weeks 1- 2 weeks 1/2 - 1week 1/2 -1week
4 - 8 weeks
#uxpa2013 #crite #androidux
1/2 - 1 week1/2 -1 week

17. Time Line - RITE
Research Analysis
Reporting
&
Prioritization
Design Prototyping
Planning
& Recruiting
1 - 2 weeks 1 - 2 weeks
2 - 4 weeks
#uxpa2013 #crite #androidux

18. Time Line - C-RITE
Research Analysis
Reporting
&
Prioritization
Design Prototyping
Planning
& Recruiting
1-2 weeks 2.5 days
1.5 - 2.5 weeks
#uxpa2013 #crite #androidux

19. Time Line - Cyclic C-RITE
Research Analysis
Reporting
&
Prioritization
Design Prototyping
Planning
& Recruiting
1 week 2.5 days
1.5 weeks
#uxpa2013 #crite #androidux

20. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
Example: TV Voice Search

21. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
1st TV Voice Search Design

22. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Research Timing
10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm
Day 1 Day 2
#uxpa2013 #crite #androidux

23. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Collaborative Analysis
10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm
Day 1 Day 2
#uxpa2013 #crite #androidux

24. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
Observation Board Example

25. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
User Goals Example
Observation Example
Goal example
P1P1 P2P2
Bob Lin Ali Nur Bob Lin Ali Nur
User knows what to say 2 1 2 2
The system understood user’s intent 5 3 1 1
The system doesn’t slow down user
w unnecessary confirmations 4 4 2 2
User can easily tell where speech
will work
2 2 2 2
User comfortable using their voice 4 4 4 4
Rating on a scale from 1- 5 1 = poor 5 = good

26. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
Collaborative Observation
#uxpa2013 #crite #androidux
• Re-call to count observations, analysis step
reduced
• Disagreements about observations are resolved
immediately
• Research protocol can be adjusted based on
stakeholder feedback
• Researcher can explain what observations
mean
• Small sample size discussion is obsolete

27. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
C-RITE Analysis
10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm
Day 1 Day 2
2:00 pm 3:00 pm
#uxpa2013 #crite #androidux

28. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
From Observation to Insight
Observation Example
Goal example

29. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
Prioritizing Insights
Goal example
Insights

30. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
C-RITE Design Workshop
3:00 pm
Day 1
10:00 am 11:00 am 1:00 pm 2:00 pm 10:00 am 11:00 am 1:00 pm 2:00 pm
Day 2
6:30 pm5:30 pm
#uxpa2013 #crite #androidux

31. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
Design Workshop

32. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
Design Workshop Actions
Observation Example
Goal example

33. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
Action Item Owners & Due Dates

34. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
C-RITE Design Workshop
#uxpa2013 #crite #androidux
• Turn panic into confidence

35. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
C-RITE Design Workshop
#uxpa2013 #crite #androidux
• Turn panic into confidence
• Designers demonstrate problem solving skills

36. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
C-RITE Design Workshop
#uxpa2013 #crite #androidux
• Turn panic into confidence
• Designers demonstrate problem solving skills
• Engineers solve technical constraints

37. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
C-RITE Design Workshop
#uxpa2013 #crite #androidux
• Turn panic into confidence
• Designers demonstrate problem solving skills
• Engineers solve technical constraints
• Researchers guide insight interpretation

38. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
C-RITE Design Workshop
#uxpa2013 #crite #androidux
• Turn panic into confidence
• Designers demonstrate problem solving skills
• Engineers solve technical constraints
• Researchers guide insight interpretation
• PM gives business and scheduling input

39. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
C-RITE Design Workshop
#uxpa2013 #crite #androidux
• Turn panic into confidence
• Designers demonstrate problem solving skills
• Engineers solve technical constraints
• Researchers guide insight interpretation
• PM gives business and scheduling input
• Everyone participates in creative process

40. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
Before & After

41. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Reporting
Day 1
Send action items to team
Day 2
Send action items to team Short report
Day 3
#uxpa2013 #crite #androidux

42. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Prototyping
3:00 pm10:00 11:00 1:00 2:00
Day 2
6:30 pm
Day 1
10:00 11:00 1:00 2:00 5:30 pm
#uxpa2013 #crite #androidux

43. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Cycles
Week1 Week 2 Week 3
#uxpa2013 #crite #androidux

44. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Recruiting
• Create a panel of users that meet
general recruitment criteria
• Panel members commit to availability at
short notice on research days
• Each week, schedule panel members
• Recruit for specific criteria when needed
#uxpa2013 #crite #androidux

45. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Room Design
• Comfortable seating for all
to observe sessions
• Whiteboard to record
observations, and drawing
during design workshop
• Sunlight and windows is
good for creativity
#uxpa2013 #crite #androidux

46. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Preparations
• Set expectations with all stakeholders
beforehand about time commitment,
collaboration and workshop format
• Decision makers need to participate in
C-RITE studies and workshops
#uxpa2013 #crite #androidux

47. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
C-RITE Summary
#uxpa2013 #crite #androidux
• Iterative UX research method suitable for agile
environments
• From research to new design solution in 1 day
• 7 research sessions in 2 days
• Collaborative design workshop each day
• Prototype changes between day 1 and 2
• Can be applied weekly
• Stakeholders collaborate in all UX phases

48. Research
Observation
& Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
Thank you
#uxpa2013 #crite #androidux
jhilmil.jain@gmail.com @jhilmiljain
@helenaroeberhelena.roeber@gmail.com

49. Appendix

50. Agile Methods: RITE
• Changes are made as soon as a problem is
identified, sometimes after 1 session
• 1-2 sessions a day. Breaks between sessions long
enough to make changes
• 1-2 weeks of sessions
• Requires stakeholder attendance and participation
Medlock, Wixon, Fulton, Terrano and Romero, UPA, 2002
#uxpa2013 #crite #androidux

51. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
Agile Methods: KRUG + RITE
• 4 participants over 2 days, at least 1 hour
between sessions to make changes
• Researcher sends out observations and
recommendations 30 min after last session
• Team debrief of observations and action items on
day 2, 1 hour after last session
• No collaborative design workshop
McGinn, Chen, Journal of Usability Studies, 2013

52. Research Analysis
Reporting &
Prioritization
Design Prototyping
Planning
& Recruiting
Agile Methods: PULSE
• 3 participants, 1 day every week
• Analysis and write-up of results by researcher
within 2 days
• No collaborative analysis and design workshop
Konno, Agile 2012

53. Research Analysis
Reporting &
Prioritization
Design Implementation
Planning
& Recruiting
Variations We’ve Tried
• C-RITE Remote
• If time zones allow, stakeholders participate through ‘Hangouts’
• Sessions are recorded and watched with time delay
• Next day debriefs and design workshops for incompatible time
zones
• C-RITE Diary Studies
• In-lab diary kick-offs and debriefs are run in C-RITE format
• Researcher analyzes and summarizes diary data for debrief and
design workshop