C(ollab) RITE
How to run impactful iterative studies in a fast paced environment

Helena Roeber
Jhilmil Jain
Google

#uxpa...
C-RITE Impact
• User research insights are adopted quickly
• Product team has clear sense of progress
• Researchers and de...
C-RITE
• Iterative UX research method suitable for agile
environments
• From research to new design solution in 1 day
• 7 ...
User Research Stages

Planning
& Recruiting

Research

Analysis

Reporting
&
Prioritization

Design

Prototyping

#uxpa201...
The Players and Their Roles
The
Players
The
Researcher

#uxpa2013 #crite #androidux
Traditional Researcher Role

Planning
& Recruiting

Research

Researcher

Analysis

Reporting
&
Prioritization

Design

Pr...
The
Designer

#uxpa2013 #crite #androidux
Traditional Designer Role

Planning
Planning
& Recruiting
& Recruiting

Research

Analysis

Reporting
Reporting
&
&
Priori...
The
Product
Manager

#uxpa2013 #crite #androidux
Traditional PM Role

Planning
Planning
& Recruiting
& Recruiting

Research

Analysis

Reporting
Reporting
&
&
Prioritizati...
The
Engineer

#uxpa2013 #crite #androidux
Traditional Engineering Role

Planning
Planning
& Recruiting
& Recruiting

Research

Analysis

Reporting
Reporting
&
&
Pri...
Collaboration in C-RITE

Planning
& Recruiting

Research

Researcher
Designer
PM
Engineer

Analysis

Reporting
&
Prioritiz...
Time Line - Traditional Usability

Planning
& Recruiting

Research

Analysis

1-2 weeks

1-2 weeks

1/2 - 1week

Reporting...
Time Line - Traditional Usability

Planning
& Recruiting

Research

Analysis

1- 2 weeks

1- 2 weeks

1/2 - 1week

Reporti...
Time Line - RITE

Planning
& Recruiting

1 - 2 weeks

Research

Analysis

Reporting
&
Prioritization

Design

Prototyping
...
Time Line - C-RITE

Planning
& Recruiting

1-2 weeks

Research

Analysis

Reporting
&
Prioritization

Design

Prototyping
...
Time Line - Cyclic C-RITE

Planning
& Recruiting

1 week

Research

Analysis

Reporting
&
Prioritization

Design

Prototyp...
Example: TV Voice Search

Planning
& Recruiting

Research

Analysis

Reporting &
Prioritization

Design

Prototyping
1st TV Voice Search Design

Planning
& Recruiting

Research

Analysis

Reporting &
Prioritization

Design

Prototyping
C-RITE Research Timing

10:00 am

11:00 am

1:00 pm

Day 1

10:00 am

11:00 am

1:00 pm

2:00 pm

Day 2
#uxpa2013 #crite #...
C-RITE Collaborative Analysis

10:00 am

11:00 am

1:00 pm

10:00 am

Day 1

11:00 am

1:00 pm

2:00 pm

Day 2
#uxpa2013 #...
Observation Board Example

Planning
& Recruiting

Research

Analysis

Reporting &
Prioritization

Design

Prototyping
User Goals Example
P1
Bob

Lin

P2
Ali

Nur

2
1
Observation Example

2

2

Goal example

User knows what to say

The syst...
Collaborative Observation
• Re-call to count observations, analysis step
reduced
• Disagreements about observations are re...
C-RITE Analysis

10:00 am

11:00 am

1:00 pm

2:00 pm

Day 1

10:00 am

11:00 am

1:00 pm

2:00 pm

3:00 pm

Day 2
#uxpa20...
From Observation to Insight
Observation Example
Goal example

Planning
& Recruiting

Research

Analysis

Reporting &
Prior...
Prioritizing Insights
Insights
Goal example

Planning
& Recruiting

Research

Analysis

Reporting &
Prioritization

Design...
C-RITE Design Workshop

10:00 am

11:00 am

1:00 pm

2:00 pm

5:30 pm

Day 1

10:00 am

11:00 am

1:00 pm

2:00 pm

3:00 p...
Design Workshop

Planning
& Recruiting

Research

Analysis

Reporting &
Prioritization

Design

Implementation
Design Workshop Actions
Observation Example
Goal example

Planning
& Recruiting

Research

Analysis

Reporting &
Prioritiz...
Action Item Owners & Due Dates

Planning
& Recruiting

Research

Analysis

Reporting &
Prioritization

Design

Implementat...
Before & After

Planning
& Recruiting

Research

Analysis

Reporting &
Prioritization

Design

Implementation
C-RITE Design Workshop
• Turn panic into confidence
• Designers demonstrate problem solving skills
• Engineers solve techni...
C-RITE Reporting

Day 3

Day 2

Day 1

Send action items to team

Send action items to team

Short report

#uxpa2013 #crit...
C-RITE Prototyping

10:00

11:00

1:00

2:00

5:30 pm

10:00

11:00

1:00

2:00

3:00 pm

6:30 pm

Day 2

Day 1

#uxpa2013...
C-RITE Cycles

Week1

Week 2

Week 3

#uxpa2013 #crite #androidux
Planning
& Recruiting

Research

Analysis

Reporting &
P...
C-RITE Recruiting
• Create a panel of users that meet
general recruitment criteria
• Panel members commit to availability ...
C-RITE Room Design
• Comfortable seating for all
to observe sessions
• Whiteboard to record
observations, and drawing
duri...
C-RITE Preparations

• Set expectations with all stakeholders
beforehand about time commitment,
collaboration and workshop...
C-RITE Summary
• Iterative UX research method suitable for agile
environments
• From research to new design solution in 1 ...
Thank you

helena.roeber@gmail.com
jhilmil.jain@gmail.com

@helenaroeber
@jhilmiljain
#uxpa2013 #crite #androidux
Appendix
Agile Methods: RITE
• Changes are made as soon as a problem is
identified, sometimes after 1 session
• 1-2 sessions a day. ...
Agile Methods: KRUG + RITE
• 4 participants over 2 days, at least 1 hour
between sessions to make changes
• Researcher sen...
Agile Methods: PULSE

• 3 participants, 1 day every week
• Analysis and write-up of results by researcher
within 2 days
• ...
Variations We’ve Tried
• C-RITE Remote
•

If time zones allow, stakeholders participate through ‘Hangouts’

•

Sessions ar...
Upcoming SlideShare
Loading in …5
×

C(ollab) RITE: How to run impactful iterative studies in a fast paced environment

456 views

Published on

This talk describes C-RITE , a method that maximizes impact through cross-disciplinary collaboration on research observation, analysis, and design exploration within an agile, user-centered development framework. We showcase real-world tested techniques that Google’s Android and TV teams have developed and share tactics for reducing logistical overhead in a lean, iterative user-centered design process.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
456
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

C(ollab) RITE: How to run impactful iterative studies in a fast paced environment

  1. 1. C(ollab) RITE How to run impactful iterative studies in a fast paced environment Helena Roeber Jhilmil Jain Google #uxpa2013 #crite #androidux @helenaroeber @jhilmiljain
  2. 2. C-RITE Impact • User research insights are adopted quickly • Product team has clear sense of progress • Researchers and designers earn understanding and respect for their work • Advocates for the user centered design process are born #uxpa2013 #crite #androidux
  3. 3. C-RITE • Iterative UX research method suitable for agile environments • From research to new design solution in 1 day • 7 research sessions in 2 days • Collaborative design workshop each day • Prototype changes between day 1 and 2 • Can be repeated weekly • Stakeholders collaborate in all UX phases #uxpa2013 #crite #androidux
  4. 4. User Research Stages Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping #uxpa2013 #crite #androidux
  5. 5. The Players and Their Roles The Players
  6. 6. The Researcher #uxpa2013 #crite #androidux
  7. 7. Traditional Researcher Role Planning & Recruiting Research Researcher Analysis Reporting & Prioritization Design Prototyping
  8. 8. The Designer #uxpa2013 #crite #androidux
  9. 9. Traditional Designer Role Planning Planning & Recruiting & Recruiting Research Analysis Reporting Reporting & & Prioritization Prioritization Design Prototyping Researcher Designer Designer Engineer Engineers
  10. 10. The Product Manager #uxpa2013 #crite #androidux
  11. 11. Traditional PM Role Planning Planning & Recruiting & Recruiting Research Analysis Reporting Reporting & & Prioritization Prioritization Design Prototyping Researcher Designer PM Designer PM
  12. 12. The Engineer #uxpa2013 #crite #androidux
  13. 13. Traditional Engineering Role Planning Planning & Recruiting & Recruiting Research Analysis Reporting Reporting & & Prioritization Prioritization Design Prototyping Researcher Designer PM Designer PM Engineer
  14. 14. Collaboration in C-RITE Planning & Recruiting Research Researcher Designer PM Engineer Analysis Reporting & Prioritization Design Prototyping
  15. 15. Time Line - Traditional Usability Planning & Recruiting Research Analysis 1-2 weeks 1-2 weeks 1/2 - 1week Reporting & Prioritization Design Prototyping 1/2 -1week 3 - 6 weeks #uxpa2013 #crite #androidux
  16. 16. Time Line - Traditional Usability Planning & Recruiting Research Analysis 1- 2 weeks 1- 2 weeks 1/2 - 1week Reporting & Prioritization 1/2 -1week Design Prototyping 1/2 -1 week 1/2 - 1 week 4 - 8 weeks #uxpa2013 #crite #androidux
  17. 17. Time Line - RITE Planning & Recruiting 1 - 2 weeks Research Analysis Reporting & Prioritization Design Prototyping 1 - 2 weeks 2 - 4 weeks #uxpa2013 #crite #androidux
  18. 18. Time Line - C-RITE Planning & Recruiting 1-2 weeks Research Analysis Reporting & Prioritization Design Prototyping 2.5 days 1.5 - 2.5 weeks #uxpa2013 #crite #androidux
  19. 19. Time Line - Cyclic C-RITE Planning & Recruiting 1 week Research Analysis Reporting & Prioritization Design Prototyping 2.5 days 1.5 weeks #uxpa2013 #crite #androidux
  20. 20. Example: TV Voice Search Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  21. 21. 1st TV Voice Search Design Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  22. 22. C-RITE Research Timing 10:00 am 11:00 am 1:00 pm Day 1 10:00 am 11:00 am 1:00 pm 2:00 pm Day 2 #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  23. 23. C-RITE Collaborative Analysis 10:00 am 11:00 am 1:00 pm 10:00 am Day 1 11:00 am 1:00 pm 2:00 pm Day 2 #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  24. 24. Observation Board Example Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  25. 25. User Goals Example P1 Bob Lin P2 Ali Nur 2 1 Observation Example 2 2 Goal example User knows what to say The system understood user’s intent 5 3 1 4 4 2 2 2 2 4 4 4 Nur 2 User comfortable using their voice Ali 2 User can easily tell where speech will work Lin 1 The system doesn’t slow down user w unnecessary confirmations Bob 4 Rating on a scale from 1- 5 Planning & Recruiting Research 1 = poor 5 = good Analysis Reporting & Prioritization Design Prototyping
  26. 26. Collaborative Observation • Re-call to count observations, analysis step reduced • Disagreements about observations are resolved immediately • Research protocol can be adjusted based on stakeholder feedback • Researcher can explain what observations mean • Small sample size discussion is obsolete #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  27. 27. C-RITE Analysis 10:00 am 11:00 am 1:00 pm 2:00 pm Day 1 10:00 am 11:00 am 1:00 pm 2:00 pm 3:00 pm Day 2 #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  28. 28. From Observation to Insight Observation Example Goal example Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  29. 29. Prioritizing Insights Insights Goal example Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  30. 30. C-RITE Design Workshop 10:00 am 11:00 am 1:00 pm 2:00 pm 5:30 pm Day 1 10:00 am 11:00 am 1:00 pm 2:00 pm 3:00 pm 6:30 pm Day 2 #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  31. 31. Design Workshop Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  32. 32. Design Workshop Actions Observation Example Goal example Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  33. 33. Action Item Owners & Due Dates Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  34. 34. Before & After Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  35. 35. C-RITE Design Workshop • Turn panic into confidence • Designers demonstrate problem solving skills • Engineers solve technical constraints • Researchers guide insight interpretation • PM gives business and scheduling input • Everyone participates in creative process #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation
  36. 36. C-RITE Reporting Day 3 Day 2 Day 1 Send action items to team Send action items to team Short report #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  37. 37. C-RITE Prototyping 10:00 11:00 1:00 2:00 5:30 pm 10:00 11:00 1:00 2:00 3:00 pm 6:30 pm Day 2 Day 1 #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  38. 38. C-RITE Cycles Week1 Week 2 Week 3 #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  39. 39. C-RITE Recruiting • Create a panel of users that meet general recruitment criteria • Panel members commit to availability at short notice on research days • Each week, schedule panel members • Recruit for specific criteria when needed #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  40. 40. C-RITE Room Design • Comfortable seating for all to observe sessions • Whiteboard to record observations, and drawing during design workshop • Sunlight and windows is good for creativity #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  41. 41. C-RITE Preparations • Set expectations with all stakeholders beforehand about time commitment, collaboration and workshop format • Decision makers need to participate in C-RITE studies and workshops #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  42. 42. C-RITE Summary • Iterative UX research method suitable for agile environments • From research to new design solution in 1 day • 7 research sessions in 2 days • Collaborative design workshop each day • Prototype changes between day 1 and 2 • Can be applied weekly • Stakeholders collaborate in all UX phases #uxpa2013 #crite #androidux Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  43. 43. Thank you helena.roeber@gmail.com jhilmil.jain@gmail.com @helenaroeber @jhilmiljain #uxpa2013 #crite #androidux
  44. 44. Appendix
  45. 45. Agile Methods: RITE • Changes are made as soon as a problem is identified, sometimes after 1 session • 1-2 sessions a day. Breaks between sessions long enough to make changes • 1-2 weeks of sessions • Requires stakeholder attendance and participation Medlock, Wixon, Fulton, Terrano and Romero, UPA, 2002 #uxpa2013 #crite #androidux
  46. 46. Agile Methods: KRUG + RITE • 4 participants over 2 days, at least 1 hour between sessions to make changes • Researcher sends out observations and recommendations 30 min after last session • Team debrief of observations and action items on day 2, 1 hour after last session • No collaborative design workshop McGinn, Chen, Journal of Usability Studies, 2013 Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  47. 47. Agile Methods: PULSE • 3 participants, 1 day every week • Analysis and write-up of results by researcher within 2 days • No collaborative analysis and design workshop Konno, Agile 2012 Planning & Recruiting Research Analysis Reporting & Prioritization Design Prototyping
  48. 48. Variations We’ve Tried • C-RITE Remote • If time zones allow, stakeholders participate through ‘Hangouts’ • Sessions are recorded and watched with time delay • Next day debriefs and design workshops for incompatible time zones • C-RITE Diary Studies • In-lab diary kick-offs and debriefs are run in C-RITE format • Researcher analyzes and summarizes diary data for debrief and design workshop Planning & Recruiting Research Analysis Reporting & Prioritization Design Implementation

×