The RITE+Krug Method
Getting crazy good stakeholder involvement
in Agile user research
Jen McGinn & Ana Chang
December 5, ...
Format

We’ll tell you our story

We’ll listen to your stories
We’ll give you summaries and links to
further reading
Hopef...
Before we tell you our story, let’s start with you

Show of hands:
– How many of you are working with Agile development
te...
Late 2011: Traditional usability tests were conducted

I can’t believe he clicked
the blue button.
I assumed he would clic...
Later 2011: The stakeholders didn’t buy into the results

The results are not
statistically significant.

This is not a ty...
The feedback came too late for our Agile environment
Current state of the
application

Proposed
new design

Current state ...
Now it’s your turn
• First names only
• Agile User Research only
• In less than a minute, tell us:
– What user research yo...
Ana had attended a conference

Hey, what about
RITE testing?

Jen McGinn and Ana Chang

PS SIG CHI Monthly Meeting, Dec 20...
Great – so now what?

Rapid Iterative Test Environment (RITE)
– Developed by Dennis Wixon & the folks at Microsoft
games
–...
Early 2012: What’s been written on Agile + RITE?

• When I started looking for answers 18+ months ago:
• Lots and lots on ...
Steve Krug … why does that name sound familiar?

http://www.amazon.com/Steve-Krug/e/B001KHCFUU/

11
The Krug Approach*
“A morning a sprint, that’s all we ask.”
• Test 3 users in a morning

• Invite everyone to attend: prod...
Our RITE + Krug Combination
• Elements from both RITE and KRUG
– Require that a stakeholder from dev, design and PM attend...
Artifacts: scaled down script

Jen McGinn and Ana Chang

PS SIG CHI Monthly Meeting, Dec 2013

14
Artifacts: example wiki page with schedule and impact

Jen McGinn and Ana Chang

PS SIG CHI Monthly Meeting, Dec 2013

15
RITE+Krug Tips & Tricks
• Before sessions: Hold a kickoff meeting with stakeholders to
get buy-in into the process and tim...
Let’s talk about Diana

The conversation has
changed from ‘what is it?’
to ‘how can we master it
and make it our own?’
- D...
Diana’s Agile Research Recommendations*
• Conduct research before the development cycle if possible
– But if you can’t, fi...
Jeff Gothelf’s Approach

• Continuous & collaborative
– Continuous: every sprint
– Collaborative: conduct
research togethe...
Test everything

“In order to maintain a regular cadence of user testing,
your team must adopt a “test everything” policy....
What I do now: Continuous and Collaborative
• Collaborative
– Dev, Design, Doc, and PM stakeholders participate or observe...
What else have people written on User Research + Agile?

Agile + RITE testing, finally (!) described in detail (2013):
• “...
Agile & Lean UX resources that include user research

• “Adapting Usability Investigations for Agile User-centered Design”...
Takeaways

• You and your team can incorporate user research into Agile
development environments
• Align research with the...
Questions?

?
jen dot mcginn at oracle dot com & ana dot ramirez dot chang at oracle dot com
Jen McGinn and Ana Chang

PS ...
The RITE+Krug Method
Getting crazy good stakeholder involvement in Agile
user research
Jen McGinn & Ana Chang
December 5, ...
Small, think-aloud study

Ease of Use

Large A/B study

Different versions of the UI

Jen McGinn and Ana Chang
A/B testing
• A validation technique that you can use when you have:
– Working code on the web
– Large numbers of particip...
“Formative” Feedback Sessions
• A discovery technique that you can use when you have:
– drawings, sketches, wireframes
– s...
Upcoming SlideShare
Loading in...5
×

Talk to PS SIG CHI on Agile user research

756

Published on

Ana Chang and I presented this talk to PS SIG CHI on December 5, 2013. We also presented the material at Google and Chase on December 6th.

Published in: Design, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
756
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
12
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • A big usability study takes a long time and when you finally have the results, they may no longer be applicable.
  • We want to uncover the largest usability problems first, and then we can refine the UI and fix the smaller usability problems. A small, think-aloud usability study will not uncover all usability problems, but it will uncover the largest problems. Many stakeholders think large A/B studies are synonymous with user research. We need to introduce them to other kids of user research.
  • Talk to PS SIG CHI on Agile user research

    1. 1. The RITE+Krug Method Getting crazy good stakeholder involvement in Agile user research Jen McGinn & Ana Chang December 5, 2013
    2. 2. Format We’ll tell you our story We’ll listen to your stories We’ll give you summaries and links to further reading Hopefully, you’ll leave with something helpful And we’ll pick up some tricks from you  Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 2
    3. 3. Before we tell you our story, let’s start with you Show of hands: – How many of you are working with Agile development teams? – How many of you are conducting user research (feedback sessions) with those Agile teams? – How many of you are feeling really AWESOME about how the dev team is involved in that research and that they make changes as a result? Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 3
    4. 4. Late 2011: Traditional usability tests were conducted I can’t believe he clicked the blue button. I assumed he would click the gray button. We should fix that! Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 4
    5. 5. Later 2011: The stakeholders didn’t buy into the results The results are not statistically significant. This is not a typical user. That’s not a realistic task. I don’t agree with your interpretation of what that means. Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 5
    6. 6. The feedback came too late for our Agile environment Current state of the application Proposed new design Current state of the application At the beginning of the study Jen McGinn and Ana Chang Start of study When the study results are published PS SIG CHI Monthly Meeting, Dec 2013 6
    7. 7. Now it’s your turn • First names only • Agile User Research only • In less than a minute, tell us: – What user research you are conducting now (method, frequency, # of participants) – One thing that is AWESOME about that OR – One thing that’s not working, that you wish you could fix Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 7
    8. 8. Ana had attended a conference Hey, what about RITE testing? Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 8
    9. 9. Great – so now what? Rapid Iterative Test Environment (RITE) – Developed by Dennis Wixon & the folks at Microsoft games – Can change the (live) system between participants – Developers must attend the UT sessions – Keep testing until show-stoppers are fixed – Number of participants and time to conduct undefined Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 9
    10. 10. Early 2012: What’s been written on Agile + RITE? • When I started looking for answers 18+ months ago: • Lots and lots on Agile UX + design • Not so much on Agile UX + user feedback • RITE (described in general terms, but no specifics) • Co-design • Customer/user “proxies” • However, there were some elements of Steve Krug’s approach that stuck with me Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 10
    11. 11. Steve Krug … why does that name sound familiar? http://www.amazon.com/Steve-Krug/e/B001KHCFUU/ 11
    12. 12. The Krug Approach* “A morning a sprint, that’s all we ask.” • Test 3 users in a morning • Invite everyone to attend: product owners, developers, designers, writers, marketing … • Debrief over lunch: 1 hour – Only people who observed can attend/vote (sort of) – Focus ruthlessly on most serious problems • Report takes less than 30 minutes to write – Just an email – 2 page max., mostly bullet lists * Originally presented by Steve Krug at the Boston UXPA 2013 Conference, May 2013 PS SIG CHI Monthly Meeting, Dec 2013 12
    13. 13. Our RITE + Krug Combination • Elements from both RITE and KRUG – Require that a stakeholder from dev, design and PM attend each session – Can change the prototype between participants – 3 or 4 participants – Findings discussed in 1-hour debrief – Minimal “report” (bullets on a wiki page) • Sessions went viral: 12+ observers • Could test every as often as every 2 weeks http://www.upassoc.org/upa_publications/jus/2013may/JUS_McGinn_May_2013.pdf Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 13
    14. 14. Artifacts: scaled down script Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 14
    15. 15. Artifacts: example wiki page with schedule and impact Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 15
    16. 16. RITE+Krug Tips & Tricks • Before sessions: Hold a kickoff meeting with stakeholders to get buy-in into the process and time required • During: Keep a running list of observations and email them to the stakeholders within 30 minutes of the last participant • After sessions: Ask attendees of the stakeholder meeting to review your list of observations prior to the meeting • During debrief: Walk through the findings to get agreement on what everyone saw (optional: agree to next steps to fix) • After debrief: Immediately document the observations and list of changes agreed to and send out/post on wiki Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 16
    17. 17. Let’s talk about Diana The conversation has changed from ‘what is it?’ to ‘how can we master it and make it our own?’ - Diana DeMarco Brown (over lunch one day) http://www.amazon.com/Agile-User-Experience-Design-Practitioners/dp/0124159532/ PS SIG CHI Monthly Meeting, Dec 2013 17
    18. 18. Diana’s Agile Research Recommendations* • Conduct research before the development cycle if possible – But if you can’t, fit it in anyway • Predictable and repeatable user feedback sessions during the cycle • Complement user feedback sessions with fast, low-cost data collection methods: – Surveys – Customer phone calls – Web analytics – Card sorts – Un-moderated usability tests * Originally presented by Diana Brown at the Boston UXPA 2013 Conference, May 2013 PS SIG CHI Monthly Meeting, Dec 2013 18
    19. 19. Jeff Gothelf’s Approach • Continuous & collaborative – Continuous: every sprint – Collaborative: conduct research together (dev, design, PM, &research) http://www.amazon.com/Lean-UX-Applying-Principles-Experience/dp/1449311652 Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 19
    20. 20. Test everything “In order to maintain a regular cadence of user testing, your team must adopt a “test everything” policy. Whatever is ready on testing day is what goes in front of the users… you’ll find yourself taking advantage of your weekly test sessions to get insight at every stage of design and development.” – Lean UX, Jeff Gothelf Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 20
    21. 21. What I do now: Continuous and Collaborative • Collaborative – Dev, Design, Doc, and PM stakeholders participate or observe – User researcher will facilitate or advise – Your opportunity to get “out of the building” • Continuous – Sessions take place at regular intervals, say once a sprint – User researcher will recruit 3 – 4 target participants per session + debrief same day Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 21
    22. 22. What else have people written on User Research + Agile? Agile + RITE testing, finally (!) described in detail (2013): • “The RITE Way to Prototype”, Shirey, Charng, & Nguyen http://uxmag.com/articles/the-rite-way-to-prototype Agile + Cognitive walkthroughs (2013): • “Informal cognitive walkthroughs (ICW): paring down and pairing up for an agile world”, Grigoreanu & Mohanna http://dx.doi.org/10.1145/2470654.2466421 Agile + Wizard of Oz testing (2006): • “Adding Usability Testing to an Agile Project”, Meszaros & Aston http://dx.doi.org/10.1109/AGILE.2006.5 Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 22
    23. 23. Agile & Lean UX resources that include user research • “Adapting Usability Investigations for Agile User-centered Design”, Sy (2007) http://www.upassoc.org/upa_publications/jus/ 2007may/agile-ucd.pdf • “Successful User Experience in an Agile Enterprise Environment”, Federoff & Courage (2009) http://link.springer.com/chapter/10.1007%2F978-3-642-02556-3_27 • Jeff Gothelf’s blog: http://www.jeffgothelf.com/blog/ • 5 tips for applying Lean UX to User Research, Jeff Sauro (2013) http://www.measuringusability.com/blog/lean-research.php Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 23
    24. 24. Takeaways • You and your team can incorporate user research into Agile development environments • Align research with the Agile development & design cycles • Use lightweight research methods • Use a few participants • Require that key stakeholders attend (and/or conduct) user research • Get feedback at regular intervals (once a sprint) • Get feedback on everything Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 24
    25. 25. Questions? ? jen dot mcginn at oracle dot com & ana dot ramirez dot chang at oracle dot com Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 25
    26. 26. The RITE+Krug Method Getting crazy good stakeholder involvement in Agile user research Jen McGinn & Ana Chang December 5, 2013
    27. 27. Small, think-aloud study Ease of Use Large A/B study Different versions of the UI Jen McGinn and Ana Chang
    28. 28. A/B testing • A validation technique that you can use when you have: – Working code on the web – Large numbers of participants – A narrowly-defined problem • “Which one of the designs results in fewer abandoned carts?” • “Which one of the designs results in a higher click-through rate?” “Unless you are a huge property like – like Bing or Google … you’ll have more things to test than you have traffic – Lean Analytics, Croll & Yaskovitz, Eric Reis, ed Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 28
    29. 29. “Formative” Feedback Sessions • A discovery technique that you can use when you have: – drawings, sketches, wireframes – small numbers of participants – questions about features, mental models, terminology, iconography, usability of your prototypes, current practices, pain points, values, design alternatives, or want to collect other qualitative data “Initially, you’re looking for qualitative data … you’re speaking to people … You’re exploring. You’re getting out of the building.” – Lean Analytics, Croll & Yaskovitz, Eric Reis, ed Jen McGinn and Ana Chang PS SIG CHI Monthly Meeting, Dec 2013 29
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×