![Distance Demands Documentation
Scope of Research Question
Range of Audience
Text vs. Visual
“I'll write up, here's what I think the
main findings were in basically a one-
page kind of thing.”
Time to Create
Shelf Life
“Everybody knows that producing
written documents in most
organizational cultures is not the
way to affect change.”
“Usually, we're bumping up at the
time when the product manager
has to hand [the product] over to
engineering for development.”
Sharing in the
Integrated Model
“It's mostly just the bare bones: what
did we observe, what did we learn
from it, what do we need to do now.”](https://image.slidesharecdn.com/kaiser-thereportisdeadlonglivethereport-uxpa2023-230628202448-4e6caf1c/75/The-Report-is-Dead-Long-Live-the-Report-How-to-Communicate-UX-Research-Findings-for-Maximum-Impact-20-2048.jpg)
Uploaded byKathi Kaiser
The Report is Dead, Long Live the Report! How to Communicate UX Research Findings for Maximum Impact
Kathi Kaiser's report emphasizes the evolution of UX research communication over 25 years, urging researchers to adapt their methods based on stakeholder proximity and context. Key insights reveal that successful sharing relies on collaboration and understanding audience attention, emphasizing process over artifacts for impactful results. In summary, the report argues that traditional reports may be outdated, advocating for dynamic engagement with stakeholders throughout the research process.
More Related Content
Similar to The Report is Dead, Long Live the Report! How to Communicate UX Research Findings for Maximum Impact
![[BROCHURE] Italy Tour Project | @SlideON](https://cdn.slidesharecdn.com/ss_thumbnails/brochure8-251215152319-2805af68-thumbnail.jpg?width=640&height=640&fit=bounds)
The Report is Dead, Long Live the Report! How to Communicate UX Research Findings for Maximum Impact
- 1. The Report isDead, Long Live the Report! How to Communicate Usability Research Findings for Maximum Impact Kathi Kaiser Centralis @kathikaiser www.linkedin.com/in/kathikaiser
- 2. • Co-founder ofCentralis • 20+ years making products easier, mostly through user research • Lots of training/mentoring of new UXers along the way • Observer and creator of various reporting approaches Hello! Hi! I’m Kathi Kaiser.
- 3. Back in theday… When I began in UX in the late 1990s, few people knew what UX research was.
- 4. Nowadays… In 25 years,UX researchers have grown in strength and numbers.
- 5. And so… Given ourrange of approaches, we should think critically about what we do, and why. • How do we share research results? • What seems to work best? • What challenges are we facing? • How should we adapt to different contexts? • How can we get beyond “It depends…”?
- 6. We researched… We solicitedinput from ~100 UX practitioners, via two surveys and follow-up interviews.
- 7. The greater thedistance between the researcher and the stakeholders, the more formal results sharing tends to be. And we learned... 1. Distance demands documentation.
- 8. Gaining buy-in forUX research results is more about a collaborative process than the artifacts produced. And we learned... 2. Process outperforms artifacts.
- 9. The best wayto package results depends on the researcher’s available time and the stakeholders’ available attention. And we learned... 3. Optimize time and attention.
- 10.
- 11. Distance Demands Documentation Researchersshare results in a wide range of ways, usually by creating an “artifact”.
- 12. Distance Demands Documentation ProductExecs Product Owner Designers Devs Imagine a sphere…
- 13. Employee Vendor Not on the ProductTeam On the Product Team Distance Demands Documentation Product Team Execs Product Owner Designers Devs Where do UX researchers fit in?
- 14. Where do UXresearchers fit in? The Integrated Model (Employee) Product Team Execs Product Owner Designers Devs Product Team Execs Product Owner Embedded UX Researcher Designers Devs Employee Vendor Not on the Product Team On the Product Team Distance Demands Documentation
- 15. Product Execs Product Owner Designers Devs Employee Vendor Not onthe Product Team On the Product Team Product Execs Product Owner Embedded UX Researcher Designers Devs Distance Demands Documentation The Integrated Model (Contractor) Product Team Execs Embedded UX Researcher Product Owner Designers Devs
- 16. Product Team Execs Product Owner Designers Devs Employee Vendor Not on the ProductTeam On the Product Team Distance Demands Documentation The Service Model In-House UX Research Consultant
- 17. Product Team Execs Product Owner Designers Devs Employee Vendor Not on the ProductTeam On the Product Team Distance Demands Documentation The Outsourced Model External UX Research Consultant
- 18. Product Team Execs Product Owner Designers Devs Employee Vendor Not on the ProductTeam On the Product Team Distance Demands Documentation The Outsourced Model In-House UX Research Consultant External UX Research Consultant
- 19. Product Team Execs Product Owner Designers Devs Employee Vendor Not on the ProductTeam On the Product Team Distance Demands Documentation Research must span different distances to reach the stakeholders in each model. Product Team Execs Product Owner Integrated Model Designers Devs Service Model Outsourced Model
- 20. Distance Demands Documentation Scopeof Research Question Range of Audience Text vs. Visual “I'll write up, here's what I think the main findings were in basically a one- page kind of thing.” Time to Create Shelf Life “Everybody knows that producing written documents in most organizational cultures is not the way to affect change.” “Usually, we're bumping up at the time when the product manager has to hand [the product] over to engineering for development.” Sharing in the Integrated Model “It's mostly just the bare bones: what did we observe, what did we learn from it, what do we need to do now.”
- 21. Distance Demands Documentation Scopeof Research Question Range of Audience Text vs. Visual “I sit within our Consumer Insights Department, which is an overarching research center of excellence – that area supports the entire enterprise.” Time to Create Shelf Life “There's like 15 years of research…some of the insights, drivers, motivations, those things are still the same. And so they're still relevant. And so it's helpful.” Sharing in the Service Model “The way we write our reports is having that executive summary up front, because we are thinking about the executives… tell me what you're going to do. What needs to be fixed and why?
- 22. Distance Demands Documentation Scopeof Research Question Range of Audience Text vs. Visual A: “Maybe less than 20 hours for a straightforward usability study…a big user research project, like could be like 40 or 50 or more hours.” Time to Create Shelf Life “I always walk through the whole report. So usually I book an hour, which is not always enough.” “I had a client reach out to me a couple of months ago who was like, we found your report from three years ago and it's incredible…we'd like you to do a new project with us.” Sharing in the Outsourced Model Q: “How long does it take you to write a usability report?” A: “Oh, forever. It takes forever.”
- 23. Distance Demands Documentation Scopeof Research Question Range of Audience Text vs. Visual Time to Create Shelf Life Sharing in the Outsourced Model Scope of Research Question Range of Audience Text vs. Visual Time to Create Shelf Life Sharing in the Service Model Scope of Research Question Range of Audience Text vs. Visual Time to Create Shelf Life Sharing in the Integrated Model
- 24. Distance Demands Documentation “Youwant your stakeholders to DO something with your report; otherwise, your report is useless. Why are you doing this work? It’s because you want change to happen. The way to make change happen is to involve the stakeholders from the beginning.”
- 25.
- 26. “Making the resultseasy to comprehend and showing visual examples” “Including an exec summary of key insights/ recommendations so they can quickly grasp the "greatest hits".” “Sharing participant quotes/anecdotes from testing” “…video clips showing participants failing (and succeeding).” Process Over Artifacts Q: What have you found most effective for sharing usability findings?(2020) Q: What have you found most effective for getting buy-in to the results of usability testing?(2023) “Building relationships and asking questions about the plans and roadmap” “Require that stakeholders observe usability testing.” “Collaboration on the goals and planning of the test” “Review the test plan with PM and UX and Engineering key stakeholders” “Including responsible stakeholders in planning and analysis.” “Having stakeholders watch… take notes on a shared sticky note space and ‘affinitizing’ the findings in a group workshop.” “Give stakeholders the opportunity to discuss/ask questions” “Cross-functional, collaborative analysis.” “Socializing findings ahead of the formal report also helps, as do clear recommendations.”
- 27. Process Over Artifacts ACommon Usability Research Process
- 28. Process Over Artifacts Levelof Involvement R Level of Involvement R R R R
- 29. Process Over Artifacts Levelof Involvement: Integrated Model R Level of Involvement R R R R S S S S S Integrated Model “I know the stakeholders I'm working with and I know what problem we're focused on at that time.”
- 30. Process Over Artifacts Levelof Involvement: Service Model R Level of Involvement R R R R S S S S S “If we're going to do research for you, you as a business owner need to participate. You cannot just wait for the report to be thrown over the wall.” Service Model
- 31. Process Over Artifacts Levelof Involvement: Outsourced Model R Level of Involvement R R R R S S S S S Outsourced Model “And then there's, you know, an analysis phase…usually it's just me, where I try to make sense of all the stuff that we saw in the usability testing.”
- 32. Process Over Artifacts Levelof Involvement R Level of Involvement R R R R S S S S S S S S S S S S S S S Integrated Model Service Model Outsourced Model
- 33. R S S Process Over Artifacts BoostingStakeholder Involvement Level of Involvement R R R S S S S S S S S S S R I S S R I S S R S S S Integrated Model Service Model Outsourced Model
- 34. R S S Process Over Artifacts BoostingStakeholder Involvement Level of Involvement R R R S S S S S S S S S S R I S S R I S S R S S S Integrated Model Service Model Outsourced Model ?
- 35. Process Over Artifacts 35 •The debrief session told us what we need to know • No one has time to write a report • No one will read a report • The design has already moved on • The debrief is for the team; the report is for the executives • The report is our memory • The report is our roadmap • The report is our product The Report is Dead! Long Live the Report! Integrated Model Service Model Outsourced Model
- 36. Optimize your timeand their attention.
- 37. Optimizing for Timeand Attention A Taxonomy of Reports (Cooley & Barnum, 2019) Cooley, D., Barnum, C. (2019). How Do Other People Do It? A Comparative Review of Usability Study Reports. User Experience Magazine, 19(1). Retrieved from https://uxpamagazine.org/how-do-other-people-do-it-a-comparative-review-of-usability-study-reports/ Visuals from DialogDesign: https://www.dialogdesign.dk/cue-studies/ ‘Traditional’ Reports ‘Visual’ Reports ‘Lean’ Reports
- 38. Optimizing for Timeand Attention Report Building Blocks 38 Key Findings Product Images Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method What we did How to illustrate it What to do about it What we learned
- 39. Optimizing for Timeand Attention Report Building Blocks 39 Key Findings Detailed Findings Severity Ratings Recos Method What we did How to illustrate it What to do about it Traditional Focus What we learned Product Images Quotes Video Clips Data Viz Session Metrics
- 40. Optimizing for Timeand Attention Report Building Blocks 40 Key Findings Detailed Findings Severity Ratings Recos Method What we did How to illustrate it What to do about it Visual Focus What we learned Product Images Quotes Video Clips Data Viz Session Metrics
- 41. Lean Focus Optimizing forTime and Attention Report Building Blocks 41 Key Findings Detailed Findings Severity Ratings Recos Method What we did How to illustrate it What to do about it Lean Focus What we learned Product Images Quotes Video Clips Data Viz Session Metrics
- 42. Optimizing for Timeand Attention Report Building Blocks Key Findings Product Images Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time
- 43. Optimizing for Timeand Attention The Attention Paradox: An Exercise 43 N P R F D R C I A P D Q B M W J F K
- 44. Optimizing for Timeand Attention The Attention Paradox: An Exercise 44 N P R F D R C I A P D Q B M W J F K
- 45. Optimizing for Timeand Attention The Attention Paradox: An Exercise 45 N P R F D R C I A P D Q B M W J F K N P R F D R C IA P D Q B M W J F K
- 46. Optimizing for Timeand Attention The Attention Paradox 46 Our attention is limited. The less attention needed to process something, the more attention available for processing it. We capture more attention by requiring less attention.
- 47. Optimizing for Timeand Attention Report Building Blocks Key Findings Product Images Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time
- 48. Optimizing for Timeand Attention Report Building Blocks Product Images Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings
- 49. Optimizing for Timeand Attention The Elevator Pitch Game Claire Slattery, https://claireslatterycoaching.com/
- 50. Optimizing for Timeand Attention Report Building Blocks Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images
- 51. Optimizing for Timeand Attention Report Building Blocks Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Quotes
- 52. Optimizing for Timeand Attention Otter.ai 52
- 53. Optimizing for Timeand Attention Report Building Blocks Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Quotes Key Findings Product Images Quotes Video Clips
- 54. Optimizing for Timeand Attention Reduct.video 54
- 55. Optimizing for Timeand Attention Report Building Blocks Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Quotes Key Findings Product Images Quotes Video Clips Key Findings Product Images Quotes Video Clips Data Viz
- 56. Method Method Optimizing for Timeand Attention Report Building Blocks Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Quotes Key Findings Product Images Quotes Video Clips Key Findings Product Images Quotes Video Clips Data Viz Severity Ratings Session Metrics Move to Appendix Move to Debrief & Preview Exclude Severity Ratings
- 57. Optimizing for Timeand Attention Report Building Blocks Detailed Findings Recos Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Key Findings Product Images Key Findings Product Images Stakeholder Attention Researcher Time Detailed Findings Detailed Findings Product Images Detailed Findings Product Images Quotes Detailed Findings Product Images Quotes Video Clips Detailed Findings Product Images Quotes Video Clips Data Viz Quotes Quotes Video Clips Quotes Video Clips Data Viz Quotes Quotes Video Clips Quotes Video Clips Data Viz
- 58. Optimizing for Timeand Attention Report Building Blocks 58 Finding = Behavior Why did they do that? How did they react? What happened next? + Cause + Mindset + Impact
- 59. Optimizing for Timeand Attention Story Arcs 59
- 60. Optimizing for Timeand Attention Progressive Specificity Navigation to Screen Screen Layout Individual Screen Elements Call to Action Next Screen >
- 61. Optimizing for Timeand Attention Should researchers make recommendations? No Yes Recos
- 62. Optimizing for Timeand Attention Should researchers make recommendations? 62 “That can create a silo effect and potentially, antagonism. Research has come here to tell you all the things that are wrong and all the things you should have done under the guise of ‘recommendations’.” “You don't tell me what to do and I won't tell you what to do because obviously you're a professional who has skills and knowledge that I don’t.” No Yes
- 63. Optimizing for Timeand Attention Should researchers make recommendations? 63 No Yes “Some product managers were like, who do you think you are, trying to drive these next steps. That was when I was new. Now, I’m like, hey, we're having next steps meeting from the research and we're going to all discuss together what we're doing next.”
- 64. Optimizing for Timeand Attention Should researchers make recommendations? 64 “Recommendations are a key part of what we do. Why be the one who holds all of the knowledge on the findings and not tell people what they should, ideas for things they should do about that?” No Yes
- 65. Optimizing for Timeand Attention Should researchers make recommendations? 65 “I usually just do it because I need it to make the next steps in an iteration.” No Yes
- 66.
- 67. The greater thedistance between the researcher and the stakeholders, the more formal results sharing tends to be. Choose your reporting approach in part based on whether you work in an integrated, service, or outsourced model. In Summary 1. Distance demands documentation.
- 68. Gaining buy-in forUX research results is more about a collaborative process than the artifacts produced. Loop in stakeholders early and often. In Summary 2. Process outperforms artifacts.
- 69. The best wayto package results depends on the researcher’s available time and the stakeholders’ available attention. Choose your components wisely, and learn how to lighten their load on everyone. In Summary 3. Optimize time and attention.
- 70. In Summary The Reportis Dead!
- 71. In Summary The Reportis Dead! Long Live the Report! (and also collaborate along the way…)
- 72.