SlideShare a Scribd company logo

The Report is Dead, Long Live the Report ! Communicating Usability Research Findings for Maximum Impact

Centralis
Centralis

The best way to improve products is to have people use them, but UX researchers struggle to share what they’ve learned in a way that has immediate and long-lasting impact. How do we keep the design process moving while grounding it thoroughly in research? This talk presents evidence for and against reports, and explores characteristics of reports that make them more and less successful at effecting change. We describe where approaches like debriefs, co-design, and video have succeeded and fallen short. Based on survey data from UX practitioners and experiences in the field, we address these questions: Is it worth it to write a report? Are there quicker, more engaging alternatives? What makes a compelling report? How do we make usability research usable? We offer a framework for choosing the best reporting approach, and share best practices for determining what to communicate, and how.

1 of 72
The Report is Dead, Long Live the Report!
How to Communicate Usability Research Findings
for Maximum Impact
Kathi Kaiser
Centralis
@kathikaiser
www.linkedin.com/in/kathikaiser
• Co-founder of Centralis
• 20+ years making products easier, mostly
through user research
• Lots of training/mentoring of new UXers
along the way
• Observer and creator of various reporting
approaches
Hello!
Hi! I’m Kathi Kaiser.
Back in the day…
When I began in UX in the late 1990s, few people knew what UX research was.
Nowadays…
In 25 years, UX researchers have grown in strength and numbers.
And so…
Given our range of approaches, we should think critically about what we do, and why.
• How do we share research results?
• What seems to work best?
• What challenges are we facing?
• How should we adapt to different contexts?
• How can we get beyond “It depends…”?
We researched…
We solicited input from ~100 UX practitioners, via two surveys and follow-up interviews.

Recommended

User Experience Design + Agile: The Good, The Bad, and the Ugly
User Experience Design + Agile: The Good, The Bad, and the UglyUser Experience Design + Agile: The Good, The Bad, and the Ugly
User Experience Design + Agile: The Good, The Bad, and the UglyJoshua Randall
 
Rapid Product Design in the Wild, Agile 2013
Rapid Product Design in the Wild, Agile 2013Rapid Product Design in the Wild, Agile 2013
Rapid Product Design in the Wild, Agile 2013Michele Ide-Smith
 
Basics of-usability-testing
Basics of-usability-testingBasics of-usability-testing
Basics of-usability-testingWBC Software Lab
 
ux_standards_slides
ux_standards_slidesux_standards_slides
ux_standards_slidesWarren Tsang
 

More Related Content

Similar to The Report is Dead, Long Live the Report ! Communicating Usability Research Findings for Maximum Impact

User Experience Design: an Overview
User Experience Design: an OverviewUser Experience Design: an Overview
User Experience Design: an OverviewJulie Grundy
 
Becoming A User Advocate
Becoming A User AdvocateBecoming A User Advocate
Becoming A User AdvocateKarl Kaufmann
 
Scaling Product Thinking with SAFe - The Secret Sauce for Meaningful Product ...
Scaling Product Thinking with SAFe - The Secret Sauce for Meaningful Product ...Scaling Product Thinking with SAFe - The Secret Sauce for Meaningful Product ...
Scaling Product Thinking with SAFe - The Secret Sauce for Meaningful Product ...Cprime
 
Usability Testing for Qualitative Researchers - QRCA NYC Chapter event
Usability Testing for Qualitative Researchers - QRCA NYC Chapter eventUsability Testing for Qualitative Researchers - QRCA NYC Chapter event
Usability Testing for Qualitative Researchers - QRCA NYC Chapter eventKay Aubrey
 
Ux matters2016-final
Ux matters2016-finalUx matters2016-final
Ux matters2016-finalNada Cbo
 
Building a UX Process at Salesforce that Promotes Focus and Creativity
Building a UX Process at Salesforce that Promotes Focus and CreativityBuilding a UX Process at Salesforce that Promotes Focus and Creativity
Building a UX Process at Salesforce that Promotes Focus and Creativityuxpin
 
Conducting User Research
Conducting User ResearchConducting User Research
Conducting User ResearchJeremy Horn
 
Designing user experience (ux) for digital products
Designing user experience (ux) for digital productsDesigning user experience (ux) for digital products
Designing user experience (ux) for digital productsVijay Morampudi
 
Embedding service design and product thinking at Cancer Research UK | Heads o...
Embedding service design and product thinking at Cancer Research UK | Heads o...Embedding service design and product thinking at Cancer Research UK | Heads o...
Embedding service design and product thinking at Cancer Research UK | Heads o...CharityComms
 
Agile User Experience Design
Agile User Experience DesignAgile User Experience Design
Agile User Experience DesignAlla Zollers
 
Product Design & Development Process By- Achia Nila
Product Design & Development Process  By- Achia NilaProduct Design & Development Process  By- Achia Nila
Product Design & Development Process By- Achia NilaAchia Nila
 
Lean Business Analysis and UX Runway - Natalie Warnert
Lean Business Analysis and UX Runway - Natalie WarnertLean Business Analysis and UX Runway - Natalie Warnert
Lean Business Analysis and UX Runway - Natalie WarnertNatalie Warnert
 
Lean Business Analysis and UX Runway: Managing Value by Reducing Waste (Natal...
Lean Business Analysis and UX Runway: Managing Value by Reducing Waste (Natal...Lean Business Analysis and UX Runway: Managing Value by Reducing Waste (Natal...
Lean Business Analysis and UX Runway: Managing Value by Reducing Waste (Natal...IT Arena
 
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden InsightsUXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden InsightsUXPA International
 

Similar to The Report is Dead, Long Live the Report ! Communicating Usability Research Findings for Maximum Impact (20)

User Experience Design: an Overview
User Experience Design: an OverviewUser Experience Design: an Overview
User Experience Design: an Overview
 
UX in an Agile World
UX in an Agile WorldUX in an Agile World
UX in an Agile World
 
What ux is
What ux isWhat ux is
What ux is
 
Becoming A User Advocate
Becoming A User AdvocateBecoming A User Advocate
Becoming A User Advocate
 
Scaling Product Thinking with SAFe - The Secret Sauce for Meaningful Product ...
Scaling Product Thinking with SAFe - The Secret Sauce for Meaningful Product ...Scaling Product Thinking with SAFe - The Secret Sauce for Meaningful Product ...
Scaling Product Thinking with SAFe - The Secret Sauce for Meaningful Product ...
 
What's Next For UX?
What's Next For UX?What's Next For UX?
What's Next For UX?
 
Jan Moons at WUD16
Jan Moons at WUD16Jan Moons at WUD16
Jan Moons at WUD16
 
Usability Testing for Qualitative Researchers - QRCA NYC Chapter event
Usability Testing for Qualitative Researchers - QRCA NYC Chapter eventUsability Testing for Qualitative Researchers - QRCA NYC Chapter event
Usability Testing for Qualitative Researchers - QRCA NYC Chapter event
 
Ux matters2016-final
Ux matters2016-finalUx matters2016-final
Ux matters2016-final
 
Building a UX Process at Salesforce that Promotes Focus and Creativity
Building a UX Process at Salesforce that Promotes Focus and CreativityBuilding a UX Process at Salesforce that Promotes Focus and Creativity
Building a UX Process at Salesforce that Promotes Focus and Creativity
 
Conducting User Research
Conducting User ResearchConducting User Research
Conducting User Research
 
UX @ Harvard's IQSS (Elizabeth Quigley)
UX @ Harvard's IQSS (Elizabeth Quigley)UX @ Harvard's IQSS (Elizabeth Quigley)
UX @ Harvard's IQSS (Elizabeth Quigley)
 
Designing user experience (ux) for digital products
Designing user experience (ux) for digital productsDesigning user experience (ux) for digital products
Designing user experience (ux) for digital products
 
Embedding service design and product thinking at Cancer Research UK | Heads o...
Embedding service design and product thinking at Cancer Research UK | Heads o...Embedding service design and product thinking at Cancer Research UK | Heads o...
Embedding service design and product thinking at Cancer Research UK | Heads o...
 
Agile User Experience Design
Agile User Experience DesignAgile User Experience Design
Agile User Experience Design
 
Product Design & Development Process By- Achia Nila
Product Design & Development Process  By- Achia NilaProduct Design & Development Process  By- Achia Nila
Product Design & Development Process By- Achia Nila
 
Lean Business Analysis and UX Runway - Natalie Warnert
Lean Business Analysis and UX Runway - Natalie WarnertLean Business Analysis and UX Runway - Natalie Warnert
Lean Business Analysis and UX Runway - Natalie Warnert
 
Lean Business Analysis and UX Runway: Managing Value by Reducing Waste (Natal...
Lean Business Analysis and UX Runway: Managing Value by Reducing Waste (Natal...Lean Business Analysis and UX Runway: Managing Value by Reducing Waste (Natal...
Lean Business Analysis and UX Runway: Managing Value by Reducing Waste (Natal...
 
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden InsightsUXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden Insights
 
Os Leung
Os LeungOs Leung
Os Leung
 

More from Centralis

Using Competitors To Do Better UX Research - UXPA 2018
Using Competitors To Do Better UX Research - UXPA 2018Using Competitors To Do Better UX Research - UXPA 2018
Using Competitors To Do Better UX Research - UXPA 2018Centralis
 
How Designers Can Make the World a Happier Place
How Designers Can Make the World a Happier PlaceHow Designers Can Make the World a Happier Place
How Designers Can Make the World a Happier PlaceCentralis
 
How to Design a Company
How to Design a CompanyHow to Design a Company
How to Design a CompanyCentralis
 
Museums, Tech, and UX: The Future of the Museum Experience
Museums, Tech, and UX: The Future of the Museum ExperienceMuseums, Tech, and UX: The Future of the Museum Experience
Museums, Tech, and UX: The Future of the Museum ExperienceCentralis
 
Deconstructing Happiness: Where Happy Endings Begin
Deconstructing Happiness: Where Happy Endings BeginDeconstructing Happiness: Where Happy Endings Begin
Deconstructing Happiness: Where Happy Endings BeginCentralis
 
Usability in the Gallery: Missing Methods in the Museum Evaluator's Toolkit
Usability in the Gallery: Missing Methods in the Museum Evaluator's ToolkitUsability in the Gallery: Missing Methods in the Museum Evaluator's Toolkit
Usability in the Gallery: Missing Methods in the Museum Evaluator's ToolkitCentralis
 
Red Herrings: Debunking the Pop Psychology of Color (UXPA 2014 Elizabeth Allen)
Red Herrings: Debunking the Pop Psychology of Color (UXPA 2014 Elizabeth Allen)Red Herrings: Debunking the Pop Psychology of Color (UXPA 2014 Elizabeth Allen)
Red Herrings: Debunking the Pop Psychology of Color (UXPA 2014 Elizabeth Allen)Centralis
 
If It Can Go Wrong, It Will! How to Bulletproof Your User Research
If It Can Go Wrong, It Will! How to Bulletproof Your User ResearchIf It Can Go Wrong, It Will! How to Bulletproof Your User Research
If It Can Go Wrong, It Will! How to Bulletproof Your User ResearchCentralis
 
User Experience & Visitor Experience: How to Improve Museum Apps
User Experience & Visitor Experience: How to Improve Museum AppsUser Experience & Visitor Experience: How to Improve Museum Apps
User Experience & Visitor Experience: How to Improve Museum AppsCentralis
 
Adapting Focus Groups for User Experience (UX) Research
Adapting Focus Groups for User Experience (UX) ResearchAdapting Focus Groups for User Experience (UX) Research
Adapting Focus Groups for User Experience (UX) ResearchCentralis
 
Integrating UX into Voice of the Customer Programs
Integrating UX into Voice of the Customer ProgramsIntegrating UX into Voice of the Customer Programs
Integrating UX into Voice of the Customer ProgramsCentralis
 
Is User Experience a Cult?
Is User Experience a Cult?Is User Experience a Cult?
Is User Experience a Cult?Centralis
 

More from Centralis (12)

Using Competitors To Do Better UX Research - UXPA 2018
Using Competitors To Do Better UX Research - UXPA 2018Using Competitors To Do Better UX Research - UXPA 2018
Using Competitors To Do Better UX Research - UXPA 2018
 
How Designers Can Make the World a Happier Place
How Designers Can Make the World a Happier PlaceHow Designers Can Make the World a Happier Place
How Designers Can Make the World a Happier Place
 
How to Design a Company
How to Design a CompanyHow to Design a Company
How to Design a Company
 
Museums, Tech, and UX: The Future of the Museum Experience
Museums, Tech, and UX: The Future of the Museum ExperienceMuseums, Tech, and UX: The Future of the Museum Experience
Museums, Tech, and UX: The Future of the Museum Experience
 
Deconstructing Happiness: Where Happy Endings Begin
Deconstructing Happiness: Where Happy Endings BeginDeconstructing Happiness: Where Happy Endings Begin
Deconstructing Happiness: Where Happy Endings Begin
 
Usability in the Gallery: Missing Methods in the Museum Evaluator's Toolkit
Usability in the Gallery: Missing Methods in the Museum Evaluator's ToolkitUsability in the Gallery: Missing Methods in the Museum Evaluator's Toolkit
Usability in the Gallery: Missing Methods in the Museum Evaluator's Toolkit
 
Red Herrings: Debunking the Pop Psychology of Color (UXPA 2014 Elizabeth Allen)
Red Herrings: Debunking the Pop Psychology of Color (UXPA 2014 Elizabeth Allen)Red Herrings: Debunking the Pop Psychology of Color (UXPA 2014 Elizabeth Allen)
Red Herrings: Debunking the Pop Psychology of Color (UXPA 2014 Elizabeth Allen)
 
If It Can Go Wrong, It Will! How to Bulletproof Your User Research
If It Can Go Wrong, It Will! How to Bulletproof Your User ResearchIf It Can Go Wrong, It Will! How to Bulletproof Your User Research
If It Can Go Wrong, It Will! How to Bulletproof Your User Research
 
User Experience & Visitor Experience: How to Improve Museum Apps
User Experience & Visitor Experience: How to Improve Museum AppsUser Experience & Visitor Experience: How to Improve Museum Apps
User Experience & Visitor Experience: How to Improve Museum Apps
 
Adapting Focus Groups for User Experience (UX) Research
Adapting Focus Groups for User Experience (UX) ResearchAdapting Focus Groups for User Experience (UX) Research
Adapting Focus Groups for User Experience (UX) Research
 
Integrating UX into Voice of the Customer Programs
Integrating UX into Voice of the Customer ProgramsIntegrating UX into Voice of the Customer Programs
Integrating UX into Voice of the Customer Programs
 
Is User Experience a Cult?
Is User Experience a Cult?Is User Experience a Cult?
Is User Experience a Cult?
 

Recently uploaded

Copy of VSST102 Digital Portfolio.pdffff
Copy of VSST102 Digital Portfolio.pdffffCopy of VSST102 Digital Portfolio.pdffff
Copy of VSST102 Digital Portfolio.pdffffschell
 
Heathers Symbols for Gerald Freedman Theatre.pdf
Heathers Symbols for Gerald Freedman Theatre.pdfHeathers Symbols for Gerald Freedman Theatre.pdf
Heathers Symbols for Gerald Freedman Theatre.pdfLizShek
 
impact of packaging on consumer buying behaviour project report pdf.pdf
impact of packaging on consumer buying behaviour project  report pdf.pdfimpact of packaging on consumer buying behaviour project  report pdf.pdf
impact of packaging on consumer buying behaviour project report pdf.pdfSanjivanisShirodkar
 
VSST103 - Digital Portfolio.pdf character
VSST103 - Digital Portfolio.pdf characterVSST103 - Digital Portfolio.pdf character
VSST103 - Digital Portfolio.pdf characterschell
 
Friendship Hospital _ Case Study-MaHi.pptx
Friendship Hospital _ Case Study-MaHi.pptxFriendship Hospital _ Case Study-MaHi.pptx
Friendship Hospital _ Case Study-MaHi.pptxTafshirul Alam
 
Spoleto Festival23 Hang and Cable Layouts.pdf
Spoleto Festival23 Hang and Cable Layouts.pdfSpoleto Festival23 Hang and Cable Layouts.pdf
Spoleto Festival23 Hang and Cable Layouts.pdfLizShek
 
urban design case study of old city Jodhpur
urban design case study of  old city Jodhpururban design case study of  old city Jodhpur
urban design case study of old city JodhpurSreyaseeNath1
 
Communicative Grammar English .pnnnnkkptx
Communicative Grammar English .pnnnnkkptxCommunicative Grammar English .pnnnnkkptx
Communicative Grammar English .pnnnnkkptxRawalRafiqLeghari
 
Ordering Principles in Architecture-MaHi.pptx
Ordering Principles in Architecture-MaHi.pptxOrdering Principles in Architecture-MaHi.pptx
Ordering Principles in Architecture-MaHi.pptxTafshirul Alam
 
Amadeus Hang and Cable Layouts UNCSA.pdf
Amadeus Hang and Cable Layouts UNCSA.pdfAmadeus Hang and Cable Layouts UNCSA.pdf
Amadeus Hang and Cable Layouts UNCSA.pdfLizShek
 
acervus process book no cover--character
acervus process book no cover--characteracervus process book no cover--character
acervus process book no cover--characterschell
 
Mother Courage and Her Children Layouts.pdf
Mother Courage and Her Children Layouts.pdfMother Courage and Her Children Layouts.pdf
Mother Courage and Her Children Layouts.pdfLizShek
 
Bashundhara Residential _ Analysis and Case Study-MaHi.pptx
Bashundhara Residential _ Analysis and Case Study-MaHi.pptxBashundhara Residential _ Analysis and Case Study-MaHi.pptx
Bashundhara Residential _ Analysis and Case Study-MaHi.pptxTafshirul Alam
 
Interior Landscape and Landscape Features Extraction
Interior Landscape and Landscape Features ExtractionInterior Landscape and Landscape Features Extraction
Interior Landscape and Landscape Features ExtractionNityaSharma77
 
Studio Surfaced - Textile Design Studio based out of Delhi NCR
Studio Surfaced - Textile Design Studio based out of Delhi NCRStudio Surfaced - Textile Design Studio based out of Delhi NCR
Studio Surfaced - Textile Design Studio based out of Delhi NCRSurfaced - Designs By Meera Narula
 
sustainable-interior-design-strategy-with-case-study_TANIA-BERA.ppt
sustainable-interior-design-strategy-with-case-study_TANIA-BERA.pptsustainable-interior-design-strategy-with-case-study_TANIA-BERA.ppt
sustainable-interior-design-strategy-with-case-study_TANIA-BERA.pptAr. TANIA BERA
 
homescreen revamp homescreen revamphomescreen revamp
homescreen revamp homescreen revamphomescreen revamphomescreen revamp homescreen revamphomescreen revamp
homescreen revamp homescreen revamphomescreen revampvidya934261
 
SECMOL School In Leh- A Role Model of Vernacular , Passive and Sustainable H...
SECMOL School In Leh- A  Role Model of Vernacular , Passive and Sustainable H...SECMOL School In Leh- A  Role Model of Vernacular , Passive and Sustainable H...
SECMOL School In Leh- A Role Model of Vernacular , Passive and Sustainable H...JIT KUMAR GUPTA
 

Recently uploaded (20)

Copy of VSST102 Digital Portfolio.pdffff
Copy of VSST102 Digital Portfolio.pdffffCopy of VSST102 Digital Portfolio.pdffff
Copy of VSST102 Digital Portfolio.pdffff
 
Bring Your Office Interiors Back to Life The Enchantment of Personalized Wall...
Bring Your Office Interiors Back to Life The Enchantment of Personalized Wall...Bring Your Office Interiors Back to Life The Enchantment of Personalized Wall...
Bring Your Office Interiors Back to Life The Enchantment of Personalized Wall...
 
Heathers Symbols for Gerald Freedman Theatre.pdf
Heathers Symbols for Gerald Freedman Theatre.pdfHeathers Symbols for Gerald Freedman Theatre.pdf
Heathers Symbols for Gerald Freedman Theatre.pdf
 
impact of packaging on consumer buying behaviour project report pdf.pdf
impact of packaging on consumer buying behaviour project  report pdf.pdfimpact of packaging on consumer buying behaviour project  report pdf.pdf
impact of packaging on consumer buying behaviour project report pdf.pdf
 
VSST103 - Digital Portfolio.pdf character
VSST103 - Digital Portfolio.pdf characterVSST103 - Digital Portfolio.pdf character
VSST103 - Digital Portfolio.pdf character
 
Friendship Hospital _ Case Study-MaHi.pptx
Friendship Hospital _ Case Study-MaHi.pptxFriendship Hospital _ Case Study-MaHi.pptx
Friendship Hospital _ Case Study-MaHi.pptx
 
Spoleto Festival23 Hang and Cable Layouts.pdf
Spoleto Festival23 Hang and Cable Layouts.pdfSpoleto Festival23 Hang and Cable Layouts.pdf
Spoleto Festival23 Hang and Cable Layouts.pdf
 
urban design case study of old city Jodhpur
urban design case study of  old city Jodhpururban design case study of  old city Jodhpur
urban design case study of old city Jodhpur
 
Communicative Grammar English .pnnnnkkptx
Communicative Grammar English .pnnnnkkptxCommunicative Grammar English .pnnnnkkptx
Communicative Grammar English .pnnnnkkptx
 
Ordering Principles in Architecture-MaHi.pptx
Ordering Principles in Architecture-MaHi.pptxOrdering Principles in Architecture-MaHi.pptx
Ordering Principles in Architecture-MaHi.pptx
 
Amadeus Hang and Cable Layouts UNCSA.pdf
Amadeus Hang and Cable Layouts UNCSA.pdfAmadeus Hang and Cable Layouts UNCSA.pdf
Amadeus Hang and Cable Layouts UNCSA.pdf
 
acervus process book no cover--character
acervus process book no cover--characteracervus process book no cover--character
acervus process book no cover--character
 
Maximize Your Brand Impact with a Box Truck Wrap Design
Maximize Your Brand Impact with a Box Truck Wrap DesignMaximize Your Brand Impact with a Box Truck Wrap Design
Maximize Your Brand Impact with a Box Truck Wrap Design
 
Mother Courage and Her Children Layouts.pdf
Mother Courage and Her Children Layouts.pdfMother Courage and Her Children Layouts.pdf
Mother Courage and Her Children Layouts.pdf
 
Bashundhara Residential _ Analysis and Case Study-MaHi.pptx
Bashundhara Residential _ Analysis and Case Study-MaHi.pptxBashundhara Residential _ Analysis and Case Study-MaHi.pptx
Bashundhara Residential _ Analysis and Case Study-MaHi.pptx
 
Interior Landscape and Landscape Features Extraction
Interior Landscape and Landscape Features ExtractionInterior Landscape and Landscape Features Extraction
Interior Landscape and Landscape Features Extraction
 
Studio Surfaced - Textile Design Studio based out of Delhi NCR
Studio Surfaced - Textile Design Studio based out of Delhi NCRStudio Surfaced - Textile Design Studio based out of Delhi NCR
Studio Surfaced - Textile Design Studio based out of Delhi NCR
 
sustainable-interior-design-strategy-with-case-study_TANIA-BERA.ppt
sustainable-interior-design-strategy-with-case-study_TANIA-BERA.pptsustainable-interior-design-strategy-with-case-study_TANIA-BERA.ppt
sustainable-interior-design-strategy-with-case-study_TANIA-BERA.ppt
 
homescreen revamp homescreen revamphomescreen revamp
homescreen revamp homescreen revamphomescreen revamphomescreen revamp homescreen revamphomescreen revamp
homescreen revamp homescreen revamphomescreen revamp
 
SECMOL School In Leh- A Role Model of Vernacular , Passive and Sustainable H...
SECMOL School In Leh- A  Role Model of Vernacular , Passive and Sustainable H...SECMOL School In Leh- A  Role Model of Vernacular , Passive and Sustainable H...
SECMOL School In Leh- A Role Model of Vernacular , Passive and Sustainable H...
 

The Report is Dead, Long Live the Report ! Communicating Usability Research Findings for Maximum Impact

  • 1. The Report is Dead, Long Live the Report! How to Communicate Usability Research Findings for Maximum Impact Kathi Kaiser Centralis @kathikaiser www.linkedin.com/in/kathikaiser
  • 2. • Co-founder of Centralis • 20+ years making products easier, mostly through user research • Lots of training/mentoring of new UXers along the way • Observer and creator of various reporting approaches Hello! Hi! I’m Kathi Kaiser.
  • 3. Back in the day… When I began in UX in the late 1990s, few people knew what UX research was.
  • 4. Nowadays… In 25 years, UX researchers have grown in strength and numbers.
  • 5. And so… Given our range of approaches, we should think critically about what we do, and why. • How do we share research results? • What seems to work best? • What challenges are we facing? • How should we adapt to different contexts? • How can we get beyond “It depends…”?
  • 6. We researched… We solicited input from ~100 UX practitioners, via two surveys and follow-up interviews.
  • 7. The greater the distance between the researcher and the stakeholders, the more formal results sharing tends to be. And we learned... 1. Distance demands documentation.
  • 8. Gaining buy-in for UX research results is more about a collaborative process than the artifacts produced. And we learned... 2. Process outperforms artifacts.
  • 9. The best way to package results depends on the researcher’s available time and the stakeholders’ available attention. And we learned... 3. Optimize time and attention.
  • 11. Distance Demands Documentation Researchers share results in a wide range of ways, usually by creating an “artifact”.
  • 12. Distance Demands Documentation Product Execs Product Owner Designers Devs Imagine a sphere…
  • 13. Employee Vendor Not on the Product Team On the Product Team Distance Demands Documentation Product Team Execs Product Owner Designers Devs Where do UX researchers fit in?
  • 14. Where do UX researchers fit in? The Integrated Model (Employee) Product Team Execs Product Owner Designers Devs Product Team Execs Product Owner Embedded UX Researcher Designers Devs Employee Vendor Not on the Product Team On the Product Team Distance Demands Documentation
  • 15. Product Execs Product Owner Designers Devs Employee Vendor Not on the Product Team On the Product Team Product Execs Product Owner Embedded UX Researcher Designers Devs Distance Demands Documentation The Integrated Model (Contractor) Product Team Execs Embedded UX Researcher Product Owner Designers Devs
  • 16. Product Team Execs Product Owner Designers Devs Employee Vendor Not on the Product Team On the Product Team Distance Demands Documentation The Service Model In-House UX Research Consultant
  • 17. Product Team Execs Product Owner Designers Devs Employee Vendor Not on the Product Team On the Product Team Distance Demands Documentation The Outsourced Model External UX Research Consultant
  • 18. Product Team Execs Product Owner Designers Devs Employee Vendor Not on the Product Team On the Product Team Distance Demands Documentation The Outsourced Model In-House UX Research Consultant External UX Research Consultant
  • 19. Product Team Execs Product Owner Designers Devs Employee Vendor Not on the Product Team On the Product Team Distance Demands Documentation Research must span different distances to reach the stakeholders in each model. Product Team Execs Product Owner Integrated Model Designers Devs Service Model Outsourced Model
  • 20. Distance Demands Documentation Scope of Research Question Range of Audience Text vs. Visual “I'll write up, here's what I think the main findings were in basically a one- page kind of thing.” Time to Create Shelf Life “Everybody knows that producing written documents in most organizational cultures is not the way to affect change.” “Usually, we're bumping up at the time when the product manager has to hand [the product] over to engineering for development.” Sharing in the Integrated Model “It's mostly just the bare bones: what did we observe, what did we learn from it, what do we need to do now.”
  • 21. Distance Demands Documentation Scope of Research Question Range of Audience Text vs. Visual “I sit within our Consumer Insights Department, which is an overarching research center of excellence – that area supports the entire enterprise.” Time to Create Shelf Life “There's like 15 years of research…some of the insights, drivers, motivations, those things are still the same. And so they're still relevant. And so it's helpful.” Sharing in the Service Model “The way we write our reports is having that executive summary up front, because we are thinking about the executives… tell me what you're going to do. What needs to be fixed and why?
  • 22. Distance Demands Documentation Scope of Research Question Range of Audience Text vs. Visual A: “Maybe less than 20 hours for a straightforward usability study…a big user research project, like could be like 40 or 50 or more hours.” Time to Create Shelf Life “I always walk through the whole report. So usually I book an hour, which is not always enough.” “I had a client reach out to me a couple of months ago who was like, we found your report from three years ago and it's incredible…we'd like you to do a new project with us.” Sharing in the Outsourced Model Q: “How long does it take you to write a usability report?” A: “Oh, forever. It takes forever.”
  • 23. Distance Demands Documentation Scope of Research Question Range of Audience Text vs. Visual Time to Create Shelf Life Sharing in the Outsourced Model Scope of Research Question Range of Audience Text vs. Visual Time to Create Shelf Life Sharing in the Service Model Scope of Research Question Range of Audience Text vs. Visual Time to Create Shelf Life Sharing in the Integrated Model
  • 24. Distance Demands Documentation “You want your stakeholders to DO something with your report; otherwise, your report is useless. Why are you doing this work? It’s because you want change to happen. The way to make change happen is to involve the stakeholders from the beginning.”
  • 26. “Making the results easy to comprehend and showing visual examples” “Including an exec summary of key insights/ recommendations so they can quickly grasp the "greatest hits".” “Sharing participant quotes/anecdotes from testing” “…video clips showing participants failing (and succeeding).” Process Over Artifacts Q: What have you found most effective for sharing usability findings?(2020) Q: What have you found most effective for getting buy-in to the results of usability testing?(2023) “Building relationships and asking questions about the plans and roadmap” “Require that stakeholders observe usability testing.” “Collaboration on the goals and planning of the test” “Review the test plan with PM and UX and Engineering key stakeholders” “Including responsible stakeholders in planning and analysis.” “Having stakeholders watch… take notes on a shared sticky note space and ‘affinitizing’ the findings in a group workshop.” “Give stakeholders the opportunity to discuss/ask questions” “Cross-functional, collaborative analysis.” “Socializing findings ahead of the formal report also helps, as do clear recommendations.”
  • 27. Process Over Artifacts A Common Usability Research Process
  • 28. Process Over Artifacts Level of Involvement R Level of Involvement R R R R
  • 29. Process Over Artifacts Level of Involvement: Integrated Model R Level of Involvement R R R R S S S S S Integrated Model “I know the stakeholders I'm working with and I know what problem we're focused on at that time.”
  • 30. Process Over Artifacts Level of Involvement: Service Model R Level of Involvement R R R R S S S S S “If we're going to do research for you, you as a business owner need to participate. You cannot just wait for the report to be thrown over the wall.” Service Model
  • 31. Process Over Artifacts Level of Involvement: Outsourced Model R Level of Involvement R R R R S S S S S Outsourced Model “And then there's, you know, an analysis phase…usually it's just me, where I try to make sense of all the stuff that we saw in the usability testing.”
  • 32. Process Over Artifacts Level of Involvement R Level of Involvement R R R R S S S S S S S S S S S S S S S Integrated Model Service Model Outsourced Model
  • 33. R S S Process Over Artifacts Boosting Stakeholder Involvement Level of Involvement R R R S S S S S S S S S S R I S S R I S S R S S S Integrated Model Service Model Outsourced Model
  • 34. R S S Process Over Artifacts Boosting Stakeholder Involvement Level of Involvement R R R S S S S S S S S S S R I S S R I S S R S S S Integrated Model Service Model Outsourced Model ?
  • 35. Process Over Artifacts 35 • The debrief session told us what we need to know • No one has time to write a report • No one will read a report • The design has already moved on • The debrief is for the team; the report is for the executives • The report is our memory • The report is our roadmap • The report is our product The Report is Dead! Long Live the Report! Integrated Model Service Model Outsourced Model
  • 36. Optimize your time and their attention.
  • 37. Optimizing for Time and Attention A Taxonomy of Reports (Cooley & Barnum, 2019) Cooley, D., Barnum, C. (2019). How Do Other People Do It? A Comparative Review of Usability Study Reports. User Experience Magazine, 19(1). Retrieved from https://uxpamagazine.org/how-do-other-people-do-it-a-comparative-review-of-usability-study-reports/ Visuals from DialogDesign: https://www.dialogdesign.dk/cue-studies/ ‘Traditional’ Reports ‘Visual’ Reports ‘Lean’ Reports
  • 38. Optimizing for Time and Attention Report Building Blocks 38 Key Findings Product Images Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method What we did How to illustrate it What to do about it What we learned
  • 39. Optimizing for Time and Attention Report Building Blocks 39 Key Findings Detailed Findings Severity Ratings Recos Method What we did How to illustrate it What to do about it Traditional Focus What we learned Product Images Quotes Video Clips Data Viz Session Metrics
  • 40. Optimizing for Time and Attention Report Building Blocks 40 Key Findings Detailed Findings Severity Ratings Recos Method What we did How to illustrate it What to do about it Visual Focus What we learned Product Images Quotes Video Clips Data Viz Session Metrics
  • 41. Lean Focus Optimizing for Time and Attention Report Building Blocks 41 Key Findings Detailed Findings Severity Ratings Recos Method What we did How to illustrate it What to do about it Lean Focus What we learned Product Images Quotes Video Clips Data Viz Session Metrics
  • 42. Optimizing for Time and Attention Report Building Blocks Key Findings Product Images Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time
  • 43. Optimizing for Time and Attention The Attention Paradox: An Exercise 43 N P R F D R C I A P D Q B M W J F K
  • 44. Optimizing for Time and Attention The Attention Paradox: An Exercise 44 N P R F D R C I A P D Q B M W J F K
  • 45. Optimizing for Time and Attention The Attention Paradox: An Exercise 45 N P R F D R C I A P D Q B M W J F K N P R F D R C IA P D Q B M W J F K
  • 46. Optimizing for Time and Attention The Attention Paradox 46 Our attention is limited. The less attention needed to process something, the more attention available for processing it. We capture more attention by requiring less attention.
  • 47. Optimizing for Time and Attention Report Building Blocks Key Findings Product Images Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time
  • 48. Optimizing for Time and Attention Report Building Blocks Product Images Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings
  • 49. Optimizing for Time and Attention The Elevator Pitch Game Claire Slattery, https://claireslatterycoaching.com/
  • 50. Optimizing for Time and Attention Report Building Blocks Quotes Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images
  • 51. Optimizing for Time and Attention Report Building Blocks Video Clips Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Quotes
  • 52. Optimizing for Time and Attention Otter.ai 52
  • 53. Optimizing for Time and Attention Report Building Blocks Data Viz Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Quotes Key Findings Product Images Quotes Video Clips
  • 54. Optimizing for Time and Attention Reduct.video 54
  • 55. Optimizing for Time and Attention Report Building Blocks Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Quotes Key Findings Product Images Quotes Video Clips Key Findings Product Images Quotes Video Clips Data Viz
  • 56. Method Method Optimizing for Time and Attention Report Building Blocks Detailed Findings Severity Ratings Session Metrics Recos Method Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Quotes Key Findings Product Images Quotes Video Clips Key Findings Product Images Quotes Video Clips Data Viz Severity Ratings Session Metrics Move to Appendix Move to Debrief & Preview Exclude Severity Ratings
  • 57. Optimizing for Time and Attention Report Building Blocks Detailed Findings Recos Stakeholder Attention Researcher Time Key Findings Key Findings Product Images Key Findings Product Images Key Findings Product Images Key Findings Product Images Stakeholder Attention Researcher Time Detailed Findings Detailed Findings Product Images Detailed Findings Product Images Quotes Detailed Findings Product Images Quotes Video Clips Detailed Findings Product Images Quotes Video Clips Data Viz Quotes Quotes Video Clips Quotes Video Clips Data Viz Quotes Quotes Video Clips Quotes Video Clips Data Viz
  • 58. Optimizing for Time and Attention Report Building Blocks 58 Finding = Behavior Why did they do that? How did they react? What happened next? + Cause + Mindset + Impact
  • 59. Optimizing for Time and Attention Story Arcs 59
  • 60. Optimizing for Time and Attention Progressive Specificity Navigation to Screen Screen Layout Individual Screen Elements Call to Action Next Screen >
  • 61. Optimizing for Time and Attention Should researchers make recommendations? No Yes Recos
  • 62. Optimizing for Time and Attention Should researchers make recommendations? 62 “That can create a silo effect and potentially, antagonism. Research has come here to tell you all the things that are wrong and all the things you should have done under the guise of ‘recommendations’.” “You don't tell me what to do and I won't tell you what to do because obviously you're a professional who has skills and knowledge that I don’t.” No Yes
  • 63. Optimizing for Time and Attention Should researchers make recommendations? 63 No Yes “Some product managers were like, who do you think you are, trying to drive these next steps. That was when I was new. Now, I’m like, hey, we're having next steps meeting from the research and we're going to all discuss together what we're doing next.”
  • 64. Optimizing for Time and Attention Should researchers make recommendations? 64 “Recommendations are a key part of what we do. Why be the one who holds all of the knowledge on the findings and not tell people what they should, ideas for things they should do about that?” No Yes
  • 65. Optimizing for Time and Attention Should researchers make recommendations? 65 “I usually just do it because I need it to make the next steps in an iteration.” No Yes
  • 67. The greater the distance between the researcher and the stakeholders, the more formal results sharing tends to be. Choose your reporting approach in part based on whether you work in an integrated, service, or outsourced model. In Summary 1. Distance demands documentation.
  • 68. Gaining buy-in for UX research results is more about a collaborative process than the artifacts produced. Loop in stakeholders early and often. In Summary 2. Process outperforms artifacts.
  • 69. The best way to package results depends on the researcher’s available time and the stakeholders’ available attention. Choose your components wisely, and learn how to lighten their load on everyone. In Summary 3. Optimize time and attention.
  • 71. In Summary The Report is Dead! Long Live the Report! (and also collaborate along the way…)