UXPA 2023 Poster: Atomic Research in Practice: Using a Feedback Repository to Make User-Driven Decisions

1. AtomicUXResearchinPractice:DevelopingaFeedbackRepositorytoMakeUser-DrivenDecisions
Authors:BoKim,AlexMarantz,AbbyOlivier,QingyanMa
MemorialSloanKetteringCancerCenter(MSK),NewYork,NY
Havequestions?
ContactBoatkimb2@mskcc.org
Background
To apply human-centered design in an agile software development setting, it is vital to incorporate user’s input during each
iteration. However, qualitative user feedback is often collected across multiple channels and projects, and static documentation of
research findings makes accessing and using the data outside the span cope of each project challenging. Atomic User
Experience(UX) Research, a concept developed by the Head of UX at WeWork, Tomer Sharon, proposes a systematic way of
aggregating, compartmentalizing, and managing research insights. Organizations can effectively employ the voice of the user to
influence every phase of the product development process by breaking down research insights into smaller ‘nuggets’ of information
and putting them in a unified, evolving repository. A user feedback repository developed at Memorial Sloan Kettering Cancer
Center(MSKCC)’s UX team exemplifies the process of applying Atomic UX Research into practice and evangelizing human-centered
design in a large academic medical center setting.
Research
Report in
PDFs and
decks
Ad hoc
emails from
stakeholders
One off
interviews Siloed
feedback
collection
WITHOUT ATOMIC UX RESEARCH with atomic ux research
Atomic unit of research evidence
Method
Data Collection and Aggregation
We compiled a comprehensive list of all channels that we receive qualitative
feedback from our users, such as customer support tickets, qualitative
interviews, internal feedback, and satisfaction surveys. We streamlined the
pipeline to consistently ingest incoming data into a single repository, with a
focus on the user satisfaction survey, which was the highest source of user
feedback. After aggregating the feedback within two products, we gradually
expanded to other products targeting the same user group.
Centralized user feedback
repository
CHANNEL X
CHANNEL Y
CHANNEL Z
Taxonomy Development
We developed an initial set of taxonomy coding the ingested data. In the early stage,
we used affinity mapping to categorize and consolidate the tags into tag groups.
During this process, we engaged partners from different disciplines, especially
Product, to test the taxonomy with real feedback. We continuously tested the
taxonomy with incoming data, iterated it, and developed a documentation to generate
a shared understanding of the taxonomy.
Qualitative
FEEDBACK
TAXONOMY
Test Test
Review & Iterate Review & Iterate
Synthesis and Application
The UX team synthesizes the obtained data into themes in order to identify
recurring patterns and opportunities for improvement. When there is a notable
trend in cross-departmental issues, we triage the feedback relating to those
departments, such as operations and engineering. We also monitor the volume
of themes and tags over time to measure the impact of our updates and quickly
resolve reported issues based on their priority and actionability.
Theme A Theme B Theme C Theme D
VOLUME
FIX on update on
Results
The feedback repository now has over 10,000 qualitative feedback collected from four products. A robust repository can have a
substantial influence to the product development process. First, it consolidates dispersed feedback into a single place. Rather than
searching for previous research reports, the UX and research team can simply search for and retrieve relevant data depending on
their research questions. Second, it supports measuring success. In addition to quantitative and analytical data, UX and Product
can employ qualitative metrics to examine the impact of each iteration and expand their understanding of user behavior. The UX
team was able to manage and develop the feedback repository with little reliance on other teams. This level of agency contrasts
with quantitative metrics, which often require the involvement of Engineering and Business Intelligence for implementation. Third,
it empowers the organization to validate and inform decisions using user input. The UX and Product team used the themes and
trends in feedback to create business cases and gauge roadmap’s alignment with user needs.
ConclusionandFutureDirection
A feedback repository leveraging Atomic UX is an operationalized application of the human-centered design process. It allows
users to directly influence the roadmap, prevents redundant research, and democratizes research insights by connecting
stakeholders and members who have little exposure to their end-users. A high volume of feedback may necessitate a specialized
resource, such as Design Ops or Research Ops, to annotate and analyze the data, which may be a limitation for smaller teams.
Machine learning models in natural language processing(NLP) can be a viable method for scaling and accelerating the feedback
synthesis process.
References
Ross, L., Sharon, T. “Atomic research: From reports to consumable insights.” 2020. dovetail.com/blog/atomic-research/