This talk was for Product School in October 2019. The audience was primarily product managers, so this talk focused on the research process and some education on that aspect with what to expect with the relationship with the UX researcher.
6. Phases of Research
Assess
Measure performance of a
product against itself or
competition.
Sample Methods:
● A/B Testing
● Usability Benchmarking
● Online assessments
● Surveys
● Analytics
Explore
Exploring who your users are,
including their behaviors,
motivations, goals, needs, and
challenges/problems are.
Sample Methods:
● Field studies
● Diary studies
● Interviews
● Data Mining
● Analytics
Evaluate
Evaluate designs, content,
organization, or flows to
optimize, reduce risk, and
increase usability.
Sample Methods:
● Card sorting
● Field studies
● Participatory design
● Usability testing
● Prototype testing
Reference: https://www.nngroup.com/articles/which-ux-research-methods/
7. Why & How
to Fix
How Many &
How Much
What People Do
What People Say
8. 1
2
3
Explore
Evaluate
Assess
Research Phases
1 2
3
1
1
When Research Phases Apply in Product Management Framework
2 2 3
3
3
Where research
phases fit in to the
Pragmatic
framework for
product
development.
Reference: https://www.pragmaticmarketing.com/framework
1 3
10. Define Research
Objectives
With key stakeholders, define
research objectives, goals, and
outcomes / deliverables.
Develop Research Plan
If there aren’t answers from previous
research, build a research plan with
appropriate methodology to get
desired outcomes / deliverables. Recruit & Schedule
Participants
After research plan is finalized, recruit
participants that meet the
requirements of the research (e.g,
current customers, potential users,
etc.). This can be a long phase
depending on complexity of recruit.
Analyze Data & Create
Deliverables
Spend time analyzing and
synthesizing the data. Spending time
with the data ensures a better
understanding in the outputted
insights. After analysis, create a
deliverable from the insights meeting
needs set out at the beginning of the
effort.
Execute Research
Complete research sessions with
informed consent, record (audio or
video), etc. depending on
methodology.
Simplified UX Research (UXR) Process
05
01
02 03
04
17. How to Foster
the BFF
Relationship as a
Product
Manager
● Prioritize what you want to learn
collaboratively with the UXR
● Let the UX Researcher guide you
● Participate / observe when able
● Use the learnings and insights in your work
18. How to Foster
the BFF
Relationship as a
UX Researcher
● Research the PM
● Help the PM by scoping a project down and
prioritize learnings to make a bigger impact
● Keep going back to “What do you want to
learn?”
● Involve the PM
In other words - how to build a great working relationship!
Imagine a world where Product managers knew what direction to head in, potential competitive advantages on the horizon, what their current users’ needs were, and how much their changes to the product have improved their customers’ experiences. All of this based on research and not on guesses.
Photo by Nadine Shaabana on Unsplash
This is where a UX Researcher can come in. A strategic partnership with UXR can help you build that type of world. The key is knowing how to leverage and work with the UXR in your organization. This is how you can become best friends and foster a great working relationship
I’d love to know who is in the audience - Raise your hand if you’re a UX Researcher. Raise your hand if you’re a PM. PMs - keep your hand raised if you have a dedicated UX Researcher at your organization
How many of you have a dedicated UXR at your organization?
For those of you who don’t, maybe this will inspire you to get a UXR as a function in your organization
Who am I to talk to you about this budding friendship.
I am the Senior Lead of UX Research at GoCanvas building out the research capability of the organization. I have a background in Human Factors Psychology from GMU, I have applied research skills in varying fidelities at a variety of organizations, I’ve had experience in government contracting, insurance, and tech startups.
In each of my roles, I have had to partner with key stakeholders, including product managers, to ensure that a product or service was successful. Through my experiences building out a research practice as the first researcher in an organization, I have created some processes to help PMs and UXRs collaborate well.
First I’d like to give you an overview of the different phases of research, sample of methods used, and how these phases apply to product management.
Explore
Who our users are including behaviors, motivations, goals, needs, and challenges/problems
Example methods that can be used in this phase
Evaluate
Evaluate designs, content, organization, or flows to optimize, reduce risk, and increase usability
This could range from evaluating current designs, content, flows, or organization to testing new ones
Assess
Measure performance of a product against itself or competition
See how things are performing once live, testing a new design in an A/B test that went well in usability testing, surveying the population to understand perceived satisfaction over time, etc.
https://www.nngroup.com/articles/which-ux-research-methods/
Here are some samples of methods that we have in a toolbox. Attitudinal to behavioral and qualitative and quantitative. This is not an exhaustive list, as people are coming up with creative methods all the time, but this is a framework to choose methods based on what you’re trying to learn.
https://www.nngroup.com/articles/which-ux-research-methods/
Now, here are examples of when the research phases can apply in the Pragmatic’s Product Management Framework, showing where you could leverage different types of research depending on the type of information that you’re looking to learn.
Now I’m going to go over a light version of the UX Research process that I use in my organization and have used in the past, and the ways I have worked with Product Managers that have built close relationships.
Talk through each part briefly
Define Research Objectives
This is defining the outcomes of research - what the goals are, what you’re aiming to learn, what these learnings are going to help you accomplish
Develop Research Plan
Choosing appropriate methodology and building a plan to meet those goals if we don’t have past research that can help answer the questions
Recruit & Schedule Participants
Based on the objectives and research plan, recruit representative participants. This is potentially a long phase depending on the complexity of the research.
Execute Research
Conduct the research sessions with appropriate measures (e.g., informed consents, audio recording, notetaking, etc.)
Analyze Data & Create Deliverables
Spending time in the data to identify patterns, which turn into insights
Build out deliverables that were determined earlier on, even some additional ones if the insights support it
-----
These areas are areas where the Product Manager can be involved in the process to benefit both themselves as well as the researcher in understanding where the data is coming from. We will dive into these areas more deeply now.
The UXR will work with you to define research objectives, build hypotheses, etc. depending on the needs of the research.
You can also think of the UXR as your translator - someone who can understand human behavior and translate that for you. I often tell stakeholders to ask the questions they want to ask if the customer / prospect, etc. could tell them the truth, and that I will translate their questions into research approaches and methods to get as close as possible to those answers.
From knowing what you want answered we can determine the deliverable / outcome of research to answer that question, the type/phase of research (e.g., explore, evaluate, or assess), and then finally determine the method.
If I had a Product Manager of mine come to me wanting to learn more about businesses in a specific industry that we serve, for example, what their pain points were, where opportunities might be, then I get to live the researchers dream and plan some exploratory research.
As the UXR builds out the research plan, they will likely pull you into the loop to review. I will do this to ensure a) that nothing has changed on the stakeholder’s side so we don’t spend time researching the wrong thing and b) that we have mutual understanding, again, so we don’t research the wrong thing.
Pulling that example in, an exploratory study such as field research could have a research plan resulting in 20+ pages. Including recruiting screeners, schedules, number of participants needed, travel budget, incentive budgets, etc, basically everything needed to ensure that that research project went off without a hitch. Once the Product Manager, any other stakeholders, and I are on the same page then… (next slide)
I work through the hardest part of research, recruiting! The UXR or in collaborating with a firm or another colleague, depending on your organization, to ensure you’re getting the right people invited to your research study based on the thorough planning we have worked on together.
Lots of phone calls, follow up emails, reminders, etc. go into this phase.
In a recent field research study, I had emailed customers several times to get responses, followed up with phone calls, confirmations, etc to ensure that the day of our visit would go on as smoothly as possible.
Depending on the type of research, the UXR may invite you to participate. This may be coming along and taking notes, or observing one or two sessions. This allows you to be a part of the picture and see what the UXR is seeing. You also will then understand where the insights and deliverable is coming from.
Remember that only observing ONE or TWO sessions won’t give you all the answers. The UXR is going to be collecting a lot more data to analyze and synthesize that you may see.
Going back to field research, I will bring a Product Manager with me and ask them to take notes. I also give them an understanding of what the ideal is, what will likely go wrong, and how we’ll pivot. Nothing ever goes 100% to plan, especially in the field.
Analysis! We’ve spent all that time collecting data now we’ve got to do analysis and synthesis which will turn into deliverables. We will comb through the data to identify meaningful patterns which will result in insights and learnings.
Depending on the type of project, the UXR may include you in analysis process. I do this when I am unable to bring stakeholders with me, for example, and do mini analysis or brain dumps of what I learned after a session or trip. This allows the stakeholder to fine tune questions for additional sessions and iterate on the research if needed.
Deliverables are defined at the beginning of the research project, though depending on the effort there may be multiple deliverables and outputs, some which may not have been apparent at the beginning of the study. This could be a survey summary deck, a list of insights, personas, or a mental model. It really depends on the questions that need to be answered.
Back to field research example. Especially when things are more exploratory maybe there isn’t a defined deliverable other than answering some questions. But then you learned about the ecosystem of the industry, so you map that out, and you learned where there might be an opportunity to expand your product, so you map that out.
Now that you know the process and which parts of the process we collaborate heavily, let’s jump into ways we can both foster the budding friendship.
Prioritize what you want to learn collaboratively with the UXR -> this will allow them to focus and help you scope down projects into multiple manageable efforts
Let the UXR guide you to scope out research objectives, hypothesis, etc. and translate what you want to ask into ways we can identify that from human behavior. Collaborate here to ensure that you’re both on the same page
Participate/observe when able. Seeing things happen live will give you opportunities to see UXR in action, understand the process, and also empathize more deeply with your customers / people who use your products.
Please, please, please, use the insights we learn together in your work to make decisions. :-)
Research the PM - super important, learn what their goals are, what they are trying to accomplish, what will make them look good to their bosses. This will help you understand their perspective and what they want to learn at a deeper level and help them in ways they may not even realize you can.
Help the PM by scoping a project down and prioritizing what you want to learn. We all know we can’t research EVERYTHING in one project.
Keep going back to “What do you want to learn?”, this will help you scope the project down and get your Product Manager or stakeholder back to the root need of why they need research support.
Involve the PM - keep inviting them, even if they can’t come. This will help build a trusting relationship over time. I have PMs who used to watch all sessions, now trust me to just give them a quick readout after a usability test is done and will take that and run with it now that we have that close trusting relationship.
Now that we have are building a great friendship and working relationship, we can venture out into the world of research to find the untraveled path to successfully build and enhance our products together.