UX researchers can deliver more value by optimizing how they work with research sponsors at two key stages of a study: defining study questions and delivering results. When defining study questions (i.e., scoping and framing the study), researchers can improve upon initial input from sponsors by (1) enlarging the problem frame and (2) refining the questions posed to study participants. When delivering results, researchers can use two tactics: (1) preserving freedom of action and (2) adding breathing room between findings and recommendations. The recommended practices in this talk arose my idiosyncratic reflections and solutions to challenges I’ve encountered in conducting UX research with project teams. These practices have been validated in numerous engagements, and shared informally with colleagues in multiple organizations.

1. UX Research: Optimizing collaboration
with project study sponsors
Paul McInerney

2. Requester Researcher
product manager
product owner
designer
strategist
executive
etc.
Questions
Answers
UX researcher
user researcher
design researcher
etc.
Requestor-Researcher collaboration

3. Requester Researcher
1. Request a study
Questions
Answers
3. Conduct the study
Context of a study
4. Deliver study results
2. Plan the study
5. Consume results

4. Agenda
• Interaction #1: Defining questions
4 suggested practices
• Interaction #2: Delivering answers
3 suggested practices
• Q & A
Approach
• Case-studies
• Your perspective and stories
after each practice

5. Requester Researcher
Questions
Defining questions for a study
Start: Requestor poses questions for a study
End: Agreement on revised questions

6. Importance of high quality study questions
Low quality study questions
Low value answers

7. Creating high quality study questions
Low quality study questions
High quality questions
4 practices
High value answers

8. Study phases
• Kick-off engagement
• Plan study
• Prepare for data collection
• Collect data
• Analyze
• Report
• Deliver
Study plan
• Project background
• Study questions
• Method & Participants
• Project plan
Questions

9. Relative contribution
Study plan Requestor Researcher
Project
Background
80% 20%
Study
Questions
50% 50%
Method 20% 80%
Project Plan
20% 80%

10. Relative contribution Two perspectives on
study questions
Project value Study quality and
feasibility
Study plan Requestor Researcher
Project
Background
80% 20%
Study
Questions
50% 50%
Method 20% 80%
Project Plan
20% 80%

11. Relative contribution Two perspectives on
study questions
Project value
Study quality and
feasibility
Study plan Requestor Researcher
Project
Background
80% 20%
Study
Questions
50% 50%
Method 20% 80%
Project Plan
20% 80%

12. Requester Researcher
Questions
Resolving ambiguities in
the problem statement

13. Original
Clarified
or

14. “We want to find out why our product is
not succeeding in the marketplace.”

15. Business problem Study approach
Acquisition: "Our product is not acquiring
sufficient new customers"
Recruit non-customers and understand reasons for
lack of adoption
Retention: "Our product is not retaining
existing customers"
Recruit former customers and understand reasons for
dissatisfaction or leaving
“We want to find out why our product is
not succeeding in the marketplace.”

16. “We want to find out why our product is
not succeeding in the marketplace
acquiring sufficient new customers”

17. “Our team has a long term roadmap
for a major UI transformation and
we’re working on the initial release.
We want to validate this with users."
Ambiguity:
• near term: the initial release?
• long term: the full roadmap?
“Our team has a long term roadmap
for a UI transformation and
we’re working on the initial release.
We want to validate this with users."

18. “Our team has a long term roadmap
for a UI transformation and
we’re working on the initial release.
We want to validate the initial release with users."

19. Resolving ambiguities in
the problem statement
Your Turn
Your take &
your stories
Some common ambiguities:
• Acquire vs. retain customers (Case 1)
• Near term vs. long term solution (Case 2)
• Granularity: big picture vs. fine detail
• Understanding users vs. evaluating a design

20. Requester Researcher
Questions
Addressing the full picture

21. “Conduct a study on X”
“No. Because ...
• Topic Y is not in scope
• We already know about Topic Y
“Also Y?”
• We just need to validate that Design
X is good enough
• Design Y is not feasible”

22. X
X
Y
Scope of the
question
NO!

23. Departure point:
Requester says NO to including Y
Destination:
Requestor agrees to act on
findings related to Y
1. Collect data on Y
2. Consider the results
3. Act on the results
X+Y
X
Journey to address the full picture
Y
X+Y

24. Journey to address the full picture: Agree to …
1. collect data on Part B
2. consider the results on Part B
3. address the issue(s) with Part B
Customers complaining
about Part A
Part B too?
Part A
Part B
No … we already know
complaints concern Part A

25. Journey to address the full picture: Agree to …
1. generate and evaluate enhancements to Design A
2. review results and agree enhancement is better
3. deliver enhancement
Usability test our design
Also design enhancements?
Design
Enhancement
No … just validate our
design is good enough

26. Addressing the full picture
Your Turn
Your take &
your stories
X
X
No. Because ...
• Topic Y is not in scope
• We already know about Topic Y
• We just need to validate that Design X is good enough
• Design Y is not feasible
Y

27. Requester Researcher
Questions
Transforming draft study materials

28. Internal design review questions
Q1: Is the amount of information
overwhelming?
Q2: Is this the right way to present
eligibility criteria?
Q3: Should we include the point
about digital signatures?

29. Internal design review questions John’s proposed questions for study participants
Q1: Is the amount of information
overwhelming?
Is the amount of information overwhelming? Yes | No
Q2: Is this the right way to present
eligibility criteria?
How easy is it to determine whether you are eligible for
each program?
(1=very difficult; 5=very easy)
Q3: Should we include the point
about digital signatures?
How useful is the point about digital signatures?
(1=not useful at all; 5=very useful)

30. Requestor’s proposed
study materials Transform Refined study materials
Clarify
Refine
Reframe
Elaborate
Enrich with context
Extend to adjacent topics
Etc.
Fine-tuning material presented to participants

31. Internal design reviewers Part of the design
Q1: Is the amount of
information overwhelming?
Eligibility criteria
Additional information
Q2: Is this the right way to
present eligibility criteria?
Eligibility criteria
Q3: Should we include the
point about digital signatures?
Additional info > digital signature

32. Internal design reviewers
Q1: Is the amount of information overwhelming?
Q2: Is this the right way to present eligibility criteria?
Q3: Should we include the point about digital signatures?
Eligibility Criteria
• Q1A: Are eligibility criteria overwhelming?
• Q2: Is that the right way to present eligibility criteria?
Additional Info
• Q1B: Is the additional information overwhelming?
Digital signature
• Q3: Include this point?
Transform
Requestor’s proposed study materials (Partly) refined study materials

33. Is this the right way to present
eligibility criteria?
Research strategy: Compare original design to
one or more alternatives:
• Original design
• Alternative 1
• Alternative 2
Questions presented to participants
How satisfied are you which each alternative?
Which alternative do you prefer and why?

34. Is this the right way to present
eligibility criteria?
Are you satisfied with this design?
Do it prefer it over the alternative?
Transform
Requestor’s proposed study materials Refined study materials

35. Should we include the point about
digital signatures? (Yes or No)
Research strategy: Probe underlying attitudes:
• Is providing a digital signature something that concerns people?
• If people are concerned, what is the nature of the concern?
Are the concerns more about privacy/security or the ease of use?
Questions presented to participants
E.g., Do you have any security concerns about providing digital
signatures on websites? If so, please describe them.

36. Should we include the point about
digital signatures? (Yes or No)
Should we …
1. Keep the point Improve the wording (to
address concerns)
2. Remove the point (because people are
not concerned about this)?
Transform
Requestor’s proposed study materials Refined study materials

37. Transforming draft study materials
Your Turn Your take & your stories
Requestor’s proposed
study materials Transform Refined study materials

38. Requester Researcher
Questions
Committing to a decision

39. Why?
“It would be interesting”
“To see if there’s any
issues with our design.”

40. Project need ←?-- Study question
Project need ← Improved study question
Project need ← Decision ← Improved study question + Decision Rule*
* Decision rule format
If study results are X then do action Y
Otherwise, then do action Z

41. Study goal #3: Determine whether the content about
digital signatures is suitable*:
* “Suitable” means that the amount of reassurance provided matches
the level of user concern.
The results will guide the project actions as follows:
• If the results show that people have little or no
concern, then this content will be omitted to
streamline the page.
• If the results show that people are very concerned,
then this content will be enhanced to address the
concerns.

42. Committing to a decision
Your Turn
Your take &
your stories
Study goal: …
If the results show …
then project will do …

43. Requester Researcher
1. Request a study
Answers
4. Deliver study results
5. Consume results
Questions 2. Plan the study
3. Conduct the study
1. Request a study

44. Typical considerations for preparing results
• What insights are most “actionable”
• Amount & type of supporting evidence to provide
• If/how to include recommendations
• Crafting visually compelling PowerPoint slides

45. Considerations addressed in this talk
Theme: Empathize and support the requestor’s goals
• Get answers to questions
• Include their judgment in decision making
• Get good recommendations

46. Requester Researcher
Organizing findings
Answers

47. Common ways reports organize findings
Survey studies
• One slide per survey question – i.e., follow the survey outline
• One slide summarizing results from all related survey questions
User tests
• One slide per session task
• One slide per part of the design evaluated

48. Recommended approach
Use study questions as a table of contents for the findings section
Study questions*
Q 1
• Q 1.1
• Q 1.2
Q 2
Q 3
Finding section
Findings for Q 1
• Findings for Q 1.1
• Findings for Q 1.2
Findings for Q 2
Findings for Q 3
Study Plan Study Report
* AKA Study goals

49. Study questions are the lynchpin a study
Study
Plan
Study
Report
Study
Questions

50. Case study
Understand the popularity of
call centres vs. self-help options
for obtaining assistance.
Study questions (goals)
How popular is the call centre relative to the
following other sources of assistance?
• <list of 4 sources of assistance of interest>
How much does the popularity of the call centre
vary across the following scenarios?
• <list of 9 scenarios of interest>
What are the pros and cons of the call centre vs.
other sources of assistance?
Study Plan

51. Study questions (goals)
How popular is the call centre relative to 4 other
sources of assistance of interest?
How much does the popularity of the call centre
vary across 9 scenarios of interest?
What are the pros and cons of the call centre vs.
other sources of assistance?
Study Plan Study Report
Finding section (slide titles)
Popularity of the call centre vs. other sources of assistance
Popularity of the call centre for different scenarios
Attributes of the most popular scenarios
Attributes of the least popular scenarios
Comparing sources of assistance (quant)
Comparing sources of assistance (quat)

52. Refining study questions
1. Initial study questions
3. Revised study questions
2. Envision future findings
Study
Plan
Study
Report
Study
Questions

53. Organizing findings
Tip
Organize findings to answer each question
in the study plan
Why?
Study questions (goals) are the lynchpin in
a study
• The study plan poses questions, and the
report answers them
Project teams don’t want data – they
want answers to their questions
Your Turn
Your take &
your stories

54. Requester Researcher
Preserving freedom
Answers

55. (Forced) Ranking
Rank these items in order of importance
vs.
(Priority) Rating
Rate the importance of each item
What are the top 3 items?
Item A
Item B
Item C
Item D
Item E
Item F
Item G

56. (Forced) Ranking
Rank the items
(Priority) Rating
Rate the importance of the items
Ranking Item
1 Item B
2 Item F
3 Item G
4 Item C
5 …
Rating Item
4.5 Item B
4.2 Item F
3.9
3.8
Item G
Item C
… …
What are the top 3 items?
VS

57. Ranking Rating
Precision ✓ Better Worser
Freedom of
action
Worser ✓ Better
Trade-off between rankings and ratings

58. Freedom of action explained
• Results inform* the decision, but don’t dictate it.
*Decision maker can combine study results + their
judgement to make a decision.
• Applies when results suggest users don’t have a strong
preference among the options.
How to preserve freedom of action
• Give participants the opportunity to express
indifference among the options
• For example, by using ratings rather than rankings

59. 5-point scale
❑ Strongly prefer A
❑ Somewhat prefer A
❑ No preference
❑ Somewhat prefer B
❑ Strongly prefer B
Do customers prefer option A or B?
3-point scale
❑ Strongly prefer A
❑ No preference
❑ Strongly prefer B

60. 5-point scale 3-point scale
Precision ✓ Better Worser
Do customers prefer option A or B?

61. “Option A is a little better than Option B,
but not much better.
So, I’m going to choose “No preference.”
Strongly prefer A 10%
Somewhat prefer A 35%
No preference 30%
Somewhat prefer B 20%
Strongly prefer B 5%
Strongly prefer A 25%
No preference 60%
Strongly prefer B 15%
5-point scale 3-point scale

62. Would customers prefer option A or B?
5-point scale 3-point scale
Precision ✓ Better Worser
Freedom of
action
Worser ✓ Better
Overall Worser ✓ Better

63. Preserving freedom
✓ More precise
but
 Ties your hands
VS
 Less precise
but
✓ Free to make final decision
Your Turn
Do you agree with this trade-off?
How do you address it?
Where else does it apply?

64. Requester Researcher
Adding breathing room
Answers

66. • Finding 1 → Recommendation
• Finding 2 → Recommendation
• Finding 3 → Recommendation
• Finding 4 → Recommendation
• Finding 5 → Recommendation
• …

67. Finding 1
Recommendation
for finding 1
Finding 1
Recommendation for finding 1
Finding 2
Recommendation
for finding 2
Finding 2
Recommendation for finding 2
Etc. Etc.

68. Finding→
Finding → (breathing space) →
Breathing
Room
Recommendation
Recommendation

69. Finding 1 →
Finding 2 →
Finding 3 →
Finding 4 →
Finding 5 →
Finding 6 →
Look at findings
holistically to
discern larger themes
→ Recommendation A
→ Recommendation B

70. Controversial
finding Resolve controversy
Clarify or revise
finding Recommendation

71. Finding
Consider additional
factors
→
→
Recommendation
→

72. Adding breathing room
Adding breathing room improve:
• Understanding of the findings
• Recommendations
Your Turn
Your take &
your stories
Breathing
Room
Recommendation
Finding → →

73. UX Research: Optimizing collaboration
with project study sponsors
Paul McInerney
Questions
Resolving ambiguities in the problem statement
Addressing the full picture
Transforming draft study materials
Committing to a decision
Answers
Preserving freedom
Organizing findings
Adding breathing room

74. UX Research: Optimizing collaboration
with project study sponsors
Paul McInerney