This presentation was provided by Serena Rosenhan of ProQuest, during Session Four of the NISO event "Agile Product and Project Management for Information Products and Services," held on June 4, 2020.

1. User Research
Agile Product and Project Management
for Information Products and Services
NISO Training Series
June 4, 2020
Serena Rosenhan
VP User Experience, ProQuest

2. What
questions
do you hope
to have
answered
today?

3. • I. Why? What? & When?
• II. How?
• III. And then what?
User Research Session Outline

4. Why? What? & When?

5. • What happens before they click?
• Do we have the right navigation?
• How can we make it easier to . . .
In Discovery phase – often called “experiments”
Starts with a question

6. User research in context
Discover
user needs
and pain
points
Learn if an idea has
value
Validate if a product is
usable
Understand user
behaviors and
motivations
https://www.jpattonassociates.com/
dual-track-development/

7. HYPOTHESIZE / IDEATE
Contextual Inquiry
Card sorting
Analytics
Heat-maps
Survey Data
VALIDATE
Prototyping
Usability Testing
Beta Testing
A/B Testing
Eye-tracking
Users, needs,
behaviors
Interaction
design
Different reasons for User Research

8. Many Research Methods
Qualitative
Contextual Inquiry
Card sorting
Diary studies
Usability testing*
Focus group
Quantitative
Survey
Analytics / usage
A/B testing
Eye tracking*
* Don’t forget to consult secondary research if available

9. Watch users do what they normally do in
their own environments
– Traditionally in-person
– Context of use (external factors)
– Natural task-flows (show me)
– Open ended – inductive process
– Best for learning about user needs & tasks
Contextual Research
Observation Interview

10. Ask users to group and label items
– Validates/reveals mental
model
– labeling
– useful for designing
information architecture,
workflows, menu structure,
or web site navigation paths
Card Sorting

11. Participants independently record
experiences and data
– Provides detail over time
– May share with researchers during
or after study
– Best for longitudinal studies or
questions about change, adapting,
learning or growth
Diary Studies

12. Moderated Testing
• Qualitative user interviews
• Remote or in-person
• Lab or lunch room
• Task-based
• Allows for follow up questions and feedback
Un-moderated (Remote) Testing
• Requires specific tools/services
• Recruiting is often included
• Quick turn around allows for iterative testing
• Ideal for testing alternative solutions for specific features
Usability Testing

13. Allows for comparative data with larger
number of users
– Demographic and characteristic
information
– Sentiment & satisfaction
– Task completion
– Open-ended feedback
– Users self-report
– Ask if you can follow up
– Use in conjunction with other
methods
Survey

14. Different goals
Different measurements
Different granularity
But . . . they inform one another . . .
Market research vs. user research
Market Users

15. • Shows what happens
in the aggregate (but
doesn’t explain why)
• Requires online tools,
some expertise
• Tools like heatmaps
help visualize the
experience
Analytics / Usage
120 minutes
61,421 clicks
9,451 visitors
• Red/orange colors for
click volume
• Blue/green colors for
mouse move traffics

16. • Paper
• Linked mockups
• Rapid prototyping
• HTML prototypes*
*Invaluable for mobile-first and
responsive design.
Prototype testing

17. Section I
Discussion

18. How?

19. Who’s job is it?
UX
Researcher
or person
with
research
skills
Good interviewer
Survey/question design
Can avoid bias
Observant
Synthesis

20. Make a Research Plan
Objectives
Hypothesis
Method
Participants
Timeline
Logistics
Budget
What will your
research
accomplish?
Who are the
stakeholders or
sponsors?
How will the data be
used?

21. Make a Research Plan
Objectives
Hypothesis
Method
Participants
Timeline
Logistics
Budget
Clearly state what you are trying to learn:
Hypothesis:
We believe if we do ___________
For ______________
We will see ____________
Which we can measure by ___________
Exploratory question:
How does our service/tool fit into the user’s larger workflow?
Usability question:
Can users successfully complete a task on our site?

22. Make a Research Plan
Objectives
Hypothesis
Method
Participants
Timeline
Logistics
Budget
Select the available method(s) that will help you
answer your question.
Consider:
• Who needs to be informed/convinced (stakeholders?)
• Team skill set / tools / time
• Effort/ timing
• Level of risk if you are wrong
• What you can and can’t learn from each method
• Sample size
Tip: You don’t have to learn everything in one study.
Research should be iterative and can build on itself.

23. Make a Research Plan
Objectives
Hypothesis
Method
Participants
Timeline
Logistics
Budget
Decide who you need:
• Which users? (Which personas?)
• How many?
• Target characteristics? (role, geography, field of study, gender,
etc.)
Screening criteria (who will be disqualified)
Decide how you will find them:
• External services
• Built into some tools
• DIY
• In-product survey
• Flyer
• Friends & family referrals (NOT participants)
Tip: Check if your study will require IRB approval at your institution

24. Make a Research Plan
Objectives
Hypothesis
Method
Participants
Timeline
Logistics
Budget
Session outline

25. Make a Research Plan
Objectives
Hypothesis
Method
Participants
Timeline
Logistics
Budget
Session outline
Capture the people, tools, places, and
communication you will need for the test to be
successful.
• Where will the research take place?
• What technology is required? (Mobile device? Video recorder?
Meeting software with screen share capabilities, test accounts,
addresses,
• Legal review? (e.g. Is a consent form needed?)
• Communication templates
• Facilitator
• Notetaker (how will notes and data be captured consistently?)
• Observer

26. Make a Research Plan
Objectives
Hypothesis
Method
Participants
Timeline
Logistics
Budget
Session outline
What costs will you have?
• Participant incentives (if needed)
• Travel
• Physical equipment (video, memory cards)
• Software/tools
• Assistance
• Recruiting

27. Make a Research Plan
Objectives
Hypothesis
Method
Participants
Timeline
Logistics
Budget
Session outline

28. 1. Talk too much
2. Test with (listen to) the wrong users (or not enough)
3. Ask “yes or no” questions
4. Ask the user if they like your product idea
5. Include the terms or ideas you are looking for in your questions
6. Test by checklist
7. Ignore the influence of test order
8. Ignore mocked up content in prototypes
9. Decide how to analyzed the results after the research sessions
10. Lead the witness (Remember: “There are no right answers”)
10 ways to introduce bias
into your user research:

29. Section II
Discussion

30. And then what?

31. User Research Product
Design
31

32. Analyzing the data
Qualitative
Transcription
Annotation
Coding
Semantic analysis
Affinity mapping
Quantitative
Calculations
Cross-tabulation
Significance
Outliers
Involve observers
Look for patterns,
themes, significance
Decide how to handle
outlier data
•Unique and circumstantial
•Possible early indicator
•Articulator
Tip: Flag quotes and video clips as
you go
Tip: Avoid the weeds – stay focused
on answering your research
questions.

33. • Establish what you learned . . .
– Focus on insights and observations
– “My hypothesis is . . .”
• . . . And what you didn’t learn
• Decide which findings are important to act on and
prioritize
• Identify your next experiment
Resist the temptation to solution too early

34. • Adapt to stakeholder and team needs
– Formal report → Informal summary
– Discovery goal is rapid learning – keep it
lightweight
– Focus on creating empathy and shared
understanding over reporting facts
– Include the “so what?”
• Common research artifacts may be more
effective at documenting what you learned
– Personas
– Journey maps with pain points/opportunities
– Mental models
Documenting Research
“I wish there wasn’t so
much trial and error in
finding sources. You
think you find a source
that is relevant and read
half of it and it just isn’t
what you need.”
– Anthony, Law Grad, U of
Chicago

35. Share what you learned
Create shared
understanding
Let the users speak
for themselves
Don’t expect people
to read a report
• Share in sprint
review
• Send out a
recording

36. Section III
Discussion

37. Final Thoughts

38. Can I do User Research?
Q: I don’t have all those fancy tools . . .
A: They are great, but you really only need two
things:
. . . and a willingness to experiment and learn
Determination Discipline User insights!

39. Can I do User Research?
Plan
• Be clear about what you
want to learn
• Isolate “experiments” (test
one thing a time)
Test
• Get some users (even 5 or
6 will tell you a lot!)
• Ask open ended questions
or task prompts
• Don’t create bias
Learn
• Describe the behavior or
insight before jumping to
solution

40. Serena H. Rosenhan, PhD
VP, User Experience Design at ProQuest
serena.rosenhan@proquest.com
Thank you