The document discusses how to prepare for successful user research sessions by anticipating and planning for unexpected situations. It recommends identifying potential issues before a study, such as participants being late or unable to complete tasks. The authors advise establishing backup plans, like scheduling extra participants or creating paper surveys. They also suggest preparing the product, space, and technology to avoid technical problems. Guidelines are provided for briefing participants and observers. The document stresses practicing moderating skills and technical setups to improve sessions. The overall message is that thorough preparation can help user researchers successfully handle unexpected challenges during studies.

1. Donna Tedesco
Fiona Tranquada
Expecting the Unexpected:
Preparing for Successful
User Research Sessions
UXPA Boston 2013

2. How did we
get here?

3. Identify what you can do
before your study to:
 Reduce the chances of
encountering
annoying, tricky, and sticky
situations – especially the
ones you don’t usually expect
– during a session
 Set yourself up to
successfully handle those
situations when they occur
Our Goal

4. Frequent
 Participant is late or a no-
show
 Participant not thinking
aloud
 Participant is unable to
complete a necessary task
Infrequent
 Participant makes you feel
uncomfortable
 Participant frustrated by
prototype limitations
 Observer interrupts the
session
 Natural disaster
What do we mean by “unexpected”?
Being prepared will help you with both types of situations!

5.  Recruit
 Study Plan
 Product/Space/Technology
 Observers
 Yourself
What We’ll Cover

6. Identifying
the “right”
participants
Recruit

7.  Identify your
“dealbreakers”
 Look for behavior in
addition to role
 Avoid “giveaway”
questions
Be specific with your criteria

8.  Confirmation
letter/email, including
directions to facility or
remote session
 1 on 1
 Recording and observers
 If you need them
 to bring anything
 to do anything during the
session (e.g., watch them
process payroll)
 to prepare
Set expectations about the session

9.  Backup – 2 participants
scheduled for 1 timeslot
 Floater – 1 participant
scheduled to cover multiple
timeslots
 Incremental cost but may
be worth it if you have high-
profile observers
 If you don’t have backup or
floaters, plan for no shows!
Schedule backup or floater participants

10. Establishing
priorities and
making
backup plans
Study Plan

11.  Think about how you can
adjust your plan if you
encounter:
 Time constraints
 Misrecruits
 Personality challenges
 Technical difficulties
 Create a backup such as a
paper survey or
questionnaire
Prioritize your tasks/questions

12.  Reiterate the points covered
in your consent form:
 Recording/observers
 How data will be used
 Goals for the session
 Take a break or end the session
at any time
 Set clear expectations for:
 How the session time will be
used
 What they’ll be using (e.g., a
prototype)
 Your behavior (e.g., may not
answer questions)
 Verify qualifications
Prepare the participant briefing

13. Avoiding
technical and
environmental
challenges
Product,
Space, &
Technology

14. Prototype
 “Illusion of functionality”
 Agree on if/when any updates
will be made to the prototype
Actual “live” product
 Be aware of the product
schedule
 Research any A/B tests
 Have a backup if necessary
(e.g., screenshots) or set
appropriate expectations
Prepare the Product

15.  Walk through your full setup
 If participants are remote, do a
walkthrough with them as well
 Document everything!
Include screenshots!
 Order of operations
 Lab/space setup
 Recording
 Screen sharing / Audio
conferencing
 Controls for observers (Mute All)
 “Kick out” options
 Prototype/Product
 URLs
 Login/Password
 Paths
 Any known issues (e.g., browser
incompatibilities)
 Any backup equipment
 Technical support contact
Document your setup

16.  Emergency exits
 Restrooms
 Kitchen / Vending areas
 Security / Emergency
procedures
 Printers / Copiers
 Climate controls
 If going off-site,
 Equipment prohibitions
 Seating (e.g., if you have
observers)
 Internet access
 Noise levels
 Other environmental factors
Explore your research space

17. Establishing
guidelines
for behavior
and
interactions
Observers

18.  Establish a code of conduct
 Remember – you’re responsible for the
participant’s well-being!
 Share with observers ahead of time
 Via email
 Poster on the wall of research space
 Printouts
 Hold observers accountable
Provide ground rules

19. Core ground rules to include:
 Keep quiet; no laughing at or making fun of participant!
 Keep participant’s identity confidential
 Avoid slamming doors
 If observers are remote,
 Keep phone muted
 Avoid in-meeting chat feature
 If observers are in the room with you,
 Avoid multitasking
 Be careful with body language
 What to do if they have questions or want to communicate with
you/participant
Provide ground rules cont.

20.  May not exactly follow study plan
 May let participant follow tangents
 Assists and diversions
 When you might use them
 How to tell if that’s what you’re doing
 May pretend to not know an answer
Communicate your approach

21.  Make sure you always have
at least 1 person observing
 Avoid going to someone’s
space without 1 other
person
 “Safe word” to indicate
when you need assistance
Establish safety precautions

22. Improving
your
technique
Yourself

23.  Ethical obligations
 Your attitude
 Your body language
 What you say
 Your tone
Review good moderating behavior

24.  Helpful if you’re:
 New to moderating
 Running a new kind of
research
 Expect certain behaviors
 Trying to adjust your style
 Include in your study plan –
don’t be afraid to put it on
every page
Jot down key phrases or reminders

25. Examples of what you might jot down:
 Wait 3 seconds before responding to a participant
 Neutral acknowledgements (“Mmhmm”, “Uh-huh”)
 Turn question around
 “If you were doing this at home, and I wasn’t sitting there next to
you, what would you do?”
 Prompt non-responsive participant
 “What are you trying to do right now?
 Reassure
 "This is just the kind of feedback we want to hear..."
 Redirect or cut short
 "I'm sorry, I just want to interrupt you here for a second. For the sake of
time I’d like you to return to the task/question, and we can revisit this
topic at the end of the session if time permits.
Jot down key phrases or reminders cont.

26.  Watch recordings of yourself
 Practice with a colleague
 Have your colleague “role-play” problematic situations
 Ask colleagues to watch you moderate and provide feedback
 Watch others moderate
Practice!

27.  Set expectations
 With participant
 With observers
 Dive deep
 Your space
 Your technology
 Your study plan
 Create backups
 For your technology
 For your participants
 For your product
 Practice
 Your technical setup
 Your moderating
Preparation Takeaways

28. Thank you!
Donna Tedesco (donna.tedesco@gmail.com)
Fiona Tranquada (ftranquada@gmail.com)
Our (forthcoming) book: @ModSurvivalUX
Questions?