UX Camp Brighton 2019 slides

1. @laura_yarrow
Difficult research participants
…and (non-sinister) ways of dealing with them

2. @laura_yarrowDifficult user testing participants 2
★ Research in labs/in office
★ In-house or agency
★ Recruiting participants
★ Organising the day
★ Managing a client and observers
This covers:

3. @laura_yarrowDifficult user testing participants 3
Sometimes, no matter what you do,
tricky participants just happen.

4. @laura_yarrowDifficult user testing participants 4
Disclaimer:
Most user testing participants are
reacting to an unnatural situation

5. @laura_yarrowDifficult user testing participants 5
The many types of
participant to deal with…

6. @laura_yarrowDifficult user testing participants 6
The Chatterbox
★ Spend all the session off topic
★ Tell you their life story - use the session as a social event
★ Lengthy discussions about the task
★ Don’t take it seriously
Tips:
✓ Put a large clock on the desk
✓ Time-boxed script
✓ Don’t be afraid to interrupt
✓ Use closed questions at the end to get to the point
MR CHATTERBOX

7. @laura_yarrowDifficult user testing participants 7
2. The Mute
★ Nervous, detached, despondent, or naturally quiet
★ Recent trauma or bad situation
★ Won’t talk through what they’re doing, afraid we’re
testing them
✓ Be kind
✓ Be ready with open ended questions
✓ Have an introduction at the start that states they’re
not being tested
MR QUIET
Tips:

8. @laura_yarrowDifficult user testing participants 8
3. The Pleasers
★ Wants to tell you what you want to hear
★ Won’t criticise anything, even if they struggle
★ If asked to rate anything, will give 10/10
★ Will rush through everything like a race
✓ Screen for their sentiment towards brands/clients
✓ Emphasise your neutrality
✓ Play devils advocate - say you have to bring back a
balanced set of feedback
MR PERFECT
Tips:

9. @laura_yarrowDifficult user testing participants 9
4. The Nosey Ones
★ Grill you about the client or organisation
★ Find it hard to focus on the tasks and research
★ May start using their phone/your tech to look up
details!
✓ Again, reiterate your independence
✓ Let them know you can tell them at the end
✓ Don’t name anyone
✓ Test a range of products/sites/apps to maintain neutrality
MR NOSEY
Tips:

10. @laura_yarrowDifficult user testing participants 10
5. The Cheaters
★ Lied on the screener!
★ Trying to make a profession out of being a research participant
★ Will skew your research findings
★ Say they use the tech or devices required but can’t in the session
✓ Compare previous answers to screeners
✓ Keep a blacklist - take note of names that respond regularly
✓ Always recruit a backup
LITTLE MISS
NAUGHTY
Tips:

11. @laura_yarrowDifficult user testing participants 11
6. The Undercover Agent
★ Refuses to be recorded or observed
★ Highly suspicious, very wary of recording devices
★ May have had a past bad experience
✓ Ask them in the screener if they are happy to be recorded in the session
✓ Give them a copy of the consent form
✓ Let them know exactly what you do with the recordings
✓ In the worst case, fall back to audio recording, or good note-taking!
MR WORRY
Tips:

12. @laura_yarrowDifficult user testing participants 12
7. The Free Spirit
✓ Time-box your script and tasks
✓ State “I want to spend 5 minutes talking about…”
✓ Keep a clock on the desk and keep referring to it
✓ Send instructions before hand about thinking out loud so they can practice
LITTLE MISS
SCATTERBRAIN
★ Regularly goes off course, distracted
★ Opening new tabs in browsers, not sticking to tasks
★ Incoherently narrates, can’t think out loud
★ Will show you photos on their phone of their cat/grandkids/plant
★ Possibly eating a MacDonalds lunch throughout
Tips:

13. @laura_yarrowDifficult user testing participants 13
8. The Reluctant Participant
★ Treat it like an interview
★ Won’t touch the tech or engage
★ Think it’s a quiz and try and guess the answers to tasks
without doing them
✓ Set expectations if you want them to use some devices
✓ “Can you show me how you would do this?”
✓ “Without telling me…”
✓ Gesture at the devices you want them to use…
LITTLE MISS
STUBBORN
Tips:

14. @laura_yarrowDifficult user testing participants 14
9. The Confused
★ Not sure why they’re here / never done this before
★ Doesn’t normally use the tech you’re expecting them to use
★ Thinks they can bring their friend/child/dog…
MR MUDDLE
Tips:
✓ Send a guide before hand
✓ Brief them in at the start (what/why), and have they done this before?
✓ Only one question at a time
✓ Screen for devices they’re familiar with / technical ability

15. @laura_yarrowDifficult user testing participants 15
10. The Inappropriate Person
★ Wants to talk about you more than the tasks
★ Makes inappropriate comments
★ Poor hygiene etc
★ Inadvertently in-appropriate
✓ Don’t go alone, take a colleague who’s always watching
✓ End the test if you feel uncomfortable
✓ You can position them where you want!
LITTLE MISS
SCARY
Tips:

16. @laura_yarrowDifficult user testing participants 16
11. The No Show
★ Doesn’t let you know
★ Doesn’t answer the phone
★ May turn up really late and try and slot themselves in
★ Will suggest a date in 4 weeks when they have a slot free…
✓ Blacklist
✓ Capture their phone number and text the night before
✓ Have a backup participant, or “floater”
✓ Allow breaks between each session where you could squeeze them in
MR NOBODY
Tips:

17. @laura_yarrowDifficult user testing participants 17
Summary
✓ Send out information before
✓ Have a time-boxed script to follow
✓ Text the night before
✓ Have a backup participant
✓ Screen properly (device, sentiment, job…)
✓ Brief the participant at the start
✓ Hide the tech
✓ Tell them what you do with recordings
✓ Allow for 5 minutes of settling in at the start
✓ Take a colleague
MR CLEVER

18. @laura_yarrowDifficult user testing participants 18
Thanks, any questions?
laurayarrow@gmail.com
Email:
Twitter:
@laura_yarrow
All images courtesy of www.mrmen.com