Donna TedescoFiona TranquadaExpecting the Unexpected:Preparing for SuccessfulUser Research SessionsUXPA Boston 2013
How did weget here?
Identify what you can dobefore your study to: Reduce the chances ofencounteringannoying, tricky, and stickysituations – e...
Frequent Participant is late or a no-show Participant not thinkingaloud Participant is unable tocomplete a necessary ta...
 Recruit Study Plan Product/Space/Technology Observers YourselfWhat We’ll Cover
Identifyingthe “right”participantsRecruit
 Identify your“dealbreakers” Look for behavior inaddition to role Avoid “giveaway”questionsBe specific with your criteria
 Confirmationletter/email, includingdirections to facility orremote session 1 on 1 Recording and observers If you need...
 Backup – 2 participantsscheduled for 1 timeslot Floater – 1 participantscheduled to cover multipletimeslots Incrementa...
Establishingpriorities andmakingbackup plansStudy Plan
 Think about how you canadjust your plan if youencounter: Time constraints Misrecruits Personality challenges Technic...
 Reiterate the points coveredin your consent form: Recording/observers How data will be used Goals for the session Ta...
Avoidingtechnical andenvironmentalchallengesProduct,Space, &Technology
Prototype “Illusion of functionality” Agree on if/when any updateswill be made to the prototypeActual “live” product Be...
 Walk through your full setup If participants are remote, do awalkthrough with them as well Document everything!Include...
 Emergency exits Restrooms Kitchen / Vending areas Security / Emergencyprocedures Printers / Copiers Climate control...
Establishingguidelinesfor behaviorandinteractionsObservers
 Establish a code of conduct Remember – you’re responsible for theparticipant’s well-being! Share with observers ahead ...
Core ground rules to include: Keep quiet; no laughing at or making fun of participant! Keep participant’s identity confi...
 May not exactly follow study plan May let participant follow tangents Assists and diversions When you might use them...
 Make sure you always haveat least 1 person observing Avoid going to someone’sspace without 1 otherperson “Safe word” t...
ImprovingyourtechniqueYourself
 Ethical obligations Your attitude Your body language What you say Your toneReview good moderating behavior
 Helpful if you’re: New to moderating Running a new kind ofresearch Expect certain behaviors Trying to adjust your st...
Examples of what you might jot down: Wait 3 seconds before responding to a participant Neutral acknowledgements (“Mmhmm”...
 Watch recordings of yourself Practice with a colleague Have your colleague “role-play” problematic situations Ask col...
 Set expectations With participant With observers Dive deep Your space Your technology Your study plan Create back...
Thank you!Donna Tedesco (donna.tedesco@gmail.com)Fiona Tranquada (ftranquada@gmail.com)Our (forthcoming) book: @ModSurviva...
Upcoming SlideShare
Loading in …5
×

Expecting the Unexpected: Preparing for Successful User Research Sessions (Donna Tedesco & Fiona Tranquada, UXPA Boston 2013)

714 views

Published on

UXPABOS13

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
714
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Welcome everyone. What we want to talk about today is preparing for successful user research sessions. This came up as we were writing a book – a “survival guide” - on how to handle tricky situations that come up while moderating. There are steps that you can take ahead of time that will help prevent – or mitigate – those tricky situations.  Specifically, how to prepare keeping in mind that the “unexpected” situations that we talk to each other about when sharing horror stories aren’t really as unexpected or infrequent as we think. It’s not enough to think that you can handle complications on the fly, which experienced and inexperienced moderators are prone to do. You need to anticipate what might happen and plan accordingly. Think of it like preparing for a trip. You create an itinerary but you should also make sure your insurance covers helicopter evacuations.   
  • Hope for the best, but prepare for the worst!All types of user research sessions – different methods, different goals (qualvs quant), remote or in-person, at your location or on-site
  • Very simple to build on your regular study preparation so you can minimize the effect of infrequent (unexpected) events.
  • Remember – we’re building on established good practices for each of these with a specific focus on avoiding problems. Consider this a call to arms to treat all of these more seriously – no more complacency!
  • What you’ll avoid: participants who are not qualified (who will waste your time), participants who you can’t understandTalk with your stakeholders ahead of time about the criteria and dealbreakersA good way to identify behavior that you need – in addition to talking to stakeholders – is talking to a target usersTalk with your recruiter, especially if recruiting internal employees! (e.g., representative users not top performers). Understand how they work and the process they use to find participants – you need to trust them. Giveaway questions – instead of multiple choice or yes/no, try to use free form (“tell me about the last item you purchased on Amazon” instead of “Was the last item you purchased on Amazon a book?”)
  • What you’ll avoid: participants who expect a different kind of session (like focus group) and may not show up because of that, or who bring someone, or are unprepared Just like you are inviting someone to attend an event (you’d tell them time, location, dress code, etc.)If internal employee, their direct team will not be observingEXAMPLE: Double-jacking w/call center, need to ask them ahead of time to have the special jacks availableOr remote participant, have a hands free device
  • What you’ll avoid: empty timeslots, not enough data, unhappy stakeholdersYou can put together an activity or survey for these backup/floater participants so you’re able to get some feedback
  • Study plan -> moderator’s guide, interview script – any level of formality
  • What you’ll avoid: (list above) + technical/environmental difficultiesHow will you adjust your study plan, based on your research goals, if you encounter…Time constraints (Late participant; Slow/through participant)Misrecruit (Unqualified participant; Unprepared participant)Personality challenges (Reluctant or obstinate participant)Technical difficulties (Uncooperative technology)Adjustments should be based on your research goals. E.g., quantitative. Work with your stakeholders to prioritize. Know which tasks/questions can be skipped, or moved around. Don’t be afraid to adjust or even end the session early depending on what’s happening and your participant’s comfort level
  • What you’ll avoid/limit: confusion over your behavior, frustration about limited prototype functionalityAdjust this per study – don’t just use the same template over and over againVerify participant qualifications early in the session – you can fit it naturally into the briefing so it comes across as less of an info dump and more of a conversationYou can refer back to this throughout the session
  • Help avoid: Unexpected prototype/product changes; participant frustration with a prototype’s limitations; technical difficultiesPrototype – we all have limited time (especially Agile), so illusion can be important.Also set expectations at beginning of session that everything may not be working. Updates to prototype – even if Agile/rapid, avoid changes in the middle of a session (will confuse participant and you)Live Products:Try to avoid updates during your research (if you are in-house)A/B tests – either turn off, or find way to always see same version if you need consistency
  • Avoid: Wasted time spent troubleshooting in the middle of a sessionHelpful if anything goes wrong in the middle of a session, or if multiple people are moderatingBackup equipment – e.g., if you’re going on-site and need to be prepared for different types of setupsIf you have time, ask someone on your team to “break” something in your setup as a troubleshooting exercise
  • Environmental, e.g., smell levels at animal testing facility, or wearing closed-toe hard soled shoes at manufacturing facility
  • Observers – whether stakeholders, team members, etc. Anyone who will be watching the session!
  • Avoid: interrupting/misbehaving observersCONTINUES on next slide with examples of ground Hold observers accountable: Remember your obligations to the participant and to the protection of your organization
  • These will vary depending on your study goalsIdeally discuss these in person with your observers/stakeholders ahead of time, in personAvoid: Confusion over your behavior, interruptions
  • Person observing can be stakeholder or team member, as long as they can be paying attentionLiability issues!Avoid running research on your own!
  • ….Because sometimes, no matter what preparation you do, things will go wrong
  • Participant comfort comes first!Your attitudeBe neutral but kind and respectfulNot their buddy!Take responsibilityYour body languagePay attentionBe consistentWhat you sayLet participant talkAvoid leading questionsYour toneCurious and interestedNot defensive
  • Expect certain behaviors – e.g., if you need a participant to attempt every task but expect to run short of time, you might write down a way to smoothly transition them to the next taskTrying to adjust your style – e.g., if you’re trying to train yourself to avoid/include moderating behaviors such as giving the participant more time to answer a questionTONE is importantDepending on your goals, you may want different kinds of things jotted down (e.g., if you suspect that you’ll need to provide assists, you may want to write down wording for how you’d do this)EXAMPLES on next page
  • TONE is importantDepending on your goals, you may want different kinds of things jotted down (e.g., if you suspect that you’ll need to provide assists, you may want to write down wording for how you’d do this)
  • Expecting the Unexpected: Preparing for Successful User Research Sessions (Donna Tedesco & Fiona Tranquada, UXPA Boston 2013)

    1. 1. Donna TedescoFiona TranquadaExpecting the Unexpected:Preparing for SuccessfulUser Research SessionsUXPA Boston 2013
    2. 2. How did weget here?
    3. 3. Identify what you can dobefore your study to: Reduce the chances ofencounteringannoying, tricky, and stickysituations – especially theones you don’t usually expect– during a session Set yourself up tosuccessfully handle thosesituations when they occurOur Goal
    4. 4. Frequent Participant is late or a no-show Participant not thinkingaloud Participant is unable tocomplete a necessary taskInfrequent Participant makes you feeluncomfortable Participant frustrated byprototype limitations Observer interrupts thesession Natural disasterWhat do we mean by “unexpected”?Being prepared will help you with both types of situations!
    5. 5.  Recruit Study Plan Product/Space/Technology Observers YourselfWhat We’ll Cover
    6. 6. Identifyingthe “right”participantsRecruit
    7. 7.  Identify your“dealbreakers” Look for behavior inaddition to role Avoid “giveaway”questionsBe specific with your criteria
    8. 8.  Confirmationletter/email, includingdirections to facility orremote session 1 on 1 Recording and observers If you need them to bring anything to do anything during thesession (e.g., watch themprocess payroll) to prepareSet expectations about the session
    9. 9.  Backup – 2 participantsscheduled for 1 timeslot Floater – 1 participantscheduled to cover multipletimeslots Incremental cost but maybe worth it if you have high-profile observers If you don’t have backup orfloaters, plan for no shows!Schedule backup or floater participants
    10. 10. Establishingpriorities andmakingbackup plansStudy Plan
    11. 11.  Think about how you canadjust your plan if youencounter: Time constraints Misrecruits Personality challenges Technical difficulties Create a backup such as apaper survey orquestionnairePrioritize your tasks/questions
    12. 12.  Reiterate the points coveredin your consent form: Recording/observers How data will be used Goals for the session Take a break or end the sessionat any time Set clear expectations for: How the session time will beused What they’ll be using (e.g., aprototype) Your behavior (e.g., may notanswer questions) Verify qualificationsPrepare the participant briefing
    13. 13. Avoidingtechnical andenvironmentalchallengesProduct,Space, &Technology
    14. 14. Prototype “Illusion of functionality” Agree on if/when any updateswill be made to the prototypeActual “live” product Be aware of the productschedule Research any A/B tests Have a backup if necessary(e.g., screenshots) or setappropriate expectationsPrepare the Product
    15. 15.  Walk through your full setup If participants are remote, do awalkthrough with them as well Document everything!Include screenshots! Order of operations Lab/space setup Recording Screen sharing / Audioconferencing Controls for observers (Mute All) “Kick out” options Prototype/Product URLs Login/Password Paths Any known issues (e.g., browserincompatibilities) Any backup equipment Technical support contactDocument your setup
    16. 16.  Emergency exits Restrooms Kitchen / Vending areas Security / Emergencyprocedures Printers / Copiers Climate controls If going off-site, Equipment prohibitions Seating (e.g., if you haveobservers) Internet access Noise levels Other environmental factorsExplore your research space
    17. 17. Establishingguidelinesfor behaviorandinteractionsObservers
    18. 18.  Establish a code of conduct Remember – you’re responsible for theparticipant’s well-being! Share with observers ahead of time Via email Poster on the wall of research space Printouts Hold observers accountableProvide ground rules
    19. 19. Core ground rules to include: Keep quiet; no laughing at or making fun of participant! Keep participant’s identity confidential Avoid slamming doors If observers are remote, Keep phone muted Avoid in-meeting chat feature If observers are in the room with you, Avoid multitasking Be careful with body language What to do if they have questions or want to communicate withyou/participantProvide ground rules cont.
    20. 20.  May not exactly follow study plan May let participant follow tangents Assists and diversions When you might use them How to tell if that’s what you’re doing May pretend to not know an answerCommunicate your approach
    21. 21.  Make sure you always haveat least 1 person observing Avoid going to someone’sspace without 1 otherperson “Safe word” to indicatewhen you need assistanceEstablish safety precautions
    22. 22. ImprovingyourtechniqueYourself
    23. 23.  Ethical obligations Your attitude Your body language What you say Your toneReview good moderating behavior
    24. 24.  Helpful if you’re: New to moderating Running a new kind ofresearch Expect certain behaviors Trying to adjust your style Include in your study plan –don’t be afraid to put it onevery pageJot down key phrases or reminders
    25. 25. Examples of what you might jot down: Wait 3 seconds before responding to a participant Neutral acknowledgements (“Mmhmm”, “Uh-huh”) Turn question around “If you were doing this at home, and I wasn’t sitting there next toyou, what would you do?” Prompt non-responsive participant “What are you trying to do right now? Reassure "This is just the kind of feedback we want to hear..." Redirect or cut short "Im sorry, I just want to interrupt you here for a second. For the sake oftime I’d like you to return to the task/question, and we can revisit thistopic at the end of the session if time permits.Jot down key phrases or reminders cont.
    26. 26.  Watch recordings of yourself Practice with a colleague Have your colleague “role-play” problematic situations Ask colleagues to watch you moderate and provide feedback Watch others moderatePractice!
    27. 27.  Set expectations With participant With observers Dive deep Your space Your technology Your study plan Create backups For your technology For your participants For your product Practice Your technical setup Your moderatingPreparation Takeaways
    28. 28. Thank you!Donna Tedesco (donna.tedesco@gmail.com)Fiona Tranquada (ftranquada@gmail.com)Our (forthcoming) book: @ModSurvivalUXQuestions?

    ×