Recruiting Better User Research Participants: UPA 2011
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Recruiting Better User Research Participants: UPA 2011

  • 3,316 views
Uploaded on

 

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,316
On Slideshare
3,308
From Embeds
8
Number of Embeds
3

Actions

Shares
Downloads
67
Comments
0
Likes
4

Embeds 8

http://www.linkedin.com 5
https://www.linkedin.com 2
http://twitter.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Failing to recruit participants or recruiting the wrong participants dooms everything that comes afterwards.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Someone has to do the difficult job of finding, contacting, screening, and scheduling the participants. You can do it yourself, leave it up to your clients, or hire a recruiting company. There are advantages and disadvantages for each of these approaches.
  • Makes the most sense when you have easy access to a list of potential participants. When you’re recruiting your fellow employees or customers.
  • Makes the most sense when you have easy access to a list of potential participants. When you’re recruiting your fellow employees or customers.
  • Give you more credibility if you have a connection to these people - fellow employees, they are your customers, you are a member
  • People visiting your website are often the exact people you want to reach. The intercept leads to a screening questionnaire. You can either contact those volunteers immediately or they are added to a database of participants to contact later. This is far less expensive over time than using a recruiting company.Can continue to recruit participants over time
  • If your client has the relationship with employees, customers, members; have him/her do the recruiting to take advantage of that relationship.
  • Client not good at recruiting They’re not going to go through a complicated screenerToo many or too few participants of a particular type Recruiting inappropriate participants - Bob, the super user, because he knows the most about the systemYour client may not be good at scheduling participants in the desired timeframe, delaying the study or spreading it out over a longer time period.
  • Clients need guidance of who to recruit and who not to recruit. Otherwise they may not recruit the right people.Screeners are too complicated for them.
  • Schedule will constrain the sessions to a reasonable time frame.Otherwise the sessions may be too spread out over time.Emphasize the need to not schedule sessions back to back, leave enough time for breaks, meals, travel between locations, etc.
  • To prevent the client from passing on incorrect information, give them the email text to use.
  • Greater chance that you’ll get people who lie to get into the study just to get the money. May not care about your research.
  • Some recruiting companies specialize in certain types of participantsGood to have several recruiting companies to use - otherwise you tend to scrape the bottom of the barrel if you use the same company all the time
  • Since you’re having someone else do the recruiting, a screener is essential.
  • The recruiting company should send you periodic updates on the participants are recruited.
  • The screener is one of the most important documents in a research project. Yet it’s often the one that’s most taken for granted. Few books or schools detail how to write an effective screener, and as a result we often learn to write them through trial and error, rely on screeners created by the marketing department, or reuse screeners from previous projects without considering whether they are still appropriate.
  • Because people will assume you’re a telemarketer, you need to get right to the point and quickly establish that you’re not selling anything. Mentioning the money up front gets attention and keeps people on the phone.
  • More questions just add to the burden on the recruiter and the participants
  • Recruiting companies sometimes rely too heavily on their participant databases before calling “fresh” participants. Many are great participants, but others volunteer a little too often. They may be supplementing their income as a professional research participantRecruiting companies should first use their database to screen out the participants they will call based on your criteria.
  • Recruiting companies sometimes rely too heavily on their participant databases before calling “fresh” participants. Many are great participants, but others volunteer a little too often. They may be supplementing their income as a professional research participantRecruiting companies should first use their database to screen out the participants they will call based on your criteria.
  • If you’re using a recruiter, you have to rely on their ability to correctly judge the response and whether it fits these categories.
  • [Ask the audience to try to guess the answer that will get them into the study as a participant.]
  • If you’re using a recruiter, they have to have a good ability to interpret the person’s answer to know which to terminate
  • Sometimes you do need to ask multiple choice questions in which you read the choices to the person.
  • Because the middle three answers are so specific, they seem like safe choices to avoid being eliminated. It’s obvious that “12 or less” and possibly “31 or more” are the cutoff points. The wording “12 or less” and “31 or more” implies that if you watch that little or that much TV, the recruiter doesn’t even care about the exact number of hours, and you’ll be eliminated.
  • Sure, I’m “considering”. Maybe, who knows. I guess I might need to buy a car, so I qualify.
  • “Professional” participants play it safe by selecting all or most of the answers, knowing that there’s one expected answer that will get them in the study.
  • Sure, you may eliminate some legitimate people, but you’ll also eliminate cheaters. So be careful to make it a reasonable limit. A first time home buyer might reasonably buy many of these items.
  • Bring in their credit card from a particular bankBring in their iPadGraceful exit allows people to save face and screen themselves out instead of remaining with the lie and allowing themselves to be scheduled, knowing they won’t actually show up
  • Some clients want to get a distribution of demographics. That doesn’t really matter, but if needed, put those questions at the end.Unless it’s a dating site, in many cases marital status doesn’t matter.
  • Use personas or user profiles to determine the behaviors and attitudes
  • You don’t want people with a strong bias against your client
  • You can interpret attitudes towards Japanese cars from the answers to this question.
  • You can interpret attitudes towards Japanese cars from the answers to this question.
  • Requires you to first think of the average hours per day and then multiply by days per week
  • It’s easier to think of hours per day. No mental math required.
  • Information gathering only adds to the length of the screening process.
  • Ensure that the participants match the computer and web experience of your user groups. For example, in a usability test, you don’t want to confuse interface problems with problems that new computer users face in general. Similarly, you often don’t want to test with web developers or designers.
  • In self-assessments, some people will inflate their self-ratings, but this will help eliminate those that truly feel uncomfortable with computers.Eliminates people who show up and get nervous, “I’m not good with computers.”Self-assessments are very subjective, and some people are reluctant to admit their inexperience.
  • Be specific. For example, I added “without assistance” to the following question after we recruited a woman who later said she used the Web to read her email by having her niece use the computer to read the email to her.
  • It’s great to have eloquent participants who can easily think aloud and express their thoughts and opinions. It requires a lot of work to drag useful information out of quiet participants.Question should be related to your topic because people won’t always be talkative about subjects they are not interested in.
  • Real user groups include both expressive and quiet people. Including some of each type of participant will give you a more representative group and still give you enough interesting video clips.
  • These things sound so obvious, that you forget to mention them until a participant comes in who can’t see the screen.I once had a woman who didn’t bring her glasses and asked, “Can’t you just read what’s on the screen to me?”
  • Sure, we give them this information at the beginning of the session, but most people don’t feel able to back out at that point.Many participants assume they will be part of a focus group, because that is what recruiting companies most commonly recruit for.
  • Reveals holes and logical inconsistenciesA good recruiting company reveals logic problems and when your screener is too strict and may result in no participants
  • There will always be no-shows, but there are steps you can take to make it more likely that your participants will show up for their sessions.
  • Obviously, the more convenient the location of your research, the more likely your participants will attend.
  • Conduct your research sessions at times convenient to the participants.
  • Tuesdays through Thursdays tend to be the best days for weekday research
  • Let’s face it, for most people it’s the money that encourages them keep their commitment to attend a research session. If you don’t offer the right incentive for the effort involved, your no-show risk increases. For example, employees using poorly designed business systems are often grateful that someone is finally asking for their opinions. The chance to improve the application is incentive enough.
  • Recruit backup participants to replace no-shows and problem participants. Two approaches to securing backup participants are using floaters and over-recruiting.
  • To completely eliminate risk, each participant can be covered by a floater. However, this is very expensive. Instead, you can balance cost and risk by having each floater cover multiple sessions. This means you may run out of participants if you use the floater and then don’t have a floater to cover another no-show participant, but that’s the risk you take.
  • To completely eliminate risk, each participant can be covered by a floater. However, this is very expensive. Instead, you can balance cost and risk by having each floater cover multiple sessions. This means you may run out of participants if you use the floater and then don’t have a floater to cover another no-show participant, but that’s the risk you take.
  • To completely eliminate risk, each participant can be covered by a floater. However, this is very expensive. Instead, you can balance cost and risk by having each floater cover multiple sessions. This means you may run out of participants if you use the floater and then don’t have a floater to cover another no-show participant, but that’s the risk you take.
  • Phone screening is not economical or practical since people have to have a link to click to get to the study.
  • Emailing makes the most sense if you have a list of potential participants.
  • Emailing a link to your study is an effective method if you can easily identify and obtain email addresses of potential participants. However, emailing a large number of people can be time consuming.
  • For example, if you want 100 participants to complete the study, email 2,000 people (assuming a five-percent-response rate).
  • Some unmoderated tools allow you to end the study when it reaches a maximum number of participants. This can help you avoid paying too much in incentives.
  • Save the study instructions until people get into the study itself.
  • If there are non-monetary incentives, appeal to the benefits of participating. Your participation will help make the application you use everyday more efficient and better suited to your needs.
  • They add up and it can be time consuming to send checks to every participant.
  • It can be more enticing and less expensive.It’s easier than having to send individual incentives to everyone.Beware of legal issues around contests and sweepstakes. Consult a lawyer.
  • For example, an Xbox appeals much more to some people than others, which will affect the types of people who enter.
  • - Although mass emails require less work, individual emails are more effective at establishing trust and are therefore less likely to be ignored.
  • Few people will contact you, but you show you’re putting your information out there for people to check you out. You’re willing to share with them.
  • People are wary of clicking links in emails from people they don’t know. Emails from a familiar person or organization seem more credibleOr name drop someone the person is familiar with and provide their contact information
  • This usually results in additional participants. If you still need more participants, email a new set of people that you haven’t yet contacted.
  • If you think you’ll have difficulty finding participants on your own, an online panel is a good option.
  • Provided by many recruiting companies
  • Provided by many recruiting companies
  • You don’t need to provide a screener to a panel recruiter, as you would with a moderated study.
  • They are more tolerant of additional questions. They are more familiar with what’s expected.
  • For example, if 1,000 people are invited, and 100 complete the study, you will only be charged for those 100 people. The panel company handles the incentives. You just pay one fee.
  • Recruiting is done automatically It can continue indefinitely
  • Don’t make the screening too long though. That leads to abandonment.
  • Intercepts reach “real users” who are already on the website for a reason with real goals and motivations
  • Less motivation to complete the study – People agree to them on a whim. No personal email from someone they know, not part of a panel and wanting to be invited for future studies
  • Discussion pointsWhat do you find are the most difficult aspects of recruiting?Are there other good methods to get participants?Other recruiting tips or advice?

Transcript

  • 1. 23 June 2011
    1
    Recruiting Better User Research Participants
    Recruiting Better User Research Participants
    Principal of Research & Strategy
    Electronic Ink
    UPA 2011
    23 June, 2011
  • 2. Why talk about recruiting?
    23 June 2011
    Recruiting Better User Research Participants
    2
    User research is only as good as the participants you recruit.
    It’s difficult to recruit participants who …
    • Are representative
    • 3. Can express themselves and provide useful information
    • 4. Show up
  • Why talk about recruiting?
    23 June 2011
    Recruiting Better User Research Participants
    3
    Have you ever …
    • Recruited someone who didn’t fit the profile or was just odd?
  • Why talk about recruiting?
    23 June 2011
    Recruiting Better User Research Participants
    4
    Have you ever …
    • Recruited someone who lied to get into the study and was just in it for the money?
  • Why talk about recruiting?
    23 June 2011
    Recruiting Better User Research Participants
    5
    Have you ever …
    • Recruited people who couldn’t express themselves?
  • Why talk about recruiting?
    23 June 2011
    Recruiting Better User Research Participants
    6
    Have you ever …
    • Had too many no shows and cancellations?
  • Why talk about recruiting?
    23 June 2011
    Recruiting Better User Research Participants
    7
    What other recruiting problems have you had?
  • 5. What we’ll talk about
    23 June 2011
    Recruiting Better User Research Participants
    8
    Recruiting for moderated research
    • Who should do the recruiting?
    • 6. Writing a better screener
    • 7. Minimizing no-shows
    • 8. Recruiting backup participants
    Recruiting for unmoderated, remote research
    • Emailing participants
    • 9. Using an online panel
    • 10. Using online intercepts
  • Recruiting for Moderated Research.
    23 June 2011
    Recruiting Better User Research Participants
    9
  • 11. Recruiting for Moderated Research.
    23 June 2011
    Recruiting Better User Research Participants
    10
    Who should do the recruiting?
  • 12. Who should do the recruiting?
    23 June 2011
    Recruiting Better User Research Participants
    11
    Recruiting options
    You
    Your client
    A recruiting company
  • 13. Recruiting for Moderated Research.
    23 June 2011
    Recruiting Better User Research Participants
    12
    Doing the recruiting yourself
  • 14. Doing the recruiting yourself
    23 June 2011
    Recruiting Better User Research Participants
    13
    Advantages
    You know what you’re looking for.
    You have more control over who is recruited.
    You sound more human and less like a telemarketer.
    You save money.
  • 15. Doing the recruiting yourself
    23 June 2011
    Recruiting Better User Research Participants
    14
    Disadvantages
    It’s difficult if you don’t already have a list to work from.
    It’s time consuming to find, contact, screen, and schedule.
    It may require a lot of cold calling.
  • 16. Doing the recruiting yourself
    23 June 2011
    Recruiting Better User Research Participants
    15
    Start with lists of potential participants - customer lists, membership lists, employee directories, etc.
  • 17. Or gather participants from your website with a recruiting page or an online intercept.
    Doing the recruiting yourself
    23 June 2011
    Recruiting Better User Research Participants
    16
  • 18. Or advertise or connect through social media and other sources.
    Doing the recruiting yourself
    23 June 2011
    Recruiting Better User Research Participants
    17
  • 19. Recruiting for Moderated Research.
    23 June 2011
    Recruiting Better User Research Participants
    18
    Having your client do the recruiting
  • 20. Having your client do the recruiting
    23 June 2011
    Recruiting Better User Research Participants
    19
    Advantages
    Clients have access to employee, customer, and member lists.
    Employees, customers, or members are more likely to volunteer.
    It saves you money and time.
  • 21. Having your client do the recruiting
    23 June 2011
    Recruiting Better User Research Participants
    20
    Disadvantages
    Clients may not be very good at recruiting.
    Clients may not possess the time and organizational skills required.
    Clients often pass on vague or incorrect information about the research.
  • 22. Having your client do the recruiting
    23 June 2011
    Recruiting Better User Research Participants
    21
    Provide your client with detailed instructions.
    Profile of who to recruit
    And who not to recruit
  • 23. Having your client do the recruiting
    23 June 2011
    Recruiting Better User Research Participants
    22
    Provide your client with detailed instructions.
    Schedule of sessions
  • 24. Having your client do the recruiting
    23 June 2011
    Recruiting Better User Research Participants
    23
    Provide your client with detailed instructions.
    Description of the study to give to participants
  • 25. Having your client do the recruiting
    23 June 2011
    Recruiting Better User Research Participants
    24
    Ensure they do it correctly.
    Delegate to someone with time and organizational skills.
    Monitor the recruited participants.
    Email the scheduled participants to introduce yourself and provide more details.
  • 26. Recruiting for Moderated Research.
    23 June 2011
    Recruiting Better User Research Participants
    25
    Using a recruiting company
  • 27. Using a recruiting company
    23 June 2011
    Recruiting Better User Research Participants
    26
    Advantages
    It saves you a lot of time and effort.
    It’s a good option when you have no existing lists or other resources.
    With the right information, a good recruiting company can get the best participants.
  • 28. Using a recruiting company
    23 June 2011
    Recruiting Better User Research Participants
    27
    Disadvantages
    It’s the most expensive recruiting option.
    You might get “professional” participants who are only in it for the money.
    With poor direction or a bad recruiting company, you can get the worst participants.
  • 29. Using a recruiting company
    23 June 2011
    Recruiting Better User Research Participants
    28
    Selecting a recruiting company
    Get referrals from other researchers.
    Get quotes from several recruiting companies - cost and confidence.
    Look for good communication.
    Cultivate several recruiting companies you can use regularly.
  • 30. Using a recruiting company
    23 June 2011
    Recruiting Better User Research Participants
    29
    A strong and clearly written screener is essential.
  • 31. Using a recruiting company
    23 June 2011
    Recruiting Better User Research Participants
    30
    Review the list of recruited participants.
    Ensure they fit what you’re looking for.
    Have your client review and approve the list also - CYA!
  • 32. Recruiting for Moderated Research.
    23 June 2011
    Recruiting Better User Research Participants
    31
    Deciding who will do the recruiting
  • 33. Deciding who will do the recruiting
    23 June 2011
    Recruiting Better User Research Participants
    32
    Do your own recruiting when:
    You have more time than money.
    You don’t mind or are effective at cold calling.
    You can hire someone else to do the recruiting – intern, staff recruiter.
    You can use existing lists, recruit from your website, or advertise.
    You already have a connection with potential participants.
  • 34. Deciding who will do the recruiting
    23 June 2011
    Recruiting Better User Research Participants
    33
    Have your client do the recruiting when:
    Your client has detailed contact lists of employees, customers, or members.
    Your client already has a connection with potential participants.
    Your client has the time and coordination to do this effectively.
    Your client has successfully done recruiting in the past.
  • 35. Deciding who will do the recruiting
    23 June 2011
    Recruiting Better User Research Participants
    34
    Hire a recruiting company when:
    You have more money than time.
    You don’t want to “cold call” and screen participants.
    You don’t have an easy way to find potential participants.
  • 36. Recruiting for Moderated Research.
    23 June 2011
    Recruiting Better User Research Participants
    35
    Writing a better screener
  • 37. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    36
    Keep people from hanging up on you
    Quickly get to the point.
    You’re not selling anything.
    You’re recruiting for a paid study.
  • 38. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    37
    Ask likely elimination questions first
    Only the most promising will go through all the questions.
  • 39. Eliminate the usual suspects
    Eliminate recent participants.
    How would you answer this question if you wanted to get into the study?
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    38
  • 40. Eliminate the usual suspects
    “Professional” participants might lie.
    Recruiters should check their records before calling people.
    “A few years ago.”
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    39
  • 41. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    40
    Eliminate conflicts of interest
    Existing relationship with your client - employee, employee of a competitor, etc.
    Too much inside knowledge - web designer, developer, marketer, etc.
  • 42. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    41
    Avoid reading multiple choice answers
    The “correct” answer is often obvious
    How would you answer this question if you wanted to get into the study?
  • 43. Avoid reading multiple choice answers
    The “correct” answer is often obvious
    “None of these”
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    42
  • 44. Ask open-ended questions instead of reading the answers
    How would you answer this question if you wanted to get into the study?
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    43
  • 45. Ask open-ended questions instead of reading the answers
    Requires the recruiter to know which professions should terminate the screening
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    44
  • 46. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    45
    Don’t make the elimination answers obvious
    How would you answer this question if you wanted to get into the study?
  • 47. Don’t make the elimination answers obvious
    “19 – 24” seems like a safe answer.
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    46
  • 48. Include multiple elimination answers.
    How would you answer this question if you wanted to get into the study?
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    47
  • 49. Include multiple elimination answers
    It’s not obvious which answers will get you eliminated.
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    48
  • 50. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    49
    Avoid giving away what you’re looking for
    How would you answer this question if you wanted to get into the study?
  • 51. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    50
    Avoid giving away what you’re looking for
    “Yes”
  • 52. Avoid giving away what you’re looking for
    Providing many choices obscures what you’re looking for
    How would you answer this question if you wanted to get into the study?
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    51
  • 53. Avoid giving away what you’re looking for
    Providing many choices obscures what you’re looking for
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    52
  • 54. Avoid giving away what you’re looking for
    Beware of those who choose everything to try to get into the study.
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    53
  • 55. Avoid giving away what you’re looking for
    Set a reasonable limit to eliminate fakers.
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    54
  • 56. Avoid giving away what you’re looking for
    Use open-ended questions to completely hide what you’re looking for.
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    55
  • 57. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    56
    Ask questions that only qualified participants can answer
  • 58. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    57
    Ask questions that only qualified participants can answer
  • 59. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    58
    Require proof
    Require participants to provide proof of qualification.
    Provide a graceful exit, so that people don’t have to be caught in a lie.
  • 60. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    59
    Recruit based on behavior and attitudes
    Most demographics aren’t very important for small-scale research.
  • 61. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    60
    Recruit based on behavior and attitudes
    Screen for desired behaviors.
  • 62. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    61
    Recruit based on behavior and attitudes
    Negative attitudes towards your client or subject matter may disqualify people.
  • 63. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    62
    Recruit based on behavior and attitudes
    Screen for desired attitudes.
  • 64. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    63
    Recruit based on behavior and attitudes
    Interpret attitudes and behaviors based on answers.
    What do you think we’re looking for here?
  • 65. Recruit based on behavior and attitudes
    Interpret attitudes and behaviors based on answers.
    People interested in Japanese cars
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    64
  • 66. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    65
    Don’t ask questions that are difficult to answer
    Can you answer this question easily and accurately?
  • 67. Don’t ask questions that are difficult to answer
    Estimating the average number of hours per day is easier.
    Is this question easier to answer?
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    66
  • 68. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    67
    Don’t use the screener to gather information
    Only include questions that screen participants.
    Save information gathering for a pre-study questionnaire.
  • 69. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    68
    Screen for computer, Web, and technology experience
    You may want to eliminate those with very low and very high experience.
  • 70. Screen for computer, Web, and technology experience
    Self-assessments allow those who are truly uncomfortable with technology to excuse themselves from the study.
    But don’t rely only on self-assessments.
    Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    69
  • 71. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    70
    Screen for computer, Web, and technology experience
    Make your own assessment based on activities and behaviors.
  • 72. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    71
    Screen for eloquence
    Ask an open-ended question related to your topic to judge expressiveness.
  • 73. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    72
    Screen for eloquence
    But be careful not to eliminate qualified, quiet participants.
    “Normal” people may not be as entertaining as “professional” participants.
    Recruit representative participants, not only those who speak in perfect sound bites.
  • 74. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    73
    Ensure that people are physically able to participate
  • 75. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    74
    Let people know what they’ll be doing
    Eliminate surprises and potential discomfort.
    Allow those who would be uncomfortable to back out.
  • 76. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    75
    Test your screener with a colleague
    Ask a colleague to try to fake his/her way through the screener.
    Recruiting company should point out problems in logic.
  • 77. Writing a better screener
    23 June 2011
    Recruiting Better User Research Participants
    76
    Get your client to approve the screener – CYA!
    Your client becomes responsible for the types of people recruited.
    It’s difficult for them to complain later about the quality of the participants.
  • 78. Recruiting for Moderated Research.
    23 June 2011
    Recruiting Better User Research Participants
    77
    Minimize No-Shows
  • 79. Minimize No-Shows
    23 June 2011
    Recruiting Better User Research Participants
    78
    Locate your research conveniently.
    Where are your participants usually located during your session times?
    How easy is it to get to and find your location?
    Traffic? Public transportation? Parking?
    Is it more convenient to go to your participants or conduct remote sessions?
  • 80. Minimize No-Shows
    23 June 2011
    Recruiting Better User Research Participants
    79
    Schedule your research conveniently
    What are your ideal participants usually doing during your session times?
    Schedule a variety of session times.
    Who’s most available for weekday research?
  • 81. Minimize No-Shows
    23 June 2011
    Recruiting Better User Research Participants
    80
    Avoid days and conditions that result in more no-shows
    Fridays
    Days before holidays
    Bad weather – snowstorms, rainstorms, etc.
  • 82. Minimize No-Shows
    23 June 2011
    Recruiting Better User Research Participants
    81
    Provide appropriate incentives.
    Make it worth their while to show up.
    Make it appropriate to the person and to the activity.
  • 83. Minimize No-Shows
    23 June 2011
    Recruiting Better User Research Participants
    82
    Contact participants to confirm
    Remind them to attend.
    Give directionsand parking / transportation information.
    A good way to receive early warning of no-shows and get a head start on recruiting replacements
  • 84. Recruiting for Moderated Research.
    23 June 2011
    Recruiting Better User Research Participants
    83
    Recruit Backup Participants
  • 85. Recruit Backup Participants
    23 June 2011
    Recruiting Better User Research Participants
    84
    Two approaches to backup participants
    Floaters
    Over-recruiting
    Floaters
    Over-recruiting
  • 86. Recruit Backup Participants
    23 June 2011
    Recruiting Better User Research Participants
    85
    Floaters
    Wait on standby to fill in for no-shows or inappropriate participants
  • 87. Recruit Backup Participants
    23 June 2011
    Recruiting Better User Research Participants
    86
    Floaters
    Wait on standby to fill in for no-shows or inappropriate participants
  • 88. Recruit Backup Participants
    23 June 2011
    Recruiting Better User Research Participants
    87
    Floaters
    Wait on standby to fill in for no-shows or inappropriate participants
  • 89. Recruit Backup Participants
    23 June 2011
    Recruiting Better User Research Participants
    88
    Over-recruiting
    Recruit several extra participants to use for no-shows
    Dismiss and pay those you don’t need
  • 90. Recruit Backup Participants
    23 June 2011
    Recruiting Better User Research Participants
    89
    Over-recruiting
    Recruit several extra participants to use for no-shows
    Dismiss and pay those you don’t need
  • 91. Recruit Backup Participants
    23 June 2011
    Recruiting Better User Research Participants
    90
    Over-recruiting
    Recruit several extra participants to use for no-shows
    Dismiss and pay those you don’t need
  • 92. Recruit Backup Participants
    23 June 2011
    Recruiting Better User Research Participants
    91
    Use floaters when
    You want to be sure your observers always have a participant to observe
    You don’t have time for an extra day of backup sessions
    You don’t have money to pay for an extra day of sessions at a research facility
  • 93. Recruit Backup Participants
    23 June 2011
    Recruiting Better User Research Participants
    92
    Use over-recruiting when
    It’s not important that your observers have a participant to watch for each session
    You have time and money for an extra day of backup sessions
    You want to save money on incentives
    Your participants don’t want to wait around as floaters
  • 94. Recruiting for Unmoderated, Remote Research.
    23 June 2011
    Recruiting Better User Research Participants
    93
  • 95. Recruiting for Unmoderated, Remote Research
    23 June 2011
    Recruiting Better User Research Participants
    94
    What is different about unmoderated, remote research?
    Large numbers of participants from anywhere in the world can participate
    Participants visit a website to complete the study on their own time
    Unmoderated tools automatically capture and present the data
  • 96. Recruiting for Unmoderated, Remote Research
    23 June 2011
    Recruiting Better User Research Participants
    95
    With a large number of potential participants, different methods are required to:
    Find
    Screen
    Contact
    And convince people to participate
  • 97. Recruiting for Unmoderated, Remote Research
    23 June 2011
    Recruiting Better User Research Participants
    96
    Three recruiting methods for unmoderated studies
    Emailing participants
    Online panel
    Online intercept
  • 98. Recruiting for Unmoderated, Remote Research.
    23 June 2011
    Recruiting Better User Research Participants
    97
    Emailing Potential Participants
  • 99. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    98
  • 100. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    99
    Gather a list of potential participants
    Use existing lists of customers, members, employees, etc.
    Recruit from your website
    Advertise
  • 101. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    100
    Narrow the list of participants using existing information.
  • 102. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    101
    Avoid adding screening questions to your study
    Each question adds to the length of the study, increasing abandonment.
  • 103. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    102
    Determine how many people to email
    Determine the minimum number of participants for acceptable results.
    Tullis and Wood showed 30 - 50 is needed for online card sorting. [1]
    Assume a low response rate of 5% and email an appropriately larger population.
    [1] Tullis, Tom, and Larry Wood. “How Many Users Are Enough for a Card-Sorting Study?” Proceedings UPA 2004, Minneapolis, MN, June 7–11, 2004. Retrieved March 7, 2011: http://home.comcast.net/~tomtullis/publications/UPA2004CardSorting.pdf
  • 104. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    103
    Determine how many people to email
    Determine the maximum number of participants to accept.
    Avoid paying too much in incentives.
  • 105. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    104
    Keep it short, briefly describe the study, and provide a link to the study
  • 106. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    105
    Don’t provide the study instructions in the email
  • 107. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    106
    Highlight the incentives, both monetary and altruistic
  • 108. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    107
    Provide appropriate incentives: individual incentives are small but add up.
  • 109. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    108
    Provide appropriate incentives: entry into a drawing is often easier.
  • 110. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    109
    Provide appropriate incentives: with equal appeal to everyone.
  • 111. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    110
    Establish credibility with individual emails in a friendly, professional tone.
  • 112. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    111
    Establish credibility by providing contact information.
  • 113. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    112
    Establish credibility by sending the emails from a familiar person.
  • 114. Emailing Potential Participants
    23 June 2011
    Recruiting Better User Research Participants
    113
    Monitor participation and send reminders
    Monitor the number and type of people who have completed the study.
    If necessary, send reminder emails.
  • 115. Recruiting for Unmoderated, Remote Research.
    23 June 2011
    Recruiting Better User Research Participants
    114
    Using an Online Panel
  • 116. Using an Online Panel
    23 June 2011
    Recruiting Better User Research Participants
    115
    A panel is a database of people who have volunteered for surveys.
  • 117. Using an Online Panel
    23 June 2011
    Recruiting Better User Research Participants
    116
    Panels are provided by many recruiting and market research companies.
  • 118. Using an Online Panel
    23 June 2011
    Recruiting Better User Research Participants
    117
    Panel members are already pre-screened
    You provide the details of the people you’re looking for instead of a screener.
    The recruiter filters the database to find people matching those characteristics.
  • 119. Using an Online Panel
    23 June 2011
    Recruiting Better User Research Participants
    118
    Screening
    You can also add additional screening questions to the beginning of your study.
    Panel participants are motivated to take surveys, so you don’t have to worry as much about extra questions leading to abandonment.
  • 120. Using an Online Panel
    23 June 2011
    Recruiting Better User Research Participants
    119
    You pay the panel company based on the number of “completes.”
    For each person who completes the study, you pay a recruiting fee and small incentive.
    Limit your costs by ending the study after a maximum number of participants.
  • 121. Using an Online Panel
    23 June 2011
    Recruiting Better User Research Participants
    120
    Quality of panel participants
    They want to participate in studies, and they want to be invited back.
    This motivates them to take the study seriously and provide thorough feedback.
  • 122. Using an Online Panel
    23 June 2011
    Recruiting Better User Research Participants
    121
    Quality of panel participants
    They are more likely to be “professional” participants, primarily in it for the money.
    They may not represent your typical user.
  • 123. Recruiting for Unmoderated, Remote Research.
    23 June 2011
    Recruiting Better User Research Participants
    122
    Using Online Intercepts
  • 124. Using Online Intercepts
    23 June 2011
    Recruiting Better User Research Participants
    123
    Online intercepts pop-up on websites to invite people to participate.
  • 125. Using Online Intercepts
    23 June 2011
    Recruiting Better User Research Participants
    124
    Those who accept are taken to a brief screening questionnaire.
  • 126. Using Online Intercepts
    23 June 2011
    Recruiting Better User Research Participants
    125
    Those who pass screening can either be …
    • Added to a participant database
    • 127. Contacted immediately for a remote, moderated study
    • 128. Taken directly into an unmoderated study
  • Using Online Intercepts
    23 June 2011
    Recruiting Better User Research Participants
    126
    Creating an online intercept
    • Some unmoderated tools provide the ability to create intercepts.
    • 129. Otherwise you may need a developer to create the intercept and tie it into the site.
  • Using Online Intercepts
    23 June 2011
    Recruiting Better User Research Participants
    127
    Advantages of online intercepts
    • They recruit “real users” who are already on the website.
    • 130. Automatic screening and recruiting saves time.
    • 131. They can remain on a website indefinitely, continually recruiting for long-term studies.
  • Using Online Intercepts
    23 June 2011
    Recruiting Better User Research Participants
    128
    Disadvantages of online intercepts
    • Most people ignore them and don’t volunteer.
    • 132. Intercepted participants have less motivation to complete the study.
    • 133. Require a lot of site traffic or waiting a long time to get enough participants.
  • Using Online Intercepts
    23 June 2011
    Recruiting Better User Research Participants
    129
    Incentives
    • Providing incentives to each participant is difficult logistically and financially.
    • 134. Using prize drawings may be an easier, less expensive incentive.*
    * Be careful of the legal issues surrounding contests and sweepstakes. Consult a lawyer if needed.
  • 135. Recruiting for Unmoderated, Remote Research.
    23 June 2011
    Recruiting Better User Research Participants
    130
    Choosing Between These Methods
  • 136. Choosing Between These Methods
    23 June 2011
    Recruiting Better User Research Participants
    131
    Email the participants if …
    • You have access to a list of potential participants.
    • 137. You don’t need to do much additional screening.
    • 138. The emails can be sent from someone the participants know.
    • 139. You have more time than money.
  • Choosing Between These Methods
    23 June 2011
    Recruiting Better User Research Participants
    132
    Use an online panel if …
    • You can’t easily get a list of potential participants.
    • 140. You need a very specific type of participant, which would require a lot of screening.
    • 141. Your study requires a lot of time and effort for the participants to complete.
    • 142. You have more money than time.
  • Choosing Between These Methods
    23 June 2011
    Recruiting Better User Research Participants
    133
    Use online intercepts if …
    • You can’t easily get a list of potential participants.
    • 143. You want to use your site’s current users.
    • 144. Your site gets enough traffic or you can wait for enough responses to accumulate.
    • 145. You’ll be running ongoing studies on your site.
  • Recruiting Better User Research Participants
    23 June 2011
    Recruiting Better User Research Participants
    134
    Conclusion
  • 146. Conclusion
    23 June 2011
    Recruiting Better User Research Participants
    135
    Recruiting can be difficult, but it’s a crucial first step for effective research.
    • It’s worth taking the extra time to ensure that you get representative participants, who provide useful feedback, and who show up!
  • Questions and Discussion.
    23 June 2011
    Recruiting Better User Research Participants
    136
  • 147. 23 June 2011
    Recruiting Better User Research Participants
    137
    Jim Ross, Principal of Research & Strategy
    jross@electronicink.com