Your SlideShare is downloading. ×
Recruiting Better User Research Participants: UPA 2011
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Recruiting Better User Research Participants: UPA 2011

2,803
views

Published on

Published in: Business, Technology

0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,803
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
68
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Failing to recruit participants or recruiting the wrong participants dooms everything that comes afterwards.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Have audience raise hands to each question. With “Had other recruiting problems”, ask them to mention or talk about other recruiting problems they’ve had.
  • Someone has to do the difficult job of finding, contacting, screening, and scheduling the participants. You can do it yourself, leave it up to your clients, or hire a recruiting company. There are advantages and disadvantages for each of these approaches.
  • Makes the most sense when you have easy access to a list of potential participants. When you’re recruiting your fellow employees or customers.
  • Makes the most sense when you have easy access to a list of potential participants. When you’re recruiting your fellow employees or customers.
  • Give you more credibility if you have a connection to these people - fellow employees, they are your customers, you are a member
  • People visiting your website are often the exact people you want to reach. The intercept leads to a screening questionnaire. You can either contact those volunteers immediately or they are added to a database of participants to contact later. This is far less expensive over time than using a recruiting company.Can continue to recruit participants over time
  • If your client has the relationship with employees, customers, members; have him/her do the recruiting to take advantage of that relationship.
  • Client not good at recruiting They’re not going to go through a complicated screenerToo many or too few participants of a particular type Recruiting inappropriate participants - Bob, the super user, because he knows the most about the systemYour client may not be good at scheduling participants in the desired timeframe, delaying the study or spreading it out over a longer time period.
  • Clients need guidance of who to recruit and who not to recruit. Otherwise they may not recruit the right people.Screeners are too complicated for them.
  • Schedule will constrain the sessions to a reasonable time frame.Otherwise the sessions may be too spread out over time.Emphasize the need to not schedule sessions back to back, leave enough time for breaks, meals, travel between locations, etc.
  • To prevent the client from passing on incorrect information, give them the email text to use.
  • Greater chance that you’ll get people who lie to get into the study just to get the money. May not care about your research.
  • Some recruiting companies specialize in certain types of participantsGood to have several recruiting companies to use - otherwise you tend to scrape the bottom of the barrel if you use the same company all the time
  • Since you’re having someone else do the recruiting, a screener is essential.
  • The recruiting company should send you periodic updates on the participants are recruited.
  • The screener is one of the most important documents in a research project. Yet it’s often the one that’s most taken for granted. Few books or schools detail how to write an effective screener, and as a result we often learn to write them through trial and error, rely on screeners created by the marketing department, or reuse screeners from previous projects without considering whether they are still appropriate.
  • Because people will assume you’re a telemarketer, you need to get right to the point and quickly establish that you’re not selling anything. Mentioning the money up front gets attention and keeps people on the phone.
  • More questions just add to the burden on the recruiter and the participants
  • Recruiting companies sometimes rely too heavily on their participant databases before calling “fresh” participants. Many are great participants, but others volunteer a little too often. They may be supplementing their income as a professional research participantRecruiting companies should first use their database to screen out the participants they will call based on your criteria.
  • Recruiting companies sometimes rely too heavily on their participant databases before calling “fresh” participants. Many are great participants, but others volunteer a little too often. They may be supplementing their income as a professional research participantRecruiting companies should first use their database to screen out the participants they will call based on your criteria.
  • If you’re using a recruiter, you have to rely on their ability to correctly judge the response and whether it fits these categories.
  • [Ask the audience to try to guess the answer that will get them into the study as a participant.]
  • If you’re using a recruiter, they have to have a good ability to interpret the person’s answer to know which to terminate
  • Sometimes you do need to ask multiple choice questions in which you read the choices to the person.
  • Because the middle three answers are so specific, they seem like safe choices to avoid being eliminated. It’s obvious that “12 or less” and possibly “31 or more” are the cutoff points. The wording “12 or less” and “31 or more” implies that if you watch that little or that much TV, the recruiter doesn’t even care about the exact number of hours, and you’ll be eliminated.
  • Sure, I’m “considering”. Maybe, who knows. I guess I might need to buy a car, so I qualify.
  • “Professional” participants play it safe by selecting all or most of the answers, knowing that there’s one expected answer that will get them in the study.
  • Sure, you may eliminate some legitimate people, but you’ll also eliminate cheaters. So be careful to make it a reasonable limit. A first time home buyer might reasonably buy many of these items.
  • Bring in their credit card from a particular bankBring in their iPadGraceful exit allows people to save face and screen themselves out instead of remaining with the lie and allowing themselves to be scheduled, knowing they won’t actually show up
  • Some clients want to get a distribution of demographics. That doesn’t really matter, but if needed, put those questions at the end.Unless it’s a dating site, in many cases marital status doesn’t matter.
  • Use personas or user profiles to determine the behaviors and attitudes
  • You don’t want people with a strong bias against your client
  • You can interpret attitudes towards Japanese cars from the answers to this question.
  • You can interpret attitudes towards Japanese cars from the answers to this question.
  • Requires you to first think of the average hours per day and then multiply by days per week
  • It’s easier to think of hours per day. No mental math required.
  • Information gathering only adds to the length of the screening process.
  • Ensure that the participants match the computer and web experience of your user groups. For example, in a usability test, you don’t want to confuse interface problems with problems that new computer users face in general. Similarly, you often don’t want to test with web developers or designers.
  • In self-assessments, some people will inflate their self-ratings, but this will help eliminate those that truly feel uncomfortable with computers.Eliminates people who show up and get nervous, “I’m not good with computers.”Self-assessments are very subjective, and some people are reluctant to admit their inexperience.
  • Be specific. For example, I added “without assistance” to the following question after we recruited a woman who later said she used the Web to read her email by having her niece use the computer to read the email to her.
  • It’s great to have eloquent participants who can easily think aloud and express their thoughts and opinions. It requires a lot of work to drag useful information out of quiet participants.Question should be related to your topic because people won’t always be talkative about subjects they are not interested in.
  • Real user groups include both expressive and quiet people. Including some of each type of participant will give you a more representative group and still give you enough interesting video clips.
  • These things sound so obvious, that you forget to mention them until a participant comes in who can’t see the screen.I once had a woman who didn’t bring her glasses and asked, “Can’t you just read what’s on the screen to me?”
  • Sure, we give them this information at the beginning of the session, but most people don’t feel able to back out at that point.Many participants assume they will be part of a focus group, because that is what recruiting companies most commonly recruit for.
  • Reveals holes and logical inconsistenciesA good recruiting company reveals logic problems and when your screener is too strict and may result in no participants
  • There will always be no-shows, but there are steps you can take to make it more likely that your participants will show up for their sessions.
  • Obviously, the more convenient the location of your research, the more likely your participants will attend.
  • Conduct your research sessions at times convenient to the participants.
  • Tuesdays through Thursdays tend to be the best days for weekday research
  • Let’s face it, for most people it’s the money that encourages them keep their commitment to attend a research session. If you don’t offer the right incentive for the effort involved, your no-show risk increases. For example, employees using poorly designed business systems are often grateful that someone is finally asking for their opinions. The chance to improve the application is incentive enough.
  • Recruit backup participants to replace no-shows and problem participants. Two approaches to securing backup participants are using floaters and over-recruiting.
  • To completely eliminate risk, each participant can be covered by a floater. However, this is very expensive. Instead, you can balance cost and risk by having each floater cover multiple sessions. This means you may run out of participants if you use the floater and then don’t have a floater to cover another no-show participant, but that’s the risk you take.
  • To completely eliminate risk, each participant can be covered by a floater. However, this is very expensive. Instead, you can balance cost and risk by having each floater cover multiple sessions. This means you may run out of participants if you use the floater and then don’t have a floater to cover another no-show participant, but that’s the risk you take.
  • To completely eliminate risk, each participant can be covered by a floater. However, this is very expensive. Instead, you can balance cost and risk by having each floater cover multiple sessions. This means you may run out of participants if you use the floater and then don’t have a floater to cover another no-show participant, but that’s the risk you take.
  • Phone screening is not economical or practical since people have to have a link to click to get to the study.
  • Emailing makes the most sense if you have a list of potential participants.
  • Emailing a link to your study is an effective method if you can easily identify and obtain email addresses of potential participants. However, emailing a large number of people can be time consuming.
  • For example, if you want 100 participants to complete the study, email 2,000 people (assuming a five-percent-response rate).
  • Some unmoderated tools allow you to end the study when it reaches a maximum number of participants. This can help you avoid paying too much in incentives.
  • Save the study instructions until people get into the study itself.
  • If there are non-monetary incentives, appeal to the benefits of participating. Your participation will help make the application you use everyday more efficient and better suited to your needs.
  • They add up and it can be time consuming to send checks to every participant.
  • It can be more enticing and less expensive.It’s easier than having to send individual incentives to everyone.Beware of legal issues around contests and sweepstakes. Consult a lawyer.
  • For example, an Xbox appeals much more to some people than others, which will affect the types of people who enter.
  • - Although mass emails require less work, individual emails are more effective at establishing trust and are therefore less likely to be ignored.
  • Few people will contact you, but you show you’re putting your information out there for people to check you out. You’re willing to share with them.
  • People are wary of clicking links in emails from people they don’t know. Emails from a familiar person or organization seem more credibleOr name drop someone the person is familiar with and provide their contact information
  • This usually results in additional participants. If you still need more participants, email a new set of people that you haven’t yet contacted.
  • If you think you’ll have difficulty finding participants on your own, an online panel is a good option.
  • Provided by many recruiting companies
  • Provided by many recruiting companies
  • You don’t need to provide a screener to a panel recruiter, as you would with a moderated study.
  • They are more tolerant of additional questions. They are more familiar with what’s expected.
  • For example, if 1,000 people are invited, and 100 complete the study, you will only be charged for those 100 people. The panel company handles the incentives. You just pay one fee.
  • Recruiting is done automatically It can continue indefinitely
  • Don’t make the screening too long though. That leads to abandonment.
  • Intercepts reach “real users” who are already on the website for a reason with real goals and motivations
  • Less motivation to complete the study – People agree to them on a whim. No personal email from someone they know, not part of a panel and wanting to be invited for future studies
  • Discussion pointsWhat do you find are the most difficult aspects of recruiting?Are there other good methods to get participants?Other recruiting tips or advice?
  • Transcript

    • 1. 23 June 2011
      1
      Recruiting Better User Research Participants
      Recruiting Better User Research Participants
      Principal of Research & Strategy
      Electronic Ink
      UPA 2011
      23 June, 2011
    • 2. Why talk about recruiting?
      23 June 2011
      Recruiting Better User Research Participants
      2
      User research is only as good as the participants you recruit.
      It’s difficult to recruit participants who …
      • Are representative
      • 3. Can express themselves and provide useful information
      • 4. Show up
    • Why talk about recruiting?
      23 June 2011
      Recruiting Better User Research Participants
      3
      Have you ever …
      • Recruited someone who didn’t fit the profile or was just odd?
    • Why talk about recruiting?
      23 June 2011
      Recruiting Better User Research Participants
      4
      Have you ever …
      • Recruited someone who lied to get into the study and was just in it for the money?
    • Why talk about recruiting?
      23 June 2011
      Recruiting Better User Research Participants
      5
      Have you ever …
      • Recruited people who couldn’t express themselves?
    • Why talk about recruiting?
      23 June 2011
      Recruiting Better User Research Participants
      6
      Have you ever …
      • Had too many no shows and cancellations?
    • Why talk about recruiting?
      23 June 2011
      Recruiting Better User Research Participants
      7
      What other recruiting problems have you had?
    • 5. What we’ll talk about
      23 June 2011
      Recruiting Better User Research Participants
      8
      Recruiting for moderated research
      • Who should do the recruiting?
      • 6. Writing a better screener
      • 7. Minimizing no-shows
      • 8. Recruiting backup participants
      Recruiting for unmoderated, remote research
      • Emailing participants
      • 9. Using an online panel
      • 10. Using online intercepts
    • Recruiting for Moderated Research.
      23 June 2011
      Recruiting Better User Research Participants
      9
    • 11. Recruiting for Moderated Research.
      23 June 2011
      Recruiting Better User Research Participants
      10
      Who should do the recruiting?
    • 12. Who should do the recruiting?
      23 June 2011
      Recruiting Better User Research Participants
      11
      Recruiting options
      You
      Your client
      A recruiting company
    • 13. Recruiting for Moderated Research.
      23 June 2011
      Recruiting Better User Research Participants
      12
      Doing the recruiting yourself
    • 14. Doing the recruiting yourself
      23 June 2011
      Recruiting Better User Research Participants
      13
      Advantages
      You know what you’re looking for.
      You have more control over who is recruited.
      You sound more human and less like a telemarketer.
      You save money.
    • 15. Doing the recruiting yourself
      23 June 2011
      Recruiting Better User Research Participants
      14
      Disadvantages
      It’s difficult if you don’t already have a list to work from.
      It’s time consuming to find, contact, screen, and schedule.
      It may require a lot of cold calling.
    • 16. Doing the recruiting yourself
      23 June 2011
      Recruiting Better User Research Participants
      15
      Start with lists of potential participants - customer lists, membership lists, employee directories, etc.
    • 17. Or gather participants from your website with a recruiting page or an online intercept.
      Doing the recruiting yourself
      23 June 2011
      Recruiting Better User Research Participants
      16
    • 18. Or advertise or connect through social media and other sources.
      Doing the recruiting yourself
      23 June 2011
      Recruiting Better User Research Participants
      17
    • 19. Recruiting for Moderated Research.
      23 June 2011
      Recruiting Better User Research Participants
      18
      Having your client do the recruiting
    • 20. Having your client do the recruiting
      23 June 2011
      Recruiting Better User Research Participants
      19
      Advantages
      Clients have access to employee, customer, and member lists.
      Employees, customers, or members are more likely to volunteer.
      It saves you money and time.
    • 21. Having your client do the recruiting
      23 June 2011
      Recruiting Better User Research Participants
      20
      Disadvantages
      Clients may not be very good at recruiting.
      Clients may not possess the time and organizational skills required.
      Clients often pass on vague or incorrect information about the research.
    • 22. Having your client do the recruiting
      23 June 2011
      Recruiting Better User Research Participants
      21
      Provide your client with detailed instructions.
      Profile of who to recruit
      And who not to recruit
    • 23. Having your client do the recruiting
      23 June 2011
      Recruiting Better User Research Participants
      22
      Provide your client with detailed instructions.
      Schedule of sessions
    • 24. Having your client do the recruiting
      23 June 2011
      Recruiting Better User Research Participants
      23
      Provide your client with detailed instructions.
      Description of the study to give to participants
    • 25. Having your client do the recruiting
      23 June 2011
      Recruiting Better User Research Participants
      24
      Ensure they do it correctly.
      Delegate to someone with time and organizational skills.
      Monitor the recruited participants.
      Email the scheduled participants to introduce yourself and provide more details.
    • 26. Recruiting for Moderated Research.
      23 June 2011
      Recruiting Better User Research Participants
      25
      Using a recruiting company
    • 27. Using a recruiting company
      23 June 2011
      Recruiting Better User Research Participants
      26
      Advantages
      It saves you a lot of time and effort.
      It’s a good option when you have no existing lists or other resources.
      With the right information, a good recruiting company can get the best participants.
    • 28. Using a recruiting company
      23 June 2011
      Recruiting Better User Research Participants
      27
      Disadvantages
      It’s the most expensive recruiting option.
      You might get “professional” participants who are only in it for the money.
      With poor direction or a bad recruiting company, you can get the worst participants.
    • 29. Using a recruiting company
      23 June 2011
      Recruiting Better User Research Participants
      28
      Selecting a recruiting company
      Get referrals from other researchers.
      Get quotes from several recruiting companies - cost and confidence.
      Look for good communication.
      Cultivate several recruiting companies you can use regularly.
    • 30. Using a recruiting company
      23 June 2011
      Recruiting Better User Research Participants
      29
      A strong and clearly written screener is essential.
    • 31. Using a recruiting company
      23 June 2011
      Recruiting Better User Research Participants
      30
      Review the list of recruited participants.
      Ensure they fit what you’re looking for.
      Have your client review and approve the list also - CYA!
    • 32. Recruiting for Moderated Research.
      23 June 2011
      Recruiting Better User Research Participants
      31
      Deciding who will do the recruiting
    • 33. Deciding who will do the recruiting
      23 June 2011
      Recruiting Better User Research Participants
      32
      Do your own recruiting when:
      You have more time than money.
      You don’t mind or are effective at cold calling.
      You can hire someone else to do the recruiting – intern, staff recruiter.
      You can use existing lists, recruit from your website, or advertise.
      You already have a connection with potential participants.
    • 34. Deciding who will do the recruiting
      23 June 2011
      Recruiting Better User Research Participants
      33
      Have your client do the recruiting when:
      Your client has detailed contact lists of employees, customers, or members.
      Your client already has a connection with potential participants.
      Your client has the time and coordination to do this effectively.
      Your client has successfully done recruiting in the past.
    • 35. Deciding who will do the recruiting
      23 June 2011
      Recruiting Better User Research Participants
      34
      Hire a recruiting company when:
      You have more money than time.
      You don’t want to “cold call” and screen participants.
      You don’t have an easy way to find potential participants.
    • 36. Recruiting for Moderated Research.
      23 June 2011
      Recruiting Better User Research Participants
      35
      Writing a better screener
    • 37. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      36
      Keep people from hanging up on you
      Quickly get to the point.
      You’re not selling anything.
      You’re recruiting for a paid study.
    • 38. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      37
      Ask likely elimination questions first
      Only the most promising will go through all the questions.
    • 39. Eliminate the usual suspects
      Eliminate recent participants.
      How would you answer this question if you wanted to get into the study?
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      38
    • 40. Eliminate the usual suspects
      “Professional” participants might lie.
      Recruiters should check their records before calling people.
      “A few years ago.”
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      39
    • 41. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      40
      Eliminate conflicts of interest
      Existing relationship with your client - employee, employee of a competitor, etc.
      Too much inside knowledge - web designer, developer, marketer, etc.
    • 42. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      41
      Avoid reading multiple choice answers
      The “correct” answer is often obvious
      How would you answer this question if you wanted to get into the study?
    • 43. Avoid reading multiple choice answers
      The “correct” answer is often obvious
      “None of these”
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      42
    • 44. Ask open-ended questions instead of reading the answers
      How would you answer this question if you wanted to get into the study?
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      43
    • 45. Ask open-ended questions instead of reading the answers
      Requires the recruiter to know which professions should terminate the screening
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      44
    • 46. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      45
      Don’t make the elimination answers obvious
      How would you answer this question if you wanted to get into the study?
    • 47. Don’t make the elimination answers obvious
      “19 – 24” seems like a safe answer.
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      46
    • 48. Include multiple elimination answers.
      How would you answer this question if you wanted to get into the study?
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      47
    • 49. Include multiple elimination answers
      It’s not obvious which answers will get you eliminated.
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      48
    • 50. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      49
      Avoid giving away what you’re looking for
      How would you answer this question if you wanted to get into the study?
    • 51. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      50
      Avoid giving away what you’re looking for
      “Yes”
    • 52. Avoid giving away what you’re looking for
      Providing many choices obscures what you’re looking for
      How would you answer this question if you wanted to get into the study?
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      51
    • 53. Avoid giving away what you’re looking for
      Providing many choices obscures what you’re looking for
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      52
    • 54. Avoid giving away what you’re looking for
      Beware of those who choose everything to try to get into the study.
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      53
    • 55. Avoid giving away what you’re looking for
      Set a reasonable limit to eliminate fakers.
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      54
    • 56. Avoid giving away what you’re looking for
      Use open-ended questions to completely hide what you’re looking for.
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      55
    • 57. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      56
      Ask questions that only qualified participants can answer
    • 58. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      57
      Ask questions that only qualified participants can answer
    • 59. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      58
      Require proof
      Require participants to provide proof of qualification.
      Provide a graceful exit, so that people don’t have to be caught in a lie.
    • 60. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      59
      Recruit based on behavior and attitudes
      Most demographics aren’t very important for small-scale research.
    • 61. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      60
      Recruit based on behavior and attitudes
      Screen for desired behaviors.
    • 62. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      61
      Recruit based on behavior and attitudes
      Negative attitudes towards your client or subject matter may disqualify people.
    • 63. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      62
      Recruit based on behavior and attitudes
      Screen for desired attitudes.
    • 64. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      63
      Recruit based on behavior and attitudes
      Interpret attitudes and behaviors based on answers.
      What do you think we’re looking for here?
    • 65. Recruit based on behavior and attitudes
      Interpret attitudes and behaviors based on answers.
      People interested in Japanese cars
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      64
    • 66. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      65
      Don’t ask questions that are difficult to answer
      Can you answer this question easily and accurately?
    • 67. Don’t ask questions that are difficult to answer
      Estimating the average number of hours per day is easier.
      Is this question easier to answer?
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      66
    • 68. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      67
      Don’t use the screener to gather information
      Only include questions that screen participants.
      Save information gathering for a pre-study questionnaire.
    • 69. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      68
      Screen for computer, Web, and technology experience
      You may want to eliminate those with very low and very high experience.
    • 70. Screen for computer, Web, and technology experience
      Self-assessments allow those who are truly uncomfortable with technology to excuse themselves from the study.
      But don’t rely only on self-assessments.
      Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      69
    • 71. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      70
      Screen for computer, Web, and technology experience
      Make your own assessment based on activities and behaviors.
    • 72. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      71
      Screen for eloquence
      Ask an open-ended question related to your topic to judge expressiveness.
    • 73. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      72
      Screen for eloquence
      But be careful not to eliminate qualified, quiet participants.
      “Normal” people may not be as entertaining as “professional” participants.
      Recruit representative participants, not only those who speak in perfect sound bites.
    • 74. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      73
      Ensure that people are physically able to participate
    • 75. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      74
      Let people know what they’ll be doing
      Eliminate surprises and potential discomfort.
      Allow those who would be uncomfortable to back out.
    • 76. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      75
      Test your screener with a colleague
      Ask a colleague to try to fake his/her way through the screener.
      Recruiting company should point out problems in logic.
    • 77. Writing a better screener
      23 June 2011
      Recruiting Better User Research Participants
      76
      Get your client to approve the screener – CYA!
      Your client becomes responsible for the types of people recruited.
      It’s difficult for them to complain later about the quality of the participants.
    • 78. Recruiting for Moderated Research.
      23 June 2011
      Recruiting Better User Research Participants
      77
      Minimize No-Shows
    • 79. Minimize No-Shows
      23 June 2011
      Recruiting Better User Research Participants
      78
      Locate your research conveniently.
      Where are your participants usually located during your session times?
      How easy is it to get to and find your location?
      Traffic? Public transportation? Parking?
      Is it more convenient to go to your participants or conduct remote sessions?
    • 80. Minimize No-Shows
      23 June 2011
      Recruiting Better User Research Participants
      79
      Schedule your research conveniently
      What are your ideal participants usually doing during your session times?
      Schedule a variety of session times.
      Who’s most available for weekday research?
    • 81. Minimize No-Shows
      23 June 2011
      Recruiting Better User Research Participants
      80
      Avoid days and conditions that result in more no-shows
      Fridays
      Days before holidays
      Bad weather – snowstorms, rainstorms, etc.
    • 82. Minimize No-Shows
      23 June 2011
      Recruiting Better User Research Participants
      81
      Provide appropriate incentives.
      Make it worth their while to show up.
      Make it appropriate to the person and to the activity.
    • 83. Minimize No-Shows
      23 June 2011
      Recruiting Better User Research Participants
      82
      Contact participants to confirm
      Remind them to attend.
      Give directionsand parking / transportation information.
      A good way to receive early warning of no-shows and get a head start on recruiting replacements
    • 84. Recruiting for Moderated Research.
      23 June 2011
      Recruiting Better User Research Participants
      83
      Recruit Backup Participants
    • 85. Recruit Backup Participants
      23 June 2011
      Recruiting Better User Research Participants
      84
      Two approaches to backup participants
      Floaters
      Over-recruiting
      Floaters
      Over-recruiting
    • 86. Recruit Backup Participants
      23 June 2011
      Recruiting Better User Research Participants
      85
      Floaters
      Wait on standby to fill in for no-shows or inappropriate participants
    • 87. Recruit Backup Participants
      23 June 2011
      Recruiting Better User Research Participants
      86
      Floaters
      Wait on standby to fill in for no-shows or inappropriate participants
    • 88. Recruit Backup Participants
      23 June 2011
      Recruiting Better User Research Participants
      87
      Floaters
      Wait on standby to fill in for no-shows or inappropriate participants
    • 89. Recruit Backup Participants
      23 June 2011
      Recruiting Better User Research Participants
      88
      Over-recruiting
      Recruit several extra participants to use for no-shows
      Dismiss and pay those you don’t need
    • 90. Recruit Backup Participants
      23 June 2011
      Recruiting Better User Research Participants
      89
      Over-recruiting
      Recruit several extra participants to use for no-shows
      Dismiss and pay those you don’t need
    • 91. Recruit Backup Participants
      23 June 2011
      Recruiting Better User Research Participants
      90
      Over-recruiting
      Recruit several extra participants to use for no-shows
      Dismiss and pay those you don’t need
    • 92. Recruit Backup Participants
      23 June 2011
      Recruiting Better User Research Participants
      91
      Use floaters when
      You want to be sure your observers always have a participant to observe
      You don’t have time for an extra day of backup sessions
      You don’t have money to pay for an extra day of sessions at a research facility
    • 93. Recruit Backup Participants
      23 June 2011
      Recruiting Better User Research Participants
      92
      Use over-recruiting when
      It’s not important that your observers have a participant to watch for each session
      You have time and money for an extra day of backup sessions
      You want to save money on incentives
      Your participants don’t want to wait around as floaters
    • 94. Recruiting for Unmoderated, Remote Research.
      23 June 2011
      Recruiting Better User Research Participants
      93
    • 95. Recruiting for Unmoderated, Remote Research
      23 June 2011
      Recruiting Better User Research Participants
      94
      What is different about unmoderated, remote research?
      Large numbers of participants from anywhere in the world can participate
      Participants visit a website to complete the study on their own time
      Unmoderated tools automatically capture and present the data
    • 96. Recruiting for Unmoderated, Remote Research
      23 June 2011
      Recruiting Better User Research Participants
      95
      With a large number of potential participants, different methods are required to:
      Find
      Screen
      Contact
      And convince people to participate
    • 97. Recruiting for Unmoderated, Remote Research
      23 June 2011
      Recruiting Better User Research Participants
      96
      Three recruiting methods for unmoderated studies
      Emailing participants
      Online panel
      Online intercept
    • 98. Recruiting for Unmoderated, Remote Research.
      23 June 2011
      Recruiting Better User Research Participants
      97
      Emailing Potential Participants
    • 99. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      98
    • 100. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      99
      Gather a list of potential participants
      Use existing lists of customers, members, employees, etc.
      Recruit from your website
      Advertise
    • 101. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      100
      Narrow the list of participants using existing information.
    • 102. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      101
      Avoid adding screening questions to your study
      Each question adds to the length of the study, increasing abandonment.
    • 103. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      102
      Determine how many people to email
      Determine the minimum number of participants for acceptable results.
      Tullis and Wood showed 30 - 50 is needed for online card sorting. [1]
      Assume a low response rate of 5% and email an appropriately larger population.
      [1] Tullis, Tom, and Larry Wood. “How Many Users Are Enough for a Card-Sorting Study?” Proceedings UPA 2004, Minneapolis, MN, June 7–11, 2004. Retrieved March 7, 2011: http://home.comcast.net/~tomtullis/publications/UPA2004CardSorting.pdf
    • 104. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      103
      Determine how many people to email
      Determine the maximum number of participants to accept.
      Avoid paying too much in incentives.
    • 105. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      104
      Keep it short, briefly describe the study, and provide a link to the study
    • 106. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      105
      Don’t provide the study instructions in the email
    • 107. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      106
      Highlight the incentives, both monetary and altruistic
    • 108. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      107
      Provide appropriate incentives: individual incentives are small but add up.
    • 109. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      108
      Provide appropriate incentives: entry into a drawing is often easier.
    • 110. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      109
      Provide appropriate incentives: with equal appeal to everyone.
    • 111. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      110
      Establish credibility with individual emails in a friendly, professional tone.
    • 112. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      111
      Establish credibility by providing contact information.
    • 113. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      112
      Establish credibility by sending the emails from a familiar person.
    • 114. Emailing Potential Participants
      23 June 2011
      Recruiting Better User Research Participants
      113
      Monitor participation and send reminders
      Monitor the number and type of people who have completed the study.
      If necessary, send reminder emails.
    • 115. Recruiting for Unmoderated, Remote Research.
      23 June 2011
      Recruiting Better User Research Participants
      114
      Using an Online Panel
    • 116. Using an Online Panel
      23 June 2011
      Recruiting Better User Research Participants
      115
      A panel is a database of people who have volunteered for surveys.
    • 117. Using an Online Panel
      23 June 2011
      Recruiting Better User Research Participants
      116
      Panels are provided by many recruiting and market research companies.
    • 118. Using an Online Panel
      23 June 2011
      Recruiting Better User Research Participants
      117
      Panel members are already pre-screened
      You provide the details of the people you’re looking for instead of a screener.
      The recruiter filters the database to find people matching those characteristics.
    • 119. Using an Online Panel
      23 June 2011
      Recruiting Better User Research Participants
      118
      Screening
      You can also add additional screening questions to the beginning of your study.
      Panel participants are motivated to take surveys, so you don’t have to worry as much about extra questions leading to abandonment.
    • 120. Using an Online Panel
      23 June 2011
      Recruiting Better User Research Participants
      119
      You pay the panel company based on the number of “completes.”
      For each person who completes the study, you pay a recruiting fee and small incentive.
      Limit your costs by ending the study after a maximum number of participants.
    • 121. Using an Online Panel
      23 June 2011
      Recruiting Better User Research Participants
      120
      Quality of panel participants
      They want to participate in studies, and they want to be invited back.
      This motivates them to take the study seriously and provide thorough feedback.
    • 122. Using an Online Panel
      23 June 2011
      Recruiting Better User Research Participants
      121
      Quality of panel participants
      They are more likely to be “professional” participants, primarily in it for the money.
      They may not represent your typical user.
    • 123. Recruiting for Unmoderated, Remote Research.
      23 June 2011
      Recruiting Better User Research Participants
      122
      Using Online Intercepts
    • 124. Using Online Intercepts
      23 June 2011
      Recruiting Better User Research Participants
      123
      Online intercepts pop-up on websites to invite people to participate.
    • 125. Using Online Intercepts
      23 June 2011
      Recruiting Better User Research Participants
      124
      Those who accept are taken to a brief screening questionnaire.
    • 126. Using Online Intercepts
      23 June 2011
      Recruiting Better User Research Participants
      125
      Those who pass screening can either be …
      • Added to a participant database
      • 127. Contacted immediately for a remote, moderated study
      • 128. Taken directly into an unmoderated study
    • Using Online Intercepts
      23 June 2011
      Recruiting Better User Research Participants
      126
      Creating an online intercept
      • Some unmoderated tools provide the ability to create intercepts.
      • 129. Otherwise you may need a developer to create the intercept and tie it into the site.
    • Using Online Intercepts
      23 June 2011
      Recruiting Better User Research Participants
      127
      Advantages of online intercepts
      • They recruit “real users” who are already on the website.
      • 130. Automatic screening and recruiting saves time.
      • 131. They can remain on a website indefinitely, continually recruiting for long-term studies.
    • Using Online Intercepts
      23 June 2011
      Recruiting Better User Research Participants
      128
      Disadvantages of online intercepts
      • Most people ignore them and don’t volunteer.
      • 132. Intercepted participants have less motivation to complete the study.
      • 133. Require a lot of site traffic or waiting a long time to get enough participants.
    • Using Online Intercepts
      23 June 2011
      Recruiting Better User Research Participants
      129
      Incentives
      • Providing incentives to each participant is difficult logistically and financially.
      • 134. Using prize drawings may be an easier, less expensive incentive.*
      * Be careful of the legal issues surrounding contests and sweepstakes. Consult a lawyer if needed.
    • 135. Recruiting for Unmoderated, Remote Research.
      23 June 2011
      Recruiting Better User Research Participants
      130
      Choosing Between These Methods
    • 136. Choosing Between These Methods
      23 June 2011
      Recruiting Better User Research Participants
      131
      Email the participants if …
      • You have access to a list of potential participants.
      • 137. You don’t need to do much additional screening.
      • 138. The emails can be sent from someone the participants know.
      • 139. You have more time than money.
    • Choosing Between These Methods
      23 June 2011
      Recruiting Better User Research Participants
      132
      Use an online panel if …
      • You can’t easily get a list of potential participants.
      • 140. You need a very specific type of participant, which would require a lot of screening.
      • 141. Your study requires a lot of time and effort for the participants to complete.
      • 142. You have more money than time.
    • Choosing Between These Methods
      23 June 2011
      Recruiting Better User Research Participants
      133
      Use online intercepts if …
      • You can’t easily get a list of potential participants.
      • 143. You want to use your site’s current users.
      • 144. Your site gets enough traffic or you can wait for enough responses to accumulate.
      • 145. You’ll be running ongoing studies on your site.
    • Recruiting Better User Research Participants
      23 June 2011
      Recruiting Better User Research Participants
      134
      Conclusion
    • 146. Conclusion
      23 June 2011
      Recruiting Better User Research Participants
      135
      Recruiting can be difficult, but it’s a crucial first step for effective research.
      • It’s worth taking the extra time to ensure that you get representative participants, who provide useful feedback, and who show up!
    • Questions and Discussion.
      23 June 2011
      Recruiting Better User Research Participants
      136
    • 147. 23 June 2011
      Recruiting Better User Research Participants
      137
      Jim Ross, Principal of Research & Strategy
      jross@electronicink.com