Carma internet research module: Encouraging responding
Encouraging RespondingCARMA Internet Research Module Jeff Stanton
Dillman Vs. GrovesTailored Design - Dillman Leverage-Saliency – GrovesUse an advance email contact Follow-up procedures do not or pre-screening procedure work in a vacuum; theirSecond contact: Email a link to effectiveness depends the questionnaire with a upon the survey context carefully constructed and characteristics of the recruitment message; include pre-study incentive respondentThird contact: Reminder Topic interest moderates the message targeted to non- effectiveness of response- respondents enhancing techniquesSubsequent contacts: Thank Community involvement you messages, or second moderates the remindersProvide any post-study effectiveness of response- incentives for participation enhancing techniques May 15-17, 2008 Internet Data Collection Methods (Day 2-2)
Strategies for Encouraging Resp.Use multiple response enhancingstrategies to increase the proportion of 1. Pre-notificationthe sample that responds. The single 2. Publicitymost powerful technique for increasing 3. Instrument Designresponse is having an interesting topic. 4. IncentivesThe second most powerful technique is 5. Survey lengthhaving interesting questions. Each 6. Reminderstechnique may only increment 7. Opportunityresponse rate by a few percentage 8. Response Monitoringpoint. Each increment decreases the 9. Establish Importancepotential for bias, although response 10. Provide Feedbackrates under 95% do not generallyeliminate it.May 15-17, 2008 Internet Data Collection Methods (Day 2-4)
Response Enhancing Techniques and Pct Increases in Mail SurveysMonetary incentives: 12% (Yammarino et al., 1991) to 19% (Hopkins & Gullickson, 1993)Advance notice: 13% (Roth and BeVier, 1998)Follow-up/reminder: 6% (Yu and Cooper , 1983)Personalization: 9% (Martin, Duncan, Powers & Sawyer, 1989)Each strategy has been tested and found effective in Internet surveys, although different improvement %s
Anseel: Org Surveys 1995-2008Advance notice, personalization, topic salience, identification numbers (to improve anonymity), and sponsorship all have generally positive effectsCombining several response enhancing techniques is advisableResults consistent with leverage-saliency Consider how respondent characteristics relate to each response enhancing technique Surveys of managers: Mention sponsorship Surveys of executives: Mention business relevance Consumers: Don’t mention sponsorship Non-managerial employees: Incentives ineffective Non-working respondents: Incentives are effectivePersonal distribution of surveysAdvance notice and follow-up have become less important over the years
Thorpe et al. 2009 (Family Practice)Improving response rates when surveying physicians: Used Dillman’s Total Design Method including four contacts by mail, RR = 48% With the addition of registered mail with return receipt plus $25 gift cards (unconditional presentation), response rates for subsequent survey studies rose to over 75%
Personally notify your potential participants that they will be receiving a survey in the near future.Actively publicize the survey.Be sensitive to the actual physical design of your survey. For example, how questions are ordered may impact respondent participation. – A study by Roberson and Sundstrom (1990) suggests placing the more interesting and easy questions first and demographic questions last.
Provide incentives, if appropriate. Inexpensive items such as pens, key chains, or certificates for free food/drink can increase responses.Keep the survey to a reasonable length. A theory- driven approach to survey design helps determine what is absolutely necessary to include in the survey instrument.Send reminder notes. Response rates may bump up 3-7% with each reminder note, but keep in mind that theres a point of diminishing returns when you irritate people who have chosen not to participate.
Give everyone the opportunity to participate (e.g., paper surveys where required, scheduling time off the phone in the call centers, etc.). At one company for example, most surveys run for 10 business days and span across three work weeks.Track response rates so that HR generalists and/or the survey coordinators can identify units with low response rates and contact the responsible manager to increase responses.
Foster commitment to the survey effort. For example, you can involve a wide range of employees (across many levels) in the survey development process. Link the content of the survey to important business outcomes.Provide respondents with survey feedback after the project is completed. Be careful not to abandon your participants once getting the data you wanted from them. You are paving the way (or failing to do so!) for future survey efforts.Personalization of the survey invitation. Personal signature as part of cover letter.Topic salience
Even when controlling for the presence of other techniques, advance notice, personalization, identification numbers, and salience, are associated with higher response rates.Need to do more to just get the same results as the past.Target facilitation strategy to who you are surveying. – For top executives, salience of the survey topic was most key. Incentives were counterproductive – Incentives worked for unemployed individuals
Pre-screening w/Active Response Dear sampled individual: In one week, we will begin a study on this topic where you will have the opportunity to earn $X for completing a survey that is Y minutes long. Click here if you expect to participate: http://mysrv.com/screen.htm?participate=yes Please click here if you plan to decline participation or do not wish to receive the recruitment message: http://mysrv.com/screen.htm?participate=no Internet Data Collection Methods (Day 2-May 15-17, 2008 13)
Dealing with Non-Contacts Cull bad Advance Recruiting addrsScreening Message Ethics question: Who gets rewards? 1st Rem How much missing Message data allowed? 2nd Rem Message Cull Cull good refusals Distribute resps Rewards Internet Data Collection Methods (Day 2- May 15-17, 2008 14)