Mysore Call Girls 8617370543 WhatsApp Number 24x7 Best Services
Aapor incentive voicemail_short
1. Increasing response rates in cell
frames: Results from an incentive and
voicemail experiment
Kathleen Call, PhD
Professor
Division of Health Policy & Management
State Health Access Data Assistance Center
University of Minnesota, School of Public Health
AAPOR Orlando Florida
May 18, 2012
Funded by a grant from the Robert Wood Johnson Foundation
2. Acknowledgments
• Funding: MN Dept. of Health
• Co-Authors:
Jessie Kemmick Pintor Doctoral Student
Stefan Gildemeister Director, MN Dept of Health HEP
David Dutwin Vice President, SSRS
Robyn Rapoport Research Director, SSRS
2
3. Objective
• Evaluate how cell phone users respond to
different incentive and voicemail
conditions independently and combined
3
4. Experiment
No voicemail:
No 1,050
incentive Standard
voicemail: 27,050
Standard
voicemail: 7,000
Incentive $5
arms incentive Incentive
voicemail: 7,000
Standard
voicemail: 7,000
$10
incentive Incentive
voicemail: 7,000
4
5. Methods – data and sample
• 2011 MN Health Access Survey
• Dual frame survey targeting 11,000
completes
– 7000 landline
– 4000 cell
• Conducted from Sep to Dec 2011
• Experiment in cell phone frame alone
5
6. Analysis
• Compare response rates (AAPOR RR 4):
• RR4 considers complete/partial interviews, refusals/breakoffs,
non-contacts, and % of unknown eligible thought to be eligible
• Compare mean # of calls to complete among those
who received VM
• Compare the ratio of complete to eligible
• Compare % providing contact info to receive
incentive across arms, and % requesting incentive
in non-incentive conditions
6
7. Table 1. Voicemail x incentive
conditions
% complete Response
Total among rate
Arm contacts Completes eligible (AAPOR RR 4)
No incentive, standard VM 29,361 2,046 89.8% 40.5%
No incentive, no VM 1,874 139 87.4% 39.2%
$5 incentive, incentive VM 8,118 560 94.1%** 40.0%
$5 incentive, standard VM 7,449 487 90.5% 41.0%
$10 incentive, incentive VM 8,121 580 92.9%* 40.7%
$10 incentive, standard VM 7,495 515 91.0% 39.6%
Overall 62,468 4,327 90.9% 40.4%
* significantly different from no incentive at .05 level
** significantly different from no incentive at .01 level
7
8. Table 2. Mean calls to complete
among those receiving voicemail
Total
Arm contacts Completes Mean calls (SD)
No incentive, standard VM 13,223 1,106 5.08 (2.42)
$5 incentive, incentive VM 3,629 321 5.28 (2.55)
$5 incentive, standard VM 3,348 265 5.14 (2.51)
$10 incentive, incentive VM 3,665 320 5.03 (2.46)
$10 incentive, standard VM 3,343 269 5.13 (2.60)
Overall 27,208 2,281 5.11 (2.47)
AUDIENCE PARTICIPATION!
8
9. Table 3. Percent of respondents
providing contact info for incentive
%
provided
Requested contact
Arm Completes incentive info
No 337
2,185 73.6%
incentive (15.4%)
$5
1,047 N/A 54.7%***
incentive
$10
1,095 N/A 64.8%**,***
incentive
** significantly different at .01 level
*** significantly different at .001 level
9
10. Table 4. Incentive conditions:
Who provides contact information?
$5 incentive $10
incentive
Female No effect +
Young adults + No effect
Race/ethnicity No effect No effect
Lower family income + +
Lower education + +
Employed – –
Home owner – –
Household size No effect No effect
Greater MN No effect No effect
10
11. Table 5. No incentive conditions:
Who requests incentives and who
provides contact information?
Requested Provided contact
incentive info
Female No effect No effect
Age No effect No effect
Race/ethnicity (non- + No effect
White)
Family income No effect No effect
Lower education + No effect
Employed No effect No effect
Home owner No effect No effect
Household size No effect No effect
Greater MN No effect +
11
12. Summary
• No impact on response rates for VM, incentive,
or joint VM/incentive conditions
• Incentive VM had no impact on # call attempts
• Incentive VM appears to influence completion
among those eligible
– 87% vs. 90% vs. 94%
• Respondents requesting compensation are
more likely to provide contact information
• Respondents offered larger compensation and
those from disadvantaged populations are more
likely to provide contact information
12
13. Implications
• Offering incentives/reimbursement in order
to increase response rates in cell frames
may not be necessary
• Providing incentive increases efficiency,
somewhat
• Even if offering incentive, not all callers will
provide contact information to receive
• Provide incentive to those who request
13
14. Contact Information
Kathleen Call
callx001@umn.edu
State Health Access Data Assistance Center
University of Minnesota, Minneapolis, MN
(612) 624-4802
www.shadac.org
Sign up to receive our
newsletter and updates at
www.shadac.org
@shada
c 14
Editor's Notes
½ of sample assigned to NO INCENTIVE condition – with a small portion receiving NO VMRemaining ½ of sample split evenly among the incentive and VM arms. Targeted 4000 completes out of 56,000 cell users contactedLL and Cell - No compensation version:Hello. This is not a sales or marketing call. We’re calling to include your household in an important study about health insurance coverage in Minnesota. We will try to reach you again soon. You can also call us toll-free at 1-800-307-5184.Thank you for your time.Cell - Compensation version:Hello. This is not a sales or marketing call. We’re calling to include you in an important study about health insurance coverage in Minnesota. We will try to reach you again soon. If you are eligible for and complete the interview, we can send you a $5/$10 check to reimburse you for your cell minutes.You can also call us toll-free at 1-800-307-5184.Thank you for your time.Should I also reveal that for those we get on the phone and are determined eligible, they are told during the interview they will be reimbursed for their minutes. “We can send you a ($5/$10) check to reimburse you for cell minutes. I will collect your contact information at the end of the survey.”
NOTE: MN cell only rates is between 25-30%
RR3 includes an estimate of what proportion of cases of unknown eligibility are actually eligible.RR4 includes partials as completes so is less conservative than RR3. The analysis we are presenting is for the total frame, but we also ran the analysis removing cases that were resolved on the first call and therefore would not be exposed to the voicemail condition. Although this alters the magnitude of the response rates the pattern of results are identical.
In the interest of time rather than present each condition separately I go direction to interaction of the VM and Incentive condition – offering an incentive and leaving a message informing the R about the incentive leads to significantly more completes among those eligible for the survey. Col 1: Total contacts = the number of attempts to obtain a complete for cases in each arm of the experimentCol 2: Total completed interviews in each armCol 3: The represents the Ratio of completed surveys among those phone numbers determined to be an eligible household – that is, the cell number is attached to a person 18+ who lives in MN. OLD TABLE 1 notes: As shown – the incentive arm yielded a significantly higher % of completed interviews than the No VM condition, but this does not translate into a significantly higher RR (39.2 vs 40.4). OLD TABLE 2 notes: Same table this time looking at the incentive condition – no impact on RR rates but increase in % completes among eligibles.Used simple Z-test to explore difference between arms
In this analysis we wanted to learn whether leaving a VM would result in a shorter number of contacts to achieve a completed interviewHypothesis: those hearing the voicemail promising of reimbursement for participation would require fewer contacts to achieve the complete. This analysis is restricted to those who actually received a voicemail, which in the cell frame was left on the first attempt. That is we remove those who completed the survey on the first call attempt or who were determined to be ineligible on the first attempt (not MN resident, business, group quarters, minor, nonworking number)AUDIENCE PARTICIPATION: PLEASE STAND IF YOU HAVE A CELL PHONESIT DOWN IF YOU DON’T HAVE A VOICEMAIL FUNCTION OR HAVE NEVER SET UP VM FUNCTIONSIT DOWN IF YOU RARELY or never LISTEN TO YOUR VOICEMAILS
When conducting cell phone surveys, even if those conducting the survey decide against offering an incentive, money must still budgeted for respondents who ask to be reimbursed for their minutes. The middle column indicates that of the 337 people in the NO $ condition asked to be compensated for the call – these people were told they would be sent a $5 check. This represents 15.4% of no incentive armAt the end of the survey the interviewer request address information to send the check – saving this information to a separate file so that identifying information is not connected with their survey responses.As shown in the last column – a larger portion of those who proactively requested compensation actually provided the information needed to send the incentive. And a greater share of those promised the larger incentive ($10) end up providing contact info than those promised the smaller incentive ($5).JKP: THOSE FROM NO INCENTIVE PROACTIVELY REQUESTING COMPENSATION WERE SIGNIFICANTLY MORE LIKELY TO PROVIDE CONTACT INFO THAN $5 INCENTIVE GROUP (AT .001 LEVEL) AND $10 (AT .01 LEVEL); THOSE IN $10 INCENTIVE GROUP WERE SIGNIFICANLTY MORE LIKELY TO PROVIDE CONTACT INFO THAN THOSE IN $5 GROUP AT .001 LEVEL
We wanted to know a bit more about who is likely to take up the incentive offer.As shown, for both the $5 and $10 conditions those from more disadvantaged backgrounds (low income and education) are more likely to provide contact info.Those who work and own their homes are less likely to do so.In addition, young people will go for the $5 and females are more likely to go for the $10 incentive.
Among those in the No incentive condition We wanted to know more about who requested the incentive, AND among those who requested, who actually went on to provide contact information. In this study non whites and those with lower educational attainment were more likely to request an incentive, but only those who live outside the metro area are likely to provide contact information so we can send the incentive.
Bullets1. No obvious trends in response rates among VM or incentive conditions or VM*incentive conditions2. So doesn’t make interviewing more efficient in terms of call attempts HOWEVER3. The VM incentive appears to lead to higher completion rates among those eligible, which is also an efficiency measure. YETone could argue that the difference between 87 and 94% may not be substantively important.4. Those who request compensation are more likely to provide contact information necessary to get the incentive5. Those offered larger comp and those from more disadvantaged pops are more likely to provide contact information.
What does this all meanIf the goal of offering an incentive to cell phone users is to increase RR or sample coverage, our study shows that this may not be necessary.Incentives do seem to increase efficiency a bit, but not sure what the impact on overall project costs would be for the effect sizes we are seeing. If you do offer an incentive, not everyone will take you up on the offer – only 50% of those offered $5 did and about 65% of those offered $10 did. Not providing contact info could be for privacy reasons or because the caller doesn’t think it’s necessary to receive incentive (perhaps survey shorter than they anticipated or the promise of an incentive hadn’t influenced their decision to participate in the first place).We do recommend providing compensation to those who ask for it.
