• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Toepoel response quantity,-response_quality,_and_costs-164
 

Toepoel response quantity,-response_quality,_and_costs-164

on

  • 195 views

A GOR presentation

A GOR presentation

Statistics

Views

Total Views
195
Views on SlideShare
195
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Toepoel response quantity,-response_quality,_and_costs-164 Toepoel response quantity,-response_quality,_and_costs-164 Presentation Transcript

    • Response Quantity, Quality, and Costs ofBuilding an Online Panel via Social Contacts Dr. Vera Toepoel v.toepoel@uvt.nl Leisure Sciences GOR, March 2011
    • Research Problem • More than 50% of all survey data in the Netherlands are collected via Internet > Problem 1: majority of the Dutch people are not willing to join a web panel > Problem 2: the minority (20%) of the web panellists fills out the majority (80%) of the questionnaires (NOPVO, 2006) • Are results representative of the Dutch population?! > NO! E.g. Internet panels contain too many (heavy) Internet users and SP-7 mei 2012 2 voters and too few CDA-voters, church-
    • Traditional Approach to Building a Web Panel 1. Volunteer panel • Recruitment via email, banners, self- selection > Pro: cheap, fast, many panel members > Con: no sampling frame available 2. Probability-based panel • Recruitment via probability sampling (address-based sampling) > Pro: representativeness > Con: Unaffordable for most people7 mei 2012 3
    • Unconventional Approach to Building a Web Panel • Snowball-effect via social networks • Leisure Panel > Sample base: administrative records of Breda University of Applied Sciences(7000 students with a national spread) > Each student is asked to recruit panel members (each year new students to recruit new panel members) > Experiment with instruction to recruit new panel members (those not in other panels, ethnic minorities etc.)7 mei 2012 4
    • Recruiting via social networks7 mei 2012 5
    • Mail chain7 mei 2012 6
    • Mail chain7 mei 2012 7
    • Literature • Majority of Dutch people is not willing to be in a web panel (NOPVO, 2006) • Announcement boosts response rates (Dillman, 2007) • E-mail augmentation boosts response rates (Dillman, 2010) • Pre (unconditional) incentives are more powerful than post (conditional) incentives (Sanchez-Fernandez et al., 2010) • Cash is more powerful than gifts (Goritz, 2010) • Response depends on relevance7 mei 2012 8
    • Experiment/design 1 • Majority of Dutch people is not willing to be in a web panel > Request to fill out single questionnaire- >request to be a panel member > Request to be a panel member ->request to fill out single questionnaire • Announcement boosts response rates (Dillman, 2007) > Announcement letter: mail/email/no • E-mail augmentation boosts response rates (Dillman, 2010) > Reminder email7 mei 2012 9
    • Experiment/design 2 • Pre (unconditional) incentives are more powerful than post (conditional) incentives (Sanchez-Fernandez et al., 2010) > Unconditional incentive versus no incentive • Cash is more powerful than gifts (Goritz, 2010) > 5 euro versus lottery ticket • Response depends on relevance > Response to “Leisure Panel” dependent on Academy for Leisure,7 mei 2012 Tourism, Media 10
    • Recruitment • Week 48 2011: pre-announcement • Week 49 2011: recruitment mail • Week 50 2011: marketing (newsletter, tv) • Week 51: 2011: reminder7 mei 2012 11
    • Marketing7 mei 2012 12
    • Design: 3 different academies Frequency Tourism 2045 Leisure 1453 Media 1009 Total students 45077 mei 2012 13
    • Design: announcement Frequency Letter 1000 (600 euro) Email 1000 No announcement 2507 Total students 4507 Incentives in letter Frequency 5 euro 333 (1700 euro) Lottery ticket (costs 3 euro) 333 (1000 euro) No incentive 334 Total students 4507 Across 3 academies7 mei 2012 14
    • Research questions • When should we time the request to be a panel member? • Are the costs associated with sending an announcement letter worth it? • Does a reminder mail increase response rates? • Does an unconditional incentive boosts response rates and is it worth the costs? • Is cash more powerful than a lottery ticket? • Does response to “Leisure Panel” differ between three academies (Leisure, Tourism, Media)?7 mei 2012 15
    • Response???7 mei 2012 16
    • Response???7 mei 2012 17
    • Response???7 mei 2012 18
    • When should we time the request to be a panel member? Panel Panel member member no yes 1st questionnaire 23% 77% 1st panel request 9% 91% N 203 Chi Square 7.79 p .0057 mei 2012 19
    • Are the costs associated with sending an announcement letter worth it? • Costs 600 euro Students Recruited % panel members Letter 1000 (334) 56 (9) 6% (3%) Mail 1000 32 3% No announcement 2498 41 2% N 4498 Chi Square 40.68 p .000 Letter without incentives between parentheses Costs per panel member: 11 euro Announcement does not significantly boost7 mei 2012 response rates and is not worth the costs 20
    • Does a reminder mail increase response rates? • Recruitment mail: 93 panelmembers • Reminder: 27 panelmembers • Increase=23%7 mei 2012 21
    • Does an unconditional incentive boosts response rates and is it worth the costs? • 2700 euro (333*5 euro+333*lottery ticket) Students Recruited % panel member No Incentive 334 9 3% Incentive 666 46 7% N 1000 Chi Square 7.59 p .01 Note: incentive is provided with announcement letter Unconditional incentive boosts response rate significantly7 mei 2012 Costs per panel member: 59 euro 22
    • Is cash more powerful than a lottery ticket? • 1700 euro (333*5 euro) • 1000 euro (333*lottery ticket) Students Recruited % panel member 5 euro 333 25 8% Lottery 333 22 7% ticket No 334 9 3% incentive N 1000 Chi Square 8.26 p .02 Costs per panel member: 68 euro/cash; 45 euro/lottery7 mei 2012 Difference between cash and lottery not significant 23
    • Does response to “Leisure Panel” differ between three academies (Leisure, Tourism, Media)? Academies Students Recruited % panel member Leisure 1453 57 4% Tourism 2045 55 3% Media 1000 15 2% N 4498 Chi Square 12.92 p .00 Academy for Leisure produces significantly more panel members than Academy for Media (difference with Academy for Tourism not significant) The higher the relevance, the more panel members are recruited7 mei 2012 24
    • Conclusion • Response quantity is poor. Students are not really motivated to recruit panel members • Quality is poor (mostly students) • Students do not understand request/Students do not know the sender of the request • First panel request, then questionnaire yields most panel members (response actual waves?) • Announcement letter does not work • Reminder works7 mei 2012 25 • Incentives work, but are not really worth
    • Future actions • Lecturers are ask to discuss Leisure Panel and request • Mail is send via directors of academies • Different types of incentives > 50*2 tickets for event > 2*250 euro in cash > Does this work???7 mei 2012 26