Response Quantity, Quality, and Costs                    ofBuilding an Online Panel via Social Contacts                Dr....
Research Problem             • More than 50% of all survey data in the               Netherlands are collected via Interne...
Traditional Approach to Building             a Web Panel             1. Volunteer panel             •   Recruitment via em...
Unconventional Approach to             Building a Web Panel             • Snowball-effect via social networks             ...
Recruiting via social networks7 mei 2012                                    5
Mail chain7 mei 2012                6
Mail chain7 mei 2012                7
Literature             • Majority of Dutch people is not willing to               be in a web panel (NOPVO, 2006)         ...
Experiment/design 1             •   Majority of Dutch people is not willing to be in a                 web panel          ...
Experiment/design 2             • Pre (unconditional) incentives are more               powerful than post (conditional) i...
Recruitment             • Week 48 2011: pre-announcement             • Week 49 2011: recruitment mail             • Week 5...
Marketing7 mei 2012               12
Design: 3 different academies                               Frequency             Tourism           2045             Leisu...
Design: announcement                                             Frequency             Letter                          100...
Research questions              •   When should we time the request to be a panel                  member?              • ...
Response???7 mei 2012                 16
Response???7 mei 2012                 17
Response???7 mei 2012                 18
When should we time the request to be a                 panel member?                                      Panel    Panel ...
Are the costs associated with sending an                 announcement letter worth it?                 • Costs 600 euro   ...
Does a reminder mail increase response             rates?             • Recruitment mail: 93 panelmembers             • Re...
Does an unconditional incentive boosts                  response rates and is it worth the costs?                  • 2700 ...
Is cash more powerful than a lottery                  ticket?                  • 1700 euro (333*5 euro)                  •...
Does response to “Leisure Panel” differ between                 three academies (Leisure, Tourism, Media)?             Aca...
Conclusion             • Response quantity is poor. Students are               not really motivated to recruit panel      ...
Future actions             • Lecturers are ask to discuss Leisure Panel               and request             • Mail is se...
Upcoming SlideShare
Loading in …5
×

Toepoel response quantity,-response_quality,_and_costs-164

193 views
165 views

Published on

A GOR presentation

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
193
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Toepoel response quantity,-response_quality,_and_costs-164

  1. 1. Response Quantity, Quality, and Costs ofBuilding an Online Panel via Social Contacts Dr. Vera Toepoel v.toepoel@uvt.nl Leisure Sciences GOR, March 2011
  2. 2. Research Problem • More than 50% of all survey data in the Netherlands are collected via Internet > Problem 1: majority of the Dutch people are not willing to join a web panel > Problem 2: the minority (20%) of the web panellists fills out the majority (80%) of the questionnaires (NOPVO, 2006) • Are results representative of the Dutch population?! > NO! E.g. Internet panels contain too many (heavy) Internet users and SP-7 mei 2012 2 voters and too few CDA-voters, church-
  3. 3. Traditional Approach to Building a Web Panel 1. Volunteer panel • Recruitment via email, banners, self- selection > Pro: cheap, fast, many panel members > Con: no sampling frame available 2. Probability-based panel • Recruitment via probability sampling (address-based sampling) > Pro: representativeness > Con: Unaffordable for most people7 mei 2012 3
  4. 4. Unconventional Approach to Building a Web Panel • Snowball-effect via social networks • Leisure Panel > Sample base: administrative records of Breda University of Applied Sciences(7000 students with a national spread) > Each student is asked to recruit panel members (each year new students to recruit new panel members) > Experiment with instruction to recruit new panel members (those not in other panels, ethnic minorities etc.)7 mei 2012 4
  5. 5. Recruiting via social networks7 mei 2012 5
  6. 6. Mail chain7 mei 2012 6
  7. 7. Mail chain7 mei 2012 7
  8. 8. Literature • Majority of Dutch people is not willing to be in a web panel (NOPVO, 2006) • Announcement boosts response rates (Dillman, 2007) • E-mail augmentation boosts response rates (Dillman, 2010) • Pre (unconditional) incentives are more powerful than post (conditional) incentives (Sanchez-Fernandez et al., 2010) • Cash is more powerful than gifts (Goritz, 2010) • Response depends on relevance7 mei 2012 8
  9. 9. Experiment/design 1 • Majority of Dutch people is not willing to be in a web panel > Request to fill out single questionnaire- >request to be a panel member > Request to be a panel member ->request to fill out single questionnaire • Announcement boosts response rates (Dillman, 2007) > Announcement letter: mail/email/no • E-mail augmentation boosts response rates (Dillman, 2010) > Reminder email7 mei 2012 9
  10. 10. Experiment/design 2 • Pre (unconditional) incentives are more powerful than post (conditional) incentives (Sanchez-Fernandez et al., 2010) > Unconditional incentive versus no incentive • Cash is more powerful than gifts (Goritz, 2010) > 5 euro versus lottery ticket • Response depends on relevance > Response to “Leisure Panel” dependent on Academy for Leisure,7 mei 2012 Tourism, Media 10
  11. 11. Recruitment • Week 48 2011: pre-announcement • Week 49 2011: recruitment mail • Week 50 2011: marketing (newsletter, tv) • Week 51: 2011: reminder7 mei 2012 11
  12. 12. Marketing7 mei 2012 12
  13. 13. Design: 3 different academies Frequency Tourism 2045 Leisure 1453 Media 1009 Total students 45077 mei 2012 13
  14. 14. Design: announcement Frequency Letter 1000 (600 euro) Email 1000 No announcement 2507 Total students 4507 Incentives in letter Frequency 5 euro 333 (1700 euro) Lottery ticket (costs 3 euro) 333 (1000 euro) No incentive 334 Total students 4507 Across 3 academies7 mei 2012 14
  15. 15. Research questions • When should we time the request to be a panel member? • Are the costs associated with sending an announcement letter worth it? • Does a reminder mail increase response rates? • Does an unconditional incentive boosts response rates and is it worth the costs? • Is cash more powerful than a lottery ticket? • Does response to “Leisure Panel” differ between three academies (Leisure, Tourism, Media)?7 mei 2012 15
  16. 16. Response???7 mei 2012 16
  17. 17. Response???7 mei 2012 17
  18. 18. Response???7 mei 2012 18
  19. 19. When should we time the request to be a panel member? Panel Panel member member no yes 1st questionnaire 23% 77% 1st panel request 9% 91% N 203 Chi Square 7.79 p .0057 mei 2012 19
  20. 20. Are the costs associated with sending an announcement letter worth it? • Costs 600 euro Students Recruited % panel members Letter 1000 (334) 56 (9) 6% (3%) Mail 1000 32 3% No announcement 2498 41 2% N 4498 Chi Square 40.68 p .000 Letter without incentives between parentheses Costs per panel member: 11 euro Announcement does not significantly boost7 mei 2012 response rates and is not worth the costs 20
  21. 21. Does a reminder mail increase response rates? • Recruitment mail: 93 panelmembers • Reminder: 27 panelmembers • Increase=23%7 mei 2012 21
  22. 22. Does an unconditional incentive boosts response rates and is it worth the costs? • 2700 euro (333*5 euro+333*lottery ticket) Students Recruited % panel member No Incentive 334 9 3% Incentive 666 46 7% N 1000 Chi Square 7.59 p .01 Note: incentive is provided with announcement letter Unconditional incentive boosts response rate significantly7 mei 2012 Costs per panel member: 59 euro 22
  23. 23. Is cash more powerful than a lottery ticket? • 1700 euro (333*5 euro) • 1000 euro (333*lottery ticket) Students Recruited % panel member 5 euro 333 25 8% Lottery 333 22 7% ticket No 334 9 3% incentive N 1000 Chi Square 8.26 p .02 Costs per panel member: 68 euro/cash; 45 euro/lottery7 mei 2012 Difference between cash and lottery not significant 23
  24. 24. Does response to “Leisure Panel” differ between three academies (Leisure, Tourism, Media)? Academies Students Recruited % panel member Leisure 1453 57 4% Tourism 2045 55 3% Media 1000 15 2% N 4498 Chi Square 12.92 p .00 Academy for Leisure produces significantly more panel members than Academy for Media (difference with Academy for Tourism not significant) The higher the relevance, the more panel members are recruited7 mei 2012 24
  25. 25. Conclusion • Response quantity is poor. Students are not really motivated to recruit panel members • Quality is poor (mostly students) • Students do not understand request/Students do not know the sender of the request • First panel request, then questionnaire yields most panel members (response actual waves?) • Announcement letter does not work • Reminder works7 mei 2012 25 • Incentives work, but are not really worth
  26. 26. Future actions • Lecturers are ask to discuss Leisure Panel and request • Mail is send via directors of academies • Different types of incentives > 50*2 tickets for event > 2*250 euro in cash > Does this work???7 mei 2012 26

×