Representativeness and qualityof online sampling<br />German Loewe<br />Founder & CEO<br />
No method…<br />CONVENIENCE<br />SAMPLING<br />Unstructured data<br />Yes method…<br /><ul><li>Known inclusion probabilities
Known estimates error</li></ul>PROBABILITY <br />SAMPLING<br /><ul><li>Unknown inclusion probabilities
Unknown error
Representation via quotas</li></ul>QUOTA SAMPLING<br />Main sampling techniques<br />
Sample Frame<br />Sample Frame exists = target population<br />1<br />Units sampled randomly (known distribution)<br />2<b...
Sample Frame<br />Sample Frame exists = target population<br />1<br />Units sampled randomly (known distribution)<br />2<b...
Internet Population<br />Target Population<br />Population<br />No such thing as a Sample Frame exists<br />1<br />…no ran...
Probability Sampling<br />Quota Sampling<br />So how is online working at all??<br /><ul><li>no sample frames available
low response rates
no time / money for design</li></li></ul><li>Probability Sampling<br />Quota Sampling<br />…and what about off-line??<br /...
low response rates
no time / money for design</li></li></ul><li>Quota Sampling<br />Quota Sampling<br />Why is online so widely accepted<br /...
Cheap
Multimedia
Low incidence
 Coverage??
no sampleframesavailable
low response rates
no time / money for design</li></ul>On-line<br />Probability Sampling<br /><ul><li> Fast??
Upcoming SlideShare
Loading in …5
×

Representativeness and quality of online sampling by Germán Loewe - Netquest 2011

2,977 views

Published on

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,977
On SlideShare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Representativeness and quality of online sampling by Germán Loewe - Netquest 2011

  1. 1. Representativeness and qualityof online sampling<br />German Loewe<br />Founder & CEO<br />
  2. 2. No method…<br />CONVENIENCE<br />SAMPLING<br />Unstructured data<br />Yes method…<br /><ul><li>Known inclusion probabilities
  3. 3. Known estimates error</li></ul>PROBABILITY <br />SAMPLING<br /><ul><li>Unknown inclusion probabilities
  4. 4. Unknown error
  5. 5. Representation via quotas</li></ul>QUOTA SAMPLING<br />Main sampling techniques<br />
  6. 6. Sample Frame<br />Sample Frame exists = target population<br />1<br />Units sampled randomly (known distribution)<br />2<br />The three pillars of representativity…<br />Target Population<br />Population<br />
  7. 7. Sample Frame<br />Sample Frame exists = target population<br />1<br />Units sampled randomly (known distribution)<br />2<br />Unbiased non-response<br />3<br />The three pillars of representativity…<br />Target Population<br />Population<br />
  8. 8. Internet Population<br />Target Population<br />Population<br />No such thing as a Sample Frame exists<br />1<br />…no random (unknown inclusion probabilities)<br />2<br />…also non-response<br />3<br />Online’s methodological limbo…<br />
  9. 9. Probability Sampling<br />Quota Sampling<br />So how is online working at all??<br /><ul><li>no sample frames available
  10. 10. low response rates
  11. 11. no time / money for design</li></li></ul><li>Probability Sampling<br />Quota Sampling<br />…and what about off-line??<br /><ul><li>no sampleframesavailable
  12. 12. low response rates
  13. 13. no time / money for design</li></li></ul><li>Quota Sampling<br />Quota Sampling<br />Why is online so widely accepted<br />Probability Sampling<br /><ul><li>Fast
  14. 14. Cheap
  15. 15. Multimedia
  16. 16. Low incidence
  17. 17. Coverage??
  18. 18. no sampleframesavailable
  19. 19. low response rates
  20. 20. no time / money for design</li></ul>On-line<br />Probability Sampling<br /><ul><li> Fast??
  21. 21. Cheap??
  22. 22. Multimedia??
  23. 23. Low incidence?
  24. 24. Coverage
  25. 25. no sampleframesavailable
  26. 26. low response rates
  27. 27. no time / money for design</li></ul>Off-line<br />
  28. 28. How big is the coverage disadvantage?<br />?<br />On-line<br />Off-line<br />
  29. 29. How big is the coverage disadvantage?<br /><ul><li>Not connected to the Internet
  30. 30. Not visiting web sites were you recruit
  31. 31. Not wanting to participate in on-line surveys
  32. 32. Abandoning on-line surveys longer than X min</li></ul>?<br />On-line<br /><ul><li>Not accessible via Phone or f2f
  33. 33. Not visiting places were you recruit
  34. 34. Not wanting to participate in off-line surveys
  35. 35. Abandoning off-line surveys longer than Y min</li></ul>Off-line<br />
  36. 36. How big is the coverage disadvantage?<br /><ul><li>Not connected to the Internet
  37. 37. Not visiting web sites were you recruit
  38. 38. Not wanting to participate in on-line surveys
  39. 39. Abandoning on-line surveys longer than X min</li></ul>?<br />On-line<br /><ul><li>Not accessible via Phone or f2f
  40. 40. Not visiting places were you recruit / not at home
  41. 41. Not wanting to participate in off-line surveys
  42. 42. Abandoning off-line surveys longer than Y min</li></ul>Off-line<br />
  43. 43. How big is the coverage disadvantage?<br /><ul><li>Not connected to the Internet
  44. 44. Not visiting web sites were you recruit
  45. 45. Not wanting to participate in on-line surveys
  46. 46. Abandoning on-line surveys longer than X min</li></ul>?<br />On-line<br /><ul><li>Not accessible via Phone or f2f
  47. 47. Not visiting places were you recruit / not at home
  48. 48. Not wanting to participate in off-line surveys
  49. 49. Abandoning on-line surveys longer than Y min</li></ul>Off-line<br />
  50. 50. How big is the coverage disadvantage?<br /><ul><li>Not connected to the Internet
  51. 51. Not visiting web sites were you recruit
  52. 52. Not wanting to participate in on-line surveys
  53. 53. Abandoning on-line surveys longer than X min</li></ul>?<br />On-line<br /><ul><li>Not accessible via Phone or f2f
  54. 54. Not visiting places were you recruit / not at home
  55. 55. Not wanting to participate in off-line surveys
  56. 56. Abandoning off-line surveys longer than Y min</li></ul>Off-line<br />
  57. 57. DEPENDS ON:<br />Typeofproductorserviceyou are askingabout:<br />A PDA device?<br />Oran ice cream?<br />So how much should you worry about coverage?<br />
  58. 58. Measurement Error<br />4<br />PROBABILITY <br />SAMPLING<br />…what about the measurement error?<br />1<br />2<br />Coverage Error<br />Sampling Error<br />“Sample Frame does not match target population”<br />Statistical error<br />(under control)<br />3<br />Interviewer<br />Mode-effects<br />Social desirability<br />Situational representativity<br />Control via technology<br />Non-response<br />“Only people wanting to share opinions participate”<br />Online ok!!!<br />
  59. 59. What is quality in online panels then?<br /><ul><li>No professional respondents
  60. 60. No monetary incentives
  61. 61. Maximum 2-3 surveys a month (different categories)
  62. 62. Updated information
  63. 63. Continuous automatic updates (mandatory)
  64. 64. No fraud
  65. 65. Lower duplicates thanks to no media buying
  66. 66. Higher control over recruiting campaigns</li></li></ul><li>What is quality in online panels, then?<br /><ul><li>No self-selection
  67. 67. Actively recruited only panelists
  68. 68. Incentivize recruiting
  69. 69. No incentive or content announcement in the field
  70. 70. Reduced non-observation</li></ul> (non-response + non-coverage)<br /><ul><li>Multisource (incentivized) recruiting
  71. 71. Incentivize proportional to time (including filter-outs)
  72. 72. High response rate due to sure incentives</li></li></ul><li>If you cannot look inside, <br />how can you know find the quality providers??<br />“It’s the response rate, stupid”<br />
  73. 73. If you cannot look inside, <br />how can you know find the quality providers??<br />“It’s the observation rate, stupid”<br />
  74. 74. Thankyou!<br />German Loewe<br />gloewe@netquest.es<br />

×