Does Sample Source Matter? [Part 1]

795
-1

Published on

A comparison of some leading sample sources including river sample and router sample.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
795
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Does Sample Source Matter? [Part 1]

  1. 1. Does the Sample Source Matter?A comparison of some leading sample sourcesOctober 3, 2011
  2. 2. Sample is the soiland water ofresearch.The quality of itdetermines thevalue of what youproduce.
  3. 3. We compared SBA to four other samplesources that VC regularly usesWe conducted a five minute survey Nov 4 to 7, 2011with a sample of about 500 people each, using samplefrom Vision Critical’s Springboard America, and four ofthe leading sample providers that we have anonymisedas “Panel B”, “ Panel C”, “Panel D” and “River E”.The questionnaire included items from the US 2010General Social Survey, as well as a questions aboutsnack consumption and a concept test for a snack food.We counted the number of statistically significantdifferences between the panels (95% CI)
  4. 4. Sample Quality: Dropout rate during the survey Dropouts as a percent of all who started the survey20% 16%15% 10%10% 8% 7% 5% 4% 0% SBA Panel B River C Panel D River E
  5. 5. Data quality: straight-lining % who gave the same answer for all 13 trust ratings 10% 9% 7.9% 7.8% 8% 7% 6.5% 6% 5% 3.9% 4% 3% 2% 1% 0.5% 0% SBA Panel B River C Panel D River E
  6. 6. Data quality: use of “don’t know” Average % "Dont Know"8%7% 6% 6% 6%6%5% 4% 4%4%3%2%1%0% SBA Panel B River C Panel D River E Average percent who choose the option "don’t know", "cant choose", "not sure", "prefer not to say".
  7. 7. Differences between panels in how peopleanswered all questions Percent of times panel differed from the others15%13% 11%11% 9% 7% 7% 5% 6% Expected 5% 4% by chance 3% 1%-1% SBA Lightspeed SSI Panel C Research Now Panel D Fulcrum River Panel B Count of the number of statistically significant differences (95% CI) as a percent of the total number of differences possible
  8. 8. Differences between panels in how peopleanswered snack purchase questions Purchase behaviour: Percent of times panel differed from the others15% 13% They were13% less likely to11% buy snacks9%7% 6% 6%5%3% 2% 2%1%-1% SBA Panel B River C Panel D River E Count of the number of statistically significant differences (95% CI) as a percent of the total number of differences possible
  9. 9. Differences between panels in how peopleanswered concept test questions Concept Test: Percent of times panel differed from the others30% 26%25%20% 16%15% 13% 11%10% 6% 5% 0% SBA Lightspeed Panel B SSI Panel C Research Now Panel D Fulcrum River Count of the number of statistically significant differences (95% CI) as a percent of the total number of differences possible
  10. 10. Differences between panels in how peopleanswered the purchase intent question Definitely will Probably will45%40%35%30%25% 28% 21%20% 23% 21%15% 15%10% 5% 13% 12% 9% 8% 6% 0% SBA Panel B River C Panel D River E Count of the number of statistically significant differences (95% CI) as a percent of the total number of differences possible
  11. 11. ConclusionsSample matters. A lot.Springboard America has fewer drop-outs, notablybetter data quality, and is “right down the middle”when it comes to comparison to the other panels onthe findings, unlike some panels which show realvariation—particularly on the concept test.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×