Does Sample Source Matter? [Part 1]

  • 703 views
Uploaded on

A comparison of some leading sample sources including river sample and router sample.

A comparison of some leading sample sources including river sample and router sample.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
703
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
5
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Does the Sample Source Matter?A comparison of some leading sample sourcesOctober 3, 2011
  • 2. Sample is the soiland water ofresearch.The quality of itdetermines thevalue of what youproduce.
  • 3. We compared SBA to four other samplesources that VC regularly usesWe conducted a five minute survey Nov 4 to 7, 2011with a sample of about 500 people each, using samplefrom Vision Critical’s Springboard America, and four ofthe leading sample providers that we have anonymisedas “Panel B”, “ Panel C”, “Panel D” and “River E”.The questionnaire included items from the US 2010General Social Survey, as well as a questions aboutsnack consumption and a concept test for a snack food.We counted the number of statistically significantdifferences between the panels (95% CI)
  • 4. Sample Quality: Dropout rate during the survey Dropouts as a percent of all who started the survey20% 16%15% 10%10% 8% 7% 5% 4% 0% SBA Panel B River C Panel D River E
  • 5. Data quality: straight-lining % who gave the same answer for all 13 trust ratings 10% 9% 7.9% 7.8% 8% 7% 6.5% 6% 5% 3.9% 4% 3% 2% 1% 0.5% 0% SBA Panel B River C Panel D River E
  • 6. Data quality: use of “don’t know” Average % "Dont Know"8%7% 6% 6% 6%6%5% 4% 4%4%3%2%1%0% SBA Panel B River C Panel D River E Average percent who choose the option "don’t know", "cant choose", "not sure", "prefer not to say".
  • 7. Differences between panels in how peopleanswered all questions Percent of times panel differed from the others15%13% 11%11% 9% 7% 7% 5% 6% Expected 5% 4% by chance 3% 1%-1% SBA Lightspeed SSI Panel C Research Now Panel D Fulcrum River Panel B Count of the number of statistically significant differences (95% CI) as a percent of the total number of differences possible
  • 8. Differences between panels in how peopleanswered snack purchase questions Purchase behaviour: Percent of times panel differed from the others15% 13% They were13% less likely to11% buy snacks9%7% 6% 6%5%3% 2% 2%1%-1% SBA Panel B River C Panel D River E Count of the number of statistically significant differences (95% CI) as a percent of the total number of differences possible
  • 9. Differences between panels in how peopleanswered concept test questions Concept Test: Percent of times panel differed from the others30% 26%25%20% 16%15% 13% 11%10% 6% 5% 0% SBA Lightspeed Panel B SSI Panel C Research Now Panel D Fulcrum River Count of the number of statistically significant differences (95% CI) as a percent of the total number of differences possible
  • 10. Differences between panels in how peopleanswered the purchase intent question Definitely will Probably will45%40%35%30%25% 28% 21%20% 23% 21%15% 15%10% 5% 13% 12% 9% 8% 6% 0% SBA Panel B River C Panel D River E Count of the number of statistically significant differences (95% CI) as a percent of the total number of differences possible
  • 11. ConclusionsSample matters. A lot.Springboard America has fewer drop-outs, notablybetter data quality, and is “right down the middle”when it comes to comparison to the other panels onthe findings, unlike some panels which show realvariation—particularly on the concept test.