Your SlideShare is downloading. ×
Does Sample Source Matter? [Part 2]
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Does Sample Source Matter? [Part 2]

753

Published on

A comparison of some leading sample sources including river sample and router sample.

A comparison of some leading sample sources including river sample and router sample.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
753
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Does the Sample Source Matter?Wave 2 of a comparison of leading sample sourcesSept 07, 2012
  • 2. Sample is the soiland water ofresearch.The quality of itdetermines thevalue of what youproduce.
  • 3. We compared other sample sources that VCregularly usesWe conducted a five minute survey April 12, 2012 witha sample of about 500 people each, using sample fromVision Critical’s Springboard America, and four of theleading sample providers that we have anonymised as“Panel B”, “River C”, “Panel D” and “River E”.The questionnaire included items from the US 2010General Social Survey, as well as a questions aboutsnack consumption and a concept test for a snack food.We counted the number of statistically significantdifferences between the panels (95% CI)
  • 4. Methodology cont.The questionnaire included items from the US 2010General Social Survey, as well as a questions aboutsnack consumption and a concept test for a snack food.We counted the number of statistically significantdifferences between the panels (95% CI), and looked atdrop-out rates, straight lining, and the use of “don’tknow”.
  • 5. Sample Quality: Dropout rate during the survey Drop-outs as a percent of all who started the survey Nov-11 Apr-1220% 19%18% 16%16%14%12% 10%10% 8% 9% 8% 7% 6% 5% 5% 5% 4% 4% 4% 2% 2% 0 0 0% SBA - SBA - Panel B Lightspeed Panel C SSI Research Panel D River United Panel E Targeted Routed Now Sample
  • 6. Data quality: straight-lining10% 9% 7.9% 7.8% 8% 7% 6.5% 6% 5.5% 5.6% 5% 4.8% 3.9% 4% 3.1% 3% 2.7% 2.1% 2% NA 1% 0.7% 0.5% 0% SBA Panel B SBA Routed Lightspeed Panel C SSI Panel D Research SB River Panel E uSamp Now
  • 7. Data quality: use of “don’t know” Nov-11 Apr-1210% 9% 8% 7% 6% 6% 6% 6% 5% 5% 4% 4% 4% 4% 4% 4% 4% 4% 3% 3% 2% 1% 0 0 0% SBA - SBA - Panel B Lightspeed Panel C SSI Research SB River Panel E Panel D uSamp Targeted Routed Now Average percent who choose the option "don’t know", "cant choose", "not sure", "prefer not to say".
  • 8. Differences between SBA and other panels inhow people answered all questions10% 9% 8% 7% 7% 6% Expected 5% 4% by chance 4% 4% 3% 3% 2% 2% 2% 1% 0% Panel B SBA Routed Lightspeed Panel C SSI Panel D Research Panel E uSamp River Now Count of the number of statistically significant differences (95% CI) as a percent of the total number of questions
  • 9. Differences between SBA and other panels inhow people answered all questionsWhile, for the most part, these differences appearwithin the realm of random, there were somedifferences that seem to be very real.For example, in both April and November Panel Brespondents were less likely to purchase snackproducts and had lower purchase intent scores fora snack concept.
  • 10. Differences between panels in how peopleanswered the purchase intent question Nov-11 Apr-1250% 46% 44% 44%45% 41% 40% 39%40% 34% 33%35% 31% 30%30% 29%25% 20%20%15%10% 5% 0 0 0% SBA - SBA - Lightspeed Panel C Research Panel B SSI Panel D River Panel E United Targeted Routed Now Sample Which statement below best describes how likely you would be purchase this snack if it were available today?—top 2 box: definitely and probably
  • 11. ConclusionsSample matters. A lot.Springboard America has fewer drop-outs, notablybetter data quality, and is “right down the middle”when it comes to comparison to the other panels onthe findings, unlike some panels which show realvariation—particularly on the concept test.SBA also shows consistency in how peopleanswered, as illustrated by reaction to the concept.
  • 12. ConclusionsThe same cannot be said for the other panels,however. They showed considerable variationbetween time 1 and time 2 in how they reacted tothe snack concept, which was a non-trendy concept.This may be a concern for tracking. I will dig deeperinto the comparison between waves in anotherreport.

×