• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Mixing the Right Sample Ingredients
 

Mixing the Right Sample Ingredients

on

  • 384 views

At the Advertising Research Foundation’s (ARF) 2011 annual re:think convention, a key issues forum presentation was held entitled Mixing the Right Sample Ingredients. The presentation was given by ...

At the Advertising Research Foundation’s (ARF) 2011 annual re:think convention, a key issues forum presentation was held entitled Mixing the Right Sample Ingredients. The presentation was given by Jackie Lorch, VP of Global Knowledge Management for Survey Sampling International. The presentation discussed which factors to blend and emphasized the importance of implementing multi-source testing.

Statistics

Views

Total Views
384
Views on SlideShare
384
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Mixing the Right Sample Ingredients Mixing the Right Sample Ingredients Presentation Transcript

    • Mixing the Right SampleIngredientsA New Source Recipe Jackie Lorch VP, Global Knowledge Management Survey Sampling International
    • What is the ProblemWe Are Trying to Solve?1. A systematic problem with online access panels
    • What is the ProblemWe Are Trying to Solve?2. A scarce resource
    • Many Reasons People Are Not in Panels Why have you not joined an Why do you no longer belong to an online research panel?1 online research panel?2I don’t want any more emails 7% Too many surveys 4%I think they are scams 6% Not enough surveys 27%It’s a waste of time 3% I never qualified for a survey 44%Not enough time to take surveys 11% Poor respondent experience 4%Never heard of an online research panel 20% Not enough time to take surveys 17%Don’t want to commit to a panel 22% Didn’t want to commit to a panel anymore 8%Never had any opportunities to join a panel 37% Other 8%Other 6% 1. Asked of people online who have never joined a research panel 2. Asked of people online, on a panel previously, not currently Multiple answers allowed
    • What is the ProblemWe Are Trying to Solve?
    • What is the ProblemWe Are Trying to Solve?
    • A Better Breed of Online Sample
    • Real Differences in BehaviorOwn an MP3 player
    • Why Not Use Socio-Demographics?• Demographics may not be the most helpful or relevant stabilizers• Demographic quotas work if the stratification chosen is relevant to questionnaire topic• A product may be liked across all demographics
    • Which are the right quotas? Sample 1 Sample 2 Balanced on: Balanced on: Age Ethnicity Gender Region
    • From Sources to People• Using thousands of sources• Metric must be at “person” level
    • Beyond Socio-DemographicsWith effective stabilization variables, no need tomaintain same blend of sources project-to-project
    • Factors for the BlendBroad set of factors defining groups of people,spanning multiple disciplinesPersonality traits Need for cognition (Cacioppo et al)Music preferences (Rentfrow) Neurographics measuresCognitive ability Propensity factors e.g., to participate,(Kahneman and Frederick) share, risk averse, attitude to privacyGeographic / personality alignment Chronotypes, “lark” or “owl”(Gosling and Rentfrow) (Roenneberg)Social Values. Schwarz has international Disruption/orienting reflex measure,benchmarks habituation disruption
    • Factors for the Blend• Questions created around 162 variables• Tested factor performance against questions on technology, hobbies, interests, brand preference, loyalty, ad awareness• Variables tested iteratively on power to “move the needle” on dependent variables• 14 variables identified as explaining more variance than socio-demographics alone; from these created 8 clusters to control
    • Multi-Source Testing• Broad set of factors defining groups of people, spanning multiple disciplines• Factors included such topics as attitudes to trust, gender roles, online behavior, personality traits, ideas and beliefs• Results varied by source
    • Results Closer to “Truth”On measures where benchmarks are available,blend comes closer to it Social Media iPod/MP3 Smartphone (Facebook or Ownership Ownership MySpace)SSI SurveySpot panel 37 16 60Benchmark 43 c.20-22 69SSI Blend 43 20 68Benchmark sources: Pew Research; comScore; Harris Interactive
    • Implementing the Blend
    • Taming the Stream
    • Lock and Control the Stream
    • Simulation Without ControlMP3 ownership – uncontrolled
    • Simulation Holding Clusters ConstantMP3 ownership – controlled
    • Practical Steps When Blending• Know your sources• Use a consistent blend for the entire project• Use benchmarks and calibrate• Understand blending techniques being used
    • In Summary• Multi-sourcing is the future, offering superior sample• We can keep multi-sourced samples consistent by pre-profiling participants• Standard socio-demographics aren’t enough• A concise list of variables can be used for a broad range of research subjects• Participant experience is key