Sample Balancing in 2012Carter Cathey, Vice President, Excellence InitiativesHow to set and manage your sample balancingop...
Sample Balancing in 2012    Webinar Agenda     Introduction to Balancing              Quick Poll: What type of     Limi...
Sample Balancing in 2012    Introduction to Balancing     Market Research = Learning      about Specific Populations      ...
Sample Balancing in 2012    Limits of Balancing    Type and Mechanism of Balancing MUST be CUSTOM.    The wrong type/mecha...
Sample Balancing in 2012    Types of Balancing – “Good Mix”      Definition: An acceptable mix of ages, genders, and othe...
Sample Balancing in 2012    Types of Balancing – “Representative”      Definition: Representative of some known universe....
Sample Balancing in 2012    Types of Balancing – Internet Representative      Definition: Representative of the online un...
Sample Balancing in 2012    Types of Balancing – “Balanced”      Definition: Generally refers to equal amounts of demogra...
Sample Balancing in 2012    Types of Balancing – Census Balancing      Definition: Generally refers to the census-based U...
Sample Balancing in 2012     Mechanisms of Balancing – Outbound       Definition: Creating a balanced set of respondents ...
Sample Balancing in 2012     Mechanisms of Balancing – Inbound       Definition: Creating a set of respondents to invite ...
Sample Balancing in 2012     Mechanisms of Balancing – Completes       Definition: Balancing on “Completes” is essentiall...
Sample Balancing in 2012     To Nest or Not to Nest… This is the question.       Top-Line (aka un-nested) Quotas: These q...
Sample Balancing in 2012     To Nest or Not to Nest… This is the question.       Key Benefits:              Ensures opti...
Sample Balancing in 2012 – 2010 Census15   © 2012 Research Now
Sample Balancing in 2012 – 2010 Census16   © 2012 Research Now
Sample Balancing in 2012     2010 Census – Key Changes   Quick Poll: What version of                                 the c...
Sample Balancing in 2012     What does it all mean? Research Now Recommendation       Always Balance: At a minimum, balan...
Questions?     Carter Cathey     ccathey@Researchnow.com19   © 2012 Research Now
Upcoming SlideShare
Loading in …5
×

April Webinar: Sample Balancing in 2012

1,798 views

Published on

How to set and manage your sample balancing options to ensure quality data and happy clients.

Presentation by: Carter Cathey, Vice President, Excellence Initiatives

Published in: Business, Technology
  • Be the first to comment

April Webinar: Sample Balancing in 2012

  1. 1. Sample Balancing in 2012Carter Cathey, Vice President, Excellence InitiativesHow to set and manage your sample balancingoptions to ensure quality data and happyclients.
  2. 2. Sample Balancing in 2012 Webinar Agenda  Introduction to Balancing  Quick Poll: What type of  Limits of Balancing balancing do you use? 1. “Good Mix”  Types of Balancing 2. Representative  “Good Mix” 3. Internet Representative  Representative 4. Population Sampling  Internet Rep  Population Sampling 5. Balanced Sampling  Balanced 6. Census Balancing  Census Balancing 7. I use something different. 8. All of the above, depending  Mechanisms to Achieve Balancing on the needs of the study.  Outbound 9. I don’t know / don’t care.  Inbound Balancing is the responsibility  Completes of the panel provider, right?  To nest of not to nest?  2010 Census and Implications for Balancing2 © 2012 Research Now
  3. 3. Sample Balancing in 2012 Introduction to Balancing Market Research = Learning about Specific Populations To Learn about Specific Populations: Respondents must Represent Target Audience. This is the heart of sample balancing.3 © 2012 Research Now
  4. 4. Sample Balancing in 2012 Limits of Balancing Type and Mechanism of Balancing MUST be CUSTOM. The wrong type/mechanism of balancing can lead to:  Bad Data  Feasibility Issues  Higher Project Costs Balancing doesn’t always make sense with TARGETED sample.4 © 2012 Research Now
  5. 5. Sample Balancing in 2012 Types of Balancing – “Good Mix”  Definition: An acceptable mix of ages, genders, and other demos, without major skews.  Feasibility Impact: This type of balancing is generally the least panel restrictive.  Key Concerns: This is fine for most research.5 © 2012 Research Now
  6. 6. Sample Balancing in 2012 Types of Balancing – “Representative”  Definition: Representative of some known universe.  Feasibility Impact: Depends on the universe requested.  Key Question: Representative of what?  Key Concerns:  Clients sometimes request “representative” sampling without being specific on the details of the known universe.  Or, they can request “representative” sampling for universes that are unknown or unreachable.6 © 2012 Research Now
  7. 7. Sample Balancing in 2012 Types of Balancing – Internet Representative  Definition: Representative of the online universe.  Feasibility Impact, Key Questions, and Key Concerns: Largely the same as “Representative Balancing. Types of Balancing – Population Sampling  Definition: Population sampling is representative balancing to the details of a specific population.  Feasibility Impact, Key Questions, and Key Concerns: Largely the same as “Representative Balancing.7 © 2012 Research Now
  8. 8. Sample Balancing in 2012 Types of Balancing – “Balanced”  Definition: Generally refers to equal amounts of demographic groups, like 50/50 gender, 33/33/33 on three age ranges.  Feasibility Impact: Depends on the elements balanced.  Key Question: What elements need to be balanced?  Key Concerns: This is fine for most research.8 © 2012 Research Now
  9. 9. Sample Balancing in 2012 Types of Balancing – Census Balancing  Definition: Generally refers to the census-based US population, with varying levels of nested targets.  Feasibility Impact: Typically, this is the most difficult to deliver and most panel restrictive.  Key Question: What elements need to be balanced to the Census?  Key Concerns: This type of balancing can be difficult to deliver and greatly reduce the amount of completed interviews Research Now is able to deliver.9 © 2012 Research Now
  10. 10. Sample Balancing in 2012 Mechanisms of Balancing – Outbound  Definition: Creating a balanced set of respondents to invite to the research engagement.  Feasibility Impact: Marginal impact.  Key Question: What are the balancing goals?  Key Benefits: Can be easier to apply weighting controls.  Key Concerns: Frequently does not take into account differential response rates.10 © 2012 Research Now
  11. 11. Sample Balancing in 2012 Mechanisms of Balancing – Inbound  Definition: Creating a set of respondents to invite to the survey designed to deliver representative respondents to the first question of the engagement.  Feasibility Impact: Can be more significant.  Key Question: What are the balancing goals?  Key Benefits: Corrects for differential response rates.  Key Concerns:  Can make applying weighting controls more challenging.  Response rates are not always consistent.  Can be harder to deliver.11 © 2012 Research Now
  12. 12. Sample Balancing in 2012 Mechanisms of Balancing – Completes  Definition: Balancing on “Completes” is essentially using the quota management functionality of the survey instrument to manage the distribution and attributes of the completed interviews.  Feasibility Impact: Varies to a high degree based on the quotas.  Key Benefits:  More control.  Easier to use multiple sample providers.  Minimizes impact of sampling errors.  Key Concerns:  Not a good fit for all research.  Must avoid impossible quota cells.  More sophisticated survey programming platform.12 © 2012 Research Now
  13. 13. Sample Balancing in 2012 To Nest or Not to Nest… This is the question.  Top-Line (aka un-nested) Quotas: These quotas are not related to each other.  N=1600  Male: n=800  Female: n=800  < 35 YO: n=800  > 35 YO: n=800  HHI >$35k: n=800  HHI <$35K: n=800  Nested Quotas: These quotas are related to each other.13 © 2012 Research Now
  14. 14. Sample Balancing in 2012 To Nest or Not to Nest… This is the question.  Key Benefits:  Ensures optimal distribution of sample.  Ensures that you will have minimum sample sizes to make assertions about these sub-sets of the respondent pool.  Key Concerns:  Can very easily create nonsensical quota cells. Think 18 year-old millionaires and Un-acculturated Hispanics with Graduate Degrees.  Can extend field time and make sampling more difficult.  Can lower qualification rates and raise project costs.14 © 2012 Research Now
  15. 15. Sample Balancing in 2012 – 2010 Census15 © 2012 Research Now
  16. 16. Sample Balancing in 2012 – 2010 Census16 © 2012 Research Now
  17. 17. Sample Balancing in 2012 2010 Census – Key Changes Quick Poll: What version of the census do you use? 1. 2000 Census 2. 2010 Census 3. A hybrid that I alone have master-engineered 4. I don’t use the census.17 © 2012 Research Now
  18. 18. Sample Balancing in 2012 What does it all mean? Research Now Recommendation  Always Balance: At a minimum, balance for age, gender, and income.  Always Share Balancing Objectives: Always share your specific balancing needs, expectations, and objectives with your partners and providers.  Consider Nesting: When appropriate, nesting age, gender, and region should be applied.  Don’t Always Nest Ethnicity: Ethnicity targets should only be nested when required by the needs of a specific project.  Quotas should be discussed, and never assumed. Using quotas (rather than sampling targets) to control for sample mix may have a more significant effect.  Use 2010 Census as Default. If you haven’t already, you should consider using the 2010 estimates.  Tracking Studies. There will likely be a noticable impact in the data if you switch a currently fielding tracking study from 2000 to 2010 Census balancing.18 © 2012 Research Now
  19. 19. Questions? Carter Cathey ccathey@Researchnow.com19 © 2012 Research Now

×