Your SlideShare is downloading. ×
  • Like
Reducing Bias in Public Opinion Polls
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Reducing Bias in Public Opinion Polls

  • 1,938 views
Published

Reducing Bias in Public Opinion Polls - By Autumn Carter …

Reducing Bias in Public Opinion Polls - By Autumn Carter
Presentation Delivered at Stanford University, Summer Research College, August 2009

Presentation of some of my results from my research. Over the course of the project, I analyzed 7 survey experiments across 9 unique vendors, who conducted the survey via phone or the internet and using either a probability sample or a non-probability sample. In all, about 10000 unique individuals were surveyed.

This presentation highlights the question bias of Agree/Disagree type questions and the response bias associated with the Acquiescence Effect.

I conducted my research through Stanford\'s Political Psychology Research Group. pprg.stanford.edu

Published in Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,938
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
26
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Reducing Bias in Public Opinion Polls Autumn Carter Professor Jon Krosnick Summer Research College 2009
  • 2. Satisfy Your Constituents... Or Suffer the Consequences.
  • 3. So, how do we effectively determine what Americans are feeling?
  • 4. Public Opinion Polling
  • 5. But Are the Polls Accurate? ? =
  • 6. Critics Don't Think So... “ The dirty little secret of the polling industry is that, all too often, its findings are based on flawed methodology and dubious assumptions.” -Arianna Huffington, Huffington Post Founder
  • 7. Sample Public Opinion Question Form 1 Individuals are more to blame than social conditions for crime and lawlessness in this country. Agree Disagree
  • 8. Sample Public Opinion Question Form 1 Individuals are more to blame than social conditions for crime and lawlessness in this country. Agree: 59.6% Disagree: 40.4%
  • 9. Sample Public Opinion Question Form 1 Individuals are more to blame than social conditions for crime and lawlessness in this country. Agree: 59.6% Disagree: 40.4% Form 2 Social conditions are more to blame than individuals for crime and lawlessness in this country. Agree Disagree
  • 10. Sample Public Opinion Question Form 1 Individuals are more to blame than social conditions for crime and lawlessness in this country. Agree: 59.6% Disagree: 40.4% Form 2 Social conditions are more to blame than individuals for crime and lawlessness in this country. Agree: 56.8% Disagree: 43.2% Acquiescence Effect: 16.4%
  • 11. What is Acquiescence? “ ...a presumed tendency for respondents to agree with attitude statements presented to them.” -Schuman and Presser (1981)
  • 12. Why do Respondents Acquiesce?
    • Norms of Conduct : Be polite and agreeable
    • Status Differential : Defer to higher status
    • Satisficing : Choosing the easiest response because it requires less thinking
  • 13. Can we reduce acquiescence by administering the survey via the Internet instead of by telephone?
  • 14. Would Internet Administration Reduce Acquiescence?
    • Norms of Conduct : Be polite and agreeable
        • Do you have to be polite to a computer?
    • Status Differential : Defer to higher status
        • Is there a status difference without a human interviewer?
    • Satisficing : Choosing the easiest response because it requires less thinking
        • Are people less likely to satisfice on Internet surveys? (Chang & Krosnick, in press)
  • 15. The Present Research
    • Replicate Schuman & Presser (1981)
    • Mode effects: Telephone vs. the Internet
    • Sample effects: Representative vs. Non-Representative
  • 16. How Do You Survey a Representative Sample Via the Internet?
    • Knowledge Networks randomly dialed phone numbers and invited people to join a panel (of about 50,000 adults).
    • Provided Internet access to homes without it.
    • Periodically emails about 1,000 panelists to complete surveys and receive incentives.
  • 17. A Classic Acquiescence Experiment Form 1 Individuals are more to blame than social conditions for crime and lawlessness in this country. Agree Disagree Form 2 Social conditions are more to blame than individuals for crime and lawlessness in this country. Agree Disagree
  • 18. Replication in a National Survey Telephone survey of a representative sample: Internet survey of a representative sample: * p < 0.05 + p <0.10 11.5%* 7.1%+ TelephoneXInternet Interaction: n.s.
  • 19. What about Internet surveys of people who are not representative of the national population?
  • 20. Non-Probability Sample Internet Surveys
    • Volunteers click on banner ads and “opt-in” into the panel.
  • 21. You've all seen these...
  • 22. Non-Probability Sample Internet Surveys
    • Volunteers click on banner ads and “opt-in” to the panel.
    • Participants receive money or prize.
    • Perhaps they acquiescence less because they like taking surveys and have more practice.
    • In fact, previous research has found less acquiescence vs. a telephone survey (Chang and Krosnick, in press)
  • 23. Acquiescence Bias * * * * * † * † † *
  • 24. If anything, non-probability samples increase acquiescence. So we should be cautious when interpreting these types of surveys.
  • 25. What if we just didn't ask questions that ask respondents to agree or disagree with a statement?
  • 26. Balanced Forced-Choice Questions Form 1 Which do you think is more responsible for crime and lawlessness in this country: individuals or social conditions ? Individuals Social Conditions Form 2 Which do you think is more responsible for crime and lawlessness in this country: social conditions or individuals ? Social Conditions Individuals
  • 27. Response Order Effects ns ns ns ns ns ns ns ns ns
  • 28. Conclusions
    • Acquiescence bias is real and can undermine public opinion polls and possibly the democratic process.
    • Internet administration did not significantly reduce acquiescence.
    • Non-probability sample surveys showed, if anything, more acquiescence.
    • BUT WAIT! You can simply ask the questions using the balanced forced choice format, eliminating order effects.