Your SlideShare is downloading. ×
0
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
AAPOR 2013 SSRS Langer CapInsight Context Effects
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

AAPOR 2013 SSRS Langer CapInsight Context Effects

311

Published on

Published in: News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
311
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Since at least the 1950s researchers have established: that survey responses are susceptible to pronounced context effects, including question wording, format and order e.g., Hyman & Sheatsley, 1950; Schwarz Strack, 1991; Krosnick & Schuman, 1988; Knauper, Schwarz, Park, & Fritsch, 2007
  • *** When asked about their attitudes, people are unlikely to have an answer ready for use available in their memories even when they hold a general attitude on the topic, issue or person asked about
  • *** E.g., Of which party has Gen. Colin Powell recently become a member? (Stapel & Schwarz, 1998) The highly-respected Colin Powell’s membership results in more positive evaluations of the Republican Party
  • *** Responses are more negative than they’d be without the prior question, if information was positive. E.g., Which party asked Gen. Colin Powell to run as its presidential candidate? (Stapel & Schwarz, 1998) The highly respected Powell (the positive standard) makes the Republican party look less good (Powell had declined the offer)
  • For example: 1. Would you say you trust President Richard Nixon a great deal, somewhat, not too much or not at all? (category member) 2. Would you say you trust politicians a great deal, somewhat, not too much or not at al? (general category)
  • For example: 1. Would you say you trust President Richard Nixon a great deal, somewhat, not too much or not at all? (category member) 2. Would you say you trust politicians a great deal, somewhat, not too much or not at al? (general category)
  • For example: 1. Do you have a favorable or unfavorable impression of [politician A]? (specific) 2. Do you have a favorable or unfavorable impression of [politician B]? (specific)
  • Difference in Obama’s favorability ratings when asked first compared to when asked second. * No strong pattern in favorability – slight tendency for more favorability when asked first
  • Difference in Obama’s favorability ratings when asked first compared to when asked second.
  • Difference in Obama’s favorability ratings when asked first compared to when asked second. DKs hardly affected at all
  • * Week effect, if any on Romney's favorable ratings
  • * But a very pronounced effect on Romney's unfavorable percentage, especially earlier in the campaign: when Obama is asked first – Romney I much more unfavorable
  • * The difference is in DK: when Romney is asked first DKs are significantly higher then when Obama is asked first Obama provides a context (which translates negatively, but that’s another story) Effect weaken toward the end of the campaign
  • * Almost no differences on Obama FAV
  • * This is consistent with the belief that Obama provides context for those with those less likely to have a Romney attitude cognitively available (low ed=low exposure and low interest)
  • * This is consistent with the belief that Obama provides context for those with those less likely to have a Romney attitude cognitively available (low ed=low exposure and low interest)
  • Possible overlap with education Non-Republicans: most likely to NOT know Romney (and like Obama)
  • ***The favorability of the lesser-known Santorum is more susceptible to order effect
  • I’m arguing that Obama’s role as polarized political figure aids his “favorable” supporters in recognizing Romney as a member of the out-group, someone who threatens someone they like.
  • *
  • Transcript

    • 1. Context Effects in CandidateFavorability Ratings:Lessons from the 2012 ElectionsEran Ben-PorathSSRSDamla Ergun, Gary Langer, Greg HolykLanger Research AssociatesScott Clement, Jon CohenCapital Insight
    • 2.  Context Effects◦ and Question WordingResearch QuestionsMethod: Favorability StudiesFindingsImplications
    • 3.  Attitude reports, as all social judgments, are contextdependent Prior research: Responses are susceptible topronounced context effects, including question wording,format and order Context effects may occur at several points in thejudgment process◦ Comprehension of the question◦ Generating the judgment◦ Response formatting
    • 4.  When asked about their attitudes, people are unlikely tohave an answer ready for use available in theirmemories Hence, respondents need to form a judgment based oninformation they have Thus, people’s responses reflect both constructive andmemory-based processes◦ E.g., Feldman & Lynch, 1988
    • 5.  During this judgment formation process,respondents◦ Rarely retrieve all information that may be relevant to thetask at hand◦ Often truncate the search process as soon as they havesome certainty that they have enough information toform a judgment
    • 6.  Reported attitudes are based on◦ A subset of relevant information that is mostaccessible in memory temporarily accessible◦ and information that’s available regardless ofcontextual cues chronically accessible (i.e., unprompted) Often, the temporarily accessible information isbrought to mind in the process of answering apreceding question
    • 7.  Once respondents understand the intendedmeaning of the question, they recall relevantinformation The impact of information depends on how it’sused In a survey, a previous question may provideadditional information for judgment formation intwo ways:◦ Assimilation effects◦ Contrast effects
    • 8.  Information brought to mind by a previousquestion is used to create a representation ofthe attitude object◦ Example: Given his popularity, people who know Gen.Colin Powell’s party membership evaluate theRepublican Party more positively (Stapel & Schwarz, 1998)
    • 9.  Information brought to mind by the previousquestion is used to create a standard ofcomparison◦ Example: People who know that Gen. Colin Powelldeclined to run as a Republican presidential candidateevaluate the Republican Party more negatively (Stapel & Schwarz, 1998)
    • 10.  Studies often focus on how question order affectsevaluation of the general category vs. a specificmember of the category◦ e.g., trustworthiness of politicians vs. Richard Nixon Schwarz & Bless, 1992
    • 11.  Studies often focus on how question order affectsevaluation of the general category vs. a specificmember of the category◦ e.g., trustworthiness of politicians vs. Richard Nixon Schwarz & Bless, 1992 Are there order effects when the objects are of thesame category?
    • 12.  Are question-order effects observable in basicevaluations of public figures when the two questionsdon’t conform to the category-member pattern? Can one public figure provide context for evaluatinganother? What predicts susceptibility to the effect?◦ Education/information?◦ Partisanship/interest?
    • 13.  Data collected during the 2012 primaries and thepresidential election campaign ABC News/Washington Post polls Field work by Social Science ResearchSolutions via its Excel omnibus survey 1,000 weekly random-sample telephoneinterviews inc. 300 via cell phone
    • 14.  Favorability questions are a basic measure of apublic figure’s personal popularity:◦ Overall, do you have a favorable or unfavorableimpression of (ITEM)? Do you feel that way strongly orsomewhat?◦ e.g., Barack Obama, Mitt Romney Respondents randomly assigned to differentquestion orders
    • 15. Obama First(%)Romney First(%)∆ (%)Obama Favorable 51.2 48.9 2.3*Obama Unfavorable 44.9 47.2 -2.3*DK ObamaFavorability3.9 3.9 0Romney Favorable 38.9 40.4 -1.5RomneyUnfavorable49.7 43.7 6.0*DK RomneyFavorability11.4 15.9 -4.5*Obama: N=10,743Romney: N=10,658
    • 16.  Effect for Obama was inconsistent (3 out of 9months Obama was less favorable/moreunfavorable when asked first) No observed effect on DK about Obama
    • 17.  For Romney: effect for unfavorability wassignificant 7 out of 9 months Effect for DK: 6 out of 9 months Effect weakens over time; then disappears
    • 18. ObamaFavorabilityObama First(%)RomneyFirst (%)∆(%)Less than HS graduate Favorable 58.2 56.1 2.1Unfavorable 36.3 36.9 -0.6Dont know 5.5 7.0 -1.5HS graduate Favorable 49.0 46.8 2.2Unfavorable 46.0 47.4 -1.4Dont know 5.0 5.8 -0.8Some college Favorable 47.7 46.1 1.6Unfavorable 48.1 50.7 -2.6Dont know 4.2 3.2 1.0College graduate Favorable 51.1 48.0 3.1Unfavorable 46.4 49.8 -3.4Dont know 2.4 2.2 0.2Graduate school or more Favorable 58.1 57.5 0.6Unfavorable 39.6 41.3 -1.7Dont know 2.3 1.2 1.1
    • 19. ObamaFavorabilityObama First(%)RomneyFirst (%)∆(%)Less than HS graduate Favorable 58.2 56.1 2.1Unfavorable 36.3 36.9 -0.6Dont know 5.5 7.0 -1.5HS graduate Favorable 49.0 46.8 2.2Unfavorable 46.0 47.4 -1.4Dont know 5.0 5.8 -0.8Some college Favorable 47.7 46.1 1.6Unfavorable 48.1 50.7 -2.6Dont know 4.2 3.2 1.0College graduate Favorable 51.1 48.0 3.1Unfavorable 46.4 49.8 -3.4Dont know 2.4 2.2 0.2Graduate school or more Favorable 58.1 57.5 0.6Unfavorable 39.6 41.3 -1.7Dont know 2.3 1.2 1.1*** Effect is weak and inconsistent; very slight variation amongeducation levels
    • 20. RomneyFavorabilityObama First(%)RomneyFirst (%)∆(%)Less than HS graduate Favorable 25.9 30. -4.1Unfavorable 50.8 38.7 12.1*Dont know 23.2 31.3 -8.1*HS graduate Favorable 36.9 36.7 0.2Unfavorable 48.7 42.7 6.0*Dont know 14.4 20.6 -6.2*Some college Favorable 39.6 41.9 -2.3Unfavorable 50.3 42.9 7.4*Dont know 10.1 15.1 -5.0*College graduate Favorable 44.3 45.8 -1.5Unfavorable 47.2 44.5 2.7Dont know 8.4 9.7 -1.3Graduate school or more Favorable 41.9 42.3 -0.4Unfavorable 53.0 50.7 2.3Dont know 5.1 6.9 -1.8
    • 21. RomneyFavorabilityObama First(%)RomneyFirst (%)∆(%)Less than HS graduate Favorable 25.9 30. -4.1Unfavorable 50.8 38.7 12.1*Dont know 23.2 31.3 -8.1*HS graduate Favorable 36.9 36.7 0.2Unfavorable 48.7 42.7 6.0*Dont know 14.4 20.6 -6.2*Some college Favorable 39.6 41.9 -2.3Unfavorable 50.3 42.9 7.4*Dont know 10.1 15.1 -5.0*College graduate Favorable 44.3 45.8 -1.5Unfavorable 47.2 44.5 2.7Dont know 8.4 9.7 -1.3Graduate school or more Favorable 41.9 42.3 -0.4Unfavorable 53.0 50.7 2.3Dont know 5.1 6.9 -1.8*** Effect is between unfavorable and DK and is apparent in lowereducation levels
    • 22. ObamaFavorabilityObama First(%)RomneyFirst (%)∆(%)Republican Favorable 14.1 13.7 .4Unfavorable 83.8 84.8 -1.0Dont know 2.1 1.5 .6Democrat Favorable 85.3 84.3 1Unfavorable 12.9 13.2 -.3Dont know 1.8 2.5 -.7Independent Favorable 47.3 44.7 2.6Unfavorable 47.4 50.2 -2.8Dont know 5.2 5.1 .1*** Weak non-significant effect, apparent only among independents
    • 23. RomneyFavorabilityObama beforeRomney (%)Romney beforeObama (%)∆(%)Republican Favorable 72.4 74.5 -2.1Unfavorable 20.9 17.7 3.2*Dont know 6.7 7.9 -1.2Democrat Favorable 15.7 17.2 -1.5Unfavorable 74.8 68.4 6.4*Dont know 9.6 14.4 -4.8*Independent Favorable 38.6 38.4 .2Unfavorable 49.0 43.4 5.6*Dont know 12.4 18.2 -5.8**** Order effect is highest among Democrats and independents.
    • 24. Gingrich AskedFirst (%)Romney AskedFirst (%)∆(%)Gingrich Favorable 30.3 28.5 1.8Gingrich Unfavorable 49.2 54.8 -5.6DK GingrichFavorability20.5 16.7 3.8Romney Favorable 34.1 36.1 -2.0Romney Unfavorable 48.7 41.6 7.1DK RomneyFavorability17.2 22.3 -5.1Gingrich: N=654Romney: N=649
    • 25. Santorum AskedFirst (%)Romney First (%) ∆(%)Santorum Favorable 31.1 37.9 -6.8Santorum Unfavorable 37.8 39.5 -1.8DK SantorumFavorability31.1 22.6 8.5Romney Favorable 37.6 35.0 2.6Romney Unfavorable 43.6 44.6 -1.0DK Romney Favorability 18.8 20.3 -1.5Santorum: N=325Romney: N=326
    • 26.  We find pronounced question-order effects:◦ Observed higher “unfavorable” and lower “no opinion”ratings for Mitt Romney, the lesser known challenger– whenasked after Obama◦ Effect weakened over course of campaign◦ Strongest effect among respondents with lower education,political independents and Democrats Modest effects observed for other less-knownfigures
    • 27.  Role of familiarity of the attitude object◦ Decreased effect over time correlates with closer attentionto campaign
    • 28.  Which order is preferable?◦ These are almost different questions because of thefamiliarity-gap◦ Asking “better-known first”: reduces respondent burden andis a better approximation of vote choice◦ The first question position has the benefit of not introducingcues, but is difficult to maintain if asking a series (3+names)◦ Rotation/randomization and larger series may limit the ordereffect of a single item

    ×