Your SlideShare is downloading. ×
AAPOR 2013 SSRS Langer CapInsight Context Effects
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

AAPOR 2013 SSRS Langer CapInsight Context Effects

295

Published on

Published in: News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
295
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Since at least the 1950s researchers have established: that survey responses are susceptible to pronounced context effects, including question wording, format and order e.g., Hyman & Sheatsley, 1950; Schwarz Strack, 1991; Krosnick & Schuman, 1988; Knauper, Schwarz, Park, & Fritsch, 2007
  • *** When asked about their attitudes, people are unlikely to have an answer ready for use available in their memories even when they hold a general attitude on the topic, issue or person asked about
  • *** E.g., Of which party has Gen. Colin Powell recently become a member? (Stapel & Schwarz, 1998) The highly-respected Colin Powell’s membership results in more positive evaluations of the Republican Party
  • *** Responses are more negative than they’d be without the prior question, if information was positive. E.g., Which party asked Gen. Colin Powell to run as its presidential candidate? (Stapel & Schwarz, 1998) The highly respected Powell (the positive standard) makes the Republican party look less good (Powell had declined the offer)
  • For example: 1. Would you say you trust President Richard Nixon a great deal, somewhat, not too much or not at all? (category member) 2. Would you say you trust politicians a great deal, somewhat, not too much or not at al? (general category)
  • For example: 1. Would you say you trust President Richard Nixon a great deal, somewhat, not too much or not at all? (category member) 2. Would you say you trust politicians a great deal, somewhat, not too much or not at al? (general category)
  • For example: 1. Do you have a favorable or unfavorable impression of [politician A]? (specific) 2. Do you have a favorable or unfavorable impression of [politician B]? (specific)
  • Difference in Obama’s favorability ratings when asked first compared to when asked second. * No strong pattern in favorability – slight tendency for more favorability when asked first
  • Difference in Obama’s favorability ratings when asked first compared to when asked second.
  • Difference in Obama’s favorability ratings when asked first compared to when asked second. DKs hardly affected at all
  • * Week effect, if any on Romney's favorable ratings
  • * But a very pronounced effect on Romney's unfavorable percentage, especially earlier in the campaign: when Obama is asked first – Romney I much more unfavorable
  • * The difference is in DK: when Romney is asked first DKs are significantly higher then when Obama is asked first Obama provides a context (which translates negatively, but that’s another story) Effect weaken toward the end of the campaign
  • * Almost no differences on Obama FAV
  • * This is consistent with the belief that Obama provides context for those with those less likely to have a Romney attitude cognitively available (low ed=low exposure and low interest)
  • * This is consistent with the belief that Obama provides context for those with those less likely to have a Romney attitude cognitively available (low ed=low exposure and low interest)
  • Possible overlap with education Non-Republicans: most likely to NOT know Romney (and like Obama)
  • ***The favorability of the lesser-known Santorum is more susceptible to order effect
  • I’m arguing that Obama’s role as polarized political figure aids his “favorable” supporters in recognizing Romney as a member of the out-group, someone who threatens someone they like.
  • *
  • Transcript

    • 1. Context Effects in CandidateFavorability Ratings:Lessons from the 2012 ElectionsEran Ben-PorathSSRSDamla Ergun, Gary Langer, Greg HolykLanger Research AssociatesScott Clement, Jon CohenCapital Insight
    • 2.  Context Effects◦ and Question WordingResearch QuestionsMethod: Favorability StudiesFindingsImplications
    • 3.  Attitude reports, as all social judgments, are contextdependent Prior research: Responses are susceptible topronounced context effects, including question wording,format and order Context effects may occur at several points in thejudgment process◦ Comprehension of the question◦ Generating the judgment◦ Response formatting
    • 4.  When asked about their attitudes, people are unlikely tohave an answer ready for use available in theirmemories Hence, respondents need to form a judgment based oninformation they have Thus, people’s responses reflect both constructive andmemory-based processes◦ E.g., Feldman & Lynch, 1988
    • 5.  During this judgment formation process,respondents◦ Rarely retrieve all information that may be relevant to thetask at hand◦ Often truncate the search process as soon as they havesome certainty that they have enough information toform a judgment
    • 6.  Reported attitudes are based on◦ A subset of relevant information that is mostaccessible in memory temporarily accessible◦ and information that’s available regardless ofcontextual cues chronically accessible (i.e., unprompted) Often, the temporarily accessible information isbrought to mind in the process of answering apreceding question
    • 7.  Once respondents understand the intendedmeaning of the question, they recall relevantinformation The impact of information depends on how it’sused In a survey, a previous question may provideadditional information for judgment formation intwo ways:◦ Assimilation effects◦ Contrast effects
    • 8.  Information brought to mind by a previousquestion is used to create a representation ofthe attitude object◦ Example: Given his popularity, people who know Gen.Colin Powell’s party membership evaluate theRepublican Party more positively (Stapel & Schwarz, 1998)
    • 9.  Information brought to mind by the previousquestion is used to create a standard ofcomparison◦ Example: People who know that Gen. Colin Powelldeclined to run as a Republican presidential candidateevaluate the Republican Party more negatively (Stapel & Schwarz, 1998)
    • 10.  Studies often focus on how question order affectsevaluation of the general category vs. a specificmember of the category◦ e.g., trustworthiness of politicians vs. Richard Nixon Schwarz & Bless, 1992
    • 11.  Studies often focus on how question order affectsevaluation of the general category vs. a specificmember of the category◦ e.g., trustworthiness of politicians vs. Richard Nixon Schwarz & Bless, 1992 Are there order effects when the objects are of thesame category?
    • 12.  Are question-order effects observable in basicevaluations of public figures when the two questionsdon’t conform to the category-member pattern? Can one public figure provide context for evaluatinganother? What predicts susceptibility to the effect?◦ Education/information?◦ Partisanship/interest?
    • 13.  Data collected during the 2012 primaries and thepresidential election campaign ABC News/Washington Post polls Field work by Social Science ResearchSolutions via its Excel omnibus survey 1,000 weekly random-sample telephoneinterviews inc. 300 via cell phone
    • 14.  Favorability questions are a basic measure of apublic figure’s personal popularity:◦ Overall, do you have a favorable or unfavorableimpression of (ITEM)? Do you feel that way strongly orsomewhat?◦ e.g., Barack Obama, Mitt Romney Respondents randomly assigned to differentquestion orders
    • 15. Obama First(%)Romney First(%)∆ (%)Obama Favorable 51.2 48.9 2.3*Obama Unfavorable 44.9 47.2 -2.3*DK ObamaFavorability3.9 3.9 0Romney Favorable 38.9 40.4 -1.5RomneyUnfavorable49.7 43.7 6.0*DK RomneyFavorability11.4 15.9 -4.5*Obama: N=10,743Romney: N=10,658
    • 16.  Effect for Obama was inconsistent (3 out of 9months Obama was less favorable/moreunfavorable when asked first) No observed effect on DK about Obama
    • 17.  For Romney: effect for unfavorability wassignificant 7 out of 9 months Effect for DK: 6 out of 9 months Effect weakens over time; then disappears
    • 18. ObamaFavorabilityObama First(%)RomneyFirst (%)∆(%)Less than HS graduate Favorable 58.2 56.1 2.1Unfavorable 36.3 36.9 -0.6Dont know 5.5 7.0 -1.5HS graduate Favorable 49.0 46.8 2.2Unfavorable 46.0 47.4 -1.4Dont know 5.0 5.8 -0.8Some college Favorable 47.7 46.1 1.6Unfavorable 48.1 50.7 -2.6Dont know 4.2 3.2 1.0College graduate Favorable 51.1 48.0 3.1Unfavorable 46.4 49.8 -3.4Dont know 2.4 2.2 0.2Graduate school or more Favorable 58.1 57.5 0.6Unfavorable 39.6 41.3 -1.7Dont know 2.3 1.2 1.1
    • 19. ObamaFavorabilityObama First(%)RomneyFirst (%)∆(%)Less than HS graduate Favorable 58.2 56.1 2.1Unfavorable 36.3 36.9 -0.6Dont know 5.5 7.0 -1.5HS graduate Favorable 49.0 46.8 2.2Unfavorable 46.0 47.4 -1.4Dont know 5.0 5.8 -0.8Some college Favorable 47.7 46.1 1.6Unfavorable 48.1 50.7 -2.6Dont know 4.2 3.2 1.0College graduate Favorable 51.1 48.0 3.1Unfavorable 46.4 49.8 -3.4Dont know 2.4 2.2 0.2Graduate school or more Favorable 58.1 57.5 0.6Unfavorable 39.6 41.3 -1.7Dont know 2.3 1.2 1.1*** Effect is weak and inconsistent; very slight variation amongeducation levels
    • 20. RomneyFavorabilityObama First(%)RomneyFirst (%)∆(%)Less than HS graduate Favorable 25.9 30. -4.1Unfavorable 50.8 38.7 12.1*Dont know 23.2 31.3 -8.1*HS graduate Favorable 36.9 36.7 0.2Unfavorable 48.7 42.7 6.0*Dont know 14.4 20.6 -6.2*Some college Favorable 39.6 41.9 -2.3Unfavorable 50.3 42.9 7.4*Dont know 10.1 15.1 -5.0*College graduate Favorable 44.3 45.8 -1.5Unfavorable 47.2 44.5 2.7Dont know 8.4 9.7 -1.3Graduate school or more Favorable 41.9 42.3 -0.4Unfavorable 53.0 50.7 2.3Dont know 5.1 6.9 -1.8
    • 21. RomneyFavorabilityObama First(%)RomneyFirst (%)∆(%)Less than HS graduate Favorable 25.9 30. -4.1Unfavorable 50.8 38.7 12.1*Dont know 23.2 31.3 -8.1*HS graduate Favorable 36.9 36.7 0.2Unfavorable 48.7 42.7 6.0*Dont know 14.4 20.6 -6.2*Some college Favorable 39.6 41.9 -2.3Unfavorable 50.3 42.9 7.4*Dont know 10.1 15.1 -5.0*College graduate Favorable 44.3 45.8 -1.5Unfavorable 47.2 44.5 2.7Dont know 8.4 9.7 -1.3Graduate school or more Favorable 41.9 42.3 -0.4Unfavorable 53.0 50.7 2.3Dont know 5.1 6.9 -1.8*** Effect is between unfavorable and DK and is apparent in lowereducation levels
    • 22. ObamaFavorabilityObama First(%)RomneyFirst (%)∆(%)Republican Favorable 14.1 13.7 .4Unfavorable 83.8 84.8 -1.0Dont know 2.1 1.5 .6Democrat Favorable 85.3 84.3 1Unfavorable 12.9 13.2 -.3Dont know 1.8 2.5 -.7Independent Favorable 47.3 44.7 2.6Unfavorable 47.4 50.2 -2.8Dont know 5.2 5.1 .1*** Weak non-significant effect, apparent only among independents
    • 23. RomneyFavorabilityObama beforeRomney (%)Romney beforeObama (%)∆(%)Republican Favorable 72.4 74.5 -2.1Unfavorable 20.9 17.7 3.2*Dont know 6.7 7.9 -1.2Democrat Favorable 15.7 17.2 -1.5Unfavorable 74.8 68.4 6.4*Dont know 9.6 14.4 -4.8*Independent Favorable 38.6 38.4 .2Unfavorable 49.0 43.4 5.6*Dont know 12.4 18.2 -5.8**** Order effect is highest among Democrats and independents.
    • 24. Gingrich AskedFirst (%)Romney AskedFirst (%)∆(%)Gingrich Favorable 30.3 28.5 1.8Gingrich Unfavorable 49.2 54.8 -5.6DK GingrichFavorability20.5 16.7 3.8Romney Favorable 34.1 36.1 -2.0Romney Unfavorable 48.7 41.6 7.1DK RomneyFavorability17.2 22.3 -5.1Gingrich: N=654Romney: N=649
    • 25. Santorum AskedFirst (%)Romney First (%) ∆(%)Santorum Favorable 31.1 37.9 -6.8Santorum Unfavorable 37.8 39.5 -1.8DK SantorumFavorability31.1 22.6 8.5Romney Favorable 37.6 35.0 2.6Romney Unfavorable 43.6 44.6 -1.0DK Romney Favorability 18.8 20.3 -1.5Santorum: N=325Romney: N=326
    • 26.  We find pronounced question-order effects:◦ Observed higher “unfavorable” and lower “no opinion”ratings for Mitt Romney, the lesser known challenger– whenasked after Obama◦ Effect weakened over course of campaign◦ Strongest effect among respondents with lower education,political independents and Democrats Modest effects observed for other less-knownfigures
    • 27.  Role of familiarity of the attitude object◦ Decreased effect over time correlates with closer attentionto campaign
    • 28.  Which order is preferable?◦ These are almost different questions because of thefamiliarity-gap◦ Asking “better-known first”: reduces respondent burden andis a better approximation of vote choice◦ The first question position has the benefit of not introducingcues, but is difficult to maintain if asking a series (3+names)◦ Rotation/randomization and larger series may limit the ordereffect of a single item

    ×