1. Context Effects in Candidate
Favorability Ratings:
Lessons from the 2012 Elections
Eran Ben-Porath
SSRS
Damla Ergun, Gary Langer, Greg Holyk
Langer Research Associates
Scott Clement, Jon Cohen
Capital Insight
3. Attitude reports, as all social judgments, are context
dependent
Prior research: Responses are susceptible to
pronounced context effects, including question wording,
format and order
Context effects may occur at several points in the
judgment process
◦ Comprehension of the question
◦ Generating the judgment
◦ Response formatting
4. When asked about their attitudes, people are unlikely to
have an answer ready for use available in their
memories
Hence, respondents need to form a judgment based on
information they have
Thus, people’s responses reflect both constructive and
memory-based processes
◦ E.g., Feldman & Lynch, 1988
5. During this judgment formation process,
respondents
◦ Rarely retrieve all information that may be relevant to the
task at hand
◦ Often truncate the search process as soon as they have
some certainty that they have enough information to
form a judgment
6. Reported attitudes are based on
◦ A subset of relevant information that is most
accessible in memory
temporarily accessible
◦ and information that’s available regardless of
contextual cues
chronically accessible (i.e., unprompted)
Often, the temporarily accessible information is
brought to mind in the process of answering a
preceding question
7. Once respondents understand the intended
meaning of the question, they recall relevant
information
The impact of information depends on how it’s
used
In a survey, a previous question may provide
additional information for judgment formation in
two ways:
◦ Assimilation effects
◦ Contrast effects
8. Information brought to mind by a previous
question is used to create a representation of
the attitude object
◦ Example: Given his popularity, people who know Gen.
Colin Powell’s party membership evaluate the
Republican Party more positively
(Stapel & Schwarz, 1998)
9. Information brought to mind by the previous
question is used to create a standard of
comparison
◦ Example: People who know that Gen. Colin Powell
declined to run as a Republican presidential candidate
evaluate the Republican Party more negatively
(Stapel & Schwarz, 1998)
10. Studies often focus on how question order affects
evaluation of the general category vs. a specific
member of the category
◦ e.g., trustworthiness of politicians vs. Richard Nixon
Schwarz & Bless, 1992
11. Studies often focus on how question order affects
evaluation of the general category vs. a specific
member of the category
◦ e.g., trustworthiness of politicians vs. Richard Nixon
Schwarz & Bless, 1992
Are there order effects when the objects are of the
same category?
12. Are question-order effects observable in basic
evaluations of public figures when the two questions
don’t conform to the category-member pattern?
Can one public figure provide context for evaluating
another?
What predicts susceptibility to the effect?
◦ Education/information?
◦ Partisanship/interest?
13. Data collected during the 2012 primaries and the
presidential election campaign
ABC News/Washington Post polls
Field work by Social Science Research
Solutions via its Excel omnibus survey
1,000 weekly random-sample telephone
interviews inc. 300 via cell phone
14. Favorability questions are a basic measure of a
public figure’s personal popularity:
◦ Overall, do you have a favorable or unfavorable
impression of (ITEM)? Do you feel that way strongly or
somewhat?
◦ e.g., Barack Obama, Mitt Romney
Respondents randomly assigned to different
question orders
19. Effect for Obama was inconsistent (3 out of 9
months Obama was less favorable/more
unfavorable when asked first)
No observed effect on DK about Obama
20.
21.
22.
23. For Romney: effect for unfavorability was
significant 7 out of 9 months
Effect for DK: 6 out of 9 months
Effect weakens over time; then disappears
24. Obama
Favorability
Obama First
(%)
Romney
First (%)
∆
(%)
Less than HS graduate Favorable 58.2 56.1 2.1
Unfavorable 36.3 36.9 -0.6
Don't know 5.5 7.0 -1.5
HS graduate Favorable 49.0 46.8 2.2
Unfavorable 46.0 47.4 -1.4
Don't know 5.0 5.8 -0.8
Some college Favorable 47.7 46.1 1.6
Unfavorable 48.1 50.7 -2.6
Don't know 4.2 3.2 1.0
College graduate Favorable 51.1 48.0 3.1
Unfavorable 46.4 49.8 -3.4
Don't know 2.4 2.2 0.2
Graduate school or more Favorable 58.1 57.5 0.6
Unfavorable 39.6 41.3 -1.7
Don't know 2.3 1.2 1.1
25. Obama
Favorability
Obama First
(%)
Romney
First (%)
∆
(%)
Less than HS graduate Favorable 58.2 56.1 2.1
Unfavorable 36.3 36.9 -0.6
Don't know 5.5 7.0 -1.5
HS graduate Favorable 49.0 46.8 2.2
Unfavorable 46.0 47.4 -1.4
Don't know 5.0 5.8 -0.8
Some college Favorable 47.7 46.1 1.6
Unfavorable 48.1 50.7 -2.6
Don't know 4.2 3.2 1.0
College graduate Favorable 51.1 48.0 3.1
Unfavorable 46.4 49.8 -3.4
Don't know 2.4 2.2 0.2
Graduate school or more Favorable 58.1 57.5 0.6
Unfavorable 39.6 41.3 -1.7
Don't know 2.3 1.2 1.1
*** Effect is weak and inconsistent; very slight variation among
education levels
26. Romney
Favorability
Obama First
(%)
Romney
First (%)
∆
(%)
Less than HS graduate Favorable 25.9 30. -4.1
Unfavorable 50.8 38.7 12.1*
Don't know 23.2 31.3 -8.1*
HS graduate Favorable 36.9 36.7 0.2
Unfavorable 48.7 42.7 6.0*
Don't know 14.4 20.6 -6.2*
Some college Favorable 39.6 41.9 -2.3
Unfavorable 50.3 42.9 7.4*
Don't know 10.1 15.1 -5.0*
College graduate Favorable 44.3 45.8 -1.5
Unfavorable 47.2 44.5 2.7
Don't know 8.4 9.7 -1.3
Graduate school or more Favorable 41.9 42.3 -0.4
Unfavorable 53.0 50.7 2.3
Don't know 5.1 6.9 -1.8
27. Romney
Favorability
Obama First
(%)
Romney
First (%)
∆
(%)
Less than HS graduate Favorable 25.9 30. -4.1
Unfavorable 50.8 38.7 12.1*
Don't know 23.2 31.3 -8.1*
HS graduate Favorable 36.9 36.7 0.2
Unfavorable 48.7 42.7 6.0*
Don't know 14.4 20.6 -6.2*
Some college Favorable 39.6 41.9 -2.3
Unfavorable 50.3 42.9 7.4*
Don't know 10.1 15.1 -5.0*
College graduate Favorable 44.3 45.8 -1.5
Unfavorable 47.2 44.5 2.7
Don't know 8.4 9.7 -1.3
Graduate school or more Favorable 41.9 42.3 -0.4
Unfavorable 53.0 50.7 2.3
Don't know 5.1 6.9 -1.8
*** Effect is between unfavorable and DK and is apparent in lower
education levels
32. We find pronounced question-order effects:
◦ Observed higher “unfavorable” and lower “no opinion”
ratings for Mitt Romney, the lesser known challenger– when
asked after Obama
◦ Effect weakened over course of campaign
◦ Strongest effect among respondents with lower education,
political independents and Democrats
Modest effects observed for other less-known
figures
33.
34. Role of familiarity of the attitude object
◦ Decreased effect over time correlates with closer attention
to campaign
35. Which order is preferable?
◦ These are almost different questions because of the
familiarity-gap
◦ Asking “better-known first”: reduces respondent burden and
is a better approximation of vote choice
◦ The first question position has the benefit of not introducing
cues, but is difficult to maintain if asking a series (3+
names)
◦ Rotation/randomization and larger series may limit the order
effect of a single item
Editor's Notes
Since at least the 1950s researchers have established: that survey responses are susceptible to pronounced context effects, including question wording, format and order e.g., Hyman & Sheatsley, 1950; Schwarz Strack, 1991; Krosnick & Schuman, 1988; Knauper, Schwarz, Park, & Fritsch, 2007
*** When asked about their attitudes, people are unlikely to have an answer ready for use available in their memories even when they hold a general attitude on the topic, issue or person asked about
*** E.g., Of which party has Gen. Colin Powell recently become a member? (Stapel & Schwarz, 1998) The highly-respected Colin Powell’s membership results in more positive evaluations of the Republican Party
*** Responses are more negative than they’d be without the prior question, if information was positive. E.g., Which party asked Gen. Colin Powell to run as its presidential candidate? (Stapel & Schwarz, 1998) The highly respected Powell (the positive standard) makes the Republican party look less good (Powell had declined the offer)
For example: 1. Would you say you trust President Richard Nixon a great deal, somewhat, not too much or not at all? (category member) 2. Would you say you trust politicians a great deal, somewhat, not too much or not at al? (general category)
For example: 1. Would you say you trust President Richard Nixon a great deal, somewhat, not too much or not at all? (category member) 2. Would you say you trust politicians a great deal, somewhat, not too much or not at al? (general category)
For example: 1. Do you have a favorable or unfavorable impression of [politician A]? (specific) 2. Do you have a favorable or unfavorable impression of [politician B]? (specific)
Difference in Obama’s favorability ratings when asked first compared to when asked second. * No strong pattern in favorability – slight tendency for more favorability when asked first
Difference in Obama’s favorability ratings when asked first compared to when asked second.
Difference in Obama’s favorability ratings when asked first compared to when asked second. DKs hardly affected at all
* Week effect, if any on Romney's favorable ratings
* But a very pronounced effect on Romney's unfavorable percentage, especially earlier in the campaign: when Obama is asked first – Romney I much more unfavorable
* The difference is in DK: when Romney is asked first DKs are significantly higher then when Obama is asked first Obama provides a context (which translates negatively, but that’s another story) Effect weaken toward the end of the campaign
* Almost no differences on Obama FAV
* This is consistent with the belief that Obama provides context for those with those less likely to have a Romney attitude cognitively available (low ed=low exposure and low interest)
* This is consistent with the belief that Obama provides context for those with those less likely to have a Romney attitude cognitively available (low ed=low exposure and low interest)
Possible overlap with education Non-Republicans: most likely to NOT know Romney (and like Obama)
***The favorability of the lesser-known Santorum is more susceptible to order effect
I’m arguing that Obama’s role as polarized political figure aids his “favorable” supporters in recognizing Romney as a member of the out-group, someone who threatens someone they like.
