A Methodological Review of PCAC in the Free-Choice Paradigm

1. A Methodological Review of
PCAC in the Free-Choice
Paradigm
Amir Mohammad Tahamtan
Graduate School of Management and Economics
Sharif University of Techology
May 7, 2016
5/6/2016, Friday 1

2. A historical overview of the methodology of
PCAC in the FCP
1. Chen (unpublished) , 2008, Rationalization and cognitive dissonance: do choices affect or reflect
preferences
2. Sagarin & Skowronski , 2008, The implications of imperfect measurement for free-choice carry-over
effects
3. Chen and Risen, 2008, Is choice a reliable predictor of choice?
4. Sagarin & Skowronski , 2009, In pursuit of the proper null
5. Chen and Risen, 2010, How Choice Affects and Reflects Preferences: Revisiting the Free-Choice
Paradigm
6. Risen and Chen, 2010, How to Study Choice-Induced Attitude Change: Strategies for Fixing the Free-
Choice Paradigm
7. Alós-Ferrer et al., 2012, Choices and preferences: Evidence from implicit choices and response times
8. Izuma & Murayama, 2013, Choice-induced preference change in the free-choice paradigm: a critical
methodological review
9. Alós-Ferrer & Shi, 2015, Choice-induced preference change and the free-choice paradigm: A
clarification
5/6/2016, Friday 2

3. Scope of the problem in FCP type 1 (Chen 2008)
• Subjects are never completely indifferent
between two options.
• subjects should be expected to choose good C
66% of the time.
• CD would suggest numbers significantly higher
than 66%.
• Three-door problem (or Montey Hall effect) in
economics.
 FCP type 1: Shifts in choices;
 Egan et al (2007): Children choose C 63% and capuchins choose C 60% of the time;
Use a continuous scroll for rating
5/6/2016, Friday 3

4. Scope of the problem in FCP type 2 (Chen 2008)
• FCP type 2: Shifts in rankings or ratings; most of
the literature.
• If the initial ranking is an imperfect measure of
preferences, then a subject’s choices teach us
something new about their preferences;
• The selection bias is weaker in easy choices
because fewer choose against their initial
rankings, and absent in no choice.
• If a subject’s choices always conformed to their
initial ratings, their choices would contain no
new information.
• If 25% of subjects who initially rank A over B
then choose B, then we could easily expect an
increase in the spread of preferences of at least
one ranking point, completely absent CD.
P(A rises and B falls | choosing A) > P(B rises and A falls | choosing A)
P(A rises and B falls) = P(B rises and A falls)
P(choosing A | A rises and B falls) > P(choosing A | B rises and A falls)
5/6/2016, Friday 4

5. Sagarin & Skowronsky (2009)
• Choice behavior tends to be probabilistic:
options that are slightly more valuable are
chosen only slightly more often than options
perceived as slightly less valuable.
• a statistical ‘‘expected value” for choice-making
behavior:
• The probability of choosing the preferred
item over the less preferred item
• The probability that the chosen item
correctly identifies the under-lying
preference.
5/6/2016, Friday 5

6. Chen & Risen’s reply(2009)
• It is essential that researchers experimentally control for revealed preferences rather than
speculate how much of a role they may play.
• a misunderstanding of the function of the null-hypothesis :
• the correct analysis would be to calculate what would render the observed results
statistically indistinguishable from, rather than identical to experimental results.
• a misunderstanding of preference-measurement psychometrics.
• The fact that external measures may contain very little predictive power for an individual’s
choice does not imply that choice itself must be only very loosely tied to the preferences the
individual has at that moment.
• This is very trivial that when you force the subject to only and only choose one option, finally
he chooses one of them but you cannot be sure that this was due to his definite preference.
How do you know that why he has chosen that item??!!
5/6/2016, Friday 6

7. Sagarin & Skowronsky’s reply (2009)
• Subjects would select a preferred option over
a less preferred option only a proportion of
the time, and that this proportion is likely
related to the magnitude of the preference
difference between the two options.
• The more often the assumptions made by
Chen (2008) overstate subjects’ real
behavioral tendencies in choice-making, the
more the expected value for the second
choice in a typical two-choice study falls
below 66.7% and approaches 50%.
• sufficiently accurate pretest (or suite of
pretests) are needed.
 How common are intransitive permutations?
Establishing a relation between the magnitude of any pretest
difference and choice probabilities
Transitive permutations
Intransitive permutations
5/6/2016, Friday 7

8. The Problem for Studying Moderators and
Mediators in the FCP
(Chen 2008, Chen & Risen 2010)
• The criticism also applies to forms of the FCP that measure spreading of neural activation after a
choice or examine moderators or mediators of spreading.
• Example: Japanese subjects show less cognitive dissonance than Canadians. One possible
confound is that even if never asked to make choices, Japanese subjects may rate and re-rate
goods more consistently that do Canadians. If this is true, then their subsequent choices contain
less information and induce less selection.
5/6/2016, Friday 8

9. The Problem of Self-selection for spread (Chen 2008,
Chen & Risen 2010)
• Because participants are not randomly assigned to make their choice, this procedure results in
participants ‘self-selecting’ how chosen spread is calculated.
• If participants choose different items because they have different underlying preferences for the
two items, then it is unclear whether the documented effects in the FCP are the result of attitude
change following choice or, a reflection of existing preferences that are revealed by choice.
5/6/2016, Friday 9

10. Chen & Risen (2010)
• With a formal mathematical proof and based on three assumptions demonstrate that the FCP
will measure positive chosen spread even if people have perfectly stable preferences.
1. People’s ratings ⁄ rankings are at least partially (meaningfully) guided by their preferences.
2. People’s choices are at least partially guided by their preferences.
3. People’s ratings ⁄ rankings are often not a perfect measure of their preferences.
 What is the difference between first and last assumption?
5/6/2016, Friday 10

11. Proposed solutions to fix FCP, Chen and Risen,
2010
• Researchers can isolate the effect of the choice process on subsequent preferences by
i. ensuring that all participants make the same choice
ii. controlling for the information revealed by choice
iii. removing the information revealed by choice
iv. manipulating the choices that people make
5/6/2016, Friday 11

12. 1. Ensure that everyone makes the same
choice
• First, researchers must effectively get all participants to make the same choice.
• Second, in doing so, researchers must avoid directly manipulating preferences.
• It is not a preference-based choice, it is not a real choice, it is an induced one, a random one.
Something similar to ESTEKHKARE or flipping coins. This process does not make people to believe
a free-choice. If there is an attitude change, it is not Choice-induced attitude change but
acceptance-induced attitude change. Do the writers mean this to be used as control condition?
5/6/2016, Friday 12

13. 2. Controlling for the information revealed by
choice
• Rate-Rate-Choose (RRC)
• The spreading found for RRC participants demonstrates that spreading can occur in the absence
of dissonance reduction.
• using a within-subject RCRC design, researchers can calculate the RRC and RCR spread for each
participant, allowing for a paired comparison.
• I think it is wrong since the first RCR would affect the subsequent C.
5/6/2016, Friday 13

14. 3. Removing the information from choice
• so that participants’ choices do not reflect their preference.
• 2nd assumption no longer holds.
• for example, by making the choice ‘blind’.
5/6/2016, Friday 14

15. 4. Manipulation the choice
• Participants rank 15 art prints.
• Before choosing between #7 and #9, they are asked to flip a coin.
• If heads, they will get $1 extra if they choose #7
• If tails, they will get $1 extra if they choose #9.
• If the $1 shifts people’s choices, but does not directly influence preferences, then researchers can
test for attitude change following choice by calculating how much #7 improves and #9 declines if
the coin lands on heads and how much #9 improves and #7 declines if the coin land on tails.
• If spreading is positive, it suggests that the choice process influenced subsequent preferences.
5/6/2016, Friday 15

16. One more recent solution (Alós-Ferrer et al., 2012)
• Implicit choice paradigm
• measure preference changes for two items that are not compared in a direct choice between
them, but rather through other comparisons.
• participants made choices between a and h, and b and l.
• h > a >b > l in the first preference rating. participants were likely to choose h over a, and b over l.
• Then, preference change between the first and second rating were compared between the
rejected item a and the chosen item b.
• If one of choices in the a-h and b-l pairings was not as expected, the four items were excluded
from the analysis (i.e., selection bias).
5/6/2016, Friday 16

17. Alós-Ferrer & Shi (2015)
• We show that the result in Chen & Risen (2010) is mathematically incorrect.
• Specifically, we present a formal model of decision making which satisfies all assumptions in that
article but such that spreading needs not be positive in the absence of choice-induced preference
change.
• Hence, although the free-choice paradigm is flawed, the present research shows that reasonable
models of human behavior need not predict positive spreading. As a consequence, experimental
results remain informative.
5/6/2016, Friday 17