Valuation and choice: Using values
PSYC 596F
10/20/2009
Presenter: David Yokum
Chapter 28: Antonio Rangel. The computation...
Outline
• Rangel: How does the brain compute and
compare values? The random-walk model.
• Platt and Padoa-Schioppa: What a...
Value-based decision making
1) Create representation of decision problem
2) Assign values to considered actions
3) Compare...
Simplifications
1) Create representation of decision problem
2) Assign values to considered actions
3) Compare valuations
...
Random dot motion task
Fig. 28.1
Random-walk model (RWM): measure
of relative evidence
1. Encode moment-to-moment sensory evidence
2. Accumulator measuring...
RWM: behavioral evidence
Fig. 28.3
RWM: neural correlates (1)
• Middle temporal area contains groups of
neurons that respond preferentially to stimuli
moving...
RWM: neural correlates (2)
• Time-courses of neural activity in lateral
intraparietal area rise to single threshold
about ...
RWM: neural correlates (3)
• What about threshold computation?
• Maybe LIP? Maybe superior colliculus?
Simple goal-directed choice
Fig. 28.7
RWM redux: attention-driven choice
• Relative value again computed, but this time
an advantage is given to whatever item i...
AD-RWM: behavioral evidence
Fig. 28.9
Random walk model: Evaluation?
• Filippo: Rangel is worried that the model is
limited to binary choice, but this concern i...
Neural substrates of value
computations
• Orbitofrontal cortex (OFC): absolute value of
good
• Lateral intraparietal parie...
Economic choice task
Fig. 29.1
Economic choice task
Fig. 29.1
OFC and chosen value (i.e., value of
whatever juice ultimately chosen)
Fig. 29.2
OFC and offer value (i.e., value of one
of the two juices alone)
Fig. 29.3
OFC and taste
Fig. 29.3
OFC invariant for menu changes
Fig. 29.4
OFC upshot
• Representation of value in OFC is independent
of visuo-motor contingencies and menu
invariant.
• Essentially ...
OFC data: Evaluation?
• Trevor: “I don't know much about how dementia
works. When dementia patients have OFC
damage, is it...
OFC data: Evaluation?
• Trevor: “The authors present at least 4
different functional types [of info encoded by
OFC neurons...
OFC data: Cardinal versus ordinal
• Filippo: Problems with conclusion that OFC activity
represents cardinal value. Opposin...
Lateral intraparietal cortex (LIP)
• Linking sensory signals with motor commands,
as well as guiding sensory attention.
• ...
LIP: pay-per-view task
Fig. 29.5
Fig. 29.6
LIP: abstract, relative value
LIP signal independent of modality
• “These findings suggest that decisions based
on value operate on a common currency th...
Question
• Phil: “Is there a difference between "primary" rewards
(such as food/drink), "secondary" rewards (such as
money...
Question
• Trevor: “I found the brief discussion of
mutlimodality valuation very interesting and
wanted more information a...
LIP data: Evaluation?
• Mirre: “Are LIP neurons only involved in
perceptual decision-making or has LIP activity
only been ...
LIP data: Evaluation?
• Kaisa: “The fact that different economical variables are reflected in the LIP has
been puzzling me...
LIP data: Evaluation?
• Luke: “No disrespect to Michael Platt and colleagues, but I
found something lacking in their argum...
“…Not determined to give up on studying
regions that have no clear role in calculating
value, he decides to focus on the p...
Posterior Cingulate Cortex (CGp)
• Anatomical and neurophysiological evidence
suggest a role in “signaling motivationally
...
CGp signals are subjective
CGp and evaluation of stimuli
• CGp neurons respond most strongly to visual
events that are unpredictable in space or time...
CGp: Evaluation?
• Luke: “Not determined to give up on studying regions that
have no clear role in calculating value, he d...
CGp: Evaluation?
• Luke: “Why are goal values (i.e. motivations)
always neglected in models of value? If things
like appet...
Question
• Assuming neuronal correlates of value exist in
the OFC, LIP, and CGp, why are there so many
representations of ...
Question
• Mirre: “Different neural correlates of value
have been found in the brain. In sensory
areas, value signals cont...
Goods- versus Actions-based
• GBM: economic choice is modular and before
action selection
• ABM: economic choice “unfolds ...
Other questions
• Trevor: “…Both chapters this week (and last
week) sort of handwaved at the notion that
understanding the...
Other questions
• Luke “Things that are more interesting to discuss then the scientific
career of Michael Platt might incl...
Upcoming SlideShare
Loading in …5
×

Yokum 10.20.09 Presentation Valuation And Choice Using Values

429 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
429
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Rewarded some amount (e.g., quantity of juice squirt) for correct fixation; otherwise nothing. Probability of correct response = %coherence.
  • RWM Predictions:
    - logistic relation between choice and coherence
    - RT right-skewed, with RT longer for error trials
    - RT decrease with stimulus coherence, & accuracy improves
    - Implements sequential probability ratio test
  • Salzman et al (1990, 1992) showed that microstimulation of MT cells biased perceptual judgment in favor of those cells’ response field

    Solid curve – in MT response field; dashed – not in response field

    Encoding of moment-to-moment sensory evidence?
  • also has neurons with response fields; activity increases with coherence;

    Microstimulation of LIP neurons also biases toward response field

    Shadlen & Newsome (2001)
  • Now there is no probability estimation – just a value computation (viz., identify outcome and assign it a value) .
  • Predicts, like RWM, that choice and difficult logistic function; reaction times increase with difficulty; but also:

    Last fixations are shorter
    Last fixation is to chosen item (if sufficiently better than item in next-to-last trial
    Choice bias in favor of first seen item (and last)
  • Longer exposure times increase probability of appetitive item being chosen
  • Bottom graphs just demonstrate that it is subjective value of juice, not anything about physical properties, that is being encoded; on bottom right, e.g., red and green lines represent whether movement was to left or right, respectively.

    OFC neurons, unlike other reviewed neurons, do not depend on visuo-motor contingencies of choice.
  • Also, not shown, some neurons just preferred one type of juice to another – taste.
  • 1.2*2.1=2.5
  • Juice amounts versus type of picture. Right hand figure is of men and pictures of women.
  • Did NOT observe value modulation if monkeys not permitted to choose, in line with idea that LIP neurons signal the relative value of options AVAILABLE for orienting

    Value encoding is RELATIVE
  • Predict: modulation of sensory motor processing in cortical areas like LIP, which presumably lie downstream of OFC, should be independent of the modality of the desired outcome
  • CPg neurons respond most strongly to visual events that are unpredictable in space or time, and strength of activity predicts how accurately monkeys subsequently respond; also respond to unpredictable rewards or omission of predictable rewards

    For example, BOLD signal in CGp elevated if chocholate rated pleasant or unpleasant, but not if just neutral
  • Consistent with attentional theories of learning, which posit that reward prediction errors highlight motivationally significant events
  • Yokum 10.20.09 Presentation Valuation And Choice Using Values

    1. 1. Valuation and choice: Using values PSYC 596F 10/20/2009 Presenter: David Yokum Chapter 28: Antonio Rangel. The computation and comparison of value in goal-directed choice. Chapter 29: Michael Platt and Camillo Padoa-Schioppa. Neuronal representations of value.
    2. 2. Outline • Rangel: How does the brain compute and compare values? The random-walk model. • Platt and Padoa-Schioppa: What are the neural substrates of value computations? OFC, LIP, and CGp. • Throughout, your questions and comments!
    3. 3. Value-based decision making 1) Create representation of decision problem 2) Assign values to considered actions 3) Compare valuations (select highest valued action) 4) Measure desirability of outcomes 5) Use feedback to update
    4. 4. Simplifications 1) Create representation of decision problem 2) Assign values to considered actions 3) Compare valuations (select highest valued action) 4) Measure desirability of outcomes 5) Use feedback to update  Goal-directed choice: – two options – familiar outcomes – immediate consumption
    5. 5. Random dot motion task Fig. 28.1
    6. 6. Random-walk model (RWM): measure of relative evidence 1. Encode moment-to-moment sensory evidence 2. Accumulator measuring net evidence 3. Thresholds implementing decision criterion Fig. 28.2
    7. 7. RWM: behavioral evidence Fig. 28.3
    8. 8. RWM: neural correlates (1) • Middle temporal area contains groups of neurons that respond preferentially to stimuli moving in certain directions (i.e., response fields); activity increases with coherence. Fig. 28.5
    9. 9. RWM: neural correlates (2) • Time-courses of neural activity in lateral intraparietal area rise to single threshold about 100 ms before saccade; an accumulator signal? Fig. 28.6
    10. 10. RWM: neural correlates (3) • What about threshold computation? • Maybe LIP? Maybe superior colliculus?
    11. 11. Simple goal-directed choice Fig. 28.7
    12. 12. RWM redux: attention-driven choice • Relative value again computed, but this time an advantage is given to whatever item is being attended to. Fig. 28.8
    13. 13. AD-RWM: behavioral evidence Fig. 28.9
    14. 14. Random walk model: Evaluation? • Filippo: Rangel is worried that the model is limited to binary choice, but this concern is unwarranted. • How is threshold determined?
    15. 15. Neural substrates of value computations • Orbitofrontal cortex (OFC): absolute value of good • Lateral intraparietal parietal cortex (LIP): relative value of good • Posterior cingulate cortex (CGp): subjective value; update LIP
    16. 16. Economic choice task Fig. 29.1
    17. 17. Economic choice task Fig. 29.1
    18. 18. OFC and chosen value (i.e., value of whatever juice ultimately chosen) Fig. 29.2
    19. 19. OFC and offer value (i.e., value of one of the two juices alone) Fig. 29.3
    20. 20. OFC and taste Fig. 29.3
    21. 21. OFC invariant for menu changes Fig. 29.4
    22. 22. OFC upshot • Representation of value in OFC is independent of visuo-motor contingencies and menu invariant. • Essentially a cardinal, or absolute, representation of value.
    23. 23. OFC data: Evaluation? • Trevor: “I don't know much about how dementia works. When dementia patients have OFC damage, is it selective to a particular part of the OFC, or to neurons in the OFC that either get their inputs from or project to specific regions, or is it just widespread, unselective damage to the OFC? If it is selective, is it possible that the cell death corresponds to neurons of particular functional types (chosen, offer, or taste) neurons?”
    24. 24. OFC data: Evaluation? • Trevor: “The authors present at least 4 different functional types [of info encoded by OFC neurons]: chosen value, offer value A, offer value B, and taste. Do we know anything about the properties of these different neurons? Are "offer" neurons and "chosen" neurons different in any meaningful ways or part of different functional networks in the brain?”
    25. 25. OFC data: Cardinal versus ordinal • Filippo: Problems with conclusion that OFC activity represents cardinal value. Opposing data from Tremblay and Schultz (1999). “[M.P. and C.P-S.] briefly discuss the clash between the two lines of research by noting that i) they may have recorded from different orbital areas and ii) that the paradigm adopted in Tremblay et al. may have induced adaptation, which in turn may explain the differences between the two sets of results. They then conclude this brief line of reasoning by claiming that the “cardinal hypothesis” is confirmed by empirical data. “Apart from the fact that the strength of the argument is questionable, it is also not clear how one can discern encoding of ‘absolute value’ from other possible information…” • Luke: “…the conclusion that it [OFC] calculates "objective" value seems overstated and tenuous at best.”
    26. 26. Lateral intraparietal cortex (LIP) • Linking sensory signals with motor commands, as well as guiding sensory attention. • Modulated by expected value of different stimuli? – Enter Michael Platt’s c.v. – LIP neuronal activity proportional to EV of target (e.g., altered juice amount or probability of cue)
    27. 27. LIP: pay-per-view task Fig. 29.5
    28. 28. Fig. 29.6 LIP: abstract, relative value
    29. 29. LIP signal independent of modality • “These findings suggest that decisions based on value operate on a common currency that is independent of the modality of the goods under consideration or the actions they motivate” (450).
    30. 30. Question • Phil: “Is there a difference between "primary" rewards (such as food/drink), "secondary" rewards (such as money), and "social" rewards (such as viewing pictures of attractive members of the opposite sex): Clearly the value of food/drink varies with how hungry the animal (rat/monkey/human) is, while a secondary reward has value only in that it can be exchanged for a primary reward (i.e. Monopoly money is fairly useless), but social rewards might be different - for instance, can one ever get bored of looking at photos of pretty women? If hunger/thirst mediate the value of food/drink, what might mediate the value of photos of attractive women?...”
    31. 31. Question • Trevor: “I found the brief discussion of mutlimodality valuation very interesting and wanted more information about studies in humans that have tried to assess whether valuation is done the same across different types of information. Aside from the studies with female and male images discussed in the chapter, and some of the charitable giving research, are there any good examples of studies that have tried to look at decisions in which multiple modalities must be considered when making a choice?”
    32. 32. LIP data: Evaluation? • Mirre: “Are LIP neurons only involved in perceptual decision-making or has LIP activity only been measured using saccadic decision- making tasks? Do LIP neurons in human beings function similarly as in monkeys?”
    33. 33. LIP data: Evaluation? • Kaisa: “The fact that different economical variables are reflected in the LIP has been puzzling me for quite a while. What does it really tell us in studies where monkeys are answering with eye movements? Somehow I have gotten the feeling that always when there is a significant difference in the behavior, LIP neurons also show the same effect (sometimes amazingly accurately). Are the measurements of LIP activity and behavioral responses (saccades) actually indicating the same thing at different stages, i.e., selection of motor action and the action itself? Is the choice calculated elsewhere and fed forward to LIP? Could the monkeys still detect the direction of random dot motion if they would not have means of responding at the time of motion display (no knowledge which saccade to make)? If the monkeys could learn to use delayed response instructions (appearing after the motion display disappeared), what kind of activity would there be in LIP prior to the saccade (fast accumulation in every condition)? Does anybody know if this kind of studies have been made? And if monkeys are able to make 'abstract' choices about the direction of motion, which areas are involved in the computations? There could be a region coding for the choice irrespective of any spatial location, similarly to the OFC coding value of goods.”
    34. 34. LIP data: Evaluation? • Luke: “No disrespect to Michael Platt and colleagues, but I found something lacking in their argument and overall approach. For reasons that have never been clearly spelled out, he has spent over a decade studying value in regions in which there is no clear evidence (nor ever has been) supporting their role in its calculation. At the conclusion of the chapter he notes what has been painfully obvious to everyone else for years - lesioning the LIP does not result in an impaired ability to calculate value. However, it does impair attention and ability to initiate visual saccades (a behavior that is crucial for his tasks…”
    35. 35. “…Not determined to give up on studying regions that have no clear role in calculating value, he decides to focus on the posterior cingulate gyrus…” - Luke
    36. 36. Posterior Cingulate Cortex (CGp) • Anatomical and neurophysiological evidence suggest a role in “signaling motivationally significant events and actions, as well as perhaps their subjective value for guiding future behavior.”
    37. 37. CGp signals are subjective
    38. 38. CGp and evaluation of stimuli • CGp neurons respond most strongly to visual events that are unpredictable in space or time (Dean et al., 2004) • CGp neurons respond following delivery of unpredictable reward or omission of predictable reward (McCoy et al., 2003) • Attention for learning
    39. 39. CGp: Evaluation? • Luke: “Not determined to give up on studying regions that have no clear role in calculating value, he decides to focus on the posterior cingulate gyrus. However, rather than actually describing how it might be involved in calculating value, he gives us a list of random citations justifying how it might play a role (sounding strangely like a grant proposal). After reading a dense 20 page chapter on value, I was hoping to have at least learned something about how value might be calculated in the brain, but instead I felt like I was reading a 20 page justification of why we should pick random areas in the brain that are not well studied and then not be surprised when nothing conclusive emerges.
    40. 40. CGp: Evaluation? • Luke: “Why are goal values (i.e. motivations) always neglected in models of value? If things like appetitive desires (presumably processed in the hypothalamus) are crucial for calculating goal values, why in the world would we think the furthest region away from them (both spatially and evolutionarily) would be an interesting place to begin? How might internal motivational states (e.g. hunger, sexual drive, temperature regulation) be integrated with external demands (e.g. reputation)?”
    41. 41. Question • Assuming neuronal correlates of value exist in the OFC, LIP, and CGp, why are there so many representations of value in the brain? • MP & C P-S: value signals expressed in different areas contribute to different processes – Frontal areas: economic choice – Sensory areas: perceptual choice – Motor areas: motor choice
    42. 42. Question • Mirre: “Different neural correlates of value have been found in the brain. In sensory areas, value signals contribute to perceptual attention, in frontal areas, value signals contribute to economic choice and in motor areas value signals contribute to action selection. How are these signals integrated? Which value signal determines the eventual choice? What happens when there is a conflict between value signals?”
    43. 43. Goods- versus Actions-based • GBM: economic choice is modular and before action selection • ABM: economic choice “unfolds as a process of action selection” • For GBM, choice should be “completely processed within an abstract representation of goods” (e.g., in the OFC)
    44. 44. Other questions • Trevor: “…Both chapters this week (and last week) sort of handwaved at the notion that understanding the valuation process would be beneficial for clinical research, but never really justified this claim. How do you think understanding how valuation is accomplished in the brain can inform assessment and treatment of various clinical disorders?”
    45. 45. Other questions • Luke “Things that are more interesting to discuss then the scientific career of Michael Platt might include: Is there a unitary valuation system? I think this will prove to be a hotly contested issue. Are abstract representations somehow funneled into the OFC or ventral striatum and then converted into a unitary currency? Or are there independent systems involved in risk (insula, Nacc) reward (NAcc, amygdala, OFC), and punishment (insula, amygdala)? What about more abstract representations like future consequences? What about motivational values originating either exogenously or endogenously? Is there any reason to believe that the assumptions about choice held by economists have any validity in the brain? Is there actually a system involved in representing value and another one for selecting the action with the highest expected value?

    ×