Games in Humans and Non-human Primates:
         Scanners to Single Units
      The prospects for game theoretical approac...
From matrix to real world
   The gap between the experiments and
    real world in decision making studies

• Traditional ...
The combination of game theoretical tasks and
 modern neuroscientific methods is the solution.

• This chapter examines
  ...
Game theory attempts to mathematically capture behavior in
            strategic situations, in which an individual's succ...
Ultimatum Game
Player 1 can offer a fair (F) or unfair
(U) proposal; player 2 can accept (A)
or reject (R).




          ...
Trust Game




48 = 12 MU own endowment + 36 MU tripled transfer

A rational and selfish trustee will never honor the trus...
Prisoner’s dilemma game-
      simultaneous trust game
• The Nash Equilibrium is mutual defection.
• Players exhibit 50% m...
Public goods game: a generalized
          form of PDG
• The self-interested solution is to free-ride
  and hope that ever...
Mixed strategy game
 (e.g. matching pennies and the inspection game)

• A pure strategy provides a complete
  definition o...
It is unclear whether the decisions emerge
   from strategic or altruistic motivations.

• Ultimatum Game proposer behavio...
The benefits of combined approach between
experimental economics and neuroscience methods

• More precise characterization...
The animal model
          (Rhesus Macaque)
• To investigate “black box” during social
  interactions
• For studying highe...
Advantages and disadvantages of a systems
neurophysiology (invasive animal model) approach

• Advantage:
   – direct acces...
Saccadic ( jumping-fixating) eye
            movements
• A saccade is a fast movement of an eye,
  head or other part of a...
Benefits of visuosaccadic system
   investigation for decision making


• Efficiently extract visual information in
  perc...
Adapting games for non-human primates

LIP: estimating the desirability of sensory stimuli
SC: selecting/preparing actions...
Encoding the desirability of choice
          stimuli in LIP
• At the end of visual
  processing stream
• Encode the salie...
Visuosaccadic version of mixed-
   strategy inspection game
                                           Nash Eq.




      ...
LIP firing rates varies with desirability,
      not probability of outcome




                          Dorris and Glimc...
Correlation between LIP Firing Rates and an
Estimate of the Relative Desirability of Choices




                         ...
Evolving response selection in
   midbrain superior colliculus
• Intimately involved in saccade generation
• See Figure 6....
Evaluating the consequences of
     actions in frontal cortex
• dlPFC is involved in integrating choice
  (i.e. left vs ri...
General significance of decision making studies in
                non-human primates

• Recording single neurons allow fo...
Games in humans
• Imaging studies of social interaction
  – fMRI, PET

  – Stimuli of other human faces (static, 2D)
    (...
Automatic and intentional brain responses
     during evaluation of trustworthiness of faces

 Explicit judgments: whether...
Reading the mind in cartoons and stories




medial prefrontal cortex (paracingulate cortex)
                             ...
Problems in the previous studies
• Does the pattern of brain activation in
  response to the picture of a static, 2D face
...
Solutions to improve ecological validity of
     experiments in neuroeconomics

• To interact with other people in real
  ...
Another solution is to manipulate
      specific neurotransmitter systems
• Tryptophan depletion can be used to decrease b...
Trust Game behavior

                 Oxytocin increased trust.




                 Oxytocin: filled bars
               ...
Fool me once, shame on you.
Fool me twice, shame on oxytocin.




                         Baumgartner et al. Neuron 2008
What is the role of PFC in emotion vs.
     cognition conflict decision making?

Economic rationality:                  So...
Current research direction:
             social motivation reward
• Striatum: scale reward magnitude (Cromwell and Schultz...
Caution should be used when attempting to
  “reverse engineer” from patterns of brain
   activity to cognitive and social ...
Competition, Cooperation and Coordination



• Classical models of decision making,
  both utility theory and game theory,...
The emotional brain
  The Papez circuit theory (1937) of the functional neuroanatomy of emotion

                         ...
MacLean’s limbic system theory of the
       functional neuroanatomy of emotion
the hippocampus received sensory inputs
fr...
Map of brain areas commonly activated in
         social decision-making studies.




Theory of Mind: STS, MPFC, OFC




 ...
Neuroscientific studies of emotions
• Negative emotional states have been observed behaviorally
  as a result of both ineq...
Behavior of social interaction
                in the Ultimatum Game




Participants had a stronger emotional reaction to...
Neuroimaging results of the
              Ultimatum Game



                                                          Sanf...
dlPFC is linked to cognitive processes

• Frontal, “top-down” processes in reward
  studies suppress striatal activation.
...
Modulating affective system in
         the Ultimatum Game
• People in a sad mood (watching a 5-minute
  video rated as “s...
Significance of neuroscience
   approach to decision making
• Decision making appears to involve the
  interaction among m...
Upcoming SlideShare
Loading in …5
×

Games in humans and non-human primates - the prospects for game theoretical approach to neuroscience of decision making

6,266 views
6,011 views

Published on

prepared for neuroeconomics book study
chapter 6. by Sanfey and Dorris

Published in: Technology, Economy & Finance
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
6,266
On SlideShare
0
From Embeds
0
Number of Embeds
140
Actions
Shares
0
Downloads
53
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Games in humans and non-human primates - the prospects for game theoretical approach to neuroscience of decision making

  1. 1. Games in Humans and Non-human Primates: Scanners to Single Units The prospects for game theoretical approach to neuroscience of decision making Prepared for Neuroeconomics book study Chapter 6. by Sanfey and Dorris 2009.04.07, 07p.m. Kyongsik Yun, Ph.D. Candidate KAIST yunks@kaist.edu
  2. 2. From matrix to real world The gap between the experiments and real world in decision making studies • Traditional decision making studies were done in the confined experimental settings, isolating a specific aspect of complex decision making (e.g. risky, uncertain, ambiguous). • However, real life decisions are more complex and made in the context of a social interaction.
  3. 3. The combination of game theoretical tasks and modern neuroscientific methods is the solution. • This chapter examines – Game theory – Invasive electrophysiological techniques in monkeys during game-theoretic tasks – Non-invasive imaging techniques in humans during game-theoretic tasks
  4. 4. Game theory attempts to mathematically capture behavior in strategic situations, in which an individual's success in making choices depends on the choices of others. – von Neumann and Morgenstern (1947) Utility Theory Utility theory is unable to explain why people are often simultaneously attracted to both insurance and gambling. • Prospect theory – (Kahneman & Tversky 1979) Real world decision making is less selfish and more willing to consider reciprocity and equity (Camerer 2003).
  5. 5. Ultimatum Game Player 1 can offer a fair (F) or unfair (U) proposal; player 2 can accept (A) or reject (R). Guth et al. 1982 Nash Equilibrium is at odds: Low offers of less than 20% of the total amount are rejected about half of the time. Neuroscience has begun to offer clues as to the mechanisms underlying these decisions.
  6. 6. Trust Game 48 = 12 MU own endowment + 36 MU tripled transfer A rational and selfish trustee will never honor the trust given by the investor. In real situation, a majority of investors send some amount of their money to the trustee. Baumgartner et al. Neuron 2008
  7. 7. Prisoner’s dilemma game- simultaneous trust game • The Nash Equilibrium is mutual defection. • Players exhibit 50% mutual cooperation. Cooperate defect Cooperate defect
  8. 8. Public goods game: a generalized form of PDG • The self-interested solution is to free-ride and hope that everyone else contributes. • Players on average contribute about half of their endowment to the public good.
  9. 9. Mixed strategy game (e.g. matching pennies and the inspection game) • A pure strategy provides a complete definition of how a player will play a game. • A mixed strategy is an assignment of a probability to each pure strategy. • The Nash Equilibrium is to select randomly with equal probabilities. • Players attempt to infer the strategy of our opponent, using theory-of-mind processes.
  10. 10. It is unclear whether the decisions emerge from strategic or altruistic motivations. • Ultimatum Game proposer behavior – fairness? Fear to be rejected? • Examining these games in a neural context can begin to offer clues as to the motivations behind the decisions.
  11. 11. The benefits of combined approach between experimental economics and neuroscience methods • More precise characterizations of behavior (how decisions are actually made) • Neuroscience can provide important biological constraints on the processes involved.
  12. 12. The animal model (Rhesus Macaque) • To investigate “black box” during social interactions • For studying higher-order decision processes • General organization of their nervous system is similar to that of humans • Monkeys and humans display comparable strategies in mixed-strategy games
  13. 13. Advantages and disadvantages of a systems neurophysiology (invasive animal model) approach • Advantage: – direct access to the neural substrate – Exquisite temporal (<1ms) and spatial (ind. neurons) resolution – Artificial manipulation of neuronal activity can provide causal evidence, complementing the correlational evidence provided by neuronal recordings. • Disadvantages – Limitations exist in using non-human primates to infer the neural processes underlying human social interactions – Not suitable for more sophisticated games (UG, PDG) – No verbal instruction and using only operant conditioning techniques
  14. 14. Saccadic ( jumping-fixating) eye movements • A saccade is a fast movement of an eye, head or other part of an animal's body or device. It can also be a fast shift in frequency of an emitted signal or other quick change.
  15. 15. Benefits of visuosaccadic system investigation for decision making • Efficiently extract visual information in perceptual decision making • The neural circuitry underlying visual processing and saccadic control is well understood.
  16. 16. Adapting games for non-human primates LIP: estimating the desirability of sensory stimuli SC: selecting/preparing actions, (control eye movement) PFC: evaluating the consequences of actions LIP – gateway for the emergence of the brain Newsome and Shadlen 2001 Nature
  17. 17. Encoding the desirability of choice stimuli in LIP • At the end of visual processing stream • Encode the saliency of visual targets Platt and Glimcher 1999 Nature Sugrue, Corrado, Newsome 2005 NRN
  18. 18. Visuosaccadic version of mixed- strategy inspection game Nash Eq. 0.9 0.3 0.5 Inspection cost Dorris and Glimcher 2004 Neuron
  19. 19. LIP firing rates varies with desirability, not probability of outcome Dorris and Glimcher 2004 Neuron
  20. 20. Correlation between LIP Firing Rates and an Estimate of the Relative Desirability of Choices Dorris and Glimcher 2004 Neuron
  21. 21. Evolving response selection in midbrain superior colliculus • Intimately involved in saccade generation • See Figure 6.4 (Dorris unpublished) – B) Preferred (black)/ unpreferred (red) – C) Stimulus induced saccades deviate slightly towards the target. – D) angular deviation of saccades increases as the time of target presentation approaches. (일찍 자극, 적게 변화)
  22. 22. Evaluating the consequences of actions in frontal cortex • dlPFC is involved in integrating choice (i.e. left vs right) and reward (i.e. rewarded or unrewarded) information. • dACC encoded critical information about the temporal delay of previous rewards within a sequence of responses (Seo and Lee 2007)
  23. 23. General significance of decision making studies in non-human primates • Recording single neurons allow for moment-to-moment correlations between neuronal activity and behavioral responses • Providing unprecedented insight into the neuronal mechanisms underlying stochastic choice.
  24. 24. Games in humans • Imaging studies of social interaction – fMRI, PET – Stimuli of other human faces (static, 2D) (Winston et al. 2002) – Mentalizing (Gallagher et al. 2000) – Moral reasoning (Greene et al. 2001) Pioneering works of its kind
  25. 25. Automatic and intentional brain responses during evaluation of trustworthiness of faces Explicit judgments: whether an individual was trustworthy Implicit judgments: an unrelated age assessment Insula STS l. Amyg r. Amyg. The findings extend a proposed model of social cognition by highlighting a functional dissociation between automatic engagement of amygdala versus intentional engagement of STS in social judgment. Winston, Strange, O’Doherty, & Dolan NN 2002
  26. 26. Reading the mind in cartoons and stories medial prefrontal cortex (paracingulate cortex) Gallagher et al. 2000
  27. 27. Problems in the previous studies • Does the pattern of brain activation in response to the picture of a static, 2D face accurately reflect the brain’s response to the dynamic, embodied faces that we encounter in everyday life? • Is the pattern of brain activation in response to reasoning regarding hypothetical, fictitious scenarios the same as when grappling with significant real life social problems?
  28. 28. Solutions to improve ecological validity of experiments in neuroeconomics • To interact with other people in real social exchanges from outside the scanner (McCabe et al. 2001) • Hyperscanning technology (Montague et al. 2002) – Data collection efficiency – Open new vistas in social cognitive neuroscience
  29. 29. Another solution is to manipulate specific neurotransmitter systems • Tryptophan depletion can be used to decrease brain serotonin levels, reducing cooperative behavior (Wood et al. 2006) • Oxytocin elevation increases trust (Kosfeld et al. 2005) • TMS to temporarily activate or deactivate a brain region and then examine its effects on decision making (van’t Wout et al. 2005) • Lesion study (Koenigs and Tranel 2007)
  30. 30. Trust Game behavior Oxytocin increased trust. Oxytocin: filled bars Placebo: open bars Baumgartner et al. Neuron 2008
  31. 31. Fool me once, shame on you. Fool me twice, shame on oxytocin. Baumgartner et al. Neuron 2008
  32. 32. What is the role of PFC in emotion vs. cognition conflict decision making? Economic rationality: Social rationality: • economic value maximization Social value maximization • Fairness monitoring • • Absolute valuation Relative valuation • • vmPFC lesion -> R↑ dlPFC disruption -> A↑ • Knoch et al. Science 2006 Koenigs & Tranel, J Neurosci. 2007
  33. 33. Current research direction: social motivation reward • Striatum: scale reward magnitude (Cromwell and Schultz 2003), specifically the magnitude of monetary reward or punishment (O’Doherty 2004, Knutson and Cooper 2005) • Striatum may register social prediction errors to guide decisions about reciprocity (Rilling et al. 2002) • Caudate: related to how much reciprocity the investor had shown on previous trials, “intention to trust” (King-Casas et al. 2005) • caudate: altruistically punish (de Quervain et al. 2004) • Social altruism: the striatum was engaged both by receiving money and by donations to charity (Moll et al. 2006), and the activation was enhanced when the donation was voluntary as opposed to forced (Harbauch et al. 2007). -> important implications for informing public policy
  34. 34. Caution should be used when attempting to “reverse engineer” from patterns of brain activity to cognitive and social processes • Correlation analysis, TMS modulation, or patient work is crucial to buttress the causal relationship between behavior and brain activity. • These results appear to demonstrate that complex social processes recruit more basic mechanisms within the human brain, providing support for the notion that the brain uses a common reward metric. This also furthers the connection between the disparate branches of neuroeconomics from primary and secondary rewards (food, money) to more abstract social rewards (reciprocity, fairness).
  35. 35. Competition, Cooperation and Coordination • Classical models of decision making, both utility theory and game theory, have largely ignored the influence of emotions on how decisions are made, but recent research has begun to demonstrate their powerful effect.
  36. 36. The emotional brain The Papez circuit theory (1937) of the functional neuroanatomy of emotion Emotional experiences or feelings occur when the cingulate cortex integrates these signals from the hypothalamus with information from the sensory cortex. stream of thinking stream of feeling Output from the cingulate cortex to the hippocampus (3) and then to the hypothalamus (4) allows top–down cortical control of emotional responses. Dalgleish 2004 NRN
  37. 37. MacLean’s limbic system theory of the functional neuroanatomy of emotion the hippocampus received sensory inputs from the outside world as well as information from the internal bodily environment (viscera and body wall). Emotional experience was a function of integrating these internal and external information streams. Dalgleish 2004 NRN
  38. 38. Map of brain areas commonly activated in social decision-making studies. Theory of Mind: STS, MPFC, OFC Sanfey 2007 Science
  39. 39. Neuroscientific studies of emotions • Negative emotional states have been observed behaviorally as a result of both inequity and non-reciprocity, such as unfair offers in a UG. • These emotional reactions have evolved precisely to foster mutual reciprocity, to make reputation important, and to punish those seeking to take advantage of others (Nawak et al. 2000) • Neuroscienctific studies of this nature offer the potential to go beyond speculation and to examine the causal relationship between an emotional reaction and subsequent social decision, as well as investigating whether areas specialized for the processing of basic emotions may be coopted for more complex affective reactions.
  40. 40. Behavior of social interaction in the Ultimatum Game Participants had a stronger emotional reaction to unfair offers from humans than to those from computers. 40 Sanfey et al. Science, 2003
  41. 41. Neuroimaging results of the Ultimatum Game Sanfey et al. Science, 2003 Anterior insula: pain, distress, hunger, thirst (Denton et al. 1999), autonomic arousal (Critchley et al. 2000) Right anterior insula: aversive conditioning (Seymour et al. 2005) Defection rate in PDG is correlated with right anterior insula activation (Rilling et al. 2008) Anterior insula may play a role in marking a social interaction as aversive, and thus discouraging trust.
  42. 42. dlPFC is linked to cognitive processes • Frontal, “top-down” processes in reward studies suppress striatal activation. • TMS increased acceptance rate of unfair offers, providing strong evidence for a causal relationship between activation in this area and social decision making (Knoch et al. 2006) – Why social? Disruption of dlPFC using TMS might reduce cognitive processes to accept the offer, decreasing acceptance rates.
  43. 43. Modulating affective system in the Ultimatum Game • People in a sad mood (watching a 5-minute video rated as “sad”) reject more 'unfair' offers (Harle and Sanfey 2007). • Examining decision making performance in participants with disregulated emotional processing, such as patients with depression or schizophrenia, may be a useful future avenue of research. • Contradictory study: vmPFC lesion patients also reject unfair offers more frequently than do controls (Koenigs and Tranel 2007)
  44. 44. Significance of neuroscience approach to decision making • Decision making appears to involve the interaction among multiple subsystems governed by different parameters and possibly even different principles. • Neuroscience approach to decision making has the potential to inform economic theories of interactive decision making (inequity aversion, social utility functions) • Economic models based on the underlying neural patterns may provide useful constraint.

×