Oct. 21, 2013•0 likes•1,179 views

Download to read offline

Report

Technology

Economy & Finance

Michael BlumFollow

- 1. The social Bayesian brain Brain and Spine Institute - Paris “Motivation, Brain & Behaviour” group Jean Daunizeau
- 4. The Bayesian brain hypothesis Cerebral information processing was optimized through natural selection [Friston 2005, Fiorillo 2010] The brain uses a model of the world that (i) is optimal on average, but (ii) can induce systematic biases [Weiss 2002, Alais 2004]
- 5. the social Bayesian brain Theory of mind = ability to attribute mental states (e.g., beliefs, desires, ...) to others [Premack 1978] • teaching, persuading, deceiving, … → success in social interactions [Baron-Cohen 1999] • develops very early in life [Surian 2007, Kovacs 2010] • impairment → severe psychiatric disorders [Baron-Cohen 1985, Frith 1994, Brüne 2005] • meta-cognitive insight: others’ behaviour is driven by their beliefs [Frith 2012] The social “Bayesian brain” hypothesis • the brain’s model of other brains should assume they are Bayesian too! → ToM = meta-Bayesian inference [Daunizeau 2010a, Daunizeau 2010b] • although it is limited, is ToM optimal in an evolutionary sense?
- 6. Overview of the talk Inverse Bayesian Decision Theory Meta-Bayesian modelling of Theory of Mind Limited ToM sophistication: did evolution fool us?
- 7. Overview of the talk Inverse Bayesian Decision Theory Meta-Bayesian modelling of Theory of Mind Limited ToM sophistication: did evolution fool us?
- 8. BDT: from observations to beliefs • The “amount of information” is related to probability in the sense that one’s belief “vagueness” is well characterized in terms of the dispersion of a probability distribution that measures how (subjectively) plausible is any “state of affairs”. • The subjective plausibility of any “state of affairs” is not captured by the objective frequency of the ensuing observable event. This is because beliefs are shaped by all sorts of (implicit or explicit) prior assumptions that act as a (potentially very complex) filter upon sensory observations. x hidden (unknown) state of the world u accessible observations or data • likelihood: p u x, m • priors: p x m • posterior: p x u, m p u x, m p x m
- 9. BDT: from beliefs to decisions • Bayesian Decision Theory (BDT) is concerned with how decisions are made or should be made, in ambiguous or uncertain situations: - normative: optimal/rational decision? (cf. statistical test) - descriptive: what do people do? (cf. behavioural economics) • BDT is bound to a perspective on preferences → utility theory: - utility functions: surrogate for the task goal (reward contingent on a decision) - subsumes game theory and control theory a x, a alternative actions or decisions loss function → expected cost (posterior risk): u, a E x, a u, m → BDT-optimal decision rule: a* arg min u, a a
- 10. BDT example: speed-accuracy trade-off • loss = estimation error + estimation time: • generative model: 2 x, a, t x a t p ut x, m N x,1 ut x t( y ) ( ) p x m N 0 , 0 2 x 0 t t 2 t 0 t u 0 1 : 1 t 0 2 t • posterior belief: p x u1:t , m N t , t 2 • expected cost: u, a, t t a t 2 t 2
- 11. BDT example: speed-accuracy trade-off (2) 12 posterior variance time cost (K=1) expected loss (K=1) time cost (K=2) expected loss (K=2) 10 8 6 4 expected inaccuracy (K=2) expected inaccuracy (K=1) 2 0 0 0.5 1 t* t* (K=2) (K=1) 1.5 2 decision time
- 12. inverse BDT: the complete class theorem There (almost) always exist a duplet of prior belief p x m and loss function such that any observed decision can be interpreted as BDT-optimal t* arg min u, a, t t 1 02 0 → interpreting BDT-optimal responses (e.g., decision times): weak duality x, a
- 13. xt 1 p xt | xt 1 , , m(1) xt Perceptual priors meta-Bayesian model Perceptual model m(1) ut p ut | xt , , m(1) Perceptual likelihood m(2) u Free-energy maximization (optimal learner assumption) t f t 1 , ut , q xt | t f : t 1 arg max F t u, approximate posterior Posterior risk minimization (optimal decider assumption) g arg min u, a a Response model m (2) y p , | m(2) y p y | , ( ), m(2) priors likelihood Daunizeau et al., 2010a
- 14. inverse BDT: the meta-Bayesian approach Andy Murray’s belief -3 4 meta-Bayesian model m x 10 3 (2) 2 t 1 u ball position 0 t , t 0 -10 -5 0 5 10 t prior uncertainty meta-Bayesian estimate of Andy Murray’s belief Andy Murray’s belief -3 4 cost of time x 10 3 2 y reaction time 1 0 -10 -5 0 5 10
- 15. Overview of the talk Inverse Bayesian Decision Theory Meta-Bayesian modelling of Theory of Mind Limited ToM sophistication: did evolution fool us?
- 16. Recursivity and limited sophistication • Operational definition of ToM: taking the intentional stance [Dennett 1996] – Infer beliefs and preferences from observed behavior – Use them to understand/predict behaviour • In reciprocal/repeated social interactions, ToM is potentially recursive [Yoshida 2008] – « I think that you think », « I think that you think that I think », … – ToM sophistication levels induce different learning rules / behavioural policies • Questions: – Does the meta-Bayesian approach reallistically capture peoples’ ToM? – Can people appeal to these sophistication levels (e.g. by pure reinforcement) or are these (social) priors that are set by the context? – What is the inter-individual variability of ToM sophistication?
- 17. 0-ToM 0-ToM does not apply the intentional stance → 0-ToM is a Bayesian agent with: - beliefs (about non-intentional contingencies) - preferences « I think that you will hide behind the tree »
- 18. 1-ToM 1-ToM learns how the other learns → 1-ToM is a meta-Bayesian agent with: - beliefs (about other’s beliefs and preferences) - preferences « I think that you think that I will hide behind the tree »
- 19. 2-ToM 2-ToM learns how the other learns and her ToM sophistication level → 2-ToM is a meta-Bayesian agent with: - beliefs (about other’s beliefs – about one’s beliefs - and preferences) - preferences « I think that you think that I think, … »
- 20. Recursive meta-Bayesian modelling • k-ToM learns how the other learns and her ToM sophistication level: k ( k ) f (1) , a ,1( k ) • k-ToM acts according to her beliefs and preferences: k) p a1, 1 ( k ) exp (1 , a1, 1 2( k ) • This induces a likelihood for a k+1-ToM observer: p a1, (1,..., k ) k , , mk 1 p a1, ' k ' 0 '1 (k ) k ' • Deriving the ensuing Free-Energy yields the k+1-ToM learning rule: k (1 1) f ( k 1) , a ,1( k 1) f : ( k 1) arg max F( k 1) ( k 1) 1 F( k 1) ln p a1, (1,...,k ) , , mk 1 ln p (1,...,k ) , mk 1 ln q (1,...,k ) ,
- 21. Performance in competitive games outcome table (« hide and seek ») hider: a1 = 1 hider: a1 = 0 seeker: a2 = 1 -1, 1 1, -1 seeker: a2 = 0 1, -1 -1, 1 0-ToM1 0.02 2-ToM3 0 3-ToM4 -0.02 4-ToM5 -0.04 4 5 4-ToM 3 3-ToM 2 2-ToM 1 1-ToM 512 1-ToM2 0-ToM simulated behavioural performance (#wins/trial) 0.04
- 22. Everybody is somebody’s fool 1-ToM predicts 0-ToM 0-ToM predicts 1-ToM
- 23. Behavioural task design social framing (game « hide and seek ») non-social framing (casino gambling task) You are playing against Player 1 Session 1 alternative options (1.2 sec) 1 2 1 2 subject’s choice feedback (1sec) Well done! You win!
- 24. the social framing effect: group results (N=26) group-average performance (cumulated earnings after 60 game repetitions) 6 * 4 2 0 -2 -4 2 -6 random biased 1.8 * 0-ToM 1.6 1.4 2.1 1-ToM 2 2-ToM 1.9 1.8 1.7 1.6 non-social framing social framing 1.5 inter-individual variability in cognitive skills: regression on performance against 1-ToM 6 * 4 2 0 -2 -4 -6 IMT false belief Frith-Happe EQ WSCT Go-NoGo 3-back * mean
- 25. Volterra decompositions: group results k p at 1 s 0 ( k )ut() ... k Volterra 1st-order kernels: own action opponent's action Volterra weight: S-NS Volterra weight 0.1 0 -0.1 -0.2 -0.3 -0.4 * own action -0.5 1 2 0.75 4 5 lag lag 3 6 7 lag 4 -0.4 * -0.5 1 6 3 -0.3 8 0.6 2 -0.2 2 3 4 5 6 7 8 5 6 7 8 chance level 0-ToM (acc=86%) 1-ToM (acc=75%) 2-ToM (acc=74%) 0.65 1 -0.1 lag 0.7 0.1 0 -0.1 -0.2 -0.3 -0.4 -0.5 0 lag weight Volterra weight: S-NS Volterra weight 0.1 5 6 7 8 0.1 0 -0.1 -0.2 -0.3 -0.4 -0.5 1 2 3 4
- 26. Group-level Bayesian model comparison (I) log model evidences (group average) free energies 4 2 0 -2 2 1.8 1.6 1.4 -4 2.1 2 1.9 1.8 1.7 1.6 non-social framing social framing 1.5 -6 Volterra Nash WSLS RL 2-ToM’s best fit: subject #7 against 0-ToM (acc=79%) Model fit: <g(x)|y,m> versus y 0-ToM 1-ToM 3-ToM 2-ToM’s worst fit: subject #21 against 0-ToM (acc=43%) Model fit: <g(x)|y,m> versus y 1.2 1 1 y observed choices 1.2 observed choices y 2-ToM 0.8 0.6 0.4 0.8 0.6 0.4 0.2 0.2 0 0 -0.2 0 0.2 0.4 0.6 <g(x)|y,m> modelled choices 0.8 1 -0.2 0 0.2 0.4 0.6 <g(x)|y,m> modelled choices 0.8 1
- 27. Group-level Bayesian model comparison (II) modelmodel families: exceedance probabilities families: exceedance probabilities 1 0.8 2 1.8 1.6 1.4 0.6 2.1 2 1.9 1.8 1.7 1.6 non-social framing social framing 1.5 0.4 0.2 0 no-ToM ToM estimated model frequencies (social condition) model frequencies (social condition) 0.6 0.5 0.4 0.3 0.2 0.1 0 Volterra Nash WSLS no-ToM family RL 0-ToM 1-ToM 2-ToM ToM family 3-ToM
- 28. What about Rhesus-macaques? [Thom et al., 2013]
- 29. Rhesus-macaques: group-results (N=4) 15 10 5 0 -5 -10 random biased own action opponent's action 0.6 0.4 0.4 Volterra weight 0.6 Volterra weight 1-ToM 0.2 0 -0.2 -0.4 0.2 0 -0.2 1 2 3 4 5 lag 6 7 8 -0.4 1 2 3 4 5 lag 6 7 8
- 30. Rhesus-macaques: group-results (N=4) log model evidences (group average) free energies 20 10 0 -10 -20 Volterra Nash WSLS RL 0-ToM no-ToM family 1-ToM ToM family model families: exceedance probabilities 1 0.8 0.6 0.4 0.2 0 no-ToM 2-ToM ToM
- 31. Overview of the talk Inverse Bayesian Decision Theory Meta-Bayesian modelling of Theory of Mind Limited ToM sophistication: did evolution fool us?
- 32. Competitive versus cooperative games « hide and seek » P1: a1 = 1 P1: a1 = 0 P2: a2 = 1 -1, 1 1, -1 P2: a2 = 0 1, -1 « battle of the sexes » P1: a1 = 1 -1, 1 P2: a2 = 1 0-ToM1 2, 0 -1, -1 P2: a2 = 0 0.04 P1: a1 = 0 -1, -1 0, 2 0.8 0-ToM 1 0.6 0.02 1-ToM2 1-ToM 2 0.4 2-ToM3 0 2-ToM 3 3-ToM 4 4-ToM 5 1 2 3 4 5 4-ToM 5 4-ToM 4 3-ToM 3 2-ToM 2 1-ToM 0-ToM 1 -0.2 3-ToM -0.04 2-ToM 4-ToM5 0 1-ToM -0.02 0-ToM 3-ToM4 0.2
- 33. Being right is as good as being smart 1-ToM predicts 0-ToM « hide and seek » « battle of the sexes » 0-ToM predicts 1-ToM
- 34. Biases in ToM induction 2-ToM vs 2-ToM 3-ToM vs 3-ToM 4-ToM vs 4-ToM
- 35. Evolutionary game theory Can we explain the emergence of the natural bound on ToM sophistication? → Average adaptive fitness: • is a function of the behavioural performance, relative to other phenotypes • depends upon the frequency of other phenotypes within the population sk frequency of phenotype k within the population i frequency of game i Q(i ) expected payoff matrix of game i at round τ → Replicator dynamics [Maynard-Smith 1982, Hofbauer 1998]: ds Diag s i Q(i ) s i sT Q (i ) s dt i i evolutionary stable states: s lim s t t
- 36. Replicator dynamics and ESS EGT replicator dynamics 1 type and frequency of EGT steady states « hide and seek » traits' frequencies 0.8 0.6 0.4 0.2 0 0 200 400 600 800 k=0 k=1 k=2 k=3 k=4 1000 evolutionary time EGT replicator dynamics 1 « battle of the sexes » traits' frequencies 0.8 EGT steady state k=0 k=1 k=2 k=3 k=4 0.6 0.4 0.2 0 0 200 400 600 evolutionary time 800 1000
- 37. ESS: phase portrait k=0 lambda (frequency of cooperative game) 1 k=1 0.9 k=2 0.8 k=3 0.7 k=4 0.6 0.5 0.4 0.3 0.2 0.1 0 1 2 4 8 16 32 64 128 256 512 tau (game duration)
- 38. Can non-ToM agents invade? • RL = reinforcement learning agent (cannot adapt to game rules) • Nash = « rational » agent (cannot adapt to opponent) « hide and seek » « battle of the sexes » Nash Nash RL Nash 4-ToM RL 3-ToM RL 2-ToM 4-ToM 1-ToM 4-ToM 0-ToM 3-ToM Nash 3-ToM RL 2-ToM 4-ToM 2-ToM 3-ToM 1-ToM 2-ToM 1-ToM 1-ToM 0-ToM 0-ToM 0-ToM
- 39. 4-ToM What do ToM agents think of RL and Nash? 2-ToM 500 3-ToM 500 2-ToM 4-ToM 500 3-ToM 500 400 500 400 400 300 400 300 300 200 300 200 300 200 200 100 200 100 200 100 100 0 100 0 100 0 4-ToM 500 400 400 300 RL Nash 0 0-ToM 1-ToM 0-ToM 1-ToM 0 0-ToM 0-ToM 2-ToM 500 1-ToM 1-ToM 2-ToM 2-ToM 0 3-ToM 500 2-ToM 0-ToM 1-ToM 2-ToM 3-ToM 0-ToM 1-ToM 2-ToM 3-ToM 4-ToM 500 3-ToM 500 400 500 400 500 400 400 300 400 300 400 300 300 200 300 200 300 200 200 100 200 100 200 100 100 0 100 0 100 0 4-ToM 0 0-ToM 1-ToM 0-ToM 1-ToM 0 0-ToM 1-ToM 2-ToM 0-ToM 1-ToM 2-ToM 0 0-ToM 1-ToM 2-ToM 3-ToM 0-ToM 1-ToM 2-ToM 3-ToM
- 40. ESS: phase portrait (2) k=0 lambda (frequency of cooperative game) 1 k=1 0.9 k=2 0.8 k=3 0.7 k=4 0.6 RL 0.5 Nash 0.4 0.3 0.2 0.1 0 1 2 4 8 16 32 64 128 256 512 tau (game duration)
- 41. The social Bayesian brain: summary • Meta-Bayesian inference - the brain’s model of other brains assumes they are Bayesian too! - there is an inevitable inflation of the uncertainty of others’ beliefs • Theory of mind: - reciprocal social interaction → recursive beliefs - Humans → social framing effect (mentalize or be fooled) - Macaque monkeys → no intentional stance (but training?) • Evolution of ToM: - cooperation+learning during evolution → natural bounds to ToM sophistication (“being right is as good as being smart”) - evolutionary stable ToM distribution = mixed!
- 42. Dealing with uncertain motives: advice taking task probabilistic cue player’s decision outcome informed advice ? or * P progress bar A Gold target = 20 CHF Silver target = 10 CHF P A Diaconescu et al., in prep.
- 43. Dealing with uncertain motives: results (N=16) 2-ToM: worst subject Model fit: <g(x)|y,m> versus y 1.2 1 2-ToM: <g(x)|y,m> versus y Model fit: best subject 1.2 R2=45.2% 1 0.6 0.6 y 0.8 y 0.8 R2=73.0% 0.4 0.4 0.2 0.2 0 0 -0.2 0 0 0.2 0.2 0.4 0.6 <g(x)|y,m> 0.4 0.6 0.8 0.8 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 0.2 0.4 0.6 0.8 <g(x)|y,m> 1 model attributions exceedance probabilities 1 0.8 5.0 subjects -0.2 0 1 1 0.6 0.4 0.2 2-ToM 0 0 1-ToM 1 2 3 4 5 6 7 8 9 01 11 HGF 0 HGF 1-ToM 2-ToM 1
- 44. Bayes Laplace Helmoltz Jaynes Friston Marie Devaine I also would like to thank Dr. S. Bouret & Dr. A. San-Gali (monkey experiments @MBB)