Beyond Nash Equilibrium Correlated Equilibrium and Evolutionary Equilibrium Jie Bao 2003-05-05 Iowa State University
Equilibrium in Games <ul><li>Pure strategy Nash equilibrium  </li></ul><ul><li>Mixed strategy Nash equilibrium  </li></ul>...
Correlated Equilibrium <ul><li>Aumann 1974 </li></ul><ul><li>A generalization of “rational” solution </li></ul><ul><li>In ...
Example : BoS <ul><li>Bach or Stravinsky Game (or Battle of Sex)  </li></ul><ul><li>NE: strategy profile -> payoff profile...
BoS: Mixed Strategy   (b)  Agent 2’s expected utility, blue line is it’s best response to given p (agent 1’s strategy) (d)...
BoS: CE - Toss Coin Equ. <ul><li><N,(A i ),(u i )>= <{1,2} ,{Bach , Stravinsky}, payoff matrix U> </li></ul><ul><li>Probab...
CE: formal definition <ul><li>A strategic game <N,(A i ),(u i )> </li></ul><ul><li>A finite probability space ( Ω,π )  </l...
CE contains Mixed NE <ul><li>For every mixed strategy Nash equilibriumαof a finite game <N,(A i ),(u i )>, there is a corr...
Convex Combination of CE->CE <ul><li>Let G=<N,(A i ),(u i )> be a strategic game. Any convex combination of correlated equ...
Example <ul><li>Pure NE payoff profile: (7,2) (2,7)  </li></ul><ul><li>Mixed NE payoff profile: (4 2 / 3  ,4 2 / 3 ) </li>...
States and outcomes in CE <ul><li>Let G=<N,(A i ),(u i )> be a strategic game. Every probability distribution over outcome...
Notes about CE <ul><li>If players hold different beliefs, additional equilibrium payoff profiles are possible. </li></ul><...
Evolutionary Equilibrium <ul><li>ESS: Maynard Smith & Price, 1972 </li></ul><ul><li>A steady state in which all organism t...
ESS: Definition: <ul><li>Let G=<{1,2},(B,B),(u i )>be a symmetric strategic game, where u 1 (a,b)=u 2 (b,a)=u(a,b) for som...
Example: Hawk--Dove <ul><li>Choose to be Hawk or dove? </li></ul><ul><li>Pure NE: (D,H) & (H,D) </li></ul><ul><li>Mixed NE...
HD Game – mixed strategy (a)  Agent 1’s expected utility, red line is it’s best response to given q (agent 2’s strategy) (...
HD Game – ESS invading <ul><li>In all those games, ESS (half-dove-half-hawk mixed strategy (0.5,0.5)) starts from a percen...
HD Game – ESS being invaded <ul><li>In all those games, ESS (half-dove-half-hawk mixed strategy (0.5,0.5)) starts from a p...
Not all NE are ESS <ul><li>A strict equilibrium b* is an ESS </li></ul><ul><ul><li>(b*,b*) if a symmetric NE  </li></ul></...
More about ESS <ul><li>Widely used in sociobiology </li></ul><ul><ul><li>See  Dawkins <selfish gene>, chapter 6 </li></ul>...
Upcoming SlideShare
Loading in …5
×

Beyond Nash Equilibrium - Correlated Equilibrium and Evolutionary Equilibrium

1,868 views

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,868
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
29
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Beyond Nash Equilibrium - Correlated Equilibrium and Evolutionary Equilibrium

  1. 1. Beyond Nash Equilibrium Correlated Equilibrium and Evolutionary Equilibrium Jie Bao 2003-05-05 Iowa State University
  2. 2. Equilibrium in Games <ul><li>Pure strategy Nash equilibrium </li></ul><ul><li>Mixed strategy Nash equilibrium </li></ul><ul><li>Correlated equilibrium </li></ul><ul><li>Evolutionary equilibrium </li></ul><ul><li>Bayesian Nash equilibrium </li></ul><ul><li>… </li></ul><ul><li>NE is too strict on what is “ration”… </li></ul>
  3. 3. Correlated Equilibrium <ul><li>Aumann 1974 </li></ul><ul><li>A generalization of “rational” solution </li></ul><ul><li>In a CE </li></ul><ul><ul><li>the action played by any player is a best response (in the expected payoff sense) to the conditional distribution over the other players given that action, </li></ul></ul><ul><ul><li>and thus no player has a unilateral incentive to deviate from playing their role in the CE. </li></ul></ul><ul><li>Example: Traffic Signal- a single bit of shares information allows a fair split of waiting times. </li></ul><ul><ul><li>“ running a light” can’t bring greater expected payoff </li></ul></ul><ul><ul><li>The actions of players are “correlated” </li></ul></ul>
  4. 4. Example : BoS <ul><li>Bach or Stravinsky Game (or Battle of Sex) </li></ul><ul><li>NE: strategy profile -> payoff profile </li></ul><ul><ul><li>(Bach, Stravinsky) ->(2 ,1) </li></ul></ul><ul><ul><li>(Stravinsky, Bach) ->(1 ,2) </li></ul></ul><ul><ul><li>(1/3 Bach, 1/3 Bach) -> (2/3, 2/3) </li></ul></ul><ul><li>Another equilibrium: the player observe the outcome of a public coin toss, which determines which of the two pure strategy Nash equilibria they play.->(3/2,3/2) </li></ul>? 1,2 1,0 Stravinsky 0,1 2,1 Bach Stravinsky Bach
  5. 5. BoS: Mixed Strategy   (b) Agent 2’s expected utility, blue line is it’s best response to given p (agent 1’s strategy) (d) Look from the top, (1/3,1/3) is the only mixed NE. pure NEs include (0,1) and (1,0) (a) Agent 1’s expected utility, red line is it’s best response to given q (agent 2’s strategy) (c) Overlap of a and b
  6. 6. BoS: CE - Toss Coin Equ. <ul><li><N,(A i ),(u i )>= <{1,2} ,{Bach , Stravinsky}, payoff matrix U> </li></ul><ul><li>Probability space ( Ω,π ) </li></ul><ul><ul><li>Ω = {x,y} </li></ul></ul><ul><ul><li>π(x) = π(y) = ½ </li></ul></ul><ul><li>information partition of each agent </li></ul><ul><ul><li>P1 = P2 = {{x},{y}} </li></ul></ul><ul><li>For each player {1,2} action function σ i : Ω->A i , </li></ul><ul><ul><li>σ 1 (x) = σ 2 (x) = Bach </li></ul></ul><ul><ul><li>σ 1 (y) = σ 2 (y) = Stravinsky </li></ul></ul><ul><li>Payoff profile: ½(B,B), ½(S,S) -> (3/2, 3/2) </li></ul><ul><ul><li>Compared with (for agent 1): </li></ul></ul><ul><ul><li>½(S,B), ½(S,S) ->1/2 </li></ul></ul><ul><ul><li>½(B,B), ½(B,S) ->1 </li></ul></ul><ul><ul><li>½(S,B), ½(B,S) ->0 </li></ul></ul>
  7. 7. CE: formal definition <ul><li>A strategic game <N,(A i ),(u i )> </li></ul><ul><li>A finite probability space ( Ω,π ) </li></ul><ul><ul><li>Ωis a set of states and π is a probability measure on π </li></ul></ul><ul><li>For each player i∈N, a partition P i of Ω(player i’s information partition) </li></ul><ul><li>For each player i∈N, a function σ i : Ω->A i , with σ i (w)= σ i (w’) whenever w∈ P i , and w’∈ P i for some P∈ P i , (σ i is player i’s strategy) </li></ul><ul><ul><li>such that for every I∈N and every function τ i : Ω->A i for which τ i (w)=τ i (w’) whenever w∈ P i , and w’∈ P i for some P∈ P i (i.e. for every strategy of player i) we have </li></ul></ul><ul><li>Note that we assume the players share a common belief about the probabilities with which the states occur. </li></ul>
  8. 8. CE contains Mixed NE <ul><li>For every mixed strategy Nash equilibriumαof a finite game <N,(A i ),(u i )>, there is a correlated equilibrium <( Ω,π ), P i ,σ i > in which for each player i∈N, the distribution on A i induced by σ i is α i . </li></ul><ul><li>Construct CE from Mixed NE </li></ul><ul><ul><li>Ω= A=(X j ∈N A j ) – strategy profiles </li></ul></ul><ul><ul><li>π(α)=Π j ∈N α j (a j ) – prob. Of the strategy profiles </li></ul></ul><ul><ul><li>P i (b i )={a ∈A: a i = b i }, P i consist of the |A i | sets P i (b i ) </li></ul></ul><ul><ul><li>σ i (a)=a i </li></ul></ul><ul><li>CE is a more general concept than Mixed NE and Pure NE </li></ul>NE CE Mixed NE
  9. 9. Convex Combination of CE->CE <ul><li>Let G=<N,(A i ),(u i )> be a strategic game. Any convex combination of correlated equilibrium payoff of G is a correlated equilibrium payoff of G </li></ul><ul><li>Interpret: first a public random device determines which of the K correlated equilibria is to played, and then the random variable corresponding to the k th correlated equilibrium is realized. </li></ul><ul><li>CE: <(Ω k ,π k ), P i k ,σ i k > </li></ul><ul><li>CE payoff profiles: U 1 ,.. U k </li></ul><ul><li>c 1 ,…c k , all c i >=0, and Σ c i =1 </li></ul><ul><li>Construct a new CE </li></ul><ul><ul><li>Ω = union of all Ω k </li></ul></ul><ul><ul><li>π(w) = c k π k (w) , if w in Ω k </li></ul></ul><ul><ul><li>P i =union (on k) of P i k </li></ul></ul><ul><ul><li>σ i (w) = σ i k (w) </li></ul></ul><ul><li>Payoff profiles Σ c k U k </li></ul>
  10. 10. Example <ul><li>Pure NE payoff profile: (7,2) (2,7) </li></ul><ul><li>Mixed NE payoff profile: (4 2 / 3 ,4 2 / 3 ) </li></ul><ul><li>CE: Ω={x,y,z}, π(x)= π(y)= π(z)=1/3, P1={{x},{y,z}}, P2={{x,y},{z}}, σ 1 (x)=B, σ 1 (y)= σ 1 (z)=T, σ 2 (x)= σ 2 (y)=L, σ 2 (z)=R, -> (5,5) </li></ul><ul><li>The CE is outside the convex hull of Pure / Mixed NE payoff profiles </li></ul>0,0 7,2 B 2,7 6,6 T R L 1 2 - x B z y T R L 1 2
  11. 11. States and outcomes in CE <ul><li>Let G=<N,(A i ),(u i )> be a strategic game. Every probability distribution over outcomes that can be obtained in a CE of G can be obtained in a CE’ in which </li></ul><ul><ul><li>the set of state Ω’ is A and </li></ul></ul><ul><ul><li>for each i∈N, player i’s information partition P i ’ consists of all sets of the form {a∈A i : a i =b i } for some action b i ∈A i </li></ul></ul><ul><ul><li>[π’(a) = π k ({w ∈ Ω: σ(w)=a}) σ i ’ (a) = σ i k (a i )] </li></ul></ul><ul><li>This theorem allows us to confine attention to equilibria in which the set states is the set of outcomes </li></ul>
  12. 12. Notes about CE <ul><li>If players hold different beliefs, additional equilibrium payoff profiles are possible. </li></ul><ul><li>Nash equilibrium is a special case of CE in which we demand that πbe a product distribution for some distribution π i , so every player acts independently of all others </li></ul><ul><li>It’s possible to compute CE via linear programming in polynomial time, while NE is exponentially complex! </li></ul>
  13. 13. Evolutionary Equilibrium <ul><li>ESS: Maynard Smith & Price, 1972 </li></ul><ul><li>A steady state in which all organism take this action and not mutant can invade the population . </li></ul><ul><li>Example: the sex ratio in bee population is 1(male):3(female) </li></ul>Selfish Selfish Selfish Selfless Selfless Selfless Selfless Selfless Selfless Selfish
  14. 14. ESS: Definition: <ul><li>Let G=<{1,2},(B,B),(u i )>be a symmetric strategic game, where u 1 (a,b)=u 2 (b,a)=u(a,b) for some function u. </li></ul><ul><li>An evolutionarily stable strategy(ESS) of G is an action </li></ul><ul><ul><li>b*∈B for which (b*,b*) is a NE of G </li></ul></ul><ul><ul><li>and u(b,b)<u(b*,b) for every best response b∈B to b* with b≠b*. </li></ul></ul>
  15. 15. Example: Hawk--Dove <ul><li>Choose to be Hawk or dove? </li></ul><ul><li>Pure NE: (D,H) & (H,D) </li></ul><ul><li>Mixed NE: (0.5D/0.5H, 0.5D/05H) </li></ul><ul><li>If the players have the freedom to choose to be hawk or dove in a repeated game, and utility will be used to reproduce their offspring, what’ll be the optimal strategy? </li></ul>(1-c/)2, (1-c/)2 1,0 H 0,1 ½, ½ D H D
  16. 16. HD Game – mixed strategy (a) Agent 1’s expected utility, red line is it’s best response to given q (agent 2’s strategy) (b) Agent 2’s expected utility, blue line is it’s best response to given p (agent 1’s strategy) (c) Overlap of a and b (d) Look from the top, (0.5,0.5) is the only mixed NE. pure NEs include (0,1) and (1,0) With c = 2, Action = {'Hawk' , 'Dove'} ;
  17. 17. HD Game – ESS invading <ul><li>In all those games, ESS (half-dove-half-hawk mixed strategy (0.5,0.5)) starts from a percentage of 0.1 in the population </li></ul><ul><li>Population: 200, Game Round = 1000 </li></ul><ul><li>Reproduce: proportional to total utility of each type </li></ul><ul><li>Note that Dove is not completely eliminated </li></ul>ESS vs. Dove ESS vs. Hawk ESS vs. 1/4 Hawk ESS vs. 1/4 Dove
  18. 18. HD Game – ESS being invaded <ul><li>In all those games, ESS (half-dove-half-hawk mixed strategy (0.5,0.5)) starts from a percentage of 0.9 in the population </li></ul><ul><li>Setting is same to the last slide </li></ul><ul><li>ESS can successfully defend the invasion of mutants, although it may not be a completely expelling </li></ul><ul><li>The experiment shows that ESS can be taken to be the set of mixed strategy over some finite set of actions </li></ul>ESS vs. Dove ESS vs. Hawk ESS vs. 1/4 Hawk ESS vs. 1/4 Dove
  19. 19. Not all NE are ESS <ul><li>A strict equilibrium b* is an ESS </li></ul><ul><ul><li>(b*,b*) if a symmetric NE </li></ul></ul><ul><ul><li>and no strategy other than b* is a best response to b* </li></ul></ul><ul><li>A nonstrict equilibrium may not be an ESS </li></ul><ul><li>Mixed NE (1/3, 1/3, 1/3) expected payoff t/3 Can be invaded by any pure strategy </li></ul><ul><ul><li>Receives expected utility t/3 when it encounters MixedNE </li></ul></ul><ul><ul><li>Receives expected t when it encounters another pure strategy </li></ul></ul>Example: ESS Mixed NE t,t -1,1 1,-1 1,-1 t,t -1,1 -1,1 1,-1 t,t
  20. 20. More about ESS <ul><li>Widely used in sociobiology </li></ul><ul><ul><li>See Dawkins <selfish gene>, chapter 6 </li></ul></ul><ul><ul><li>Wilson , < sociobiology – New Synthesis>, chapter 5 </li></ul></ul><ul><li>And in politic science and sociology </li></ul><ul><ul><li>See <the evolution of cooperation>, where tit-for-tat is a ESS in Evolutionary Pensioner Dilemma Game </li></ul></ul>

×