Successfully reported this slideshow.
Upcoming SlideShare
×

Game theory

5,839 views

Published on

Published in: Technology, Economy & Finance
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Game theory

1. 1. Game TheoryEnrico Franchi (efranchi@ce.unipr.it)
2. 2. IntroductionGame theory can be deﬁned as the studyof mathematical models of conﬂict andcooperation between intelligent rationaldecision-makers. R. Myerson
3. 3. Rationality and Intelligence • A decision-maker is rational if he makes decisions consistently in pursuit of his own objectives • A player is intelligent if he knows everything that we know about the game and he can make inferences about the situation that we can make
4. 4. Outcomes• Let O be a ﬁnite set of outcomes• A lottery is a probability distribution over O l = [ p1 : o1 ,…, pk : ok ] oi ∈O pi ∈[0,1] k ∑p i =1 i=1• We assume agents can rank outcomes and lotteries with a utility function
5. 5. TCP Game Prisoner’s Dilemma C D C -1,-1 -4,0 D 0,-4 -3,-3D: defective implementationC: correct implementation
6. 6. Game in Normal Form• A ﬁnite n-person normal form game is a tuple (N, A, u) where • N is a ﬁnite set of n players • A=A1⨉...⨉An, where Ai is a ﬁnite set of actions available to player i • u=(u1,...,un), where ui:A↦R is a real valued utility function• a=(a1,...,an) is an action proﬁle
7. 7. TCP Game (again) A2 • N={1,2} C D • A={C,D}⨉{C,D} C -1,-1 -4,0 D 0,-4 -3,-3 A1 A2 u1 u2A1 C C -1 -1 C D -4 0 D C 0 -4 D D -3 -3
8. 8. Actions• Actions can be “arbitrarily complex”• Ex.: international draughts • an action is not a move • an action maps every possible board conﬁguration to the move to be played if the conﬁguration occurs 2·1022
10. 10. Rock-Paper-Scissor Rock Paper Scissors Rock 0,0 -1,1 1,-1 Paper 1,-1 0,0 -1,1Scissors -1,1 1,-1 0,0
11. 11. Strategy• A pure strategy is selecting an action and playing it• A mixed strategy for player i is an element of the set Si=!(Ai) of probability distributions over Ai• The support of a mixed strategy is the set of pure strategies {ai|si(ai) > 0}• The set of mixed-strategy proﬁles is S1⨉...⨉Sn and a mixed strategy proﬁle is a tuple (s1, ..., sn)• The utility of a mixed strategy proﬁle is n ui (s) = ∑ ui (a)∏ s j (a j ) a∈A j=1
12. 12. Solution Concept• Games are complex, the environment can be stochastic, other player’s choices affect the outcome• Game theorists study certain subsets of outcomes that are interesting in one sense or another which are called solution concepts
13. 13. Pareto Efﬁciency• The strategy proﬁle s Pareto dominates the strategy proﬁle s’ if for some players the utility for s is strictly higher and for the others is not worse• A strategy proﬁle is Pareto optimal if there is no other strategy proﬁle dominating it
14. 14. Nash Equilibrium• Player’s i best response to the strategy proﬁle s-i is a mixed strategy s*i ∈Si such that ui(s*i ,s-i)≥ui(si, s-i) for every si ∈Si• A strategy proﬁle s=(s1,...,sn) is a Nash equilibrium if, for all agents , si is a best response to s-i• Weak Nash (≥), Strong Nash (>)
15. 15. Battle of the Sexes LW WL • Both pure strategies are Nash EquilibriaLW 2,1 0,0 • Are there any other NashWL 0,0 1,2 Equilibria? • There is at least another mixed-strategy equilibrium (usually very tricky to compute, but can be done with simple examples)
16. 16. Battle of the Sexes LW WL • Suppose the husband chooses LW with probability p and WL withLW 2,1 0,0 probability p-1 • The wife should be indifferentWL 0,0 1,2 between her available options, otherwise she would be better off choosing a pure strategy Being “indifferent” means • What is the p which allows the obtaining the same utility, not wife to be really indifferent? playing indifferently • Please notice that the pure strategies are Pareto optimal
17. 17. Battle of the Sexesp: probability that husband plays LW r: probability that wife plays LWU wife (LW) = U wife (WL) U husband (LW) = U husband (WL)1·p + 0·(1− p) = 0·p + 2·(1− p) 2·r + 0·(1− r) = 0·r + 1·(1− r) 2 1p = 2− 2p p= 2r = 1− r r= 3 3U w (s) = 2(1− p)(1− r) + pr ⎛ 2 1 ⎞ 2 2 2U w ⎜ sw (r), LW + WL ⎟ = (1− r) + r = ⎝ 3 3 ⎠ 3 3 3U h (s) = (1− p)(1− r) + 2 pr ⎛ 1 2 ⎞ 2 2 2U h ⎜ sh ( p), LW + WL ⎟ = (1− p) + p = ⎝ 3 3 ⎠ 3 3 3
18. 18. Matching Pennies Heads TailsHeads 1,-1 -1,1 • Do we have any pure strategies? Tails -1,1 1,-1 • No • Do we have mixedU1 (H) = U1 (T) strategies?1·p + (−1)·(1− p) = −1·p + 1·(1− p) • Yes 12 p − 1 = 1− 2 p p= 2
19. 19. Rock-Paper-Scissor Rock Paper Scissors • Do we have any pure Rock 0,0 -1,1 1,-1 strategies?Paper 1,-1 0,0 -1,1 • NoScissors -1,1 1,-1 0,0 • Do we have mixed strategies?pr + p p + ps = 1 • YesU1 (R) = U1 (P) = U1 (S)
20. 20. Rock-Paper-Scissorpr + p p + ps = 1 R P SU1 (R) = U1 (P) = U1 (S) R 0,0 -1,1 1,-1 P 1,-1 0,0 -1,1⎧0 pr + (−1) p p + 1ps = 1pr + 0 p p + (−1) ps S -1,1 1,-1 0,0⎨⎩1pr + 0 p p + (−1) ps = (−1) pr + 1p p + 0 ps ⎧ 1 ps + p p ⎪ ps =⎧2 ps = pr + p p 2 ps = + pp 3⎨ 2 ⎪ 2 pr = ps + p p ⎪ 1⎩ 4 ps = ps + 3p p ⎨ pr = ⎪ 3ps = p p = pr ⎪p = 1 ⎪ p 3 ⎩
21. 21. Existence of Nash Equilibria • We have seen that not every game has a pure strategy Nash equilibrium • Does every game have a Nash equilibrium (random or pure)? Theorem (Nash, 1951) Every game with a ﬁnite number of players and action proﬁles has at least one Nash equilibrium
22. 22. Computing Nash Equilibria • There are algorithms which compute Nash equilibria, but they are exponential in the size of the game • It is not known if there are polynomial algorithms (but the consensus is that there are none)
23. 23. Dominated Strategies Deﬁnitions• Let si and si’ be two strategies of player i and S-i the set of all strategy proﬁles of the remaining players • si strictly dominates si’ if for all s-i ∈S-i, it is the case that ui(si,s-i)>ui(si’,s-i) • si weakly dominates si’ if for all s-i ∈S-i, it is the case that ui(si,s-i)≥ui(si’,s-i) and for at least one s-i ∈S-i it is the case that ui(si,s-i)>ui(si’,s-i) • si very weakly dominates si’ if for all s-i ∈S-i, it is the case that ui(si,s-i)≥ui(si’,s-i)
24. 24. Dominated Strategies Example L C R L CU 3,1 0,1 0,0 U 3,1 0,1M 1,1 1,1 5,0 M 1,1 1,1D 0,1 4,1 0,0 D 0,1 4,1 L C U 3,1 0,1 D 0,1 4,1
25. 25. Dominated Strategies Prisoner’s Dilemma C D C DC -1,-1 -4,0 D 0,-4 -3,-3D 0,-4 -3,-3 D D -3,-3
26. 26. Dominated Strategies as solution concepts• The set of all strategy proﬁles that assign 0 probability to playing any action that would be removed through iterated removal of strictly dominated strategies is a solution concept• Sometimes, no action can be “removed”, sometimes we can solve the game (we say the game is solvable by iterated elimination)
27. 27. Dominated Strategies Costs• Iterated elimination ends after a ﬁnite number of steps• Iterated elimination preserves Nash equilibria • We can use it to reduce the size of the game• Iterated elimination of strictly dominated strategies can occur in any order without changing the results• Checking if a (possibly mixed) strategy is dominated can be done in polynomial time • Domination by pure strategies can be checked with a very simple iterative algorithm • Domination by mixed strategies can be checked solving a linear problem • Iterative elimination needs only to check pure strategies
28. 28. Dominated Strategies (domination by pure-strategies) forall  pure  strategies  ai∈Ai  for  player   i  where  ai≠si  do dom  ←  true forall  pure-­‐strategy  profiles  a-­‐i∈A-­‐i  do if  ui(si,a-­‐i)≥ui(ai,a-­‐i)  then dom  ←  false break if  dom  =  true  then return  true    return  false
29. 29. Other Solution Concepts: Maxmin & Minmax• The maxmin strategy for player i in an n player, general sum game is a not necessarily unique (mixed) strategy that maximizes i’s worst case payoff• The maxmin value (or security level) is the minimum payoff level guaranteed by a maxmin strategy arg max si min s− i ui (si , s−i )• In two player general sum games the minmax strategy for player i against player –i is the strategy that keeps the maximum payoff for –i at minimum• It is a punishment arg min si max s− i ui (si , s−i )
30. 30. Other Solution Concepts: Minmax, n-player• In an n-player game, the minmax strategy for player i against player j ≠ i is i’s component of the mixed-strategy proﬁle s-j in the expression: arg min s− j max s j u j (s j , s− j ) where –j denotes the set of players other that j.• Player i receives his minmax value if players –i choose their strategies to minimize i utility “after” he chose strategy si• A player maxmin value is always less than or equal to his minmax value
31. 31. Maxmin & Minmax Examples • Matching Pennies • Maxmin: 0.5 T + 0.5 H • Minmax: 0.5 T + 0.5 H • Battle of Sexes • Maxmin: H→0.66 LW + 0.33 WL W→0.33 LW + 0.66 WL • Minmax: H→0.66 LW + 0.33 WL W→0.33 LW + 0.66 WL
32. 32. Minmax TheoremTheorem (von Neumann, 1928) In any ﬁnite,two-player, zero-sum game, in any Nashequilibrium each player receives a payoff that isequal to both his maxmin value and his minmaxvalue• Each player’s maxmin equals his minmax (value of the game)• Maxmin strategies coincide with minmax strategies• Any maxmin strategy proﬁle is a Nash equilibrium and any Nash equilibrium is a maxmin strategy proﬁle
33. 33. Matching Pennies Matching Pennies for P1 1P1 Expected Utility 0.5 0.9 0.6 0 0.3 0 0.1 0.2 0 0.3 0.4 0.5 -0.5 0.6 0.7 0.8 0.9 1 P2 P(heads) -1 P1 P(heads)
34. 34. Minimax Regret• An agent i’s regret for playing an action ai if other agents adopt action proﬁle a-i is deﬁned as: ⎡ ⎤ ⎢ ai′ ∈Ai ui ( ai′ , a−i ) ⎥ − ui ( ai , a−i ) ⎣ max ⎦• An agent i’s maximum regret for playing an action ai is deﬁned as: ⎛⎡ ⎤ ⎞ max ⎜ ⎢ max ui ( ai′ , a−i ) ⎥ − ui ( ai , a−i )⎟ a− i ∈A− i ⎝ ⎣ ai ′ ∈Ai ⎦ ⎠• Minimax regret actions for agent i are deﬁned as: ⎡ ⎛⎡ ⎤ ⎞⎤ arg min ⎢ max ⎜ ⎢ max ui ( ai′ , a−i ) ⎥ − ui ( ai , a−i )⎟ ⎥ ai ∈Ai ⎣ a− i ∈A− i ⎝ ⎣ ai ′ ∈Ai ⎦ ⎠⎦
35. 35. Maxmin vs. Minmax Regret L Rregret(T ,[R])= 1− 1+  = regret(B,[R]) = 1− 1 = 0regret(T ,[L]) = 100 − 100 = 0 T 100, a 1-!, bregret(B,[L]) = 100 − 2 = 98max regret(T )= max{,0} =  B 2, c 1, dmax regret(B) = max{98,0} = 98 P1 Maxmin is B (why?), his Minimax Regret strategy is T
36. 36. Computing Equilibria
37. 37. Computing Nash-Equilibriumfor two-players zero-sum games• Consider the class of two-player zero-sum games ( Γ = {1, 2} , A1 × A2 , ( u1 ,u2 ) ) *• Ui is the expected utility for player i in equilibrium• In the next slide, the LP for computing player 2 and player 1 strategies are given• Linear Programs are rather inexpensive to compute
38. 38. Computing Nash-Equilibriumfor two-players zero-sum games * minimize U 1 subject to ∑ u (a , a ) ⋅ s 1 1 j k 2 k 2 ≤U * 1 ∀j ∈A1 k∈A2 ∑s k 2 =1 k∈A2 s ≥0 k 2 ∀k ∈A2• Constants: u1(...)• Variables: s2, U1* U1* is a maxmin value!
39. 39. Computing Nash-Equilibriumfor two-players zero-sum games * maximize U 1 subject to ∑ u (a , a ) ⋅ s 1 1 j k 2 1 j ≥U * 1 ∀k ∈A2 j∈A1 ∑s 1 j =1 j∈A1 s ≥0 1 j ∀j ∈A1• Constants: u1(...)• Variables: s1, U1* U1* is a maxmin value!
40. 40. Computing Nash-Equilibriumfor two-players zero-sum games * minimize U 1 subject to ∑ u (a , a ) ⋅ s 1 1 j k 2 k 2 + r1 = U j * 1 ∀j ∈A1 k∈A2 ∑s k 2 =1 k∈A2 s ≥0 k 2 ∀k ∈A2 r1 ≥ 0 j ∀j ∈A1• Constants: u1(...)• Variables: s2, U1* U1* is a maxmin value!
41. 41. Computing Nash-Equilibriumfor two-players zero-sum games * minimize U 1 subject to ∑ u (a , a ) ⋅ s 1 1 j k 2 1 j +r =U 2 k * 1 ∀k ∈A2 j∈A1 ∑s 1 j =1 j∈A1 s ≥0 1 j ∀j ∈A1 r ≥0 1 k ∀k ∈A2• Constants: u1(...)• Variables: s2, U1* U1* is a maxmin value!
42. 42. Computing maxmin & minmaxfor two-players general-sum games • We know how to compute minmax & maxmin strategies for two-players 0-sum games • It is sufﬁcient to transform the general-sum game in a 0-sum game • Let G be an arbitrary two-player game G=({1,2}, A1⨉A2,(u1,u2)); we deﬁne G’= ({1,2}, A1⨉A2,(u1,-u1)) • G’ is 0-sum: a strategy that is part of a Nash equilibrium for G’ is a maxmin strategy for 1 in G’ • Player 1 maxmin strategy is independent of u2 • Thus player’s 1 maxmin strategy is the same in G and in G’ • A minmax strategy for Player 2 in G’ is a minmax strategy for 2 in G as well (for the same reasons)
43. 43. Two Players General sum Games • We can formulate the∑ u (a , a ) ⋅ s 1 1 j k 2 k 2 + r1 = U j * 1 ∀j ∈A1 game as a linear complimentarity problemk∈A2∑ u (a , a ) ⋅ s (LCP) 1 1 j k 2 1 j +r =U2 k * 1 ∀k ∈A2j∈A1 • This is a constraint ∑s 1 j =1 ∑s k 2 =1 satisfaction problem j∈A1 k∈A2 (feasibility, not optimization) s ≥ 0, s ≥ 0 1 j k 2 ∀j ∈A1 , ∀k ∈A2 r1 ≥ 0, r ≥ 0 j k ∀j ∈A1 , ∀k ∈A2 • The Lemke-Howson 2 algorithm is the best r1 ·s = 0, r ·s = 0 j 1 j 2 k k 2 ∀j ∈A1 , ∀k ∈A2 suited to solve this kind of problems
44. 44. Computing n-players Nash • Could be formulated as a nonlinear complementarity problem (NLCP), thus it would not be easily solvable • A sequence of linear complementarity problems (SLCP) can be used; it is not always convergent, but if we’re lucky it’s fast • It is possible to formulate as the computation of the minimum of a speciﬁc function, both with or without constraints
45. 45. Software ToolsMcKelvey, Richard D., McLennan,Andrew M., and Turocy, Theodore L.(2010). Gambit: Software Tools for GameTheory, Version 0.2010.09.01.http://www.gambit-project.org.
46. 46. Other Games
47. 47. Extensive Form Game Example Outline Introduction Game Representations Reductions Extended Form of Game-1 • Each player bets a et (2,-2) me coin 1.a raise 2.0 pa ss fo ld (1,-1) • Player 1 draw a (re 5 d) . (1,-1) card and is the only 0 one to see it (-2,2) et .5 ) (b me la ck 1.b raise 2.0 • Player 1 also plays pa ss before Player 2 fo ld (1,-1) (-1,1)
48. 48. Repeated Games• What happens if the same NF game is repeated: • An inﬁnite number of times? • A ﬁnite number of times? • A ﬁnite but unknown number of times?
49. 49. Bayesian Games• Represent uncertainty about the game being played; there is a set of possible games • with the same number of agents and same strategy spaces but different payoffs • Agents beliefs are posteriors obtained conditioning a common prior on individual private signals