von Neumann Poker

771 views
654 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
771
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

von Neumann Poker

  1. 1. On Von Neumann Pokerwith Community Cards<br />Reto Spöhel<br />Joint work with Nicla Bernasconi and Julian Lorenz<br />TexPoint fonts used in EMF. <br />Read the TexPoint manual before you delete this box.: AAAAA<br />
  2. 2. Poker<br /><ul><li>According to Wikipedia,</li></ul>Poker is a popular type of card game in which players gamble on the superior value of the card combination ("hand") in their possession, by placing a bet into a central pot. The winner is the one who holds the hand with the highest value according to an established hand rankings hierarchy, or otherwise the player who remains "in the hand" after all others have folded.<br />
  3. 3. Outline<br />Introduction<br />von Neumann Poker<br />von Neumann Poker withcommunity cards<br />Outlook: the Newman model<br />Conclusion<br />
  4. 4. Research on poker<br />Thegame of poker has beenstudiedfrommany different perspectives:<br />game-theory[this talk]<br />artificalintelligence (heuristics)<br />machinelearning (opponentmodeling)<br />behaviouralpsychology<br />etc.<br />In thegame-theoreticapproach, oneassumes best play for all playersinvolved.<br />allowsdevelopment and application of mathematicaltheories<br />neglectsmanyotherfactors<br />
  5. 5. Game-theoreticresearch on poker<br />almostexclusivelyfortwo-playergame<br />twomainlines of attack:<br />simplifiedmodels of „real“ poker<br />e.g., 2 suitswith 5 cards each<br />solutionbybrute-forcecalculation; lots of computational power needed<br />moreabstractmodels, whichhopefullycapture essential features of poker[this talk]<br />e.g., handsarenumbersu.a.r. from [0,1]<br />hopefullyanalyticallysolvable<br />mostimportantmodel: von Neumann poker<br />
  6. 6. Von Neumann Poker<br />Pchipsare in thepot at thebeginning.<br />X and Y are dealt independent handsx,y2[0,1] u.a.r.<br />X maymake a bet of aor pass („check“).<br />If X checks, bothhandsarerevealed („showdown“), and theplayerwiththehigher hand winsthepot.<br />If X makes a bet, Y caneithermatchthe bet („call“) orconcedethepot to X („fold“).<br />If Y folds, X winsthepot (and gets his bet back).<br />If Y callsX‘s bet, bothhandsarerevealed and theplayerwiththehigher hand winsthepot and thetwobets.<br />In thefollowingweassumeforsimplicityP = a = 1.<br />
  7. 7. Von Neumann Poker<br />Itseemsthat X has an advantage, since Y canonlyreact. So howshould X play to maximize his expectedpayoff?<br />Clearly, alwayscheckingguaranteeshim an expectedpayoff of P/2 = 1/2.<br />Similarly, X cannothopefor an expectedpayoff of morethan P=1, since Y canalwaysfold.<br />
  8. 8. Von Neumann Poker<br />At firstsight, onemightguessthat X should bet thebetter half of his hands, i.e., iff x ¸ 1/2.<br />However, once Y realizesthatthisisX‘sstrategy, he will onlycallwithhands y ¸ 2/3, sincethen<br />he wins P+a=2 chipswithprobability at least 1/3<br />he loses a=1 chipswithprobability at most 2/3<br />i.e., „thepotoddsare in his favor“.<br />1<br />call<br />bet<br />2/3<br />1/2<br />fold<br />check<br />0<br />x<br />y<br />
  9. 9. Von Neumann Poker<br />X‘sexpectedpayoffcanbefoundbyintegratingoverthehands of x and y and is:<br /> P ¢ 1/8 + P ¢ 1/2 ¢ 2/3 + (P+a) ¢ 1/18 – a ¢ 1/9<br /> = 1/8+1/3 = 11/24 < 1/2<br /> X loses money!<br />y<br />1<br />-a<br />call<br />bet-call<br />bet<br />P+a<br />2/3<br />2/3<br />0<br />1/2<br />bet-fold<br />check<br />fold<br />check<br />P<br />P<br />0<br />x<br />0<br />x<br />y<br />1/2<br />
  10. 10. So X has no advantageover Y and shouldnever bet?<br />NO!<br />Withthepreviousstrategy, most of X‘s good handsgo to wastebecause Y just folds.<br />However, X caninduce Y to callmoreoftenbyincludingbluffs in his strategy!<br />von Neumann gave an equilibrium pair of strategies<br />Y‘sstrategyis best response to X‘sstrategy<br />X‘sstrategyis best response to Y‘sstrategy<br />von Neumann, 1928<br />X can achieve an expected payoff of 5/9, which is optimal. <br />Von Neumann Poker<br />
  11. 11. von Neumann‘ssolution<br />Heuristic: wemakethefollowingansatz:<br />With a hand of y1, calling and foldingshouldhavethesameexpectedpayofffor Y:<br />x0¢ (P + a) – (1 – x1) ¢ a = 0<br />With a hand of x0orx1, betting and checkingshouldhavethesameexpectedpayofffor X:<br />y1¢ P + (x1 – y1) ¢ (P + a) – (1 – y1) ¢ a = x0¢ P <br />y1¢ P + (x1 –y1) ¢ (P + a) – (1 – x1) ¢ a = x1¢ P<br />1<br />value-bet<br />call<br />x1<br />y1<br /><ul><li> solution (P=a=1):
  12. 12. x0 = 1/9
  13. 13. y1 = 5/9
  14. 14. x1 = 7/9</li></ul>check<br />x0<br />fold<br />bluff-bet<br />0<br />x<br />y<br />
  15. 15. von Neumann‘ssolution<br />Thetworesultingstrategiesareindeed in equilibrium.<br />Theexpectedpayofffor X turns out to be5/9<br />thevalue of thegameis 5/9(in zero-sumgames, all equilibriahavethesamevalue!)<br />Insights:<br />Bluffing is a game-theoretic necessity!<br />You should bluff-bet your worst hands!<br />1<br />value-bet<br />call<br />x1<br />y1<br /><ul><li> solution (P=a=1):
  16. 16. x0 = 1/9
  17. 17. y1 = 5/9
  18. 18. x1 = 7/9</li></ul>check<br />x0<br />fold<br />bluff-bet<br />0<br />x<br />y<br />
  19. 19. Von Neumann Poker<br />Manyextensions of the von Neumann modelhavebeenstudied<br />allow multiple bettingrounds, raises, reraises, etc.<br />handsmaydepend on eachother…<br />etc.<br />byscientists and professionalpokerplayeralike.<br />Themathematics of Poker, Bill Chen and JerrodAnkenman, 2006<br />Chris Ferguson, PhD<br /><ul><li>2000 World Series of Poker champion
  20. 20. co-author of severalpapers on von Neumann poker</li></li></ul><li>Ourcontribution<br />In theclassical von Neumann model (and itsextensions), no furtherrandominfluencesarepresentoncebothplayershavereceivedtheirhands.<br />In real poker, community cards aredrawnbetweenbettingrounds.<br /> playersmay bet with a bad hand, hopingthatthe right card will show up and turn itinto a good hand.<br /> playerswith a good hand tend to bet moreaggressively to force otherplayers to fold.<br />Weproposethefollowingextension of the von Neumann modelthataccountsforthesefeatures:<br />Beforetheshowdown, throw an unfair coin. Withprobability q, thelower hand wins!<br />
  21. 21. Introducingtheflip<br />Introducingtheflip:<br />With a hand of y1, calling and foldingshouldhavethesameexpectedpayofffor Y:<br />before (q=0):<br />x0¢ (P + a) – (1 – x1) ¢ a = 0<br />now:<br /> [x0¢ (1 – q) + (1 – x1) ¢ q] ¢ (P + a) <br />– [(1 – x1) ¢ (1 – q) + x0¢ q] ¢ a = 0<br />1<br />value-bet<br />call<br /><ul><li> solution (P=a=1):
  22. 22. x0 = x0(q)
  23. 23. y1 = y1(q)
  24. 24. x1 =x1(q)</li></ul>x1<br />y1<br />check<br />x0<br />fold<br />bluff-bet<br />0<br />x<br />y<br />
  25. 25. Introducingtheflip<br /><ul><li>Weobtain</li></ul>7/9<br />x1<br />5/9<br />?<br />y1<br />x0<br />1/9<br />1/3 =: q0<br />
  26. 26. Beyondthecritical q<br />Whathappensfor q > q0 = 1/3?<br />Y will callevery bet sinceevenwiththeworst hand y = 0<br />he wins P+a=2 chipswithprobability at least q = 1/3<br />he loses a=1 chipswithprobability at most 1 – q = 2/3<br />Knowingthis, X will bet thebetter half of his hands.<br />1<br /><ul><li>forgeneral P and a:</li></ul> q : (1-q) = a : (P+a)<br /> q0 = a/(P+2a)<br />bet<br />call<br />1/2<br />check<br />0<br />x<br />y<br />
  27. 27. Thefullpicture<br /><ul><li>As q increases, X makesmorevaluebets and bluffsless.
  28. 28. Y isinducedbythe q to call!
  29. 29. For q ¸ q0, there‘s no point in bluffing, since Y will alwayscallanyway.
  30. 30. X value-betsmoreoften to protect his good handsfrombeingflippedinto bad hands.</li></ul>x1<br />y1<br />x0<br />q0 = 1/3<br />
  31. 31. Thevalue of thegame<br />7/12 = 0.583…<br /><ul><li>Theexpectedpayoff of X is maximal at q = q0.
  32. 32. i.e., when Y has just enoughincentive to callevery bet of X without X wastingmoney on bluffs.</li></ul>5/9 = 0.555…<br />value of thegame<br />1/2 = 0.5<br />q0 = 1/3<br />
  33. 33. An improvedmodel<br />Wheredoesthediscontinuity at q0 come from?<br />Wefixedthe bet sizea (arbitrarily) beforethegamestarted. <br />Whatifweallow X to look at his hand and then bet anyamounta ¸ 0 he likes?<br />…and withflipprobability q, 0 · q < 1/2<br />Newman, 1959<br />Bernasconi, Lorenz, S., 2007+<br />X can achieve an expected payoff of (16-q)/(28-8q), which is optimal. <br />X can achieve an expected payoff of 4/7, which is optimal. <br />
  34. 34. Newman‘ssolution<br /><ul><li>The red line isX‘s bet a(x)
  35. 35. X checksfor 1/7 · x · 4/7
  36. 36. Thegreen line isY‘scallingthreshold
  37. 37. Y callsiff (y,a) isbelowthegreen line</li></ul>(q = 0)<br />bet a<br />„value-bet“<br />„bluff-bet“<br /><ul><li>obtainedwithsimilar, butmoreinvolvedheuristicsthanbefore( differentialequations)</li></ul>1/7<br />4/7<br />hand x, resp. y<br />
  38. 38. Introducingtheflip<br /><ul><li>Similarly to before:</li></ul> q : (1-q) = a : (P+a)<br />  a0 = q/(1-2q) ¢ P<br /><ul><li>evenwith his worst hand, Y callsevery bet of at mosta0 (pot-odds!)
  39. 39. knowingthis, X neverbets an amountbetween 0 and a0
  40. 40. neither as a value bet (iftheoddsare in his favour [x ¸ 1/2], he bets at least a0),
  41. 41. nor as a bluff bet (there‘s no value bet forwhich to inducemorecalls).</li></ul>(q = 1/3)<br />bet a<br />a0 = <br />1/2<br />hand x, resp. y<br />
  42. 42. Introducingtheflip<br /><ul><li>As q increases, X bets his valuebetsmoreaggressively.</li></ul>bet a<br />(q = 0.4)<br />(q = 1/3)<br />hand x<br />
  43. 43. Thevalue of thegame<br />31/48 = 0.645…<br /><ul><li>For 0 < q · 1/2, wehave</li></ul>value(q) = (16-q)/(28-8q)<br /><ul><li> thevalueisstrictlyincreasing in qbutdiscontinuous at q=1/2, sincetrivially</li></ul>value(1/2) = 1/2.<br /> What is going on?<br />value of thegame<br />4/7 = 0.571…<br /><ul><li>Weallowedarbitrarily high bets a. Moreover, a0diverges at q = 1/2.
  44. 44. IfwelimitX‘sbankroll, thesingularitiesvanish.</li></li></ul><li>Conclusion<br />Weproposed a way of including „community cards“ (randomeffectsafterbetting) intoboththe von Neumann and the Newman model…<br />… and gavecompleteanalyticalsolutions.<br />As expected, weobservedincreasingly aggressive bettingfor larger q.<br />In bothmodels, weobserve a „phasetransition“ when<br /> q : (1-q) = a : (P+a) (“flip-odds = pot-odds”)<br />
  45. 45. Questions?<br />

×