The Strathclyde Poker Research Environment

1,045 views

Published on

This presentation was given at AISB 2011 and introduces the Strathclyde Poker Research Environment (SPREE) an open tool for Poker research. Available from Sourceforge @ https://sourceforge.net/projects/spree-poker/

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,045
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • The Strathclyde Poker Research Environment

    1. 1. SPREEThe Strathclyde Poker Research Environment L. Dicken, N. Johnstone, J. Levine, P. Rodgers Strathclyde AI and Games Research Group
    2. 2. Poker at SAIG2
    3. 3. Poker at SAIG • Evolutionary Algorithms applied to Tournament Poker. (Carter & Levine)[1] [1] - An Investigation into Tournament Poker Strategy using Evolutionary Algorithms, Carter & Levine, Proceedings of CIG20072
    4. 4. Poker at SAIG • Evolutionary Algorithms applied to Tournament Poker. (Carter & Levine)[1] • Co-Evolution for Counter Strategies in Texas Hold ‘Em (Thomson, Levine & Wotherspoon)[2] [1] - An Investigation into Tournament Poker Strategy using Evolutionary Algorithms, Carter & Levine, Proceedings of CIG2007 [2] - Evolution of Counter-Strategies: Application of Co-evolution to Texas Holdem Poker, Thomson, Levine & Wotherspoon, Proceedings of CIG20082
    5. 5. Poker at SAIG • Evolutionary Algorithms applied to Tournament Poker. (Carter & Levine)[1] • Co-Evolution for Counter Strategies in Texas Hold ‘Em (Thomson, Levine & Wotherspoon)[2] • Monte Carlo Simulation for Ten Player Texas Hold ‘Em (Birnie, Dicken & Levine) [1] - An Investigation into Tournament Poker Strategy using Evolutionary Algorithms, Carter & Levine, Proceedings of CIG2007 [2] - Evolution of Counter-Strategies: Application of Co-evolution to Texas Holdem Poker, Thomson, Levine & Wotherspoon, Proceedings of CIG20082
    6. 6. Poker at SAIG • Evolutionary Algorithms applied to Tournament Poker. (Carter & Levine)[1] • Co-Evolution for Counter Strategies in Texas Hold ‘Em (Thomson, Levine & Wotherspoon)[2] • Monte Carlo Simulation for Ten Player Texas Hold ‘Em (Birnie, Dicken & Levine) • Machine Learning for Player Archetyping (Birnie, Dicken & Levine) [1] - An Investigation into Tournament Poker Strategy using Evolutionary Algorithms, Carter & Levine, Proceedings of CIG2007 [2] - Evolution of Counter-Strategies: Application of Co-evolution to Texas Holdem Poker, Thomson, Levine & Wotherspoon, Proceedings of CIG20082
    7. 7. Incomplete Information3
    8. 8. Incomplete Information • In the Machine Learning project we found a key weakness in existing Poker research.3
    9. 9. Incomplete Information • In the Machine Learning project we found a key weakness in existing Poker research. • Based primarily on data gathered from commercial online casinos through play.3
    10. 10. Incomplete Information • In the Machine Learning project we found a key weakness in existing Poker research. • Based primarily on data gathered from commercial online casinos through play. • Significant information remains hidden, e.g. folded hands.3
    11. 11. Incomplete Information • In the Machine Learning project we found a key weakness in existing Poker research. • Based primarily on data gathered from commercial online casinos through play. • Significant information remains hidden, e.g. folded hands. ‣ “Playable” hands are the only positive cases shown3
    12. 12. Incomplete Information • In the Machine Learning project we found a key weakness in existing Poker research. • Based primarily on data gathered from commercial online casinos through play. • Significant information remains hidden, e.g. folded hands. ‣ “Playable” hands are the only positive cases shown ‣ Infer from this which hands are “Unplayable”3
    13. 13. Incomplete Information • In the Machine Learning project we found a key weakness in existing Poker research. • Based primarily on data gathered from commercial online casinos through play. • Significant information remains hidden, e.g. folded hands. ‣ “Playable” hands are the only positive cases shown ‣ Infer from this which hands are “Unplayable” ‣ Compensation through large datasets (~1,000,000 games)3
    14. 14. Incomplete Information • In the Machine Learning project we found a key weakness in existing Poker research. • Based primarily on data gathered from commercial online casinos through play. • Significant information remains hidden, e.g. folded hands. ‣ “Playable” hands are the only positive cases shown ‣ Infer from this which hands are “Unplayable” ‣ Compensation through large datasets (~1,000,000 games) • Sub-optimal approach based on availability of data3
    15. 15. Reinventing the Wheel4
    16. 16. Reinventing the Wheel • Huge portions of these projects got spent implementing Poker rules.4
    17. 17. Reinventing the Wheel • Huge portions of these projects got spent implementing Poker rules. • Wasted effort4
    18. 18. Reinventing the Wheel • Huge portions of these projects got spent implementing Poker rules. • Wasted effort • Fragmentation > Potentially flawed implementations4
    19. 19. Reinventing the Wheel • Huge portions of these projects got spent implementing Poker rules. • Wasted effort • Fragmentation > Potentially flawed implementations • Skews results4
    20. 20. Two Birds, One Stone5
    21. 21. Two Birds, One Stone • We need better data, which we can’t get from online casinos5
    22. 22. Two Birds, One Stone • We need better data, which we can’t get from online casinos ‣ But only online casinos have the kind of volume of data we need.5
    23. 23. Two Birds, One Stone • We need better data, which we can’t get from online casinos ‣ But only online casinos have the kind of volume of data we need. • We need a standard testbed in which to conduct Poker research.5
    24. 24. Two Birds, One Stone • We need better data, which we can’t get from online casinos ‣ But only online casinos have the kind of volume of data we need. • We need a standard testbed in which to conduct Poker research. ‣ Save on development time, spend more time on the AI5
    25. 25. Two Birds, One Stone • We need better data, which we can’t get from online casinos ‣ But only online casinos have the kind of volume of data we need. • We need a standard testbed in which to conduct Poker research. ‣ Save on development time, spend more time on the AI ‣ Standardise our experimental methodology, more comparable results.5
    26. 26. Introducing SPREE6
    27. 27. Introducing SPREE • The Strathclyde Poker Research Environment provides an open framework for Poker experimentation.6
    28. 28. Introducing SPREE • The Strathclyde Poker Research Environment provides an open framework for Poker experimentation. • Client/Server structure allows AI players to be easily created6
    29. 29. Introducing SPREE • The Strathclyde Poker Research Environment provides an open framework for Poker experimentation. • Client/Server structure allows AI players to be easily created • Pre-built GUI client means players can play against AI or each other.6
    30. 30. Introducing SPREE • The Strathclyde Poker Research Environment provides an open framework for Poker experimentation. • Client/Server structure allows AI players to be easily created • Pre-built GUI client means players can play against AI or each other. • Open server means we can get complete data on all games being played.6
    31. 31. Components7
    32. 32. SPREE Server8
    33. 33. SPREE Server • Controls the flow of the game.8
    34. 34. SPREE Server • Controls the flow of the game. • Stores certain state information (e.g. Player bankroll, Active tables) in a database - non-volatile.8
    35. 35. SPREE Server • Controls the flow of the game. • Stores certain state information (e.g. Player bankroll, Active tables) in a database - non-volatile. • Broken into two main components : Casino and Table8
    36. 36. SPREE Server • Controls the flow of the game. • Stores certain state information (e.g. Player bankroll, Active tables) in a database - non-volatile. • Broken into two main components : Casino and Table • Casino is the high level - logins, providing information on active tables etc.8
    37. 37. SPREE Server • Controls the flow of the game. • Stores certain state information (e.g. Player bankroll, Active tables) in a database - non-volatile. • Broken into two main components : Casino and Table • Casino is the high level - logins, providing information on active tables etc. • The Table server runs the game according to the specification it has been given.8
    38. 38. Game Specifications9
    39. 39. Game Specifications • Focus is currently on Texas Hold ‘Em9
    40. 40. Game Specifications • Focus is currently on Texas Hold ‘Em • Rules of the game specified in XML9
    41. 41. Game Specifications • Focus is currently on Texas Hold ‘Em • Rules of the game specified in XML • Modular component for the server, can swap to other game modes.9
    42. 42. Game Specifications • Focus is currently on Texas Hold ‘Em • Rules of the game specified in XML • Modular component for the server, can swap to other game modes. ‣ Note that the current client is specific to Texas Hold ‘Em9
    43. 43. Collecting Data10
    44. 44. Collecting Data • Two types of data being logged10
    45. 45. Collecting Data • Two types of data being logged ‣ “Recording” : dump GameState messages that have been generated through the game.10
    46. 46. Collecting Data • Two types of data being logged ‣ “Recording” : dump GameState messages that have been generated through the game. ‣ “Hand History” : a human-readable export of the game, compatible with Poker stats software e.g. Poker Tracker10
    47. 47. SPREE Client11
    48. 48. SPREE Client • Java implementation modelled on existing commercial casino interfaces.11
    49. 49. SPREE Client • Java implementation modelled on existing commercial casino interfaces. • Connects to Casino Server, logs in and retrieves data about tables working.11
    50. 50. SPREE Client • Java implementation modelled on existing commercial casino interfaces. • Connects to Casino Server, logs in and retrieves data about tables working. • Then connects to Table Server and plays Poker.11
    51. 51. SPREE Client • Java implementation modelled on existing commercial casino interfaces. • Connects to Casino Server, logs in and retrieves data about tables working. • Then connects to Table Server and plays Poker. • Principal difference from commercial implementations :11
    52. 52. SPREE Client • Java implementation modelled on existing commercial casino interfaces. • Connects to Casino Server, logs in and retrieves data about tables working. • Then connects to Table Server and plays Poker. • Principal difference from commercial implementations : ‣ No chat, want to avoid play devolving into Turing tests.11
    53. 53. SPREE Client12
    54. 54. SPREE Replayer13
    55. 55. SPREE Replayer • The Replayer is an offline GUI “client”13
    56. 56. SPREE Replayer • The Replayer is an offline GUI “client” • Uses data logged by the server to visualise games of Poker13
    57. 57. SPREE Replayer • The Replayer is an offline GUI “client” • Uses data logged by the server to visualise games of Poker • Gives developers the ability to dig into games and understand the behaviour of their bots.13
    58. 58. SPREE Replayer • The Replayer is an offline GUI “client” • Uses data logged by the server to visualise games of Poker • Gives developers the ability to dig into games and understand the behaviour of their bots. ‣ Or social scientists the ability to analyse the behaviour of players.13
    59. 59. SPREE Admin14
    60. 60. SPREE Admin • Dedicated administration interface.14
    61. 61. SPREE Admin • Dedicated administration interface. • Allows moderators to control the server, register new player accounts and set up tables for play.14
    62. 62. SPREE Admin • Dedicated administration interface. • Allows moderators to control the server, register new player accounts and set up tables for play. • Also allows for full-information observation of games in progress.14
    63. 63. SPREE Bots15
    64. 64. SPREE Bots • Bots can be implemented in any language provided they conform to the communication spec.15
    65. 65. SPREE Bots • Bots can be implemented in any language provided they conform to the communication spec. • A framework has been created in Java. Implementing a bot is simply implementing a single method.15
    66. 66. SPREE Bots • Bots can be implemented in any language provided they conform to the communication spec. • A framework has been created in Java. Implementing a bot is simply implementing a single method. • GameState messages update a TableModel which can be queried by the AI routine.15
    67. 67. SPREE Bots • Bots can be implemented in any language provided they conform to the communication spec. • A framework has been created in Java. Implementing a bot is simply implementing a single method. • GameState messages update a TableModel which can be queried by the AI routine. • “makeMove” method is triggered when the server asks the bot for an action, options are presented to the method15
    68. 68. Future Projects16
    69. 69. Future Projects • Revisiting the Monte Carlo and Machine Learning projects previously mentioned within the context of SPREE.16
    70. 70. Future Projects • Revisiting the Monte Carlo and Machine Learning projects previously mentioned within the context of SPREE. • Next project I’ll be supervising will be to develop some standard bots to package with future versions of the release.16
    71. 71. Future Projects • Revisiting the Monte Carlo and Machine Learning projects previously mentioned within the context of SPREE. • Next project I’ll be supervising will be to develop some standard bots to package with future versions of the release. • Scope for various experiments not just for AI but also in sociology.16
    72. 72. Future Projects • Revisiting the Monte Carlo and Machine Learning projects previously mentioned within the context of SPREE. • Next project I’ll be supervising will be to develop some standard bots to package with future versions of the release. • Scope for various experiments not just for AI but also in sociology. ‣ How does value of wager affect behaviour.16
    73. 73. Future Projects • Revisiting the Monte Carlo and Machine Learning projects previously mentioned within the context of SPREE. • Next project I’ll be supervising will be to develop some standard bots to package with future versions of the release. • Scope for various experiments not just for AI but also in sociology. ‣ How does value of wager affect behaviour. ‣ Quantification of “on tilt” pre-indicators.16
    74. 74. Availability17
    75. 75. Availability • Initial public release TODAY!17
    76. 76. Availability • Initial public release TODAY! • Licensed under GPL.17
    77. 77. Availability • Initial public release TODAY! • Licensed under GPL. • Available from Sourceforge at :17
    78. 78. Availability • Initial public release TODAY! • Licensed under GPL. • Available from Sourceforge at : http://www.sourceforge.net/projects/spree-poker/17
    79. 79. Availability • Initial public release TODAY! • Licensed under GPL. • Available from Sourceforge at : http://www.sourceforge.net/projects/spree-poker/ • Includes Server, Client, Tools and Bot Framework.17
    80. 80. Availability • Initial public release TODAY! • Licensed under GPL. • Available from Sourceforge at : http://www.sourceforge.net/projects/spree-poker/ • Includes Server, Client, Tools and Bot Framework. • Access to resources such as hand history, recordings via Sourceforge.17
    81. 81. Availability • Initial public release TODAY! • Licensed under GPL. • Available from Sourceforge at : http://www.sourceforge.net/projects/spree-poker/ • Includes Server, Client, Tools and Bot Framework. • Access to resources such as hand history, recordings via Sourceforge. • Bot API :17
    82. 82. Availability • Initial public release TODAY! • Licensed under GPL. • Available from Sourceforge at : http://www.sourceforge.net/projects/spree-poker/ • Includes Server, Client, Tools and Bot Framework. • Access to resources such as hand history, recordings via Sourceforge. • Bot API : http://personal.cis.strath.ac.uk/~luke/SPREE/Bots/API/17
    83. 83. SPREE Contact18
    84. 84. SPREE Contact • SPREE Mailing Lists & Forums on Sourceforge18
    85. 85. SPREE Contact • SPREE Mailing Lists & Forums on Sourceforge • Contact me directly :18
    86. 86. SPREE Contact • SPREE Mailing Lists & Forums on Sourceforge • Contact me directly : ‣ luke@cis.strath.ac.uk18
    87. 87. SPREE Contact • SPREE Mailing Lists & Forums on Sourceforge • Contact me directly : ‣ luke@cis.strath.ac.uk ‣ @LukeD on Twitter18

    ×