This presentation was given at AISB 2011 and introduces the Strathclyde Poker Research Environment (SPREE) an open tool for Poker research. Available from Sourceforge @ https://sourceforge.net/projects/spree-poker/

1. SPREE
The Strathclyde Poker Research Environment
L. Dicken, N. Johnstone, J. Levine, P. Rodgers
Strathclyde AI and Games Research Group

2. Poker at SAIG
2

3. Poker at SAIG
• Evolutionary Algorithms applied to Tournament Poker.
(Carter & Levine)[1]
[1] - An Investigation into Tournament Poker Strategy using Evolutionary Algorithms, Carter & Levine,
Proceedings of CIG2007
2

4. Poker at SAIG
• Evolutionary Algorithms applied to Tournament Poker.
(Carter & Levine)[1]
• Co-Evolution for Counter Strategies in Texas Hold ‘Em
(Thomson, Levine & Wotherspoon)[2]
[1] - An Investigation into Tournament Poker Strategy using Evolutionary Algorithms, Carter & Levine,
Proceedings of CIG2007
[2] - Evolution of Counter-Strategies: Application of Co-evolution to Texas Hold'em Poker, Thomson,
Levine & Wotherspoon, Proceedings of CIG2008
2

5. Poker at SAIG
• Evolutionary Algorithms applied to Tournament Poker.
(Carter & Levine)[1]
• Co-Evolution for Counter Strategies in Texas Hold ‘Em
(Thomson, Levine & Wotherspoon)[2]
• Monte Carlo Simulation for Ten Player Texas Hold ‘Em
(Birnie, Dicken & Levine)
[1] - An Investigation into Tournament Poker Strategy using Evolutionary Algorithms, Carter & Levine,
Proceedings of CIG2007
[2] - Evolution of Counter-Strategies: Application of Co-evolution to Texas Hold'em Poker, Thomson,
Levine & Wotherspoon, Proceedings of CIG2008
2

6. Poker at SAIG
• Evolutionary Algorithms applied to Tournament Poker.
(Carter & Levine)[1]
• Co-Evolution for Counter Strategies in Texas Hold ‘Em
(Thomson, Levine & Wotherspoon)[2]
• Monte Carlo Simulation for Ten Player Texas Hold ‘Em
(Birnie, Dicken & Levine)
• Machine Learning for Player Archetyping (Birnie, Dicken &
Levine)
[1] - An Investigation into Tournament Poker Strategy using Evolutionary Algorithms, Carter & Levine,
Proceedings of CIG2007
[2] - Evolution of Counter-Strategies: Application of Co-evolution to Texas Hold'em Poker, Thomson,
Levine & Wotherspoon, Proceedings of CIG2008
2

7. Incomplete Information
3

8. Incomplete Information
• In the Machine Learning project we found a key weakness in
existing Poker research.
3

9. Incomplete Information
• In the Machine Learning project we found a key weakness in
existing Poker research.
• Based primarily on data gathered from commercial online
casinos through play.
3

10. Incomplete Information
• In the Machine Learning project we found a key weakness in
existing Poker research.
• Based primarily on data gathered from commercial online
casinos through play.
• Significant information remains hidden, e.g. folded hands.
3

11. Incomplete Information
• In the Machine Learning project we found a key weakness in
existing Poker research.
• Based primarily on data gathered from commercial online
casinos through play.
• Significant information remains hidden, e.g. folded hands.
‣ “Playable” hands are the only positive cases shown
3

12. Incomplete Information
• In the Machine Learning project we found a key weakness in
existing Poker research.
• Based primarily on data gathered from commercial online
casinos through play.
• Significant information remains hidden, e.g. folded hands.
‣ “Playable” hands are the only positive cases shown
‣ Infer from this which hands are “Unplayable”
3

13. Incomplete Information
• In the Machine Learning project we found a key weakness in
existing Poker research.
• Based primarily on data gathered from commercial online
casinos through play.
• Significant information remains hidden, e.g. folded hands.
‣ “Playable” hands are the only positive cases shown
‣ Infer from this which hands are “Unplayable”
‣ Compensation through large datasets (~1,000,000 games)
3

14. Incomplete Information
• In the Machine Learning project we found a key weakness in
existing Poker research.
• Based primarily on data gathered from commercial online
casinos through play.
• Significant information remains hidden, e.g. folded hands.
‣ “Playable” hands are the only positive cases shown
‣ Infer from this which hands are “Unplayable”
‣ Compensation through large datasets (~1,000,000 games)
• Sub-optimal approach based on availability of data
3

15. Reinventing the Wheel
4

16. Reinventing the Wheel
• Huge portions of these projects got spent implementing
Poker rules.
4

17. Reinventing the Wheel
• Huge portions of these projects got spent implementing
Poker rules.
• Wasted effort
4

18. Reinventing the Wheel
• Huge portions of these projects got spent implementing
Poker rules.
• Wasted effort
• Fragmentation > Potentially flawed implementations
4

19. Reinventing the Wheel
• Huge portions of these projects got spent implementing
Poker rules.
• Wasted effort
• Fragmentation > Potentially flawed implementations
• Skews results
4

20. Two Birds, One Stone
5

21. Two Birds, One Stone
• We need better data, which we can’t get from online casinos
5

22. Two Birds, One Stone
• We need better data, which we can’t get from online casinos
‣ But only online casinos have the kind of volume of data
we need.
5

23. Two Birds, One Stone
• We need better data, which we can’t get from online casinos
‣ But only online casinos have the kind of volume of data
we need.
• We need a standard testbed in which to conduct Poker
research.
5

24. Two Birds, One Stone
• We need better data, which we can’t get from online casinos
‣ But only online casinos have the kind of volume of data
we need.
• We need a standard testbed in which to conduct Poker
research.
‣ Save on development time, spend more time on the AI
5

25. Two Birds, One Stone
• We need better data, which we can’t get from online casinos
‣ But only online casinos have the kind of volume of data
we need.
• We need a standard testbed in which to conduct Poker
research.
‣ Save on development time, spend more time on the AI
‣ Standardise our experimental methodology, more
comparable results.
5

26. Introducing SPREE
6

27. Introducing SPREE
• The Strathclyde Poker Research Environment provides an
open framework for Poker experimentation.
6

28. Introducing SPREE
• The Strathclyde Poker Research Environment provides an
open framework for Poker experimentation.
• Client/Server structure allows AI players to be easily created
6

29. Introducing SPREE
• The Strathclyde Poker Research Environment provides an
open framework for Poker experimentation.
• Client/Server structure allows AI players to be easily created
• Pre-built GUI client means players can play against AI or each
other.
6

30. Introducing SPREE
• The Strathclyde Poker Research Environment provides an
open framework for Poker experimentation.
• Client/Server structure allows AI players to be easily created
• Pre-built GUI client means players can play against AI or each
other.
• Open server means we can get complete data on all games
being played.
6

31. Components
7

32. SPREE Server
8

33. SPREE Server
• Controls the flow of the game.
8

34. SPREE Server
• Controls the flow of the game.
• Stores certain state information (e.g. Player bankroll, Active
tables) in a database - non-volatile.
8

35. SPREE Server
• Controls the flow of the game.
• Stores certain state information (e.g. Player bankroll, Active
tables) in a database - non-volatile.
• Broken into two main components : Casino and Table
8

36. SPREE Server
• Controls the flow of the game.
• Stores certain state information (e.g. Player bankroll, Active
tables) in a database - non-volatile.
• Broken into two main components : Casino and Table
• Casino is the high level - logins, providing information on
active tables etc.
8

37. SPREE Server
• Controls the flow of the game.
• Stores certain state information (e.g. Player bankroll, Active
tables) in a database - non-volatile.
• Broken into two main components : Casino and Table
• Casino is the high level - logins, providing information on
active tables etc.
• The Table server runs the game according to the
specification it has been given.
8

38. Game Specifications
9

39. Game Specifications
• Focus is currently on Texas Hold ‘Em
9

40. Game Specifications
• Focus is currently on Texas Hold ‘Em
• Rules of the game specified in XML
9

41. Game Specifications
• Focus is currently on Texas Hold ‘Em
• Rules of the game specified in XML
• Modular component for the server, can swap to
other game modes.
9

42. Game Specifications
• Focus is currently on Texas Hold ‘Em
• Rules of the game specified in XML
• Modular component for the server, can swap to
other game modes.
‣ Note that the current client is specific to Texas Hold ‘Em
9

43. Collecting Data
10

44. Collecting Data
• Two types of data being logged
10

45. Collecting Data
• Two types of data being logged
‣ “Recording” : dump GameState messages that have been
generated through the game.
10

46. Collecting Data
• Two types of data being logged
‣ “Recording” : dump GameState messages that have been
generated through the game.
‣ “Hand History” : a human-readable export of the game,
compatible with Poker stats software e.g. Poker Tracker
10

47. SPREE Client
11

48. SPREE Client
• Java implementation modelled on existing commercial casino
interfaces.
11

49. SPREE Client
• Java implementation modelled on existing commercial casino
interfaces.
• Connects to Casino Server, logs in and retrieves data about
tables working.
11

50. SPREE Client
• Java implementation modelled on existing commercial casino
interfaces.
• Connects to Casino Server, logs in and retrieves data about
tables working.
• Then connects to Table Server and plays Poker.
11

51. SPREE Client
• Java implementation modelled on existing commercial casino
interfaces.
• Connects to Casino Server, logs in and retrieves data about
tables working.
• Then connects to Table Server and plays Poker.
• Principal difference from commercial implementations :
11

52. SPREE Client
• Java implementation modelled on existing commercial casino
interfaces.
• Connects to Casino Server, logs in and retrieves data about
tables working.
• Then connects to Table Server and plays Poker.
• Principal difference from commercial implementations :
‣ No chat, want to avoid play devolving into Turing tests.
11

53. SPREE Client
12

54. SPREE Replayer
13

55. SPREE Replayer
• The Replayer is an offline GUI “client”
13

56. SPREE Replayer
• The Replayer is an offline GUI “client”
• Uses data logged by the server to visualise games of Poker
13

57. SPREE Replayer
• The Replayer is an offline GUI “client”
• Uses data logged by the server to visualise games of Poker
• Gives developers the ability to dig into games and
understand the behaviour of their bots.
13

58. SPREE Replayer
• The Replayer is an offline GUI “client”
• Uses data logged by the server to visualise games of Poker
• Gives developers the ability to dig into games and
understand the behaviour of their bots.
‣ Or social scientists the ability to analyse the behaviour of
players.
13

59. SPREE Admin
14

60. SPREE Admin
• Dedicated administration interface.
14

61. SPREE Admin
• Dedicated administration interface.
• Allows moderators to control the server, register
new player accounts and set up tables for play.
14

62. SPREE Admin
• Dedicated administration interface.
• Allows moderators to control the server, register
new player accounts and set up tables for play.
• Also allows for full-information observation of
games in progress.
14

63. SPREE Bots
15

64. SPREE Bots
• Bots can be implemented in any language provided they
conform to the communication spec.
15

65. SPREE Bots
• Bots can be implemented in any language provided they
conform to the communication spec.
• A framework has been created in Java. Implementing a bot is
simply implementing a single method.
15

66. SPREE Bots
• Bots can be implemented in any language provided they
conform to the communication spec.
• A framework has been created in Java. Implementing a bot is
simply implementing a single method.
• GameState messages update a TableModel which can be
queried by the AI routine.
15

67. SPREE Bots
• Bots can be implemented in any language provided they
conform to the communication spec.
• A framework has been created in Java. Implementing a bot is
simply implementing a single method.
• GameState messages update a TableModel which can be
queried by the AI routine.
• “makeMove” method is triggered when the server asks the
bot for an action, options are presented to the method
15

68. Future Projects
16

69. Future Projects
• Revisiting the Monte Carlo and Machine Learning projects
previously mentioned within the context of SPREE.
16

70. Future Projects
• Revisiting the Monte Carlo and Machine Learning projects
previously mentioned within the context of SPREE.
• Next project I’ll be supervising will be to develop some
standard bots to package with future versions of the release.
16

71. Future Projects
• Revisiting the Monte Carlo and Machine Learning projects
previously mentioned within the context of SPREE.
• Next project I’ll be supervising will be to develop some
standard bots to package with future versions of the release.
• Scope for various experiments not just for AI but also in
sociology.
16

72. Future Projects
• Revisiting the Monte Carlo and Machine Learning projects
previously mentioned within the context of SPREE.
• Next project I’ll be supervising will be to develop some
standard bots to package with future versions of the release.
• Scope for various experiments not just for AI but also in
sociology.
‣ How does value of wager affect behaviour.
16

73. Future Projects
• Revisiting the Monte Carlo and Machine Learning projects
previously mentioned within the context of SPREE.
• Next project I’ll be supervising will be to develop some
standard bots to package with future versions of the release.
• Scope for various experiments not just for AI but also in
sociology.
‣ How does value of wager affect behaviour.
‣ Quantification of “on tilt” pre-indicators.
16

74. Availability
17

75. Availability
• Initial public release TODAY!
17

76. Availability
• Initial public release TODAY!
• Licensed under GPL.
17

77. Availability
• Initial public release TODAY!
• Licensed under GPL.
• Available from Sourceforge at :
17

78. Availability
• Initial public release TODAY!
• Licensed under GPL.
• Available from Sourceforge at :
http://www.sourceforge.net/projects/spree-poker/
17

79. Availability
• Initial public release TODAY!
• Licensed under GPL.
• Available from Sourceforge at :
http://www.sourceforge.net/projects/spree-poker/
• Includes Server, Client, Tools and Bot Framework.
17

80. Availability
• Initial public release TODAY!
• Licensed under GPL.
• Available from Sourceforge at :
http://www.sourceforge.net/projects/spree-poker/
• Includes Server, Client, Tools and Bot Framework.
• Access to resources such as hand history, recordings via
Sourceforge.
17

81. Availability
• Initial public release TODAY!
• Licensed under GPL.
• Available from Sourceforge at :
http://www.sourceforge.net/projects/spree-poker/
• Includes Server, Client, Tools and Bot Framework.
• Access to resources such as hand history, recordings via
Sourceforge.
• Bot API :
17

82. Availability
• Initial public release TODAY!
• Licensed under GPL.
• Available from Sourceforge at :
http://www.sourceforge.net/projects/spree-poker/
• Includes Server, Client, Tools and Bot Framework.
• Access to resources such as hand history, recordings via
Sourceforge.
• Bot API :
http://personal.cis.strath.ac.uk/~luke/SPREE/Bots/API/
17

83. SPREE Contact
18

84. SPREE Contact
• SPREE Mailing Lists & Forums on Sourceforge
18

85. SPREE Contact
• SPREE Mailing Lists & Forums on Sourceforge
• Contact me directly :
18

86. SPREE Contact
• SPREE Mailing Lists & Forums on Sourceforge
• Contact me directly :
‣ luke@cis.strath.ac.uk
18

87. SPREE Contact
• SPREE Mailing Lists & Forums on Sourceforge
• Contact me directly :
‣ luke@cis.strath.ac.uk
‣ @LukeD on Twitter
18