1) Programs like Deep Blue and Stockfish can now defeat the best human chess players by using algorithms like minimax search and alpha-beta pruning to evaluate game trees up to dozens of ply deep. 2) Go programs initially struggled compared to chess programs due to the enormous game tree of Go. But programs like Crazy Stone and more recently AlphaGo used Monte Carlo tree search combined with deep neural networks to achieve superhuman performance at Go. 3) While algorithms like minimax and Monte Carlo tree search have achieved superhuman performance at perfect-information games like chess and Go, developing strong algorithms for imperfect-information games like poker remains an open research problem. Determinization is one approach but has limitations.

1. Programs that Play better than Us
Melvin Zhang
melvin@melvinzhang.net
@melvinzhangzy

2. https://en.wikipedia.org/wiki/File:ST Battle Chess.png

3. https://en.wikipedia.org/wiki/Deep Blue (chess computer)
Deep Blue (IBM, 1996)

4. http://afflictor.com/2012/09/11/chess-programs-regularly-play-at-good-amateur-level/

5. Game tree

6. Optimal play
Terminal
min player
max player

7. Optimal play
1 01 1 1Terminal
min player
max player

8. Optimal play
1 01 1 1
0
Terminal
min player
max player

9. Optimal play
1 01 1 1
0 1
Terminal
min player
max player

10. Optimal play
1 01 1 1
0 1
1
Terminal
min player
max player

11. Chess has about 1046
states!

12. Minimax algorithm
Cut-off
min player
max player

13. Minimax algorithm
.7 .1 .6 .9Cut-off
min player
max player

14. Minimax algorithm
.7 .1 .6 .9
.1
Cut-off
min player
max player

15. Minimax algorithm
.7 .1 .6 .9
.1 .6
Cut-off
min player
max player

16. Minimax algorithm
.7 .1 .6 .9
.1 .6
.6
Cut-off
min player
max player

17. https://stockfishchess.org/
Stockfish

18. https://tests.stockfishchess.org/
Testing AI changes is crucial

19. Value functions are hard!

20. http://mathworld.wolfram.com/Go.html

21. http://www.remi-coulom.fr/CrazyStone/
R´emi Coulom

22. http://www.wired.com/2014/05/the-world-of-computer-go/

23. Monte Carlo evaluations
Cut-off
min player
max player

24. Monte Carlo evaluations
Cut-off
min player
max player

25. Monte Carlo evaluations
Cut-off
min player
max player

26. Monte Carlo evaluations
Cut-off
min player
max player

27. Monte Carlo evaluations
Cut-off
min player
max player
.7

28. Monte Carlo Tree Search (MCTS)

29. by Google Deepmind
https://deepmind.com/research/alphago/

30. https://gogameguru.com/alphago-races-ahead-2-0-lee-sedol/

31. MCTS + Policy and value networks

32. http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html
MCTS + Policy and value networks

33. Some games have hidden information!

34. http://magic.wizards.com/en/events/coverage/gpsin15/father-son-2015-06-27

35. https://magarena.github.io

37. Determinization: choose a random instance of the
hidden information during simulation

38. Comparison of Minimax and MCTS
At 1s thinking time:
Minimax MCTS
1 0.88

39. Comparison of Minimax and MCTS
At 1s thinking time:
Minimax MCTS
1 0.88
At 4s thinking time:
Minimax MCTS
1 1.71

40. Open problems

41. MCTS is bad at tight tactical play.

42. MCTS plays badly when it is behind in the game.

43. Further readings

44. Further readings