1. Introduction
My Work
Future Work and Questions
Machine Learning applied to Go
Dmitry Kamenetsky
Supervisor: Nic Schraudolph
September 2006
Dmitry Kamenetsky Machine Learning applied to Go

2. Introduction
My Work
Future Work and Questions
1 Introduction
2 My Work
3 Future Work and Questions
Dmitry Kamenetsky Machine Learning applied to Go

3. Introduction
My Work
Future Work and Questions
Aim
Create a clean and accessible database of Go games
Use Machine Learning to learn from expert games
Use the learned knowledge to build a powerful Go
program
Dmitry Kamenetsky Machine Learning applied to Go

4. Introduction
My Work
Future Work and Questions
Aim
Create a clean and accessible database of Go games
Use Machine Learning to learn from expert games
Use the learned knowledge to build a powerful Go
program
Dmitry Kamenetsky Machine Learning applied to Go

5. Introduction
My Work
Future Work and Questions
Aim
Create a clean and accessible database of Go games
Use Machine Learning to learn from expert games
Use the learned knowledge to build a powerful Go
program
Dmitry Kamenetsky Machine Learning applied to Go

6. Introduction
My Work
Future Work and Questions
What is Go?
Two-player zero-sum board game
Originated in ancient China. Today very popular in China,
Japan and Korea
19x19 grid
Simple rules, but very complex strategy
Dmitry Kamenetsky Machine Learning applied to Go

7. Introduction
My Work
Future Work and Questions
Go example
Dmitry Kamenetsky Machine Learning applied to Go

8. Introduction
My Work
Future Work and Questions
Go example
Dmitry Kamenetsky Machine Learning applied to Go

9. Introduction
My Work
Future Work and Questions
Go example
Dmitry Kamenetsky Machine Learning applied to Go

10. Introduction
My Work
Future Work and Questions
Go example
Dmitry Kamenetsky Machine Learning applied to Go

11. Introduction
My Work
Future Work and Questions
Go example
Dmitry Kamenetsky Machine Learning applied to Go

12. Introduction
My Work
Future Work and Questions
Go example
Dmitry Kamenetsky Machine Learning applied to Go

13. Introduction
My Work
Future Work and Questions
Go example
Dmitry Kamenetsky Machine Learning applied to Go

14. Introduction
My Work
Future Work and Questions
Go example
Dmitry Kamenetsky Machine Learning applied to Go

15. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

16. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

17. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

18. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

19. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

20. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

21. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

22. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

23. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

24. Introduction
My Work
Future Work and Questions
Why is Go interesting for Computer Research?
Large state space
Branching factor of about 200
10172 legal board positions
10360 game tree nodes
Position evaluation is difficult
Hard to judge strength of groups statically
Requires visual pattern recognition
Stones have local as well as long-distance effects
Best programs are much weaker than humans
Dmitry Kamenetsky Machine Learning applied to Go

25. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
My Work
Influence Function
Database
Game scorer
Dmitry Kamenetsky Machine Learning applied to Go

26. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Influence Function
Stones radiate influence to neighboring intersections
Crude measure of group’s strength - useful if can be
computed quickly
Resistor Grid idea
Clamp stones: Black = +1V, White = -1V
Current (influence) spreads to neighbors via resistors (grid
lines)
Fast and accurate
Dmitry Kamenetsky Machine Learning applied to Go

27. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Influence Function
Stones radiate influence to neighboring intersections
Crude measure of group’s strength - useful if can be
computed quickly
Resistor Grid idea
Clamp stones: Black = +1V, White = -1V
Current (influence) spreads to neighbors via resistors (grid
lines)
Fast and accurate
Dmitry Kamenetsky Machine Learning applied to Go

28. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Influence Function
Stones radiate influence to neighboring intersections
Crude measure of group’s strength - useful if can be
computed quickly
Resistor Grid idea
Clamp stones: Black = +1V, White = -1V
Current (influence) spreads to neighbors via resistors (grid
lines)
Fast and accurate
Dmitry Kamenetsky Machine Learning applied to Go

29. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Influence Function
Stones radiate influence to neighboring intersections
Crude measure of group’s strength - useful if can be
computed quickly
Resistor Grid idea
Clamp stones: Black = +1V, White = -1V
Current (influence) spreads to neighbors via resistors (grid
lines)
Fast and accurate
Dmitry Kamenetsky Machine Learning applied to Go

30. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Influence Function
Stones radiate influence to neighboring intersections
Crude measure of group’s strength - useful if can be
computed quickly
Resistor Grid idea
Clamp stones: Black = +1V, White = -1V
Current (influence) spreads to neighbors via resistors (grid
lines)
Fast and accurate
Dmitry Kamenetsky Machine Learning applied to Go

31. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Influence Function
Stones radiate influence to neighboring intersections
Crude measure of group’s strength - useful if can be
computed quickly
Resistor Grid idea
Clamp stones: Black = +1V, White = -1V
Current (influence) spreads to neighbors via resistors (grid
lines)
Fast and accurate
Dmitry Kamenetsky Machine Learning applied to Go

32. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Influence example
Dmitry Kamenetsky Machine Learning applied to Go

33. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Database
9 million Go games from various sources
MySQL
Problems
70GB of raw text
Illegal and duplicate games
Inconsistent game properties
MySQL not installed
Dmitry Kamenetsky Machine Learning applied to Go

34. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Database
9 million Go games from various sources
MySQL
Problems
70GB of raw text
Illegal and duplicate games
Inconsistent game properties
MySQL not installed
Dmitry Kamenetsky Machine Learning applied to Go

35. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Database
9 million Go games from various sources
MySQL
Problems
70GB of raw text
Illegal and duplicate games
Inconsistent game properties
MySQL not installed
Dmitry Kamenetsky Machine Learning applied to Go

36. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Database
9 million Go games from various sources
MySQL
Problems
70GB of raw text
Illegal and duplicate games
Inconsistent game properties
MySQL not installed
Dmitry Kamenetsky Machine Learning applied to Go

37. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Database
9 million Go games from various sources
MySQL
Problems
70GB of raw text
Illegal and duplicate games
Inconsistent game properties
MySQL not installed
Dmitry Kamenetsky Machine Learning applied to Go

38. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Database
9 million Go games from various sources
MySQL
Problems
70GB of raw text
Illegal and duplicate games
Inconsistent game properties
MySQL not installed
Dmitry Kamenetsky Machine Learning applied to Go

39. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Database
9 million Go games from various sources
MySQL
Problems
70GB of raw text
Illegal and duplicate games
Inconsistent game properties
MySQL not installed
Dmitry Kamenetsky Machine Learning applied to Go

40. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Cooperative Scorer
Important to determine final territory
Key ideas:
Sufficient to know the list of dead stones
Players cooperate. Make moves that do not affect score
Use simple heuristics - fast!
Result voted from 11 simulations. Compare to GnuGo’s
list and original Game Record
Dmitry Kamenetsky Machine Learning applied to Go

41. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Cooperative Scorer
Important to determine final territory
Key ideas:
Sufficient to know the list of dead stones
Players cooperate. Make moves that do not affect score
Use simple heuristics - fast!
Result voted from 11 simulations. Compare to GnuGo’s
list and original Game Record
Dmitry Kamenetsky Machine Learning applied to Go

42. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Cooperative Scorer
Important to determine final territory
Key ideas:
Sufficient to know the list of dead stones
Players cooperate. Make moves that do not affect score
Use simple heuristics - fast!
Result voted from 11 simulations. Compare to GnuGo’s
list and original Game Record
Dmitry Kamenetsky Machine Learning applied to Go

43. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Cooperative Scorer
Important to determine final territory
Key ideas:
Sufficient to know the list of dead stones
Players cooperate. Make moves that do not affect score
Use simple heuristics - fast!
Result voted from 11 simulations. Compare to GnuGo’s
list and original Game Record
Dmitry Kamenetsky Machine Learning applied to Go

44. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Cooperative Scorer
Important to determine final territory
Key ideas:
Sufficient to know the list of dead stones
Players cooperate. Make moves that do not affect score
Use simple heuristics - fast!
Result voted from 11 simulations. Compare to GnuGo’s
list and original Game Record
Dmitry Kamenetsky Machine Learning applied to Go

45. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Cooperative Scorer
Important to determine final territory
Key ideas:
Sufficient to know the list of dead stones
Players cooperate. Make moves that do not affect score
Use simple heuristics - fast!
Result voted from 11 simulations. Compare to GnuGo’s
list and original Game Record
Dmitry Kamenetsky Machine Learning applied to Go

46. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Scorer Results
Verified on Martin Mueller’s 19x19 collection of 31 games
Tested on Erik van der Werf’s 9x9 collection of 18K games
96.23% agreement. Comparable to the best classifiers
Our collection of games
3,500,000 games with territory. Varying board sizes and
level of completeness
All GnuGo Game Record Coop. None
Error 3.15% 4.12% 5.66% 10.84% 76.24%
Cooperative: median time = 0.30 s, worst time = 5.12 s
GnuGo: median time = 1.70 s, worst time = 446.69 s
Dmitry Kamenetsky Machine Learning applied to Go

47. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Scorer Results
Verified on Martin Mueller’s 19x19 collection of 31 games
Tested on Erik van der Werf’s 9x9 collection of 18K games
96.23% agreement. Comparable to the best classifiers
Our collection of games
3,500,000 games with territory. Varying board sizes and
level of completeness
All GnuGo Game Record Coop. None
Error 3.15% 4.12% 5.66% 10.84% 76.24%
Cooperative: median time = 0.30 s, worst time = 5.12 s
GnuGo: median time = 1.70 s, worst time = 446.69 s
Dmitry Kamenetsky Machine Learning applied to Go

48. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Scorer Results
Verified on Martin Mueller’s 19x19 collection of 31 games
Tested on Erik van der Werf’s 9x9 collection of 18K games
96.23% agreement. Comparable to the best classifiers
Our collection of games
3,500,000 games with territory. Varying board sizes and
level of completeness
All GnuGo Game Record Coop. None
Error 3.15% 4.12% 5.66% 10.84% 76.24%
Cooperative: median time = 0.30 s, worst time = 5.12 s
GnuGo: median time = 1.70 s, worst time = 446.69 s
Dmitry Kamenetsky Machine Learning applied to Go

49. Introduction Influence Function
My Work Database
Future Work and Questions Game Scorer
Scorer Results
Verified on Martin Mueller’s 19x19 collection of 31 games
Tested on Erik van der Werf’s 9x9 collection of 18K games
96.23% agreement. Comparable to the best classifiers
Our collection of games
3,500,000 games with territory. Varying board sizes and
level of completeness
All GnuGo Game Record Coop. None
Error 3.15% 4.12% 5.66% 10.84% 76.24%
Cooperative: median time = 0.30 s, worst time = 5.12 s
GnuGo: median time = 1.70 s, worst time = 446.69 s
Dmitry Kamenetsky Machine Learning applied to Go

50. Introduction
My Work
Future Work and Questions
Future Work
Conditional Random Fields (CRF)
Grid based model. General graph model
Predict: final territory, next move
Cooperative Scorer
Improve accuracy
Compare to other methods
Incorporate into a Monte Carlo program
Complete MySQL Database
Dmitry Kamenetsky Machine Learning applied to Go

51. Introduction
My Work
Future Work and Questions
Future Work
Conditional Random Fields (CRF)
Grid based model. General graph model
Predict: final territory, next move
Cooperative Scorer
Improve accuracy
Compare to other methods
Incorporate into a Monte Carlo program
Complete MySQL Database
Dmitry Kamenetsky Machine Learning applied to Go

52. Introduction
My Work
Future Work and Questions
Future Work
Conditional Random Fields (CRF)
Grid based model. General graph model
Predict: final territory, next move
Cooperative Scorer
Improve accuracy
Compare to other methods
Incorporate into a Monte Carlo program
Complete MySQL Database
Dmitry Kamenetsky Machine Learning applied to Go

53. Introduction
My Work
Future Work and Questions
Future Work
Conditional Random Fields (CRF)
Grid based model. General graph model
Predict: final territory, next move
Cooperative Scorer
Improve accuracy
Compare to other methods
Incorporate into a Monte Carlo program
Complete MySQL Database
Dmitry Kamenetsky Machine Learning applied to Go

54. Introduction
My Work
Future Work and Questions
Future Work
Conditional Random Fields (CRF)
Grid based model. General graph model
Predict: final territory, next move
Cooperative Scorer
Improve accuracy
Compare to other methods
Incorporate into a Monte Carlo program
Complete MySQL Database
Dmitry Kamenetsky Machine Learning applied to Go

55. Introduction
My Work
Future Work and Questions
Future Work
Conditional Random Fields (CRF)
Grid based model. General graph model
Predict: final territory, next move
Cooperative Scorer
Improve accuracy
Compare to other methods
Incorporate into a Monte Carlo program
Complete MySQL Database
Dmitry Kamenetsky Machine Learning applied to Go

56. Introduction
My Work
Future Work and Questions
Future Work
Conditional Random Fields (CRF)
Grid based model. General graph model
Predict: final territory, next move
Cooperative Scorer
Improve accuracy
Compare to other methods
Incorporate into a Monte Carlo program
Complete MySQL Database
Dmitry Kamenetsky Machine Learning applied to Go

57. Introduction
My Work
Future Work and Questions
Future Work
Conditional Random Fields (CRF)
Grid based model. General graph model
Predict: final territory, next move
Cooperative Scorer
Improve accuracy
Compare to other methods
Incorporate into a Monte Carlo program
Complete MySQL Database
Dmitry Kamenetsky Machine Learning applied to Go

58. Introduction
My Work
Future Work and Questions
Questions?
Its time for me to GO!
Dmitry Kamenetsky Machine Learning applied to Go