This document discusses using Meta-Monte Carlo Tree Search (Meta-MCTS) to build an opening book for 7x7 Go. Meta-MCTS improved its play against a sparring partner that incorporated human variations. While Meta-MCTS won all games as black and white against professionals, humans found at least one variation where it did not play correctly. The document concludes that Meta-MCTS performed well but incorporating human data helped, and exactly solving 7x7 Go would require immense work collecting and solving all leaf variations.

2. - Go is not solved beyond 6x6
- We build an opening book for 7x7 for
approximately solving 7x7 Go..
Widely believed: perfect play is a draw with Komi 9.

3. Our tools
1) Monte-Carlo Tree Search
2) Meta-Monte-Carlo Tree Search
3) Senseis' partial solution
(using in particular Davies' work)

4. Monte-Carlo Tree Search
Coulom 2006,
Kocsis-Szepesvari 2006.
= combining tree search
and Monte-Carlo evaluation

5. UCT (Upper Confidence Trees)
Coulom (06)
Chaslot, Saito & Bouzy (06)
Kocsis Szepesvari (06)

6. UCT

7. UCT

8. UCT

9. UCT

10. UCT
Kocsis & Szepesvari (06)

11. Exploitation ...

12. Exploitation ...
SCORE =
5/7
+ k.sqrt( log(10)/7 )

13. Exploitation ...
SCORE =
5/7
+ k.sqrt( log(10)/7 )

14. Exploitation ...
SCORE =
5/7
+ k.sqrt( log(10)/7 )

15. ... or exploration ?
SCORE =
0/2
+ k.sqrt( log(10)/2 )

16. Our tools
1) Monte-Carlo Tree Search
2) Meta-Monte-Carlo Tree Search
3) Senseis' partial solution
(using in particular Davies' work)

17. Meta-Monte-Carlo
Tree Search
= Monte-Carlo Tree Search
with Monte-Carlo replaced by
MCTS

18. Meta-Monte-Carlo
Tree Search
I.e.:
MCTS
= MC play-outs + Tree Search
Meta-MCTS
= MCTS play-outs + Tree Search
Meta-Meta-MCTS
= Meta-MCTS play-outs + TreeSearch
...

19. Meta-Monte-Carlo
Tree Search
I.e.:
MCTS
= MC play-outs + Tree Search
Meta-MCTS
= MCTS play-outs + Tree Search
Meta-Meta-MCTS
= Meta-MCTS play-outs + TreeSearch
...

20. Our tools
1) Monte-Carlo Tree Search
2) Meta-Monte-Carlo Tree Search
3) Senseis' partial solution
(using in particular Davies' work)

21. A variation which is not in Senseis' file.
Left: black E3 should be black E5.
Right: corrected version.

22. EXPERIMENTAL
RESULTS

23. Meta-MCTS learns against a
MCTS sparring partner.
We introduce Senseis' variations
into this sparring partner
during the Meta-MCTS run.

24. Learning curve of black by Meta-MCTS:
- X-axis = log2(number of playouts)
- Y-axis = moving average (window size 55)
of winning rate in playouts
Playouts = MCTS (it's Meta-MCTS)
Komi = 8.5 (winning with komi 8.5
ensures a draw with komi 9)

25. Learning curve of white by Meta-MCTS:
- X-axis = log2(number of playouts)
- Y-axis = moving average (window size 55)
of winning rate in playouts
Playouts = MCTS (it's Meta-MCTS)
Komi = 9.5 (winning with komi 9.5
ensures a draw with komi 9)

26. Decreasing points in the curve = introduction
of Senseis variations in the opponent.
Conclusion = the algorithm did not find alone
all these variations ==> human needed.

27. Games
Against
pros.

28. MoGoTW is black.

29. MoGoTW is white

30. With komi 9.5, MoGoTW won everything as White.
With komi 8.5, MoGoTW won everything as Black.
Exciting!
Were all MoGoTW's moves perfect ?

31. With komi 9.5, MoGoTW won everything as White.
With komi 8.5, MoGoTW won everything as Black.
Exciting!
Were all MoGoTW's moves perfect ?
No :-(
In one game (at least) the human might
have won.

32. Left: this game was won by MoGoTW as black.
Chun-Yen Lin (2P) made a mistake.
Right: how Chun-Yen Lin (2P) might have won the game.

33. So, still at least one variation on
which the bot does not play correctly.
We did not introduce manually a correction,
but we introduced the variation played by the pro
in the sparring partner.
We see if the bot can find a solution by itself.

34. Learning curve as black, after introducing the
dangerous variation in the sparring partner.
Time still logarithmic.

35. CONCLUSIONS
We used Meta-MCTS for building
an opening book for 7x7 Go.
We are not aware of
remaining bad moves, which does
not mean there's no more bad move.

36. Meta-MCTS did a good job by itself,
but human inputs
( = Senseis + games against
pros) have been helpful.
Towards exact solving ?
= collecting all leafs of the OB
+ solving all of them...
= huge work.

37. Other conclusions:
- 7x7 can be very hard, even for pros
(pros made mistakes).
- MCTS alone is not enough for
very strong play in 7x7