Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Application of Monte Carlo Tree Search in a Fighting Game AI (GCCE 2016)

757 views

Published on

This are the presentation slides for the following paper:
Shubu Yoshida, Makoto Ishihara, Taichi Miyazaki, Yuto Nakagawa, Tomohiro Harada, and Ruck Thawonmas, "Application of Monte-Carlo Tree Search in a Fighting Game AI," accepted for presentation at the 5th IEEE Global Conference on Consumer Electronics (GCCE 2016), Kyoto, Japan, Oct. 11-14, 2016.

Published in: Technology
  • Be the first to comment

Application of Monte Carlo Tree Search in a Fighting Game AI (GCCE 2016)

  1. 1. Application of Monte- Carlo Tree Search in a Fighting Game AI Shubu Yoshida, Makoto Ishihara, Taichi Miyazaki, Yuto Nakagawa, Tomohiro Harada, and Ruck Thawonmas Intelligent Computer Entertainment Laboratory Ritsumeikan University
  2. 2. Outline 1.Background of this research 2.Monte-Carlo Tree Search 3.Monte-Carlo Tree Search for a Fighting Game 4.Experimental Environment 5.Experimental Method 6.Result 7.Competition result in 2016 8.Conclusion
  3. 3. Background (1/2) A Fighting Game AI Competition is held every year [1] High-ranking AIs = Rule-based (until 2015) Rule-based : a same action in a same situation Human player can easily predict the AI’s action patterns and outsmart it [1] http://www.ice.ci.ritsumei.ac.jp/~ftgaic/
  4. 4. Background (2/2)  Apply the Monte-Carlo Tree Search (MCTS) to a fighting game AI  Decides a next own action by stochastic simulations  Already successful in many games [2][3] We evaluate the effectiveness of MCTS on a fighting game [2] S. Gelly, et al. ”The Grand Challenge of Computer Go: Monte Carlo Tree Search and Extensions”, Communications of the ACM, Vol. 55, No. 3, pp. 106-113, 2012. [3] N. Ikehata and T. Ito. ”Monte-carlo tree search in ms. pac-man”. In Computational Intelligence and Games (CIG), 2011 IEEE Conference on, pp. 39-46, 2011
  5. 5. Monte-Carlo Tree Search (1/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  6. 6. Monte-Carlo Tree Search (2/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  7. 7. Formula of UCB1 ・ 𝑋𝑖 : the value of an average reward ・𝐶 : The balance parameter ・𝑁𝑖 𝑝 : The total number of times the parent node of node 𝑖 has been visited ・𝑁𝑖 : The total number of times node 𝑖 has been visited 𝑈𝐶𝐵1𝑖 = 𝑋𝑖 + 𝐶 2 ln 𝑁𝑖 𝑝 𝑁𝑖 Preferentially select a child node that has been visited less The evaluation valueExploitation Exploration
  8. 8. Monte-Carlo Tree Search (3/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  9. 9. Monte-Carlo Tree Search (4/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  10. 10. Monte-Carlo Tree Search (5/5) selection simulation backpropagation repeat until the set time has elapsed expansion
  11. 11. MCTS for a Fighting Game (1/2) 𝑈𝐶𝐵1𝑖 = 𝑋𝑖 + 𝐶 2 ln 𝑁𝑖 𝑝 𝑁𝑖 𝑋𝑖 = 1 𝑁𝑖 𝑗=1 𝑁 𝑖 𝑒𝑣𝑎𝑙𝑗 𝑒𝑣𝑎𝑙𝑗 = (𝑎𝑓𝑡𝑒𝑟𝐻𝑃𝑗 𝑚𝑦 − 𝑏𝑒𝑓𝑜𝑟𝑒𝐻𝑃𝑗 𝑚𝑦 ) −(𝑎𝑓𝑡𝑒𝑟𝐻𝑃𝑗 𝑜𝑝𝑝 − 𝑏𝑒𝑓𝑜𝑟𝑒𝐻𝑃𝑗 𝑜𝑝𝑝 )
  12. 12. MCTS for a Fighting Game (2/2) ・・・ Expansion normal fighting game ・・・ ・・・・・ Simulation
  13. 13. Experimental Environment FightingICE Used as the platform of international fighting game AI competition 1 game : 3 rounds -1 round : 60 second 𝑚𝑦𝑆𝑐𝑜𝑟𝑒 = 𝑜𝑝𝑝𝐻𝑃 𝑚𝑦𝐻𝑃+𝑜𝑝𝑝𝐻𝑃 × 1000 Response time : 16.67ms
  14. 14. Experimental Method MCTSAI(AI applying MCTS) vs high ranking 5 AIs of 2015 tournament 5 AIs : Rule-based 100 games (50 games each side) TABLE I THE PARAMETERS USED IN THE EXPERIMENTS Notations Meanings Values C Balance parameter 3 Threshold of the number of visits 10 Threshold of the depth of tree 2 The number of simulations 60 frames 𝑁 𝑚𝑎𝑥 𝐷 𝑚𝑎𝑥 𝑇𝑠𝑖𝑚
  15. 15. Result (1/5) 0 100 200 300 400 500 600 700 800 Machete Ni1mir4ri Jay_Bot RatioBot AI128200 Score vs AI names Fig. 1. The average scores against high ranking 5 AIs of 2015 tournament
  16. 16. Result (2/5) 0 100 200 300 400 500 600 700 800 Machete Ni1mir4ri Jay_Bot RatioBot AI128200 Score vs AI names Fig. 1. The average scores against high ranking 5 AIs of 2015 tournament
  17. 17. Result (3/5) P1 : MCTSAI P2 : RatioBot
  18. 18. Result (4/5) 0 100 200 300 400 500 600 700 800 Machete Ni1mir4ri Jay_Bot RatioBot AI128200 Score vs AI names Fig. 1. The average scores against high ranking 5 AIs of 2015 tournament
  19. 19. Result (5/5) P1 : MCTSAI P2 : Machete
  20. 20. Competition result in 2016 Orange 1st Blue 2nd Green 3rd Total Rank RANK BANZAI 11 DragonSurvivor 12 iaTest 7 IchibanChan 9 JayBot2016 5 KeepYourDistanceBot 10 MctsAi 3 MrAsh 4 Poring 8 Ranezi 2 Snorkel 13 Thunder01 1 Tomatensimulator 6 Triump 14
  21. 21. Conclusion Applied MCTS to fighting game AI Showed that MCTS in fighting game AI is effective Future work In fighting game, random simulation of the enemy behavior is not effective Predict the behavior of the enemy and use this information in simulation
  22. 22. Thank you for listening

×