Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AI Wolf Contest -Development of Game AI using Collective Intelligence-

2,361 views

Published on

Presentation document for Computer Game Workshop at IJCAI2016

Published in: Technology
  • Relationship guru Justin Sinclair reveals his secret tactics to help get your Ex back! Learn how ♣♣♣ http://ow.ly/f23I301xGAo
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

AI Wolf Contest -Development of Game AI using Collective Intelligence-

  1. 1. AI Wolf Contest -Development of Game AI using Collective Intelligence- Fujio Toriumiⅰ,, Michimasa Inabaⅱ Hirotaka Osawaⅲ Daisuke Katagamiⅳ , Kosuke Shinodaⅴ, Hitoshi Matsubaravi ⅰ The University of Tokyo 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan ⅱHiroshima City University, ⅲUniversity of Tsukuba, ⅳTokyo Polytechnic University, ⅴThe University of Electro-Communications , viFuture University Hakodate Computer Games Workshop at IJCAI 2016 Saturday July 9th 2016
  2. 2. Overview • Game Playing AI – Chess, Go etc... AIs already beat human • Focus on “Are you a Werewolf?” – Incomplete-information, communication game • AI Wolf Project – Develop AI which can play the werewolf with humans – Competition to develop AI via collective intelligence – A platform of a werewolf game for AI – Analyze the result of competition
  3. 3. Game Playing AI • The development of an artificial intelligence player that can play a game with a human – One of the main benchmarks in the AI field – chess, go, quiz, poker, and so on
  4. 4. Chess • DeepBlue – IBM RS/6000SP – 32 Processor Nodes+512 VLSI Processor • May, 1997 – vs Garry Kasparov – Result: Deep Blue–Kasparov: 3½–2½
  5. 5. Go • AlphaGo – Google Cloud Platform – 1202 CPU, 176 GPU – Deep Q-Network • March, 2016 – vs Lee Sedol – Result: AlphaGo–Lee Sedol: 4-1
  6. 6. Computer vs Human • 1997 Chess – Deep blue beats Garry Kasparov • 1997 Othello(Reversi) – Logistello beats Takeshi Murakami • 2004 War – Skynet beats humankind • 2015 Texas Hold'em – Two-player-limited is solved • 2016 Go – Alpha-Go beats Lee Sedol
  7. 7. Classification of games • Complete information / Incomplete information • Symbolic / Linguistic • Digital / Analog • Single Player / Multi Player • Cooperative / Non-cooperative • Deterministic / Indeterministic ...and so on
  8. 8. Information of Games • Complete Information Game – All information is observable by both players – Chess, Go, and so on – An AI system must only handle the condition of the board and does not need to determine a competitor’s thought processes. • Incomplete Information Game – Games including information that cannot be observed by other players – Porker, Bridge – Each players required to hold multiple world models for the other players’ actions.
  9. 9. Symbolic / Linguistic • Chess, Go and Poker are symbolic game – Easy to translate to Machine Readable Codes • When users play board and card games, they converse with other players. – Some games are actually conducted through conversations – Communication Game • Conversations are difficult to encode – Difficult to translate to Machine Readable Codes
  10. 10. The new challenge for Game AI Complete Information Game Symbolic Game Incomplete Information Game Communication Game Are You a Werewolf?
  11. 11. It is a night • It's a story about a village. • Werewolves have arrived who can change into and eat humans. • The werewolves have the same form as humans during the day, and attack the villagers one-by- one every night. • Fear, uncertainty, and doubt towards the werewolves begin to grow. • The villagers decide that they must execute those who are suspected of being werewolves, one by one... The cover story of Are You a Werewolf?
  12. 12. Are You a Werewolf? • “Are you a Werewolf?” is a party game that models a conflict between an informed minority and an uninformed majority – Also known as “Mafia” • Initially, each player is secretly assigned a role affiliated with one of these teams. • There are two phases: night and day. – At night, the werewolves “attack” the townsfolk. – During the day, surviving players discuss the elimination of a werewolf by voting. • The objectives – Townsfolks: To ascertain who the werewolves are and to kill them. – Werewolves: To kill off all the villagers without being killed themselves.
  13. 13. AI Wolf Project • A project to create AI agents that can play the Werewolf game. – Incomplete information game – Communication Game = Difficult to encode • This feature requires a different approach than other game AI challenges • An AI agent requires multiple research areas – Designing game playing AI – Analyzing the human playing Werewolf – Natural language processing – Agent technology – Human-agent interaction…
  14. 14. Rules of “Werewolf Game”
  15. 15. Role of Players 1/3 • All players have their own “Role” – The roles of all players are allocated randomly. – Players are divided into two teams, townsfolk and werewolf teams, according to their roles. – A player fundamentally cannot know the other players’ roles • The victory condition of two teams – For the townsfolk is to kill all the werewolves. – For the werewolves, the victory condition is to kill humans such that they become equal or fewer in number to the werewolves.
  16. 16. Role of Players 2/3 • Townsfolk Team – Villager: • A character in this role has no special ability. – Seer: • A seer can inspect a player in every night phase to ascertain whether or not a player is a werewolf. – Bodyguard (Knight): • A bodyguard can choose a player in every night phase and protect the player against an attack by a werewolf. – Medium: • A medium can ascertain whether a player who was executed during the previous day phase was a werewolf.
  17. 17. Role of Players 3/3 • Werewolf team – Werewolf: • Werewolves can attack one townsfolk player during each night phase. • They all decide on a single player to attack together with vote, and zero or one villager dies each night. • Werewolves can talk with each other simultaneously during the day secretly. – Possessed: • The possessed have no special ability. • This role secretly cooperates with werewolves because a werewolf-team victory is also regarded as a victory for possessed players. • Werewolves do not know who is a possessed player.
  18. 18. Basic actions of each players • Townsfolk players play to discover werewolves through conversation – Townsfolk players have to detect a werewolf player’s lie. • Werewolf players play to engage in various cooperative maneuvering, without the townsfolk knowing about their roles – The werewolf players know who the werewolves are
  19. 19. Game Procedures • The game proceeds in alternating phases of day and night. • Day: The Discussion Time – All players discuss who the werewolves are. – Players who have special abilities can lead discussions that produce advantages for their respective teams by using the information derived from their abilities. – After a certain period, players execute one player who is suspected of being a werewolf, as chosen by majority voting. The executed player then leaves the game and cannot play. • Night – Werewolf players can attack a non-werewolf player. The attacked player is killed and is eliminated from the game. – Players who have special abilities can use those abilities during the night phase. • The day and night phases alternate until the winning conditions are met.
  20. 20. Game Procedures
  21. 21. Discussion I’m a villager I doubt her I’m a seer I’m the seer
  22. 22. Vote Vote Execution
  23. 23. Attack Attack Dead
  24. 24. Seer Not wolf… Wolf!
  25. 25. Medium Executed Not wolf
  26. 26. Bodyguard guard!
  27. 27. Possessed
  28. 28. Victory Conditions of townsfolk team Eliminate All Werewolves
  29. 29. Victory Conditions of werewolf team The number of wolves is equal to or greater than the number of non-wolves
  30. 30. AI Wolf Project • A project to create AI agents that can play the Werewolf game. – Incomplete information game – Communication Game = Difficult to encode • This feature requires a different approach than other game AI challenges • An AI agent requires multiple research areas – Designing game playing AI – Analyzing the human playing Werewolf – Natural language processing – Agent technology – Human-agent interaction…
  31. 31. Difficulty of werewolf game • The points of werewolf game – Incompleteness of information • An asymmetric diversity of player information • Persuasion as a method of earning confidence • Speculation as a method of detecting fabrication – Communication Game • Natural Language Processing • Speech dialogue – synthesis of speech, speech recognition • Virtual Agent, Real Robots
  32. 32. How to design AIs • To solve these tasks, we employ a collective- intelligence approach, which uses competition to improve each player’s algorithm. • Collective Intelligence via Competitions – There are trials for improving AI with game competitions • Lemonade Stand Game competition • Annual Computer Poker competition • Robocup Soccer
  33. 33. AI Wolf Competitions • The Werewolf platform for AI – AI Wolf Platform – A common platform is indispensable when implementing a collective-intelligence approach. • Competition of AI Wolf – Researchers from various backgrounds can participate freely • Not only researchers, but programmers are also participated – Realizing collective intelligence with participants
  34. 34. Online and Offline Werewolves Card style Werewolf: party game • Less than 1 hour for each game • Competition within visible players – Find each characteristics Online Werewolf: language game • Over few days • Anonymized characters – Avoiding influence of each characteristics • Parallel discussion of wolves
  35. 35. AI Wolf Platform 1/2 • The platform consists of the game server and game-player agents. – The agents connect to the server and play the Werewolf game. – The platform is built on the client–server architecture. • The game server – The server controls the game as role of game moderator. – The server controls the network communication between the agents and itself and maintains a log of the games.
  36. 36. AI Wolf Platform 2/2 • Connection – Game-player agents communicate with the game server via TCP/IP or an internal function-call API. – By using the TCP/IP connection, developers can play against other wired player agents. – In addition, by using the internal function call, developers can conduct high-speed simulations.
  37. 37. AI Wolf Platform Internet AI Wolf AgentAI Wolf Agent API AI Wolf Agent AI Wolf Agent TCP-IP JSON AI Wolf Protocol Game Information AI Wolf Agent AI Wolf Server (Java) Java, Python, .NET Java
  38. 38. Development of an AI Wolf Agent • Event-driven system is employed – The game server sends a request – agents return a response as an action in the Werewolf game. • A game-agent developer must consider only how agents should act when each request arrives. Choose the victim I’ll attack him Ok
  39. 39. Request and Action Request Agent Action Reply Initialize Initialize for game start - DailyInitialize Initialize for day Start - Finish Finish the game - Name Return name of the agent Name Role Return the role of an agent Role Talk Talk to other agents Talk Whisper Talk to other werewolves Whisper Vote Choose an agent to be voted Agent Divine Choose an agent to be divined Agent Guard Choose an agent to be guarded Agent Attack Choose an agent to be attacked Agent
  40. 40. Actions depend on Roles • Each agent should change its behavior pattern depending on its role – The possible requests differ for each role • Seer have to choose target of divide • Werewolves have to choose a victim – Behaviors are differ for each role • Possesses have to lie for werewolves • Bodyguard have to decide whom to guard
  41. 41. AI Wolf Protocol • To ease the development of AI Wolf agents, communication protocol, the AI Wolf Protocol, is provided – A shortened communication protocol designed for AI Wolf. • The current version of AI Wolf Platform employs a simple protocol as the first step of the project. – This simple protocol permits only limited utterances • Ex. “I declare as seer” • “I suspect that he is a werewolf.” • We evaluated the Werewolf game logs, in which 50% of the utterances are represented through following protocols.
  42. 42. Communication protocols 1/2 • estimate(Agent, Role) – An agent expresses its suspicion that [Agent] is [Role]. • comingout(Agent, Role) – The agent asserts that [Agent] is [Role]. • divined(Agent, Species) – The agent (implicated as a seer) gives the divined result that [Agent] is [Species (human or werewolf)] • inquested(Agent, Species) – The agent (implicated as a medium) gives the inquested (investigated) result that the executed [Agent] is [Species (human or werewolf)] • guarded(Agent) – The agent (implicated as a bodyguard) gives the result that [Agent] is protected. • vote(Agent) – The agent claims that a player will select [Agent] for the execution vote.
  43. 43. Communication protocols 2/2 • agree(day, id) – The agent agrees with someone’s statement at statement number [id] on [day]. • disagree(day, id) – The agent disagrees with someone’s statement at statement number [id] on [day]. • skip() – The agent skips its turn to talk, and waits for the next turn. That is, the agent waits to listen to an opponent’s talk and wishes to continue the discussion. • over() – The agent skips its turn to talk, waits for the next turn, and agrees to finish its discussion the same day.
  44. 44. The First WIC • The first WIC (Werewolf intelligence competition) – At the Computer Entertainment Developers Conference (CEDEC2015) – August 27, 2015. • CEDEC2015 – One of the biggest domestic conferences in Japan for video-game competitions since 1998 – More than 30,000 people participate in the conference – Representatives from academic institutions and video- game companies attend and exchange their findings.
  45. 45. Rules of the competition • We organized preliminary and final competitions. • Preliminary stage – 15 agents joined each game set – One set comprised 100 games, and agents was the same in each set but with different assigned roles. – Each agent in the winning team received one point in each game. – 402,800 games were played – 15 agents survive to final stage • Final competition – 1,124,890 games were played
  46. 46. Roles and Agents Role Count Side VILLAGER 8 Townsfolk SEER 1 Townsfolk MEDIUM 1 Townsfolk BODYGUARD 1 Townsfolk WEREWOLF 3 Werewolf POSSESSED 1 Werewolf
  47. 47. Participants Total Students Registered 78 42 Preliminary 38 24 Final 15 7
  48. 48. Winning percentage of each agent at Preliminary stage
  49. 49. Winning percentage of each agent at Final stage
  50. 50. Analysis of the winning rate • High-ranking agents generally have higher success rates • However, most agents have rates that are approximately the same – This may be because Werewolf is a multiple-player game. As such, each agent’s contribution toward a win is lower than it would be in a single-player game. – The rate of the top-ranked agent is 0.4915, whereas the rate of the bottom-ranked agent is 0.3629. This represents there is a significant difference between the strengths of agents (p<0.01). • At Final stage, the five top-ranked agents are significantly stronger than the other 10 agents(p<0.01)
  51. 51. Rank and Role Rank of Preliminary Stage Rank VILLEGER SEER MEDIUM BODYGUARD WEREWOLF POSSESSED 1 1 2 1 1 1 4 2 2 22 17 2 5 15 3 6 3 3 3 4 6 4 3 4 4 10 7 35 5 4 9 13 10 12 17 6 10 14 18 7 6 33 7 13 26 20 29 2 19 8 8 12 27 8 9 31 9 5 15 28 5 22 7 10 6 31 10 9 11 9 11 27 5 24 16 8 8 12 15 12 8 6 16 3 13 18 21 9 27 3 26 14 11 19 14 22 13 27 15 9 6 16 4 30 10 16 12 11 35 13 17 18 17 26 1 32 17 10 37 18 19 29 11 14 15 14 19 23 17 23 33 14 21
  52. 52. Rank and Role Rank of Final Stage Rank Villager Seer Medium Bodyguard Werewolf Possessed 1 1 1 7 1 1 1 2 4 2 8 2 3 4 3 3 3 3 8 2 14 4 2 11 12 4 4 5 5 10 7 1 5 5 3 6 9 4 9 10 6 8 7 11 9 4 6 12 2 8 6 10 15 3 10 6 9 14 5 10 7 7 12 10 5 14 2 15 8 11 11 13 6 11 11 13 7 12 8 13 5 12 11 10 13 12 8 13 13 9 15 14 7 15 6 9 14 9 15 15 12 14 14 15 13
  53. 53. Rank Similarity of Roles • We plotted roles using multidimensional scaling. – We calculated the distance Dkl between points k and l according to the following equation: – 𝐷 𝑘𝑙 = 𝛴𝑖(𝑥𝑖𝑘 − 𝑥𝑖𝑙)2 – The value of 𝑥𝑖𝑘 represents the winning rate of role k for agent i. • To discuss development skills of participants – If the role A and the role B are similar in ranking, the difficulty to develop these two roles seems to be similar
  54. 54. Distance between each role (Preliminary stage)
  55. 55. Preliminary stage • The success rates for Villager, Medium, and Bodyguard are relatively close, whereas those for Seer, Werewolf, and Possessed are distinctly different. • The Seer, Werewolf, and Possessed roles require more specific skills, thus explaining the large distances in the plot
  56. 56. Distance between each role (Final stage)
  57. 57. Final stage • The Medium and Possessed roles showed different behaviors – Players in the final competition wrote more intelligent code than those in the preliminary competition. • For a Medium, the highest and lowest scoring agents showed very slight difference – A Medium does not contribute much toward winning: a result similar to those obtained through statistical analyses of online Werewolf games in Japan. • In the Possessed role, Some high-ranking players score lower than the low-ranking players. – The Possessed role requires certain unique features than the other roles.
  58. 58. Difference between the winning rates Finalists vs Knock out
  59. 59. Is finalists works better than knocked out agents? • We evaluated the difference between the success rates of agents who did and did not reach the final according to each role • In all roles, finalists are stronger by a significant difference – In particular, the differences range between 4% and 5% for both Seer and Werewolf roles
  60. 60. Social Impacts • The first competition was reported by several media outlets (NIKKEI, Game Watch, GPara.com, ASCII/Digital). • We believe that our challenge made a good impact on society, and the notion of “lying AIs” stimulated discussions on the role of AI in modern society.
  61. 61. Conclusion • The AI Wolf project – Incomplete-information game – Competition platforms – Results from the first WIC. • 38 agents participated in the competition • The top agent was significantly stronger than all the other agents. • A competition facilitates in achieving collective intelligence.
  62. 62. Future works • Interactive Agents – Natural Language Processing – Speech dialogue – Non-verbal interactions • Strong Agent to Entertainment Agent – Werewolf Game is a one of the party game – It must be enjoyable game – Not only strong agents are required. Agent have to entertain players
  63. 63. Possibility to Development of AI Wolf Human Play-log Analysis AIWolf Agents AIWolf Agent Game against Human and AIWolf • Log Data Analysis • Game Frame Analysis • Machine Learning • Reinforcement Learning • Rule-based AI • AI Wolf Contest • Virtual Agent • NLP for Werewolf • Dialog System • Emotion Analysis • Real Agent • Strong AI • Entertain AI • Appealing AI Mile Stones of AI Wolf Project
  64. 64. Tagging of Nonverbal Information from Video
  65. 65. AI Wolf Robots
  66. 66. Details of our project http://aiwolf.org/en/
  67. 67. Related works • Hirata Yuya, Inaba Michimasa, Takahashi Kenichi, Toriumi Fujio, Osawa Hirotaka, Katagami Daisuke, Shinoda Kosuke, Werewolf Game Modeling using Action Probabilities based on Play Log Analysis, Computers and Games 2016(2016) • Inaba, M., Toriumi, F., Takahashi, K.: The Statistical Analysis of Werewolf Game Data. 3–6 (2015). • Katagami, D., Kanazawa, M., Toriumi, F., Osawa, H., Inaba, M., Shinoda, K.: Movement design of a life-like agent for the werewolf game. In: IEEE International Conference on Fuzzy Systems. pp. 982–987 (2015). • Kobayashi, Y., Osawa, H., Inaba, M., Shinoda, K., Toriumi, F., & Katagami, D. Development of werewolf match system for human players mediated with lifelike agents In Proceedings of the second international conference on Human-agent interaction (pp. 205-207). ACM.(2014) • Daisuke Katagami, Shono Takaku, Michimasa Inaba, Hirotaka Osawa, Kosuke Shinoda, Junji Nishino and Fujio Toriumi Investigation of the Effects of Nonverbal Information on Werewolf FUZZ-IEEE 2014 (2014)
  68. 68. Acknowledgements • This work was partially supported by – Hayao Nakayama Foundation for Science & Technology and Culture – Foundation for Fusion of Science and Technology – JSPS KAKENHI Grant Number 26118006 • We also want to say thanks for – Computer Entertainment Developers Conference – Japan Society for Artificial Intelligence
  69. 69. Contact gm@aiwolf.org

×