Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

The Role of Evolutionary Computation in Game AI

Download to read offline

introductory game AI talk from an evolutionary computation perspective for the sigevo summer school 2019

  • Be the first to like this

The Role of Evolutionary Computation in Game AI

  1. 1. Discover the world at Leiden UniversityDiscover the world at Leiden University The Role of Evolutionary Computation in Game AI Mike Preuss @ S3 ACM GECCO summer school, Praha July 10, 2019
  2. 2. Discover the world at Leiden University ● my roots in Evolutionary Computation ● game AI today: a forming field ● Procedural Content Generation ● Balancing ● GVGAI/competitions/rolling horizon EA roadmap 2
  3. 3. Discover the world at Leiden University Evolutionary Computation and me ●most likely the last PhD student of Hans-Paul Schwefel (co-inventor of the Evolution Strategy) ●most important message: expect the unexpected ●focus on multi-modal optimization (detect several good, different solutions simultaneously) ●a bit on multi-objective optimization ●some work on experimental methodology 3
  4. 4. Discover the world at Leiden University PhD (2013) and book (2015) most important results: ●the advantage of niching (if any) can only come from coordinated placement ●in contrast: SLS is randomized placement ●simulations/theory: unfortunately, we cannot speed up finding the global optimum ●but if we go for coverage (find most if not all good optima) we can gain a lot ●invented NEA2 as top algorithm (2015), lesson learned: for niching, also use the objective values modern name for this stuff: quality diversity 4
  5. 5. Discover the world at Leiden University GECCO multimodal optimization competition • builds on a fixed selection of 20 problems made from 12 functions in different dimensions • averages over 3 different rankings: - average PR (peak rate) values - static F1 (takes the amount of points in the answer set into account) - dynamic F1 (same but as integral over time for the whole runtime) • only global optima are sought • the number of optima is low for most problems http://www.epitropakis.co.uk/gecco2019/ 5
  6. 6. Discover the world at Leiden University what we did not expect so early 6
  7. 7. Discover the world at Leiden University bla a forming field a forming field
  8. 8. Discover the world at Leiden University what about game AI? current game AI is: • playing well • generating content • modeling players see 2018 book: Artificial Intelligence and Games, by Yannakakis/Togelius http://gameaibook.org/book.pdf video: fully generated game worlds in No Man’s Sky (2016) 8
  9. 9. Discover the world at Leiden University when game AI becomes… artificially intelligent ongoing discussion in game AI: how intelligent is current game AI? see recent gamasutra post: http://www.gamasutra.com/view/news/253974/When_artificial_intelligence_in_video_games_becomesartificially_intellige nt.php "My prediction is a little bit controversial. I think the next giant leap of game AI is actually artificial intelligence." - Alex J. Champandard at GDC 2013 Ken de Jong (at GECCO 2017, Berlin) on the question: What do you see as dominant research direction in Evolutionary Computation for the next years? “It is time to apply our methods to real problems now.” 9
  10. 10. Discover the world at Leiden University classical AI techniques in games even in the early 2000s, lots of standard techniques in use (and developers showing little interest for a change): • finite state machines (FSM) • hierarchical FSM • scripting basically mostly used for Non-Player Character (NPC) control, also: • cellular automata • influence maps • basic optimization approaches • game theory (still useful for the right problem, but we have so much more now) 10
  11. 11. Discover the world at Leiden University modern approach • we treat all aspects of games, now also game design/production • we use what fits best for a specific problem everything is an agent: AI director, opponent AI controller, content generator, .... game AI specific algorithms classic AI / machine learning in-game AI use computational intelligence game design/production 11
  12. 12. Discover the world at Leiden University game ai overall view • game AI traditionally encompasses mostly NPC behavior and bots • control happens during the game, design support before deployment • design support is rather new, but a highly dynamic research area game AI design supportcontrol directors strategic (bots) /supportive AI PCGNPC behavior balancing NPC = non-player character, PCG = procedural content generation 12
  13. 13. Discover the world at Leiden University panoramic view game AI components, player centric view, from: Yannakakis, G.N.; Togelius, J., "A Panorama of Artificial and Computational Intelligence in Games," in IEEE TCIAIG, 2014 13
  14. 14. Discover the world at Leiden University why apply CI methods to games? contrary to board games, • game trees often not applicable • incomplete information • concurrency: during planning phase, game situation changes • quantifying a game situation is not trivial ➔ good and fast approximations are needed evolutionary optimization is • versatile, flexible, still works (somehow) • copes with noise and strange search spaces • can be asked to deliver a result at any time 14
  15. 15. Discover the world at Leiden University Evolutionary and AI •not fully up to date, but clearly shows relations •boundaries much more fuzzy nowadays 15
  16. 16. Discover the world at Leiden University CI and games: a field forming • 1999: Blondie24, learning checkers with CI • 2005 first Computational Intelligence in Games (CIG) conference • IEEE TCIAIG Journal (Transactions on CI and Artificial Intelligence in Games) since 2009 • EvoGames track in Evo*, DETA track in GECCO since 2009/2011 • many “neighbor” conferences, etc. AIIDE, FDG, gameai/nucl.ai conf. (not strictly CI, but CI welcome) • general approach target oriented, not technique oriented • 2012: first Dagstuhl seminar on AI and CI in Games • 2015: second Dagstuhl seminar (integration) • 2017: third Dagstuhl seminar (AI-supported design) • 2018/2019: journal and conference renamed to IEEE Transactions on Games and IEEE Conference on Games • 2019: first Shonan seminar on AI for Games 16
  17. 17. Discover the world at Leiden University one small step for [a] man…. what recently happened: 17 AlphaGo: Go (MCTS, expert knowledge, self-play) AlphaGoZero: Go (MCTS-, self-play) Deep Reinforcement Learning: ALE (pixel input, self-play) AlphaStar: StarCraft (MCTS?, pixel input, self-play, expert knowledge) AlphaZero: Go, Chess, Shogi (MCTS-, self-play) Go-Explore: Montezuma (population based exploration) Obstacle Tower Challenge (population based training) ForTheWin: Capture the Flag (2 speed rnns, population based learning, population based training)
  18. 18. Discover the world at Leiden University combining EC and Deep Reinforcement Learning •some (limited) successes in weight optimization with EC •population-based learning: - GANs, multiple agent architectures - resembles co-evolutionary approaches, same problems (how to control) •population-based based training: - generating test cases as needed - procedural content generation (PCG), mainly done via EC methods •not really exploited yet: multi-modal/multi-objective approaches •maybe more things to transfer? 18
  19. 19. Discover the world at Leiden University what I did in Game AI ●procedural generation of strategy game maps ● organizing StarCraft I competitions, survey ● some works on balancing (also using EC): ○there are usually many solutions in balancing ○the problem is very multi-modal ●recently: facet orchestration (how to combine several generators, e.g. visuals, audio) ●very recently: transfer AlphaAnything to new problems (not from the games domain) ●understand how it works on small games 19
  20. 20. Discover the world at Leiden University Obstacle Tower Challenge current DRL challenge for artificial general intelligence (AGI) • levels increase in difficulty • levels procedurally generated • decorations change • includes minigames (Sokoban) runs in 2 phases: - qualification (gone) - finals (up to July 15th, 2019) https://www.aicrowd.com/challenges/unity- obstacle-tower-challenge 20
  21. 21. Discover the world at Leiden University bla Procedural Content Generation
  22. 22. Discover the world at Leiden University search-based vs. generate-and-test 22
  23. 23. Discover the world at Leiden University Evolutionary Algorithms • initial solutions needed: existing or randomly generated solutions • fitness function shall measure the targeted properties (designer input) • matching variation operators need to be established 23
  24. 24. Discover the world at Leiden University why using EAs in PCG? pros: - flexible, only needs good utility function - not very sensitive concerning noise/mistakes - anytime method: we can obtain a result whenever needed cons: - optimality cannot be guaranteed (except for very few special cases) - stagnation phases of unpredictable length (break?) - depends on matching representation and operators - slow on simple problems 24
  25. 25. Discover the world at Leiden University back in 2009: galactic arms race 25
  26. 26. Discover the world at Leiden University how are the new weapons produced? • very small population, 3 solutions (candidates, individuals) • interactive selection: player removes existing solution when new one is generated • weapons are created via a parametrized particle system by complex neural network • cgNEAT: neural network topology optimized by means of evolutionary algorithm 26
  27. 27. Discover the world at Leiden University a sample gallery 27
  28. 28. Discover the world at Leiden University StarCraft I map evolution (2009-2012) 28
  29. 29. Discover the world at Leiden University RTS map design with Sentient Sketchbook Preuss, Liapis, Togelius. Searching for Good and Diverse Game Levels. IEEE CIG 2014. 29
  30. 30. Discover the world at Leiden University gamesbyangelina and the procjam • Michael Cook’s AI that is designed to intelligently design video games: http://www.gamesbyangelina.org/ • PROCJAM: yearly game jam with strong emphasis on PCG http://itch.io/jam/procjam • coming soon: facet orchestration 30
  31. 31. Discover the world at Leiden University Project seminar nd balancing
  32. 32. Discover the world at Leiden University Game AI hot topics • procedural content generation (PCG) - large game worlds are expensive to make - represent specific (designer) styles • believability of non-player character (NPC) behavior - more humanlike behavior (also evoke and show emotions) - better cooperation of units (team AI) - human/AI interaction • personalization - preference modeling (what do they like?), player type analysis - dynamic adaptation of game content and mechanics balancing is everywhere 32
  33. 33. Discover the world at Leiden University balancing project with BlueByte • prototype from BlueByte • ZombieVillage game, related to tower defense • several parameters: can we balance automatically? 33
  34. 34. Discover the world at Leiden University balancing project with BlueByte • number of (most important) parameters reduced to 5 • interchange between manual and automated (integrated) balancing • core result: both steps needed, many different solutions exist Marlene Beyer, Aleksandr Agureikin, Alexander Anokhin, Christoph Laenger, Felix Nolte, Jonas Winterberg, Marcel Renka, Martin Rieger, Nicolas Pflanzl, Mike Preuss, Vanessa Volz: An integrated process for game balancing. CIG 2016 34
  35. 35. Discover the world at Leiden University tackling a complex RTS Mike Preuss, Thomas Pfeiffer, Nicolas Pflanzl, Vanessa Volz: Integrated Balancing of an RTS Game: Case Study and Toolbox Refinement. CIG 2018 35
  36. 36. Discover the world at Leiden University bla nd general game AI/competitions
  37. 37. Discover the world at Leiden University can we also have general (game) AI? • instead of this: • we rather want this: game 1 game 2 game 3AI 3 AI 2 AI 1 develop for develop for develop for game 1 game 2 game 3 AI 1 learn to play learn to play learn to play 37
  38. 38. Discover the world at Leiden University GVGAI ENVIRONMENT • general video-game playing (GVGP) succeeds general game-playing • targeted at more complex, non-deterministic games • language first shaped during Dagstuhl seminar CI and AI in games 2012 • we want to obtain controllers that can play “any game” • important publications: John Levine, Clare B. Congdon, Michal Bída, Marc Ebner, Graham Kendall, Simon Lucas, Risto Miikkulainen, Tom Schaul, and Tommy Thompson. General Video Game Playing. Dagstuhl Follow-up, 6:1–7, 2013. Tom Schaul. A Video Game Description Language for Model-based or Interactive Learning. In Proceedings of the IEEE Conference on Computational Intelligence in Games, Niagara Falls, 2013. IEEE Press http://www.gvgai.net 38
  39. 39. Discover the world at Leiden University GVGAI competition • build general AI controllers • learn to play many games, not one • many different challenges • many successful controllers use Monte-Carlo Tree search • but Rolling Horizon EA also an option • but how can this work? • “traditional track”: 1-player planning • employs forward model (action simulation, may be non-deterministic) • new: 2-player planning, learning (no forward model) • now also: level generation, rule generation 39
  40. 40. Discover the world at Leiden University some example games Infection: Missile Command: Pacman 40
  41. 41. Discover the world at Leiden University VGDL – game and level definitions 41
  42. 42. Discover the world at Leiden University A SAMPLE (RANDOM) CONTROLLER 42
  43. 43. Discover the world at Leiden University Rolling Horizon EA for search based planning • basic idea: evolve whole sequence of moves at once, apply only first action • hybrids with MCTS like playouts are also possible • simple extension with shift buffer (only remove latest action) performs better Raluca Gaina, Simon Lucas, Diego Perez. Rolling Horizon Evolution Enhancements in General Video Game Playing. IEEE CIG 2017 43
  44. 44. Discover the world at Leiden University current competitions held at COG 2019 • 4th Angry Birds Level Generation Competition • Bot Bowl I • Fighting Game AI Competition • First TextWorld Problems: A Reinforcement and Language Learning Challenge • Geometry Friends Game AI Competition • General Video Game AI Competitions (Learning Track) • Hanabi Competition • Hearthstone AI competition • MicroRTS AI Competition • Short Video Competition • StarCraft AI Competition • Strategy Card Game AI Competition lots of demo and explanation videos included: http://ieee-cog.org/competitions_conference/ 44
  45. 45. Discover the world at Leiden University conclusions •EC is part of AI and has always been •Procedural Content Generation getting very important for games industry but also for Deep Learning •evolutionary approaches have great potential for improving other algorithms, e.g. RHEA (rolling horizon EA) instead of MCTS •3 important applications of EC algorithms in game AI: PCG, balancing, search-based planning 45
  46. 46. Discover the world at Leiden UniversityDiscover the world at Leiden University Don’t panic! 46

introductory game AI talk from an evolutionary computation perspective for the sigevo summer school 2019

Views

Total views

442

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

6

Shares

0

Comments

0

Likes

0

×