Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

슬라이드 1


Published on

  • Be the first to comment

  • Be the first to like this

슬라이드 1

  1. 1. GA를 활용한 게임 에이전트 동작-CIG Competition PacMan-<br />YonseiUniv Soft Computing Lab<br />
  2. 2. 1/34<br />NOT YOUR FAULT!<br />
  3. 3. 2/34<br />다음 중 GA 를 사용하여 풀기에 적합한 문제는?<br />7차 방정식<br />사용자 상태 추론<br />게임에서의 최적화 경로 찾기<br />하드웨어 설계<br />게임 전략 생성<br />
  4. 4. 3/34<br />1. 탐색범위가 넓은 문제!<br />2. 반드시 제일 좋은 해를 구할 필요가 없는 문제!<br />
  5. 5. 4/34<br />DO YOU BELIEVE IN GA?<br />
  6. 6. 동기<br />5/34<br />실제 동물은 물리 환경에서 최적화된 움직임을 하고 있을까?<br />
  7. 7. 문제 설계<br />6/34<br />
  8. 8. 문제 설계(유전자)<br />7/34<br />Time(1.5 sec)<br />Angular motor를 이용 각 관절의 각속도를 제어한다.<br />Motor의 속도는 5단계(-2, -1, 0, 1, 2 )로 구성되어있다. 여기서 (-)값은 음의 방향의 각속도를 말한다.<br />Joint Degree(38)<br />
  9. 9. 유전자 평가<br />8/34<br />0<br />1<br />1<br />0<br />1<br />1<br />1<br />0<br />0<br />0<br />0<br />1<br />0<br />0<br />0<br />1<br />0<br />0<br />1<br />1<br />64<br />361<br />169<br />576<br />얼마나 전진했는가?<br />적합도<br />염색체<br />
  10. 10. 실험결과<br />9/34<br />
  11. 11. 확인<br />10/34<br />동영상!<br />
  12. 12. 확인<br />11/34<br />이런 모양도!<br />
  13. 13. CIG<br />IEEE Symposium on Computational Intelligence and Games<br />Tutorial<br />Session Presentation<br />Competition<br /><br />Competitions Chair: Simon M. Lucas<br />12/34<br />
  14. 14. CIG Competition<br />13/34<br />
  15. 15. Unreal Tournament<br />Sponsored by 2K Australia<br />The competition task is to create a computer game bot<br />Indistinguishable from a human player<br />major prize of A$7,000 cash<br />Trip to 2K Australia's studio<br />DeathMatch game type for the FPS<br />Chatting will be disabled. <br />Some aspects of the game play will be modified in a way not known to competitors. <br />14/34<br />
  16. 16. TORCS<br />Car Racing<br />Daniele Loiacono, Julian Togelius and Pier Luca Lanzi<br />Learn (or otherwise develop) a controller<br />The distance raced in a fixed amount of time<br />Best few controllers will race against each other<br />Mac OS X, Windows and Linux, using Java and C++<br />Neural networks, Fuzzy logic, Force fields, Expression trees<br />15/34<br />
  17. 17. Ms Pac-Man<br />Non-deterministic game<br />The world record for a human player currently stands at 921,360<br />Can anyone develop a software agent to beat that?<br />the best software controller for the game of Ms Pac-Man<br />Screen-capture to get the current state of the game<br />16/34<br />
  18. 18. AI for CS different than AI for Games<br />17/34<br /><ul><li>Must be smart, but purposely flawed
  19. 19. Loose in a fun, challenging way
  20. 20. No unintended weaknesses
  21. 21. No “golden path” to defeat
  22. 22. Must not look dumb
  23. 23. Must perform in real time (CPU)
  24. 24. Configurable by designers
  25. 25. Not hard coded by programmer
  26. 26. “Amount” and type of AI for game can vary
  27. 27. RTS needs global strategy, FPS needs modeling of individual units at “footstep” level
  28. 28. RTS most demanding: 3 full-time AI programmers
  29. 29. Puzzle, street fighting: 1 part-time AI programmer</li></li></ul><li>Goals<br />Games are entertainment<br />Important that things behave naturally<br />Not necessarily perfect<br />“things’ are not always creature<br />Follow(the game’s) natural laws<br />And avoid cheating<br />Characters should be aware<br />18/34<br />
  30. 30. Game Agents<br />19/34<br />Act<br />Sense<br />Think<br /><ul><li>Most AI focuses around game agent
  31. 31. think of agent as NPC, enemy, ally or neutral
  32. 32. Loops through: sense-think-act cycle
  33. 33. Acting is event specific, so talk about sense and think first, then a bit on act</li></li></ul><li>Game Agents – Sensing (1 of 2)<br />Gather current world state: barriers, opponents, objects, …<br />Needs limitations: avoid “cheating” by looking at game data<br />Typically, same constraints as player (vision, hearing range, etc.)<br />Vision<br />Can be quite complicated (CPU intensive) to test visibility (ie- if only part of an object visible)<br />Compute vector to each object<br />Check magnitude (ie- is it too far away?)<br />Check angle (dot product) (ie- within 120° viewing angle?)<br />Check if obscured. Most expensive, so do last.<br />
  34. 34. Game Agents – Sensing (2 of 2)<br /><ul><li>Hearing
  35. 35. Ex- tip-toe past, enemy doesn’t hear, but if run past, enemy hears (stealth games, like Thief)
  36. 36. Implement as event-driven
  37. 37. When player performs action, notify agents within range
  38. 38. Rather than sound reflection (complicated) usually distance within bounded area
  39. 39. Can enhance with listen attributes by agent (if agent is “keen eared” or paying attention)
  40. 40. Communication
  41. 41. Model sensing data from other agents
  42. 42. Can be instant (ie- connected by radio)
  43. 43. Or via hearing (ie- shout)
  44. 44. Reaction times
  45. 45. Sensing may take some time (ie- don’t have agent react to alarm instantly, seems unrealistic)
  46. 46. Build in delay. Implement with simple timer.</li></li></ul><li>Game Agents – Thinking (1 of 3)<br />Evaluate information and make decision<br />As simple or elaborate as required<br />Generally, two ways: <br />Pre-coded expert knowledge<br />Typically hand-crafted “if-then” rules + “randomness” to make unpredictable<br />Search algorithm for best (optimal) solution<br />Ex- MinMax<br />22/34<br />
  47. 47. Game Agents – Thinking (2 of 3)<br />23/34<br /><ul><li>Expert Knowledge
  48. 48. Finite State Machines, decision trees, … (FSM most popular, details next)
  49. 49. Appealing since simple, natural, embodies common sense and knowledge of domain
  50. 50. Ex: See enemy weaker than you?  Attack. See enemy stronger?  Go get help
  51. 51. Trouble is, often does not scale
  52. 52. Complex situations have many factors
  53. 53. Add more rules, becomes brittle
  54. 54. Still, often quite adequate for many AI tasks
  55. 55. Many agents have quite narrow domain, so doesn’t matter</li></li></ul><li>Game Agents – Thinking (3 of 3)<br />24/34<br /><ul><li>Search
  56. 56. Look ahead and see what move to do next
  57. 57. Ex: piece on game board (MinMax), pathfinding (A*)
  58. 58. Works well with known information (ie- can see obstacles, pieces on board)
  59. 59. Machine learning
  60. 60. Evaluate past actions, use for future action
  61. 61. Techniques show promise, but typically too slow</li></li></ul><li>Game Agents – Acting (1 of 2)<br />25/34<br /><ul><li>Learning and Remembering
  62. 62. May not be important in many games where agent short-lived (ie- enemy drone)
  63. 63. But if alive for 30+ seconds, can be helpful
  64. 64. ie- player attacks from right, so shield right
  65. 65. Implementation - too avoid too much information, can have fade from memory (by time or by queue that becomes full)</li></li></ul><li>Game Agents – Acting (2 of 2)<br />26/34<br /><ul><li>Making agents stupid
  66. 66. Many cases, easy to make agents dominate
  67. 67. Ex: FPS bot always makes head-shot
  68. 68. Dumb down by giving “human” conditions, longer reaction times, make unnecessarily vulnerable, have make mistakes
  69. 69. Agent cheating
  70. 70. Ideally, don’t have unfair advantage (such as more attributes or more knowledge)
  71. 71. But sometimes might “cheat” to make a challenge
  72. 72. Remember, that’s the goal, AI lose in challenging way
  73. 73. Best to let player know</li></li></ul><li>Game Agents – AI<br />Deterministic AI<br />Preplanned, predictable events<br />Set by rules<br />Walk forward ten steps then backwards<br />If player is within 5 squares move closer<br />Go to town … take the shortest path using shortest path algorithm<br />Attack player with least health/most damage potential/etc<br />Non deterministic AI<br />Unpredictable movement<br />Random selection of directions<br />React to environment to learn movement<br />Player nearby, go slower<br />Got shot going around corner last time, then take a wider path this time<br />27/34<br />
  74. 74. Base Environment<br /><br />Java<br />Image Detector, Simulator, Game <br /><br />
  75. 75. 문제 설계<br />29/34<br />Sensing<br />Thinking<br />Acting<br />IF then<br />IF then<br />If then <br />.<br />.<br />.<br />If then<br />상<br />하<br />좌<br />우<br />
  76. 76. 문제 설계<br />30/34<br />1. 죽지 말자 <br />2. 제일 가까운 고스트를 피하자 <br />3. 고스트는 나를 기준으로 어디에 있는가?<br />4. 고스트의 반대방향으로!<br />
  77. 77. 문제 설계<br />31/34<br />팩맨의 움직임<br />고스트의 위치<br />상<br />하<br />좌<br />상<br />하<br />좌<br />우<br />우<br />좌상<br />우상<br />좌하<br />우하<br />탐색범위?<br />256<br />
  78. 78. 문제 설계<br />32/34<br />Evaluating!<br />문제를 잘 이해하는 게 관건!<br />많이 해보자!<br />
  79. 79. 문제 설계<br />33/34<br />Expert Rule Based System(High Priority)<br />GA Rule(Middle Priority)<br />Expert Rule Based System(Low Priority)<br />
  80. 80. 정리<br />34/34<br />1. GA 활용예<br />2. CIG Competition 소개<br />3. 게임의 범용 AI 설계법 소개<br />4. Pac_manAI 코드 소개<br />5. Pac_manAI를 위한 GA 기초설계<br />