Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Long Lin at AI Frontiers : AI in Gaming


Published on

Games have been leveraging AI since the 1950s, when people built a rules-based AI engine that played tic-tac-toe. With technological advances over the years, AI has become increasingly popular and widely used in the gaming industry. The typical characteristics of games and game development makes them an ideal playground for practicing and implementing AI techniques, especially deep learning and reinforcement learning. Most games are well scoped; it is relatively easy to generate and use the data; and states/actions/rewards are relatively clear. In this talk, I will show a couple of use cases where ML/AI helps in-game development and enhances player experience. Examples include AI agents playing game and services that provide personalized experience to players.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Long Lin at AI Frontiers : AI in Gaming

  1. 1. AI in Gaming Long Lin Director of Engineering Data & AI @ EA
  2. 2. 2
  3. 3. “At EA, we envision a future in which games go even further beyond the immersive experiences players enjoy today. I’m talking about games that offer living, breathing worlds that constantly evolve.” - Ken Moss
  4. 4. AI Everywhere Creating games: • AI Agent & Simulation • Animation • AI-driven game balancing • AI development tools Operating games: • Bad actor detection • Content creation • Player acquisition Playing games: • Personalization • Mentors & Assistants • Matchmakers • Conversational interfaces • Dynamic experiences Not an exhaustive list
  5. 5. Games Have Been Instrumental to AI ● Crushing humans is not the only use-case of AI in games ● Can we create AI agents that play the games similar to humans do ● Can we leverage AI to benefit games
  6. 6. Gaming - Playground for AI • Human Interaction • All virtual - faster iteration • Can be well scoped • Relatively easy to generate and use the data • States/Actions/Rewards are relatively clear • Result is measurable, and can be visualized
  7. 7. AI Agents Agents that learn to play the game with different goals ● NPC ● Simulation ● Exploration of game space ● Game design ● Game balancing ● Optimal solution ● Fast test and feedback loops ● Find defects
  8. 8. ● Dev Build ● Fast Simulation ● Near-Realtime Metrics ● Multiple agents
  9. 9. Designer Questions ● Is there a significant imbalance in relationship categories? ● How many actions are needed to progress in the careers? ● How do objects impact career progress? ● How much impact did the build changes actually make, and does these change align with the designer’s plan?
  10. 10. Markov Decision Process Tuple (S, A, P, R) ● States: S ● Action: A ● State Dynamics/Simulator: P(S,A) → S’ ● Reward: R Learn a Policy: π(S) → A *Randomness of Simulator and Policy
  11. 11. AI Agents Can be Difficult to Train • Large state space • Large action space • Large number of steps in game episodes • Reward sparse environments • Simultaneous actions • Multi-agent interaction • Complex reward function and long term strategy
  12. 12. Luckily • Better/cheaper computing power • Distributed model - Computation, Data • A lot of Data • More mature AI libraries & frameworks • Framebuffer vs Game state parameters, Joystick vs abstract actions • Scope the problem - start with something simple • Does not have to start from scratch - Demonstration & IL
  13. 13. Game Interface Platform Game
  14. 14. Reinforcement Learning Images by Stephane Ross Calculate Reward Initialize Policy Trials Update Policy
  15. 15. (Deep) Reinforcement Learning Learning from Rewards, Trials and Errors Goal: Optimal solution (i.e., maximize cumulative reward) Pros: ➔ Explore the world beyond the skill of experts ➔ Superhuman game play ➔ Fast simulation/iteration Cons: ➔ Complexity around Knowledge representation for non-framebuffer approach ➔ Algorithmic efficiency (time to converge, might not converge) in transfer data to policy depends on high efficient representation of knowledge
  16. 16. Imitation Learning Images by Stephane Ross Expert Demonstration State/Action Pair Policy
  17. 17. (Deep) Imitation Learning Learning from experts/players’s demonstration & feedback Goal: Human-like behavior (i.e., minimize the difference between policy and the demonstration) Pros: ➔ Simple, efficient ➔ Works well when at state space that has enough demonstrations coverage ➔ Great at picking up styles Cons: ➔ Limited state-space coverage of the expert data, tend to over-fit ➔ Limited by the speed/scale human player can generate data ➔ No long term planning
  18. 18. IL & RL Images by Stephane Ross Calculate Reward Initialize Policy Trials Update Policy Expert Demonstration State/Action Pair
  19. 19. Agent Training Workflow Platform Game ● Training Environment ● Policy Storage ● Agent Management ● Agent Execution ● Data Pipeline
  20. 20. Looking Ahead • State of Art Methodologies • More Complex Environment • Multi-Agent Interaction • Distributed Training
  21. 21. Q&A