Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Applying AI in Games (GDC2019)


Published on

Gaming AI use cases presented at GDC 2019.

Published in: Technology
  • Today, I want to share with you my own "unfair advantage" ... An honest crack at an insider's edge that's so effective it's nothing less than performance enhancing for your own bottom line profits! ★★★
    Are you sure you want to  Yes  No
    Your message goes here
  • You're Going to Want to Read This Before Tonight ■■■
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Applying AI in Games (GDC2019)

  1. 1. Confidential + Proprietary Applying Cloud AI in games with DeNA Samir Hammoudi, Gaming Technical Specialist, Google Cloud Jun Ernesto Okumura, AI System Dept. Project Leader, DeNA Ikki Tanaka, AI System Dept. Data Scientist, DeNA
  2. 2. Create + Connect + Scale
  3. 3. Many Applications of AI In Games Player against smarter AI (AI playing like a pro player) Smarter NPC Automated QA testing Replace a human player by a bot when they leave a game Fraud/Cheat detection Player toxicity Break language barrier Recommend the appropriate deck/card/character/equipement IAP recommendations Better matchmaking And so many more!!
  4. 4. Confidential + Proprietary Atari Go Starcraft Games are an ideal environment for AI Possible Actions 17 361 Millions No. of Moves Per Game 100’s of moves 100’s of moves 1000’s of moves
  5. 5. 21M Developers <1000’s Deep Learning Researchers <1M Data Scientists Helping Democratize Machine Learning
  6. 6. Confidential + Proprietary AI building blocks for all developers Cloud Vision Cloud Translation Cloud Natural Language Cloud Speech Cloud Video Intelligence
  7. 7. Confidential + Proprietary Google Cloud enables your AI journey Pre-packaged AI solutions Powerful image analysis Vision API Natural chatbot interactions DialogFlow Powerful text analysis Natural Language API Train custom machine learning models AutoML A new middle pathway Custom ML models Support for custom ML Models ML Engine Open source ML TensorFlow Hardware optimised for machine learning Cloud TPUs Create ML models using standard SQL BigQuery ML
  8. 8. Confidential + Proprietary Why customers choose Google for AI Scale Speed Quality Best performance for AI workloads with customized hardware and Cloud TPUs Instant access to thousands of machines with Google Cloud Pre-trained AI building blocks solve business needs, with the highest quality Accessible Cloud AutoML and ML Engine to customize models for users ranging from citizen data scientists to researchers 1 2 3 4
  9. 9. Confidential + Proprietary More Advanced? Let’s put some AI in your game! You’re new to ML? Cloud Vision Cloud Translation Cloud Natural Language Cloud Speech Cloud Video Intelligence Cloud ML Engine Cloud AutoML
  10. 10. Building Game AI for better user experiences Jun Ernesto Okumura / Ikki Tanaka (DeNA)
  11. 11. Confidential + Proprietary Jun Ernesto Okumura AI System Dept., AI Team Lead / ML Engineer Speaker Gaming AI team Ikki Tanaka AI System Dept., Data Scientist / ML Engineer Speaker Takeshi Okada AI System Dept., ML Engineer Yu Kono AI System Dept., AI Researcher
  12. 12. Confidential + Proprietary Gaming in DeNA Gyakuten Othellonia Inhouse Nintendo Alliance Titles ● Super Mario Run ● Fire Emblem Heroes ● Animal Crossing: Pocket Camp Megido 72 Alliance ● Final Fantasy Record Keeper ● Uta Macross 3rd Party ● Granblue Fantasy
  13. 13. Confidential + Proprietary Problem Setting and Approach
  14. 14. Strategic app game Based on Board Game (Othello / Reversi) Variety of Characters / Skills Released in 2016 23M downloads so far Region: Japan / Taiwan Gyakuten Othellonia
  15. 15. Deck and Character Deck: 16 characters (among more than 3,000 options) Character: Each character has its own skills, and status Player Deck Character Detail Status: HP (hit-point) ATK (attack-point) Skill description: When this character can flip just one opponent piece, the attack point will double
  16. 16. 1. High churn-rate 2. Hard to construct optimal decks / No training field 3. Need better on-boarding support for beginners Issues to be solved
  17. 17. Our approach Support beginners by two AI functions Deck Recommendation select appropriate characters for deck (Association Analysis) Othellonia Dojo (battle AI) support human-level AI for practicing (Supervised Learning w/ Deep Neural Network)
  18. 18. Available Data Most recent 9 months, ~billion log entries ● Extract high-skilled player’s deck/battle logs ● Data Augmentation (battle AI) ○ Board rotation, hand piece permutation, … preprocessed logsBigQuery
  19. 19. Deck Recommendation Released on 6th Nov. 2018 A player select 1. deck archetype, 2. deck cost and 3. favorite character Metrics: Usage, Acceptance rate, Win rate, ... Build my deck!Association Rules Recommendation API Server Deck logs
  20. 20. Othellonia Dojo (AI Player) Feature released last week A player select 1. deck archetype and 2. AI level Metrics: Usage, Win rate, ... Battle record Deep Neural Network Trained AI model Let’s battle! Battle API Server game status Inference result
  21. 21. Confidential + Proprietary Deck Recommendation
  22. 22. Algorithm of Deck Recommendation Player Inputs Select Leader Select Base Characters Select Well-Associated Characters Recommended Deck
  23. 23. How Are the Relationships Extracted? ● Association analysis ○ Statistical method to extract the relationships inside large-scale data ○ Relations are evaluated quantitatively with some metrics GOOD RELATION! BAD RELATION… GOOD RELATION!
  24. 24. Why Did We Choose GCP for Our ML Systems? Powerful and flexible tools for ML and Data Science Scalable and robust infrastructure services App Engine Cloud Functions Cloud Pub/Sub Cloud Datastore Stackdriver Logging Compute Engine Kubernetes Engine BigQuery Cloud Datalab Cloud Storage Cloud Machine Learning
  25. 25. System Architecture with Google Cloud Platform batch Deck Logs BigQuery Association Analysis Compute Engine Cloud Storage Association Rules Master Data of Game (sqlite) batch Cloud Pub/Sub Cloud Storage Cloud Datastore App Engine Cloud Functions Update Time Association Rules Master Data of Game (pickle) (cron) Recommendation API App Engine User ID Characters Obtained Options of Recommendation Recommended Deck
  26. 26. Confidential + Proprietary Deep-Learning AI-bot (Othellonia Dojo)
  27. 27. 1 6 2 4 9 2 2 5 3 1 8 3 0 1 5 0 9 2 8 7 2 7 3 4 1 4 3 2 3 0 3 5 1 9 8 3 1 6 2 4 9 2 2 5 3 1 8 3 0 1 5 0 9 2 8 7 2 7 3 4 1 4 3 2 3 0 3 5 1 9 8 3 Deep Learning Makes AI Like A Human Player 1 6 2 4 9 2 2 5 3 1 8 3 0 1 5 0 9 2 8 7 2 7 3 4 1 4 3 2 3 0 3 5 1 9 8 3 Deep Neural Network Action Probability Battle logs of top-tier players Update Network >5000 features >10m battles
  28. 28. Training Phase AI model trained in the Google Compute Engine is stored in the Cloud Storage training PvP Match Training AI model BigQuery Preprocessing Compute Engine Training Compute Engine Cloud Storage Cloud Storage Battle Record Battle Record AI model AI model
  29. 29. Inference Phase Deploy the trained AI model stored in the Cloud Storage to CMLE serving Inference API App Engine Scores of Actions AI PvE Match GAE/Python3.7 Auto-scaling Cloud ML Engine Auto-scaling Cloud Storage Inference by AI Cloud Machine Learning Features Scores of Actions Battle Record AI model
  30. 30. Testing of the Systems High Traffic Estimated RPS (requests per second)RPS Time
  31. 31. Load / Scalability Testing Auto-scaling of CMLE is very helpful for our case ○ Averaged latency is less than 1 second ○ ~1.3% of requests exceeded our latency limit Requests Errors How to remove this errors? 1:30 PM 2:00 PM 2:39 PM 1:30 PM 2:00 PM 2:39 PM
  32. 32. Our Approach to Eliminate the Errors We set another backup-endpoint to receive the request that fails to be operated in the main-endpoint Inference API App Engine Inference by AI Cloud Machine Learning Prediction request Successful response Prediction request Inference by AI Cloud Machine Learning main backup 5 sec limit something bad...
  33. 33. Result of Several Testings No errors at all! Total Error Rates 0.00% 0.50% 1.00% 1.50% No Backup Backup (minNode=10) Backup (minNode=20)
  34. 34. Confidential + Proprietary Results and Future
  35. 35. Good adoption from beginners Time#ofuses(beginnerclass) Event to promote deck building Release Deck Recommendation
  36. 36. Good adoption from beginners Win rate of beginners increased by 5% Time winrateatbattle (beginnerclass) Release Deck Recommendation
  37. 37. Latency The algorithm is complex, but the latency is low Thanks to GCP, especially auto-scaling, it is running error-free Sat 23 Latency 24ms 22ms 20ms 18ms 16ms Stable latency with most requests Sun 24 Mon 25 Tue 26 Wed 27 Thu 28 Mar 1 Sat 2 Sun 3 Mon 4 Tue 5 Wed 6Fri 22
  38. 38. Othellonia Dojo (AI Player) Preliminary Results ● Good enough win rate as a practice partner for beginners ● Dojo is used for beginners well ● Latency is stable and fast
  39. 39. Confidential + Proprietary Best practices Record logs that completely reproduce battles Creating headless game simulator Scalable system components (CMLE, GAE) Project structure & communication
  40. 40. How You Can Get Started With ML
  41. 41. Chat translation ● Increasingly a global gaming audience ● Connect users with emotes ● Chat enables deeper conversation ● Language doesn’t have to be a barrier
  42. 42. Player Toxicity ● People can be horrible to each other ● Bad actors ruin the experience for everyone ● Detecting toxic communications is a manual effort ● Real-time detection
  43. 43. Simplify Games ● QA & testing ● Player assistance ● Non-player characters
  44. 44. Confidential + Proprietary Thank you