AWS Webcast - Database in the Cloud Series - Scalable Games and Analytics with AWS

8,052 views

Published on

Discover how mobile, AAA, and MMO game companies are leveraging Amazon Web Services to build cost-effective and scalable games.

In this session we will show you how to:
• Build leaderboards that can scale to very high user counts using DynamoDB and ElastiCache
• Host low-latency online game servers on the new network-enhanced C3 instance
• Stream game telemetry to the cloud in real-time using Kinesis
• Analyze player data with Redshift and Elastic MapReduce

AWS Webcast - Database in the Cloud Series - Scalable Games and Analytics with AWS

  1. 1. AWS Gaming Solutions Scalable Games and Analytics with AWS Nate Wiger @nateware Principal Gaming Solutions Architect April 2014 Artist: Abel Oroz Vicente
  2. 2. AWS Gaming Solutions What's In It For Me? •  Why AWS for Games? •  Core Game Backend •  Scaling with DynamoDB •  Low-Latency Multiplayer with C3 •  Game Analytics with Redshift and Kinesis
  3. 3. AWS Gaming Solutions Gratuitous Logo Slide
  4. 4. AWS Gaming Solutions Handle Viral Spikes
  5. 5. AWS Gaming Solutions 10 Regions 51 CloudFront POPs Continuous Expansion Global is Good
  6. 6. AWS Gaming Solutions Big picture slide
  7. 7. AWS Gaming Solutions Elastic Beanstalk •  Managed Container •  6 Languages •  Git Push or Zip Upload •  ELB, EC2, DB, DNS •  Web Dashboard •  Same Performance •  Full Access •  So Yeah, Use It
  8. 8. AWS Gaming Solutions Hill Of Beans ELB S3 •  Beanstalk Manages •  ELB •  EC2 •  Auto Scaling •  Monitoring •  RDS •  Add Other Services •  S3 •  CloudFront •  SNS Mobile Push EC2 Elastic Beanstalk Container EC2 CloudFront CDN
  9. 9. AWS Gaming Solutions
  10. 10. AWS Gaming Solutions Push It cd  MyGame   eb  init     vi  app.rb   require  'sinatra'   get  '/hello.json'  do      {message:  "Hello  World!"}   end     git  commit  -­‐m  "app  updates"  -­‐a   git  aws.push   •  Initialize Beanstalk •  Write Code •  Git Commit •  Git Push to Beanstalk •  Coffee / Beer •  Rinse, Repeat
  11. 11. AWS Gaming Solutions http://bit.ly/ubisoft-aws
  12. 12. AWS Gaming Solutions
  13. 13. AWS Gaming Solutions Region Writing is Painful Availability Zone A Availability Zone B S3 EC2 •  Games are Write Heavy •  Constant State Updates •  Key Value Key Value •  Binary Data •  Caching of Limited Use •  Database = Bottleneck ELB EC2 CloudFront CDN
  14. 14. AWS Gaming Solutions DynamoDB Availability Zone A Availability Zone B S3 •  NoSQL datastore •  Fully-managed •  Provisioned throughput •  PUT/GET keys •  Secondary indexes •  Handles any SANE Gaming use case EC2 EC2 ELB CloudFront CDN Elastic Beanstalk Container
  15. 15. AWS Gaming Solutions DynamoDB: Low Latency at Scale Docs, SDK's: http://aws.amazon.com/dynamodb/developer-resources/ Auto Scaling: https://github.com/sebdah/dynamic-dynamodb
  16. 16. AWS Gaming Solutions DynamoDB: Durable at Scale WRITES Replicated to 2 AZ’s before OK Persisted to disk (SSD) READS Strong or eventual consistency No trade-off in latency
  17. 17. AWS Gaming Solutions Leaderboard in DynamoDB •  Hash key = Primary key •  Range key = Sub key •  Others attributes are unstructured •  How to sort based on High Score per Game? •  Secondary Index on Game + Score
  18. 18. AWS Gaming Solutions Python DynamoDB Leaderboard table  =  Table('scores',  schema=[      HashKey('user'),      RangeKey('level')   ],   throughput={      'read':  5,  'write':  15   },  global_indexes=[      GlobalAllIndex('highscore',          parts=[              HashKey('level'),              RangeKey('score',  data_type=NUMBER)          ],          throughput={              'read':  5,  'write':  15          }      )   ])   new_score  =      Item(table,  data={          'user':    user,          'level':  level,          'score':  score        })   new_score.save()     topscores  =  table.query(index='highscore')     for  ts  in  topscores:      print(ts['user'],  ts['score'])    
  19. 19. AWS Gaming Solutions
  20. 20. AWS Gaming Solutions
  21. 21. AWS Gaming Solutions EC2: New C3 Instance •  High packets per second •  Low latency, no jitter •  Intel Ivy Bridge CPU •  SSD's •  Built for games •  10.5 cents / hour
  22. 22. AWS Gaming Solutions Enhanced Networking (SR-IOV) Before: Hypervisor After: Hardware
  23. 23. AWS Gaming Solutions Step 1: HVM Machine Image
  24. 24. AWS Gaming Solutions Step 2: Choose C3 Instance
  25. 25. AWS Gaming Solutions Step 3: Launch in VPC
  26. 26. AWS Gaming Solutions Step 3: Launch in VPC
  27. 27. AWS Gaming Solutions Recap: Enabling SR-IOV ①  Use HVM image (Linux or Windows) ②  Select C3 instance ③  Must launch in VPC ④  Verify with CLI aws  ec2  describe-­‐instance-­‐attribute        -­‐-­‐attribute  sriovNetSupport  -­‐-­‐instance-­‐id  i-­‐12345678   ⑤  Full docs: http://bit.ly/ec2-enhanced-net
  28. 28. AWS Gaming Solutions EC2: New G2 Instance •  NVIDIA Kepler GPU •  Game Streaming •  Rendering Assets •  Build Servers •  AppStream! EVE Online: http://bit.ly/appstream-eve-online
  29. 29. AWS Gaming Solutions http://bit.ly/red5-aws
  30. 30. AWS Gaming Solutions http://bit.ly/rovio-aws
  31. 31. AWS Gaming Solutions Analyze What? Emotions •  Enjoying game •  Engaged •  Like/dislike new content •  Stuck on a level •  Bored •  Abandonment Behaviors •  Hours played day/week •  Number of sessions/day •  Level progression •  Friend invites/referrals •  Response to mobile push •  Money spent/week
  32. 32. AWS Gaming Solutions Analytics Flow Ingest Store Process Analyze Profit!
  33. 33. AWS Gaming Solutions Off The Shelf Analytics •  Easy To Integrate •  Pre-Baked Reports •  Rate Limits •  Retention Windows •  Data Lock-In
  34. 34. AWS Gaming Solutions •  Write Events File on Device •  Periodically Upload to S3 •  Process into Redshift •  Point GUI Tool to Redshift Start Simple 2014-­‐01-­‐24,nateware,e4df,login   2014-­‐01-­‐24,nateware,e4df,gamestart   2014-­‐01-­‐24,nateware,e4df,gameend   2014-­‐01-­‐25,nateware,a88c,login   2014-­‐01-­‐25,nateware,a88c,friendlist   2014-­‐01-­‐25,nateware,a88c,gamestart   Profit!
  35. 35. AWS Gaming Solutions Tableau + Redshift http://bit.ly/tableau-redshift
  36. 36. AWS Gaming Solutions •  Also Collect Server Logs •  Periodically Upload to S3 •  Stuff into Redshift •  External Analytics Data Too More Data Sources EC2 External Analytics
  37. 37. AWS Gaming Solutions •  Integrate Game DB •  Load Directly into Redshift •  Redshift does Intelligent Merge •  Tracks DynamoDB Columns Direct From DynamoDB EC2
  38. 38. AWS Gaming Solutions Real-Time Event Stream 2014-­‐03-­‐17T09:52:08-­‐07:00,nateware,e4b5,login   2014-­‐03-­‐17T09:52:54-­‐07:00,nateware,e4b5,gamestart   2014-­‐03-­‐17T09:53:15-­‐07:00,nateware,e4b5,levelup   2014-­‐03-­‐17T09:54:06-­‐07:00,nateware,e4b5,gameend   2014-­‐03-­‐17T09:54:23-­‐07:00,nateware,30a4,gamestart   2014-­‐03-­‐17T09:55:14-­‐07:00,nateware,30a4,gameend   2014-­‐03-­‐17T09:55:41-­‐07:00,nateware,30a4,gamestart   2014-­‐03-­‐17T09:57:12-­‐07:00,nateware,6ebd,levelup   2014-­‐03-­‐17T09:58:50-­‐07:00,nateware,6ebd,levelup   2014-­‐03-­‐17T09:59:52-­‐07:00,nateware,6ebd,gameend    
  39. 39. AWS Gaming Solutions •  Stream Data With Kinesis •  Multiple Writers and Readers •  Still Output to Redshift Speeding Towards Real-Time EC2
  40. 40. AWS Gaming Solutions •  Stream Data With Kinesis •  Multiple Writers and Readers •  Still Output to Redshift •  Stream to Spark on EMR •  Storm via Kinesis Spout •  Custom EC2 Workers Speeding Towards Real-Time EC2 EC2
  41. 41. AWS Gaming Solutions Wrap It Up Already •  Start Simple With Beanstalk •  Go Directly to DynamoDB, Do Not Pass Go •  Leverage the EC2 C3 Instance •  Analytics 1-2-3: CSV => S3 => Redshift •  Kinesis + Spark/EMR for Real-Time Analytics
  42. 42. AWS Gaming Solutions http://bit.ly/minecraft-aws

×