Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Riot Games 글로벌 게임 운영을 위한 Docker 및 Amazon ECS 활용사례 - AWS Summit Seoul 2017

687 views

Published on

Published in: Technology
  • Be the first to comment

Riot Games 글로벌 게임 운영을 위한 Docker 및 Amazon ECS 활용사례 - AWS Summit Seoul 2017

  1. 1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Adam Rozumek, Systems Engineer RIOT GAMES Standardizing Application Deployments Using Amazon ECS
  2. 2. RIOT GAMES Standardizing Application Deployments Using Amazon ECS
  3. 3. SYSTEMS ENGINEER ADAM ROZUMEK INTRODUCTIONS WHO AM I?
  4. 4. AMAZON ECS & VIDEO GAMES WHAT TO EXPECT ECS CONSOLIDATION WINS & LESSONS Adapting existing deployments & infrastructure INFRASTRUCTURE ENCAPSULATION Object Oriented Development Operations MULTI-GAME MODULAR SERVICE DESIGN A different kind of scaling 1 2 3
  5. 5. AMAZON ECS & VIDEO GAMES WHAT TO EXPECT ECS CONSOLIDATION WINS & LESSONS Adapting existing deployments & infrastructure INFRASTRUCTURE ENCAPSULATION Object Oriented Development Operations MULTI-GAME MODULAR SERVICE DESIGN A different kind of scaling 1 2 3
  6. 6. AMAZON ECS & VIDEO GAMES WHAT TO EXPECT ECS CONSOLIDATION WINS & LESSONS Adapting existing deployments & infrastructure INFRASTRUCTURE ENCAPSULATION Object Oriented Development Operations MULTI-GAME MODULAR SERVICE DESIGN A different kind of scaling 1 2 3
  7. 7. AMAZON ECS & VIDEO GAMES WHAT TO EXPECT ECS CONSOLIDATION WINS & LESSONS Adapting existing deployments & infrastructure INFRASTRUCTURE ENCAPSULATION Object Oriented Development Operations MULTI-GAME MODULAR SERVICE DESIGN A different kind of scaling 1 2 3
  8. 8. 7.5MILLION PEAK CONCURRENT PLAYERS 100MILLION MONTHLY ACTIVE PLAYERS MORE THAN 27MILLION DAILY ACTIVE PLAYERS MORE THAN MORE THAN 2016 LEAGUE OF LEGENDS STATS
  9. 9. DATA PRODUCTS & SERVICES OUR MISSION Empower teams at Riot to make timely, data-informed products by maintaining a scalable and reliable data platform AWS re:Invent 2015 | (GAM303) Riot Games: Migrating Mountains of Big Data to AWS Sean Maloney
  10. 10. PROBLEM
  11. 11. PROBLEM SCALING TOTAL OWNERSHIP ON AWS Total ownership We want to empower developers to: • Provision their own infrastructure • Execute their own deployments • Monitor their own metrics
  12. 12. PROBLEM SCALING TOTAL OWNERSHIP ON AWS Resource attribution • Who owns these EBS volumes? • What applications depend on these security groups? • Can these AMIs be deleted?
  13. 13. PROBLEM SCALING TOTAL OWNERSHIP ON AWS Security
  14. 14. PROBLEM SCALING TOTAL OWNERSHIP ON AWS Security • Auditing is important, but it’s reactive • Operational time sink Security Monkey AWS Trusted Advisor
  15. 15. PROBLEM SCALING TOTAL OWNERSHIP ON AWS Amazon EC2 Container Service
  16. 16. CONTAINERS
  17. 17. CONTAINERS STANDARDIZED APPLICATION UNITS… ON AWS • Dockerfiles capture application dependencies
  18. 18. CONTAINERS STANDARDIZED APPLICATION UNITS… ON AWS • Dockerfiles capture application dependencies • Common use cases have great community support
  19. 19. CONTAINERS STANDARDIZED APPLICATION UNITS… ON AWS • Dockerfiles capture application dependencies • Common use cases have great community support • Profit from our own engineering community
  20. 20. CONTAINERS STANDARDIZED APPLICATION UNITS… ON AWS
  21. 21. CONTAINERS STANDARDIZED APPLICATION UNITS… ON AWS Embrace the abstraction • Plan for failure at all levels • Avoid manual intervention whenever possible • It’s ephemeral all the way down
  22. 22. CONTAINERS STANDARDIZED APPLICATION UNITS… ON AWS Scheduling is hard We need to: • Quickly and fairly run tasks • Prevent resource conflicts • Provide reasonable fault tolerance
  23. 23. CONTAINERS STANDARDIZED APPLICATION UNITS… ON AWS Provisioning AWS hardware • Enable total ownership of the AWS resources backing our containers • Avoid the security, resource attribution, and convention degradation pitfalls
  24. 24. AMAZON ECS
  25. 25. AMAZON EC2 CONTAINER SERVICE STANDARDIZED APPLICATION UNITS… ON ECS! • ECS AMI provides all necessary software • Designed with AWS integration in mind • Free!
  26. 26. AMAZON EC2 CONTAINER SERVICE KEY FEATURE TIMELINE Nov 2014 ECS ANNOUNCED Re:Invent 2014
  27. 27. AMAZON EC2 CONTAINER SERVICE KEY FEATURE TIMELINE Nov 2014 April 2015 Dec 2015 August 2016 ECS ANNOUNCED Re:Invent 2014 ECS GENERALLY AVAILABLE ECR & NEW REGIONS ECS becomes available in the final missing region we need (Frankfurt) for our globally deployed applications
  28. 28. AMAZON EC2 CONTAINER SERVICE KEY FEATURE TIMELINE Nov 2014 April 2015 Dec 2015 May 2016 July 2016 August 2016 ECS ANNOUNCED Re:Invent 2014 ECS GENERALLY AVAILABLE ECR & NEW REGIONS ECS becomes available in the final missing region we need (Frankfurt) for our globally deployed applications SERVICE SCALING Automatic task count scaling based on CloudWatch metrics introduced
  29. 29. AMAZON EC2 CONTAINER SERVICE KEY FEATURE TIMELINE Nov 2014 April 2015 Dec 2015 May 2016 July 2016 August 2016 ECS ANNOUNCED Re:Invent 2014 ECS GENERALLY AVAILABLE ECR & NEW REGIONS ECS becomes available in the final missing region we need (Frankfurt) for our globally deployed applications SERVICE SCALING Automatic task count scaling based on CloudWatch metrics introduced TASK SPECIFIC IAM ROLES Unlocked a lot of cluster sharing potential
  30. 30. AMAZON EC2 CONTAINER SERVICE KEY FEATURE TIMELINE Nov 2014 April 2015 Dec 2015 May 2016 July 2016 August 2016 ECS ANNOUNCED Re:Invent 2014 ECS GENERALLY AVAILABLE ECR & NEW REGIONS ECS becomes available in the final missing region we need (Frankfurt) for our globally deployed applications SERVICE SCALING Automatic task count scaling based on CloudWatch metrics introduced TASK SPECIFIC IAM ROLES Unlocked a lot of cluster sharing potential APPLICATION LOAD BALANCERS Several key improvements for ECS
  31. 31. BEYOND ECS INFRASTRUCTURE AS CODE • At scale, orchestrating infrastructure in a consistent, reproducible way is key
  32. 32. BEYOND ECS INFRASTRUCTURE AS CODE • At scale, orchestrating infrastructure in a consistent, reproducible way is key Total ownership
  33. 33. PROVISIONING
  34. 34. VPC NAT Gateway Route 53 Hosted Zone Route Tables VPN Gateway VPC Internet Gateway Application Subnets Tools Instances Instances Instances Availability Zone C Availability Zone B Availability Zone A PROVISIONING INFRASTRUCTURE AS OBJECT-ORIENTED CODE
  35. 35. CASE STUDIES
  36. 36. ECS CLUSTER TERRAFORM BUILDING BLOCKS VPC NAT Gateway Route 53 Hosted Zone Route Tables VPN Gateway VPC Internet Gateway Application Subnets Tools Instances Instances Instances Availability Zone C Availability Zone B Availability Zone A ECS Cluster Autoscaling Group Security Group Security Group Security Group Instance Instance Instance Instance Instance Instance Launch Configuration User Data IAM Role CloudWatch Alarms
  37. 37. ECS CLUSTER TERRAFORM BUILDING BLOCKS Module 1 Module 2 Module 3
  38. 38. ECS CLUSTER TERRAFORM BUILDING BLOCKS
  39. 39. MICROSERVICES WITHOUT SERVICE ENDPOINTS ECS Cluster Autoscaling Group Security Group Security Group Security Group Instance Instance Instance Instance Instance Instance Launch Configuration User Data IAM Role CloudWatch Alarm ECS Service Task Definition CloudWatch Alarms IAM Role
  40. 40. MICROSERVICES WITH SERVICE ENDPOINTS ECS Cluster ECS Service Task Definition CloudWatch Alarms IAM Role Application Load Balancer Monitoring CloudWatch Alarms SNS Topics Security Group Security Group Listeners Target Groups Route 53
  41. 41. PERSISTENT DATA LOSE ECS HOSTS WITHOUT LOSING DATA ECS Cluster ECS Service Application Load Balancer Monitoring CloudWatch Alarms SNS Topics Security Group Security Group Listeners Target Groups Route 53 Attachment Group EBS EBS EBS Elastic Network Interface Elastic IP
  42. 42. LESSONS LEARNED
  43. 43. LESSONS WHAT WORKED FOR US • Break apart your stacks ECS Cluster
  44. 44. LESSONS WHAT WORKED FOR US • Break apart your stacks, but don’t overdo it ECS Service ALB SNS Topics Route 53 IAM Role Task Definition CloudWatch Alarms
  45. 45. LESSONS WHAT WORKED FOR US • Be liberal with your cluster provisioning • Don’t risk resource contention in production • With good orchestration, additional clusters != additional operational overhead
  46. 46. LESSONS WHAT WORKED FOR US • Tag everything all of the time • Keep your tags organized in your Terraform templates • Have top level variables that get applied to every resource • Create a tag for every dimension that is useful to your business
  47. 47. LESSONS WHAT WORKED FOR US • Centralize your logs or Amazon CloudWatch Logs
  48. 48. LESSONS WHAT WORKED FOR US • Stay up to date with release blogs and application updates • ECS updates on the AWS blog
  49. 49. LESSONS WHAT WORKED FOR US • Capture your AMI generation process
  50. 50. LESSONS WHAT WORKED FOR US • Profile your memory requirements, monitor for scheduling issues
  51. 51. RIOT ENGINEERING BLOG THESE PROBLEMS AND MORE https://engineering.riotgames.com/ Riot engineering How we use data http://na.leagueoflegends.com/en/tag/insights
  52. 52. 함께 해주셔서 감사합니다!

×