Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS


Published on

There are many things to consider when architecting to scale a Kubernetes or Amazon EC2 Container Service (Amazon ECS), especially in heterogeneous environments, which are made up of different machine types and sizes. To increase a cluster’s efficiency, it is crucial to choose the right instance size and type for the right workload. In this talk, we discuss the two important concepts of k8s autoscaling: headroom and two-levels scaling. In addition, we review the different k8s deployment tools, including Kubernetes Operations (kops). This session is brought to you by AWS Partner, Spotinst.

  • Be the first to comment

DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS

  1. 1. Advanced Auto scaling and Deployment tools for Kubernetes & ECS Amiram Shachar Founder & CEO Spotinst
  2. 2. Founded ~2 years ago Spotinst Snapshot San Francisco Tel Aviv London 60+ Employees $17M raised Intel Capital 1000+ Customers
  3. 3. 1,500 VMs 100,000 VMs Jan 2016 Dec 2017 Spotinst powers workloads with cloud excess capacity Managed VMs by Spotinst, per month Ensures SLA 60-80% Cost SavingsThe Fastest Growing Company in Deloitte Fast 50 of 2017
  4. 4. Agenda Anatomy of k8s “Heterogenous” or “Tetris” scaling Scaling k8s - the old school way The problems of old school scaling k8s autoscaler concepts and implementation
  5. 5. Anatomy of k8s Kubernetes Scheduler Kubernetes Node Pod containercontainer Pod containercontainer Ingress Controller Kubernetes Node Pod containercontainer Pod containercontainer Kubernetes Node Pod containercontainer Pod containercontainer
  6. 6. 2 Layers Heterogenous Autoscaling c3.large c3.2xlarge m3.medInfrastructure Distributed Cluster Service auto scaling Infrastructure auto scaling Containers
  7. 7. How do you scale (Infrastructure Layer) today? When total Memory / CPU Reservation & Utilization meet a certain threshold
  8. 8. Scenario 1 - Wrong Pod Size c3.large c3.2xlargeInfrastructure Distributed Cluster Containers = 1 vCPU & 512mb RAM
  9. 9. Scenario 2 - Wrong instance type c3.2xlargeInfrastructure Distributed Cluster Containers
  10. 10. Scenario 3 - Wrong Scale Down Infrastructure Distributed Cluster Containers c3.2xlarge
  11. 11. k8s auto-scaler No scaling policies required Scale according to cluster needs )events) Fast scaling - don’t wait for thresholds, satisfy your tasks needs Scale down when instances are fragmented (But!) Keep headroom for incoming pods Head room is not reservation, but rather units of work
  12. 12. Catching the right events Insufficient Memory Insufficient CPU No nodes are available that match all of the following predicates:: Insufficient memory, PodToleratesNodeTaints. No nodes are available that match all of the following predicates:: Insufficient cpu, PodToleratesNodeTaints.
  13. 13. Scale up - Logic According to the pending Pods, sum the required amount of resources - CPU and memory (e.g 10 cores, 28 GB RAM) Determine the instance type that can handle the most resource- consuming task
  14. 14. Scale down - Defragmentation Look for idle instances - CPU & memory below 40% 3 consecutive periods of 1 min Make sure the pods running on this instance can run on other instances CPU RAM All pods’ ports are available on other instances Drain the instance Wait for the pods to be rescheduled on other instances Terminate the instance
  15. 15. Scale down - Defragmentation
  16. 16. Headroom A buffer of spare capacity that makes sure that when we want to scale more tasks, we don't have to wait for new instances. Headroom is defined as follows: unit : CPU and RAM. number of units.
  17. 17. Headroom - Example For example: Unit: 1024 CPU units & 512 MB RAM 5 units c3.2xlargeInfrastructure Distributed Cluster Containers
  18. 18. Utilized Cluster
  19. 19. Thank you We’re hiring! $ KOPS_FEATURE_FLAGS="+SpotinstCloudProvider" $ KOPS_CLOUD_PROVIDER="spotinst"