SlideShare a Scribd company logo
1 of 19
Download to read offline
Advanced Auto scaling and Deployment tools for
Kubernetes & ECS
Amiram Shachar
Founder & CEO
Spotinst
Founded
~2 years
ago
Spotinst Snapshot
San Francisco
Tel Aviv
London
60+
Employees
$17M
raised
Intel
Capital
1000+
Customers
1,500 VMs
100,000 VMs
Jan 2016 Dec 2017
Spotinst powers workloads with cloud excess capacity
Managed VMs by Spotinst, per month
Ensures SLA
60-80% Cost SavingsThe Fastest
Growing
Company in
Deloitte Fast 50
of 2017
Agenda
Anatomy of k8s
“Heterogenous” or “Tetris” scaling
Scaling k8s - the old school way
The problems of old school scaling
k8s autoscaler concepts and implementation
Anatomy of k8s
Kubernetes Scheduler
Kubernetes Node
Pod
containercontainer
Pod
containercontainer
Ingress Controller
Kubernetes Node
Pod
containercontainer
Pod
containercontainer
Kubernetes Node
Pod
containercontainer
Pod
containercontainer
2 Layers Heterogenous Autoscaling
c3.large c3.2xlarge m3.medInfrastructure
Distributed Cluster
Service auto scaling
Infrastructure auto scaling
Containers
How do you scale (Infrastructure Layer)
today?
When total Memory / CPU Reservation & Utilization meet a certain
threshold
Scenario 1 - Wrong Pod Size
c3.large c3.2xlargeInfrastructure
Distributed Cluster
Containers
m3.med
= 1 vCPU & 512mb RAM
Scenario 2 - Wrong instance type
m3.med c3.2xlargeInfrastructure
Distributed Cluster
Containers
m3.med
Scenario 3 - Wrong Scale Down
Infrastructure
Distributed Cluster
Containers
m3.medm3.med c3.2xlarge
k8s auto-scaler
No scaling policies required
Scale according to cluster needs )events)
Fast scaling - don’t wait for thresholds, satisfy your tasks needs
Scale down when instances are fragmented (But!)
Keep headroom for incoming pods
Head room is not reservation, but rather units of work
Catching the right events
Insufficient Memory
Insufficient CPU
No nodes are available that match all of the following
predicates:: Insufficient memory, PodToleratesNodeTaints.
No nodes are available that match all of the following
predicates:: Insufficient cpu, PodToleratesNodeTaints.
Scale up - Logic
According to the pending Pods, sum the required amount of resources -
CPU and memory (e.g 10 cores, 28 GB RAM)
Determine the instance type that can handle the most resource-
consuming task
Scale down - Defragmentation
Look for idle instances - CPU & memory below 40%
3 consecutive periods of 1 min
Make sure the pods running on this instance can run on other
instances
CPU
RAM
All pods’ ports are available on other instances
Drain the instance
Wait for the pods to be rescheduled on other instances
Terminate the instance
Scale down - Defragmentation
Headroom
A buffer of spare capacity that makes sure that when we want to
scale more tasks, we don't have to wait for new instances.
Headroom is defined as follows:
unit : CPU and RAM.
number of units.
Headroom - Example
For example:
Unit: 1024 CPU units & 512 MB RAM
5 units
c3.2xlargeInfrastructure
Distributed Cluster
Containers
m3.medm3.med m3.med
Utilized Cluster
Thank you
amiram@spotinst.com
We’re
hiring!
$ KOPS_FEATURE_FLAGS="+SpotinstCloudProvider"
$ KOPS_CLOUD_PROVIDER="spotinst"

More Related Content

What's hot

[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...Amazon Web Services
 
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...Amazon Web Services
 
Builders' Day - Building Data Lakes for Analytics On AWS LC
Builders' Day - Building Data Lakes for Analytics On AWS LCBuilders' Day - Building Data Lakes for Analytics On AWS LC
Builders' Day - Building Data Lakes for Analytics On AWS LCAmazon Web Services LATAM
 
Hands-On with Amazon ElastiCache for Redis - Workshop (DAT309-R1) - AWS re:In...
Hands-On with Amazon ElastiCache for Redis - Workshop (DAT309-R1) - AWS re:In...Hands-On with Amazon ElastiCache for Redis - Workshop (DAT309-R1) - AWS re:In...
Hands-On with Amazon ElastiCache for Redis - Workshop (DAT309-R1) - AWS re:In...Amazon Web Services
 
Backing Up Amazon EC2 with Amazon EBS Snapshots (CMP301-R1) - AWS re:Invent 2018
Backing Up Amazon EC2 with Amazon EBS Snapshots (CMP301-R1) - AWS re:Invent 2018Backing Up Amazon EC2 with Amazon EBS Snapshots (CMP301-R1) - AWS re:Invent 2018
Backing Up Amazon EC2 with Amazon EBS Snapshots (CMP301-R1) - AWS re:Invent 2018Amazon Web Services
 
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018Amazon Web Services
 
Developing with .NET Core on AWS: What's New (DEV318-R1) - AWS re:Invent 2018
Developing with .NET Core on AWS: What's New (DEV318-R1) - AWS re:Invent 2018Developing with .NET Core on AWS: What's New (DEV318-R1) - AWS re:Invent 2018
Developing with .NET Core on AWS: What's New (DEV318-R1) - AWS re:Invent 2018Amazon Web Services
 
Amazon EC2 Instances, Featuring Performance Optimisation Best Practices
Amazon EC2 Instances, Featuring Performance Optimisation Best PracticesAmazon EC2 Instances, Featuring Performance Optimisation Best Practices
Amazon EC2 Instances, Featuring Performance Optimisation Best PracticesAmazon Web Services
 
Automating Backup and Archiving on AWS with Commvault (STG358) - AWS re:Inven...
Automating Backup and Archiving on AWS with Commvault (STG358) - AWS re:Inven...Automating Backup and Archiving on AWS with Commvault (STG358) - AWS re:Inven...
Automating Backup and Archiving on AWS with Commvault (STG358) - AWS re:Inven...Amazon Web Services
 
Amazon EC2 and Amazon VPC Hands-on Workshop
Amazon EC2 and Amazon VPC Hands-on WorkshopAmazon EC2 and Amazon VPC Hands-on Workshop
Amazon EC2 and Amazon VPC Hands-on WorkshopAmazon Web Services
 
Amazon Relational Database Service – How is it different to what you do today ?
Amazon Relational Database Service – How is it different to what you do today ?Amazon Relational Database Service – How is it different to what you do today ?
Amazon Relational Database Service – How is it different to what you do today ?Amazon Web Services
 
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018Amazon Web Services
 
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...Amazon Web Services
 
Transform Your Organization with Real Real-Time Monitoring
Transform Your Organization with Real Real-Time MonitoringTransform Your Organization with Real Real-Time Monitoring
Transform Your Organization with Real Real-Time MonitoringAmazon Web Services
 
Deep Dive on PostgreSQL Databases on Amazon RDS (DAT324) - AWS re:Invent 2018
Deep Dive on PostgreSQL Databases on Amazon RDS (DAT324) - AWS re:Invent 2018Deep Dive on PostgreSQL Databases on Amazon RDS (DAT324) - AWS re:Invent 2018
Deep Dive on PostgreSQL Databases on Amazon RDS (DAT324) - AWS re:Invent 2018Amazon Web Services
 
Accelerate Analytics at Scale with Amazon EMR - AWS Summit Sydney 2018
Accelerate Analytics at Scale with Amazon EMR - AWS Summit Sydney 2018Accelerate Analytics at Scale with Amazon EMR - AWS Summit Sydney 2018
Accelerate Analytics at Scale with Amazon EMR - AWS Summit Sydney 2018Amazon Web Services
 
Serverlesss Big Data Analytics with Amazon Athena and Quicksight
Serverlesss Big Data Analytics with Amazon Athena and QuicksightServerlesss Big Data Analytics with Amazon Athena and Quicksight
Serverlesss Big Data Analytics with Amazon Athena and QuicksightAmazon Web Services
 

What's hot (20)

[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
 
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
 
Oracle on AWS
Oracle on AWSOracle on AWS
Oracle on AWS
 
Builders' Day - Building Data Lakes for Analytics On AWS LC
Builders' Day - Building Data Lakes for Analytics On AWS LCBuilders' Day - Building Data Lakes for Analytics On AWS LC
Builders' Day - Building Data Lakes for Analytics On AWS LC
 
Hands-On with Amazon ElastiCache for Redis - Workshop (DAT309-R1) - AWS re:In...
Hands-On with Amazon ElastiCache for Redis - Workshop (DAT309-R1) - AWS re:In...Hands-On with Amazon ElastiCache for Redis - Workshop (DAT309-R1) - AWS re:In...
Hands-On with Amazon ElastiCache for Redis - Workshop (DAT309-R1) - AWS re:In...
 
Backing Up Amazon EC2 with Amazon EBS Snapshots (CMP301-R1) - AWS re:Invent 2018
Backing Up Amazon EC2 with Amazon EBS Snapshots (CMP301-R1) - AWS re:Invent 2018Backing Up Amazon EC2 with Amazon EBS Snapshots (CMP301-R1) - AWS re:Invent 2018
Backing Up Amazon EC2 with Amazon EBS Snapshots (CMP301-R1) - AWS re:Invent 2018
 
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
 
Developing with .NET Core on AWS: What's New (DEV318-R1) - AWS re:Invent 2018
Developing with .NET Core on AWS: What's New (DEV318-R1) - AWS re:Invent 2018Developing with .NET Core on AWS: What's New (DEV318-R1) - AWS re:Invent 2018
Developing with .NET Core on AWS: What's New (DEV318-R1) - AWS re:Invent 2018
 
Amazon EC2 Instances, Featuring Performance Optimisation Best Practices
Amazon EC2 Instances, Featuring Performance Optimisation Best PracticesAmazon EC2 Instances, Featuring Performance Optimisation Best Practices
Amazon EC2 Instances, Featuring Performance Optimisation Best Practices
 
Automating Backup and Archiving on AWS with Commvault (STG358) - AWS re:Inven...
Automating Backup and Archiving on AWS with Commvault (STG358) - AWS re:Inven...Automating Backup and Archiving on AWS with Commvault (STG358) - AWS re:Inven...
Automating Backup and Archiving on AWS with Commvault (STG358) - AWS re:Inven...
 
Amazon EC2 and Amazon VPC Hands-on Workshop
Amazon EC2 and Amazon VPC Hands-on WorkshopAmazon EC2 and Amazon VPC Hands-on Workshop
Amazon EC2 and Amazon VPC Hands-on Workshop
 
Amazon Relational Database Service – How is it different to what you do today ?
Amazon Relational Database Service – How is it different to what you do today ?Amazon Relational Database Service – How is it different to what you do today ?
Amazon Relational Database Service – How is it different to what you do today ?
 
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
 
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
Migrating Massive Databases and Data Warehouses to the Cloud - ENT327 - re:In...
 
Transform Your Organization with Real Real-Time Monitoring
Transform Your Organization with Real Real-Time MonitoringTransform Your Organization with Real Real-Time Monitoring
Transform Your Organization with Real Real-Time Monitoring
 
Deep Dive on PostgreSQL Databases on Amazon RDS (DAT324) - AWS re:Invent 2018
Deep Dive on PostgreSQL Databases on Amazon RDS (DAT324) - AWS re:Invent 2018Deep Dive on PostgreSQL Databases on Amazon RDS (DAT324) - AWS re:Invent 2018
Deep Dive on PostgreSQL Databases on Amazon RDS (DAT324) - AWS re:Invent 2018
 
Builders' Day- Mastering Kubernetes on AWS
Builders' Day- Mastering Kubernetes on AWSBuilders' Day- Mastering Kubernetes on AWS
Builders' Day- Mastering Kubernetes on AWS
 
EC2 and VPC Workshop
EC2 and VPC WorkshopEC2 and VPC Workshop
EC2 and VPC Workshop
 
Accelerate Analytics at Scale with Amazon EMR - AWS Summit Sydney 2018
Accelerate Analytics at Scale with Amazon EMR - AWS Summit Sydney 2018Accelerate Analytics at Scale with Amazon EMR - AWS Summit Sydney 2018
Accelerate Analytics at Scale with Amazon EMR - AWS Summit Sydney 2018
 
Serverlesss Big Data Analytics with Amazon Athena and Quicksight
Serverlesss Big Data Analytics with Amazon Athena and QuicksightServerlesss Big Data Analytics with Amazon Athena and Quicksight
Serverlesss Big Data Analytics with Amazon Athena and Quicksight
 

Similar to DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS

Advanced Autoscaling for Kubernetes & Amazon ECS
Advanced Autoscaling for Kubernetes & Amazon ECS Advanced Autoscaling for Kubernetes & Amazon ECS
Advanced Autoscaling for Kubernetes & Amazon ECS Amazon Web Services
 
Advanced Autoscaling for Kubernetes & Amazon ECS - DEM05 - Anaheim AWS Summit
Advanced Autoscaling for Kubernetes & Amazon ECS - DEM05 - Anaheim AWS SummitAdvanced Autoscaling for Kubernetes & Amazon ECS - DEM05 - Anaheim AWS Summit
Advanced Autoscaling for Kubernetes & Amazon ECS - DEM05 - Anaheim AWS SummitAmazon Web Services
 
Deep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceDeep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceAmazon Web Services
 
Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...
Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...
Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...Amazon Web Services
 
Deep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceDeep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceAmazon Web Services
 
Deep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instancesDeep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instancesAmazon Web Services
 
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...Amazon Web Services
 
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...Amazon Web Services
 
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...Amazon Web Services
 
Kubernetes: Reducing Infrastructure Cost & Complexity
Kubernetes: Reducing Infrastructure Cost & ComplexityKubernetes: Reducing Infrastructure Cost & Complexity
Kubernetes: Reducing Infrastructure Cost & ComplexityDevOps.com
 
Super scaling singleton inserts
Super scaling singleton insertsSuper scaling singleton inserts
Super scaling singleton insertsChris Adkin
 
Deep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceDeep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceAmazon Web Services
 
CMP301_Deep Dive on Amazon EC2 Instances
CMP301_Deep Dive on Amazon EC2 InstancesCMP301_Deep Dive on Amazon EC2 Instances
CMP301_Deep Dive on Amazon EC2 InstancesAmazon Web Services
 
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-FinalSizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-FinalVigyan Jain
 
Cassandra in Operation
Cassandra in OperationCassandra in Operation
Cassandra in Operationniallmilton
 
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech TalksDeep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech TalksAmazon Web Services
 
AWS Webcast - An Introduction to High Performance Computing on AWS
AWS Webcast - An Introduction to High Performance Computing on AWSAWS Webcast - An Introduction to High Performance Computing on AWS
AWS Webcast - An Introduction to High Performance Computing on AWSAmazon Web Services
 
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian KöhntoppOSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian KöhntoppNETWAYS
 
Presto at Tivo, Boston Hadoop Meetup
Presto at Tivo, Boston Hadoop MeetupPresto at Tivo, Boston Hadoop Meetup
Presto at Tivo, Boston Hadoop MeetupJustin Borgman
 

Similar to DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS (20)

Advanced Autoscaling for Kubernetes & Amazon ECS
Advanced Autoscaling for Kubernetes & Amazon ECS Advanced Autoscaling for Kubernetes & Amazon ECS
Advanced Autoscaling for Kubernetes & Amazon ECS
 
Advanced Autoscaling for Kubernetes & Amazon ECS - DEM05 - Anaheim AWS Summit
Advanced Autoscaling for Kubernetes & Amazon ECS - DEM05 - Anaheim AWS SummitAdvanced Autoscaling for Kubernetes & Amazon ECS - DEM05 - Anaheim AWS Summit
Advanced Autoscaling for Kubernetes & Amazon ECS - DEM05 - Anaheim AWS Summit
 
Deep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceDeep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance Performance
 
Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...
Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...
Choosing the Right EC2 Instance and Applicable Use Cases - AWS June 2016 Webi...
 
Deep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceDeep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance Performance
 
Deep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instancesDeep Dive on Amazon EC2 instances
Deep Dive on Amazon EC2 instances
 
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
 
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
 
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
SRV402 Deep Dive on Amazon EC2 Instances, Featuring Performance Optimization ...
 
Kubernetes: Reducing Infrastructure Cost & Complexity
Kubernetes: Reducing Infrastructure Cost & ComplexityKubernetes: Reducing Infrastructure Cost & Complexity
Kubernetes: Reducing Infrastructure Cost & Complexity
 
Super scaling singleton inserts
Super scaling singleton insertsSuper scaling singleton inserts
Super scaling singleton inserts
 
Deep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceDeep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance Performance
 
Deep Dive on Amazon EC2
Deep Dive on Amazon EC2Deep Dive on Amazon EC2
Deep Dive on Amazon EC2
 
CMP301_Deep Dive on Amazon EC2 Instances
CMP301_Deep Dive on Amazon EC2 InstancesCMP301_Deep Dive on Amazon EC2 Instances
CMP301_Deep Dive on Amazon EC2 Instances
 
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-FinalSizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
Sizing MongoDB on AWS with Wired Tiger-Patrick and Vigyan-Final
 
Cassandra in Operation
Cassandra in OperationCassandra in Operation
Cassandra in Operation
 
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech TalksDeep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
 
AWS Webcast - An Introduction to High Performance Computing on AWS
AWS Webcast - An Introduction to High Performance Computing on AWSAWS Webcast - An Introduction to High Performance Computing on AWS
AWS Webcast - An Introduction to High Performance Computing on AWS
 
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian KöhntoppOSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
OSDC 2017 | Something Openshift Kubernetes Containers by Kristian Köhntopp
 
Presto at Tivo, Boston Hadoop Meetup
Presto at Tivo, Boston Hadoop MeetupPresto at Tivo, Boston Hadoop Meetup
Presto at Tivo, Boston Hadoop Meetup
 

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS

  • 1. Advanced Auto scaling and Deployment tools for Kubernetes & ECS Amiram Shachar Founder & CEO Spotinst
  • 2. Founded ~2 years ago Spotinst Snapshot San Francisco Tel Aviv London 60+ Employees $17M raised Intel Capital 1000+ Customers
  • 3. 1,500 VMs 100,000 VMs Jan 2016 Dec 2017 Spotinst powers workloads with cloud excess capacity Managed VMs by Spotinst, per month Ensures SLA 60-80% Cost SavingsThe Fastest Growing Company in Deloitte Fast 50 of 2017
  • 4. Agenda Anatomy of k8s “Heterogenous” or “Tetris” scaling Scaling k8s - the old school way The problems of old school scaling k8s autoscaler concepts and implementation
  • 5. Anatomy of k8s Kubernetes Scheduler Kubernetes Node Pod containercontainer Pod containercontainer Ingress Controller Kubernetes Node Pod containercontainer Pod containercontainer Kubernetes Node Pod containercontainer Pod containercontainer
  • 6. 2 Layers Heterogenous Autoscaling c3.large c3.2xlarge m3.medInfrastructure Distributed Cluster Service auto scaling Infrastructure auto scaling Containers
  • 7. How do you scale (Infrastructure Layer) today? When total Memory / CPU Reservation & Utilization meet a certain threshold
  • 8. Scenario 1 - Wrong Pod Size c3.large c3.2xlargeInfrastructure Distributed Cluster Containers m3.med = 1 vCPU & 512mb RAM
  • 9. Scenario 2 - Wrong instance type m3.med c3.2xlargeInfrastructure Distributed Cluster Containers m3.med
  • 10. Scenario 3 - Wrong Scale Down Infrastructure Distributed Cluster Containers m3.medm3.med c3.2xlarge
  • 11. k8s auto-scaler No scaling policies required Scale according to cluster needs )events) Fast scaling - don’t wait for thresholds, satisfy your tasks needs Scale down when instances are fragmented (But!) Keep headroom for incoming pods Head room is not reservation, but rather units of work
  • 12. Catching the right events Insufficient Memory Insufficient CPU No nodes are available that match all of the following predicates:: Insufficient memory, PodToleratesNodeTaints. No nodes are available that match all of the following predicates:: Insufficient cpu, PodToleratesNodeTaints.
  • 13. Scale up - Logic According to the pending Pods, sum the required amount of resources - CPU and memory (e.g 10 cores, 28 GB RAM) Determine the instance type that can handle the most resource- consuming task
  • 14. Scale down - Defragmentation Look for idle instances - CPU & memory below 40% 3 consecutive periods of 1 min Make sure the pods running on this instance can run on other instances CPU RAM All pods’ ports are available on other instances Drain the instance Wait for the pods to be rescheduled on other instances Terminate the instance
  • 15. Scale down - Defragmentation
  • 16. Headroom A buffer of spare capacity that makes sure that when we want to scale more tasks, we don't have to wait for new instances. Headroom is defined as follows: unit : CPU and RAM. number of units.
  • 17. Headroom - Example For example: Unit: 1024 CPU units & 512 MB RAM 5 units c3.2xlargeInfrastructure Distributed Cluster Containers m3.medm3.med m3.med