In this session, Kevin will dive into the unique challenges of keeping your Kubernetes workloads highly available while keeping costs low. You will learn about how to leverage cloud-native autoscaling, pod requirement right-sizing, resource buffer definition, cost allocation and more.

1. Kubernetes: Reducing
Infrastructure Cost
& Complexity

2. Anatomy of Kubernetes
● Control Plane
● Data Plane (Worker Nodes)
● Kubelet (“node agent”)
● Services (Ingress Controllers)
UICLI
SCHEDULER
Data Plane ManagementControl Plane

3. Kubernetes | 2 Layers of Scaling
1. IaaS
2. Pods
Kubernetes
Infrastructure as a Service
M3.MED
C5.2XLARGE M5.LARGEM3.MED
Intel® Xeon® E5 Intel® Xeon® Scalable Processor Intel® Xeon® E5

4. Horizontal Pod Auto Scaling
● HPA checks metrics every 30s
● HPA updates the # of replicas inside the Deployment or RC
● 3/5 minutes cooldown for scale up/down

5. Vertical Pod Auto Scaling
● VPA checks metrics every 10s
● VPA updates the resources inside the Deployment or RC
● Pods are restarted
● 3/5 minutes cooldown for scale up/down
VPA

6. The Problem
Meeting Application Demands Minimizing Idle Resources
● Instance Sizing
● Instance Life Cycle
● Pod Right Sizing

7. Tetris Scaling Challenges

8. Kubernetes
Infrastructure as a Service
C5.LARGE
C5.2XLARGE
M5.LARGE
Intel® Xeon® Scalable Processor
SKYLAKE
Intel® Xeon® E5
SKYLAKE
Intel® Xeon® E5
Scale Up - Pod Size to Node Mismatch
Containers
3vCPU / 4GiB RAMPending

9. Kubernetes
Infrastructure as a Service
C5.LARGE C5.2XLARGE
M3.MEDIUMIntel® Xeon® Scalable Processor
SKYLAKE
Intel® Xeon® E5
SKYLAKE
Scale Up - Wrong Node Type
Containers
3vCPU / 4GiB RAMPending

10. Kubernetes
Infrastructure as a Service
C5.LARGE M5.LARGE
C5.2XLARGE
Intel® Xeon® Scalable Processor
SKYLAKE
Intel® Xeon® E5
SKYLAKE Intel® Xeon® E5
Wrong Node Scale Down
Containers

11. A Different Approach:
Container Driven Scaling
Stop thinking about nodes and infrastructure!

12. Containers First Philosophy
Instance size, type & pricing is determined based on the Pod/Containers spec while honoring Labels, Taints,
Tolerations, Network & Storage requirements.
GPU
ON
DEMAND
G2.XL
C5.4XL
ON DEMAND
Pending Pods
T2.MED
C5.2XL
C5.XL
Instances Pending Pods Instances

13. Kubernetes Data-plane management
Data Plane Management
Control Plane

14. Achieving a Serverless or “Nodeless” Experience
What does it mean?
● No management of underlying infrastructure
● Scale by request based on real time requirements
● Utility billing - Pay only for what you use, eliminate waste
● Scale to zero - scale all the way down to 0
● Fast scaling - do not wait for thresholds, satisfy container needs immediately

15. Show Me the Numbers!
Example 1: Pod requires 6,500mb of memory and 1.5vCPUs – Different Instance Size
Pod Count Pod M Instance M Memory % Pod vCPU Instance vCPU CPU %
Cost Per Pod
(OD hourly)
1 6,500 8,000 81.25% 1.5 2 75% $0.096m5.large
2 13,000 16,000 81.25% 3 4 75% $0.096m5.xlarge
5 31,500 32,000 98.44% 7.5 8 93.75% $0.077m5.2xlarge
As seen in the chart above, by going a few sizes up, we can
achieve better resource allocation and reduce the cost by 20
percent.

16. Show Me the Numbers!
Example 2: Pod of 6500mb of memory and 1.8vCPUs – Different Instance Family
Pod Count Pod M Instance M Memory % Pod vCPU Instance vCPU CPU %
Cost Per Pod
(OD hourly)
1 6,500 8,000 81.25% 1.8 4 45% $0.17c5.xlarge
2 13,000 16,000 81.25% 3.6 4 90% $0.096m5.xlarge
As seen in the chart above, by changing instance family, we can
achieve better resource allocation and reduce the cost by 43
percent.

17. What are EC2 Spot
Instances?

18. Cloud excess capacity aka “Spot Instances”
80%Discount
Spot VMs
Preemptible
VMs
Spot Instances
Spot Instances
NOSLA

19. Know Your Cloud

20. Planning for Interruptions
● Spot instances will be preempted
● Each cloud provider has different notification mechanisms
● Capacity will be affected
● State on the instance will be lost
● Spot instances can be preempted within a short period of time
These situations can be handled, by understanding markets, regions,
and instance capacity.

21. Planning for Interruptions
Capacity Pool | a set of instances that share the same region, availability zone and os
type. Includes all life-cycle types
Spot Pool | has its own availability and price as determined by current supply and
demand conditions.
Since on-demand and Spot Instances share a Capacity Pool, Spot prices can fluctuate
from usage changes of both Spot and on-demand instances.

23. Excess Capacity Interruptions
Zone 1 Zone 2 Zone 3

24. Pricing Strategy
AWS Example
Spot Instances allow applications to use a wide variety of instance types and maintain
or even increase performance
● On-demand
○ m5.xlarge 4 vCPU - 16 GB - $0.2/hour (Linux)
● Spot - all cheaper
○ m5.xlarge 4 vCPU - 16 GB - $0.0402 (80%)
○ m5.2xlarge 8 vCPU - 32 GB - $0.0851 (57%)
○ c5.4xlarge 16 vCPU - 32 GB - $0.152 (24%)
○ r4.4xlarge 8 vCPU - 61 GB - $0.17 (15%)

25. Ocean by Spot

26. Ocean by Spot | Serverless Containers
Deploy Containers on Abstracted Infrastructure. With up to 90% Cost Optimization.
Container-Driven Autoscaling
Auto-detect Pod or Task infrastructure
requirements so the appropriate instance
size or type will always be available.
Simplify Operations
Deploy more without having to manage all
the details of the underlying Container
infrastructure.

27. Ocean
Container-Driven Infrastructure Auto-Scaling
Ensures that all Pods have resources to run on, and
systematically selects the most suitable instance type
that will facilitate the containers’ requirements.
Maximize Resource Utilization
Validates that your instances are fully utilized
before spinning up new ones, thus enabling an
additional layer of cost-efficiency.
In addition, Ocean understand the network
topology and matches “chatty” containers to the
same hosts or AZ.

28. Ocean
Bring Your own Control Plane
Ocean seamlessly integrates and supports your stack,
whether you are using Amazon ECS or Kubernetes
Orchestrators such as EKSGKEAKS.
Save Up To 90% On Infrastructure Costs
Optimize up to 90% of your cloud-compute costs
by leveraging cloud excess capacity as the
underlying infrastructure which facilitates your
containers allocation, while enabling the option to
fallback to On-Demand.

29. Ocean
Cost Management & Showback
Get a granular view of the cluster’s cost breakdown
(compute and storage) for each and every one of the
cluster’s resources such as deploymentservice, cron
jobs, Tasks and Pods.
Vertical Container Auto-Scaling
Measure in real-time the CPUMemory of Pods
provides resource actionable suggestions based
on the consumption in your cluster.

30. Demo

31. Kevin McGrath
CTO
kevin@spot.io
Thank You!