Containers are usually stateless and fault tolerant, and Amazon EC2 Spot Instances are a great match for powering your Amazon EC2 compute for these workloads. In this workshop, you learn how to provision, manage, and maintain your Amazon Kubernetes clusters with Amazon EKS at any scale on Spot Instances to architect for optimizations on cost and scale. We dive deep using hands-on material to provision and scale worker nodes, handle interruptions, and design for fault tolerance. We also demonstrate successfully managing a suddenly spiky workload. This workshop helps architects, engineers, and developers understand how to run a containerized environment on Spot Instances. All attendees must bring their own laptop (Windows, macOS, and Linux all supported). Tablets are not appropriate. We also recommend having the current version of Chrome or Firefox installed.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Running Amazon EKS Workloads on
Amazon EC2 Spot Instances
Madhuri Peri
Sr. IoT Solutions Architect
Amazon Web Services
C M P 4 0 3
Shawn O’Connor
Solutions Architect
Amazon Web Services

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Overview of Amazon EC2 Spot
Overview of Amazon EKS
Provisioning Amazon EC2 Spot in Amazon EKS
Scaling with Amazon EC2 Spot in Amazon EKS
Handling Interruptions
Workshop Overview
Workshop Cleanup

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EC2 Purchasing Options
On-Demand
Pay for compute capacity by the second
with no long-term commitments
Spiky workloads, to define
needs
Reserved Instances
Make a 1- or 3-Year commitment and
receive a significant discount off On-
Demand prices
Committed, steady-state usage
Spot Instances
Spare Amazon EC2 capacity at savings of
up to 90% off On-Demand prices
Fault-tolerant, flexible, stateless
workloads

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why Spot Instances?
Inexpensive
Significant price savings of up to
90% over On-Demand Instances.
Perfect for stateless and fault-
tolerant workloads
Faster Results
Easily run multiple projects
simultaneously and speed up job
flows to generate business results
faster and innovate faster
Easy Access
Launch spot instances via
the RunInstances function,
run-instances cmd, or from
AWS Console. Integrated with
other AWS and 3rd party
services
Resource Flexibility
Flexibility of ad-hoc provisioning
for multiple instance types with
an option to hibernate, stop or
terminate instances when
reclaimed.

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Provision Amazon EC2 Spot Instances
• Run Instances API
• Launch configuration / Auto scaling groups API’s
• EC2 Fleet and/or Spot Fleet API
• RequestSpotInstances API

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS architecture
Workers Workers Workers
Availability zone Availability zone Availability zone
etcd
Master Nodes
Availability zone Availability zone Availability zone
etcd
Master Nodes
etcd
Master Nodes

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EKS worker node provisioning with EC2 Spot
• Launch Amazon EC2 Spot instances as part of AutoScaling group
• Use Amazon EC2 Spot instances best practice of mixed instance types
• Recommend using the node labels to identify Amazon EC2 Spot instances
• Enables scheduling workloads selectively

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Auto scaling with Amazon EKS
• Two dimensions to scaling
• EC2 instance through K8s cluster AutoScaler
• Scale out Amazon EC2 Spot instances
• Pods through HPA
• Scale out pods

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Auto scaling with EKS – Con’t …
Amazon EC2 Autoscaling
• Min
• Max
• Instance type
• ASG group
K8s Cluster Autoscaler
• Min, max, instance type
• K8s node groups
• CA modes
• Auto-discovery
• Multi ASG
• Single ASG
• Master node
• HPA

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Auto scaling workflow
Instance
Instance
Instance
Auto Scaling Group
Worker Nodes
Cluster Autoscaler
Scale Up
Desired Capacity +1
Instance
Desired Capacity -1
Scale down
Do I still need all these resources?
Do I have enough resources?
Instance

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Handling interruptions
Instance gets terminated / maintenance / interrupted, workflow should be:
• Stop directing new-traffic to the pods on terminating instance
• Gracefully drain connections of traffic already being served
• Re-instantiate pods elsewhere in the cluster

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Interrupt handler
• Catch Amazon EC2 Spot interruption event
• Leverage K8s functionality to set node state to DRAIN
• Leverage K8s schedulers to re-instantiate pods

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What are we going to build
VPC
Availability zone Availability zone Availability zone
Auto Scaling
Auto Scaling
Auto Scaling
m4.large Spot
t2.medium Spot
On-Demand
Instances Instances Instances
Spot Instance
Spot Instance Spot Instance
Spot Instance
Spot Instance
Spot Instance
Spot Interruption
Cluster Autoscaler

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Workshop link
https://eksworkshop.com/spot

21. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.