Think differently when building apps in the cloud: heterogeneous environments, microservices, and disposable nodes are key components to building robust, deployment, and automated server management. In this session, we will investigate patterns for engineering scalable stateless systems, indentifying anti-patterns to avoid, and show how you can save 90% on your overall compute costs. When engineered for cloud, you will never worry about the uptime of a node again.   We will dive into best practices and new features that will help you realize immediate cost savings, maximize compute capacity within your budget, and maintain application availability and performance with less up-front and ongoing development effort. Attendees leave with practical knowledge of Spot bidding strategies, Spot market price trends, instance selection, and fault-tolerant architectures for web services. Learning Objectives: • Learn more about EC2 Spot and how you can use it to save 90% off your EC2 bill • Learn how to effective use Spot in production workloads by incorporating new Auto Scaling features • Learn best practices on bidding and choosing Spot instances Who Should Attend: • Customers who are familiar with Amazon EC2, or customers who want to understand how they can use Spot to run applications in the cloud.

1. Coding Apps in the
Cloud to reduce costs
up to 90%
Chad Schmutzer,
Specialist Solution Architect,
EC2 Spot

2. On-Demand
Pay for compute
capacity by the hour
with no long-term
commitments
For spiky workloads,
or to define needs
AWS EC2 Consumption Models
Reserved
Make a low, one-time
payment and receive
a significant discount
on the hourly charge
For committed
utilization
Spot
Bid for unused
capacity, charged at a
Spot Price which
fluctuates based on
supply and demand
For time-insensitive
or transient
workloads

3. Spare capacity at scale
AWS has more than a
million active customers
in 190 countries.
Amazon EC2 instance
usage has increased 93%
YoY, comparing Q4 2014
and Q4 2013, not
including Amazon use.

7. Capacity pools
AZ1
AZ2
SYD Total Capacity
T2 C4 M4 I2 R3 D2
Shared
Dedicated
Shared
Dedicated

8. $0.27 $0.29$0.50
1b 1c1a
8XL
$0.30 $0.16$0.214XL
$0.07 $0.08$0.082XL
$0.05 $0.04$0.04XL
$0.01 $0.04$0.01L
C3
$1.76
On
Demand
$0.88
$0.44
$.22
$0.11
Show me the markets!
Each instance family
Each instance size
Each Availability Zone
In every region
Is a separate Spot Market

9. 50% Bid
75% Bid
You pay the
market
price
Bid Price Vs Market Price
25% Bid

10. Amazon EC2 Spot – in the wild
1) We make this easy using the
Spot bid advisor
2) With deliberate pool
selection and bidding, you
will keep your Spot instance
as long as you need to.
3) And with new features like
Spot fleet diversified we do
the heavy lifting for you...

11. Spot Bid Advisor – aws-spot-labs

12. EC2 Best practices
Fault tolerance
for Spot
Stateless Multi-AZ Loosely coupled
Instance
Flexibility

13. Example customer cases

14. Why use Spot – customer examples
39 years of drug research re-processed, using over 80,000 cores, in 9
hours for $4,232
- Approximately 87,000 compute cores at peak
- Estimated 39 years of computational chemistry performed in 9 hours
- Three candidate compounds successfully identified

15. “By using AWS Spot instances, we've been able to save 75% a month
simply by changing four lines of code. It makes perfect sense for
saving money when you're running continuous integration workloads or
pipeline processing.” - Matthew Leventi, Lead Engineer, Lyft
Why use Spot – customer examples

16. The $9 Billion Experiment
Why use Spot – customer examples

17. Why use Spot – customer examples
Scaling up as many as 1000 Spot instances a day to handle real time ad
delivery
Petabyte-Scale Data Pipelines with Docker, Luigi and Elastic Spot
Instances

18. EC2 Spot fleet!

19. Spot fleet helps you
Launch Thousands of Spot Instances
with one RequestSpotFleet call.
Get Best Price
Find the lowest priced horsepower that works for you.
or
Get Diversified Resources
Diversify your fleet. Grow your availability.
And
Apply Custom Weighting
Create your own capacity unit based on your application
needs

20. Diversification with EC2 Spot fleet
Multiple EC2 Spot instances
selected
Multiple Availability Zones
selected
Pick the instances with similar
performance characteristics e.g.
c3.large, m3.large, m4.large,
r3.large, c4.large.

21. EC2 Spot fleet
Spot fleet API launched [May 18, 2015]
Weighted Bidding for EC2 Spot Instances [Aug 31,
2015]
Distribute Your Fleet Across Multiple Capacity Pools
[Sep 15, 2015]
New Spot Console [June 16, 2016]
Auto scaling for Spot fleet [Sept 01, 2016]

22. Lets see New Auto Scaling
Spot fleet in action..
Queue based processing
Stateless Applications (e.g. web tiers)
Docker powered by ECS

23. AWS cloud
Region
Amazon
S3
DynamoDB
Amazon SQS
CloudWatch172.16.0.0/16
Internet
gateway
region-1a -
172.16.0.0/20
region-1b -
172.16.16.0/20
region-1c -
172.16.32.0/20
region-1d -
172.16.48.0/20
Cluster
Controller
c3.4x Spot r3.4x Spot c3.4x Spot r3.4x Spot
c3.4x Spot r3.4x Spot
Queue based processing

24. Stateless Web Application
Elastic Load
Balancing
Stateless
Web Servers
(Spot)
Stateless
Web Servers
(Spot)
Session
State Data
Spot fleet
Availability Zone A
Availability Zone B
Stateless
Web Servers
(Spot)
Stateless
Web Servers
(Spot)

25. Docker powered by ECS
Task
Task TaskTask

26. DEMO!

27. Results - Web Application
50 instances requested,
over 30 days.
- Never dropped
below 45 instances
- 85% discount if you
wanted 50 and
could withstand
dropping to 45
0
0.02
0.04
0.06
0.08
0.1
0.12
30
35
40
45
50
55
Instances Average Price Per Instance
- If you only wanted
45 the discount is
still 83%

28. Results - Grid
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
250
350
450
550
650
750
850
950
1050
1150
Number of Cores Running Average Hourly Price Per Core
Requested 1000
vCores over 30 days
Minimum 960 vCores
Mode 1024 vCores
Average 1012 vCores
Average Price of $0.012
per vCore
Savings of over 80%

29. Thank you!

30. Appendix

31. Example from today – 6th April 2016

32. Example from today – 6th April 2016

33. Some additional considerations
Session state
Elastic Load Balancing
Two minute warning

34. Session state for the web application in DynamoDB.
• Data replicated across availability zones.
You can also choose other databases to maintain state in
your architecture.
• Amazon RDS using Multi-AZ deployments
• Amazon Elasticache
Where to store the state?

35. Since Spot fleet is configured to
span across multiple Availability
Zones, we highly recommend
enabling cross-zone load
balancing for the load balancer.
To allow in-flight requests to
complete when de-registering Spot
instances that are about to be
terminated, connection draining
can be enabled on the load
balancer with a timeout of 90
seconds.
Elastic Load Balancing

36. Capitalizing on two minute warning
When the Spot price exceeds
your bid price, the instance will
receive a two-minute warning
Check for the 2 minute spot
instance termination
notification every 5 seconds
leveraging a script invoked at
instance launch

37. Sample script – two minutes left!
1) Check for 2 minute
warning
2) If YES, detach instance
from ELB
3) OTHERWISE, do nothing
4) Sleep for 5 seconds
$ if curl -s http://169.254.169.254/latest/meta-
data/spot/termination-time |
grep -q .*T.*Z; then instance_id=$(curl -s
http://169.254.169.254/latest/meta-data/instance-id);
aws elb deregister-instances-from-load-balancer
--load-balancer-name my-load-balancer
--instances $instance_id;
/env/bin/flushsessiontoDBonterminationscript.sh; fi

38. Batch Processing with
Amazon EC2 Spot

39. Batch oriented applications can leverage on-demand
processing using EC2 Spot to save up to 90% cost:
Batch Processing with Amazon EC2 Spot
Monte Carlo
simulation
Molecular
modeling
Media
processing
High energy
simulations

40. Capitalizing on two minute warning
When the Spot price exceeds
your bid price, the instance will
receive a two-minute warning
Check for the 2 minute spot
instance termination
notification every 5 seconds
leveraging a script invoked at
instance launch

41. Sample script – two minutes left!
1) Check for 2 minute warning
2) If YES, run shutdown scripts
3) OTHERWISE, do nothing
4) Then sleep for 5 seconds
#!/bin/bash
while true
do
if curl -s http://169.254.169.254/latest/meta-
data/spot/termination-time | grep -q .*T.*Z; then
/env/bin/runterminationscripts.sh;
else
# Spot instance not yet marked for termination.
sleep 5
fi
done

42. Reference Links
EC2 Spot Documentation:
http://aws.amazon.com/ec2/spot/
http://aws.amazon.com/ec2/spot/bid-advisor/
http://aws.amazon.com/ec2/spot/getting-started/
http://aws.amazon.com/ec2/spot/faqs/
http://aws.amazon.com/ec2/spot/testimonials/
User Guide
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html
Helpful AWS Blog Posts
https://aws.amazon.com/blogs/aws/focusing-on-spot-instances-lets-talk-about-best-practices/
https://aws.amazon.com/blogs/aws/building-price-aware-applications-using-ec2-spot-instances/
https://aws.amazon.com/blogs/compute/cost-effective-batch-processing-with-amazon-ec2-spot/
https://aws.amazon.com/blogs/compute/dynamic-scaling-with-ec2-spot-fleet/