The document provides tips and strategies for optimizing costs when using cloud computing services like AWS. It discusses turning off unused instances, using auto scaling to align resources with demand, taking advantage of reserved instances for discounts, leveraging spot instances for significant savings, using different Amazon S3 storage classes, optimizing DynamoDB capacity units, buffering requests with SQS, and offloading architecture to services like CloudFront and ElastiCache. It also shares examples and case studies from customers like Pfizer, Zumba, and Airbnb that achieved cost savings through these approaches.
3. Top 5 Ways to Optimize for Cost
Efficiency with the Cloud
Nam Je Cho
Solutions Architect
4. v
What You’ll Get Out of This Session
• A lower AWS bill
• Maintain high performance while minimising cost
• A more scalable, robust, dynamic architecture
• More time to innovate
• Real-world customer examples
• Easy to implement
5. v
Cloud Computing Benefits
• Pay as you go, no up-front investments
• Low ongoing cost
• Flexible capacity
• Speed, agility, and innovation
• Focus on your business
• Go global in minutes
6. v
Cloud Computing Benefits
• Pay as you go, no up-front investments
• Low ongoing cost
• Flexible capacity
• Speed, agility, and innovation
• Focus on your business
• Go global in minutes
8. v
AWS Pricing Philosophy
Ecosystem
Global Footprint
New Features
New Services
More AWS
Usage
More
Infrastructure
Lower
Infrastructure
Costs
Reduced
Prices
More
Customers
Infrastructure
Innovation
45
price
reduc+ons
since
2006
Economies
of Scale
15. v
1. Turn Off Unused Instances
• Developer, test, training instances on evenings/weekends
• Use simple instance start and stop
• Tear down and build up all together
• Instances are disposable
• Automate, automate, automate!!!
• AWS CloudFormation
• Weekend/off-hours scripts
• Use resource tags
17. v
Customer Case Studies
Pfizer’s
high
performance
compu=ng
(HPC)
soGware
and
systems
for
worldwide
research
and
development
(WRD)
support
large-‐scale
data
analysis,
research
projects,
clinical
analy=cs,
and
modeling.
“We
accomplished
this
by
customizing
the
job
scheduler
in
our
HPC
environment
to
recognize
workload,
and
start
and
stop
instances
as
needed
to
address
the
workflow”
RunE2E
for
Kenneth
Cole
leverage
several
AWS
APIs
for
star=ng/stopping
the
system,
increasing
system
capacity
based
on
demand
needs,
and
changing
security
profiles
providing
a
low
cost
but
scalable
environment
to
run
its
business.
“Instance
ac=ons,
such
as
create,
stop,
start,
and
terminate,
can
very
easily
be
scheduled
as
an
event.
For
example,
now
we
can
terminate
all
instances
automa=cally
aGer
a
training
class,
thereby
elimina=ng
the
genera=on
of
unnecessary
cost”
21. v
Align Your Resources With Demand
h$p://aws.amazon.com/autoscaling/
22. v
Customer Case Studies
“We
believe
auto
scaling
greatly
improves
the
availability
of
our
services
and
provides
an
excellent
means
of
op=mizing
our
cloud
costs.”
h$p://techblog.neYlix.com/2012/01/auto-‐scaling-‐in-‐amazon-‐cloud.html
Siemens
Healthcare
Diagnos=cs
helps
advance
human
health
through
innova=on.
By
using
services
such
as
ELB,
Amazon
SQS,
Amazon
EC2,
Amazon
RDS,
Auto
Scaling,
Amazon
Elas=Cache,
and
AWS
CloudTrail,
Siemens
has
built
a
secure,
HIPAA-‐compliant,
and
scalable
plaYorm
on
AWS.
The
array
of
AWS
products
offers
TicketLeap
a
potent
solu=on.
The
company
uses
Amazon
EC2,
Amazon
RDS,
Amazon
CloudFront,
Amazon
S3,
ELB,
Auto
Scaling,
and
cloud
monitoring
services
as
the
founda=on
for
its
plaYorm.
24. v
Calculate RI Savings
1y
RI
Break
even
3y
RI
Break
even
h$p://calculator.s3.amazonaws.com/index.html
25. v
Reserved Instances Are Flexible
• Can be moved between AZs
• Can be moved between EC2-Classic and EC2-VPC platforms
• Size can be modified within the same instance family
h$p://aws.amazon.com/ec2/purchasing-‐op=ons/reserved-‐instances/
26. v
Simplifying the EC2 Reserved Instance Model
• Simplified Reserved Instance model
• Single type of Reserved Instance and three payment options;
• All Upfront
• Partial Upfront
• No Upfront
h$p://aws.amazon.com/ec2/purchasing-‐op=ons/reserved-‐instances/
27. v
Customer Case Studies
As
one
of
the
world’s
largest
dance
fitness
programs,
Zumba
Fitness
needed
a
scalable
and
high
performance
website.
When
Zumba
migrated
its
website
to
AWS
in
2009,
the
company
saved
39%
on
compu+ng
resources
by
using
Amazon
Reserved
Instances.
Zumba
could
also
scale
easily
to
serve
up
7
TB
of
images
to
its
customers
each
month.
29. v
Spot Instance Rules
• Price based on supply/demand
• You choose your maximum price/hour
• Your instance is started if the Spot price is lower
• Your instance is terminated if the Spot price is higher
• But, you did plan for fault tolerance, didn’t you?
• We provide guide on architecting applications to use Spot Instances
30. v
Spot Instance Example
$0.028
(11.7%)
$0.026
(10.8%)
$3.28
(1367%)
On-‐Demand:
$0.24
31. v
Spot Instance Use Cases
• Analytics
• Batch Processing
• Big Data - Amazon Elastic MapReduce
• Web crawlers
• Testing
• Image & media encoding
• High Performance Computing
• Financial modeling and analysis – Monte Carlo analysis
32. v
Spot Instances Recap
• Very dynamic pricing
• Opportunity to save 80-90% cost
• But there are risks & you need to design around it!
• Different prices per AZ
• Leverage Auto Scaling!
• One group with Spot Instances
• One group with On-Demand
• Get the best of both worlds
33. v
Customer Case Study
Cycle Computing
“HGST buys 70,000-core cloud HPC Cluster, breaks record, returns it 8 hours later”
34. v
5. Use Amazon S3 Storage Classes
• Reduced redundancy storage class
• 99.99% durability vs. 99.999999999%
• Up to 20% savings
• Everything that is easy to reproduce
• Use Amazon SNS lost object notifications
• Amazon Glacier storage class
• Same 99.999999999% durability
• 3 to 5 hours restore time
• Up to 64% savings
• Archiving, long-term backups, and old data
• Use S3 Lifecycle rules
• Archive objects to Glacier after x days
• Delete objects after y days
35. v
6. Optimize Amazon DynamoDB capacity units
• Read/write capacity units (CUs) determine most of DynamoDB cost
• By optimizing CUs, you can save a lot of money
• But:
• Need to provision enough capacity to not run into capacity errors
• Need to prepare for peaks
• Need to constantly monitor/adjust
36. v
DynamoDB Optimisation
• Use caching to save read capacity units
• Local RAM caches at app server instances
• Check out Amazon ElastiCache
• Think of strategies for optimizing CU use
• Use multiple tables to support varied access patterns
• Understand access patterns for time series data
• Compress large attribute values
• Use Amazon SQS to buffer over-capacity writes
38. v
Customer Case Studies
During
the
2012
Super
Bowl,
the
Shazam
App
ran
on
Amazon
DynamoDB,
Amazon
EMR,
and
Amazon
EC2
Cluster
Compute
Quadruple
Extra
Large
Instances
with
Elas=c
Load
Balancing.
This
infrastructure
helped
to
support
the
processing
of
millions
of
transac=ons
during
this
high-‐traffic
event.
40. v
DynamoDB Optimisation Example
Caching/Op=miza=on:
80%
saved
Cache
flush
Dynamic
DynamoDB:
20%
saved
Growth
+
new
features
41. v
7. Offload Your Architecture
• More you can offload, the less infrastructure you need to maintain, scale,
and pay for
• Three easy ways to offload:
• Use CDN - Amazon CloudFront
• Introduce Caching – Amazon ElastiCache
• Leverage existing Amazon Web Services
43. v
Customer Case Studies
Airbnb
is
a
community
marketplace
for
unique
vaca=on
spaces
around
the
world.
Airbnb
benefits
from
the
scalability,
agility
and
reliability
provided
by
Amazon
Web
Services,
including
Amazon
EC2,
Elas=c
Load
Balancing,
Amazon
S3,
Amazon
EMR,
Amazon
CloudFront,
Amazon
CloudWatch,
and
Amazon
RDS.
The
Internet
Movie
Database
(www.imdb.com)
is
one
of
the
world’s
most
popular
and
authorita=ve
sources
for
movie,
TV
and
celebrity
content
with
more
than
100
million
unique
visitors
per
month.
IMDb
uses
AWS
and
Amazon
CloudFront
to
host
search
data,
making
the
search
experience
the
fastest
possible
by
distribu=ng
the
content
physically
close
to
IMDb's
worldwide
user
base.
PBS,
a
private,
non-‐profit
corpora=on,
provides
content
through
television,
the
Web
and
mobile
applica=ons.
PBS
Interac=ve,
the
department
responsible
for
PBS’
Internet
and
mobile
presence,
improved
its
video
streaming
performance
by
migra=ng
to
AWS
to
u=lize
the
content
delivery
service
Amazon
CloudFront
and
increase
its
exis=ng
usage
of
Amazon
S3.
45. v
Customer Case Studies
PlaceIQ,
a
loca=on-‐based
intelligence
company
in
New
York
City,
provides
loca=on
intelligence
for
mobile
adver=sing.
The
company
uses
Amazon
Elas=Cache
and
AWS
to
improve
its
web
service
response
=me
by
83%.
Sanoma
Games
designs
casual
online
gaming
and
fantasy
sports
leagues
as
part
of
the
Sanoma
diversified
media
group.
The
business
unit
recently
closed
its
local
datacenter
in
order
to
build
a
scalable,
service-‐based
architecture
that
can
facilitate
expansion
into
addi=onal
markets
and
gaming
categories.
Cloud
management
specialist
Nordcloud
was
appointed
to
create
and
manage
the
new
environment,
which
now
includes
Amazon
RDS,
Amazon
EC2,
Amazon
S3,
Amazon
Elas+Cache,
and
Amazon
CloudWatch.
SpareFoot.com
helps
self-‐storage
facili=es
market
their
extra
space,
and
helps
consumers
make
educated
decisions
on
which
storage
unit
is
best
for
their
needs.
In
order
to
manage
a
network
of
consumer
facing
sites,
the
company
uses
several
AWS
products—including
Amazon
RDS,
Amazon
ELB,
Amazon
S3,
Amazon
Elas+Cache,
Amazon
CloudWatch,
and
Amazon
CloudFront—to
create
a
stable
system
and
reduce
=me,
costs,
and
labor.
46. v
Leverage Existing Services
• Amazon RDS, Amazon DynamoDB or Amazon ElastiCache for Redis, Amazon Redshift
• Instead of running your own database
• Amazon CloudSearch
• Instead of running your own search engine
• Amazon Elastic Transcoder
• Amazon Elastic MapReduce
• Amazon Cognito, Amazon SQS, Amazon SNS, Amazon Simple Workflow Service, Amazon SES,
Amazon Kinesis, and more …
Simple, more reliable, lower cost
49. v
Customer Case Studies
Learn
how
media
giant,
Dow
Jones
&
Company,
plans
to
save
over
$100
million
over
3
years
by
migra=ng
its
data
centers
to
AWS.
h$ps://www.youtube.com/watch?v=xS8Ha0Q6_CY
50. v
AWS Simple Monthly Calculator
http://calculator.s3.amazonaws.com/index.html
54. v
Let’s Recap
1. Turn off unused instances
2. Use Auto Scaling
3. Use Reserved Instances
4. Use Spot Instances
5. Leverage Amazon S3 storage classes
6. Optimize Amazon DynamoDB capacity units
7. Offload your architecture