Česko-Slovenský AWS Webinář 07 - Optimalizace nákladů v AWS

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Cost Management
and Optimization
Nazar Špak, Territory Manager Vladimír Šimek, Enterprise Solution Architect

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Contents
Ways to lower data
transfer out cost
Monitor, track, and
analyze your services
usage
Reserve and save with
the Reserved Instances
Technical Optimisation
Levers and Tools

How AWS Helps You
Realize Value and Save Cost

Cloud is the new normal with fewer organisations asking
“why” and more asking “how and how fast?”
Industrial
Revolution
Cloud
Revolution
Shift to on-demand
computing
Shift to on-demand
power

AWS allows you to fit cost to today’s demand
“Hug your workloads, not your servers!”

AWS lowers prices over time
70+ Price
Reductions!

Step 1:
Establish Cost Visibility

Speed, scale, complexity, and value at stake drives
the need for visibility

Pick the Cost Tool that provides the visibility you need
Simple, Static, Small
environment
Complex, Dynamic,
Large environment
1. Monthly AWS Invoice
2. AWS Billing
console
3. AWS Cost Explorer
and AWS Budgets
4. AWS Billing File Analysis,
DIY dashboards, and
3rd party tools

Step 2:
Technical Optimization Levers and Tools

Architectural
Optimisation
Quick-win Optimisation
The Technical Pillars of Optimisation
Right Size &
Decommission
unused Resources
Use Reserved
Instances
Increase
Elasticity
Match Storage
to Need
Design for
Cost

Pillar 1: Right Sizing your Resources
Right Sizing is about
• Selecting the most cost-efficient instance (size
and family) while meeting performance needs
• Most commonly based on CPU, RAM,
storage, and network needs
• Doing this before/during migration via
discovery tools reduces overall effort

87,5%
Saving
Right Sizing example
m4.4xlarge
$0.96 per hr
m4.large
$0.12 per hr
2. Check (CPU,
RAM, network, disc)
1. Migrate/provision
& Run
3. Right Size
5. Save!4. Review Performance

Choose the right (EC2) instance type from the start
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html
Area of need Instance Type
Burstable / intermittent compute T2, T3
Compute Optimised ~1vCPU-2GB C3, C4, C5
Memory Optimised ~1vCPU-8GB R3, R4, X1, X1e
Compute-Memory balanced ~1vCPU-4GB M3, M4, M5
Storage/throughput Optimised D2, H1, I2, I3
Accelerated Computing F1 (FPGA), P2, P3, G3

Decommissioning unused resources
• Unattached Elastic IPs
• Unattached EBS
• Idle / unattached load balancers
• Incomplete S3 transfers
• Idle EC2, RDS, Redshift instances
• Clearing old snapshots

• AWS Trusted Advisor & AWS CloudWatch
• 3rd party paid tools (including but not limited to)
• Open source tools
• Cost Optimisation: EC2 Right Sizing
https://aws.amazon.com/answers/account-management/cost-
optimization-ec2-right-sizing/
• Netflix Janitor Monkey
https://medium.com/netflix-techblog/janitor-monkey-keeping-the-cloud-
tidy-and-clean-d517ad74d648
Tools for Right Sizing and finding unused resources

Pillar 2: Increase Elasticity
AWS allows you to fit cost to today’s demand, as opposed to expected
peak demand in 3-5 years for traditional IT.
“Hug your workloads, not your servers!”

Reduce non-prod spend by 70% or more via scheduling
Fun fact:
RDS can be
turned off too
Maximise savings via automatically turning off resources at night with
manual turn-on in the morning

AWS Auto Scaling helps you match resources to demand
24 hrs x 24 m5.2xlarge @
$0.48/hr. on Linux in Sydney
= $8,294.40 / month
Autoscaling 51 m5.large instances
@ $0.12/hr. on Linux in Sydney
= $4,406.40 / month (47% saving)
Demand
$

• Guide to Auto Scaling
https://aws.amazon.com/documentation/autoscaling/
• AWS Instance Scheduler
https://aws.amazon.com/answers/infrastructure-
management/instance-scheduler/
• 3rd party paid tools (including but not limited to)
Tools for Increasing Elasticity

RI
coverage
Total use
Key
Reserved Instances for one financial services customer
resulted in 39% decrease in unit cost
Reserved Instance
purchase
Pillar 3: Use Reserved Instances

What are Reserved Instances (RIs)?
• RIs are a commitment in exchange for discount
• They behave like discount coupon booklets
• Each hourly RI coupon in the booklet tries to match to a running
instance, if matched it provides its full discount, if not it expires
• RIs are a financial construct/layer on top of your AWS
infrastructure
RI coupon booklet

EC2 RI matching properties
• Instance type: m4.xlarge
o or Instance Family: m4.
• Operating System: Linux/Unix, Windows, Windows with
SQL Server Standard, etc.
• Region: ap-southeast-2 (Sydney)
o or AZ: ap-southeast-2a
• Tenancy: Shared, dedicated

RIs are most often used for always-on instances
(can still be used to save for non-always on)
Commitment level
1 year (approx. payback 7-10 months)
3 year (approx. payback 10-20 months)
AWS services offering RIs
Amazon EC2 & EC2 Hosts
Amazon RDS
Amazon Redshift
Amazon ElastiCache
Amazon Elasticsearch (new)
Amazon DynamoDB*
Amazon CloudFront*
*Discount for commitment, but not an RI

Understanding Convertible Reserved Instances
You are committing to:
1 or 3 yrs. (not refreshed on conversion)
Region
EC2 on AWS
Spend (convert to equal or greater $)

Which EC2 RIs give greater discount?
Greater discountLess discount
All upfrontPartial upfrontNo-upfrontPayment option
3 year1 yearDuration
Linux/UnixOthers (typically)Operating System
Newer genOlder generations (typically)Instance type
StandardConvertibleClass

RIs have priority to apply to instances in the same
account as the RI
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/apply_ri.html
http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/consolidated-billing.html
RI can live here and this
account will get priority
to use it first
RIs can also live here and this account
will get priority to use the RI.
Be sure to have RI console access to
modify RIs if needed & billing console
access to check RI utilisation
Paying
Account
Linked
Account 1
Linked
Account 2

Use Cost Explorer to track RI coverage

Get notified of low RI utilization via email or SNS
via AWS Budgets

Check if RIs are underutilized via Cost Explorer

See RI Recommendations and Savings via Cost Explorer
1
2

This report can also be used to understand what your unused Convertible RIs
could be changed into
See RI Recommendations and Savings via Cost Explorer

Pillar 4:
Match Storage Class to
Business Need
Elastic Block Store (EBS)
Simple Storage Service (S3)

Reduce Elastic Block Store (EBS) spend by picking the
class that’s fit for need
https://aws.amazon.com/ebs/details/
Provisioned IOPS
SSD (io1)
General Purpose
SSD (gp2)
Throughput
Optimized HDD (st1)
Cold HDD
(sc1)
Default
Approx. 75%
cheaper than gp2
Approx. 50%
cheaper than gp2
Size matters for EBS
because you pick the
amount of storage
provisioned, whereas
S3 and EFS scale
dynamically

Storage Classes on Simple Storage Service (S3)
Active data Archive dataInfrequently accessed data
Standard (default)
Hot
Standard - Infrequent Access
Warm
Amazon Glacier
Cold

Pillar 5:
Designing for Cost
EC2 Spot and other methods

Architectural
Optimisation
Designing for Cost is most effective as a proactive strategy
& activity set, however it can be used reactively too
Right Size &
Decommission
unused Resources
Use Reserved
Instances
Increase
Elasticity
Match Storage
to Need
Design for
Cost

“By using AWS Spot instances, we've been able to
save up to 75 percent a month simply by changing
four lines of code. It makes perfect sense for saving
money when you're running continuous integration
workloads or pipeline processing.”
Matthew Leventi - Lead Engineer

With Spot the rules are simple
The price changes
infrequently based on supply
and demand of spare capacity
Just request capacity and pay
the current rate. When we
need the capacity back you’ll
get a 2 minute warning

Spot can be used for
• Production & non-production workloads
• Queue and Batch based processing
• Hadoop / Spark / Big Data workloads (EMR)
• Stateless Applications (e.g. web tiers)
• CI/CD pipelines
• Containerised workloads powered by Spot Fleet
• Video transcoding
• See http://bit.ly/whenToUseAWSSpot
EC2 Spot provides incredible savings for the
right workloads

Spot Advisor finds you suitable instance types to
include in your Spot or Spot Fleet application
https://aws.amazon.com/ec2/
spot/instance-advisor/

Recent EC2 Spot updates
• T3 on Spot
• Higher spot bids no longer
take priority over lower bids
• Spot stop or hibernate &
resume
• New price algorithm
(smoother price)
Smoother price starting
Dec 2017

EC2 Spot resources to help you get started
Getting started with EC2 Spot Instances:
http://aws.amazon.com/ec2/spot/
http://aws.amazon.com/ec2/spot/getting-started/
EC2 Spot Labs:
https://github.com/awslabs/ec2-spot-labs
EC2 Spot Case Studies:
https://aws.amazon.com/ec2/spot/testimonials/

There are many more …
Containerisation (staff productivity
and compute utilisation)
Consolidated billing (RI and
volume discount benefits)
AWS CloudFormation
(Dev time saving)
Serverless &
AWS Lambda
ELB to Application Load
Balancer
Deliver content with
AWS CloudFront
(lower compute and data transfer)
Open source platforms & databases
(reduced licensing cost)
Low cost security monitoring via
AWS GuardDuty
Running resources in a
cheaper region
Static Web Hosting on S3
and using S3 Select

Architectural
Optimisation
Summary of the Technical Levers of Optimisation
Right Size &
Decommission
unused Resources
Use Reserved
Instances
Increase
Elasticity
Match Storage
to Need
Design for
Cost

Tagging

Define and agree mandatory (cost) tags
Tagging -> Allocation -> Showback -> Chargeback -> Ownership ->
Drive responsibility and efficient behaviour

Tagging can help you quickly and accurately
answer important cost management questions
• what is the business purpose of the spend?
• what’s causing the change in spend?
• where does spend inefficiency exist?
• who is responsible for optimising spend?
• who should be notified of any environment changes?

Address improper tagging
???
App
BU
Owner

• AWS Tag Editor:
To bulk filter, add, and edit tags
Can export resources and tags to csv
• Open source tools
• Capital One’s Cloud Custodian: advanced tag compliance and remediation
https://developer.capitalone.com/opensource-projects/cloud-custodian/
https://github.com/capitalone/cloud-custodian
• Tagging coverage tools:
https://github.com/GorillaStack/auto-tag
https://github.com/Answers4AWS/graffiti-monkey
Tools for Tagging incl. Automation & Enforcement

Get Support

AWS Support, AWS Professional Services, and AWS Partners can guide
your Cost Management and Optimisation journey and initiatives, contact
your AWS Account Manager for more info.
AWS Support
https://aws.amazon.com/premiumsupport/
AWS Professional Services
https://aws.amazon.com/professional-services/
AWS Partner Network
https://aws.amazon.com/partners/

Thank You
nazaspak@amazon.com
vladsim@amazon.com

Appendix:
Content and FAQ

Contents
• EC2 and RDS RI recommendations in Cost Explorer
• Credit and RI discount sharing via billing preferences
• EC2 Convertible Reserved Instance Update – New 1-Year CRI, Merges & Splits
• EC2 Spot Update – Streamlined Access to Spot Capacity, Smooth Price Changes, Instance Hibernation
• S3 Select and Glacier Select – Retrieving Subsets of Objects
• AWS Cost Explorer API
• AWS Price List API Update – Regional Price Lists
• Per-Second Billing for EC2 and EBS
• RDS Reserved Instance Size Flexibility
• T2 Unlimited – Going Beyond the Burst with High Performance
• Amazon Aurora Serverless (preview)
• AWS Instance Scheduler
• EC2 Fleet – Manage of On-Demand and Spot Instances with One Request
• RI Amortisation in Cost Explorer
• Optimize (i.e. turn-off) vCPUs for EC2 Instances
• Default billing now shows unblended costs
• RI coverage warnings from AWS Budgets

EC2 and RDS recommendations in Cost Explorer
What is it?
• RI recommendations are now provided by Cost Explorer
• Options for 1yr or 3yr, Standard or Convertible (EC2), 3 payment options (AURI, PURI,
NURI), and Based on the past 7 days, 30days or 60 days
• Considers instance size flex
• Convertible RIs can be managed via the combined use of the RI utilization report (which
RIs to change) and the RI recommendations report (what RIs to change into)
How does this help you save?
• Quicker access to RI recommendations based on very recent data
Here is some more information:
• EC2 Landing Page
• RDS Landing Page

Credit and RI discount sharing via billing preferences
What is it?
• Control where Credit & RI discounts get applied
• Allows customers to have credits apply to the account that owns a credit,
or has previously redeemed a credit
• Allows customers to switch accounts into “non-sharing mode” where RI benefits will not
flow into or out of the account. The feature does not support creation of account groups
to share benefits.
• Doesn’t delivers savings but allows customers to have more granular control over
Credits and RIs
• Landing Page

AWS Console view of Credit and RI sharing
Bucket of accounts
that don’t share RIs

EC2 Convertible Reserved Instance Update –
New 1-Year CRI, Merges & Splits
What is it?
• 1 year Convertible RIs are now available
• Convertible RIs let you change Instance Size, Family, OS, and Tenancy
• Convertible RIs can now be split and/or merged (i.e. merging two different CRIs with
different end dates)
• Expected to boost RI coverage as customers desire flexibility and a 1 year commit
• In APAC, for popular instance families (m4, c4, r4) and common operating systems
(non-SQL), 1yr Convertible RIs typically give 5% to 10% less discount than the
corresponding 1yr Standard RI
• Blog Page

Merging and Splitting Convertible RIs
Split
Split CRIs into the part that you want to keep and the part that you want to exchange. Select the CRI,
click on Modify Reserved Instances. Create desired configuration and click on continue. Review the
request and click on Submit Modifications. The state of the CRI changes to indicate that it is being
modified. After a moment or two it will be marked as retired, replaced by a pair that are active.
Merge
You can also merge two or more Convertible Reserved Instances together and then use them as the
starting point for an exchange. To do this you simply select two or more existing CRIs, click on Action,
and choose Exchange Reserved Instances. You can see the total remaining upfront value of the
selected CRIs and proceed accordingly. You can merge CRIs that have different start dates and/or
term lengths. The merged CRI will have the expiry date of the RI that is furthest from the date of
exchange. Merging CRIs with different term lengths always produces a 3-year CRI.

EC2 Spot Update – Streamlined Access to Spot Capacity,
Smooth Price Changes, Instance Hibernation
What is it?
• New price algorithm, smoother price starting Dec 2017
• Spot instances can now stop or hibernate & resume instead of being terminated
• T2 can now be run as Spot
• Higher spot bids no longer take priority over lower bids
• EC2 Spot saves 50% to 90% off on Demand
• With the changes, more workloads are able to take advantage of EC2 Spot
• Blog Page

S3 Select and Glacier Select – Retrieving Subsets
of Objects – preview
What is it?
• Retrieve only a subset of data from an object in S3 or Glacier by using simple SQL
expressions
• Data retrieval cost and time can be reduced, in some cases it’s 5x faster, using 1/40 of
the CPU
• Blog Page

AWS Cost Explorer API
What is it?
• Directly access Cost Explorer API, via an ad-hoc query service that powers the Cost
Explorer web application
• Access to the full Cost Explorer dataset including Reserved Instance utilization and
cost allocation tags
• Enables customers to more easily build customized reports
• Simplifies cost management on AWS
• Blog Page

AWS Price List API Update
What is it?
• Access only the price data required instead of downloading the entire price list
• Key functions:
• DescribeServices – Returns sets of attribute keys that are used to define products within a service. For
example, keys returned for EC2 will include physicalProcessor, memory, operatingSystem, location and tenancy.
• GetAttributeValues – Returns all of the allowable values for a given attribute key. For example, values for the
operatingSystem key include Windows, RHEL, Linux and SUSE; values for the location key include US East (N.
Virginia) and Asia Pacific (Mumbai).
• GetProducts – Returns all of the products, along with their public prices, that match a filter expression based on
service name and attribute value.
• Avoids size and complexity of using entire pricing file in cost management tools
• Blog Page

Per-Second Billing for EC2 and EBS
What is it?
• One second billing increment for Linux/Unix instances that are launched in On-Demand,
Reserved and Spot (incl. EMR and Batch)
• Also applies to EBS volumes, Elastic GPUs, and Provisioned IOPS
• There is a 1-minute minimum charge per-instance
• Applicable to underlying instances used via the AWS Marketplace
• Saves more money for autoscaling/dynamic workloads due to finer grained billing
• Enables application of RI discounts across multiple instances for sub-1 hour workloads
(see next page)
• Blog Page

On Demand With Reserved Instance
Impact of Per-Second Billing with and without RIs
Instance 10am - 11am Before After
A 30 minutes RI RI
B 20 minutes OD RI
C 9 minutes OD RI
D 0.5 minutes OD RI
Total 59 mins 30 secs 3 hrs OD
1 hr RI billed
1 hr RI billed
Instance 10am - 11am Before After
A 30 minutes 1 hour 30 minutes
B 20 minutes 1 hour 20 minutes
C 9 minutes 1 hour 9 minutes
D 0.5 minutes 1 hour 1 minute
Total 59 mins 30 secs 4 hrs billed 1 hr billed

RDS Reserved Instance Size Flexibility
What is it?
• RDS RI size flexibility improves the ability for RDS RIs to apply and provide discount
• RDS RI size flexibility is offered in all regions for the MySQL, MariaDB, PostgreSQL, and Amazon
Aurora database engines, as well as the “bring your own license” (BYOL) edition of the Oracle
database engine
• Applies to usage of any size in the DB instance family using the same database engine.
• The RI discounted rate also applies to usage of both Single-AZ and Multi-AZ configurations for the
same database engine and DB instance family.
• Improves ability to increase RDS RI coverage
• E.g.1 db.r3.2xlarge applies to any of the following in the same Region for the same DB engine:
1 x db.r3.2xlarge, 2 x db.r3.xlarge, 4 x db.r3.large, ½ x db.r3.4xlarge (50% is RI, 50% is On
Demand), ¼ x db.r3.8xlarge (25% is RI, 75% is On Demand)
• Blog Page

T2 Unlimited – Going Beyond the Burst with High
Performance
What is it?
• Sustain high CPU performance for T2 over any desired time frame
• Enable this upon instance launch
• Small extra cost for higher CPU without needing to upgrade instance type
• For example, if you run a t2.micro instance at an average of 15% utilization (5%
above the baseline) for 24 hours you will be charged an additional 6 cents (5 cents
per vCPU-hour * 1 vCPU * 5% * 24 hours).
• Blog Page

Amazon Aurora Serverless - preview
What is it?
• Aurora Serverless starts up on demand and shuts down when not in use
• Automatically scales with no instances to manage
• Price is based on the actual per-second usage, reducing RDS cost
• No instances hence reduced instance management overhead and cost
• Product Page
• Announcement Video (0:56)
• Sign up for the preview

AWS Instance Scheduler - automatically start and
stop EC2 and RDS
What is it?
• Automatically start and stop EC2 and RDS on a schedule
• Highly configurable to account for weekday, weekend, and resource specific nuances
• Resources (excl. storage) that are turned off do not incur any charge
• Reduces operational effort of developing automation solution in-house
• Landing Page

EC2 Fleet - Manage Thousands of On-Demand
and Spot Instances with One Request
What is it?
• Create fleets that are composed of a combination of EC2 On-Demand, Reserved, and
Spot Instances with a single API call
• We will launch, manage, monitor and scale instances as needed, without the need for
scaffolding code
• Lower barrier to entry (simpler code/architecture) to leverage EC2 Spot which saves
50-90% off on-demand prices
• Blog Page

Optimize CPUs for Amazon EC2 Instances
What is it?
• Ability to specify a custom number of vCPUs when launching new instances to save
on vCPU-based licensing costs
• Disable Intel Hyper-Threading Technology (Intel HT Technology) for workloads that
perform well with single-threaded CPUs, like certain high-performance computing
(HPC) applications.
• Reduced licensing costs
• Announcement Page

Default billing now shows unblended costs
What is it?
• Customers with consolidated billing will receive linked account bills that show each
account’s unblended rates and costs
• This means Reserved Instance (RI) discounts and pricing tiers (e.g., AWS Free Tier
rates and S3 volume tiers) are reflected in the linked account bill
• Previously the total bill was the same but a single average rate was used across all
accounts
• Doesn’t save but helps with cost allocation

RI Coverage warnings from AWS Budgets
What is it?
• To maximise RI discounts, customers need to increase RI coverage (% usage hours
receiving RI discount) and RI Utilisation (% use of purchased RIs)
• AWS Budgets can provide email or SNS notifications based on user specified RI
utilization thresholds and now also can notify on RI coverage thresholds
• Automated notifications warning about RI Coverage will improve ability to address
falling RI coverage (caused by newly launched workloads or expiring RIs)

Česko-Slovenský AWS Webinář 07 - Optimalizace nákladů v AWS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Česko-Slovenský AWS Webinář 07 - Optimalizace nákladů v AWS

Similar to Česko-Slovenský AWS Webinář 07 - Optimalizace nákladů v AWS (20)

More from Vladimir Simek

More from Vladimir Simek (18)

Recently uploaded

Recently uploaded (20)

Česko-Slovenský AWS Webinář 07 - Optimalizace nákladů v AWS