Amazon S3 supports a wide range of storage classes to help you cost-effectively store your data. Each of the S3 Storage Classes is designed to support different use cases while reliably protecting your data. In this session, we will look into the different S3 Storage Classes, their respective key features, and the use cases they support, while focusing on the newest storage class S3 Intelligent-Tiering-the first cloud storage class that automatically optimizes storage costs for data with changing access patterns.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Drive Down the Cost of your Data Lake by
Using the Right Data Tiering
Boaz Ziniman
Technical Evangelist - Amazon Web Services
@ziniman
boaz.ziniman.aws
ziniman

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Broad portfolio of storage services
Amazon Elastic
Block Store
Amazon S3 Amazon Glacier
Data movement
OnlineOffline
AWS Snow family
AWS Storage
Gateway Family
AWS Direct Connect
Amazon EFS File Sync
Amazon S3
Transfer Acceleration
Storage Partners
Amazon Kinesis
Data Streams
Amazon Kinesis
Video Streams
Amazon Elastic
File System
NEW! STORAGE CLASS
Object storage
S3 Standard
S3 Intelligent-Tiering
S3 Standard-IA
S3 One Zone-IA
S3 Glacier

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Pillars of cost optimization
Application
Requirements
Data
Organization
Right
Sizing
Monitor,
Analyze,
Optimize

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Define application requirements
Media
master files
Big Data
File
Sharing
Content
Distribution
Archive
Data
Analytics
Backup &
Restore
Dynamic
Websites
Mobile sync
& backup
Disaster
Recover
Re-creatable
data

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Organize data with Object Tags and Prefixes
CustomerID =
3a24xxyz24
Department =
Finance
Project =
FinancialAnalysis
Classification =
Confidential
Environment =
Test
Control access, analyze usage,
lifecycle and replicate objects
Up to 10 mutable metadata tags
(key value pair) per object
Completely customizable (Dept.,
Project, Environment, etc.)
Tag objects when created, later, or
both

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Choose the storage class that fits best
≥ 3 AZs 1 AZ
99.99% 99.5%
Milliseconds Hours
Hours YearsFrequent Infrequent
0 Bytes 5 Terabytes
Reduce storage cost > 80% by choosing the
storage class option that best fits your use case
2 Regions
99.9%

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Monitor, Analyze, and Tier …
Monitor with S3 Inventory,
Amazon CloudWatch, AWS
CloudTrail
Tier and expire storage
with S3 Lifecycle Policy
… or just let S3 Intelligent-Tiering do the work and you save on storage costs automatically
Understand access patterns with
S3 Storage Class Analysis

8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
AWS pricing principles
No upfront
investment
Pay-as-you-go
approach
Pay less by
using more
Pay less as AWS
grows
$

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Decreasing prices and more storage options
1 2
Decreasing storage prices
S3 Standard
(2006)
Glacier
(2012)
S-IA
(2015)
Z-IA
(H1-2018)
INT
(Q4 2018)
Accelerating
innovation
2006 2018

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Your choice of Amazon S3 storage classes
Access FrequencyFrequent Infrequent
• Active, frequently
accessed data
• Milliseconds access
• > 3 AZ
• $0.0210/GB
• Data with changing
access patterns
• Milliseconds access
• > 3 AZ
• $0.0210 to $0.0125/GB
• Monitoring fee per Obj.
• Min storage duration
• Infrequently accessed
data
• Milliseconds access
• > 3 AZ
• $0.0125/GB
• Retrieval fee per GB
• Min storage duration
• Min object size
S3 Standard S3 S-IA S3 Z-IA Amazon Glacier
• Re-creatable, less
accessed data
• Milliseconds access
• 1 AZ
• $0.0100/GB
• Retrieval fee per GB
• Min storage duration
• Min object size
• Archive data
• Select minutes or hours
• > 3 AZ
• $0.0040/GB
• Retrieval fee per GB
• Min storage duration
• Min object size
S3 INT
Amazon Glacier
Deep Archive
• Archive data
• Select hours
• > 3 AZ
• $0.00099/GB
• Retrieval fee per GB
• Min storage duration
• Min object size

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
S3 Storage Class Analysis and S3 Lifecycle Policy
U se S3 Sto r ag e Cl ass A nal ysi s to
i de nti f y stor age age gr oups that ar e
l e ss f r e que ntl y ac c e sse d
Se t S3 L i f e c yc l e Po l i c y to tie r stor age
to low e r c ost stor age c lasse s and e xp i r e
sto r ag e b ase d o n ag e o f o b j e c t
G r e at f o r pr e d ic tabl e w or kl oad s ( o b j e c t
ag e i ndi c ate s ac c e ss f r e que nc y)
F i ne tune anal ysi s b y b uc ke t, p r e f i x, o r
o b j e c t tag

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
S3 Intelligent-Tiering automates cost savings

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
The story behind S3 Intelligent-Tiering
S3 Intelligent-Tiering
New cloud storage class that automates cost savings for customers
Heavy Lifting
Fragmented applications,
constraints on resources
and experience
Unmatched experience
>1M S3 customers,
Trillions of objects,
Millions of requests per second
Amazon Machine Learning
predict future access patterns,
inform storage of objects in most cost-
effective way
+ +

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
S3 Intelligent-Tiering storage class
Automatically moves objects between two
storage tiers:
• Frequent access tier optimized for
frequent use of data
• Lower cost infrequent access tier
optimized for less accessed data
Monitors access patterns and auto-tiers
on granular object level
No performance impact, no operational
overhead
Milliseconds access, > 3 AZs, Monitoring
fee per Object, Minimum storage duration

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Ideal use cases for S3 Intelligent-Tiering
Dynamic cost optimization with no performance impact and no operational overhead
Big Data, Data Lakes
Storage with changing access
patterns used by multiple
applications
Enterprises
Storage accessed by fragmented
applications from various
organizations
Startups
Constraint on resources and
experience to optimize storage
themselves
Amazon S3

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Workload pattern 1 – frequently accessed data
W o r kl o ad c har ac te r i sti c s:
• F r e q u e n t l y a c c e s s e d s t o r a g e
( > 1 0 0 % o f s t o r a g e r e t r i e v e d )
• S o m e t i m e s s m a l l o b j e c t s ( a v g .
o b j e c t s i z e ~ K B )
• S t o r a g e d u r a t i o n s o m e t i m e s
s h o r t
Co m m o n u se c ase s:
• B i g d a t a a n a l y t i c s , d y n a m i c
w e b s i t e h o s t i n g , I o T s e n s o r
d a t a , D N A s e q u e n c e s , f i n a n c i a l
s i m u l a t i o n s , o r i g i n s t o r a g e f o r
C D N
Sto r ag e c l asse s:
• S 3 S t a n d a r d , m a y b e S 3 I N T

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Workload pattern 2 – infrequently accessed data
W o r kl o ad c har ac te r i sti c s:
• O v e r t i m e i n f r e q u e n t l y a c c e s s e d
s t o r a g e ( < 1 0 0 % o f s t o r a g e
r e t r i e v e d a f t e r 9 0 d a y s )
• L a r g e o b j e c t s ( a v g . o b j e c t s i z e
~ M B )
• S t o r a g e d u r a t i o n l o n g t e r m
Co m m o n u se c ase s:
• M o b i l e s y n c & b a c k u p , d a t a l o g s ,
m e d i a a s s e t s f o r g a m i n g ,
c u s t o m e r g e n e r a t e d c o n t e n t ,
d a t a s t o r e d f o r d i s a s t e r
r e c o v e r y
Sto r ag e c l asse s:
• L i f e c y c l e f r o m S 3 S t a n d a r d t o S -
I A o r Z - I A f o r r e - c r e a t a b l e d a t a
• U s e S 3 I N T f o r a u t o m a t e d
t i e r i n g
• U s e G l a c i e r f o r A r c h i v e

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Workload pattern 3 – data with changing access
W o r kl o ad c har ac te r i sti c s:
• D a t a w i t h c h a n g i n g o r
u n p r e d i c t a b l e a c c e s s p a t t e r n s
• M i x o f o b j e c t s i z e s ( a v g . o b j e c t
s i z e ~ M B )
• S t o r a g e d u r a t i o n l o n g t e r m
Co m m o n u se c ase s:
• M a c h i n e L e a r n i n g t r a i n i n g
d a t a , S a t e l l i t e a n d G e o s p a t i a l
i m a g e r y , F i n a n c i a l T r a n s a c t i o n
R e c o r d s , A u t o n o m o u s v e h i c l e
d a t a , d a t a l a k e s
Sto r ag e c l asse s:
• S 3 I N T

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Workload pattern 4 – unknown access patterns
W o r kl o ad c har ac te r i sti c s:
• U n k n o w n w o r k l o a d
• Y o u o n l y k n o w t h a t o b j e c t s a r e
l a r g e ( ~ M B ) a n d s t o r a g e
d u r a t i o n i s l o n g ( ~ m o n t h s )
à S 3 I N T
W o r kl o ad c har ac te r i sti c s:
• U n k n o w n w o r k l o a d
• U n k n o w n o b j e c t s i z e a n d s h o r t
l i v e d o b j e c t s ( < m o n t h s )
à S t a r t w i t h S 3 S t a n d a r d a n d
a f t e r s o m e t i m e l i f e c y c l e l a r g e
o b j e c t s i n t o S 3 I N T

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
https://bit.ly/AWSCost

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Date Tiering Cost Reduction

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Pinterest S3 use cases
Operational Data Store on AWS S3
Business Intelligence: Reporting and Ad Hoc Analysis
Product Analytics and Experimentation
Machine Learning
...and all of those fun Pinterest images
...and more standard use cases like online storage backups

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Storage usage growth
Standard
IA
Glacier

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
@ziniman
Putting it all together
Understand your application requirements
Use tags and prefixes to organize your data
Optimize across all storage classes
Cost optimize on an object level (or tag, prefix, bucket)
S3 Intelligent-Tiering for automated cost savings
INT
S-
IA
Z-IA
Glacier
Std
AWS Cloud enables you to be more innovative, agile, and cost effective

29. Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Boaz Ziniman
Technical Evangelist - Amazon Web Services
@ziniman
boaz.ziniman.aws
ziniman