Get the latest on what we've been developing in Amazon S3. In this session, we share best practices for performance optimization, security, data protection, storage management, and much more. We discuss ways to optimize key naming to increase throughput, apply the appropriate AWS Identity and Access Management (IAM) and encryption configurations, and take advantage of object tagging and other features to enhance security.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Rob Wilson
Sr. Product Manager—Amazon S3, Amazon Web Services
SRV301
Latest Updates & Best Practices for
Amazon S3

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
• Overview of Amazon S3
• S3 One Zone – Infrequent Access
• S3 Select
• Recap of Recent Launches
• Best Practices

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The AWS Storage Portfolio
Data Transfer
Third-party
connectors
S3 Transfer
Acceleration
File
Amazon EFS
Object
Amazon GlacierAmazon S3
Block
Amazon EBS
(persistent)
Amazon EC2
Instance Store
(ephemeral)
AWS
Snow Family
AWS Storage
Gateway
AWS Direct
Connect
Amazon
Kinesis
Amazon EFS
File Sync

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Benefits of Amazon S3 & Amazon Glacier
Durable, Available, & Scalable Security & Compliance Query in Place
Flexible Management Ecosystem

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Choice of Storage Classes
Active data Archive dataInfrequently accessed data
Milliseconds Minutes to HoursMilliseconds
From 2.1¢-GB/mo. 0.4¢-GB/mo.1.25¢-GB/mo.
Amazon S3
Standard
Amazon S3 Standard–
Infrequent Access
Amazon Glacier
One
Amazon S3 One Zone–
Infrequent Access
1.0¢-GB/mo.

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Management
Cross-Region
Replication
Lifecycle Policies Object TagsEvent
Notifications
Amazon S3
Inventory
AWS CloudTrail
Data Events
Storage Class
Analysis
Amazon CloudWatch
Request Metrics

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
New Launch:
S3 One Zone - Infrequent Access

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 Standard, S3 Standard–IA, and Amazon Glacier
Regional services:
• Data written across three or more
physical Availability Zones (AZs)
• Data remains durable even in the
event of an entire AZ failure
Designed for:
• Durability: 99.999999999%
• Availability:
• Amazon S3 Standard: 99.99%
• Amazon S3 Standard-IA: 99.9%

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Availability Zones
S3 stores data in at least 3
AZs
Each AZ can be up to 8
physical data centers
Unavailability of a data center
or an AZ does not impact
overall S3 availability
Low latency private network
connect data centers and
AZs
Physically separate – even
extremely uncommon disasters
would only affect a single AZ
Data is automatically distributed
across a minimum of 3 AZs GEO
separated within an AWS Region

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Introducing a New Kind of Storage in Amazon S3
Data written across three or
more physical AZs

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data written across three or
more physical AZs
Not all data needs to be
stored in three or more AZs
Introducing a New Kind of Storage in Amazon S3

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data written across three or
more physical AZs
Some data is easy to
recreate or saved elsewhere
Introducing a New Kind of Storage in Amazon S3

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data written across three or
more physical AZs
Data written within one
physical AZ
Introducing a New Kind of Storage in Amazon S3

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 Standard
S3 Standard – IA
Amazon Glacier
S3 One Zone - IA
Introducing a New Kind of Storage in Amazon S3

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 One Zone – IA Stores Data Differently
By storing your data within a single AZ …
You get:
- 11 9’s of durability*
- 99.5% designed availability
- Lower-cost storage
*Data is not resilient to the physical loss of the Availability Zone

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Getting Started with S3 One Zone – IA
Getting data into S3 One Zone - IA:
• Directly PUT to S3 One Zone - IA
• Use Lifecycle to transition your data
Using your S3 One Zone - IA data:
• Works with Storage Class Analysis
• Offers similar performance to S3 Standard – IA
• Can be used with cross-region replication

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Comparing Infrequent Access Storage Classes
S3 One Zone - IA S3 Standard - IA
Designed for Durability 99.999999999% 99.999999999%
Resilience Single AZ Across AZs
Designed for Availability 99.5% 99.9%
Availability SLA 99% 99%
Minimum Object Size 128 KB 128 KB
Minimum Storage Duration 30 days 30 days
Retrieval Fee $0.01 per GB retrieved $0.01 per GB retrieved

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
When should you use S3 One Zone - IA?
Use S3 One Zone – IA to store:
• Mobile or enterprise backup data
• Offsite compliance data
• Disaster recovery data
• Derived analysis data

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Choice of Storage Classes
Active data Archive dataInfrequently accessed data
Milliseconds Minutes to HoursMilliseconds
From 2.1¢-GB/mo. 0.4¢-GB/mo.1.25¢-GB/mo.
Amazon S3
Standard
Amazon S3 Standard–
Infrequent Access
Amazon Glacier
One
Amazon S3 One Zone–
Infrequent Access
1.0¢-GB/mo.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
New Launch:
S3 Select is Generally Available

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 Select
Select a subset of your object’s data using a SQL expression

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Easy to Get Started
Familiar
Work and scales like GET requests
Integrated
AWS SDK and Presto
Simple to use
Standard SQL expression

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Simple, Faster, and Cheaper!
Available as an API – No
infrastructure or administration
Faster performance compared to
doing it yourself
Pay as you go. The less you
retrieve, the more you save.

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Up to 400% faster
Up to 80% cheaper
S3 Select filters your data at the storage layer
Amazon S3
Before
Amazon S3
S3 Select
After

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 Select filters your data at the storage layer
AfterBefore

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 Select filters your data at the storage layer
AfterBefore
5x faster with 1/40 of the CPU

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 Select Now Supports
Formats:
CSV, TSV, and JSON
Encrypted Objects (SSE)
GZIP Compressed Objects
Integration with:
S3 Console, AWS CLI, AWS SDK for Java and Python
Presto Connector

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Recent Launches

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Recent Launches
• Free Trusted Advisor checks
• Bucket permissions checks in the S3 console
• Default encryption for S3 buckets
• Encryption status in S3 inventory reports
• Creation time stamp in S3 inventory
• Athena integration with S3 inventory and AWS CloudTrail
• Event notification supports encrypted Amazon SQS queues

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Free Trusted Advisor Checks for Bucket
PermissionsChecks buckets in Amazon S3 that have open access permissions or allow
access to any authenticated AWS user.
Alert Criteria
Yellow: The bucket ACL allows list access for "Everyone" or "Any
Authenticated AWS User."
Yellow: A bucket policy allows any kind of open access.
Red: The bucket ACL allows Upload/Delete access for "Everyone" or "Any
Authenticated AWS User."

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Permission Checks in the S3 Console

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Encryption by Default
Automatically encrypts all objects written to your Amazon S3
bucket
• Choose SSE-S3 or SSE-KMS
• Makes it easy to satisfy
compliance needs

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Inventory
Saves Time Daily or Weekly Delivery CSV or ORC Format
• Includes encryption status of each object
• Amazon S3 Inventory files can be encrypted
• Available in CSV and ORC file format
• Compatible with Amazon Athena, Amazon Redshift Spectrum, and AWS Glue
• Added creation time stamp to indicate when the S3 inventory creation began
Low-cost alternative to the LIST API delivered into your bucket

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Athena Works with S3 Inventory
Allows you to run SQL queries on your S3 Inventory files to:
-Identify unencrypted storage
-Visualize your storage by file type, etc.
-View replication status of your objects
-Analyze your objects by size

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Athena Works with AWS CloudTrail
Create a table directly in the AWS CloudTrail console
Run analytics on your API calls and events
Start with the basics and iterate on your queries over time:
SELECT
useridentity.arn,
eventname,
sourceipaddress,
eventtime
FROM cloudtrail_logs
LIMIT 100;

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Event Notifications Supports SSE-Enabled SQS
Queues
• S3 event notifications are sent in response to actions in
Amazon S3 like PUT, POST, COPY, or DELETE.
• Event Notifications can be sent to Amazon SNS, Amazon
SQS, or AWS Lambda
• Messages are encrypted using a key provided by AWS Key
Management System (AWS KMS).

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Best Practices

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Best Practices
• Using encryption with Amazon S3
• AWS Key Management Service (AWS KMS)
• Cross-region replication
• With S3 One Zone - IA
• Support for AWS KMS
• Ownership overwrite
• Storage class analysis and lifecycle policies
• Amazon CloudWatch bucket and request metrics
• Performance best practices

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Encryption Support in Amazon S3
• Encryption in motion – HTTPS/TLS
• Encryption at rest
• Client side encryption – Encrypt before upload
• Server side encryption
• SSE-S3 – Amazon S3 manages the data and master
encryption keys
• SSE-C – You manage the encryption key
• SSE-KMS – Amazon S3 manages the data key; you
manage the master key in the AWS KMS

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
Region A Region B
Cross-region connectivity

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
Region A Region B
Cross-region connectivity

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
S3 Standard
Cross-region connectivity
S3 Standard

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
S3 Standard
Cross-region connectivity
S3 Standard - IA

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
S3 Standard
Cross-region connectivity
Amazon Glacier

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
S3 Standard
Cross-region connectivity
S3 One Zone - IA

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
Cross-region connectivity
S3 One Zone - IAS3 One Zone - IA

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Class Analysis
Daily storage
class analysis
Data-driven storage management and cost optimization for Amazon S3
Export storage class
analysis to your S3 bucket
Filter by bucket,
prefix, or object tags
• Monitors access patterns to understand your storage usage
• After 30 days, recommends when to move objects to other storage classes
• Export file includes a daily report of storage, retrieved bytes, and GETs by object age

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Class Analysis

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Class Analysis

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lifecycle Policies
Create rules to automatically transition or expire your storage
Lifecycle rules take action based on object age
Example policy:
• Move all objects older than 90 days to S3 Standard–Infrequent Access
• Move all objects older than 180 days to Amazon Glacier
Amazon S3
Standard
Amazon S3 Standard–
Infrequent Access
Amazon Glacier

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lifecycle Policies
Create rules to automatically transition or expire your storage
Lifecycle rules take action based on object age
Example policy:
• Move all objects older than 90 days to S3 One Zone–Infrequent Access
• Move all objects older than 180 days to Amazon Glacier
Amazon S3
Standard
Amazon S3 One Zone–
Infrequent Access
Amazon Glacier
One

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CloudWatch Metrics for Amazon S3
Free Daily Storage Metrics:
Bucket Size (Bytes) and Number of Objects

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon CloudWatch Metrics for Amazon S3
Request metrics provided at 1-minute intervals:
AllRequests
GetRequests
PutRequests
DeleteRequests
HeadRequests
PostRequests
ListRequests
BytesDownloaded
BytesUploaded
4xxErrors
5xxErrors
FirstByteLatency
TotalRequestLatency

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Getting High Throughput with Amazon S3
Amazon S3 automatically scales to thousands of requests per second
per prefix based on your steady state traffic
• Due to recent Amazon S3 performance enhancements, most customers will
never need to worry about introducing entropy in key names
• Consider using a three- or four-character hash
examplebucket/232a-2017-26-05-15-00-00/cust1234234/photo1.jpg
examplebucket/7b54-2017-26-05-15-00-00/cust3857422/photo2.jpg
examplebucket/921c-2017-26-05-15-00-00/cust1248473/photo2.jpg
examplebucket/animations/232a-2017-26-05-15-00-00/cust1234234/animation1.obj
examplebucket/videos/ba65-2017-26-05-15-00-00/cust8474937/video2.mpg
examplebucket/photos/8761-2017-26-05-15-00-00/cust1248473/photo3.jpg
A more LIST friendly version:

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Please complete the session survey
in the summit mobile app.

56. Submit Session Feedback
1. Tap the Schedule icon. 2. Select the session
you attended.
3. Tap Session
Evaluation to submit your
feedback.

57. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank You!