Get the latest on what we've been developing in Amazon S3. In this session, learn about new advances in S3 performance, security, data protection, storage management, and much more. We'll discuss how to apply the appropriate bucket policies and encryption configurations to enhance security, use S3 Select to accelerate queries, and take advantage of object tagging for data classification.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
PD Dutta
Sr. Product Manager - Amazon S3, Amazon Web Services
SRV301
Latest Updates & Best Practices for
Amazon S3

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
• AWS storage portfolio
• Overview of Amazon S3
• Choice of Storage Classes
• Query data in place with Amazon S3 Select
• Recap of additional Amazon S3 capabilities
• Best Practices

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
The broadest range of storage services
Data movement
OnlineOffline
Data security
and management
Amazon
EFS
Amazon
EBS
Amazon
S3
Amazon
Glacier
AWS KMS
IAM
Amazon CloudWatch
AWS CloudTrail
AWS CloudFormation
AWS Lambda
Amazon Macie
Amazon QuickSight
AWS Snow Family
AWS Storage Gateways
AWS Direct Connect
Amazon EFS File Sync
S3 Transfer Acceleration
Third-Party Applications
Amazon Kinesis Data
Firehose

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS offers the most ways to move data to the cloud
AWS
Direct
Connect
A private
connection
between your data
center, office, or
colocation
environment and
AWS
AWS Snow
family
(Snowball, Snowball
Edge, Snowmobile)
Secure, physical
transport
appliances that
move up to
Exabytes of data
into and out of AWS
AWS
Storage
Gateways
Hybrid storage that
seamlessly connects
on-premises
applications to AWS
storage. Ideal for
backup, DR, bursting,
tiering, or migration
Amazon
Kinesis Data
Firehose
Capture, trans-
form, & load
streaming data
into Amazon S3 for
use with Amazon
business
intelligence and
analytics tools
Amazon EFS
File
Sync
Up to 5x faster file
transfers than open-
source tools. Ideal
for migrating data
into EFS or moving
between cloud file
systems
Amazon S3
Transfer
Acceleration
Up to 300% faster
transfers into and
out of Amazon S3.
Ideal when
working with long
geographic
distances
APN
competency
partners
Integrations
between third-party
vendors and AWS
services. Ideal for
leveraging existing
software licenses
and skills
Networks Roads Hybrid

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Choice of Storage Classes
Active data Archive dataInfrequently accessed data
Milliseconds Minutes to HoursMilliseconds
From 2.1¢-GB/mo. 0.4¢-GB/mo.1.25¢-GB/mo.
Amazon S3
Standard
Amazon S3 Standard -
Infrequent Access
Amazon Glacier
One
Amazon S3 One Zone -
Infrequent Access
1.0¢-GB/mo.

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Management
Cross-Region
Replication
Lifecycle Policies Object TagsEvent
Notifications
Amazon S3
Inventory
AWS CloudTrail
Data Events
Storage Class
Analysis
Amazon CloudWatch
Request Metrics

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Storage Classes

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Regional Storage Classes
Amazon S3 Standard, Amazon S3 Standard - IA, and Amazon Glacier
Regional storage classes:
• Data written across three or more
physical Availability Zones (AZs)
• Data remains durable even in the
event of an entire AZ failure
Designed for:
• Durability: 99.999999999%
• Availability:
• Amazon S3 Standard: 99.99%
• Amazon S3 Standard-IA: 99.9%

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Availability Zones
Amazon S3 regional storage
classes store data in at least 3
AZs
Each AZ can be up to 8
physical data centers
Unavailability of a data center or
an AZ does not impact overall
Amazon S3 availability
Low latency private network
connect data centers and AZs
Physically separate – even extremely
uncommon disasters would only
affect a single AZ
Data is automatically distributed
across a minimum of 3 AZs GEO
separated within an AWS Region

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 One Zone - IA Stores Data Differently
By storing your data within a single AZ …
You get:
- 11 9’s of durability*
- 99.5% designed availability
- Lower-cost storage
*Data is not resilient to the physical loss of the Availability Zone

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Getting Started with Amazon S3 One Zone - IA
Getting data into Amazon S3 One Zone - IA:
• Directly PUT to Amazon S3 One Zone - IA
• Use Lifecycle to transition your data
Using your Amazon S3 One Zone - IA data:
• Works with Storage Class Analysis
• Offers similar performance to Amazon S3 Standard - IA
• Can be used with cross-region replication

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Comparing Infrequent Access Storage Classes
Amazon S3 One
Zone - IA
Amazon S3 Standard
- IA
Amazon S3 Standard
Designed for Durability 99.999999999% 99.999999999% 99.999999999%
Resilience Single AZ Across AZs Across AZs
Designed for Availability 99.5% 99.9% 99.99%
Availability SLA 99% 99% 99.9%
Minimum Object Size 128 KB 128 KB none
Minimum Storage Duration 30 days 30 days none
Retrieval Fee $0.01 per GB
retrieved
$0.01 per GB
retrieved
N/A

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
When should you use Amazon S3 One Zone - IA?
Use Amazon S3 One Zone - IA to store:
• Mobile or enterprise backup data
• Offsite compliance data
• Disaster recovery data
• Derived analysis data

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Choice of Storage Classes
Active data Archive dataInfrequently accessed data
Milliseconds Minutes to HoursMilliseconds
From 2.1¢-GB/mo. 0.4¢-GB/mo.1.25¢-GB/mo.
Amazon S3
Standard
Amazon S3 Standard -
Infrequent Access
Amazon Glacier
One
Amazon S3 One Zone -
Infrequent Access
1.0¢-GB/mo.

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Query Data in Place with Amazon S3 Select

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Select
Select a subset of your object’s data using a SQL expression

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Easy to Get Started
Familiar
Work and scales like GET requests
Integrated
AWS SDK and Presto
Simple to use
Standard SQL expression

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Simple, Faster, and Cheaper!
Available as an API – No
infrastructure or administration
Faster performance compared to
doing it yourself
Pay as you go. The less you
retrieve, the more you save.

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Output
Format: delimited text (CSV,
TSV), JSON …
Amazon S3 Select
Clauses Data types Operators Functions
Select String Conditional String
From Integer, Float, Decimal Math Cast
Where Time stamp Logical Math
Boolean String (Like, ||) Aggregate
Input
Format: delimited text (CSV,
TSV), JSON …
Compression: GZIP …

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Up to 400% faster
Up to 80% cheaper
Amazon S3 Select: Accelerating Big Data Workloads
Amazon S3
Before
Amazon S3
Amazon S3 Select
After

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Select filters your data at the storage layer
AfterBefore

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Select filters your data at the storage layer
AfterBefore
5x faster with 1/40 of the CPU

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Select Supports
Formats:
CSV, TSV, and JSON
Encrypted Objects (SSE)
GZIP Compressed Objects
Integration with:
Amazon S3 Console, AWS CLI, AWS SDK for Java and Python
Presto Connector

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Performance Enhancements

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 Request Performance Increase

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 Request Performance Increase

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
S3 Request Performance Increase

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Additional Amazon S3 Capabilities

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Additional Amazon S3 capabilities
• Free Trusted Advisor checks
• Bucket permissions checks in the Amazon S3 console
• Monitoring with AWS Config
• Default encryption for S3 buckets
• Encryption status in Amazon S3 inventory reports
• Creation of time stamp in Amazon S3 inventory
• Amazon Athena integration with Amazon S3 inventory and AWS
CloudTrail
• Event notification supports encrypted Amazon SQS queues

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Free Trusted Advisor Checks for Bucket Permissions
Checks buckets in Amazon S3 that have open access permissions or allow access
to any authenticated AWS user.
Alert Criteria
Yellow: The bucket ACL allows list access for "Everyone" or "Any Authenticated
AWS User."
Yellow: A bucket policy allows any kind of open access.
Red: The bucket ACL allows Upload/Delete access for "Everyone" or "Any
Authenticated AWS User."

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Permission Checks in the Amazon S3 Console

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Monitoring with AWS Config
• Managed rules to identify insecure bucket
configurations:
• s3-bucket-public-write-prohibited
• s3-bucket-public-read-prohibited
• Managed rules to verify that Amazon S3 features are
enabled
• Logging
• Versioning
• Visualize the changes in your S3 bucket configurations
(Policies, Versioning, Lifecycle Rules).

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Encryption by Default
Automatically encrypts all objects written to your Amazon S3 bucket
• Choose SSE-S3 or SSE-KMS
• Makes it easy to satisfy compliance
needs

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 Inventory
Saves Time Daily or Weekly Delivery CSV or ORC Format
• Includes encryption status of each object
• Amazon S3 Inventory files can be encrypted
• Available in CSV and ORC file format
• Compatible with Amazon Athena, Amazon Redshift Spectrum, and AWS Glue
• Added creation time stamp to indicate when the Amazon S3 inventory creation began
Low-cost alternative to the LIST API delivered into your bucket

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Athena Works with Amazon S3 Inventory
Allows you to run SQL queries on your Amazon S3 Inventory files to:
-Identify unencrypted storage
-Visualize your storage by file type, etc.
-View replication status of your objects
-Analyze your objects by size

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Athena Works with AWS CloudTrail
Create a table directly in the AWS CloudTrail console
Run analytics on your API calls and events
Start with the basics and iterate on your queries over time:
SELECT
useridentity.arn,
eventname,
sourceipaddress,
eventtime
FROM cloudtrail_logs
LIMIT 100;

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Event Notifications Supports SSE-Enabled SQS Queues
• Amazon S3 event notifications are sent in response to actions in
Amazon S3 like PUT, POST, COPY, or DELETE.
• Event Notifications can be sent to Amazon SNS, Amazon SQS, or
AWS Lambda
• Messages are encrypted using a key provided by AWS Key
Management System (AWS KMS).

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Best Practices

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Best Practices
• Security best practice
• Start with the minimum set of permissions
• Using encryption with Amazon S3
• AWS KMS
• Cross-region replication
• With Amazon S3 One Zone - IA
• Support for AWS KMS
• Ownership overwrite
• Storage class analysis and lifecycle policies
• Amazon CloudWatch bucket and request metrics

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Access to encryption keys
IAM Security Token
Service
Temporary
Credentials
Customer
Master Key
Customer Data
Keys
Ciphertext Key Plaintext Key
Amazon S3
S3 object
……
Name: MyData
Key: Ciphertext Key
…..
My Data
My Data

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Encryption Support in Amazon S3
• Encryption in motion – HTTPS/TLS
• Encryption at rest
• Client side encryption – Encrypt before upload
• Server-side encryption
• SSE-S3 – Amazon S3 manages the data and master encryption
keys
• SSE-C – You manage the encryption key
• SSE-KMS – Amazon S3 manages the data key; you manage the
master key in the AWS KMS

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
Region A Region B
Cross-region connectivity

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
Region A Region B
Cross-region connectivity

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
Amazon S3 Standard
Cross-region connectivity
Amazon S3 Standard

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
Amazon S3 Standard
Cross-region connectivity
Amazon S3 Standard - IA

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
Amazon S3 Standard
Cross-region connectivity
Amazon Glacier

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
Amazon S3 Standard
Cross-region connectivity
Amazon S3 One Zone - IA

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Cross-Region Replication with AWS KMS
Automatically replicate data to any other AWS Region
• Replicate by object, bucket, or prefix
• Support for SSE-KMS encrypted objects
• Ownership overwrite
• Change the object owner in the destination region
Cross-region connectivity
S3 One Zone - IAS3 One Zone - IA

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Class Analysis
Daily storage class
analysis
Data-driven storage management and cost optimization for Amazon S3
Export storage class analysis
to your S3 bucket
Filter by bucket, prefix,
or object tags
• Monitors access patterns to understand your storage usage
• After 30 days, recommends when to move objects to other storage classes
• Export file includes a daily report of storage, retrieved bytes, and GETs by object age

50. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Class Analysis

51. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Class Analysis

52. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lifecycle Policies
Create rules to automatically transition or expire your storage
Lifecycle rules take action based on object age
Example policy:
• Move all objects older than 90 days to Amazon S3 Standard–Infrequent Access
• Move all objects older than 180 days to Amazon Glacier
Amazon S3
Standard
Amazon S3 Standard -
Infrequent Access
Amazon Glacier

53. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Lifecycle Policies
Create rules to automatically transition or expire your storage
Lifecycle rules take action based on object age
Example policy:
• Move all objects older than 90 days to Amazon S3 One Zone - Infrequent Access
• Move all objects older than 180 days to Amazon Glacier
Amazon S3
Standard
Amazon S3 One Zone -
Infrequent Access
Amazon Glacier
One

54. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
CloudWatch Metrics for Amazon S3
Free Daily Storage Metrics:
Bucket Size (Bytes) and Number of Objects

55. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon CloudWatch Metrics for Amazon S3
Request metrics provided at 1-minute intervals:
AllRequests
GetRequests
PutRequests
DeleteRequests
HeadRequests
PostRequests
ListRequests
BytesDownloaded
BytesUploaded
4xxErrors
5xxErrors
FirstByteLatency
TotalRequestLatency

56. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Please complete the session survey in the
summit mobile app.

57. Submit Session Feedback
1. Tap the Schedule icon. 2. Select the session you
attended.
3. Tap Session Evaluation to
submit your feedback.

58. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!