Amazon S3 powers some of the most performance-sensitive workloads running in the cloud today. 2018 has brought massive improvements in Amazon S3 request rates and overall throughput. In this chalk talk, you have the opportunity to interact directly with the Amazon S3 engineering team responsible for the S3 index and many other components at the core of these improvements. We discuss what's changed, including the removal of the need for prefix randomization, and the ways you can improve your architecture to enhance performance in the unique environment of Amazon S3.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Optimizing Performance in Amazon
S3
Dave Phillips
Principal Product Manager
AWS/S3
S T G 3 6 7
Seth Markle
Principal Software
Development Engineer
AWS/S3
Vishwas Narendra
Senior Software
Development Engineer
AWS/S3

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon S3 performance optimization
• What we hear from you …
1. How do I get the highest request rates?
2. How do I saturate my compute, network, and storage resources?
3. How do I maximize my single-threaded throughput?
4. How do I achieve more predictable outlier performance?
5. How do I optimize data query performance against Amazon S3?
• For this topic, check out session STG378:
• “Querying Data in Place with AWS Object Storage Features and Analytics Tools.”
• Where do you start?
• Your object naming scheme!
• We’ll review this first, then walk through a streaming video workflow use case
as an example.

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Object naming scheme
ExampleAWSbucket/Logistics/packing-list.pdf

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Up to 3,500 PUT* tps
Up to 5,500 GET tps
ExampleAWSbucket
Let’s look at how prefixes scale
request rate performance
BucketName/Prefix:
ExampleAWSbucket/LogFiles/
ExampleAWSbucket/Logistics/
ExampleAWSbucket/…
* PUT, POST, and DELETE are all included in the PUT tps
tps = transactions per second
Initial partition

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
3,500 PUT tps
5,500 GET tps
3,500 PUT tps
5,500 GET tps
2nd partition
Initial partition
ExampleAWSbucket
BucketName/Prefix:
ExampleAWSbucket/LogFiles/
ExampleAWSbucket/Logistics/
ExampleAWSbucket/…
Let’s look at how prefixes scale
request rate performance
/Log
/…
PUT, POST, and DELETE are all included in the PUT tps
tps = transactions per second

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
3,500 PUT tps
5,500 GET tps
3,500 PUT tps
5,500 GET tps
3,500 PUT tps
5,500 GET tps
2nd partition
Files/
istics/
3rd partition
3,500 PUT tps
5,500 GET tps
…
ExampleAWSbucket
BucketName/Prefix:
ExampleAWSbucket/LogFiles/
ExampleAWSbucket/Logistics/
ExampleAWSbucket/…
Let’s look at how prefixes scale
request rate performance
/Log
/…
PUT, POST, and DELETE are all included in the PUT tps
tps = transactions per second
Initial partition

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
3,500 PUT tps
5,500 GET tps
3,500 PUT tps
0 GET tps
0 PUT tps
5,500 GET tps
Request rate performance is allocated proportionally
PUT, POST, and DELETE are all included in the PUT tps
tps = transactions per second

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Streaming video workflow example
10.1.1.1
Amazon S3
Endpoints
10.2.2.2
10.3.3.3
10.4.4.4
Storage
Objects
• How do I get the highest
request rates?
• How do I saturate my
compute, network, and
storage resources?
• How do I maximize my single-
threaded throughput?
• How do I achieve more
predictable outlier
performance?
• How do I optimize data query
performance against Amazon
S3?

10. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Dave Phillips
dvphllps@amazon.com

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.