A whole host of things on the internet -- photos, videos, podcasts, pdfs, etc. -- are served from files stored on the cloud. If files are the currency of information, then AWS S3 is like the Federal Reserve i.e. the biggest storehouse of files on earth. In this technical talk, you’ll learn practical tips to secure files on S3, so only the right parties are allowed access. The talk will be full of examples and live demonstration so the audience can get a feel for common mistakes in securing files on S3 and how to do it right.

1. Let the right one in:
Securing files on AWS S3
Sunil Kowlgi
Houston Techfest
5/5/2018

2. About me
• Founder of Outclip, a startup focused on video sharing for remote
teams: https://checkoutclip.com
• 2+ years of experience with AWS S3
• Contact: sunil@checkoutclip.com

3. Amazon S3
• Simple Storage Service (S3)
• Among the top 3 most used AWS services
• S3 is an object storage system, which is quite different than file
system:
• Flat vs. hierarchical address space: makes S3 highly scalable. E.g.
‘prefix/filename’ where prefix is not really a folder.
• Customizable metadata vs. fixed metadata: can add custom attributes to each
object
• Pricing comparison:
• Dropbox: 1TB for $9.99 a month, S3: 1TB for $23 at a minimum

4. AWS S3 Shared Responsibility Security Model

5. Demo: S3 Access Denied
• https://houston-techfest-bucket.s3.amazonaws.com/Wildlife.mp4
• https://signing-demo-bucket.s3.amazonaws.com/Wildlife.mp4

6. S3 Security Fiascos
• “Respondent has engaged in a number of practices that, taken
together, failed to provide reasonable security to prevent
unauthorized access to Rider and Driver personal information stored
in the Amazon S3 Datastore. Among other things, Respondent:
• a. Until approximately September 2014, failed to implement reasonable
access controls to safeguard data stored in the Amazon S3 Datastore. For
example, Respondent:
• i. failed to require programs and engineers that access the Amazon S3 Datastore to use
distinct access keys, instead permitting all programs and engineers to use a single AWS
access key that provided full administrative privileges over all data in the Amazon S3
Datastore;
• ii. failed to restrict access to systems based on employees’ job functions; and
• iii. failed to require multi-factor authentication for access to the Amazon S3 Datastore;”

7. S3 Security Fiascos
• In 2017, WWE’s misconfigured S3 exposed 3 million emails
• Security researchers discovered two open Amazon S3 Buckets that
contained a trove of private information collected for WWE marketing
purposes.
• An estimated 12 percent of all the information (several gigabytes) was
set to “Public” access and available for anybody with internet
connection to view and download.

8. Software Security
• While software engineering is about ensuring that certain things
happen, security is about ensuring that they don’t.
– Ross Anderson, Security Engineering
• Many systems fail because designers protect the wrong things or
protect the right things but the wrong way.
• Robust security design requires that the protection goals are made
explicit.

9. Good Security Engineering Requires Four
Things
• Policy: what are the security goals?
• Mechanism: how to implement the policy?
• Assurance : how reliable are the mechanisms?
• Incentive: understand motivation for people who both secure and try
to attack your system

10. S3 File Security Policy
1. Deny access to everyone except some users
2. Encrypt data at rest
3. Encrypt data before sending it to S3
4. Manage access through special URL
5. Manage access without special URL

11. Using S3
• Object: the thing you store e.g. .mp4, .jpg, etc.
• Bucket: where all your objects go
• Bucket policy: controls access to the bucket
• Possible operations: listObjects, getObject, putObject, etc.
• IAM (Identity and Access Management): AWS’ system to set up and
manage user access to all AWS services, not just S3
• Ways to access S3:
• Command line interface (CLI)
• Web console
• API for programmatic access

12. Demo: S3 console and CLI

13. S3 File Security Policy
1. Deny access to everyone except some users
2. Encrypt data at rest
3. Encrypt data before sending it to S3
4. Manage access through special URL
5. Manage access without special URL

14. Managing Access to S3
• Ways to manage access to buckets:
• Bucket Policy
• IAM Policy a.k.a User Policy
• ACLs
• “When Amazon S3 receives a request—for example, a bucket or an
object operation—it first verifies that the requester has the necessary
permissions. Amazon S3 evaluates all the relevant access policies,
user policies, and resource-based policies (bucket policy, bucket ACL,
object ACL) in deciding whether to authorize the request.”

15. Bucket Policy vs IAM policy
Sample Bucket Policy Sample IAM Policy

16. ACL – Access Control List
• ACLs are a legacy access control mechanism
• AWS recommends bucket policy over ACL
• Use ACL when you want to manage permissions on individual objects
Sample ACL

17. Bucket Policy vs. IAM Policy vs ACL: who
wins?
• Access is based on “least-privilege union of all the permissions”

18. Demo: Bucket policy and ACL

19. How to Write a Good Bucket Policy
• Explicitly deny everyone except a few users
• Explicitly allow a few users to perform specific operations:
• E.g. Deny everyone but allow Alice to execute S3GetObject on the bucket

20. Demo: Bucket Policy Best Practice

21. S3 File Security Policy
1. Deny access to everyone except some users
2. Encrypt data at rest
3. Encrypt data before sending it to S3
4. Manage access through special URL
5. Manage access without special URL

22. Encrypting data
• Server-side encryption
• Amazon S3 encrypts each object at rest with a unique key
• Client-side encryption
• Object is encrypted by client before it’s sent to S3
• Key management is hard, but AWS Key Management Service (KMS) comes to
the rescue

23. Demo: Server-side Encryption

24. Demo: Client-side Encryption

25. Client-side Encryption
AWS Key
Management
Service (KMS)
AWS S3

26. S3 File Security Policy
1. Deny access to everyone except some users
2. Encrypt data at rest
3. Encrypt data before sending it to S3
4. Manage access through special URL
5. Manage access without special URL

27. Manage Access Through Special URL
• Every object has a unique URL (let’s call it a normal URL):
• E.g. https://myBucket.s3.amazonaws.com/vacation.mp4
• There are cases where you want to deny access to the object’s normal
URL but allow access to the object through a special URL.
• A special URL is advantageous because it can:
• Allow access to specific object without having to relax bucket policy
• Have a start date and expiry date
• Condition based access: e.g. access based on IP address

28. Manage access through special URL
• Two ways to get a special URL for an object
• S3 Pre-signed URL
• Cloudfront signed URL
• S3 Presigned URLs can be used to both get and put objects
• Can only specify expiration date
• Cloudfront Signed URLs are used only to get objects
• Can specify start and expiration date
• Can restrict to IP address or range of IP addresses

29. What is Cloudfront?
• AWS Cloudfront is a content delivery network (CDN)
• Its primary advantage is faster access for users by caching objects
• Another advantage: Cloudfront provides an extra layer of security
policy over S3:
• Signed URLs
• Signed cookies

30. What is Cloudfront?

31. Demo: Pre-signed URL

32. Demo: Cloudfront Signed URL

33. S3 File Security Policy
1. Deny access to everyone except some users
2. Encrypt data at rest
3. Encrypt data before sending it to S3
4. Manage access through special URL
5. Manage access without special URL

34. Manage access without special URL
• There are cases when you want to give users access to multiple
objects
• For each user if you’re signing URLs for multiple objects, that’s not
scalable. To scale you’ll need to use signed cookies.

35. Cloudfront Signed Cookies
• For signed cookies you’ll need Cloudfront. Signed cookies allow similar policies as signed URLs:
• Can specify start and expiration date
• Can restrict to IP address or range of IP addresses
• With one important addition: you can use a wildcard character (‘*’) in the resource name in the policy
statement to give access to multiple objects
• Example policy:
{
Statement : [
{
Resource : "https://example.com/privatecontent/*",
Condition: { DateLessThan: { 'AWS:EpochTime’: link_expiration_timestamp } }
}
]
}

36. Summary
• AWS Shared Responsibility security model implies you’re responsible for
configuring S3 correctly. S3 misconfigurations can lead to compromised data
security.
• There are several mechanisms to protect data in S3 that enable a range of policy
options;
1. Deny access to everyone except some users
2. Encrypt data at rest
3. Encrypt data before sending it to S3
4. Manage access through special URL
5. Manage access without special URL
• Knowledge of these mechanisms along with a restrictive security policy approach
can improve the security of your data in S3

37. Appendix

38. Examples
• https://github.com/kowlgi/effective-s3-file-security

39. References
• https://d1.awsstatic.com/whitepapers/Security/AWS_Security_Best_Practices.pdf
• https://www.ftc.gov/enforcement/cases-proceedings/152-3054/uber-technologies-inc
• https://mackeepersecurity.com/post/world-wrestling-entertainment-leaks-3-million-
emails
• http://www.cl.cam.ac.uk/~rja14/book.html
• https://docs.aws.amazon.com/AmazonS3/latest/dev/how-s3-evaluates-access-
control.html
• https://aws.amazon.com/blogs/security/iam-policies-and-bucket-policies-and-acls-oh-
my-controlling-access-to-s3-resources/
• https://aws.amazon.com/about-aws/whats-new/2011/10/04/amazon-s3-announces-
server-side-encryption-support/
• https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PrivateConten
t.html