MasterclassAmazon S3 – beyond simple storage     Ryan Shuttleworth – Technical Evangelist                  @ryanAWS
Masterclass                       A technical deep dive beyond the basicsHelp educate you on how to get the best from AWS ...
Amazon S3           It’s more than just a ‘simple’ storage platform          A sophisticated 21st century distributed syst...
“Spotify needed a storage solution that  could scale very quickly without incurringlong lead times for upgrades. This led ...
You put it in S3AWS stores with 99.999999999% durability
Highly scalable web                      access to objects     You put it in S3AWS stores with 99.999999999% durabilityMul...
but it’s more than just a  simple storage service
Objects in S3  1500.000                                           1.3 Trillion  1250.000  1000.000   750.000   500.000   2...
Highly scalable data storageA web store, not a file system                                  Access via APIs               ...
A web store, not a file system                     Write once,                     read many                     (WORM) Ev...
A web store, not a file system                                             Load balancers                                 ...
A web store, not a file system                                             Load balancers                                 ...
A web store, not a file system                                             Load balancers                                 ...
A web store, not a file system                                             Load balancers                                 ...
A web store, not a file system                                             Load balancers                                 ...
A web store, not a file system                                             Load balancers                                 ...
A web store, not a file system                                             Load balancers                                 ...
A web store, not a file system                                             Load balancers                                 ...
A web store, not a file system                               Write once,                               read many          ...
New objects                                            Synchronously stores your data across multiple                     ...
A regional serviceYour data never leaves a region unless you move it
Storage classesControlling the way S3 holds your data
Amazon S3 storage classes       Standard     Designed to provide99.999999999% durability and 99.99% availability of object...
Amazon S3 storage classes                                        Reduced       Standard                    Redundancy Stor...
Amazon S3 storage classes                                        Reduced       Standard                    Redundancy Stor...
Amazon S3 storage classes                                   Reduced      Standard                Redundancy Storage       ...
NamespacesObject naming, buckets and keys
Amazon S3 namespaceGlobally uniquebucket name + object name (key)
Amazon S3 namespace                            Amazon S3         bucket                                      bucketobject ...
Amazon S3 namespace                              Amazon S3     jones-docshare                                media.mydomai...
Amazon S3 namespace    Object key  Unique with a bucket
Amazon S3 namespace                           Object keyMax 1024 bytes UTF-8                            Including ‘path’ p...
Amazon S3 namespace                           Object keyMax 1024 bytes UTF-8                              Including ‘path’...
Throughput optimisation   S3 automatically partitions based upon key prefix:Bucket:          Object keys:mynewgame        ...
Throughput optimisation   S3 automatically partitions based upon key prefix:Bucket:              Object keys:mynewgame    ...
Throughput optimisation   S3 automatically partitions based upon key prefix:Bucket:                Object keys:mynewgame  ...
Throughput optimisation   S3 automatically partitions based upon key prefix:Bucket:             Object keys:mynewgame     ...
Throughput optimisation   S3 automatically partitions based upon key prefix:Bucket:             Object keys:mynewgame     ...
EncryptionSecuring data at rest
Automatic encryption of                        data at rest  SimpleAdditional PUT                                         ...
Server side encryption                    Data                     bucketHigh level design
Server side encryption                                            Encrypted object                    Data                ...
Server side encryption                                            Encrypted object                    Data                ...
Server side encryption                                            Encrypted object                    Data                ...
Access controlsYou’re in control of who does what
You decide what to share                                         Apply policies to buckets and objects          Secure by ...
IAM        Fine grained Administer as part of  role based access Apply policies to S3 atrole, user & group levelAllowActio...
IAM            VS        Bucket Policies        Fine grained                    Fine grained Administer as part of        ...
IAM            VS        Bucket Policies            VS           ACLs        Fine grained                    Fine grained ...
Access controls – bucket policy{"Statement":[{         "Effect":"Allow",         "Principal":{"AWS":["4649-6425", "5243-00...
Access controls – bucket policy{"Statement":[{                  Accounts to allow         "Effect":"Allow",         "Princ...
Access controls – bucket policy{"Statement":[{         "Effect":"Allow",         "Principal":{"AWS":["4649-6425", "5243-00...
Access controls – bucket policy{"Statement":[{         "Effect":"Allow",         "Principal":{"AWS":["4649-6425", "5243-00...
Transitions & Lifecycle    Management Automated management of objects
Lifecycle management
Object deletionPermanently delete objects from S3 Lifecycle management
Object deletionPermanently delete objects from S3 Lifecycle management                                     Object archivin...
GlacierLong term durable archive
Durable                                             Designed for 99.999999999%                                            ...
Expiry         Logs                accessible from S3                                     time
✗Expiry         Logs                accessible from S3                                       Objects                      ...
Object                                      transition toTransition                                     Glacier invoked   ...
Object         Restoration of                                      transition to    object requestedTransition            ...
Object         Restoration of                                      transition to    object requestedTransition            ...
3-5 hour retrieval time We assume you won’t access often
using (client = new AmazonS3Client()){  var lifeCycleConfiguration = new LifecycleConfiguration()  {    Rules = new List<L...
using (client = new AmazonS3Client()){  var lifeCycleConfiguration = new LifecycleConfiguration()  {    Rules = new List<L...
using (client = new AmazonS3Client()){  var lifeCycleConfiguration = new LifecycleConfiguration()  {    Rules = new List<L...
POST /ObjectName?restore HTTP/1.1Host: BucketName.s3.amazonaws.comDate: dateAuthorization: signatureValueContent-MD5: MD5<...
POST /ObjectName?restore HTTP/1.1Host: BucketName.s3.amazonaws.comDate: dateAuthorization: signatureValueContent-MD5: MD5<...
Website hostingStatic sites straight from S3
It’s a web service……so serve up web content
Setting default documents        Redirecting requests
{    "Version":"2008-10-17",    "Statement":[{      "Sid":"PublicReadGetObject",            "Effect":"Allow",         "Pri...
Website addressing{bucket-name}.s3-website-{region}.amazonaws.come.g. mybucket.s3-website-eu-west-1.amazonaws.comNormal ad...
Record set for:aws-examples.info       R53
Record set for:                        aws-examples.info                               R53                                ...
Record set for:                        aws-examples.info                                 R53                              ...
Record set for:                        aws-examples.info                                          A Record ‘Alias’ to S3 w...
Record set for:                           aws-examples.info            CNAME for www. to:               A Record ‘Alias’ t...
Object versioning Preserving object histories
Bucket level             PersistentAutomatically preserves   Even deleted object  all copies of objects      history is held
>>>   import boto>>>   conn = boto.connect_s3()>>>   from boto.s3.bucket import Bucket>>>   from boto.s3.key import Key>>>...
>>>   import boto>>>   conn = boto.connect_s3()>>>   from boto.s3.bucket import Bucket>>>   from boto.s3.key import Key>>>...
>>> key = bucket.get_key(myfile.txt,version_id=8cjozv9Hmkzum8xj.8q8BZxR5CuXnzon’)>>> key.get_contents_as_string()this is v...
>>> key = bucket.get_key(myfile.txt,version_id=8cjozv9Hmkzum8xj.8q8BZxR5CuXnzon’)>>> key.get_contents_as_string()this is v...
>>> key = bucket.get_key(myfile.txt,version_id=8cjozv9Hmkzum8xj.8q8BZxR5CuXnzon’)>>> key.get_contents_as_string()this is v...
Error response: link      expired
MetadataSystem and user generated
System metadata                      Name        Description                                                          Edit...
User metadataKey-value pairs stored with objects and returned with requestsx-amz-meta-{your metadata key in here}
>>> key.set_metadata(’my_tag, ’my metadata)>>> key.get_metadata(’my_tag)’my metadata
Cloud FrontDelivery from global edge locations
Download                                      Enable static and dynamic assets to be                                      ...
S3 BucketCache control      Forwarding                 Optional CNAMEEdge access controls
S3 Bucket                            Optional CNAMELogging to S3
Content
Download                                                    distribution<html><script type=text/javascriptsrc=’http://d2ew...
Summary
Stop doing these:          Capacity planning      Management of storageWorrying about backing up the backup       Fixing b...
Bootstrapping                            Store scripts and drive EC2   Backups & archiveApplication backends           ins...
aws.amazon.com
Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3
Masterclass Webinar: Amazon S3
Upcoming SlideShare
Loading in...5
×

Masterclass Webinar: Amazon S3

7,213

Published on

AWS Evangelist, Ryan Shuttleworth, explores the extended features of AWS S3 in this Masterclass webinar.

AWS S3 hosts over 1.3 trillion objects and is used for storing a wide range of data, from system backups, web site assets and digital media. In this webinar we will explain the features of S3 from static website hosting, through server side encryption to Glacier integration. We'll dive deep into the feature sets of S3 to give a rounded overview of its capabilities, looking at common use cases, APIs and best practice.

To see the recording and demostration for this webinar on YouTube, please click on the following links:

Masterclass Webinar: Amazon S3 Recording - http://www.youtube.com/watch?v=HHuRJZChCYQ
Masterclass Webinar: Amazon S3 Demonstration - http://www.youtube.com/watch?v=JuffWMBeJkw

Published in: Technology
0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
7,213
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
1
Comments
0
Likes
12
Embeds 0
No embeds

No notes for slide

Masterclass Webinar: Amazon S3

  1. 1. MasterclassAmazon S3 – beyond simple storage Ryan Shuttleworth – Technical Evangelist @ryanAWS
  2. 2. Masterclass A technical deep dive beyond the basicsHelp educate you on how to get the best from AWS technologies Show you how things work and how to get things done Broaden your knowledge in ~45 mins
  3. 3. Amazon S3 It’s more than just a ‘simple’ storage platform A sophisticated 21st century distributed systemA bedrock architectural component for many applications
  4. 4. “Spotify needed a storage solution that could scale very quickly without incurringlong lead times for upgrades. This led us to cloud storage, and in that market, Amazon Simple Storage Service (Amazon S3) is the most mature large-scale product. Amazon S3 gives us confidence in ourability to expand storage quickly while also providing high data durability.” Emil Fredriksson, Operations Director
  5. 5. You put it in S3AWS stores with 99.999999999% durability
  6. 6. Highly scalable web access to objects You put it in S3AWS stores with 99.999999999% durabilityMultiple redundantcopies in a region
  7. 7. but it’s more than just a simple storage service
  8. 8. Objects in S3 1500.000 1.3 Trillion 1250.000 1000.000 750.000 500.000 250.000 0.000 835k+ peak transactions per second
  9. 9. Highly scalable data storageA web store, not a file system Access via APIs What is S3? Fast Economical Highly available & durable
  10. 10. A web store, not a file system Write once, read many (WORM) Eventually consistent
  11. 11. A web store, not a file system Load balancers Load balancers Write once, read many Web servers Web servers (WORM) Eventually consistent Indexing Storage Indexing Storage Availability Zone Availability Zone RegionConceptual diagram only
  12. 12. A web store, not a file system Load balancers Load balancers Write once, read many Web servers Web servers (WORM) Eventually consistent Indexing Storage Indexing Storage Availability Zone Availability Zone RegionConceptual diagram only
  13. 13. A web store, not a file system Load balancers Load balancers Write once, read many Web servers Web servers (WORM) Eventually consistent Indexing Storage Indexing Storage Availability Zone Availability Zone RegionConceptual diagram only
  14. 14. A web store, not a file system Load balancers Load balancers Write once, read many Web servers Web servers (WORM) Eventually consistent Indexing Storage Indexing Storage Availability Zone Availability Zone RegionConceptual diagram only
  15. 15. A web store, not a file system Load balancers Load balancers Write once, read many Web servers Web servers (WORM) Eventually consistent Indexing Storage Indexing Storage Availability Zone Availability Zone RegionConceptual diagram only
  16. 16. A web store, not a file system Load balancers Load balancers Write once, read many Web servers Web servers (WORM) Eventually consistent Indexing Storage Indexing Storage Availability Zone Availability Zone RegionConceptual diagram only
  17. 17. A web store, not a file system Load balancers Load balancers Write once, read many Web servers Web servers (WORM) Eventually consistent Indexing Storage Indexing Storage Availability Zone Availability Zone RegionConceptual diagram only
  18. 18. A web store, not a file system Load balancers Load balancers Write once, read many Web servers Web servers (WORM) Eventually consistent Indexing Storage Indexing Storage Availability Zone Availability Zone RegionConceptual diagram only
  19. 19. A web store, not a file system Write once, read many (WORM) Eventually consistentConceptual diagram only
  20. 20. New objects Synchronously stores your data across multiple facilities before returning SUCCESS A web store, not a file system Read-after-write consistency* Updates Write once, Write then read: could report key does not exist read many Write then list: might not include key in list (WORM) Overwrite then read: old data could be returned Eventually consistent Deletes Delete then read: could still get old data Delete then list: deleted key could be included in list*except US-STANDARD region
  21. 21. A regional serviceYour data never leaves a region unless you move it
  22. 22. Storage classesControlling the way S3 holds your data
  23. 23. Amazon S3 storage classes Standard Designed to provide99.999999999% durability and 99.99% availability of objects over a given year Designed to sustain theconcurrent loss of data in two facilities
  24. 24. Amazon S3 storage classes Reduced Standard Redundancy Storage Designed to provide Reduces costs by storing data at99.999999999% durability and lower levels of redundancy than 99.99% availability of objects the Standard storage over a given year Designed to provide 99.99% Designed to sustain the durability and 99.99% availabilityconcurrent loss of data in two of objects over a given year facilities
  25. 25. Amazon S3 storage classes Reduced Standard Redundancy Storage Glacier Designed to provide Reduces costs by storing data at Suitable for archiving data, where99.999999999% durability and lower levels of redundancy than data access is infrequent and a 99.99% availability of objects the Standard storage retrieval time of several hours is over a given year acceptable Designed to provide 99.99% Designed to sustain the durability and 99.99% availability Uses the very low-cost Amazonconcurrent loss of data in two of objects over a given year Glacier storage service, but managed facilities through Amazon S3
  26. 26. Amazon S3 storage classes Reduced Standard Redundancy Storage Glacier Designed to provide Reduces costs by storing data Suitable for archiving data,99.999999999%want to Objects you durability at lower levels can afford to Objects you of redundancy where data access is infrequent Objects you want to keepand 99.99% availability of than the Standard storage and a retrieval time of several have high durability lose or can recreate in archive for a long time objects over a given year Designed to provide 99.99% hours is acceptable Designed to sustain the e.g. master copy of movie e.g. durability and 99.99% different encodings of movie e.g. digital very low-cost Amazon Uses the archive of old movies &concurrent loss of data in media availability of objects over a media Glacier storage service, but broadcasts two facilities given year managed through Amazon S3
  27. 27. NamespacesObject naming, buckets and keys
  28. 28. Amazon S3 namespaceGlobally uniquebucket name + object name (key)
  29. 29. Amazon S3 namespace Amazon S3 bucket bucketobject object object object bucket object object
  30. 30. Amazon S3 namespace Amazon S3 jones-docshare media.mydomain.combeach.jpg drafts/rpt.doc img1.jpg style.css yourdomain.com swf/mediaplayer.swf img/banner1.jpg
  31. 31. Amazon S3 namespace Object key Unique with a bucket
  32. 32. Amazon S3 namespace Object keyMax 1024 bytes UTF-8 Including ‘path’ prefixes Unique with a bucket
  33. 33. Amazon S3 namespace Object keyMax 1024 bytes UTF-8 Including ‘path’ prefixes Unique with a bucket assets/js/jquery/plugins/jtables.js this is an object key
  34. 34. Throughput optimisation S3 automatically partitions based upon key prefix:Bucket: Object keys:mynewgame 2134857/gamedata/start.png
2134857/g amedata/resource.rsrc
2134857/gameda ta/results.txt
2134858/gamedata/star t.png
2134858/gamedata/resource.rsrc 
2134858/gamedata/results.txt
213485 9/gamedata/start.png
2134859/gamedat a/resource.rsrc
2134859/gamedata/res ults.txt
  35. 35. Throughput optimisation S3 automatically partitions based upon key prefix:Bucket: Object keys:mynewgame 2134857/gamedata/start.png
2134857/g amedata/resource.rsrc
2134857/gameda ta/results.txt
2134858/gamedata/star t.png
2134858/gamedata/resource.rsrc 
2134858/gamedata/results.txt
213485 9/gamedata/start.png
2134859/gamedat Incrementing a/resource.rsrc
2134859/gamedata/res game id ults.txt
  36. 36. Throughput optimisation S3 automatically partitions based upon key prefix:Bucket: Object keys:mynewgame 2134857/gamedata/start.png
2134857/g amedata/resource.rsrc
2134857/gameda ta/results.txt
2134858/gamedata/star t.png
2134858/gamedata/resource.rsrc 
2134858/gamedata/results.txt
213485 9/gamedata/start.png
2134859/gamedat Partition: a/resource.rsrc
2134859/gamedata/res mynewgame/2 ults.txt
  37. 37. Throughput optimisation S3 automatically partitions based upon key prefix:Bucket: Object keys:mynewgame 7584312/gamedata/start.png
7584312/g amedata/resource.rsrc
7584312/gameda ta/results.txt
8584312/gamedata/star t.png
8584312/gamedata/resource.rsrc 
8584312/gamedata/results.txt
958431 2/gamedata/start.png
9584312/gamedat Reversed game a/resource.rsrc
9584312/gamedata/res ID ults.txt
  38. 38. Throughput optimisation S3 automatically partitions based upon key prefix:Bucket: Object keys:mynewgame 7584312/gamedata/start.png
7584312/g amedata/resource.rsrc
7584312/gameda ta/results.txt
8584312/gamedata/star t.png
8584312/gamedata/resource.rsrc 
8584312/gamedata/results.txt
958431 Partitions: 2/gamedata/start.png
9584312/gamedat mynewgame/7 a/resource.rsrc
9584312/gamedata/res mynewgame/8 ults.txt mynewgame/9
  39. 39. EncryptionSecuring data at rest
  40. 40. Automatic encryption of data at rest SimpleAdditional PUT Durable header S3 key storage Server side encryption Strong Secure AES-256 3-way simultaneous access Self managed No need to manage a key store
  41. 41. Server side encryption Data bucketHigh level design
  42. 42. Server side encryption Encrypted object Data bucket Encrypted data Per-object keyHigh level design
  43. 43. Server side encryption Encrypted object Data bucket Encrypted data Encrypted per-object key Per-object key Master keyHigh level design
  44. 44. Server side encryption Encrypted object Data bucket Encrypted data Encrypted per-object key Per-object key Key management (monthly rotation) Master keyHigh level design
  45. 45. Access controlsYou’re in control of who does what
  46. 46. You decide what to share Apply policies to buckets and objects Secure by default Policies, ACLs & IAMUse S3 policies, ACLs or IAM to define rules
  47. 47. IAM Fine grained Administer as part of role based access Apply policies to S3 atrole, user & group levelAllowActionsPutObjectResourcearn:aws:s3:::mybucket/* Bob Jane
  48. 48. IAM VS Bucket Policies Fine grained Fine grained Administer as part of Apply policies at the bucket role based access level in S3 Apply policies to S3 at Incorporate user restrictionsrole, user & group level without using IAMAllow Allow Bob, JaneActions ActionsPutObject PutObjectResource Resourcearn:aws:s3:::mybucket/* arn:aws:s3:::mybucket/* Bob Jane mybucket
  49. 49. IAM VS Bucket Policies VS ACLs Fine grained Fine grained Coarse grained Administer as part of Apply policies at the bucket Apply access control role based access level in S3 rules at the bucket Apply policies to S3 at Incorporate user restrictions and/or object level in S3role, user & group level without using IAMAllow Allow Allow Bob, Jane Everyone, Bob, JaneActions Actions ActionsPutObject PutObject ReadResource Resourcearn:aws:s3:::mybucket/* arn:aws:s3:::mybucket/* Bob Jane mybucket mybucket myobject
  50. 50. Access controls – bucket policy{"Statement":[{ "Effect":"Allow", "Principal":{"AWS":["4649-6425", "5243-0045"]}, "Action":"*", "Resource":"/mybucket/*", "Condition":{ "IpAddress":{"AWS:SourceIp":"176.13.0.0/12"}}}]}
  51. 51. Access controls – bucket policy{"Statement":[{ Accounts to allow "Effect":"Allow", "Principal":{"AWS":["4649-6425", "5243-0045"]}, "Action":"*", "Resource":"/mybucket/*", "Condition":{ "IpAddress":{"AWS:SourceIp":"176.13.0.0/12"}}}]}
  52. 52. Access controls – bucket policy{"Statement":[{ "Effect":"Allow", "Principal":{"AWS":["4649-6425", "5243-0045"]}, "Action":"*", "Resource":"/mybucket/*", Resource "Condition":{ "IpAddress":{"AWS:SourceIp":"176.13.0.0/12"}}}]}
  53. 53. Access controls – bucket policy{"Statement":[{ "Effect":"Allow", "Principal":{"AWS":["4649-6425", "5243-0045"]}, "Action":"*", "Resource":"/mybucket/*", "Condition":{ "IpAddress":{"AWS:SourceIp":"176.13.0.0/12"}}}]} Source address to allow
  54. 54. Transitions & Lifecycle Management Automated management of objects
  55. 55. Lifecycle management
  56. 56. Object deletionPermanently delete objects from S3 Lifecycle management
  57. 57. Object deletionPermanently delete objects from S3 Lifecycle management Object archiving Move objects to Glacier and out of S3 storage
  58. 58. GlacierLong term durable archive
  59. 59. Durable Designed for 99.999999999% durability of archivesLong term Glacier archive Cost effectiveWrite-once, read-never. Cost effective forlong term storage. Pay for accessing data
  60. 60. Expiry Logs accessible from S3 time
  61. 61. ✗Expiry Logs accessible from S3 Objects expire and are deleted time
  62. 62. Object transition toTransition Glacier invoked accessible from S3 Txns ✗Expiry Logs accessible from S3 Objects expire and are deleted time
  63. 63. Object Restoration of transition to object requestedTransition Glacier invoked for x hrs accessible from S3 Txns ✗Expiry Logs accessible from S3 Objects expire and are deleted time
  64. 64. Object Restoration of transition to object requestedTransition Glacier invoked for x hrs Object held in S3 accessible from S3 RRS for x hrs Txns 3-5hrs ✗Expiry Logs accessible from S3 Objects expire and are deleted time
  65. 65. 3-5 hour retrieval time We assume you won’t access often
  66. 66. using (client = new AmazonS3Client()){ var lifeCycleConfiguration = new LifecycleConfiguration() { Rules = new List<LifecycleRule> { new LifecycleRule { Id = "Archive and delete rule", Prefix = "projectdocs/", Status = LifecycleRuleStatus.Enabled, Transition = new LifecycleTransition() { Days = 365, StorageClass = S3StorageClass.Glacier }, Expiration = new LifecycleRuleExpiration() { Days = 3650 } } } };
  67. 67. using (client = new AmazonS3Client()){ var lifeCycleConfiguration = new LifecycleConfiguration() { Rules = new List<LifecycleRule> { new LifecycleRule { Transition to Id = "Archive and delete rule", Prefix = "projectdocs/", Glacier after 1 Status = LifecycleRuleStatus.Enabled, year Transition = new LifecycleTransition() { Days = 365, StorageClass = S3StorageClass.Glacier }, Expiration = new LifecycleRuleExpiration() { Days = 3650 } } } };
  68. 68. using (client = new AmazonS3Client()){ var lifeCycleConfiguration = new LifecycleConfiguration() { Rules = new List<LifecycleRule> { new LifecycleRule { Id = "Archive and delete rule", Prefix = "projectdocs/", Status = LifecycleRuleStatus.Enabled, Transition = new LifecycleTransition() Delete object { Days = 365, after 10 years StorageClass = S3StorageClass.Glacier }, Expiration = new LifecycleRuleExpiration() { Days = 3650 } } } };
  69. 69. POST /ObjectName?restore HTTP/1.1Host: BucketName.s3.amazonaws.comDate: dateAuthorization: signatureValueContent-MD5: MD5<RestoreRequest xmlns="http://s3.amazonaws.com/doc/2006-3-01"> <Days>NumberOfDays</Days></RestoreRequest>
  70. 70. POST /ObjectName?restore HTTP/1.1Host: BucketName.s3.amazonaws.comDate: dateAuthorization: signatureValueContent-MD5: MD5<RestoreRequest xmlns="http://s3.amazonaws.com/doc/2006-3-01"> <Days>NumberOfDays</Days></RestoreRequest>Response codes:202 Accepted Restore request accepted200 OK Object already restored, number of days updated409 Conflict Restoration already in progress
  71. 71. Website hostingStatic sites straight from S3
  72. 72. It’s a web service……so serve up web content
  73. 73. Setting default documents Redirecting requests
  74. 74. { "Version":"2008-10-17", "Statement":[{ "Sid":"PublicReadGetObject", "Effect":"Allow", "Principal": { "AWS": "*" }, "Action":["s3:GetObject"], "Resource":["arn:aws:s3:::example-bucket/*" ] } ]}
  75. 75. Website addressing{bucket-name}.s3-website-{region}.amazonaws.come.g. mybucket.s3-website-eu-west-1.amazonaws.comNormal addressings3-{region}.amazonaws.com/{bucket-name}/{object-key}e.g. s3-eu-west-1.amazonaws.com/mybucket/img.png{bucket-name}.s3-{region}.amazonaws.com/{object-key}e.g. mybucket.s3-eu-west-1.amazonaws.com/img.png
  76. 76. Record set for:aws-examples.info R53
  77. 77. Record set for: aws-examples.info R53 Index. Error. bucket bucket html html Website bucket name: Website bucket name:www.aws-examples.info aws-examples.info
  78. 78. Record set for: aws-examples.info R53 Index. Error. bucket bucket html html Website redirect to: aws-examples.info Website bucket name: Website bucket name:www.aws-examples.info aws-examples.info
  79. 79. Record set for: aws-examples.info A Record ‘Alias’ to S3 website: R53 aws-examples.info @ s3-website-eu- west-1.amazonaws.com Index. Error. bucket bucket html html Website redirect to: aws-examples.info Website bucket name: Website bucket name:www.aws-examples.info aws-examples.info
  80. 80. Record set for: aws-examples.info CNAME for www. to: A Record ‘Alias’ to S3 website: www.aws-examples.info.s3- R53 aws-examples.info @ s3-website-eu-website-eu-west-1.amazonaws.com west-1.amazonaws.com Index. Error. bucket bucket html html Website redirect to: aws-examples.info Website bucket name: Website bucket name:www.aws-examples.info aws-examples.info
  81. 81. Object versioning Preserving object histories
  82. 82. Bucket level PersistentAutomatically preserves Even deleted object all copies of objects history is held
  83. 83. >>> import boto>>> conn = boto.connect_s3()>>> from boto.s3.bucket import Bucket>>> from boto.s3.key import Key>>> bucket = conn.get_bucket(’mybucket)>>> versions = bucket.list_versions()>>> for version in versions:... print version.name + version.version_id...
  84. 84. >>> import boto>>> conn = boto.connect_s3()>>> from boto.s3.bucket import Bucket>>> from boto.s3.key import Key>>> bucket = conn.get_bucket(’mybucket)>>> versions = bucket.list_versions()>>> for version in versions:... print version.name + version.version_id Object... version IDsmyfile.txt jU9eVv800OlP4PQx6zskMEyPIoExne57myfile.txt xOJzMvMmGv0Bx2v4QpIypbkkH2XE2yyqmyfile.txt 8cjozv9Hmkzum8xj.8q8BZxR5CuXnzon
  85. 85. >>> key = bucket.get_key(myfile.txt,version_id=8cjozv9Hmkzum8xj.8q8BZxR5CuXnzon’)>>> key.get_contents_as_string()this is version 1 of my file’ Grabbing the contents of a version
  86. 86. >>> key = bucket.get_key(myfile.txt,version_id=8cjozv9Hmkzum8xj.8q8BZxR5CuXnzon’)>>> key.get_contents_as_string()this is version 1 of my file’>>> key = bucket.get_key(myfile.txt,version_id=xOJzMvMmGv0Bx2v4QpIypbkkH2XE2yyq’)>>> key.get_contents_as_string()this is version 2 of my file’
  87. 87. >>> key = bucket.get_key(myfile.txt,version_id=8cjozv9Hmkzum8xj.8q8BZxR5CuXnzon’)>>> key.get_contents_as_string()this is version 1 of my file’>>> key = bucket.get_key(myfile.txt,version_id=xOJzMvMmGv0Bx2v4QpIypbkkH2XE2yyq’)>>> key.get_contents_as_string() Generating a ’10this is version 2 of my file’ minute time bombed’ url for an older>>> key.generate_url(600) versionhttps://mybucket.s3.amazonaws.com/myfile.txt?Signature=ABCD&Expires=1358857379&AWSAccessKeyId=AB&versionId=xOJzMvMmGv0Bx2v4QpIypbkkH2XE2yyq
  88. 88. Error response: link expired
  89. 89. MetadataSystem and user generated
  90. 90. System metadata Name Description Editable? Date Object creation date No Content-Length Object size in bytes No Content-MD5 Base64 encoded 128bit MD5 digest No x-amz-server-side-encryption Server side encryption enabled for object Yes x-amz-version-id Object version No x-amz-delete-marker Indicates a version enabled object is deleted No x-amz-storage-class Storage class for the object Yesx-amz-website-redirect-location Redirects request for the object to another object or external URL Yes
  91. 91. User metadataKey-value pairs stored with objects and returned with requestsx-amz-meta-{your metadata key in here}
  92. 92. >>> key.set_metadata(’my_tag, ’my metadata)>>> key.get_metadata(’my_tag)’my metadata
  93. 93. Cloud FrontDelivery from global edge locations
  94. 94. Download Enable static and dynamic assets to be served from edge locationsGlobal content distribution Streaming Serve RTMP directly from media files in buckets
  95. 95. S3 BucketCache control Forwarding Optional CNAMEEdge access controls
  96. 96. S3 Bucket Optional CNAMELogging to S3
  97. 97. Content
  98. 98. Download distribution<html><script type=text/javascriptsrc=’http://d2ew7gdzogp20x.cloudfront.net/jwplayer/jwplayer.js></script><body><div id=player></div><script type=text/javascript> jwplayer(player).setup({ file: "rtmp://s1eat02wfxn38u.cloudfront.net/cfx/st/montage-medium.mp4", width: "480", height: "270", });</script> Streaming</body> distribution</html>
  99. 99. Summary
  100. 100. Stop doing these: Capacity planning Management of storageWorrying about backing up the backup Fixing broken hardware
  101. 101. Bootstrapping Store scripts and drive EC2 Backups & archiveApplication backends instances on startup Storage gateway, 3rd Incorporate S3 SDKs into party tools your applications and start doing these Application logs Documentation Store logs and analyse Web content Store documents with with EMR versioning and security Serve content and models distribute globally
  102. 102. aws.amazon.com

×