Amazon's Simple Storage Service (S3)

2,635 views
2,547 views

Published on

These are my S3 slides from the first Mash It Up Camp at the okcCoCo.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,635
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
104
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide


































































  • Amazon's Simple Storage Service (S3)

    1. 1. AMAZON SIMPLE STORAGE SERVICE (S3) The Infinite Hard Drive in the Cloud
    2. 2. What is S3?
    3. 3. What is S3? A RESTful (or SOAP) data storage API
    4. 4. What is S3? A RESTful (or SOAP) data storage API Supports HTTP and BitTorrent protocols Control headers to serve content straight from S3
    5. 5. What is S3? A RESTful (or SOAP) data storage API Supports HTTP and BitTorrent protocols Control headers to serve content straight from S3 Full access control per file or user Preauthorize direct uploads by users
    6. 6. What is S3? A RESTful (or SOAP) data storage API Supports HTTP and BitTorrent protocols Control headers to serve content straight from S3 Full access control per file or user Preauthorize direct uploads by users Billed by capacity stored and transfer rates
    7. 7. Everything is Better Online!
    8. 8. Everything is Better Online!
    9. 9. Basic Usage
    10. 10. Basic Usage
    11. 11. Creating/Listing Buckets Globally unique “buckets” hold files on S3
    12. 12. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
    13. 13. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
    14. 14. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
    15. 15. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
    16. 16. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
    17. 17. Your buckets: Creating the bucket graysoftinc... Done. Your buckets: graysoftinc Creating/Listing Buckets Globally unique “buckets” hold files on S3
    18. 18. Uploading/Listing Files Files are “objects” stored under “keys” on S3
    19. 19. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
    20. 20. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
    21. 21. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
    22. 22. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
    23. 23. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
    24. 24. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
    25. 25. Files: Files: presentations/upload.rb ruby/upload.rb ruby/: ruby/upload.rb Uploading/Listing Files Files are “objects” stored under “keys” on S3
    26. 26. Files: Files: presentations/upload.rb ruby/upload.rb ruby/: ruby/upload.rb Uploading/Listing Files Files are “objects” stored under “keys” on S3
    27. 27. Downloading a File You can stream files to and from S3
    28. 28. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3Interface.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) open("downloaded.rb", "w") do |f| s3.get("graysoftinc", "ruby/upload.rb") do |chunk| f << chunk end end Downloading a File You can stream files to and from S3
    29. 29. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3Interface.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) open("downloaded.rb", "w") do |f| s3.get("graysoftinc", "ruby/upload.rb") do |chunk| f << chunk end end Downloading a File You can stream files to and from S3
    30. 30. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3Interface.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) open("downloaded.rb", "w") do |f| s3.get("graysoftinc", "ruby/upload.rb") do |chunk| f << chunk end end Downloading a File You can stream files to and from S3
    31. 31. The Details
    32. 32. The Details
    33. 33. The Good
    34. 34. The Good Scalable: effectively “unlimited” storage
    35. 35. The Good Scalable: effectively “unlimited” storage Reliable: 99.9% guaranteed uptime and very redundant
    36. 36. The Good Scalable: effectively “unlimited” storage Reliable: 99.9% guaranteed uptime and very redundant Inexpensive: rates for GB in cents
    37. 37. The Good Scalable: effectively “unlimited” storage Reliable: 99.9% guaranteed uptime and very redundant Inexpensive: rates for GB in cents Universal: everything supports it
    38. 38. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
    39. 39. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
    40. 40. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
    41. 41. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
    42. 42. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
    43. 43. The Not-So-Good
    44. 44. The Not-So-Good Not quite worldly: servers in the U.S., California, and Ireland
    45. 45. The Not-So-Good Not quite worldly: servers in the U.S., California, and Ireland Simple, but not quite curl/wget simple
    46. 46. The Not-So-Good Not quite worldly: servers in the U.S., California, and Ireland Simple, but not quite curl/wget simple The service is “eventually consistent”
    47. 47. Eventual Consistency All machines will “eventually” see the same data in S3
    48. 48. Now Eventual Consistency All machines will “eventually” see the same data in S3
    49. 49. Eventually Later Now Eventual Consistency All machines will “eventually” see the same data in S3
    50. 50. Eventually Later Now Eventual Consistency All machines will “eventually” see the same data in S3

    ×