Amazon's Simple Storage Service (S3)

  • 1,794 views
Uploaded on

These are my S3 slides from the first Mash It Up Camp at the okcCoCo.

These are my S3 slides from the first Mash It Up Camp at the okcCoCo.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,794
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
53
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide


































































Transcript

  • 1. AMAZON SIMPLE STORAGE SERVICE (S3) The Infinite Hard Drive in the Cloud
  • 2. What is S3?
  • 3. What is S3? A RESTful (or SOAP) data storage API
  • 4. What is S3? A RESTful (or SOAP) data storage API Supports HTTP and BitTorrent protocols Control headers to serve content straight from S3
  • 5. What is S3? A RESTful (or SOAP) data storage API Supports HTTP and BitTorrent protocols Control headers to serve content straight from S3 Full access control per file or user Preauthorize direct uploads by users
  • 6. What is S3? A RESTful (or SOAP) data storage API Supports HTTP and BitTorrent protocols Control headers to serve content straight from S3 Full access control per file or user Preauthorize direct uploads by users Billed by capacity stored and transfer rates
  • 7. Everything is Better Online!
  • 8. Everything is Better Online!
  • 9. Basic Usage
  • 10. Basic Usage
  • 11. Creating/Listing Buckets Globally unique “buckets” hold files on S3
  • 12. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
  • 13. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
  • 14. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
  • 15. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
  • 16. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') puts print "Creating the bucket graysoftinc... " s3.bucket("graysoftinc", :create, "private") puts "Done." puts puts "Your buckets: " + s3.buckets.map { |b| b.name }.join(', ') Creating/Listing Buckets Globally unique “buckets” hold files on S3
  • 17. Your buckets: Creating the bucket graysoftinc... Done. Your buckets: graysoftinc Creating/Listing Buckets Globally unique “buckets” hold files on S3
  • 18. Uploading/Listing Files Files are “objects” stored under “keys” on S3
  • 19. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
  • 20. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
  • 21. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
  • 22. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
  • 23. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
  • 24. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) bucket = s3.bucket("graysoftinc") puts "Files:", bucket.keys.map { |k| k.name } puts open(__FILE__) do |f| bucket.put("presentations/upload.rb", f) end bucket.copy_key("presentations/upload.rb", "ruby/upload.rb") puts "Files:", bucket.keys.map { |k| k.name } puts "ruby/:", bucket.keys(:prefix => "ruby/").map { |k| k.name } Uploading/Listing Files Files are “objects” stored under “keys” on S3
  • 25. Files: Files: presentations/upload.rb ruby/upload.rb ruby/: ruby/upload.rb Uploading/Listing Files Files are “objects” stored under “keys” on S3
  • 26. Files: Files: presentations/upload.rb ruby/upload.rb ruby/: ruby/upload.rb Uploading/Listing Files Files are “objects” stored under “keys” on S3
  • 27. Downloading a File You can stream files to and from S3
  • 28. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3Interface.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) open("downloaded.rb", "w") do |f| s3.get("graysoftinc", "ruby/upload.rb") do |chunk| f << chunk end end Downloading a File You can stream files to and from S3
  • 29. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3Interface.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) open("downloaded.rb", "w") do |f| s3.get("graysoftinc", "ruby/upload.rb") do |chunk| f << chunk end end Downloading a File You can stream files to and from S3
  • 30. #!/usr/bin/env ruby -KU $VERBOSE = nil require "rubygems" require "aws" # sudo gem install aws s3 = Aws::S3Interface.new( "ACCESS_KEY_ID", "SECRET_ACCESS_KEY", :logger => Logger.new("/dev/null") ) open("downloaded.rb", "w") do |f| s3.get("graysoftinc", "ruby/upload.rb") do |chunk| f << chunk end end Downloading a File You can stream files to and from S3
  • 31. The Details
  • 32. The Details
  • 33. The Good
  • 34. The Good Scalable: effectively “unlimited” storage
  • 35. The Good Scalable: effectively “unlimited” storage Reliable: 99.9% guaranteed uptime and very redundant
  • 36. The Good Scalable: effectively “unlimited” storage Reliable: 99.9% guaranteed uptime and very redundant Inexpensive: rates for GB in cents
  • 37. The Good Scalable: effectively “unlimited” storage Reliable: 99.9% guaranteed uptime and very redundant Inexpensive: rates for GB in cents Universal: everything supports it
  • 38. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
  • 39. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
  • 40. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
  • 41. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
  • 42. Transmit has FTP-like S3 Support in libraries, command-line tools, and programs
  • 43. The Not-So-Good
  • 44. The Not-So-Good Not quite worldly: servers in the U.S., California, and Ireland
  • 45. The Not-So-Good Not quite worldly: servers in the U.S., California, and Ireland Simple, but not quite curl/wget simple
  • 46. The Not-So-Good Not quite worldly: servers in the U.S., California, and Ireland Simple, but not quite curl/wget simple The service is “eventually consistent”
  • 47. Eventual Consistency All machines will “eventually” see the same data in S3
  • 48. Now Eventual Consistency All machines will “eventually” see the same data in S3
  • 49. Eventually Later Now Eventual Consistency All machines will “eventually” see the same data in S3
  • 50. Eventually Later Now Eventual Consistency All machines will “eventually” see the same data in S3