S3 is a data storage service that provides scalable, reliable storage through a RESTful API. Files are stored as objects within buckets and can be accessed via HTTP, BitTorrent, or SOAP protocols. Access is controlled on a per file or per user basis. Users are billed based on the amount of data stored and transferred. The service provides effectively unlimited storage at low costs but has some limitations around global availability and eventual consistency.

1. AMAZON SIMPLE
STORAGE SERVICE
(S3)
The Infinite Hard Drive in the Cloud

2. What is S3?

3. What is S3?
A RESTful (or SOAP) data storage API

4. What is S3?
A RESTful (or SOAP) data storage API
Supports HTTP and BitTorrent protocols
Control headers to serve content straight from S3

5. What is S3?
A RESTful (or SOAP) data storage API
Supports HTTP and BitTorrent protocols
Control headers to serve content straight from S3
Full access control per file or user
Preauthorize direct uploads by users

6. What is S3?
A RESTful (or SOAP) data storage API
Supports HTTP and BitTorrent protocols
Control headers to serve content straight from S3
Full access control per file or user
Preauthorize direct uploads by users
Billed by capacity stored and transfer rates

7. Everything is Better Online!

8. Everything is Better Online!

9. Basic Usage

10. Basic Usage

11. Creating/Listing Buckets
Globally unique “buckets” hold files on S3

12. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
puts
print "Creating the bucket graysoftinc... "
s3.bucket("graysoftinc", :create, "private")
puts "Done."
puts
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
Creating/Listing Buckets
Globally unique “buckets” hold files on S3

13. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
puts
print "Creating the bucket graysoftinc... "
s3.bucket("graysoftinc", :create, "private")
puts "Done."
puts
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
Creating/Listing Buckets
Globally unique “buckets” hold files on S3

14. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
puts
print "Creating the bucket graysoftinc... "
s3.bucket("graysoftinc", :create, "private")
puts "Done."
puts
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
Creating/Listing Buckets
Globally unique “buckets” hold files on S3

15. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
puts
print "Creating the bucket graysoftinc... "
s3.bucket("graysoftinc", :create, "private")
puts "Done."
puts
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
Creating/Listing Buckets
Globally unique “buckets” hold files on S3

16. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
puts
print "Creating the bucket graysoftinc... "
s3.bucket("graysoftinc", :create, "private")
puts "Done."
puts
puts "Your buckets: " +
s3.buckets.map { |b| b.name }.join(', ')
Creating/Listing Buckets
Globally unique “buckets” hold files on S3

17. Your buckets:
Creating the bucket graysoftinc... Done.
Your buckets: graysoftinc
Creating/Listing Buckets
Globally unique “buckets” hold files on S3

18. Uploading/Listing Files
Files are “objects” stored under “keys” on S3

19. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
bucket = s3.bucket("graysoftinc")
puts "Files:", bucket.keys.map { |k| k.name }
puts
open(__FILE__) do |f|
bucket.put("presentations/upload.rb", f)
end
bucket.copy_key("presentations/upload.rb", "ruby/upload.rb")
puts "Files:", bucket.keys.map { |k| k.name }
puts "ruby/:",
bucket.keys(:prefix => "ruby/").map { |k| k.name }
Uploading/Listing Files
Files are “objects” stored under “keys” on S3

20. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
bucket = s3.bucket("graysoftinc")
puts "Files:", bucket.keys.map { |k| k.name }
puts
open(__FILE__) do |f|
bucket.put("presentations/upload.rb", f)
end
bucket.copy_key("presentations/upload.rb", "ruby/upload.rb")
puts "Files:", bucket.keys.map { |k| k.name }
puts "ruby/:",
bucket.keys(:prefix => "ruby/").map { |k| k.name }
Uploading/Listing Files
Files are “objects” stored under “keys” on S3

21. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
bucket = s3.bucket("graysoftinc")
puts "Files:", bucket.keys.map { |k| k.name }
puts
open(__FILE__) do |f|
bucket.put("presentations/upload.rb", f)
end
bucket.copy_key("presentations/upload.rb", "ruby/upload.rb")
puts "Files:", bucket.keys.map { |k| k.name }
puts "ruby/:",
bucket.keys(:prefix => "ruby/").map { |k| k.name }
Uploading/Listing Files
Files are “objects” stored under “keys” on S3

22. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
bucket = s3.bucket("graysoftinc")
puts "Files:", bucket.keys.map { |k| k.name }
puts
open(__FILE__) do |f|
bucket.put("presentations/upload.rb", f)
end
bucket.copy_key("presentations/upload.rb", "ruby/upload.rb")
puts "Files:", bucket.keys.map { |k| k.name }
puts "ruby/:",
bucket.keys(:prefix => "ruby/").map { |k| k.name }
Uploading/Listing Files
Files are “objects” stored under “keys” on S3

23. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
bucket = s3.bucket("graysoftinc")
puts "Files:", bucket.keys.map { |k| k.name }
puts
open(__FILE__) do |f|
bucket.put("presentations/upload.rb", f)
end
bucket.copy_key("presentations/upload.rb", "ruby/upload.rb")
puts "Files:", bucket.keys.map { |k| k.name }
puts "ruby/:",
bucket.keys(:prefix => "ruby/").map { |k| k.name }
Uploading/Listing Files
Files are “objects” stored under “keys” on S3

24. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3.new( "ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null") )
bucket = s3.bucket("graysoftinc")
puts "Files:", bucket.keys.map { |k| k.name }
puts
open(__FILE__) do |f|
bucket.put("presentations/upload.rb", f)
end
bucket.copy_key("presentations/upload.rb", "ruby/upload.rb")
puts "Files:", bucket.keys.map { |k| k.name }
puts "ruby/:",
bucket.keys(:prefix => "ruby/").map { |k| k.name }
Uploading/Listing Files
Files are “objects” stored under “keys” on S3

25. Files:
Files:
presentations/upload.rb
ruby/upload.rb
ruby/:
ruby/upload.rb
Uploading/Listing Files
Files are “objects” stored under “keys” on S3

26. Files:
Files:
presentations/upload.rb
ruby/upload.rb
ruby/:
ruby/upload.rb
Uploading/Listing Files
Files are “objects” stored under “keys” on S3

27. Downloading a File
You can stream files to and from S3

28. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3Interface.new(
"ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null")
)
open("downloaded.rb", "w") do |f|
s3.get("graysoftinc", "ruby/upload.rb") do |chunk|
f << chunk
end
end
Downloading a File
You can stream files to and from S3

29. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3Interface.new(
"ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null")
)
open("downloaded.rb", "w") do |f|
s3.get("graysoftinc", "ruby/upload.rb") do |chunk|
f << chunk
end
end
Downloading a File
You can stream files to and from S3

30. #!/usr/bin/env ruby -KU
$VERBOSE = nil
require "rubygems"
require "aws" # sudo gem install aws
s3 = Aws::S3Interface.new(
"ACCESS_KEY_ID",
"SECRET_ACCESS_KEY",
:logger => Logger.new("/dev/null")
)
open("downloaded.rb", "w") do |f|
s3.get("graysoftinc", "ruby/upload.rb") do |chunk|
f << chunk
end
end
Downloading a File
You can stream files to and from S3

31. The Details

32. The Details

33. The Good

34. The Good
Scalable: effectively
“unlimited” storage

35. The Good
Scalable: effectively
“unlimited” storage
Reliable: 99.9%
guaranteed uptime and
very redundant

36. The Good
Scalable: effectively
“unlimited” storage
Reliable: 99.9%
guaranteed uptime and
very redundant
Inexpensive: rates for
GB in cents

37. The Good
Scalable: effectively
“unlimited” storage
Reliable: 99.9%
guaranteed uptime and
very redundant
Inexpensive: rates for
GB in cents
Universal: everything
supports it

38. Transmit has FTP-like S3
Support in libraries, command-line tools, and programs

39. Transmit has FTP-like S3
Support in libraries, command-line tools, and programs

40. Transmit has FTP-like S3
Support in libraries, command-line tools, and programs

41. Transmit has FTP-like S3
Support in libraries, command-line tools, and programs

42. Transmit has FTP-like S3
Support in libraries, command-line tools, and programs

43. The Not-So-Good

44. The Not-So-Good
Not quite worldly:
servers in the U.S.,
California, and Ireland

45. The Not-So-Good
Not quite worldly:
servers in the U.S.,
California, and Ireland
Simple, but not quite
curl/wget simple

46. The Not-So-Good
Not quite worldly:
servers in the U.S.,
California, and Ireland
Simple, but not quite
curl/wget simple
The service is
“eventually consistent”

47. Eventual Consistency
All machines will “eventually” see the same data in S3

48. Now
Eventual Consistency
All machines will “eventually” see the same data in S3

49. Eventually
Later
Now
Eventual Consistency
All machines will “eventually” see the same data in S3

50. Eventually
Later
Now
Eventual Consistency
All machines will “eventually” see the same data in S3