Aws introduction tocloudstorage

868 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
868
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
23
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • * 2 HDs** 5 HDsNote that you have to pay for sending the device!FedEx:~30€ from London to Dublin, 3kg, one day.
  • At 10Mbps upload speedConsidering that it takes one day to ship a device from London to Dublin
  • Aws introduction tocloudstorage

    1. 1. Amazon Web ServicesIntroduction to Cloud Storage Kevin Wittkopf Enterprise Solutions Architect…
    2. 2. Cloud Storage AWS offers multiple cloud-based storage options. • Amazon S3 (Simple Storage Service) • Amazon EBS (Elastic Block Storage) • Amazon EC2 Local Instance Store (Ephemeral volumes) • Amazon Cloudfront Each has a unique combination of performance, durability, cost, and interface Often used to form a Storage Hierarchy
    3. 3. Storage by the Numbers Cloudfront S3 EBS Local DiskScale Unlimited Unlimited Storage 1TB per volume 147 – 1690GB Temporary 5TB objects ~44TB per 1-5 volumes Storage 100 name spaces server (buckets)Durability N/A 99.999999999% .1-.5% AFR Ephemeral (standard) design requires 99.99% application-level (RRS) redundancyLatency Low High Low Very LowManagement Zero Zero Maintenance Snapshots, Manual Maintenance RAID, Redundancy ConsistencyLocality Global Region-Centric AZ-Specific In-machineInterface REST or SOAP REST or SOAP Block Device Block Device Web Services Web Services
    4. 4. S3Amazon Simple Storage Service ©2011 Amazon Web Services May not be reused or redistributed without permission
    5. 5. AmazonEC2Amazon Amazon Amazon AmazonEBS S3 RDS SimpleDB ©2011 Amazon Web Services May not be reused or redistributed without permission
    6. 6. Amazon Simple Storage ServiceAmazon S3: Simple Storage ServiceHighly scalable data storage in-the-cloudProgrammatic access via web services APIIs a Web Store, not a file system Optimized for WORM Eventually consistentFast, highly available and durableEconomical ©2011 Amazon Web
    7. 7. Is Your Data Storage This Resilient? Note: Conceptual drawing only. Actual number of nodes & datacenters may vary Region Datacenter Datacenter Datacenter Node Node 1 n ... ©2011 Amazon Web
    8. 8. Amazon S3 Adoption Rate: Billions of Objects Stored Peak Requests: 290,000+600 per second 6 months!500 490400300 262200 102100 40 14 2.9 0 Q4 2006 Q4 2007 Q4 2008 Q4 2009 Q4 2010 Q2 2011 ©2011 Amazon Web Services May not be
    9. 9. Amazon S3 Namespace Amazon S3 bucket bucketobject object object object bucket object object ©2011 Amazon Web Services May not be reused
    10. 10. Amazon S3 Namespace Amazon S3 mculver-images media.mydomain.comBeach.jpg 2009/maui/sun.jpg img1.jpg img2.jpg public.blueorigin.com Installer.msi img/pic1.jpg ©2011 Amazon Web Services May not be
    11. 11. Reduced Redundancy Storage Saves $$ Designed to provide 99.99% durability and 99.99% availability of objects over a given year This durability level corresponds to an average annual expected loss of 0.01% of objects Designed to sustain the loss of data in a single facility ©2011 Amazon Web
    12. 12. S3 Pricing has Three Dimensions Storage Data Transfer Requests Tier Norma RRS Tier Price Type Price lFrist 1 $0.14/G $0.093/GB All transfer in Free PUT, $0.01TB/mo B COPY, per POST, or 1,000 LISTNext 49 TB $0.125/G $0.083/GB First 1 GB out Free GET and $0.01 B all other per requests 10,000Next 450 $0.11/GB $0.073/GB <= 10 TB out $0.12/GTB BNext 500 $0.095/G $0.063/GB Next 40 TB $0.09/GTB B out BNext 4000 $0.08/G $0.053/GB Next 100 TB $0.07/GTB B out ©2011 Amazon Web B
    13. 13. Access Control Lists• Control access to bucket or to object• No inheritance from bucket to object• Set at create, write, or any time• List of up to 100 grants by: – Owner – Email – Any AWS account – Anyone• Permissions: – READ – Object or bucket – WRITE – Create, overwrite, delete objects in bucket – READ_ACP – Read ACL for bucket or object – WRITE_ACP – Overwrite ACL for bucket or object – FULL_CONTROL – All of the above ©2011 Amazon Web Services May not be reused or redistributed without permission
    14. 14. Bucket PolicyAllow Jason or Mark to do anything in my S3 bucket when coming from the Office internal network{"Statement":[{ "Effect":"Allow", "Principal":{"AWS":["4649-6425", "5243-0045"]}, "Action":"*", "Resource":"/mybucket/*", "Condition":{ "IpAddress":{"AWS:SourceIp":"176.13.0.0/12"}}}]} ©2011 Amazon Web
    15. 15. S3 Multipart UploadsImproved throughput—upload parts in parallelQuick recovery from any network issues—Smaller partsize minimizes the impact of restarting a failed upload due toa network error.Pause and resume object uploads— upload object parts over time Once you initiate a multipart upload there is no expiry must explicitly complete or abort the multipart uploadBegin an upload before you know the final object size—can upload as you are create. ©2011 Amazon Web
    16. 16. Even More Amazon S3 (not so simple, is it?) Website support Versioning Browser Upload to S3 Logging Requestor Pays Signed URLs Bittorrent support IAM Meta-data ©2011 Amazon Web
    17. 17. AWS Import/Export Get your data into AWS faster - load it onto a portable storage device and ship it to an Amazon data center Faster than Internet transfer and more cost effective than upgrading your connectivity Use cases: data migration, offsite backup, direct data interchange, disaster recovery ©2011 Amazon Web
    18. 18. AWS Import/Export vs. S3 upload costs$1,200$1,000 $800 $600 Import/Export S3 upload $400 $200 $0 100Gb 500Gb 1 Tb 2 Tb 3 Tb 5 Tb * 10 Tb **
    19. 19. AWS Import/Export vs. S3 upload time savings 60 50 40Days 30 Import/Export 20 S3 upload 10 0 100 Gb 500 Gb 1 Tb 2 Tb 3 Tb 5 Tb
    20. 20. AmazonEC2Amazon Amazon Amazon AmazonEBS S3 RDS SimpleDB ©2011 Amazon Web Services May not be reused or redistributed without permission
    21. 21. EBS Features Persistent storage Volume lifetime is independent of any particular EC2 instance. General purpose Raw, unformatted, block device. Use from Linux, Solaris or Windows. Performance Equal to or better than local EC2 drive. High reliability Built-in redundancy within availability zone. AFR (Annual Failure Rate) between 0.1% and 1%. Scalable Volume sizes ranging from 1 GB to 1 TB. Easy Easy to create, attach, back up, restore, and delete volumes.©2011 Amazon Web Services May not be reused or redistributed without permission
    22. 22. Amazon Web Services: Durable & Available Note: Conceptual drawing only. The number of Availability Zones may vary Japan US East Region EU West Region Availabilit Availabilit Availabilit Availabilit y Zone A y Zone B y Zone A y Zone B Availabilit Availabilit y Zone A y Zone B Availabilit US West Region Availabilit Singapore y Zone C y Zone C Availabilit Availabilit Availabilit Availabilit y Zone A y Zone B y Zone A y Zone B ©2011 Amazon Web Services May not be reused or redistributed without written permission
    23. 23. EBS Volumes are in a Single Availability Zone EBS Volume 1 EBS Volume 2 Availability Zone A Availability Zone B Note: Conceptual diagram only©2011 Amazon Web Services May not be reused or redistributed without permission
    24. 24. How EBS Interacts With EC2 and S3 AWS Cloud EC2 Availability Zone S3 Storage Create Snapshot EBS Snapshot EBS EBS EBS EBS EBS EBS EBS Snapshot Clone From Snapshot EBS Snapshot EC2 EC2 EC2 EBS Snapshot EBS Snapshot Internet©2011 Amazon Web Services May not be reused or redistributed without permission
    25. 25. What’s the Difference Between Amazon S3 and Amazon EBS? Amazon S3 EBS Paradigm Object store File system Performance Very fast Very very fast Redundancy Across data centers Within data center Security Public Key / Private Visible only to your Key EC2 Pricing $0.14/GB/Mo. stored $0.10/GB/Mo. allocatedAccess from the Net? Yes(1) No(2) Typical use case Write once, read many It’s a disk drive (1) Only with proper credentials, unless ACLs are world readable (2) Accessible from Net if mounted to server and set up as FTP, etc.©2011 Amazon Web Services May not be reused or redistributed without permission
    26. 26. EBS Pricing EBS Volumes  $0.10 per GB-month of provisioned storage  $0.10 per 1 million I/O requests  No charge for mounting/unmounting volume EBS Snapshots to Amazon S3  $0.14 per GB-month of data stored  $0.01 per 1,000 PUT requests (when saving a snapshot)  $0.01 per 10,000 GET requests (when loading a snapshot)©2011 Amazon Web Services May not be reused or redistributed without permission
    27. 27. ©2011 Amazon Web Services May not be reused or redistributed without permission EBS Volume Lifecycle Call CreateVolume 1 GB to 1 TB Create Call AttachVolume to affiliate with one EC2 instanceVast amounts of unused space Attach • Format from EC2 instance Attached OS & • Mount formatted drive Snapshot to Amazon S3 In Use Deleted CreateSnapshot Call DeleteVolume Detach Call DetachVolume
    28. 28. How Do Snapshots Work? Time Snapshot 2 Snapshot 3 Snapshot 1 Chunk 1 Block 1 S3 Chunk 2 Block 2 EBS Volume Chunk 3 Block 3 Chunk 4 Block 4©2011 Amazon Web Services May not be reused or redistributed without permission
    29. 29. Things You Should Know… (Part I) EBS is a network resource  Networks have variable performance Striping will improve performance, although it’s usually fine to start with  4-8 drives per logical volume is observed sweet spot  Great approach for high-performance database workloads EBS is already redundant behind the scenes. It makes no sense to implement RAID on top of EBS©2011 Amazon Web Services May not be reused or redistributed without permission
    30. 30. Things You Should Know… (Part II) Lazy loads from snapshots  When you create a volume, space is allocated immediately  Data from snapshot written to disk on 1st request for maximum performance  For maximum IO performance, run dd to touch all blocks Like all of AWS EBS is constantly innovating. These tips may not be appropriate in a few months©2011 Amazon Web Services May not be reused or redistributed without permission
    31. 31. AWS Import/ExportGet your data into AWS faster - load it onto aportable storage device and ship it to an Amazondata centerFaster than Internet transfer and more costeffective than upgrading your connectivityUse cases: data migration, offsite backup, directdata interchange, disaster recovery
    32. 32. Amazon CloudFrontEasy-to-use content delivery networkSame pay-as-you-go pricing as all of AWS– No up-front contracts– No long-term commitments– Self-service sign-upUses Amazon S3 as the origin storeWorldwide network of edge locations– Seattle, Palo Alto, Los Angeles, St. Louis, Dallas, Ashburn, Newark, Miami, Amsterdam, Dub lin, Frankfurt, London, Hong Kong, Tokyo

    ×