Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Intro to AWS: Storage Services


Published on

With AWS, you can choose the right storage service for the right use case. This session shows the range of AWS choices - object storage to block storage - that is available to you. We include specifics about real-world deployments from customers who are using Amazon S3, Amazon EBS, Amazon Glacier, and AWS Storage Gateway.

Published in: Technology
  • Be the first to comment

Intro to AWS: Storage Services

  1. 1. ©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved Intro to AWS: Storage Services Guy Farber, AWS Storage Business Development
  2. 2. AWS storage options • Scalable object storage • Inexpensive archive storage • Persistent block storage • Shared file system • Turnkey gateway solution
  3. 3. We are constantly producing more data
  4. 4. From all types of industries
  5. 5. Object Storage
  6. 6. Amazon S3 Simple Storage Service
  7. 7. 2012 2013 2014 2015 102% year-over-year increase in data transfer to and from S3 (Q4 2014 vs Q4 2013, not including Amazon use) S3 usage
  8. 8. 99.999999999% durability Nice try kid…
  9. 9. Reduced redundancy option 99.99% saves ~20%
  10. 10. S3 resources: buckets and objects
  11. 11. S3 website: static content
  12. 12. 1 PB raw storage 800 TB usable storage 600 TB allocated storage 400 TB application data S3 capacity pricing—pay only for what you use! Amazon S3
  13. 13. S3 continuous cost reduction • Available through 11 regions globally • Priced at per GB-month rates • 8 price reductions since launch • 51% average S3 capacity fee reduction on 4/1/2014 • TCO: comparing on-premises to S3 – Can be challenging for some customers – We can help!
  14. 14. S3 Amazon EMR
  15. 15. Amazon S3—New Features
  16. 16. S3 event notifications Delivers notifications to Amazon SNS, Amazon SQS, or AWS Lambda when events occur in S3 S3 Events SNS topic SQS queue Lambda function Notifications Foo() { … }
  17. 17. • Preserve, retrieve, and restore every version of every object stored in your bucket • S3 automatically adds new versions and preserves deleted objects with delete markers • Easily control the number of versions kept by using lifecycle expiration policies • Easy to turn on in the AWS Management Console S3 versioning
  18. 18. S3 cross-region replication Automated, fast, and reliable asynchronous replication of data across AWS regions Source (Virginia) Destination (Oregon) • Only replicates new PUTs. Once S3 is configured, all new uploads into a source bucket will be replicated • Entire bucket or prefix based • 1:1 replication between any 2 regions • Versioning required Use cases: • Compliance—store data hundreds of miles apart • Lower latency—distribute data to regional customers) • Security—create remote replicas managed by separate AWS accounts
  19. 19. Cross Region Replication Demo
  20. 20. S3 use cases • Web-scale storage capacity and performance for web applications • Single-origin store with delivery through Amazon CloudFront • Staging and persistent store for Big Data applications • Storage target for backup and active archive
  21. 21. Amazon Glacier Low-cost archiving service
  22. 22. $0.01 per GB-month
  23. 23. $120 per TB/year
  24. 24. 99.999999999% durability
  25. 25. 3–5 hours data retrieval
  26. 26. Storage costs vs. retrieval costs
  27. 27. Glacier benefits • Reduce cost for long-term archiving • Leverage unlimited storage capacity • Replace tape museums • Improve durability
  28. 28. Amazon S3 – Glacier integration Policy-based archiving service
  29. 29. S3 lifecycle policies → Key prefix “logs/” Transition objects to Glacier 30 days after creation Delete 365 days after creation date <LifecycleConfiguration> <Rule> <ID>archive-in-30-days</ID> <Prefix>logs/</Prefix> <Status>Enabled</Status> <Transition> <Days>30</Days> <StorageClass>GLACIER</StorageClass> </Transition> <Expiration> <Days>365</Days> </Expiration> </Rule> </LifecycleConfiguration
  30. 30. SoundCloud—leveraging Glacier for audio transcoding • World’s leading social sound platform • Audio files must be transcoded and stored in multiple formats S3 Glacier
  31. 31. Use cases for S3 lifecycle policies • Data tiering in the cloud • Manage object versioning for logical data protection • Deletion of data from Glacier by policy
  32. 32. Block Storage
  33. 33. Instance storage
  34. 34. Amazon EBS Elastic Block Storage
  35. 35. 1GB 16 TB
  36. 36. EBS General Purpose (SSD) Up to 16 TB 10,000 IOPS Up to 160 Mbps Provisioned IOPS (SSD) Up to 16 TB 20,000 IOPS Up to 320 Mbps
  37. 37. Price Performance EBS Magnetic General Purpose Provisioned IOPS Use cases Infrequent data access Boot volumes Small to med DBs Dev and Test I/O intensive Relational DBs NoSQL DBs Storage media Magnetic disk-backed SSD-backed SSD-backed Max IOPS 40–200 IOPS 10,000 IOPS 20,000 IOPS Latency (random read) 20–40 ms 1–2 ms 1–2 ms Availability Designed for 99.999% Designed for 99.999% Designed for 99.999% Price $.05/GB-month $.05/million I/O $.10/GB-month $.125/GB-month $.065/provisioned IOPS
  38. 38. Amazon EBS snapshots
  39. 39. EBS use cases • Persistent block storage for Amazon EC2 • Transactional workloads • File system for an instance—NTFS, ExtFS, and so on
  40. 40. Introducing Amazon Elastic File System Cloud-based file system
  41. 41. What is EFS? • Fully managed file system for EC2 instances • Provides standard file system semantics • Elastically grows to petabyte scale • Delivers performance for a wide variety of workloads • Highly available and durable simple elastic scalable 1 2 3
  42. 42. EFS is designed for a broad range of use cases, such as… • Content repositories • Development environments • Home directories • Big Data
  43. 43. EFS is simple • Fully managed – No hardware, network, or file layer – Create a scalable file system in seconds! • Seamless integration with existing tools and apps – NFS v4—open and non-proprietary – Standard file system semantics – Works with standard OS file system APIs • Simple pricing = simple forecasting – EFS price: $0.30/GB-month 1
  44. 44. EFS is elastic • File systems grow and shrink automatically as you add and remove files • No need to provision storage capacity or performance • You pay only for the storage space you use, with no minimum fee 2
  45. 45. • File systems can grow to petabyte scale • Throughput and IOPS scale automatically as file systems grow • Consistent low latencies regardless of file system size • Support for thousands of concurrent NFS connections EFS is scalable3
  46. 46. How does it all fit together? AVAILABILITY ZONE 1 REGION AVAILABILITY ZONE 2 AVAILABILITY ZONE 3 VPC EC2 EC2 EC2 EC2 Customer’s file system
  47. 47. Why does this matter?... …to app owners and developers? …to your business? • Easy to move existing code/applications/tools used today with existing NFS servers to the AWS cloud • Simple shared file storage solution for new cloud-native applications • Predictable pricing with no up-front investment • Increased agility • Spend less time managing file storage and more time focusing on your business …to IT administrators? • Eliminates need to manage and maintain file system storage at scale
  48. 48. Amazon Storage Gateway
  49. 49. Storage Gateway Your on-ramp to AWS cloud storage: • Back up into S3 • DR on EC2 • Archive into Amazon Glacier • iSCSI or VTL interface
  50. 50. Summary: the Amazon storage portfolio S3 • Object storage: data presented as buckets of objects • Data access by using APIs over the Internet EFS • File storage (analogous to NAS): data presented as a file system • Shared low-latency access from multiple EC2 instances Glacier • Archival storage: data presented as vaults/archives of objects • Lowest-cost storage, infrequent access by using APIs over the Internet Storage Gateway • Back up and archive data into S3 and Amazon Glacier EBS • Block storage (analogous to SAN): data presented as disk volumes • Lowest-latency access from single EC2 instances