Storage Solutions in the AWS Cloud                   Miles Ward – Enterprise                   Solutions Architect
Traditional Storage  On-Premise Storage Options    SAN – network-attached block devices: LUNs    DAS – local block devices...
Cloud Storage AWS offers multiple cloud-based storage options.    • Amazon S3 (Simple Storage Service)    • Amazon EBS (El...
Other Storage options? Amazon S3, EBS, and EC2 Instance Store volumes are the basic storage building blocks, but for more ...
Storage by the Numbers             S3                          EBS                LocalScale        Unlimited Storage     ...
S3 (Simple Storage Service) Pay for exactly what you use Highly Durable Limitlessly scalable Natively On-Line
S3 Best Practices Replicate Metadata Parallel IO for max speed   Multipart Upload Sorted Writes Resource-level IAM permis...
S3 Use Cases and Users Media Serving Media Sharing File Storage / Backup Static Content Big Data
Amazon EBS Pay for what you provision POSIX compliant network attached block device Native Redundancy and Write Cache; opt...
EBS Best Practices Pre-warming Snapshots Striping (RAID 0) Block-device-mapping
EBS Use Cases and Users Application hosting                          Envoy Media Group Case Stud                          ...
EC2 Instance Store or Ephemeral Volumes Free with your EC2 Instance Zero Network Overhead; local, direct attached resource...
Instance Store Best Practices App-level replication Striping (RAID 0, 1) Block-device-mapping
Instance Store Use Cases and Users Stateless Web Hosts Transcoding Caching HPC DNFS
Better Together: Cross-system patterns EBS <-> S3: Snapshots S3 <-> SimpleDB: meta-data store S3 <-> Local Storage: Caching
Short Detour: S3 vs EBS AMIs Two types of EC2 AMIs and Instances:   Instance store (“S3-backed”)     Original EC2, boot fr...
So, which should I choose? Might be all of them! S3: Static Online Content, Media Serving, Backups, Big Data, Centralized ...
Test! It costs pennies to evaluate these technologies against your use-case, and making the right choice could save you mi...
How do I get it to the Cloud? AWS Import/Export Faster than Internet transfer and more cost effective than upgrading your ...
Read the originals!   Storage Options in the AWS Cloud:https://media.amazonwebservices.com/AWS_Storage_Options.pdf   Stora...
Thanks!          http://aws.amazon.com
Upcoming SlideShare
Loading in...5
×

AWS Summit 2011: Data Storage Solutions in the AWS Cloud

1,616

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,616
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

AWS Summit 2011: Data Storage Solutions in the AWS Cloud

  1. 1. Storage Solutions in the AWS Cloud Miles Ward – Enterprise Solutions Architect
  2. 2. Traditional Storage On-Premise Storage Options SAN – network-attached block devices: LUNs DAS – local block devices (disks) NAS – network attached file systems, filers Differ in, Performance, Durability, and Cost Often used to form a Storage Hierarchy
  3. 3. Cloud Storage AWS offers multiple cloud-based storage options. • Amazon S3 (Simple Storage Service) • Amazon EBS (Elastic Block Storage) • Amazon EC2 Local Instance Store (Ephemeral volumes) Each has a unique combination of performance, durability, cost, and interface Often used to form a Storage Hierarchy
  4. 4. Other Storage options? Amazon S3, EBS, and EC2 Instance Store volumes are the basic storage building blocks, but for more specialized uses, or managed resources, we have other options as well: Cloudfront Amazon RDS SimpleDB SQS Relational Databases on EC2
  5. 5. Storage by the Numbers S3 EBS LocalScale Unlimited Storage 1TB per volume 147 – 1690GB 5TB objects ~44TB per server 1-5 volumes 100 name spaces (buckets)Durability 99.999999999% (standard) .1-.5% AFR Ephemeral design 99.99% requires application- (RRS) level redundancyLatency High Low Very LowManagement Zero Maintenance Snapshots, RAID, Manual Redundancy ConsistencyLocality Region-Centric AZ-Specific In-machineInterface REST or SOAP Block Device Block Device Web Services
  6. 6. S3 (Simple Storage Service) Pay for exactly what you use Highly Durable Limitlessly scalable Natively On-Line
  7. 7. S3 Best Practices Replicate Metadata Parallel IO for max speed  Multipart Upload Sorted Writes Resource-level IAM permissions
  8. 8. S3 Use Cases and Users Media Serving Media Sharing File Storage / Backup Static Content Big Data
  9. 9. Amazon EBS Pay for what you provision POSIX compliant network attached block device Native Redundancy and Write Cache; optimized for random writes and reads.
  10. 10. EBS Best Practices Pre-warming Snapshots Striping (RAID 0) Block-device-mapping
  11. 11. EBS Use Cases and Users Application hosting Envoy Media Group Case Stud Storage for multimedia content delivery on Amazon W Data Processing Introd Establi that of Database media and po channe implem Large Data Envoy moved their IT infrastructure from a traditional co-location advantage of -as-you- to share, and promptly delivered, Envoy selected Gluster in conce Challenges Envoy currently hosts their entire web presence on Amazon Web which has helped them meet the needs of their growing business base. One of their biggest challenges was ensuring the IT infrastru to respond to the highly dynamic nature of demand such as when launch and then grow exponentially. From a data storage standpo needed a solution that integrated easily with AWS and provided h Storing the data locally on EC2 instances is not suffcient since that persistent and would be lost if a server reboots. Amazon EBS volu to one instance at a time a problem if your business requires hu needed to ensure their data was easily shared.
  12. 12. EC2 Instance Store or Ephemeral Volumes Free with your EC2 Instance Zero Network Overhead; local, direct attached resource. High performance for sequential reads and writes Temporary; goes away when your instance does
  13. 13. Instance Store Best Practices App-level replication Striping (RAID 0, 1) Block-device-mapping
  14. 14. Instance Store Use Cases and Users Stateless Web Hosts Transcoding Caching HPC DNFS
  15. 15. Better Together: Cross-system patterns EBS <-> S3: Snapshots S3 <-> SimpleDB: meta-data store S3 <-> Local Storage: Caching
  16. 16. Short Detour: S3 vs EBS AMIs Two types of EC2 AMIs and Instances: Instance store (“S3-backed”) Original EC2, boot from ephemeral storage Can Start and Terminate only All data is ephemeral (unless separate EBS volume attached) EBS root (“boot from EBS”) Newer, boot from EBS volume Can Start, Stop, Create Image, and Terminate Ephemeral storage still available, but not exposed by default Use instance Block Device Mapping (command line and API only) Additional EBS volumes can be attached to either type
  17. 17. So, which should I choose? Might be all of them! S3: Static Online Content, Media Serving, Backups, Big Data, Centralized Resources EBS: Applications that require POSIX, Databases, Data Warehouses Ephemeral: Cache/Swap/Temp, Applications that require minimum latency
  18. 18. Test! It costs pennies to evaluate these technologies against your use-case, and making the right choice could save you millions.
  19. 19. How do I get it to the Cloud? AWS Import/Export Faster than Internet transfer and more cost effective than upgrading your connectivity Use cases: data migration, offsite backup, direct data interchange, disaster recovery WAN Optimization Asperatech, Riverbed, Tsunami-UDP, etc
  20. 20. Read the originals! Storage Options in the AWS Cloud:https://media.amazonwebservices.com/AWS_Storage_Options.pdf Storage Use Cases in the AWS Cloud:https://media.amazonwebservices.com/AWS_Storage_Use_Cases.pdf
  21. 21. Thanks! http://aws.amazon.com

×