Your SlideShare is downloading. ×
0
© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or i...
Agenda
• Common Data Storage Terms
• Introduction to AWS Storage Platforms
• Use Cases Drive Platform Selection
• In their...
Goals for this session
• 101 Level Introduction to AWS Data Storage
– Survey of AWS Data Storage Platforms
– Learn how AWS...
AWS Storage Options
• Simple Storage
• General Purpose Storage
• Archive Storage
• Storage Gateway
We are constantly producing more data
From all types of industries
Classic and Contemporary
Applications
File Systems
Database(s)
User Generated Content
Machine Generated Content
Messaging,...
First, let’s visit some storage basics
Storage Basics - Block vs. File vs. Object
• Block Storage : SAN : EBS
– Data organized as an array of unrelated blocks
– ...
Storage Basics – SAN, NAS , Object
SERVERS
Object
NAS
SAN
Datacenter Proximity
Ethernet / Optical
Datacenter Proximity
Eth...
No Up-Front
Capital Expense
Pay Only for
What You Use
Self-Service
Infrastructure
Easily Scale
Up and Down
Improve Agility...
AWS Global Infrastructure
10 Regions
25 Availability Zones
51 Edge Locations
AWS Regions and Availability Zones
You decide where your data resides
Storage Services
Amazon EBS
EBS
Block storage for use
with Amazon EC2
Amazon S3
Images
Videos
Files
Binaries
Snapshots
Int...
#1
Object Storage
●○○○○
AMAZON S3
SIMPLE STORAGE SERVICE
99.999999999%
Durability
Trillions
Of Unique Customer Objects
1.5 Million+
Peak Transactions Per Second
Storage Tiers: Buckets + Unlimited Objects
Reduced Redundancy Option
99.99% saves 20-30%
Amazon S3 Website: Content
Use Amazon S3 When You Need
• Application level connectivity
• Unlimited storage capacity
• Extremely High durability
• St...
#2
Block Storage for
General Computing
●●○○○
AMAZON EBS
ELASTIC BLOCK STORAGE
10GB
1TB
IOPS
Provisioned
4000
IOPS
Amazon EBS
Snapshots
Use Amazon EBS When You Need
• Persistent storage
• Data changes frequently
• Block storage for your databases – Provision...
AMAZON GLACIER
LOW-COST ARCHIVING SERVICE
$0.01
Per GB / Month
$120
Per TB / Year
99.999999999%
Durability
3-5 Hours
Data Retrieval
Use Amazon Glacier When You Need
• Inexpensive, Long-term archiving. Cold Use.
• Unlimited storage capacity
• Eliminated T...
Amazon S3 / Amazon Glacier Integration
POLICY-BASED ARCHIVING SERVICE
Lifecycle Rule
Archive Recovery Process with Tape
+ Days or Weeks
Archive Recovery Process with AWS &
Amazon Glacier
$$
Hours
Glacier S3 EC2
/HPC
CloudFront Generating
Business
Value
Use Amazon S3 and Amazon Glacier When You Need
• Backup and Archive in the cloud
• Archive data from Amazon S3/RRS to Amaz...
AWS STORAGE GATEWAY
The Good News is that Cloud isn’t an ‘All or Nothing’
Corporate
Data
Centers
On-Premises
Resources
Cloud
Resources
Integra...
Active Directory
Network Configuration
Encryption
Backup Appliances
Your On-Premises
Apps
Corporate
Data Centers
Users & A...
AWS Storage Gateway
What is AWS Storage Gateway?
• Integrates on-prem IT environments with Cloud
storage for departmental and remote office ba...
Scale: for hundreds of millions of users
Global: anywhere in the world
Speed: as fast as Google
Mobile: on any device
The future of mapping is open.
OpenStreetMap provides
coverage with street level detail
OpenStreetMap is
constantly improving.
Total users
1 million
as of January 2013
Mapbox Satellite
We have 4.2 billion satellite
images (map tiles) on S3
We need: - Always hot (fast) - Fast
anywhere in the world- Redundant -
Geo-redundant - Integrated into our
processing pipe...
Integrated processing pipeline
hmmm, this is
slow going upgrade
EC2 type
w00t!
killing it
spiked regional
spot pricing :p
increases $ for
spot pricing
Enterprise Scale
Launch
Imagery refresh
Wired
160,000
(views per hour)
aka 240,000,000
tiles/ hour
+600tiles/ second
ENTERPRISE
IT’S ALL ABOUT
CHOICEPERFORMANCE-ORIENTED
COST-ORIENTED
Trusted by Enterprises Around the World
Used by Government Agencies & Educational Institutions
Worldwide
2014 Premier Tier Partners
TCO
• AWS Storage Platforms Provide Lowest TCO by
13-44% as compared to classic deployments.
We take cost out for customer...
Goals for this session
• 101 Level Introduction to AWS Data Storage
– Survey of AWS Data Storage Platforms
– Learn how AWS...
© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or i...
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Choosing the Right Data Storage Solution
Upcoming SlideShare
Loading in...5
×

Choosing the Right Data Storage Solution

1,018

Published on

With AWS, you can choose the right storage service for the right use case. Given the myriad of choices, from object storage to block storage, this session will profile details and examples of some of the choices available to you, with details on real world deployments from customers who are using Amazon Simple Storage Service (Amazon S3), Amazon Elastic Block Store (Amazon EBS), Amazon Glacier, and AWS Storage Gateway.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,018
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
98
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Choosing the Right Data Storage Solution"

  1. 1. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Storage & Archiving: Choosing the Right Data Storage Solution Timothy K. Harder, Global Business Development harder@amazon.com March 26, 2014
  2. 2. Agenda • Common Data Storage Terms • Introduction to AWS Storage Platforms • Use Cases Drive Platform Selection • In their own words : Mapbox.com • TCO • What to do next
  3. 3. Goals for this session • 101 Level Introduction to AWS Data Storage – Survey of AWS Data Storage Platforms – Learn how AWS Cloud Storage compares to classic models – Cloud Storage Use Cases – How to get started
  4. 4. AWS Storage Options • Simple Storage • General Purpose Storage • Archive Storage • Storage Gateway
  5. 5. We are constantly producing more data
  6. 6. From all types of industries
  7. 7. Classic and Contemporary Applications File Systems Database(s) User Generated Content Machine Generated Content Messaging, Collaboration Backup, Archive, DR
  8. 8. First, let’s visit some storage basics
  9. 9. Storage Basics - Block vs. File vs. Object • Block Storage : SAN : EBS – Data organized as an array of unrelated blocks – Host File System places data on disk: Microsoft NTFS or Unix ZFS – Structured data is predicted to grow at 34% CAGR until 2018 • File Storage : NAS :EBS – Unrelated data blocks managed by a file (serving) system – Native file system places data on disk: EXT or Filers – Unstructured data is predicted to grow at 45.6% CAGR until 2018 • Object Storage :S3:RRS:Glacier – A new data access, data storage, and data management model • API access to data vs. traditional block or file system access • Metadata Driven, Policy-based, Self Managing storage • No host overhead for storage functions – A system that stores virtual containers that encapsulate the data, data attributes, metadata, and Object IDs
  10. 10. Storage Basics – SAN, NAS , Object SERVERS Object NAS SAN Datacenter Proximity Ethernet / Optical Datacenter Proximity Ethernet Cloud Proximity API / HTTP(S) FILESYSTEMS APPLICATIONShttps:// nfs / cifs FC / iSCSI / FCoE • SAN and NAS are integrated with clients via operating system • Cloud Storage is an application level integration
  11. 11. No Up-Front Capital Expense Pay Only for What You Use Self-Service Infrastructure Easily Scale Up and Down Improve Agility & Time-to-Market Low Cost Amazon Storage Service Benefits
  12. 12. AWS Global Infrastructure 10 Regions 25 Availability Zones 51 Edge Locations
  13. 13. AWS Regions and Availability Zones You decide where your data resides
  14. 14. Storage Services Amazon EBS EBS Block storage for use with Amazon EC2 Amazon S3 Images Videos Files Binaries Snapshots Internet scale storage via API AWS Storage Gateway S3, Glacier Integrates on-premises IT and AWS storage Amazon Glacier Images Videos Files Binaries Snapshots Storage for archiving and backup
  15. 15. #1 Object Storage ●○○○○
  16. 16. AMAZON S3 SIMPLE STORAGE SERVICE
  17. 17. 99.999999999% Durability
  18. 18. Trillions Of Unique Customer Objects
  19. 19. 1.5 Million+ Peak Transactions Per Second
  20. 20. Storage Tiers: Buckets + Unlimited Objects
  21. 21. Reduced Redundancy Option 99.99% saves 20-30%
  22. 22. Amazon S3 Website: Content
  23. 23. Use Amazon S3 When You Need • Application level connectivity • Unlimited storage capacity • Extremely High durability • Storage for user or machine generated content • Content Depots and Data Lakes • Gold copy, Backup and Data Life Cycle
  24. 24. #2 Block Storage for General Computing ●●○○○
  25. 25. AMAZON EBS ELASTIC BLOCK STORAGE
  26. 26. 10GB 1TB
  27. 27. IOPS Provisioned 4000 IOPS
  28. 28. Amazon EBS Snapshots
  29. 29. Use Amazon EBS When You Need • Persistent storage • Data changes frequently • Block storage for your databases – Provisioned IOPS volumes • Filesystem enablment NTFS, ExtFS, RAID, LVM… • Access to raw, unformatted block-level storage
  30. 30. AMAZON GLACIER LOW-COST ARCHIVING SERVICE
  31. 31. $0.01 Per GB / Month
  32. 32. $120 Per TB / Year
  33. 33. 99.999999999% Durability
  34. 34. 3-5 Hours Data Retrieval
  35. 35. Use Amazon Glacier When You Need • Inexpensive, Long-term archiving. Cold Use. • Unlimited storage capacity • Eliminated Tape Museums • Eliminate Tech Refresh • High durability
  36. 36. Amazon S3 / Amazon Glacier Integration POLICY-BASED ARCHIVING SERVICE
  37. 37. Lifecycle Rule
  38. 38. Archive Recovery Process with Tape + Days or Weeks
  39. 39. Archive Recovery Process with AWS & Amazon Glacier $$ Hours Glacier S3 EC2 /HPC CloudFront Generating Business Value
  40. 40. Use Amazon S3 and Amazon Glacier When You Need • Backup and Archive in the cloud • Archive data from Amazon S3/RRS to Amazon Glacier by policy • Delete data from Amazon Glacier by policy • Tiering Strategy to support Information Lifecycle Management
  41. 41. AWS STORAGE GATEWAY
  42. 42. The Good News is that Cloud isn’t an ‘All or Nothing’ Corporate Data Centers On-Premises Resources Cloud Resources Integration
  43. 43. Active Directory Network Configuration Encryption Backup Appliances Your On-Premises Apps Corporate Data Centers Users & Access Rules (IAM) Your Private Network (VPC) Encryption (S3, RDS, HSM) Backups (Storage Gateway) Your Cloud Apps AWS Direct Connect Integrating AWS with your existing On-Premises Infrastructure
  44. 44. AWS Storage Gateway
  45. 45. What is AWS Storage Gateway? • Integrates on-prem IT environments with Cloud storage for departmental and remote office backup and DR • Utilizes a virtual appliance that sits in your datacenter • Exposes storage and backup interface on front end • Stores primary data on-AWS in Amazon S3 or on- premise with data backed-up to Amazon S3 as Amazon EBS snapshots or Glacier
  46. 46. Scale: for hundreds of millions of users Global: anywhere in the world Speed: as fast as Google Mobile: on any device
  47. 47. The future of mapping is open. OpenStreetMap provides coverage with street level detail
  48. 48. OpenStreetMap is constantly improving. Total users 1 million as of January 2013
  49. 49. Mapbox Satellite
  50. 50. We have 4.2 billion satellite images (map tiles) on S3
  51. 51. We need: - Always hot (fast) - Fast anywhere in the world- Redundant - Geo-redundant - Integrated into our processing pipeline- Highly scalable
  52. 52. Integrated processing pipeline
  53. 53. hmmm, this is slow going upgrade EC2 type w00t! killing it spiked regional spot pricing :p increases $ for spot pricing
  54. 54. Enterprise Scale
  55. 55. Launch Imagery refresh Wired
  56. 56. 160,000 (views per hour)
  57. 57. aka 240,000,000 tiles/ hour
  58. 58. +600tiles/ second
  59. 59. ENTERPRISE
  60. 60. IT’S ALL ABOUT CHOICEPERFORMANCE-ORIENTED COST-ORIENTED
  61. 61. Trusted by Enterprises Around the World
  62. 62. Used by Government Agencies & Educational Institutions Worldwide
  63. 63. 2014 Premier Tier Partners
  64. 64. TCO • AWS Storage Platforms Provide Lowest TCO by 13-44% as compared to classic deployments. We take cost out for customers in every engagement. • When is your next capacity upgrade or refresh?
  65. 65. Goals for this session • 101 Level Introduction to AWS Data Storage – Survey of AWS Data Storage Platforms – Learn how AWS Cloud Storage compares to classic models – Cloud Storage Use Cases • How to get started – AWS.AMAZON.COM/FREE – AWS.AMAZON.COM/BACKUP-STORAGE
  66. 66. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Storage & Archiving: Choosing the Right Data Storage Solution Timothy K. Harder, Global Business Development harder@amazon.com March 26, 2014 Thank you!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×