Store and move your data
to & from
the AWS cloud
Markku Lepistö
Senior Technology Evangelist
markku@amazon.com
@markkulepi...
Agenda
• What’s the Problem?
• Data Transfer
• Data Stores
• Backup and DR
• What’s Next?
Problem
We	
  are	
  constantly	
  producing	
  more	
  data	
  
Pressure on Traditional On-Premise Solutions

Primary
Block
Storage

Primary File
Storage

Archival
Storage

Disk Based
Ba...
Traditional Backups
•  Long/slow recovery time

•  Low durability (higher probability for data loss)
•  Requires Tape Muse...
AWS Storage Services
Help?
Transfer
AWS	
  Global	
  Infrastructure	
  

10 Regions
25 Availability Zones
51 Edge Locations
AWS	
  Direct	
  Connect	
  

AWS	
  Import/Export	
  

Dedicated	
  bandwidth	
  between	
  your	
  

Physical	
  transfe...
AWS Storage Options
ElasDc	
  Block	
  Store	
  

Simple	
  Storage	
  Service	
  

Glacier	
  

High	
  performance	
  bl...
Storage Tiers & Automated Life-Cycle
SpoDfy	
  adds	
  over	
  20,000	
  tracks	
  a	
  day	
  
Reduced	
  Redundancy	
  with	
  ~20%	
  savings	
  
	
  
Raw Storage at On perm vs. Cloud

1PB	
  Raw	
  Storage	
  
800TB	
  Usable	
  Storage	
  
600TB	
  Allocated	
  Storage	
...
•  nearmap routinely captures hi-res aerial images covering 75% of
Australia’s population
•  nearmap provides browser-base...
“Only Amazon Glacier had the scale nearmap
required to store thousands of terabytes of raw
PhotoMap and digital images.”
“...
Access via Console, SDKs and Third-party Solutions
Via SDK

Via 3rd parties
Internet Web
Services API
HTTP(S)

S3	
  

Gla...
AWS Storage Gateway for offsite disk backups
Gateway-Cached Volumes
Your	
  Datacenter	
  

iSCSI

Production
Systems

Bac...
AWS Storage Gateway for departmental file shares
Gateway-Stored Volumes
Your	
  Datacenter	
  

CIFS/
NFS
iSCSI

File serv...
AWS Storage Gateway - Virtual Tape Library
Partner Solutions

• 
• 
• 
• 
• 

CIFS/NFS	
  Local	
  PresentaDon	
  
Appliance	
  or	
  Virtual	
  Machine	
  
In-­‐Lin...
Partner Solutions

It’s easy to backup Oracle DBs to Amazon S3
Read the case Amazon.com study

Corporate	
  data	
  center...
Choosing the Data Store
Structure	
  

Low	
  

High	
  
High	
  
High	
  
Low	
  
Low	
  

Amazon	
  
Glacier	
  

Amazon	
  S3	
  
Amazon	
  
El...
cost, volume, access pattern

structure
Recovering from Disasters
Backup	
  and	
  
restore	
  

Pilot	
  light	
  

Warm	
  
standby	
  

Hot	
  
standby	
  
cost

speed of recovery
Backup the DB
Backup Server
Images
Restore DB and
App Servers

Create instances
from AMIs

Restore data
from backups
Summary
Next Generation Enterprise Storage	
  
Customer	
  
Data	
  
Center	
  

Block	
  

File	
  

AWS	
  Direct	
  
Connect	
 ...
Tradi8onal:	
  On-­‐Premise	
  Solu8ons	
  

Primary	
  
Block	
  Storage	
  

Primary	
  File	
  
Storage	
  

Archival	
...
“AWS	
  allows	
  us	
  to	
  store	
  informaDon	
  in	
  a	
  cost	
  effecDve	
  manner.	
  It	
  
really	
  is	
  a	
  ...
data
abandoned

information
actionable
Business	
  Value	
  From	
  The	
  Data	
  
Managed	
  DBs	
  
Amazon	
  
DynamoDB	
  

Delivery	
  

Amazon	
  CloudFron...
Benefits of AWS Storage
Financial
– 
– 
– 
– 

Significant reduction in capital investments
Pay for only what you use: Met...
Contact Your AWS Account Manager

To discuss your use cases, opportunities to
try AWS storage services
And follow us on

a...
Demo
Thank	
  you	
  
Markku	
  Lepistö	
  –	
  Senior	
  Technology	
  Evangelist	
  
@markkulepisto	
  
Lunch and Learn - Store and Move your Data To & From the AWS Cloud, Markku Lepisto
Lunch and Learn - Store and Move your Data To & From the AWS Cloud, Markku Lepisto
Lunch and Learn - Store and Move your Data To & From the AWS Cloud, Markku Lepisto
Lunch and Learn - Store and Move your Data To & From the AWS Cloud, Markku Lepisto
Upcoming SlideShare
Loading in...5
×

Lunch and Learn - Store and Move your Data To & From the AWS Cloud, Markku Lepisto

862

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
862
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Lunch and Learn - Store and Move your Data To & From the AWS Cloud, Markku Lepisto"

  1. 1. Store and move your data to & from the AWS cloud Markku Lepistö Senior Technology Evangelist markku@amazon.com @markkulepisto
  2. 2. Agenda • What’s the Problem? • Data Transfer • Data Stores • Backup and DR • What’s Next?
  3. 3. Problem
  4. 4. We  are  constantly  producing  more  data  
  5. 5. Pressure on Traditional On-Premise Solutions Primary Block Storage Primary File Storage Archival Storage Disk Based Backup Storage Tape Infrastructure & Management Complex  /  Bri5le   Siloed  /  Expensive   Capacity  Guessing   Replicated Storage for Disaster Recovery Offsite Locations GeoResilience
  6. 6. Traditional Backups •  Long/slow recovery time •  Low durability (higher probability for data loss) •  Requires Tape Museums •  Expensive Tech Refresh
  7. 7. AWS Storage Services Help?
  8. 8. Transfer
  9. 9. AWS  Global  Infrastructure   10 Regions 25 Availability Zones 51 Edge Locations
  10. 10. AWS  Direct  Connect   AWS  Import/Export   Dedicated  bandwidth  between  your   Physical  transfer  of  media  into  and   site  and  AWS   out  of  AWS     When  -­‐  if  ever  -­‐  will  the  bandwidth  of     the  Internet  surpass  that  of  FedEx?   —Johan  Öbrink  
  11. 11. AWS Storage Options ElasDc  Block  Store   Simple  Storage  Service   Glacier   High  performance  block  storage  device   Highly  scalable  object  storage   Long  term  object  archive    1GB  to  1TB  in  size,  max  4000  IOPS/volume   Mount  as  drives  to  instances     1  byte  to  5TB  in  size   99.999999999%  durability   1c/GB/month,  $120/TB/year   99.999999999%  durability   with  snapshot/cloning  funcOonaliOes   Very  fast   mounted   volumes   Fast  object  storage   Slow,   infrequent   archive  
  12. 12. Storage Tiers & Automated Life-Cycle
  13. 13. SpoDfy  adds  over  20,000  tracks  a  day   Reduced  Redundancy  with  ~20%  savings    
  14. 14. Raw Storage at On perm vs. Cloud 1PB  Raw  Storage   800TB  Usable  Storage   600TB  Allocated  Storage   400TB  Wri5en  ApplicaDon  Storage   S3  &  Glacier  –     Only  actual   usage  is   charged  
  15. 15. •  nearmap routinely captures hi-res aerial images covering 75% of Australia’s population •  nearmap provides browser-based access to up-to-date and historical aerial imagery to thousands of companies, large and small 16  
  16. 16. “Only Amazon Glacier had the scale nearmap required to store thousands of terabytes of raw PhotoMap and digital images.” “Without Amazon Glacier’s secure and scalable archival solution, it would have been cost prohibitive for us to keep these images, which are important business IP.” - Paul Peterson VP of Engineering
  17. 17. Access via Console, SDKs and Third-party Solutions Via SDK Via 3rd parties Internet Web Services API HTTP(S) S3   Glacier  
  18. 18. AWS Storage Gateway for offsite disk backups Gateway-Cached Volumes Your  Datacenter   iSCSI Production Systems Backup Servers AWS Storage Gateway VM On-premises Host Direct Attached or Storage Area Network Disks SSL AWS  Storage   Gateway  Service   Amazon  Simple   Storage  Service   (S3)  
  19. 19. AWS Storage Gateway for departmental file shares Gateway-Stored Volumes Your  Datacenter   CIFS/ NFS iSCSI File servers AWS Storage Gateway VM On-premises Host End Users Direct Attached or Storage Area Network Disks SSL AWS  Storage   Gateway  Service   Amazon  Simple   Storage  Service   (S3)  
  20. 20. AWS Storage Gateway - Virtual Tape Library
  21. 21. Partner Solutions •  •  •  •  •  CIFS/NFS  Local  PresentaDon   Appliance  or  Virtual  Machine   In-­‐Line  De-­‐DuplicaDon   Customer  Retained  Security  Keys   Local  Cache  for  recently  archived  data  
  22. 22. Partner Solutions It’s easy to backup Oracle DBs to Amazon S3 Read the case Amazon.com study Corporate  data  center   Oracle   RMAN   Oracle   Secure   Backup   Module   S3  
  23. 23. Choosing the Data Store
  24. 24. Structure   Low   High   High   High   Low   Low   Amazon   Glacier   Amazon  S3   Amazon   ElasDCache   Amazon   DynamoDB   Am Clou azon   dSea rch   Amazon   EMR   Amazon   RDS   Amazon   Redshic   Request  rate   Cost/GB   Latency   Data  Volume   Low   Low   High   High  
  25. 25. cost, volume, access pattern structure
  26. 26. Recovering from Disasters
  27. 27. Backup  and   restore   Pilot  light   Warm   standby   Hot   standby  
  28. 28. cost speed of recovery
  29. 29. Backup the DB
  30. 30. Backup Server Images
  31. 31. Restore DB and App Servers Create instances from AMIs Restore data from backups
  32. 32. Summary
  33. 33. Next Generation Enterprise Storage   Customer   Data   Center   Block   File   AWS  Direct   Connect   AWS   Cloud   Archive   Backup   Internet  Web   Services  API   HTTP(S)   Disaster  Recovery   Storage  Use   Cases  
  34. 34. Tradi8onal:  On-­‐Premise  Solu8ons   Primary   Block  Storage   Primary  File   Storage   Archival   Storage   Disk  Based   Backup   Storage   Tape   Infrastructure  &   Management   Replicated   Storage  for   Disaster   Recovery   Offsite   LocaDons   Geo-­‐ Resilience   Next  Genera8on:  Gateway  &  Cloud  Storage   Reduce  SAN   Footprint   Reduce  NAS   Footprint   Eliminate   Archival   Hardware  &   Socware   Reduce  Complexity.   Eliminate   Backup   Hardware  &   Socware   Eliminate  Tape   Recover  in   Cloud  or  any   Data  Center   Reduce  Cost.   Eliminate   Secondary   Data  Centers  
  35. 35. “AWS  allows  us  to  store  informaDon  in  a  cost  effecDve  manner.  It   really  is  a  win-­‐win  for  us  and  our  customers.”   -­‐  Jeff  Kimsey,     Associate  Vice  President   The  Story   •  The  Global  Data  Products  division  of  NASDAQ   Business  Benefit     •  $50  the  first  month   OMX  uses  detailed  historical  stock  price  data   for  its  Market  Replay  and  Data-­‐On-­‐Demand   AWS  is  a  low  cost  soluDon  that  only  cost     •  Avoided  the  need  to  buy  expensive  hardware   products   •  and  improved  product  Dme     Wanted  to  support  their  growing  customer   to  market   base  without  worrying  about  infrastructure,   •  informaDon  to  the  millisecond   security,  or  performance   •  Needed  to  be  able  to  increase  storage  space   Can  store  ten  years  of  historical  stock  price   •  Able  to  upload  hundreds  of  thousands  of  data   and  keep  data  indefinitely   •  files  to  the  AWS  cloud  and  deliver  informaDon  to   Using  Amazon  S3     customers  in  seconds  
  36. 36. data abandoned information actionable
  37. 37. Business  Value  From  The  Data   Managed  DBs   Amazon   DynamoDB   Delivery   Amazon  CloudFront       Transcoding   Amazon   ElasDc   Transcoder   Workflow   Amazon  SWF   AnalyDcs   Amazon  ElasDc   MapReduce   Discovery   (OR)   Glacier   Amazon   Cloudsearch   DR,  Archive,  Backup  and   Compute   Amazon  VPC   Amazon  EC2   Amazon  EBS  
  38. 38. Benefits of AWS Storage Financial –  –  –  –  Significant reduction in capital investments Pay for only what you use: Metered Utility Eliminate the need to over purchase AWS prices continue to fall over time Operational –  Streamline and simplify data management operations –  Reduction of systems and data centers –  Focus employees on proactive and innovative tasks Business –  Never stop the business as capacity is endless –  Never slow the pace of innovation –  Turn IT into an innovation enabler
  39. 39. Contact Your AWS Account Manager To discuss your use cases, opportunities to try AWS storage services And follow us on at @AWSCloudSEAsia
  40. 40. Demo
  41. 41. Thank  you   Markku  Lepistö  –  Senior  Technology  Evangelist   @markkulepisto  

×