Lunch and Learn - Store and Move your Data To & From the AWS Cloud, Markku Lepisto


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Lunch and Learn - Store and Move your Data To & From the AWS Cloud, Markku Lepisto

  1. 1. Store and move your data to & from the AWS cloud Markku Lepistö Senior Technology Evangelist @markkulepisto
  2. 2. Agenda • What’s the Problem? • Data Transfer • Data Stores • Backup and DR • What’s Next?
  3. 3. Problem
  4. 4. We  are  constantly  producing  more  data  
  5. 5. Pressure on Traditional On-Premise Solutions Primary Block Storage Primary File Storage Archival Storage Disk Based Backup Storage Tape Infrastructure & Management Complex  /  Bri5le   Siloed  /  Expensive   Capacity  Guessing   Replicated Storage for Disaster Recovery Offsite Locations GeoResilience
  6. 6. Traditional Backups •  Long/slow recovery time •  Low durability (higher probability for data loss) •  Requires Tape Museums •  Expensive Tech Refresh
  7. 7. AWS Storage Services Help?
  8. 8. Transfer
  9. 9. AWS  Global  Infrastructure   10 Regions 25 Availability Zones 51 Edge Locations
  10. 10. AWS  Direct  Connect   AWS  Import/Export   Dedicated  bandwidth  between  your   Physical  transfer  of  media  into  and   site  and  AWS   out  of  AWS     When  -­‐  if  ever  -­‐  will  the  bandwidth  of     the  Internet  surpass  that  of  FedEx?   —Johan  Öbrink  
  11. 11. AWS Storage Options ElasDc  Block  Store   Simple  Storage  Service   Glacier   High  performance  block  storage  device   Highly  scalable  object  storage   Long  term  object  archive    1GB  to  1TB  in  size,  max  4000  IOPS/volume   Mount  as  drives  to  instances     1  byte  to  5TB  in  size   99.999999999%  durability   1c/GB/month,  $120/TB/year   99.999999999%  durability   with  snapshot/cloning  funcOonaliOes   Very  fast   mounted   volumes   Fast  object  storage   Slow,   infrequent   archive  
  12. 12. Storage Tiers & Automated Life-Cycle
  13. 13. SpoDfy  adds  over  20,000  tracks  a  day   Reduced  Redundancy  with  ~20%  savings    
  14. 14. Raw Storage at On perm vs. Cloud 1PB  Raw  Storage   800TB  Usable  Storage   600TB  Allocated  Storage   400TB  Wri5en  ApplicaDon  Storage   S3  &  Glacier  –     Only  actual   usage  is   charged  
  15. 15. •  nearmap routinely captures hi-res aerial images covering 75% of Australia’s population •  nearmap provides browser-based access to up-to-date and historical aerial imagery to thousands of companies, large and small 16  
  16. 16. “Only Amazon Glacier had the scale nearmap required to store thousands of terabytes of raw PhotoMap and digital images.” “Without Amazon Glacier’s secure and scalable archival solution, it would have been cost prohibitive for us to keep these images, which are important business IP.” - Paul Peterson VP of Engineering
  17. 17. Access via Console, SDKs and Third-party Solutions Via SDK Via 3rd parties Internet Web Services API HTTP(S) S3   Glacier  
  18. 18. AWS Storage Gateway for offsite disk backups Gateway-Cached Volumes Your  Datacenter   iSCSI Production Systems Backup Servers AWS Storage Gateway VM On-premises Host Direct Attached or Storage Area Network Disks SSL AWS  Storage   Gateway  Service   Amazon  Simple   Storage  Service   (S3)  
  19. 19. AWS Storage Gateway for departmental file shares Gateway-Stored Volumes Your  Datacenter   CIFS/ NFS iSCSI File servers AWS Storage Gateway VM On-premises Host End Users Direct Attached or Storage Area Network Disks SSL AWS  Storage   Gateway  Service   Amazon  Simple   Storage  Service   (S3)  
  20. 20. AWS Storage Gateway - Virtual Tape Library
  21. 21. Partner Solutions •  •  •  •  •  CIFS/NFS  Local  PresentaDon   Appliance  or  Virtual  Machine   In-­‐Line  De-­‐DuplicaDon   Customer  Retained  Security  Keys   Local  Cache  for  recently  archived  data  
  22. 22. Partner Solutions It’s easy to backup Oracle DBs to Amazon S3 Read the case study Corporate  data  center   Oracle   RMAN   Oracle   Secure   Backup   Module   S3  
  23. 23. Choosing the Data Store
  24. 24. Structure   Low   High   High   High   Low   Low   Amazon   Glacier   Amazon  S3   Amazon   ElasDCache   Amazon   DynamoDB   Am Clou azon   dSea rch   Amazon   EMR   Amazon   RDS   Amazon   Redshic   Request  rate   Cost/GB   Latency   Data  Volume   Low   Low   High   High  
  25. 25. cost, volume, access pattern structure
  26. 26. Recovering from Disasters
  27. 27. Backup  and   restore   Pilot  light   Warm   standby   Hot   standby  
  28. 28. cost speed of recovery
  29. 29. Backup the DB
  30. 30. Backup Server Images
  31. 31. Restore DB and App Servers Create instances from AMIs Restore data from backups
  32. 32. Summary
  33. 33. Next Generation Enterprise Storage   Customer   Data   Center   Block   File   AWS  Direct   Connect   AWS   Cloud   Archive   Backup   Internet  Web   Services  API   HTTP(S)   Disaster  Recovery   Storage  Use   Cases  
  34. 34. Tradi8onal:  On-­‐Premise  Solu8ons   Primary   Block  Storage   Primary  File   Storage   Archival   Storage   Disk  Based   Backup   Storage   Tape   Infrastructure  &   Management   Replicated   Storage  for   Disaster   Recovery   Offsite   LocaDons   Geo-­‐ Resilience   Next  Genera8on:  Gateway  &  Cloud  Storage   Reduce  SAN   Footprint   Reduce  NAS   Footprint   Eliminate   Archival   Hardware  &   Socware   Reduce  Complexity.   Eliminate   Backup   Hardware  &   Socware   Eliminate  Tape   Recover  in   Cloud  or  any   Data  Center   Reduce  Cost.   Eliminate   Secondary   Data  Centers  
  35. 35. “AWS  allows  us  to  store  informaDon  in  a  cost  effecDve  manner.  It   really  is  a  win-­‐win  for  us  and  our  customers.”   -­‐  Jeff  Kimsey,     Associate  Vice  President   The  Story   •  The  Global  Data  Products  division  of  NASDAQ   Business  Benefit     •  $50  the  first  month   OMX  uses  detailed  historical  stock  price  data   for  its  Market  Replay  and  Data-­‐On-­‐Demand   AWS  is  a  low  cost  soluDon  that  only  cost     •  Avoided  the  need  to  buy  expensive  hardware   products   •  and  improved  product  Dme     Wanted  to  support  their  growing  customer   to  market   base  without  worrying  about  infrastructure,   •  informaDon  to  the  millisecond   security,  or  performance   •  Needed  to  be  able  to  increase  storage  space   Can  store  ten  years  of  historical  stock  price   •  Able  to  upload  hundreds  of  thousands  of  data   and  keep  data  indefinitely   •  files  to  the  AWS  cloud  and  deliver  informaDon  to   Using  Amazon  S3     customers  in  seconds  
  36. 36. data abandoned information actionable
  37. 37. Business  Value  From  The  Data   Managed  DBs   Amazon   DynamoDB   Delivery   Amazon  CloudFront       Transcoding   Amazon   ElasDc   Transcoder   Workflow   Amazon  SWF   AnalyDcs   Amazon  ElasDc   MapReduce   Discovery   (OR)   Glacier   Amazon   Cloudsearch   DR,  Archive,  Backup  and   Compute   Amazon  VPC   Amazon  EC2   Amazon  EBS  
  38. 38. Benefits of AWS Storage Financial –  –  –  –  Significant reduction in capital investments Pay for only what you use: Metered Utility Eliminate the need to over purchase AWS prices continue to fall over time Operational –  Streamline and simplify data management operations –  Reduction of systems and data centers –  Focus employees on proactive and innovative tasks Business –  Never stop the business as capacity is endless –  Never slow the pace of innovation –  Turn IT into an innovation enabler
  39. 39. Contact Your AWS Account Manager To discuss your use cases, opportunities to try AWS storage services And follow us on at @AWSCloudSEAsia
  40. 40. Demo
  41. 41. Thank  you   Markku  Lepistö  –  Senior  Technology  Evangelist   @markkulepisto