• Save
Lunch and Learn - Store and Move your Data To & From the AWS Cloud, Markku Lepisto

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,148
On Slideshare
1,146
From Embeds
2
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
2

Embeds 2

http://www.slideee.com 2

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Store and move your data to & from the AWS cloud Markku Lepistö Senior Technology Evangelist markku@amazon.com @markkulepisto
  • 2. Agenda • What’s the Problem? • Data Transfer • Data Stores • Backup and DR • What’s Next?
  • 3. Problem
  • 4. We  are  constantly  producing  more  data  
  • 5. Pressure on Traditional On-Premise Solutions Primary Block Storage Primary File Storage Archival Storage Disk Based Backup Storage Tape Infrastructure & Management Complex  /  Bri5le   Siloed  /  Expensive   Capacity  Guessing   Replicated Storage for Disaster Recovery Offsite Locations GeoResilience
  • 6. Traditional Backups •  Long/slow recovery time •  Low durability (higher probability for data loss) •  Requires Tape Museums •  Expensive Tech Refresh
  • 7. AWS Storage Services Help?
  • 8. Transfer
  • 9. AWS  Global  Infrastructure   10 Regions 25 Availability Zones 51 Edge Locations
  • 10. AWS  Direct  Connect   AWS  Import/Export   Dedicated  bandwidth  between  your   Physical  transfer  of  media  into  and   site  and  AWS   out  of  AWS     When  -­‐  if  ever  -­‐  will  the  bandwidth  of     the  Internet  surpass  that  of  FedEx?   —Johan  Öbrink  
  • 11. AWS Storage Options ElasDc  Block  Store   Simple  Storage  Service   Glacier   High  performance  block  storage  device   Highly  scalable  object  storage   Long  term  object  archive    1GB  to  1TB  in  size,  max  4000  IOPS/volume   Mount  as  drives  to  instances     1  byte  to  5TB  in  size   99.999999999%  durability   1c/GB/month,  $120/TB/year   99.999999999%  durability   with  snapshot/cloning  funcOonaliOes   Very  fast   mounted   volumes   Fast  object  storage   Slow,   infrequent   archive  
  • 12. Storage Tiers & Automated Life-Cycle
  • 13. SpoDfy  adds  over  20,000  tracks  a  day   Reduced  Redundancy  with  ~20%  savings    
  • 14. Raw Storage at On perm vs. Cloud 1PB  Raw  Storage   800TB  Usable  Storage   600TB  Allocated  Storage   400TB  Wri5en  ApplicaDon  Storage   S3  &  Glacier  –     Only  actual   usage  is   charged  
  • 15. •  nearmap routinely captures hi-res aerial images covering 75% of Australia’s population •  nearmap provides browser-based access to up-to-date and historical aerial imagery to thousands of companies, large and small 16  
  • 16. “Only Amazon Glacier had the scale nearmap required to store thousands of terabytes of raw PhotoMap and digital images.” “Without Amazon Glacier’s secure and scalable archival solution, it would have been cost prohibitive for us to keep these images, which are important business IP.” - Paul Peterson VP of Engineering
  • 17. Access via Console, SDKs and Third-party Solutions Via SDK Via 3rd parties Internet Web Services API HTTP(S) S3   Glacier  
  • 18. AWS Storage Gateway for offsite disk backups Gateway-Cached Volumes Your  Datacenter   iSCSI Production Systems Backup Servers AWS Storage Gateway VM On-premises Host Direct Attached or Storage Area Network Disks SSL AWS  Storage   Gateway  Service   Amazon  Simple   Storage  Service   (S3)  
  • 19. AWS Storage Gateway for departmental file shares Gateway-Stored Volumes Your  Datacenter   CIFS/ NFS iSCSI File servers AWS Storage Gateway VM On-premises Host End Users Direct Attached or Storage Area Network Disks SSL AWS  Storage   Gateway  Service   Amazon  Simple   Storage  Service   (S3)  
  • 20. AWS Storage Gateway - Virtual Tape Library
  • 21. Partner Solutions •  •  •  •  •  CIFS/NFS  Local  PresentaDon   Appliance  or  Virtual  Machine   In-­‐Line  De-­‐DuplicaDon   Customer  Retained  Security  Keys   Local  Cache  for  recently  archived  data  
  • 22. Partner Solutions It’s easy to backup Oracle DBs to Amazon S3 Read the case Amazon.com study Corporate  data  center   Oracle   RMAN   Oracle   Secure   Backup   Module   S3  
  • 23. Choosing the Data Store
  • 24. Structure   Low   High   High   High   Low   Low   Amazon   Glacier   Amazon  S3   Amazon   ElasDCache   Amazon   DynamoDB   Am Clou azon   dSea rch   Amazon   EMR   Amazon   RDS   Amazon   Redshic   Request  rate   Cost/GB   Latency   Data  Volume   Low   Low   High   High  
  • 25. cost, volume, access pattern structure
  • 26. Recovering from Disasters
  • 27. Backup  and   restore   Pilot  light   Warm   standby   Hot   standby  
  • 28. cost speed of recovery
  • 29. Backup the DB
  • 30. Backup Server Images
  • 31. Restore DB and App Servers Create instances from AMIs Restore data from backups
  • 32. Summary
  • 33. Next Generation Enterprise Storage   Customer   Data   Center   Block   File   AWS  Direct   Connect   AWS   Cloud   Archive   Backup   Internet  Web   Services  API   HTTP(S)   Disaster  Recovery   Storage  Use   Cases  
  • 34. Tradi8onal:  On-­‐Premise  Solu8ons   Primary   Block  Storage   Primary  File   Storage   Archival   Storage   Disk  Based   Backup   Storage   Tape   Infrastructure  &   Management   Replicated   Storage  for   Disaster   Recovery   Offsite   LocaDons   Geo-­‐ Resilience   Next  Genera8on:  Gateway  &  Cloud  Storage   Reduce  SAN   Footprint   Reduce  NAS   Footprint   Eliminate   Archival   Hardware  &   Socware   Reduce  Complexity.   Eliminate   Backup   Hardware  &   Socware   Eliminate  Tape   Recover  in   Cloud  or  any   Data  Center   Reduce  Cost.   Eliminate   Secondary   Data  Centers  
  • 35. “AWS  allows  us  to  store  informaDon  in  a  cost  effecDve  manner.  It   really  is  a  win-­‐win  for  us  and  our  customers.”   -­‐  Jeff  Kimsey,     Associate  Vice  President   The  Story   •  The  Global  Data  Products  division  of  NASDAQ   Business  Benefit     •  $50  the  first  month   OMX  uses  detailed  historical  stock  price  data   for  its  Market  Replay  and  Data-­‐On-­‐Demand   AWS  is  a  low  cost  soluDon  that  only  cost     •  Avoided  the  need  to  buy  expensive  hardware   products   •  and  improved  product  Dme     Wanted  to  support  their  growing  customer   to  market   base  without  worrying  about  infrastructure,   •  informaDon  to  the  millisecond   security,  or  performance   •  Needed  to  be  able  to  increase  storage  space   Can  store  ten  years  of  historical  stock  price   •  Able  to  upload  hundreds  of  thousands  of  data   and  keep  data  indefinitely   •  files  to  the  AWS  cloud  and  deliver  informaDon  to   Using  Amazon  S3     customers  in  seconds  
  • 36. data abandoned information actionable
  • 37. Business  Value  From  The  Data   Managed  DBs   Amazon   DynamoDB   Delivery   Amazon  CloudFront       Transcoding   Amazon   ElasDc   Transcoder   Workflow   Amazon  SWF   AnalyDcs   Amazon  ElasDc   MapReduce   Discovery   (OR)   Glacier   Amazon   Cloudsearch   DR,  Archive,  Backup  and   Compute   Amazon  VPC   Amazon  EC2   Amazon  EBS  
  • 38. Benefits of AWS Storage Financial –  –  –  –  Significant reduction in capital investments Pay for only what you use: Metered Utility Eliminate the need to over purchase AWS prices continue to fall over time Operational –  Streamline and simplify data management operations –  Reduction of systems and data centers –  Focus employees on proactive and innovative tasks Business –  Never stop the business as capacity is endless –  Never slow the pace of innovation –  Turn IT into an innovation enabler
  • 39. Contact Your AWS Account Manager To discuss your use cases, opportunities to try AWS storage services And follow us on at @AWSCloudSEAsia
  • 40. Demo
  • 41. Thank  you   Markku  Lepistö  –  Senior  Technology  Evangelist   @markkulepisto