AWS Sydney Summit 2013 - Technical Lessons on How to do DR in the Cloud


Published on

Session 3, Presentation 2 from the AWS Sydney Summit

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • This presentation covers the first 2 points, not disaster recovery.
  • AWS Sydney Summit 2013 - Technical Lessons on How to do DR in the Cloud

    1. 1. Dean SamuelsTechnical lessons on how to doBackup and Disaster Recovery in the CloudSolution Architect, Amazon Web Services
    2. 2. Agenda• Prologue The need for Backup and DR• Customer Story Ausenco Limited• Lessons Backup and DR with AWS• Conclusion Q&A
    3. 3. The need for Backup & DR
    4. 4. • Archiving is the process of moving data that is no longer actively used toa separate data storage device for long-term retention. Data archives areindexed and have search capabilities so that files and parts of files can beeasily located and retrieved.• A backup or the process of backing up is making copies of data whichmay be used to restore the original after a data loss event. The primarypurpose is to recover data after its loss, be it by data deletion or corruption.The secondary purpose of backups is to recover data from an earlier time.• Disaster recovery (DR) is the process, policies and procedures related topreparing for recovery or continuation of technology infrastructure critical toan organization after a natural or human-induced disaster.Some Definitions
    5. 5. Where is this….Brisbane, QLDSource:
    6. 6. Where is this….Brisbane, QLD - January 13th 2011AusencoSource:
    7. 7. Milton, QLD (Lang Park)Source:
    8. 8. Milton, QLD (Lang Park) – January 13th 2011Source:
    9. 9. Coronation Drive, Milton, QLDSource:
    10. 10. Coronation Drive, Milton, QLD – January 13th 2011Source:
    11. 11. Backup, Disaster Recovery andData Tiering with AWSMike LatskyDirector, IT OperationsAusenco
    12. 12. About AusencoWe provide:Engineering,Procurement,Construction Management andOperational Servicesto the energy, resource, and processindustriesPresentation title | DD Month Year | 12
    13. 13. IT Perspective• Engineers generate large 2D and 3DCAD files• CAD performance demands closenetwork proximity• Drawings are Ausenco’s deliverableand MUST be protected
    14. 14. • ASX200 company• Head office in Brisbane• 29 Offices in 19 countries• 3500 EmployeesAusenco Offices
    15. 15. Our IT Environment• Completely virtualised• Windows• ERP - Oracle SaaS• Email, Lync, SharePoint – MS Office 365 SaaS• Onsite storage• 2 x primary data centres• 1 x secondary data centre
    16. 16. Our Challenge• Backup– Unreliable• Disaster Recovery– Sites were exposed• Local Disk Storage– At capacityPossible SolutionsAn Enterprise Backup Solution?Secondary Data Centres?More Disk Storage?
    17. 17. Our initial approach• Large Integrators• Leading backup providers• Leading archiving providers• Hardware providers• Data centre hosting providers• Leading cloud providers
    18. 18. Paradigm Shift• Why use traditional technologies?• How can we better leverage the AWS cloud?• How do we do more with less?• Address the complete data lifecycle• Flexible, Scalable, Cost EffectiveDisaster Recovery ArchivingBackup
    19. 19. Solution
    20. 20. The Result• Low effort• Incremental rollout• Cloud migration• Efficient, effective usage of bandwidth• Application/database servers• Regionalised deployment
    21. 21. Our Roadmap• Wider deployment• Decommission costly secondary data centre• Migration of production systems to AWS
    22. 22. Thank-you
    23. 23. Backup and DR with AWSBackup & DR Lessons
    24. 24. Business and Technical Drivers….Reduce costsSlash DR budgets by up to 50%Consolidate sitesEliminate the need to run asecondary siteReduce on-premiseEliminate 30%+ of on-premisephysical equipmentRemove agingtechnologiesEliminate tape for backup andarchive
    25. 25. The fundamental economic model…Utility, on-demand datacenterPrimary SiteRoutersFirewallsNetworkApplication LicensesOperating SystemsHypervisorServersSANPrimary StorageBackupArchiveAWSRoutersFirewallsNetworkApplication LicensesOperating SystemsHypervisorServersSANSnapshot StorageBackupArchiveSecondarysite costs
    26. 26. Backup and DR with AWSBackup Lessons
    27. 27. Backup Lessons – My backup should be accessibleSource: the painof physicaldata transfer
    28. 28. AWS Direct ConnectDedicated bandwidth betweenyour site and AWSAmazon Storage GatewayShrink-wrapped gateway for volumesynchronizationAWS Import/ExportPhysical transfer of media into andout of AWSGetting data into the cloud
    29. 29. Simple Storage ServiceHighly scalable object storage1 byte to 5TB in size99.999999999% durabilityElastic Block StoreHigh performance block storage device1GB to 1TB in sizeMount as drives to instances withsnapshot/cloning functionalitiesGlacierLong term object archiveExtremely low cost per gigabyte99.999999999% durabilityStorage OptionsVery fast‘instance’ disksSlow, rare accessFast web objectstorage
    30. 30. Cost($/GB/month)Performance DurabilityAmazon S3 0.095  Amazon Glacier 0.01  AWS Storage Gateway0.095(+ 125/GW) Amazon EBS 0.10  Amazon EBS (PIOPS) 0.125  *AWS Oregon pricingStorage Options
    31. 31. Snapshot oflocal volumesRestorationfrom snapshotsBackup Lessons – My backup should be accessible
    32. 32. VPNPublic / AWS Direct ConnectAWS Import/ExportBackup Lessons – My backup should be accessible
    33. 33. GW-Cached volumesGW-stored volumes“Cool” storage“Cold”Backup Lessons – My backup should be accessible
    34. 34. Backup Lessons – My backup should be accessible
    35. 35. • “Infinite” scale with Amazon S3 and Amazon Glacier• Scale to multiple regions• Seamless• No need to provision• Cost tiers (cheaper at scale)Backup Lessons – My backup should be able to scale
    36. 36. • SSL Endpoints (Amazon S3 and Amazon Glacier)• Signed API calls• Store encrypted files• Server-side encryption• Multiple copies across different data centers• Local/cloud with AWS Storage GatewayBackup Lessons – My backup should be safe
    37. 37. • Easy to integrate within AWS or Hybrid• AWS Storage Gateway: Run services on Amazon EC2 (DR)• Clear costs• Reduced costs• I decide redundancy/availability in relation to costsBackup Lessons – My backup should work with a DR policy
    38. 38. Backup Lessons – My backup should work with a DR policy
    39. 39. • Clear ownership• Permissions with IAM: Users, groups  roles• Logs• AWS supportBackup Lessons – Someone should care about it
    40. 40. 1. My backup should be accessible1. My backup should be able to scale1. My backup should be safe2. My backup should work with a DR policy3. Someone should care about itBackup Lessons
    41. 41. Backup and DR with AWSDR Lessons
    42. 42. DR is part of a wider set of policies and controls…DR & business continuityIt’s not an all or nothing thingChoose what needs to failover and what does notSome things more important than othersSome things will still be workingHigh availability Backup Disaster recoveryKeep your applicationsrunning 24x7Make sure your data is safe Get your applications anddata back after a majordisaster
    43. 43. Each set of IT assets will have different requirements…DR & business continuityRecovery TimeObjective (RTO)How quickly you need this asset to berecovered?e.g. 1min? 15min? 1hr? 4hrs? 1day?Recovery PointObjective (RPO)How ‘fresh’ the recovery must be for theasset?e.g. zero data loss, 15mins out of date?
    44. 44. Assets will sit on a spectrum of technical complexity…DR & business continuityRebuild whenrequired fromoffsite backupRun hot-hotconfiguration withauto-failover
    45. 45. DR Lessons – You NEED a DR plan in placeDR with High Availability
    46. 46. App DR with StandbyDR Lessons – You NEED a DR plan in place
    47. 47. DR Lessons – Testing your DR• Dev/test in the cloud is super easy• Spin up capacity only for the test• Regularly test your DR• Cost is minimal• What about data transfer speed?s3cmd ls --recursives3://datasets.elasticmapreduce/ngrams/books/ | awk {print $4;sub(/s3://datasets.elasticmapreduce/, "/array", $4); print $4} |parallel -j0 -N2 --progress/usr/bin/s3cmd --no-progress get{1} {2}Copying 2.4 TBdown from 48 hoursto 9 hours (5xfaster)
    48. 48. DR Lessons – Reducing Costs• Dev/test in the cloud is super easy• Spin up capacity only for the test• Regularly test your DR• Cost is minimal• What about data transfer speed?
    49. 49. DR Lessons – You can have different DR solutions• Easy to integrate existing vendors with DR on AWS• Approach: One vendor/hybrid/multiple vendors• One region/multi-regions (if you need geo-diversity)• Different DR ArchitecturesBackup & Restore Pilot lightWarm standby inAWSMulti-site solutionin AWS & on-premise
    50. 50. Pilot lightStoppedinstances
    51. 51. Pilot lightRunninginstances
    52. 52. 1. You NEED a DR plan in place1. Testing your DR1. Reducing costs2. You can have different DR solutionsDR Lessons
    53. 53. started on the free tier
    54. 54. Technical lessons on how to doBackup and Disaster Recovery in the Cloud