Dean SamuelsTechnical lessons on how to doBackup and Disaster Recovery in the CloudSolution Architect, Amazon Web Services
Agenda• Prologue The need for Backup and DR• Customer Story Ausenco Limited• Lessons Backup and DR with AWS• Conclusion Q&A
The need for Backup & DR
• Archiving is the process of moving data that is no longer actively used toa separate data storage device for long-term r...
Where is this….Brisbane, QLDSource: http://www.abc.net.au/news/specials/qld-floods/
Where is this….Brisbane, QLD - January 13th 2011AusencoSource: http://www.abc.net.au/news/specials/qld-floods/
Milton, QLD (Lang Park)Source: http://www.abc.net.au/news/specials/qld-floods/
Milton, QLD (Lang Park) – January 13th 2011Source: http://www.abc.net.au/news/specials/qld-floods/
Coronation Drive, Milton, QLDSource: http://www.abc.net.au/news/specials/qld-floods/
Coronation Drive, Milton, QLD – January 13th 2011Source: http://www.abc.net.au/news/specials/qld-floods/
Backup, Disaster Recovery andData Tiering with AWSMike LatskyDirector, IT OperationsAusenco
About AusencoWe provide:Engineering,Procurement,Construction Management andOperational Servicesto the energy, resource, an...
IT Perspective• Engineers generate large 2D and 3DCAD files• CAD performance demands closenetwork proximity• Drawings are ...
• ASX200 company• Head office in Brisbane• 29 Offices in 19 countries• 3500 EmployeesAusenco Offices
Our IT Environment• Completely virtualised• Windows• ERP - Oracle SaaS• Email, Lync, SharePoint – MS Office 365 SaaS• Onsi...
Our Challenge• Backup– Unreliable• Disaster Recovery– Sites were exposed• Local Disk Storage– At capacityPossible Solution...
Our initial approach• Large Integrators• Leading backup providers• Leading archiving providers• Hardware providers• Data c...
Paradigm Shift• Why use traditional technologies?• How can we better leverage the AWS cloud?• How do we do more with less?...
Solution
The Result• Low effort• Incremental rollout• Cloud migration• Efficient, effective usage of bandwidth• Application/databas...
Our Roadmap• Wider deployment• Decommission costly secondary data centre• Migration of production systems to AWS
Thank-you
Backup and DR with AWSBackup & DR Lessons
Business and Technical Drivers….Reduce costsSlash DR budgets by up to 50%Consolidate sitesEliminate the need to run asecon...
The fundamental economic model…Utility, on-demand datacenterPrimary SiteRoutersFirewallsNetworkApplication LicensesOperati...
Backup and DR with AWSBackup Lessons
Backup Lessons – My backup should be accessibleSource: http://www.abc.net.au/news/specials/qld-floods/a.k.a. the painof ph...
AWS Direct ConnectDedicated bandwidth betweenyour site and AWSAmazon Storage GatewayShrink-wrapped gateway for volumesynch...
Simple Storage ServiceHighly scalable object storage1 byte to 5TB in size99.999999999% durabilityElastic Block StoreHigh p...
Cost($/GB/month)Performance DurabilityAmazon S3 0.095  Amazon Glacier 0.01  AWS Storage Gateway0.095(+ 125/G...
Snapshot oflocal volumesRestorationfrom snapshotsBackup Lessons – My backup should be accessible
VPNPublic / AWS Direct ConnectAWS Import/ExportBackup Lessons – My backup should be accessible
GW-Cached volumesGW-stored volumes“Cool” storage“Cold”Backup Lessons – My backup should be accessible
Backup Lessons – My backup should be accessible
• “Infinite” scale with Amazon S3 and Amazon Glacier• Scale to multiple regions• Seamless• No need to provision• Cost tier...
• SSL Endpoints (Amazon S3 and Amazon Glacier)• Signed API calls• Store encrypted files• Server-side encryption• Multiple ...
• Easy to integrate within AWS or Hybrid• AWS Storage Gateway: Run services on Amazon EC2 (DR)• Clear costs• Reduced costs...
Backup Lessons – My backup should work with a DR policy
• Clear ownership• Permissions with IAM: Users, groups  roles• Logs• AWS supportBackup Lessons – Someone should care abou...
1. My backup should be accessible1. My backup should be able to scale1. My backup should be safe2. My backup should work w...
Backup and DR with AWSDR Lessons
DR is part of a wider set of policies and controls…DR & business continuityIt’s not an all or nothing thingChoose what nee...
Each set of IT assets will have different requirements…DR & business continuityRecovery TimeObjective (RTO)How quickly you...
Assets will sit on a spectrum of technical complexity…DR & business continuityRebuild whenrequired fromoffsite backupRun h...
DR Lessons – You NEED a DR plan in placeDR with High Availability
App DR with StandbyDR Lessons – You NEED a DR plan in place
DR Lessons – Testing your DR• Dev/test in the cloud is super easy• Spin up capacity only for the test• Regularly test your...
DR Lessons – Reducing Costs• Dev/test in the cloud is super easy• Spin up capacity only for the test• Regularly test your ...
DR Lessons – You can have different DR solutions• Easy to integrate existing vendors with DR on AWS• Approach: One vendor/...
Pilot lightStoppedinstances
Pilot lightRunninginstances
1. You NEED a DR plan in place1. Testing your DR1. Reducing costs2. You can have different DR solutionsDR Lessons
aws.amazon.comget started on the free tier
Technical lessons on how to doBackup and Disaster Recovery in the Cloud
Upcoming SlideShare
Loading in …5
×

AWS Sydney Summit 2013 - Technical Lessons on How to do DR in the Cloud

2,139 views
1,969 views

Published on

Session 3, Presentation 2 from the AWS Sydney Summit

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,139
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
38
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • This presentation covers the first 2 points, not disaster recovery.
  • AWS Sydney Summit 2013 - Technical Lessons on How to do DR in the Cloud

    1. 1. Dean SamuelsTechnical lessons on how to doBackup and Disaster Recovery in the CloudSolution Architect, Amazon Web Services
    2. 2. Agenda• Prologue The need for Backup and DR• Customer Story Ausenco Limited• Lessons Backup and DR with AWS• Conclusion Q&A
    3. 3. The need for Backup & DR
    4. 4. • Archiving is the process of moving data that is no longer actively used toa separate data storage device for long-term retention. Data archives areindexed and have search capabilities so that files and parts of files can beeasily located and retrieved.• A backup or the process of backing up is making copies of data whichmay be used to restore the original after a data loss event. The primarypurpose is to recover data after its loss, be it by data deletion or corruption.The secondary purpose of backups is to recover data from an earlier time.• Disaster recovery (DR) is the process, policies and procedures related topreparing for recovery or continuation of technology infrastructure critical toan organization after a natural or human-induced disaster.Some Definitions
    5. 5. Where is this….Brisbane, QLDSource: http://www.abc.net.au/news/specials/qld-floods/
    6. 6. Where is this….Brisbane, QLD - January 13th 2011AusencoSource: http://www.abc.net.au/news/specials/qld-floods/
    7. 7. Milton, QLD (Lang Park)Source: http://www.abc.net.au/news/specials/qld-floods/
    8. 8. Milton, QLD (Lang Park) – January 13th 2011Source: http://www.abc.net.au/news/specials/qld-floods/
    9. 9. Coronation Drive, Milton, QLDSource: http://www.abc.net.au/news/specials/qld-floods/
    10. 10. Coronation Drive, Milton, QLD – January 13th 2011Source: http://www.abc.net.au/news/specials/qld-floods/
    11. 11. Backup, Disaster Recovery andData Tiering with AWSMike LatskyDirector, IT OperationsAusenco
    12. 12. About AusencoWe provide:Engineering,Procurement,Construction Management andOperational Servicesto the energy, resource, and processindustriesPresentation title | DD Month Year | 12
    13. 13. IT Perspective• Engineers generate large 2D and 3DCAD files• CAD performance demands closenetwork proximity• Drawings are Ausenco’s deliverableand MUST be protected
    14. 14. • ASX200 company• Head office in Brisbane• 29 Offices in 19 countries• 3500 EmployeesAusenco Offices
    15. 15. Our IT Environment• Completely virtualised• Windows• ERP - Oracle SaaS• Email, Lync, SharePoint – MS Office 365 SaaS• Onsite storage• 2 x primary data centres• 1 x secondary data centre
    16. 16. Our Challenge• Backup– Unreliable• Disaster Recovery– Sites were exposed• Local Disk Storage– At capacityPossible SolutionsAn Enterprise Backup Solution?Secondary Data Centres?More Disk Storage?
    17. 17. Our initial approach• Large Integrators• Leading backup providers• Leading archiving providers• Hardware providers• Data centre hosting providers• Leading cloud providers
    18. 18. Paradigm Shift• Why use traditional technologies?• How can we better leverage the AWS cloud?• How do we do more with less?• Address the complete data lifecycle• Flexible, Scalable, Cost EffectiveDisaster Recovery ArchivingBackup
    19. 19. Solution
    20. 20. The Result• Low effort• Incremental rollout• Cloud migration• Efficient, effective usage of bandwidth• Application/database servers• Regionalised deployment
    21. 21. Our Roadmap• Wider deployment• Decommission costly secondary data centre• Migration of production systems to AWS
    22. 22. Thank-you
    23. 23. Backup and DR with AWSBackup & DR Lessons
    24. 24. Business and Technical Drivers….Reduce costsSlash DR budgets by up to 50%Consolidate sitesEliminate the need to run asecondary siteReduce on-premiseEliminate 30%+ of on-premisephysical equipmentRemove agingtechnologiesEliminate tape for backup andarchive
    25. 25. The fundamental economic model…Utility, on-demand datacenterPrimary SiteRoutersFirewallsNetworkApplication LicensesOperating SystemsHypervisorServersSANPrimary StorageBackupArchiveAWSRoutersFirewallsNetworkApplication LicensesOperating SystemsHypervisorServersSANSnapshot StorageBackupArchiveSecondarysite costs
    26. 26. Backup and DR with AWSBackup Lessons
    27. 27. Backup Lessons – My backup should be accessibleSource: http://www.abc.net.au/news/specials/qld-floods/a.k.a. the painof physicaldata transfer
    28. 28. AWS Direct ConnectDedicated bandwidth betweenyour site and AWSAmazon Storage GatewayShrink-wrapped gateway for volumesynchronizationAWS Import/ExportPhysical transfer of media into andout of AWSGetting data into the cloud
    29. 29. Simple Storage ServiceHighly scalable object storage1 byte to 5TB in size99.999999999% durabilityElastic Block StoreHigh performance block storage device1GB to 1TB in sizeMount as drives to instances withsnapshot/cloning functionalitiesGlacierLong term object archiveExtremely low cost per gigabyte99.999999999% durabilityStorage OptionsVery fast‘instance’ disksSlow, rare accessFast web objectstorage
    30. 30. Cost($/GB/month)Performance DurabilityAmazon S3 0.095  Amazon Glacier 0.01  AWS Storage Gateway0.095(+ 125/GW) Amazon EBS 0.10  Amazon EBS (PIOPS) 0.125  *AWS Oregon pricingStorage Options
    31. 31. Snapshot oflocal volumesRestorationfrom snapshotsBackup Lessons – My backup should be accessible
    32. 32. VPNPublic / AWS Direct ConnectAWS Import/ExportBackup Lessons – My backup should be accessible
    33. 33. GW-Cached volumesGW-stored volumes“Cool” storage“Cold”Backup Lessons – My backup should be accessible
    34. 34. Backup Lessons – My backup should be accessible
    35. 35. • “Infinite” scale with Amazon S3 and Amazon Glacier• Scale to multiple regions• Seamless• No need to provision• Cost tiers (cheaper at scale)Backup Lessons – My backup should be able to scale
    36. 36. • SSL Endpoints (Amazon S3 and Amazon Glacier)• Signed API calls• Store encrypted files• Server-side encryption• Multiple copies across different data centers• Local/cloud with AWS Storage GatewayBackup Lessons – My backup should be safe
    37. 37. • Easy to integrate within AWS or Hybrid• AWS Storage Gateway: Run services on Amazon EC2 (DR)• Clear costs• Reduced costs• I decide redundancy/availability in relation to costsBackup Lessons – My backup should work with a DR policy
    38. 38. Backup Lessons – My backup should work with a DR policy
    39. 39. • Clear ownership• Permissions with IAM: Users, groups  roles• Logs• AWS supportBackup Lessons – Someone should care about it
    40. 40. 1. My backup should be accessible1. My backup should be able to scale1. My backup should be safe2. My backup should work with a DR policy3. Someone should care about itBackup Lessons
    41. 41. Backup and DR with AWSDR Lessons
    42. 42. DR is part of a wider set of policies and controls…DR & business continuityIt’s not an all or nothing thingChoose what needs to failover and what does notSome things more important than othersSome things will still be workingHigh availability Backup Disaster recoveryKeep your applicationsrunning 24x7Make sure your data is safe Get your applications anddata back after a majordisaster
    43. 43. Each set of IT assets will have different requirements…DR & business continuityRecovery TimeObjective (RTO)How quickly you need this asset to berecovered?e.g. 1min? 15min? 1hr? 4hrs? 1day?Recovery PointObjective (RPO)How ‘fresh’ the recovery must be for theasset?e.g. zero data loss, 15mins out of date?
    44. 44. Assets will sit on a spectrum of technical complexity…DR & business continuityRebuild whenrequired fromoffsite backupRun hot-hotconfiguration withauto-failover
    45. 45. DR Lessons – You NEED a DR plan in placeDR with High Availability
    46. 46. App DR with StandbyDR Lessons – You NEED a DR plan in place
    47. 47. DR Lessons – Testing your DR• Dev/test in the cloud is super easy• Spin up capacity only for the test• Regularly test your DR• Cost is minimal• What about data transfer speed?s3cmd ls --recursives3://datasets.elasticmapreduce/ngrams/books/ | awk {print $4;sub(/s3://datasets.elasticmapreduce/, "/array", $4); print $4} |parallel -j0 -N2 --progress/usr/bin/s3cmd --no-progress get{1} {2}Copying 2.4 TBdown from 48 hoursto 9 hours (5xfaster)
    48. 48. DR Lessons – Reducing Costs• Dev/test in the cloud is super easy• Spin up capacity only for the test• Regularly test your DR• Cost is minimal• What about data transfer speed?
    49. 49. DR Lessons – You can have different DR solutions• Easy to integrate existing vendors with DR on AWS• Approach: One vendor/hybrid/multiple vendors• One region/multi-regions (if you need geo-diversity)• Different DR ArchitecturesBackup & Restore Pilot lightWarm standby inAWSMulti-site solutionin AWS & on-premise
    50. 50. Pilot lightStoppedinstances
    51. 51. Pilot lightRunninginstances
    52. 52. 1. You NEED a DR plan in place1. Testing your DR1. Reducing costs2. You can have different DR solutionsDR Lessons
    53. 53. aws.amazon.comget started on the free tier
    54. 54. Technical lessons on how to doBackup and Disaster Recovery in the Cloud

    ×