Alagappan M
Account Manager & AWS Certified
Solutions Architect – Associate Level
Disaster Recovery, Backup and Archive in...
Agenda
I. Prologue
I. Enterprise Ready
II. Storage Options
III. Lessons
II. Earthquake
III. Conclusions
The story of Monte...
Prologue
Part I
44
"The mind is not a vessel to be filled,
but a fire to be ignited."
- Plutarch
5
Abbey of
Monte
Cassino
66
Why is Monte Cassino important? ][
7
The Treasure of Monte Cassino ][
800 papal documents
20,500 volumes in the Old Library
60,000 in the New Library
200 man...
8
World War II ][
99
Dec 1942: Many “treasures”
are transported from Rome
and other places to Monte
Cassino, for safety
The Treasure of Mont...
110
Abbey of Monte Cassino ][
111
The Treasure of Monte Cassino ][
Feb 1944: Schlegel and Becker
(Panzer-Division Hermann
Göring) had the treasures
tran...
112
Escape from Monte Cassino ][
113
Monte Cassino after bombing (1944) ][
114
Restoration in 1954 ][
115
The Abbey of Monte Cassino today ][
End of Prologue
Enterprise Storage
1
Regions (8) GovCloud Regions (1)
(as of Nov 27th, 2012)
Global AWS Infrastructure ][
1
Availability Zones (23)
(as of Nov 27th, 2012)
Global AWS Infrastructure ][
2
Dallas (2)
St.Louis
Miami
Jacksonvill
e
Los Angeles (2)
Palo Alto
Seattle
Ashburn (2)
Newark
New York (2)
Dublin
London
...
• $5.2B retail business
• 7,800 employees
• A whole lot of servers
Every day, AWS adds enough
server capacity to power tha...
Solving Problems for Organizations Around the World
Gartner “Magic Quadrant for Cloud Infrastructurea Service,” Lydia Leong, Douglas Toombs, Bob Gill, Gregor Petri, Tiny Hayn...
“Our models reveal a
significant cost
difference, with the
cloud-based model
coming in 74% less
expensive that I&O
running...
4X More Reliable & 1/4 the Cost of On-Premises Infrastructure
In early 2012, AWS commissioned IDC to interview 11 organiza...
Architected for Enterprise Security
Requirements
“The Amazon Virtual Private
Cloud [Amazon VPC] was a
unique option that o...
Storage Options
AWS Direct Connect
Dedicated bandwidth between
your site and AWS
Amazon Storage Gateway
Shrink-wrapped gateway for volume
...
Simple Storage Service
Highly scalable object storage
1 byte to 5TB in size
99.999999999% durability
Elastic Block Store
H...
Storage Gateway – Connect On-Prem with the AWS Cloud
1. Local, low-latency access to the
most frequently used files while
...
Cost
($/GB/month)
Performance Durability
Amazon S3 0.095  
Amazon Glacier 0.01  
AWS Storage
Gateway
0.095
(...
500,000 writes per second during Super Bowl
“Spotify needed a storage solution that
could scale very quickly without incurring
long lead times for upgrades. This led ...
Nasdaq FinQloud is a WORM compliant archive for
Broker Dealer financial data
Forced to choose between saving their source data or saving
their results. With Amazon Glacier, they can keep both
3
High Availability :
Keeping services alive.
Business continuity continuum ][
3
High Availability :
Keeping services alive.
Backing up :
Process of copying and archiving of data so it may be used to
r...
3
High Availability :
Keeping services alive.
Backing up :
Process of copying and archiving of data so it may be used to
r...
DR is part of a wider set of policies and controls…
DR & business continuity
It’s not an all or nothing thing
Choose what ...
DR & business continuity
Assets will sit on a spectrum of technical complexity…
Rebuild when
required from
offsite backup
...
Lessons from Monte Cassino
Part II
1. My backup should be accessible
2. My backup should be able to scale
3. My backup should be safe
4. My backup should wor...
444
1. My backup should be accessible
a.k.a. the pain of
physical data transfer
z
VPN
Public / AWS Direct Connect
AWS Import/Export
Backup Rules – My backup should be accessible
447
2. My backup should be able to scale
• “Infinite” scale with Amazon S3 and Amazon Glacier
• Scale to multiple regions
• Seamless
• No need to provision
• Cost ...
GW-Cached volumes
GW-stored volumes
“Warm” storage
“Cold”
Backup Rules – My backup should be able to scale
3. My backup should be safe
• SSL Endpoints (Amazon S3 and Amazon Glacier)
• Signed API calls
• Store encrypted files
• Server-side encryption
• Multi...
552
4. My backup should work with a DR policy
(I don’t want to wait 10 years… )
5
Lessons from Monte Cassino ][
4. My backup should work with a DR policy
• Easy to integrate within AWS or Hybrid
• AWS S...
• Clear ownership
• Permissions with IAM: Users, groups  roles
• Logs
• AWS support
5. Backup Rules – Someone should care...
• Washington Trust Bank (WTB) is one of
the largest privately held banks in the
US with 40 offices and 750 employees
• AWS...
The Earthquake
Part II
The Earthquake
Christchurch Earthquake - Feb 22nd 2011Manchester & Glouchester Street, Christchurch....
Photos: http://www...
Some “natural” examples….
after Brisbane Floods – January 13th 2011Coronation Drive, Milton, QLD.…
Photos: http://www.abc....
Some “natural” examples….
after Hurricane Sandy – October 29th 2012Breezy Point, Queens, NY, USA.…
Photos: http://www.abc....
What about “human-made” examples….
“Everything fails, all the time”
Werner Vogels, CTO, Amazon.com
Lessons from an Earthquake
Part V
1. You NEED a DR plan in place
2. Testing your DR
3. Reducing costs
4. You can have different DR solutions
DR Lessons
6
Le...
DR Lessons – You NEED a DR plan in place
DR with High Availability
App DR with Standby
DR Lessons – You NEED a DR plan in place
666
2. Testing your DR
DR Lessons – Testing your DR
• Dev/test in the cloud is super easy
• Spin up capacity only for the test
• Regularly test y...
668
3. Reducing costs
DR Lessons – Reducing Costs
• AWS cost reduction (e.g, S3 cost
reduction on Apr 2nd 2013)
• Reduced redundancy (Amazon S3)...
• Prominent Canadian TV broadcaster
whose portfolio of conventional
networks, specialty channels, multi-
media websites, o...
771
4. You can have different DR solutions
7
Lessons from an Earthquake ][
1. Backup and Restore
2. “Pilot light” for quick recovery into AWS (Cold standby)
3. Warm ...
7
1. Easy to integrate existing vendors with DR on AWS
2. Approach: One vendor/hybrid/multiple vendors
3. One region/multi...
1. You NEED a DR plan in place
2. Testing your DR
3. Reducing costs
4. You can have different DR solutions
DR Lessons
7
Le...
Oracle RMAN to S3
Backup Oracle databases to S3
Improve speed, flexibility and costs.
Oracle RMAN to S3
Backing up Oracle ...
The number of Oracle databases that
amazon.com were maintain we
causing challenges
- Utilization and capacity planning is
...
Archival
Part V
Glacier allows you to cost-effectively and securely store
enterprise data offsite, making it simple, inexpensive and
safe ...
Glacier allows you to cost-effectively and securely store
enterprise data offsite, making it simple, inexpensive and safe
...
Amazon Glacier is cost competitive, even at scale, and
eliminates pain points like capacity planning, capital
budgeting an...
“An organization like ours thinks in centuries
when it comes to content retention, and long
term preservation of our Maste...
Backups
Disaster Recovery
Archival
Action items
Agility
Cost savings
Control
x
aws.amazon.com
get started on the free tier
DR, Backup and Archival
Q&A
AWS Summit 2013 | India - Disaster Recovery, Backup and Archive in the Cloud, Alagappan M
AWS Summit 2013 | India - Disaster Recovery, Backup and Archive in the Cloud, Alagappan M
AWS Summit 2013 | India - Disaster Recovery, Backup and Archive in the Cloud, Alagappan M
Upcoming SlideShare
Loading in …5
×

AWS Summit 2013 | India - Disaster Recovery, Backup and Archive in the Cloud, Alagappan M

998 views

Published on

Think that cloud storage is not enterprise-class? AWS provides multiple storage options for a wide range of use cases, performance and cost levels. This presentation provides an overview of how AWS cloud storage services can be used to support application development and delivery along with use cases for backup, archive and disaster recovery.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
998
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

AWS Summit 2013 | India - Disaster Recovery, Backup and Archive in the Cloud, Alagappan M

  1. 1. Alagappan M Account Manager & AWS Certified Solutions Architect – Associate Level Disaster Recovery, Backup and Archive in the Cloud
  2. 2. Agenda I. Prologue I. Enterprise Ready II. Storage Options III. Lessons II. Earthquake III. Conclusions The story of Monte Cassino Next Generation Storage Solution And accessibility Why is Monte Cassino important? Disaster Recovery and Archival ... And next steps !
  3. 3. Prologue Part I
  4. 4. 44 "The mind is not a vessel to be filled, but a fire to be ignited." - Plutarch
  5. 5. 5 Abbey of Monte Cassino
  6. 6. 66 Why is Monte Cassino important? ][
  7. 7. 7 The Treasure of Monte Cassino ][ 800 papal documents 20,500 volumes in the Old Library 60,000 in the New Library 200 manuscripts on parchment 100,000 prints and paintings (including 11 Titians) 500 incunabula A book printed before 1501 C.E. Gutenberg’s Bible was printed in 1455 C.E. Titian, one of the most influential painters ever x
  8. 8. 8 World War II ][
  9. 9. 99 Dec 1942: Many “treasures” are transported from Rome and other places to Monte Cassino, for safety The Treasure of Monte Cassino ][
  10. 10. 110 Abbey of Monte Cassino ][
  11. 11. 111 The Treasure of Monte Cassino ][ Feb 1944: Schlegel and Becker (Panzer-Division Hermann Göring) had the treasures transferred to the Vatican x
  12. 12. 112 Escape from Monte Cassino ][
  13. 13. 113 Monte Cassino after bombing (1944) ][
  14. 14. 114 Restoration in 1954 ][
  15. 15. 115 The Abbey of Monte Cassino today ][
  16. 16. End of Prologue
  17. 17. Enterprise Storage
  18. 18. 1 Regions (8) GovCloud Regions (1) (as of Nov 27th, 2012) Global AWS Infrastructure ][
  19. 19. 1 Availability Zones (23) (as of Nov 27th, 2012) Global AWS Infrastructure ][
  20. 20. 2 Dallas (2) St.Louis Miami Jacksonvill e Los Angeles (2) Palo Alto Seattle Ashburn (2) Newark New York (2) Dublin London Amsterdam (2) Stockholm Frankfurt (2)Paris Singapore (2) Hong Kong Tokyo São Paulo South Bend San Jose Osaka Milan Sydney Madrid (as of Nov 27th, 2012) Global AWS Infrastructure ][ Edge Locations (40)
  21. 21. • $5.2B retail business • 7,800 employees • A whole lot of servers Every day, AWS adds enough server capacity to power that whole $5B enterprise
  22. 22. Solving Problems for Organizations Around the World
  23. 23. Gartner “Magic Quadrant for Cloud Infrastructurea Service,” Lydia Leong, Douglas Toombs, Bob Gill, Gregor Petri, Tiny Hayn, October 18, 2012. This Magic Quadrant graphic was published by Gartner, Inc. as part of a larger research note and should be evaluated in the context of the entire report.. The Gartner report is availableupon request from Steven Armstrong (asteven@amazon.com). Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings. Gartnerresearch publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied,with respect to this research, includingany warranties of merchantability or fitness for a particularpurpose. Gartner Magic Quadrant for Cloud Infrastructure as a Service
  24. 24. “Our models reveal a significant cost difference, with the cloud-based model coming in 74% less expensive that I&O running it in-house” - Forrester Research Report
  25. 25. 4X More Reliable & 1/4 the Cost of On-Premises Infrastructure In early 2012, AWS commissioned IDC to interview 11 organizations that deployed applications on AWS.
  26. 26. Architected for Enterprise Security Requirements “The Amazon Virtual Private Cloud [Amazon VPC] was a unique option that offered an additional level of security and an ability to integrate with other aspects of our infrastructure.” Dr. Michael Miller, Head of HPC for R&D
  27. 27. Storage Options
  28. 28. AWS Direct Connect Dedicated bandwidth between your site and AWS Amazon Storage Gateway Shrink-wrapped gateway for volume synchronization AWS Import/Export Physical transfer of media into and out of AWS Getting data into the cloud
  29. 29. Simple Storage Service Highly scalable object storage 1 byte to 5TB in size 99.999999999% durability Elastic Block Store High performance block storage device 1GB to 1TB in size Mount as drives to instances with snapshot/cloning functionalities Glacier Long term object archive Extremely low cost per gigabyte 99.999999999% durability Storage Options Very fast ‘instance’ disks Slow, rare accessFast web object storage
  30. 30. Storage Gateway – Connect On-Prem with the AWS Cloud 1. Local, low-latency access to the most frequently used files while storing all data in Amazon S3 (Cached-Volumes) Or 2. Scheduled off-site backups to Amazon S3 for on-premises data (Stored-Volumes) Storage Gateway
  31. 31. Cost ($/GB/month) Performance Durability Amazon S3 0.095   Amazon Glacier 0.01   AWS Storage Gateway 0.095 (+ 125/GW)   Amazon EBS 0.10   Amazon EBS (PIOPS) 0.125   *AWS Oregon pricing Storage Options
  32. 32. 500,000 writes per second during Super Bowl
  33. 33. “Spotify needed a storage solution that could scale very quickly without incurring long lead times for upgrades. This led us to cloud storage, and in that market, Amazon Simple Storage Service (Amazon S3) is the most mature large-scale product. Amazon S3 gives us confidence in our ability to expand storage quickly while also providing high data durability.” Emil Fredriksson, Operations Director
  34. 34. Nasdaq FinQloud is a WORM compliant archive for Broker Dealer financial data
  35. 35. Forced to choose between saving their source data or saving their results. With Amazon Glacier, they can keep both
  36. 36. 3 High Availability : Keeping services alive. Business continuity continuum ][
  37. 37. 3 High Availability : Keeping services alive. Backing up : Process of copying and archiving of data so it may be used to restore the original after a data loss event Business continuity continuum ][
  38. 38. 3 High Availability : Keeping services alive. Backing up : Process of copying and archiving of data so it may be used to restore the original after a data loss event. Disaster recovery : Recovery of technology infrastructure critical to an organization after a natural or human-induced disaster. Business continuity continuum ][
  39. 39. DR is part of a wider set of policies and controls… DR & business continuity It’s not an all or nothing thing Choose what needs to failover and what does not Some things more important than others Some things will still be working High availability Backup Disaster recovery Keep your applications running 24x7 Make sure your data is safe Get your applications and data back after a major disaster
  40. 40. DR & business continuity Assets will sit on a spectrum of technical complexity… Rebuild when required from offsite backup Run hot-hot configuration with auto-failover Recovery Time Objective (RTO) How quickly you need this asset to be recovered? e.g. 1min? 15min? 1hr? 4hrs? 1day? Recovery Point Objective (RPO) How ‘fresh’ the recovery must be for the asset? e.g. zero data loss, 15mins out of date?
  41. 41. Lessons from Monte Cassino Part II
  42. 42. 1. My backup should be accessible 2. My backup should be able to scale 3. My backup should be safe 4. My backup should work with a DR policy 5. Someone should care about it Lessons from Monte Cassino
  43. 43. 444 1. My backup should be accessible a.k.a. the pain of physical data transfer
  44. 44. z
  45. 45. VPN Public / AWS Direct Connect AWS Import/Export Backup Rules – My backup should be accessible
  46. 46. 447 2. My backup should be able to scale
  47. 47. • “Infinite” scale with Amazon S3 and Amazon Glacier • Scale to multiple regions • Seamless • No need to provision • Cost tiers (cheaper at scale) Backup Rules – My backup should be able to scale
  48. 48. GW-Cached volumes GW-stored volumes “Warm” storage “Cold” Backup Rules – My backup should be able to scale
  49. 49. 3. My backup should be safe
  50. 50. • SSL Endpoints (Amazon S3 and Amazon Glacier) • Signed API calls • Store encrypted files • Server-side encryption • Multiple copies across different data centers • Local/cloud with AWS Storage Gateway Backup Rules – My backup should be safe
  51. 51. 552 4. My backup should work with a DR policy (I don’t want to wait 10 years… )
  52. 52. 5 Lessons from Monte Cassino ][ 4. My backup should work with a DR policy • Easy to integrate within AWS or Hybrid • AWS Storage Gateway: Run services on Amazon EC2 (DR) • Clear costs • Reduced costs • I decide redundancy/availability in relation to costs
  53. 53. • Clear ownership • Permissions with IAM: Users, groups  roles • Logs • AWS support 5. Backup Rules – Someone should care about it
  54. 54. • Washington Trust Bank (WTB) is one of the largest privately held banks in the US with 40 offices and 750 employees • AWS Advanced Consulting Provider IT Lifeline helped WTB institute a compliant, secure solution to protect customer data and cut disaster recovery costs • Washington Trust Bank cut data recovery costs by $36,000 a year by migrating to AWS Cloud With AWS, Washington Trust Bank Cuts Cost by $36,000 a Year
  55. 55. The Earthquake Part II
  56. 56. The Earthquake Christchurch Earthquake - Feb 22nd 2011Manchester & Glouchester Street, Christchurch.... Photos: http://www.abc.net.au/news/specials/christchurch-quake/
  57. 57. Some “natural” examples…. after Brisbane Floods – January 13th 2011Coronation Drive, Milton, QLD.… Photos: http://www.abc.net.au/news/specials/qld-floods/
  58. 58. Some “natural” examples…. after Hurricane Sandy – October 29th 2012Breezy Point, Queens, NY, USA.… Photos: http://www.abc.net.au/news/specials/hurricane-sandy-before-after-photos/
  59. 59. What about “human-made” examples…. “Everything fails, all the time” Werner Vogels, CTO, Amazon.com
  60. 60. Lessons from an Earthquake Part V
  61. 61. 1. You NEED a DR plan in place 2. Testing your DR 3. Reducing costs 4. You can have different DR solutions DR Lessons 6 Lessons from an Earthquake ][
  62. 62. DR Lessons – You NEED a DR plan in place DR with High Availability
  63. 63. App DR with Standby DR Lessons – You NEED a DR plan in place
  64. 64. 666 2. Testing your DR
  65. 65. DR Lessons – Testing your DR • Dev/test in the cloud is super easy • Spin up capacity only for the test • Regularly test your DR • Cost is minimal • What about data transfer speed? s3cmd ls --recursive s3://datasets.elasticmapreduce/ngra ms/books/ | awk '{print $4; sub(/s3://datasets.elasticmapredu ce/, "/array", $4); print $4}' | parallel -j0 -N2 --progress /usr/bin/s3cmd --no-progress get {1} {2} Copying 2.4 TB down from 48 hours to 9 hours (5x faster)
  66. 66. 668 3. Reducing costs
  67. 67. DR Lessons – Reducing Costs • AWS cost reduction (e.g, S3 cost reduction on Apr 2nd 2013) • Reduced redundancy (Amazon S3) • Retention policy • Hot/warm/cool/cold backup • Reserved capacity/tiers Price Reductions
  68. 68. • Prominent Canadian TV broadcaster whose portfolio of conventional networks, specialty channels, multi- media websites, on-demand channels and mobile applications reaches 25 million Canadians weekly • Older technology and infrastructure led to frequent power outages and downtime • With AWS Shaw Media achieved, lower operational costs with the ability to scale quickly to meet business demands Shaw Media Uses AWS to Lower Operational Costs
  69. 69. 771 4. You can have different DR solutions
  70. 70. 7 Lessons from an Earthquake ][ 1. Backup and Restore 2. “Pilot light” for quick recovery into AWS (Cold standby) 3. Warm standby solution on AWS 4. Multi-site hybrid solution (AWS + on premises) Different Types of DR Architecture
  71. 71. 7 1. Easy to integrate existing vendors with DR on AWS 2. Approach: One vendor/hybrid/multiple vendors 3. One region/multi-regions (if you need geodiversity) 4. You can have different DR solutions Lessons from an Earthquake ][
  72. 72. 1. You NEED a DR plan in place 2. Testing your DR 3. Reducing costs 4. You can have different DR solutions DR Lessons 7 Lessons from an Earthquake ][
  73. 73. Oracle RMAN to S3 Backup Oracle databases to S3 Improve speed, flexibility and costs. Oracle RMAN to S3 Backing up Oracle to the Cloud
  74. 74. The number of Oracle databases that amazon.com were maintain we causing challenges - Utilization and capacity planning is hard - The cost of backup software is high - The reliability and efficiency of tapes is poor By using the Oracle RMAN backup to S3 they reduced restore times from 10-15 hours to 2.5 hours. The also improved reliability and removed capacity planning issues. There were cost savings on hardware, software and IT Operations staff.
  75. 75. Archival Part V
  76. 76. Glacier allows you to cost-effectively and securely store enterprise data offsite, making it simple, inexpensive and safe to retain archived data for as long as desired. Common use cases include enterprise data, media assets, and research and scientific data Offsite archive
  77. 77. Glacier allows you to cost-effectively and securely store enterprise data offsite, making it simple, inexpensive and safe to retain archived data for as long as desired. Common use cases include enterprise data, media assets, and research and scientific data Libraries, historical societies, non-profit organizations and governments are increasing their efforts to preserve valuable but aging digital content such as websites, software source code, video games, user-generated content and other digital artifacts Offsite archive Digital preservation
  78. 78. Amazon Glacier is cost competitive, even at scale, and eliminates pain points like capacity planning, capital budgeting and investments, media formats, hardware refreshes, and off-site storage costs, shipping and retrieving Glacier allows you to cost-effectively and securely store enterprise data offsite, making it simple, inexpensive and safe to retain archived data for as long as desired. Common use cases include enterprise data, media assets, and research and scientific data Libraries, historical societies, non-profit organizations and governments are increasing their efforts to preserve valuable but aging digital content such as websites, software source code, video games, user-generated content and other digital artifacts Offsite archive Digital preservation Tape replacement
  79. 79. “An organization like ours thinks in centuries when it comes to content retention, and long term preservation of our Master Archives is a critical part our mission here at NYPR. Storing these core assets on traditional media such as local disk and off-site tape exposes us to corruption and even outright-loss of data. We are excited to move our archives to Amazon Glacier, which will be a better long-term solution.” Steve Shultis, CTO, New York Public Radio
  80. 80. Backups Disaster Recovery Archival Action items Agility Cost savings Control x
  81. 81. aws.amazon.com get started on the free tier
  82. 82. DR, Backup and Archival Q&A

×