STG203_Get Rid of Tape and Modernize Backup with AWS

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
G et Ri d of Tape and Mod erni ze Backup wi th AWS
D a v i d G r e e n , E n t e r p r i s e S o l u t i o n A r c h i t e c t , A W S - @ d a v i d m g r e
P e t e r B a l t a t z i d i s , P r o d u c t M a n a g e r , A W S
J e s s e M a r t i n i c h , S o u t h e r n O r e g o n U n i v e r s i t y
S T G 2 0 3
N o v e m b e r 2 7 , 2 0 1 7

Agenda
• Tape backup considerations and costs
• Backing up to AWS
• Demo: Replace Tape in 10 minutes with AWS Storage Gateway
• How Southern Oregon University replaced tape backups with AWS

Tape… Who likes it?

Well…

I am a converted tape user.

What do people want out of tape backups?


DC space,
power
Productivity
Staff Time
Tape Library
Library Maintenance
New Tape Media
Off-site Archive Service
What comes to mind with tape costs
Tape is cheap, right?
Additional cost factors
New Tape Media

Economics of tape: initial deployment
• Libraries and the media to fill them
• Drives and robotics
• Networking
• Licenses: software, tape slots, and drives
• Support contracts
Additional quantitative costs
• Two or three copies of tapes for DR
• Costs of media migration
(media, people, bandwidth...)
• Cost of under-utilization
• Offsite vaulting services
• Administrators
Harder-to-quantify, but real, costs
• Long recovery times
• Resource contention
• Bit rot, mechanical failure, and “fell
off the truck”
• Secure source of supply and support
Backing up with tape—customer responsibilities &
costs
$ $
$ $

Tape lifecycle management
LTO1
2000
LTO2
2003
LTO3
2005
LTO4
2007
LTO5
2010
LTO6
2012
LTO7
2015
LTO8
2017
https://en.wikipedia.org/wiki/Linear_Tape-Open

Backing Up with AWS

Customer responsibility
• Backup software
• Cloud storage services
• AWS Storage Gateway
• Amazon S3/Glacier/EBS
Backing up with AWS

Data movement
OnlineOffline
Data security
and management
Complete set of data building blocks
Amazon
EFS
Amazon
EBS
AWS Snow family
AWS Storage Gateway
Family
AWS Direct Connect
Amazon EFS File Sync
Amazon S3
Transfer Acceleration
Storage Partners
Amazon Kinesis
Data Streams
Amazon Kinesis
Video Streams
Amazon
S3
Amazon
Glacier
AWS KMS
AWS IAM
AWS CloudWatch
AWS CloudTrail
AWS CloudFormation
AWS Lambda
Amazon Macie
AWS QuickSight

Why switch tape backups to AWS
Retain Existing Workflows
• Continue to use same backup application
• Continue to use existing tape-based policies
Economies of scale in AWS
• 99.999999999% durable
• 11 years of scaling, enhancements, and stability
• Security
Customers can focus on core competencies
• No more library or tape media management
• Reduced infrastructure management
• Reduced staff time

1 PB raw storage
800 TB usable storage
600 TB allocated storage
400 TB application data
Storage utilization—pay as you grow
Amazon
storage

S3 Standard S3 Standard - Infrequent
Access
Amazon Glacier
Active data Archive dataInfrequently accessed data
Milliseconds Minutes to HoursMilliseconds
$0.023/GB/mo $0.004/GB/mo$0.0125/GB/mo
Choice of storage classes in AWS

Options for backup to the cloud
Amazon S3
Amazon
Glacier
AWS
Direct
Connect
Internet
Amazon S3-IA
Application
servers
AWS Storage Gateway
Local disk
Backup server
Storage Gateway
Application
servers
Cloud Connector
Local diskBackup server
with cloud
connector
Application servers
S3 Sync via CLI/API
HTTPS

Backup products with cloud connectors
1. Direct Amazon S3/Glacier API/SDK
2. Varying Amazon S3 lifecycle integration

Storage Gateway hybrid storage
U s e s t a n d a r d p r o t o c o l s t o a c c e s s A W S s t o r a g e s e r v i c e s
AWS Storage
Gateway
Amazon EBS
snapshots
Amazon S3
Amazon Glacier
AWS Identity and Access
Management (IAM)
AWS Key Management
Service (AWS KMS)
AWS
CloudTrail
Amazon
CloudWatch
Files (NFS)
Volumes (iSCSI)
Tapes (VTL-iSCSI)
On-premises
AWSCloud

Backup to AWS via Storage Gateway
T h r e e O p t i o n s t o wr i t e o n - p r e m i s e s b a c k u p s t o AW S
Customer Premises
File
Gateway
GlacierS3-IA
Backup
Server
Customer Bucket
Volume
Gateway
iSCSI
Tape
Gateway
Volume Gateway
S3
Glacier
Tape Gateway VTL
Customer Environment
EBS Snapshots
S3
S3
Standard

AWS Storage Gateway—file gateway
Customer Premises
File
Gateway
• Backup data files stored and retrieved from your S3 buckets
• One-to-one mapping from files-to-objects
• File metadata object metadata
• Bucket access managed by IAM role you own and manage
• S3 features available: Lifecycle Policies, versioning, or CRR
• Recovery in EC2 with second gateway and backup software
GlacierS3
Standard
S3
Standard–
IA
HTTPS
NFS
v3 / v4.1
Backup
Server

Adaptive Server Enterprise cloud backup
with AWS Storage Gateway
File Gateway
VM
HTTPSNFS v3/v4.1
share
GlacierS3 Standard S3 Standard -
Infrequent
Access
Storage
Gateway
Region
Backup
Server
Customer Premises
Adaptive Server
Enterprise • Backup server mounts the GW via NFS
• DB from ASE is dumped to FGW via backup server
• DB dump is stored in S3 and tiered to SIA & Glacier
• Many-to-one ASE to Backup server/FGW
• CloudWatch Event for upload notification
CloudWatch
Event

AWS Storage Gateway – Tape Gateway
• Highly secure, virtual tape storage in S3 and Glacier accessed via tape gateway
• Supports leading backup applications - seamless media lifecycle management
• Data compressed in-transit and at-rest
• Secure, encrypted data
• Unlimited number of virtual tapes
• Up to 1500 tapes immediately accessible in virtual tape library
Archived Tapes
stored in
Amazon Glacier
Customer Premises
Virtual Tapes
stored in
Amazon S3
Backup
Server
Tape Gateway
iSCSI HTTPS
TAPE
DRIVE
MEDIA
CHANGER

Tape gateway supported backup products

At-Scale Cost Comparison - Amazon Glacier vs. on-premises
cost comparison
@SonyDADCNMS Tape Refresh

Costs: A customer example
$56.8K / year
Every month, 100 TB written and 1 tape retrieval; 1-year retention
AWS Storage
Gateway - VTL
Data written via Gateway
Data transfer out and tape retrieval
On-premises Tape
$34.5 K / year
Virtual tapes archived in Glacier
Admin time Admin time
Tape library maintenance
Robotic tape library
Tape media costs
Off-site archive service

Benefits of migrating to AWS virtual tapes
• Seamless replacement for existing systems and workflows
• Reduce maintenance costs by eliminating physical tape devices & hardware
• Low Cost: simple licensing and predictable storage costs
• Security: military-grade encryption of all data in flight and at rest
• Durability: 11-9’s durability and standard tape format
• Accessibility: all backup data in a single console; never swap a tape again
Photo: Brett Coomer, Houston Chronicle

Demo!

Overview of demo: replace tape in 10 min
1. Create new Tape Gateway
2. Create some virtual tapes at the click of a button
3. Connect Tape Gateway to Backup Application
4. Import new tapes into Backup Application
5. Kick Off a Backup Job
6. Archive a Tape

AWS Storage Gateway: How Southern
Oregon University Replaced Tape Backup
Jesse Martinich, Infrastructure Services Manager

Infrastructure Services @ SOU
16,000+ User Accounts
1,000 End-User Machines
350+ Virtual Machines
100+ Applications
800 Multi-User Machines
3.5 Staff

In the beginning, there was tape…
What I want to do
• Provide a reliable backup solution
• Enable the university to operate in a disaster
recovery situation
What I don’t want to do
• Fix tape libraries
• Spend staff hours managing tapes
• Spend money on hardware support
• Buy more hardware

This could be your datacenter

Our data is off-site, but there is also this

Evaluation Process: Req’s & Considerations
• No more tape
• Work with Veeam
• Keep it simple
• Manage less
• Budget
• Implementation costs
• Geographic separation
or
CloudColo
(self-managed)
Cloud
or
Managed
Service Provider

Why AWS?
It was easy to get started
• Low barriers to entry
• Low sticker price
• No commitment
Steps used to evaluate
1. Research
2. Compare
3. Download
4. Test

So what does it look like?
Point backup software at local
gateway VM to write to “tapes”
Virtual tapes reside in AWS Storage
Gateway VTL, aka “Tape Gateway”
SOU Ashland Data Center
AWS Storage Gateway
VM appliance
VTL - iSCSI
Glacier
Tape Gateway
S3
10Gb WAN Link
US-EAST 1 - Virginia
Data encrypted
& compressed
vSphere
cluster
NAS
Windows
servers
Virtual Tape
Archive

How we structure our backups
• Local Veeam backups on disk for quick recovery
• Monthly backups using two VTL Storage Gateways
• Archive tapes from VTL to Glacier
on job completion
• Retain tapes in Glacier for 400 days,
then delete by script
• Rotate backup jobs to spread out
workload, e.g., for 10 TB backups
• Example:
• Tape Gateway 1 Jobs
• Tape Gateway 2 Jobs
Dev VMs Win. VDI
Prod. Linux VMs Prod. Win. VMs
Dept Shares Inf. Linux VMs User Shares
SQL VMs Virt. Appliances Inf. Win VMs
Course Shares

cutoff = datetime.datetime.utcnow() - datetime.timedelta(days=400)
tapeCount = 0
print("Establishing connection to Amazonn")
sg = boto3.client('storagegateway’)
print("Gathering Tape Informationn")
tapeDict = sg.describe_tape_archives()
marker = tapeDict['Marker']
if marker == '':
oldTapes = [(i['TapeARN'], i['CompletionTime'].replace(tzinfo=None)) for i in
tapeDict['TapeArchives'] if i['CompletionTime'].replace(tzinfo=None) < cutoff]
Scripting tape deletion
# This bit converts the “CompletionTime” (archive time) to UTC,
and keeps the tapes older than our cutoff in a list
# Get your archived tapes
# Get your gateway
# We want tapes older than 400 days

for tape in oldTapes:
try:
response = sg.delete_tape_archive(TapeARN=tape[0])
print('%s') % response
except:
print("Unable to process: %s") % tape[0]
continue
try:
tapeDict = sg.describe_tape_archives(Marker=marker)
newmarker = tapeDict['Marker']
if marker != newmarker:
marker = newmarker
else:
break
except:
break
Scripting tape deletion (cont.)
# Delete the tape
# Get next set of tapes
# No more tapes, exit!

Tape recovery workflow
Identify Retrieve Restore

Recovery options
SOU Ashland Data Center
AWS Storage Gateway
VM appliance
VTL - iSCSI
Glacier
Tape Gateway
S3
10GB WAN Link
US-EAST 1 - Virginia
Data encrypted
& compressed
vSphere
cluster
NAS
Windows
servers
Virtual Tape
Archive
Alternative Oregon DC
Hosts
Current Recovery
to SOU, existing or
new gateway &
equipment

Cost
It's around $2,000 per month
On an annual basis, less than the total cost of our previous solution:
• Tape library acquisition and maintenance (two libraries with future expansion
anticipated)
• Tapes (even with re-use, we needed hundreds)
• Off-site storage (weekly pickup and drop-off)
• Labor cost of a student assistant to manage tapes (10+ hours per week)
In addition, we don't have any physical media to manage or store anymore

What does that mean for SOU IT?
• Reduced time and effort to achieve goals
• Better focus on services for the university
• More stable backup architecture
• Greatly improved DR capabilities and flexibility
• Confidence in protection
Would I do it again? Yes.

What’s Next? How Do I Get Started?

Instead of building a
physical vault—

Backup on AWS!

How Do I Get Started?
• Visit aws.amazon.com/storagegateway/vtl
• Drop by one of the demo booths
• See one of us

Learn More in These Sessions
• STG201 – Storage State of the Union
• STG204 – Migrating Large Scale Data Sets to the Cloud
• STG310 – Files in AWS, featuring Celgene Corp
• STG317 – Chalk Talk: Deploying HA Windows File Services w/ Cloud Tiering
• STG320 – Workshop: Hybrid Storage Unleashed, featuring Storage Gateway
• ENT322 – Disaster Recovery with AWS: Tiered Approaches

For Enterprise Storage Engineers
• Learn how to architect and
manage highly available
solutions on AWS storage
services
• Advance toward AWS
certifications
• Help your organization migrate
to the cloud faster
Online at www.aws.training
• Access 100+ new digital
training courses including
advanced training on storage
• Deep Dives on S3, EFS, and EBS
• Migrating and Tiering Storage
to AWS (Hybrid Solutions)
At re:Invent
• Visit Hands-on Labs at the
Venetian
• Attend a proctored
“Introduction to EFS” Spotlight
Lab on Thursday at 3pm at the
Venetian
• Meet Storage experts at the Ask
the Experts in Hands-on Labs
room at the Venetian
New Storage Training

Thank you!

STG203_Get Rid of Tape and Modernize Backup with AWS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to STG203_Get Rid of Tape and Modernize Backup with AWS

Similar to STG203_Get Rid of Tape and Modernize Backup with AWS (20)

More from Amazon Web Services

More from Amazon Web Services (20)

STG203_Get Rid of Tape and Modernize Backup with AWS