The document describes Amazon Web Services storage portfolio including Storage Gateway, which is a hybrid storage solution that allows on-premises applications to access objects in Amazon S3 or files in Amazon EFS. It provides three options: file gateway, volume gateway and tape gateway. File gateway stores file data as objects in S3, volume gateway stores volume data in S3 and allows for EBS snapshots, and tape gateway provides virtual tape storage in S3 and Glacier with tape library emulation. The document also provides use cases and examples of migrating and recovering data using Storage Gateway.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The AWS Storage Portfolio
Cloud Data Migration
Direct ConnectSnow data
transport family
Third-Party
Connectors
Transfer
Acceleration
Storage
Gateway
Kinesis Firehose
Object
Amazon GlacierAmazon S3
Block
Amazon EBS
(persistent)
Amazon EC2
Instance Store
(ephemeral)
File
Amazon EFS

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Storage Gateway Hybrid Storage Solutions
Use standard storage protocols to access AWS storage services
AWS Storage
Gateway
Amazon EBS
snapshots
Amazon S3
Amazon Glacier
AWS Identity and Access
Management (IAM)
AWS Key Management
Service (KMS)
AWS
CloudTrail
Amazon
CloudWatch
Files
Volumes
Tapes
On-premises
AWSCloud

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Storage Gateway Use Cases
• Backup from on-premises applications using existing workflow
• Archive to AWS (Amazon S3 and S3-IA, Amazon Glacier) using native protocols
• Disaster recovery for on-premises data and application servers
• Snapshot to the cloud for data protection and recovery
• Third-party application support for Amazon S3 and S3-IA
• Online data migration to AWS with minimal downtime

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
File Gateway
On-premises file storage maintained as objects in Amazon S3
• Data stored and retrieved from your Amazon S3 buckets
• One-to-one mapping from files to objects
• File metadata stored in object metadata
• Bucket access managed by IAM role that you own and manage
• Use Amazon S3 lifecycle policies, versioning, or CRR to manage data
Amazon
Glacier
Amazon S3
Standard
S3 Standard-
Infrequent
Access
Storage
Gateway
Application
Server File Gateway
VM
HTTPSNFS v3/v4.1
Customer Premises
share
Region

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Enabling Cloud Workloads
• NFS-based applications to write to Amazon S3 without modification
• Virtually limitless capacity
• Leverage Amazon S3-enabled compute resources against NFS stored datasets
Storage
Gateway
NFS Clients
File Gateway
VM
Customer Premises
share
Region
Files stored as
objects
HTTPS
EC2
Spot
Instances
Amazon
EMR
Amazon
Athena

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Tape Gateway
Virtual tape storage in Amazon S3 and Amazon Glacier with VTL
management
• Virtual tape storage in Amazon S3 and Amazon Glacier accessed via
tape gateway
• Data compressed in transit and at rest
• Up to 1 PB total tape storage per gateway, unlimited archive
capacity
• Supports leading backup applications
• **3-5 hour tape retrieval from Amazon Glacier
Media Changer
Virtual tapes
stored in
Amazon S3
Archived tapes stored in
Amazon Glacier
Backup
Server
iSCSI
Tape
Gateway
Tape Gateway VM
Tape Drive
HTTPS
Customer Premises Region

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Remote Region Backup
Media Changer
Virtual tapes
stored in
Amazon S3
Archived tapes stored in
Amazon Glacier
Backup
Server
iSCSI
Tape
Gateway
Tape Gateway VM
Tape Drive
HTTPS
Customer Premises
Region
us-east-1
Availability Zone #1
EC2 instance EC2 instance
Windows
Server• Tapes are written to remote region for safekeeping
• Virtual tapes are more durable than traditional tape
• Pilot light DR options are more easily available

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
On-premises volume storage backed by Amazon S3 with EBS snapshots
• Block storage in Amazon S3 accessed via the volume gateway
• Compression of data in transit and at rest
• Back up on-premises volumes to EBS snapshots
• Create on-premises volumes from EBS snapshots
• Up to 1 PB of total volume storage per gateway
• Comes in two flavors…
Volume Gateway
Application
Server
Amazon EBS
snapshots
Storage Gateway
Service
Volume
Gateway VM
Volume stored in
Amazon S3
HTTPSiSCSI
Customer Premises Region

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Volume Stored
volume Amazon EBS
snapshots
Customer Premises
iSCSI
Storage Gateway VM
Region
stored
volume
upload
bufferApp
Server
Complete volume
stored in the cloud
and on premises
Amazon
CloudWatch
Storage
Gateway
Service

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Volume Stored (Write Operation)
volume Amazon EBS
snapshots
Customer Premises
iSCSI
Storage Gateway VM
Region
stored
volume
upload
bufferApp
Server
Amazon
CloudWatch
Storage
Gateway
Service
compress
encryption

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Volume Stored (Read Operation)
Amazon EBS
snapshots
Customer Premises
iSCSI
Storage Gateway VM
volume
Region
stored
volume
upload
bufferApp
Server
Amazon
CloudWatch
Storage
Gateway
Service

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Backup and Disaster Recovery
Region
Storage
Gateway
Service
Amazon EBS
snapshots
EC2 instance EC2 instance
Windows
Server
Availability Zone #1
EC2 instance
Windows
Server
EBS
Volume
DATA
SGW
Application
Server
Volume
Gateway VM
Customer Premises
Data
Data
iSCSI
Data
Availability Zone #1
(Clone)
• Gateway volumes are continually written to
Amazon S3
• Point in time protection via EBS snaps
• Ability to recover to EBS from snaps
• Ability to clone the latest volume to another
GW in the event of a disaster

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Volume Cached
volume Amazon EBS
snapshots
Customer Premises
App
Server
iSCSI
Storage Gateway VM
Region
virtual
volume
Complete volume
stored in the cloud
Virtual volume stored
on premises backed
by cache and cloud
Amazon
CloudWatch
Storage
Gateway
Service
cache
upload
buffer

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
App On-Premises Blocks in the Cloud
Private Data Center
Windows
Clients
Windows Server
Region
Storage
Gateway
Service
Volume Gateway
VM
Amazon EBS
snapshots
iSCSISMB HTTPS
Reduce on-premises
storage through
cached mode
Complete volume
stored in the cloud
• Application server on premises, backed by cache
and cloud
• Reduces on-premises storage
• Durably stores on-premises accessed block data in
S3
• Additional protection though EBS snaps

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Volume Cached (Write Operation)
volume Amazon EBS
snapshots
Customer Premises
App
Server
iSCSI
Storage Gateway VM
Region
cache
virtual
volume
upload
buffer
Amazon
CloudWatch
Storage
Gateway
Service
compress
encryption

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Volume Cached (Read from Cache)
volume Amazon EBS
snapshots
Customer Premises
App
Server
iSCSI
Storage Gateway VM
Region
virtual
volume
Amazon
CloudWatch
Storage
Gateway
Service
cache
upload
buffer

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Volume Cached (Read from Cloud)
volume Amazon EBS
snapshots
Customer Premises
App
Server
iSCSI
Storage Gateway VM
Region
virtual
volume
Amazon
CloudWatch
Storage
Gateway
Service
cache
upload
buffer

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration with Storage Gateway Volumes
Customer Premises Region
App
Server
Existing
Volume
Existing Volume
SAN, VMDK, VHD based
volume

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration with Storage Gateway Volumes
volume
Customer Premises Region
App
Server
Storage
Gateway
Service
stored
volume
iSCSI
Initial Sync
HTTPS
Volume Stored Mode
Storage Gateway VM
Attach volume to GW
Disconnect volume from app
server and connect it to GW
(data in place)
Accessible
over iSCSI

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration with Storage Gateway Volumes
volume Amazon EBS
snapshots
Customer Premises Region
App
Server
Storage
Gateway
Service
stored
volume
iSCSI HTTPS
Volume Stored Mode
Storage Gateway VM
Create EBS snaps
For protection or cloning

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration with Storage Gateway Volumes
volume Amazon EBS
snapshots
Customer Premises Region
App
Server
stored
volume
iSCSI HTTPS
Volume Stored Mode
Storage Gateway VM
Availability Zone #1
EC2 instance
Windows
Server
EBS
Volume
Storage
Gateway
Service
Recover/Clone
EBS snaps can be used
to launch instances with
the volume

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration with Storage Gateway Volumes
volume Amazon EBS
snapshots
Customer Premises Region
App
Server
Storage
Gateway
Service
stored
volume
iSCSI HTTPS
volume
HTTPS
Volume Cached Mode
Storage Gateway VM
Volume Stored Mode
Storage Gateway VM
Availability Zone #1
EC2 instance
Windows
Server
EBS
Volume
Gateway Volume
from snapshot
Create a cached volume
Re-present the volume back
on premises in cached mode

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Migration with Storage Gateway Volumes
Amazon EBS
snapshots
Customer Premises Region
App
Server
Storage
Gateway
Service
stored
volume
volume
HTTPS
Volume Cached Mode
Storage Gateway VM
Availability Zone #1
EC2 instance
Windows
Server
EBS
Volume
Gateway Volume
from snapshot
Shrink on-premises storage
Decommission original
storage and run off cached
volume

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Scenarios

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Recovery Scenario 2
Workshop Scenario One: Volume Gateway
Availability Zone #1
eu-west-1
Availability Zone #1
eu-central-1
Storage
Gateway
Service
Amazon EBS
snapshots
EC2 instance
E:
EC2 instance
Windows
Server
eu-west-2
EC2 instance EC2 instance
Windows
Server
Availability Zone #1
Recovery Scenario 1
EC2 instance
Windows
Server
E:
E:
“DR data center”
SGW SGW
“Your data
center”

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deploy Windows Server
Workshop Scenario One: Volume Gateway
eu-west-1
Availability Zone #1
EC2 instance
Windows
Server
D:
C:
(root)
Deploy Windows Server
Simulate on-premises server
with a root EBS volume and
data EBS volume (E:) in an
AWS region
“Your data
center”

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Scenario One: Volume Gateway
eu-west-1
Availability Zone #1
eu-central-1
Storage
Gateway
Service
Amazon EBS
snapshots
EC2 instance
E:
E:
Stored in
Amazon S3
EC2 instance
Windows
Server
D:
C:
(root)
Deploy Gateway, Migrate Data, and Protect
Deploy GW, Migrate, Protect
1. Deploy gateway backed by
remote region
2. Create a volume and attach
it to the Windows server
(via iSCSI)
3. Migrate the data from D: to
E: (copy)
4. Enable EBS snapshots on
the gateway volume (and
manually create one too)
“Your data
center”

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Scenario One: Volume Gateway
eu-west-1
Availability Zone #1
eu-central-1
Storage
Gateway
Service
Amazon EBS
snapshots
EC2 instance
E:
Availability Zone #1
E:
Stored in
Amazon S3
Recovery Scenario 1
EC2 instance
Windows
Server
D:
C:
(root)
EC2 instance
Windows
Server
E:
C:
(root)
Recover Windows Server
Create an EBS volume from
EBS snapshots and launch the
instance with replica volume
attached
(point in time recover)
How long did it take?
Can you make it faster?
“Your data
center”

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Scenario One: Volume Gateway
eu-west-1
Availability Zone #1
eu-central-1
Storage
Gateway
Service
EC2 instance
E:
E:
Stored in
Amazon S3
EC2 instance
Windows
Server
D:
C:
(root)
How long did it take?
Can you make it faster?
Recovery Scenario 2
Availability Zone #1
E: Clone
Stored in
Amazon S3
eu-west-2
EC2 instance
E:
EC2 instance
Windows
Server
C:
(root)
Recover Windows Server
Create an EBS volume from
EBS snapshots and launch the
instance with replica volume
attached
(point in time recover)
“DR data center”
“Your data
center”

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Scenario Two: File Gateway
eu-west-1
Availability Zone #1
EC2 instance
Linux
Server
root volume
EC2 instance
share
S3 Bucket 1
file data
(as objects)
eu-central-1
S3 Bucket 2
file data
(as objects)
Amazon
Glacier
Lifecycle
Policy
Storage
Gateway
Service
S3 IA
eu-west-2

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Scenario Two: File Gateway
eu-west-1
Availability Zone #1
EC2 instance
Linux
Server
root volume
(with file data)
Deploy Linux Server
Deploy Linux Server
Simulate on-premises server
with a root volume and file
data (provided)

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Scenario Two: File Gateway
eu-west-1
Availability Zone #1
EC2 instance
Linux
Server
S3 Bucket 1
eu-central-1
S3 Bucket 2
Amazon
Glacier
Lifecycle
Policy
S3 IA
eu-west-2
Secondary bucket with CRR objects + Lifecycle to Amazon Glacier
Set Up Remote Buckets
1. Create “bucket 1” in a secondary region
2. Create “bucket 2” in a tertiary region
3. Enable CRR from b1 to b2
4. Create a lifecycle policy on the ”bucket
2” to move old data to Amazon Glacier
root volume
(with file data)

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Scenario Two: File Gateway
eu-west-1
Availability Zone #1
EC2 instance
Linux
Server
EC2 instance
share
S3 Bucket 1
file data
(as objects)
eu-central-1
S3 Bucket 2
file data
(as objects)
Lifecycle
Policy
Storage
Gateway
Service
S3 IA
eu-west-2
root volume
(with file data)
Deploy Gateway, Create Share, and Migrate Data
Deploy, Create, Migrate
1. Deploy and activate FGW as an EC2 instance in region 1
and activate it in region 2
2. Create a share connected to remote bucket 1
3. Mount the share on the Linux Server
4. Copy data from EBS on Linux Server to the mounted
share
Amazon
Glacier

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Scenario Two: File Gateway
eu-west-1
Availability Zone #1
EC2 instance
Linux
Server
root volume
EC2 instance
share
S3 Bucket 1
file data
(as objects)
eu-central-1
S3 Bucket 2
file data
(as objects)
Lifecycle
Policy
Storage
Gateway
Service
S3 IA
Access Content
Access the content from the
Amazon S3 buckets, using
native S3 commands
eu-west-2
Amazon
Glacier

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Workshop Resources
• GitHub: https://github.com/awslabs/hybrid-storage-workshop
• Work together or on your own
• $25 dollars in credits will be distributed at exit
• Ask us about activating GW across regions if you get stuck
• Don’t forget to clean up!