This session provides IT pros and application owners an overview of AWS options for building hybrid storage architectures or even entirely migrating datacenter storage to the AWS cloud. The AWS Storage Gateway connects existing on-premises block, file or tape storage systems to AWS cloud storage over the WAN in a hybrid model. The AWS Snow family of physical devices can capture, pre-process and migrate data into and out of AWS without any network connection at all. Join us to learn how you can close down datacenters, reduce storage footprints, and build solutions for tiering, data lakes, backup, disaster recovery, and migration.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Paul Reed, Principal Product Manager, AWS
Ian Perez Ponce, Sr. Business Development Manager, AWS
Smitha Sriram, Sr. Product Manager, AWS
SRV302
Deep Dive on AWS Data Transfer Services
for Migration, Hybrid, & Edge Use Cases

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why are we here?
You have IT resources
We’ll be talking about moving your data
, and want to move them to the cloud
• Data tied to applications you’re
migrating
• Data in bulk storage
(e.g. NAS filers)
• Data connected to
on-premises applications
• Data as part of continuous
edge & hybrid workflows

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Why move data storage to the cloud?
Retirement Management Cost

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Data movement
OnlineOffline
Data security
and management
Complete set of data building blocks
Amazon
EFS
Amazon
EBS
AWS Snow family
AWS Storage Gateway
family
AWS Direct Connect
Amazon EFS File Sync
Amazon S3
Transfer Acceleration
Storage Partners
Amazon Kinesis
Data Streams
Amazon Kinesis
Video Streams
Amazon
S3
Amazon
Glacier
AWS KMS
AWS IAM
Amazon CloudWatch
AWS CloudTrail
AWS CloudFormation
AWS Lambda
Amazon Macie
Amazon QuickSight

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migrating data: Five key questions
S3EC2
EFS EBS Aurora EMR
Amazon Glacier RDSIoT
Amazon
Redshift
1) What kind of data is it, and where is it going?
4) How much data & time do you have?
2) One time or continuous movement?
3) One way or bi-directional?
5) How about the WAN?
Files Block volumes Databases IoT Streams

6. AWS data movement & hybrid storage capabilities
AWS
Direct
Connect
A private
connection
between your data
center, office, or
colocation
environment and
AWS.
AWS Snow
family
(Snowball, Snowball
Edge, Snowmobile)
Secure, physical
transport
appliances that
move up to
exabytes of data
into and out of
AWS.
AWS
Storage
Gateway
Hybrid storage that
seamlessly connects
on-premises
applications to AWS
storage. Ideal for
backup, DR, bursting,
tiering, or migration.
Amazon
Kinesis Data
Firehose
Capture, trans-
form, & load
streaming data
into Amazon S3 for
use with Amazon
business
intelligence and
analytics tools.
Amazon EFS
File
Sync
Up to 5x faster file
transfers than open-
source tools. Ideal
for migrating data
into Amazon EFS or
moving between
cloud file systems.
Amazon S3
Transfer
Acceleration
Up to 300% faster
transfers into and
out of Amazon S3.
Ideal when
working with long
geographic
distances.
APN
competency
partners
Integrations
between third-party
vendors and AWS
services. Ideal for
leveraging existing
software licenses
and skills.
Networks Roads Hybrid

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Backup & Restore Archive Primary Storage BC/DRData Migration
AWS Partner Network: Migration & storage

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Large-scale offline data transfer
& edge processing
with the AWS Snow family

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Snowball AWS Snowball Edge
What is AWS Snowball, and why
did we build it?
Moving large volumes of data over the internet can take
years – We ship secure physical devices to you to transfer
your data at the source before shipping it back for bulk
import to the cloud.
The cloud is not always accessible from remote locations
where connectivity is limited or intermittent – Deploy
ruggedized devices at the edge with local storage and
compute capacity to process data without network
dependencies.
Traditional shipping of conventional hard drives is
laborious and error prone – Our E-Ink shipping label and
chain of custody tracking simplifies logistics at scale.

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Snowball AWS Snowball Edge AWS Snowmobile
• 80-TB storage capacity
• 10GE networking
• Data encryption end-to-end
• Rugged 8.5-G impact case
• Rain and dust-resistant
• 100-TB storage capacity
• 10/25/40 GE networking
• Data encryption end-to-end
• Rugged 8.5-G impact case
• Rain and dust resistant
• AWS Greengrass support for local
compute, messaging, and caching
• EC2/AMI support for edge
compute
• Exabyte-scale storage in a 45-ft
container
• Data encryption end-to-end
• Dedicated security personnel
• GPS tracking, alarm monitoring,
24/7 surveillance, and optional
additional security
AWS Snow family
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
EC2 compute instances on Snowball Edge
What are we announcing
• Support for EC2 C5 instance
family
• Three AMIs available on AWS
Marketplace
• SBE1 instances feature 1.8 GHz
Intel Xeon D processors, up to 24
vCPUs, 32 GiB of memory
• Up to 1-TB disk volumes
Supported EC2 instance sizes
Model vCPU Mem (GiB) Storage (GB)
sbe1.large 2 4 1 x 128
sbe1.xlarge 4 8 1 x 256
sbe1.2xlarge 8 16 1 x 512
sbe1.4xlarge 16 32 1 x 1024
Snowball
Edge

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
How EC2 Instances work on Snowball Edge
Customer AWS Snowball Edge VM import/exportPlaces order using Console/CLI/SDK
Provides AMI
Snowball accepts job and provides job id
Provides amis with s-ami.*
Snowball installs
KVM images on device
Snowball asks VM Import/Export
to convert AMI to KVM
Device is shipped to customer
customer can use EC2 commands to manage instances
VM Import/Export
provides KVM images

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Where do we see edge computing in effect?
Research Industrial Healthcare Transportation
Various industries ranging from research to transportation necessitate edge
computing capabilities to help cope with environments where data generation is
decentralized, data volumes are significant, and network connectivity is either
inaccessible or intermittent.

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Healthcare
Using Snowball Edge to support medical imaging or
optical scanning MRI machines
• Store the image files as they are captured
• Gives local users and administration systems
immediate access
• Using a cluster of Snowball Edge devices, to
stage imaging data for later into Amazon S3
without disrupting systems on site
“Snowball Edge enables us to extend the
innovative capabilities of HealthSuite,
our cloud-enabled connected health
ecosystem of devices, applications, and
digital tools supported by AWS, even
when there is no network support.
- Dale Wiggins,
Business Leader, HealthSuite digital
platform, Philips

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Transportation &
automotive
Problem
Auto manufacturers and suppliers operating
autonomous vehicle fleets for research and
development are challenged with the increasing
volume and sophistication of instrument data
being collected.
Solution
Several leading manufacturers and suppliers are
looking to Snowball Edge to not only help with
migrating data to the cloud, but to also process
data at the edge where localized machine
learning and low-latency data analytics can be
performed.
Impact
Petabyte scale datasets being generated on a
monthly basis no longer have to be migrated to
the cloud in their entirety reducing both costs
and time-to-value as a result of edge
computing.

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid cloud storage architectures

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
So, how can you make storage hybrid?
Gateways

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Storage Gateway enables a spectrum of hybrid use cases
Backup | DR | Archive
Enterprise data centers
Amazon
EBS
Amazon
S3
Amazon
Glacier
App. serversFile servers
Research sites
AWS Storage Gateway
DevicesDatabasesMultimedia content
Analytics | File Services | Machine Learning | Data Processing
Data Distribution | Backup | DR | Archive | Migration
Amazon
EC2
AWS
Lambda
Amazon
CloudFront
Amazon
Athena
Amazon
EMR
Backup serversUsers
Remote offices Small to medium businesses
Amazon
Rekognition
Amazon
MachineLearning

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Storage Gateway family
File gateway
Store and access objects in Amazon S3 from file-based applications
with local caching
Volume gateway
Block storage on-premises backed by cloud storage with local
caching, Amazon EBS snapshots, and clones
Tape gateway
Drop-in replacement for physical tape infrastructure backed by cloud
storage with local caching
Hybrid storage service enabling applications to seamlessly use AWS storage

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Storage Gateway family
Native storage in AWS
• Objects in Amazon S3 (file)
• Snapshots in Amazon EBS (volume)
• Archival in Amazon Glacier (tape)
Gateway provides applications
• Protocol conversion and device emulation
• Caching (read-through / write-back)
• Optimized data transfer
Amazon EC2
AWS Identity and Access
Management (IAM)
Amazon
CloudWatch
AWSKeyManagementService
(AWSKMS)
AWS CloudTrail
Files
(NFS or SMB)
Volumes
(iSCSI)
Tapes
(iSCSI VTL)
Amazon S3
Amazon Glacier
Amazon EBS
snapshots
Your Data Center
AWS Storage Gateway
HTTPS
Gateway Service

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
File gateway for hybrid cloud file workloads
(online archives, content editing & distribution,
data processing, backups)

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
File gateway
Store and access objects in Amazon S3 from file-based applications with local caching
Customer Premises
HTTPS
NFS or
SMB
File Gateway Objects in your
S3 bucket
Application
Server
Reduce on-premises
storage infrastructure
and management
Fully managed
local cache for
low latency access
Durability, scalability,
and reliability of
Amazon S3 storage
Flexible options for
data access, storage
class & security

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
File gateway support for SMB
Store and access objects in Amazon S3 buckets
from file-based Windows applications
File gateway Objects in your
Amazon S3 bucket
SMB v2 or v3
client
On-premises Active
Directory or AWS
Directory Service
Share-level access can
be restricted to
configured users and
groups
POSIX ACLs
(compatible subset of
NTFS) for object-level
permissions
Objects created
directly in Amazon S3
inherit ACLs from
parent folder

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
File gateway: Low latency access to active data
AWS RegionCustomer premises
Objects in your S3
bucket
NFS or SMB
client
Data
cache
Metadata
cache
File
share
File gateway
Fully managed cache of
some of your data
Durable storage of all of
your data

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS RegionCustomer premises
Objects in your S3
bucket
NFS or SMB
client
Data
cache
Metadata
cache
File
share
File gateway
File gateway: Caching details
Cache holds parts of files
based on what the client
reads/writes
Data remains in cache until
space is needed for more
recently accessed data

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS RegionCustomer premises
Objects in your S3
bucket
NFS or SMB
client
Data
cache
Metadata
cache
File
share
File gateway
File gateway: Caching details
Metadata is cached based on
operations performed by the
file client.
Bucket can have billions of
objects in it, Gateway
discovers them on demand.

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon
Glacier
Amazon S3
Standard
S3-Infrequent
Access
File Gateway
PetroBank
Application
ServersLTO
NAS
Active archive migration from disk & tape
Cost-effective storage in AWS with local data access
AWS Direct
Connect
Halliburton data center
1
2
Use Snowball to ship data from on-premises offline archives1
2 Online access to all data through gateway
Minimal on-premises storage reduces cost
Time-to-date by reduced by days or weeks

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid file use case: Content distribution
Seattle data center
File gateway
(read-only)
Application
Boston data center
Objects in your S3
bucket
AWS Region
File gateway
1
Application
2
Application in Seattle writes files, which are uploaded to Amazon S3 by gateway1
2 After refresh cache, files are visible to applications in Boston
Local cache improves access performance

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid file use case: Low-latency content editing
Los Angeles office
Content repository in
Amazon S3
AWS Region
File gateway
Editing
workstation
Editing
workstation
Low latency access for data actively being edited1
Changes are all durably stored in master repository in Amazon S32
2
1

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid file use case:
Demonstration of SQL Server backup with
native SQL tools and file gateway

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Native SQL backup to Amazon S3 via SMB
Corporate data center
Storage Gateway VM
SQL Server
(native SQL agent)
Domain
controller
US-West-1
Amazon
SQS
Amazon
CloudWatch
AWS
Storage
Gateway
Amazon Glacier
Amazon S3 - Infrequent Access
Backup
bucket Expire / delete backup after x years
Share
(SMB)
Local
cache
Lifecycle after 30 days

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Hybrid volumes for recovery & migration
with volume gateway
and Amazon EBS snapshots

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Application
server
Amazon EBS
snapshots
Storage GatewayVolume
gateway
appliance
Volume stored
in
Amazon S3
HTTPSiSCSI
Customer premises Region
Volume gateway
Cloud-backed block storage presented on-premises
• Tier snapshots or whole volumes to the cloud to reduce SAN/NAS mgt.
• Flexible recoveries in-cloud or on-premises with snapshots and clones
• Common uses: backup and restore, disaster recovery, data migrations

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Application
server
Amazon EBS
snapshots
AWS Storage
Gateway
Volume
gateway
appliance
Volume stored
in
Amazon S3
HTTPSiSCSI
Customer premises Region
100% of volume stored in
AWS & on-premises
Volume gateway: Stored mode
Low-latency access to all your data with point-in-time backup to the
cloud through Amazon EBS snapshots

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EBS
snapshots
Storage Gateway Volume stored
in
Amazon S3
HTTPSiSCSI
Customer premises Region
100% of volume
stored in AWS
Volume gateway VM
Virtual
volume
Fully-managed cache of
frequently used data
Application
server
Volume gateway: Cached mode
Reduce on-premises storage, caching frequently used data local to
your application, with 100% of your data in the cloud

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Volume gateway data protection w/ snaps & clones
Volume clones
Instant real-time copy of a cached volume
Represents current state of volume stored in AWS
Restore as a Storage Gateway volume
EBS snapshots
Point-in-time backups of a stored or caches volumes
Created on-demand or on a configurable schedule
Restore either as an EBS or a Storage Gateway volume

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
1. Restore to your
data center
Data recovery & migration with volume gateway
Storage Gateway
3. Recovery to a 2nd
DR site
EC2EBSGateway
volume
1
2
3
2. Migrate to AWS

38. Canada’s largest biotech firm
• Data sovereignty required local hot files
& tape archives in 10 global offices
• Volume Gateway eliminated 50-hour backup
windows and tape archive systems
• Cut on-premises. storage CAPEX 40%;
reduced RTO from 48 hours to 10 minutes
• Meets cloud strategy while retaining local
ownership and data sovereignty
• Enabled data center exit in next 6–12 months
“It made no sense to keep buying
big disk siloes, especially as we
opened up new global offices, and
now we can recover in the cloud
from a snapshot if we ever had to.”
Adam Leggett
IT manager
Stemcell’s backup & restore with volume gateway

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Migrate tape backup workflows to AWS
with Tape Gateway

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tape gateway: Drop-in replacement for tape backups
• Emulates a tape library. Virtual tapes on Amazon S3 and Amazon Glacier
• Works with common backup apps, to support existing backup workflows
• Low-cost: Predictable costs and reduced management

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tape gateway: Cost comparison, a customer example
Every month, 100 TB written and 1 tape retrieval; 1-year retention
Tape gateway
Data written via
gateway
Data transfer out
and tape retrieval
On-premises tape
$34.5 K / year
Virtual tapes archived
in Amazon Glacier
Admin time
Admin time
Tape library
maintenance
Robotic tape library
Tape media costs
Offsite archive
service
$56.8K / year

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tape gateway: How the VTL works
Customer premises
Region
Storage Gateway
Backup server
Upload buffer
Cache
Media
changer
Tape Drive
Tape Drive
Tape Drive
Tape Drive
Tape Drive
Tape Drive
Tape library
(Amazon S3))
Tape shelf
(AmazonGlacier)
• Emulates a physical tape device with a media changer and tape drives
• Scalable: Virtually unlimited tape storage in AWS
• Virtual tapes are written to S3: Data is in S3 when tape is in virtual library drive or slot
• “Ejected” virtual tapes are marked read-only and moved to “Tape Shelf” on Amazon Glacier
• Recovery: Retrieve tapes to library (3–5 hours) and read data to same or different gateway
Tape gateway VM
Tape Drive

43. Submit Session Feedback
1. Tap the Agenda icon.
2. Select the session you
attended.
3. Tap on Session Evaluation to
submit your feedback.

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Questions?

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you.