Data migration at a petabyte scale is now a simple service from AWS. You can easily move large volumes of data from onsite environments to the cloud, or quickly get started with the cloud as a backup target using data transfer services, like AWS Snowball or AWS Storage Gateway. Learn about the various data migration options available to you and understand which one is the right fit for your requirements.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Rashim Gupta, Principal Product Manager, Storage Services
March 2017
AWS Snowball, Snowball Edge & Snowmobile

2. Data Migration, Batch and Stream Processing
Direct
Connect
Snowball,
Snowball Edge,
Snowmobile
3rd Party
Connectors
Transfer
Acceleration
Storage
Gateway
Kinesis Firehose
File
Amazon EFS
Block
Amazon EBS
(persistent)
Object
Amazon GlacierAmazon S3 Amazon EC2
Instance Store
(ephemeral)

3. Why are we here?
Large Customer:
“We want to move 250 PBs into AWS, but it would
take years to do that over the network.
How can we upload our data faster into AWS?”

4. Why are we here?
Data is growing at a tremendous pace!
4k/8k
Genomics
Seismic
Financial
Logs
IoT
100TB over the Internet: 2 years
1PB over the internet: 22 years
100PB over T3: 609 years

5. AWS Snow Family
Snowball Snowball Edge Snowmobile
Petabyte-scale
data migration
Compute & Storage for
Hybrid/Edge workloads
Exabyte-scale
data migration

6. AWS Snow Family
Snowball Snowball Edge Snowmobile
Petabyte-scale
data migration
Compute & Storage for
Hybrid/Edge workloads
Exabyte-scale
data migration

7. The AWS data transport solution
AWS Snowball—petabyte scale data transport
• Rugged 8.5G impact case
• Rain and dust resistant
• Data encryption end-to-end
• 50 TB capacity/10G network
• E-ink label
E-ink shipping
label
RE:INVENT 2015 LAUNCH

8. Snowball features released post re:Invent 2015
• Data export
• 80 TB Snowball
• Snowball SDK
• S3 Adaptor
• HDFS import
• HIPAA eligibility

9. How is my data transported securely?
• 256-bit encryption
• Data is encrypted by the
Snowball client before it reaches
the Snowball appliance
• Keys are managed by AWS and
are never sent to the Snowball

10. How is my data transported securely?
• Strong chain of custody
• Tamper-resistant case
• Tamper-resistant electronics
(TPM)
• Each Snowball is erased
according to NIST 800-88 media
sanitization guidelines between
every job

11. How Snowball moves data into and out of AWS
Create
a job
Connect the
Snowball
Copy data to
the Snowball
Your data
moved to
Amazon S3
In transit to you Delivered to you Delivered to AWS At AWS
Job created Job completed

12. Snowball to date
SPEED
COSTLOGISTICS
SECURITY • Circled the earth 100x
• 63 total launches
• 10 AWS Regions
100x

13. When to use AWS Snowball
Cloud
Migration
Disaster
Recovery
Datacenter
Decommission
Content
Distribution

14. Customer use case: S3 as a data lake
Amazon EMR Amazon Kinesis
Amazon
Redshift
Amazon
DynamoDB
Amazon RDS
AWS Data Pipeline
Spark StreamingCassandra Storm
Amazon
S3

15. Case Study: Oregon State University
Use-case:
• Collect and analyze oceanic and coastal images
• 60TB of data per week
• Environmental and ocean ecosystem research
Architecture before Snowball:
• Transferred data with many small hard drives
• Used to take weeks to months to upload data
• $4MM+ in infrastructure investment
• Expensive and inefficient
Snowball lets OSU migrate TBs of data in days at a fraction of the cost

16. What does Snowball cost?
• Per Job shipping
- $250 / Job plus shipping (80TB)
- $200 / Job plus shipping (50TB)
• Includes 10 days to fill the device at your site
• $15/day after the tenth day on site
• Standard S3 charges apply
• $0.00/GB to transfer data in

17. AWS Snow Family
Snowball Snowball Edge Snowmobile
Petabyte-scale
data migration
Compute & Storage for
Hybrid/Edge workloads
Exabyte-scale
data migration

18. AWS Snowball Edge
Petabyte-scale hybrid device with onboard compute and storage
• 100 TB local storage
• Local compute equivalent to an
Amazon EC2 m4.4xlarge instance
• 10GBase-T, 10/25Gb SFP28, and
40Gb QSFP+ networking
• Ruggedized and rack-mountable
RE:INVENT 2016 LAUNCH

19. Snowball Edge key features
S3-compatible endpoint
File interface (NFS)
Clustering
Run AWS Lambda functions
Faster data transfer
Encryption

20. Hybrid capabilities beyond data migrationMIGRATION
Create job Copy data, Ship to AWS Moved to S3
Collect & Process data
PROCESSING
Create job Ship to AWS Moved to S3

21. Snowball Edge can…
Extend of your
data center
Process data Expedite
move
Encrypted, secure,
and embedded
compute
Write data directly
as data is generated
Offers a fast and
cost effective way to
ensure data can be
quickly transferred to
and from the cloud
Simplify
data transfer
Uses standard
and familiar tools
for the data
transfer process

22. Snowball Edge applicability
• Incorporate Snowball Edge into existing applications, i.e.
• Backup destination
• File/object store
• Data cluster
• …
• Build new, end to end applications with Snowball Edge
• In the manufacturing plant
• In the hospital
• In automobiles
• …

23. Snowball Edge use cases
Offline
Staging
Local Tiering
and Compute
IoT
Local
Transformation

24. Aviation
•Use Cases:
- Planes equipped with cameras
- Fly over regions to build 3D models
- Run algorithms on content for data mining
•Current solution:
- Cameras stream data to massive HDDs which are shipped to DC
•How Snowball helps?
- Migrate directly to AWS
•How Snowball Edge helps?
- Identify operational failures in trips for immediate action

25. Healthcare Industry
 Use Cases:
- Typical hospital's data repository can exceed 100s of TBs
- Expensive (time and $$$) to transfer data to AWS
- How to integrate with solutions like Philips HeathSuite to provide
healthcare professionals with data quickly
 How Snowball helps?
- Can transfer data in days instead of weeks
- Can easily scale to hundreds of hospitals
- Meets HIPAA requirements
 How Snowball Edge helps?
- While data is being stored, can we detect anomalies to instantly
raise alerts?

26. Media & Entertainment
•Use Cases:
- 100s of cameras covering a sport
- How to migrate data to cloud to enable offline streaming?
•Current solution:
- Data takes up to 30-60 days to transfer from local DC to AWS
•How Snowball helps?
- Order Snowballs to quickly migrate data to AWS

27. Oil & Gas
Ben Wilson (CTO of GE Oil and Gas):
“PIGs and Snowballs—a match made in heaven! AWS
Snowball 25 TB of pipeline PIG data to be managed at
AWS. That is our GE PIG we pulled some of the data
from. It’s always fun to try new AWS features and try to
break them!”
•Use Cases:
- PIG is a device to inspect cracks in pipelines using sensors
- Device fits inside pipeline and sent through to measure quality
•How Snowball helps?
- Use Snowball to migrate data to AWS

28. IoT
• Use Cases:
- Essess provides SaaS to collect/analyze energy efficiency
- Multiple vehicles equipped with thermal-imaging technology to create heat-maps
- Imaging vehicles detect leaks in windows, doors, walls
• How Snowball helps?
- Each car has a Snowball to store data that is sent to AWS

29. Security
• Use Cases:
- Biometric system uses face detection for security systems
- Replacing security guards with automated security systems
- Dozens of video cameras recording 8-10 hours/day
- Tens of thousands of videos a week
• How Snowball helps?
- When video is recorded, goes to local storage and then shipped to AWS
• How Snowball Edge helps?
- Do lot of image processing; Can use Lambda to do on premise
processing

30. Demo

31. Getting started with Snowball Edge
Step 1: Create a Job and wait for Snowball to ship to you

32. Getting started with Snowball Edge
Step 2: Power it on and connect to network
Step 3: Install the Snowball Edge Client Software
Step 4: Note the IP address on the Kindle Fire of device
Step 5: Get the Manifest file and Unlock code from AWS
Console

33. Getting started with Snowball Edge
Step 6: Unlock the device
snowballEdge unlock -i 10.10.4.4 -m C:JIDcf879535-3be9-438d-
a750-074835618cd0_manifest.bin -u 4d666-84d13-d450f-b860e-195bb
Step 7: Check the status of the Snowball
snowballEdge status -i 10.10.4.4 -m C:JIDcf879535-3be9-438d-
a750-074835618cd0_manifest.bin -u 4d666-84d13-d450f-b860e-195bb
Step 8: Get the S3 credentials
snowballEdge credentials -i 10.10.4.4 -m C:JIDcf879535-3be9-
438d-a750-074835618cd0_manifest.bin -u 4d666-84d13-d450f-b860e-
195bb

34. S3 operations
You can use AWS CLI for S3 commands:
• Store the S3 credentials under .aws/credentials
• aws s3 ls s3://rashimgupta --endpoint http://10.10.4.4:8080

35. NFS operations
• Get the NFS IP Address from the Kindle Fire
• Use the command below to mount it:
mount -o nolock 10.10.4.5:/rashimgupta Z:

36. Using CloudBerry with Snowball

37. Third-party tools and gateways
•Consumer grade: less than $50
• Example: Cloudberry, Arq (Haystack Software)
•Small / medium business: $500 - $1,000
• Example: Synology, Veeam, QNap
•Enterprise grade gateway (price varies)
• Example: NetApp AltaVault, Commvault, TSM, EMC Networker,
BackupExec

38. What does Snowball Edge cost?
• Per Job shipping
- $300 / Job plus shipping
• Includes 10 days to fill the device at your site
• $30/day after the tenth day on site
• Standard S3 charges apply
• $0.00/GB to transfer data in

39. AWS Snow Family
Snowball Snowball Edge Snowmobile
Petabyte-scale
data migration
Compute & Storage for
Hybrid/Edge workloads
Exabyte-scale
data migration

40. What
• 10-100PB in a 45 foot-long, secure (256-bit) ruggedized container truck
Where & When
• Can be made available in all AWS regions
How
• Data transferred via multiple 40Gbps interfaces up to 1Tb/s (100PB in a few weeks)
• Appears as NFS mount point
• Customer orders a Snowmobile, we dispatch it to their site, they hook it up and fill it, it returns
How much does it cost
• $0.005/GB/mo based on provisioned capacity (from site departure to AWS ingestion completion)

41. Snowmobile key features
Massively Scalable
Customized for your needs
Faster data transfer
Strong Encryption
Rugged, Durable, and Secure

42. Snowmobile Case Study: DigitalGlobe
Use-case: Seeing a better world
• DigitalGlobe takes Satellite imagery of the Earth
• 100PB image library = 6 billion square kilometers
• 1PB new image every year
Architecture before Snowmobile:
• Stored data in their own data center
• Needed elastic compute power to retrieve and analyze images
• Wanted to move data to the cloud, but no feasible solution
Snowmobile lets DigitalGlobe migrate 100PB of data to the cloud

43. Summary
Storage 50TB, 80TB 100TB 100PB
Compute N/A Comparable to m4.4xlarge N/A
Select Service
capabilities
N/A Lambda, File Gateway, Tell us
if you want more services
N/A
NFS Endpoint N/A Yes Yes
S3 Endpoint Yes Yes Roadmap
Encryption Yes Yes Yes
Clustering N/A Available N/A
Typical job lifetime <1 month Can be long lasting Weeks – months
HIPAA Yes Roadmap No
Snowball SnowmobileSnowball Edge

44. Questions?
Thank you