Learning Objectives: - Learn what storage regulations to be aware of when developing and deploying a cloud based storage solution - Gain awareness of various strategies to address compliance - Examine solutions available from AWS, including Amazon S3, Amazon Glacier and the Vault Lock feature, AWS Snowball and their data ingestion services.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Henry Zhang
Senior Product Manager, Amazon Glacier
October, 2017
Deploy and Enforce Compliance Controls
When Archiving Large Scale Data Stores

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Cloud Data Migration
Direct
Connect
Snow* data
transport
family
3rd Party
Connectors
Transfer
Acceleration
Storage
Gateway
Kinesis Firehose
The AWS Storage Portfolio
Object
Amazon GlacierAmazon S3
Block
Amazon EBS
(persistent)
Amazon EC2
Instance Store
(ephemeral)
File
Amazon EFS

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Satellite Image Archive
• DigitalGlobe takes Satellite imagery of the Earth
• 100PB image library = 6 billion square kilometers
• 1PB new image every year
• Images to be archived and retained for decades

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Patient data–Philips Healthcare
• HealthSuite digital platform powered by AWS
• 15 petabytes of patient data
• Archived for decades (beyond the lifetime of patients)
• Uses AWS HIPAA-eligible services in the BAA

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Public sector–King County
• Most populous county in Washington state
• Replaced tape solution for backup from 17 agencies
• Meets compliance requirement
• Saved $1MM in first year; no more tape refresh or
management churn

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Archive Storage Options and Considerations

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Traditional archiving approaches
• Tape libraries, robots, drives, media
• Onsite (online and offline)
• Offsite tape out/vaulting
• Specialized software and personnel
• Tape refresh every 3-5 years

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How can AWS help with your archival?
Metered usage:
Pay as you go
No capital investment
No commitment
No risky capacity planning
Avoid risks of physical
media handling
Control your
geographic locality for
performance and
compliance

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Object Storage Options
S3 Standard
Active data Archive dataInfrequently accessed data
S3 - Infrequent Access Amazon Glacier
Synchronous access Async accessSynchronous access
$0.023/GB/mo. $0.004/GB/mo.$0.0125/GB/mo.

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
- Transition Standard to Standard-IA
- Transition Standard-IA to Amazon Glacier
- Transition based on object tags
- Expiration and versioning
Data lifecycle management
T T+3 days T+5 days T+ 15 days T + 25 days T + 30 days T + 60 days T + 90 days T + 150 days T + 250 days T + 365 days
Data access frequency over time

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Transition older videos to Standard-IA

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Save money on storage
45% saving over S3 Standard-IA
68% saving over S3 Standard-IA
* Assumes the highest public pricing tier

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1 PB raw storage
800 TB usable storage
600 TB allocated storage
400 TB application data
Storage pricing - pay only for what you use
Traditional storage AWS Cloud
StoragePrice Drop on 11/21/2016
- Amazon S3 23% price drop to $0.023/GB/month
- Amazon Glacier 43% price drop to $0.004/GB/month

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
99.999999999%
Durability
Durability for long-term preservation
Built-in Fixity Checking
Automatic recovery

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Accessing Amazon S3 and Glacier
1. Direct service API/SDK
2. Amazon S3 lifecycle integration
3. Third-party tools and gateways
FastGlacier

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Glacier – Direct access/APIs
Create
Vault
Configure
Access
Upload
Archives
Register
Archive ID
Data Upload
Initiate
Retrieval
Async
Retrieval
Completion
Completion
Notification
Download
Data
Data Retrieval

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Glacier – Third-party tools and gateways
• Consumer grade: less than $50
• Example: Cloudberry, FastGlacier, Arq (Haystack Software)
• Small / medium business: $500 - $1,000
• Example: Synology, Veeam, QNap
• Enterprise gateway and data management software
• Example: NetApp AltaVault, CommVault, StorNext, StoreReduce,
Vidispine, Preservica

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Which option should I choose?
• Use S3 lifecycle managed Amazon Glacier if the S3
object keys are sufficient for index/search capability
• Use Amazon Glacier directly if you already plan to store
more metadata/indices in a database
• Use 3rd party tools to minimize coding

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Flexible Data Retrieval Options
All of your Glacier data is accessible with any of three retrieval options.
Standard Retrieval
• Current model
• 3-5 hours
• $0.01/GB
Bulk Retrieval
• Batch/Bulk access
• 5-12 hours
• $0.0025/GB
Expedited Retrieval
• Rare urgent access
• 1-5 minutes
• $0.03/GB
On-site tape replacement Off-site tape replacement

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Compliance Use Case – Regulatory Retention

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Glacier Vault Lock allows you to easily
set compliance controls on individual vaults and enforce
them via a lockable policy
Time-based retention
MFA authentication
Controls govern all
records in a vault
Immutable policy
Two-step locking
Compliance storage with Vault Lock

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock for compliance storage
• Non-overwrite, non-erasable records
• Time-based retention with “ArchiveAgeInDays” control
• Policy lockdown (strong governance)
• Legal hold with vault-level tags
• Configure optional designated third-party access and grant
temporary access

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Glacier received a third-party assessment
from Cohasset Associates on how Amazon Glacier
with Vault Lock can be used to meet the requirements
of SEC Rule 17a-4(f) and CFTC 1.31(b)-(c).

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example control: 1-year record retention
• Deny delete archive operation
• From anybody (root, administrators, users, business partners)
• When ArchiveAgeInDays is <= 365 days
Archive age computed from the time an archive lands in a vault

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example control: 1-year record retention

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock: Two-step locking
• InitiateVaultLock
– Effectuates a retention policy for testing (in-progress state)
– Returns a unique lock ID (expires after 24 hours)
• AbortVaultLock
– Deletes an in-progress policy
– Ability to modify a policy before locking it down
• CompleteVaultLock
– Locks down the vault with the appropriate lock ID
– Vault Lock cannot be aborted afterwards

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Legal hold with vault-level tags
• Set up a legal hold tag
– Configure a vault-level tag “LegalHold”
– Set initial value to “False”
• Add compliance control for legal hold in a Vault Lock policy
– Deny delete archive operation
– From anybody (root, administrators, users, business partners)
– When LegalHold tag = “True”
• Place/lift legal hold by updating the tag value

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Example control: Legal hold

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

45. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vault Lock in the Amazon Glacier console

46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Proofpoint
• Cloud-based security and compliance for the enterprise:
threat research, email, mobile, social, digital risk
• Founded 2002, public in 2012
• $350M annual revenue, $3B market cap
• Big AWS user

47. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Proofpoint SocialPatrol
Policy controls and enforcement for social
• Combats fraudulent brand impersonation
• Moderates content at scale
• Ensures compliance in publishing
• Integrates with social APIs
• 150+ classifiers using NLP and ML
• Text, links, images, meta data
• Ingesting >1M social posts per day
• Built in AWS

48. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Proofpoint SocialPatrol Archive with Glacier
SEC Rule 17a-4(f)-compliant archive, purpose-built for
social, enabled by Amazon Glacier and Vault Lock
PFPT in AWS
Policy engine MySQL/C*/SolrSocial
Amazon Glacier
& Vault Lock

49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Proofpoint SocialPatrol Archive
Via AWS API,
we lock the vault,
and specify policy
to observe a
legal hold via a tag.

50. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Proofpoint SocialPatrol Archive
As social content flows in, we record its purge date and
surface that to the user. Each piece of social content is an
archive in the vault.

51. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Migration Options

52. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Snow Family
Snowball Snowball Edge Snowmobile
Petabyte-scale data
migration
Compute & Storage for
Hybrid/Edge workloads
Exabyte-scale data
migration

53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS Snowball Edge
Petabyte-scale hybrid device with onboard compute and storage
• 100 TB local storage
• Local compute equivalent to an
Amazon EC2 m4.4xlarge instance
• 10GBase-T, 10/25Gb SFP28, and
40Gb QSFP+ networking
• Ruggedized and rack-mountable
Hardware Update
S3-compatible endpoint
File interface (NFS)
Clustering
Run AWS Lambda functions
Faster data transfer
Encryption
Key Features

54. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What
• 10-100PB in a 45 foot-long, secure (256-bit) ruggedized container truck
Where & When
• Can be made available in all AWS regions
How
• Data transferred via multiple 40Gbps interfaces up to 1Tb/s (100PB in a few weeks)
• Appears as NFS mount point
• Customer orders a Snowmobile, we dispatch it to their site, they hook it up and fill it, it returns
How much does it cost
• $0.005/GB/mo based on provisioned capacity (from site departure to AWS ingestion completion)

55. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!