AWS Storage State of the Union & APN Storage Ecosystem

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Pop-up Loft
Storage State of the Union
Drew Meyer,
Storage Product Marketing Head

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved
Two Facts of Life

Data Has Gravity
Move the processing to the data!
4k/8k
Genomics
Seismic
Financial
Logs
IoT
3

Cloud Data Migration
Direct ConnectSnow* data
transport
family
3rd Party
Connectors
Transfer
Acceleration
Storage
Gateway
Kinesis Firehose
The AWS Storage Portfolio
Object
Amazon GlacierAmazon S3
Block
Amazon EBS
(persistent)
Amazon EC2
Instance Store
(ephemeral)
File
Amazon EFS

Migration
Storage Use Cases
File (EFS)
Web/Content Serving
Enterprise Applications
Media Processing
workflows
Container Storage
Object
(S3 & Glacier)
User-Generated Content
Active Archive
(Media, Imagery, Compliance)
Serverless Computing
Big Data (Analytics Engines and Data Lakes)
Block (EBS)
Relational and
NoSQL Databases
Data Warehousing
(Teradata, Vertica)
Backup, Restore and Disaster Recovery

Container Storage
Web/Content Serving
Media Processing Workflows
Object
Block
File
Partner Ecosystem
& Certifications
Scale and Experience
Durability and Availability
Active Archive
Serverless Computing
User-Generated Content
Relational Databases,
Data Warehousing,
Backup & Restore, Compliance,
Disaster Recovery, Big Data
Security and Encryption
Storage
Management

S3 & Glacier
EBSEFS
Big Data
Analytics
Query in Place
Amazon
Athena
Amazon
Redshift
Spectrum
7
Artificial Intelligence:
Frameworks
Services
Platforms

Use Case: NoSQL Databases
Librato (SolarWinds)
• Operates custom cloud monitoring and management dashboards
• Depends on Cassandra for intensive, random workloads
• 2.2x large, 1.5TB RAID0 on 160 instance stores
• Had to pay for and scale everything in realtime (no persistence)
• Migrated from Instance Store to EBS volumes
Results
• Separate compute and storage
• Instances and volumes designed around workload needs
• 1 week (realtime): c4.x4 large, 2TB gp2, 600GB st1
• 1 year (long retention) m4.2x large, 4TB, gp2, 600GB st1
• Reduced costs 35%, increased retention from 1 week to 1 year
• Increased Operational Efficiency and DR with data persistence and EBS snapshots
• Increased durability with EBS may allow for lower replication factors at the application
level (future cost savings)
8

Use Case: Content Serving
AutoDesk’s Creative Market
• Digital storefront for fonts, graphics, themes, photos, and software
• Gluster was a PAIN
– Minor revisions would break functionality
– Documentation seemed inaccurate
– Routine maintenance meant the whole site went down (15 hrs/mo)
– Consumed one dedicated employee
Results:
• Moved to fully-managed EFS
• No administrative headaches
• Annual costs down 95%

Use Case: Active Archive
Sony DADC
• Wanted on-demand media supply chain and delivery solution
• High performance, secure, economical
• Decrease time to market for finished customer content
Results:
• Workflow pipelines can now run in parallel
• Content delivery SLA is now 1 hour
• Fully migrating away from on-prem infrastructure
10

What are customers asking for now?
Data Lakes
Help me build a
place to collect,
store and share the
insights from across
my data
Backup and Restore
& Tape Replacement
Help me achieve
quick, low-risk, high
ROI wins
with the cloud
Disaster Recovery
Help me build and
automate fault-
tolerance
or failover

Use Case: Data Lake
FINRA
• Quickly ingests data from many sources and store it efficiently in a single location (S3)
• Apply multiple analytics/processing tools (eg, Athena, EMR, Rekognition, Redshift) to
speed time to value
• Migrated on-premise Data Warehouses, Hadoop & Big Data clusters to AWS
Results
• Decoupled storage and compute to scale performance and cost optimally
• Eliminated data silos
• Apply the right analytic tools (Athena, EMR, Rekognition, Redshift) for each job, and evolve
as requirements change with no data migration
• Increased agility and speed to results
• Saving $10M-$20M per year over on-premises

Use Case: Data Lake
AlertLogic
• SaaS security & compliance solution built on AWS
• Needed performant and scalable multi-tenant storage for 4000 customers
• 110% data growth and 35% customer growth - annually
Results:
• Moved multiple PB from on-prem NAS into S3 and Glacier
• Analyzed data access patterns, built lifecycle policies and tagged data by customer
• Cross-region replication
• Glacier expedited retrievals for multi-region availability
“We couldn’t have completed the migration, optimized performance, cost optimized &
classified our 4000+ customers without S3’s analytics, tagging and lifecycle features”
13

Use Case: Backup
Southern Oregon University
• Replaced tapes with an AWS Storage Gateway Virtual Tape Library
• Works with existing backup software to store tapes in S3 and Glacier
• Wanted geographic redundancy in case of disaster
Results
• More durable than physical tape
• Improved reliability and performance, using two AWS regions added
geographic site separation
• Lower infrastructure cost, less management, and minor disruption
“Tapes are a headache, prone with hardware failures, offsite storage costs, and constant
maintenance needs. AWS Storage Gateway provided the most cost-effective and simple
alternative.”

Use Case: Backup
Dow Jones
• 30,000 offsite tapes to manage
• Poor ROI and very poor recovery SLAs
• Existing Commvault licenses and talent; moving to cloud
Results
• CommVault manages data directly in S3 and Glacier
• Manages over 4,000 instances at lower cost and higher RTO
• Maintains existing software, skills, catalog, paradigms

Use Case: Disaster Recovery
City of Asheville, NC
• Provides services (power, water, lights, public safety) and revenue (7500 seat US
Cellular Center entertainment venue)
• “DR facility” only two blocks away in hurricane country
• Incomplete application protection and failover testing
Results:
• Built a solution with CloudVelox
• RTO reduced by 75%, achieved geographic diversity
• Agility: low cost pilot light model spins up in moments, and is testable at
any time

What are customers beginning to ask about?
Query in Place
Help me collect,
analyze and
distribute my data
without copying or
moving it into ETL
systems
Hybrid Cloud
Storage
Help me integrate
cloud storage with
my existing skills,
workloads and
workflows
Edge
Help me capture
and move large
amounts to the
cloud but still
support local
workloads and
devices
17

Query in Place: more queries across more data in less time
Now anyone can use SQL to quickly analyze large-scale datasets
S3
Amazon
Athena
Serverless, on-
demand SQL
queries directly
against S3
Unstructured
object data
S3
Amazon Redshift
Spectrum
Serverless, on-
demand SQL
queries against
Redshift and S3
Data warehouses
and unstructured
objects
Redshift

Use Case: hybrid file system workloads
Moderna Therapeutics
• Building new genetic medicines based on mRNA science
• On-prem medical devices that speak NFS
• Cloud-based analytics tools that use S3
Results:
• Storage Gateway NFS (file) interface connects instruments to cloud-based
applications
• Now we can unleash the full power of AWS on “captive” local instrument data

Use Case: Edge
Oregon State University
• Collecting and analyzing 60TB/wk of oceanic and coastal images
• Took weeks/months to upload using many small HDDs
• $4MM+ in infrastructure investment
Results:
• Lower on-prem infrastructure & management cost
• WAN impractical
• Faster than disk
“Snowball Edge lets us collect and migrate TBs of data in days at a fraction of the cost”

Archive
Solutions that leverage AWS for durable,
cost-effective long-term data storage
(often for compliance)
Solutions that leverage AWS
for durable data backup
Backup and Restore BC/DR
Solutions that utilize AWS to enable
recovery strategies focused on RTO
and RPO requirements
Primary Storage/Tiering
Solutions that blend AWS
file, block, object, data formats
with on-premises storage
Storage Competency Partners
Complete list at aws.amazon.com/backup-recovery/partner-solutions/

The AWS Security Ecosystem
What do I have?
Amazon Macie
CloudTrail auditing
S3 Inventory
Reports
Is it protected?
Encryption at rest
and in flight for
nearly all services*
VPC Endpoints
*EFS not in flight
What if I move it?
AWS Key
Management
System
Object Tagging

Don’t lose my data!
Durability: even if it’s offline, make
sure it’s still there
Redundancy: protect against
geographic disasters
- Single facilities
- Networks
- Media
Customers ask infrastructure to do two things
Preserve access to my data!
Availability: help me withstand
worst case scenarios
Replication: let me switch
services between regions easily
so I can vary architectures for
applications

AWS Regional Durability and Redundancy
• Services are delivered by region
• A region contains multiple
Availability Zones
• Each AZ is made up of multiple
discrete facilities
• Each AZ is separated by at least 10
kilometers (6.2 miles)
• Each AZ has redundant power,
networking, connectivity

AWS Availability and Replication
Copy and use data anywhere

What’s new with EBS?
Elastic Volumes
- Expand live volumes
- Change live volume types
- Align storage with use cases
and workloads
- Automate with AWS CloudWatch
and AWS Lambda
Securing public snapshots
Snapshot cost allocation
Tag on Create
2

What’s new with EFS?
Encryption
“At rest” announced
August 14, 2017
2017 regional expansions:
EU (Frankfurt)
Asia Pacific (Sydney)
NFS Enhancements
Elastic Beanstalk integration
2

What’s new with Object Storage?
Classify
Quicksight integration
Amazon Rekognition
Tags, Analysis, Inventory
File Gateway
Manage
New console
Bulk and Expedited
Retrievals
HIPAA on S3TA
Automate
CloudWatch
CloudTrail
Aurora exports
Arcserve VTL support

AWS Storage State of the Union & APN Storage Ecosystem

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to AWS Storage State of the Union & APN Storage Ecosystem

Similar to AWS Storage State of the Union & APN Storage Ecosystem (20)

More from Amazon Web Services

More from Amazon Web Services (20)

AWS Storage State of the Union & APN Storage Ecosystem