Learning Objectives: - Recognize why and when to use Amazon EFS and the economic benefits versus other solutions - Understand best practices for deploying enterprise applications with Amazon EFS - Learn tips for a successful deployment

1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vince Carreon, Sr. Product Manager – Amazon
EFS
March 27, 2018
Amazon EFS
Enterprise Applications with Amazon EFS

2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What to expect from this webinar
Intro to Amazon EFS
Enterprise applications and Amazon EFS
Things to consider
Q&A

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Why and When to Use Amazon EFS

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What do you think about
when choosing a storage solution?
EconomicsFeatures
and performance
Storage type

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
File
Three types of storage
ObjectBlock

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
File
Three types of storage
Data stored as files in a
directory hierarchy
Shared over a network

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Block
Three types of storage
Data stored as blocks on a
disk or disks
Locally attached

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Object
Three types of storage
Data is stored as an object that’s
identified by a key in a flat space
Simple API to get and put data
based on key

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Why is file storage so popular?
Works natively with operating systems
Provides shared access while providing consistency guarantees
and locking functionality
Provides hierarchical namespace

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
File
How does performance compare?
Object
Block
Latency
Throughput

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Before Amazon EFS… DIY file storageAZ-a
Clients
Storage
volumes
AZ-b
Storage
volumes
File server
File server

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon EFS
A fully managed file service

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key features of Amazon EFS
ScalableSimple Elastic
Highly available and durable

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Designed for a wide spectrum of needs
Scale-out jobs Metadata-intensive jobs
Analytics
Media workflows
Web serving
Content management
Database backups
Enterprise apps
and messaging
Dev tooling
Home directories
Container storage
Low latency and serial I/OHigh throughput and parallel I/O

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon EFS customers
Web serving + content management
Media and entertainment workflowsDatabase backups
Big data and analytics
Enterprise applications
Developer tools Home directories Container storage

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Security model and security features
Control
network traffic
using Amazon VPC security
groups and network ACLs
Control file and
directory access
using POSIX
permissions
Control administrative
access (API access)
using AWS IAM
(action-level and
resource-level permissions)
Encrypt
data at rest
using keys managed
in
AWS KMS

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Access EFS from various environments
Amazon
EFS
Amazon
EC2
Corporate data
center
VMware Cloud
on AWS

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon EFS economics
$0.30/GB-Month (US Oregon, N. Virginia, Ohio)
$0.33/GB-Month (EU Ireland, US N. California)
$0.36/GB-Month (EU Frankfurt)
$0.36/GB-Month (AP Sydney)
No minimum commitments
or up-front fees
No need to provision
storage in advance
No other fees, charges,
or billing dimensions

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Before EFS…costs of DIY file storageAZ-a
Clients
Storage
volumes
AZ-b
Storage
volumes
File server
File server
Amazon EC2
instance costs
Inter-AZ data
transfer costs
Amazon EBS
volume costs

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
TCO example: For storing 500 GB,
Amazon EFS is 70% less than DIY
Amazon EFS cost: (500GB * $0.30/GB-month) = $150 per month
For DIY, you might provision 600 GB of Amazon EBS (i.e., ~85% utilization):
Storage (2x 600 GB EBS gp2 volumes): $120 per month
Compute (2x m4.xlarge instances): $290 per month
Inter-AZ data transfer costs (est.): $130 per month
Total $540 per month

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Where is Amazon EFS available today?
• US West (Oregon)
• US West (Northern California)
• US East (N. Virginia)
• US East (Ohio)
• EU (Ireland)
• EU (Frankfurt)
• Asia Pacific (Sydney)
More coming soon!

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data movement
OnlineOffline
Data security
and management
Complete set of data building blocks
Amazon
EFS
Amazon
EBS
AWS Snow Family
AWS Storage Gateway
Family
AWS Direct Connect
Amazon EFS File Sync
Amazon S3
Transfer Acceleration
Storage Partners
Amazon Kinesis
Data Streams
Amazon Kinesis
Video Streams
Amazon
S3
Amazon
Glacier
AWS KMS
AWS IAM
AWS CloudWatch
AWS CloudTrail
AWS CloudFormation
AWS Lambda
Amazon Macie
AWS QuickSight

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Enterprise Applications and
Amazon EFS

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Enterprise applications
Mission critical applications
Running 24/7
Availability, agility, and flexibility are key

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Why Amazon EFS
Seamless support for existing enterprise applications
Standard file system interface and file system semantics
Amazon EFS
Scalable, elastic, highly available, and durable

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Enterprise Application Examples

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Atlassian Jira
Jira Software is an agile
project management
tool
Use Amazon EFS to
store artifacts in a
common location that
is accessible to multiple
Jira nodes.

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Atlassian Confluence
Confluence is a content
collaboration solution for
the enterprise.
Use Amazon EFS to store
artifacts like attachments,
avatars, icons, and
plugins in a common
location that is accessible
to all Confluence nodes.

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Informatica PowerCenter
PowerCenter is a data
integration solution
Use Amazon EFS to write
files such as cache, source,
and target files.

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IBM MQ
MQ enables applications to
communicate at different
times in many diverse
computing environments.
Use Amazon EFS for
distributed storage to ensure
high availability message data
in case of server failure.

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
TIBCO EMS
EMS is a message-oriented-
middleware that makes it
easy for enterprises to
integrate applications.
Use Amazon EFS to ensure
EMS persisted messages are
always available.

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What to consider

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Do you need an EFS file system?
If you have an application (EC2 or on-premises) or use case that
requires a file system AND
• Requires multi-attach OR
• GBs/s throughput OR
• Multi-AZ availability/durability OR
• Requires automatic scaling (grow/shrink) of storage

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Functional testing
POSIX compliant & compliant with NFS
4.0 & 4.1
But you always need to test your
application

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Choose the performance mode
best suited to your workload
Mode What it is for Advantages Trade-offs When to use
General
purpose
(default)
Latency-sensitive
applications and
general-purpose
workloads
Lowest latencies for
file operations
Limit of 7k
ops/sec
Best choice for
most workloads
Max I/O
Large-scale and
data-heavy
applications
Virtually unlimited
ability to scale out
throughput/IOPS
Slightly
higher
latencies
Consider for large
scale-out
workloads

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Monitor PercentIOLimit
CloudWatch metric
PercentIOLimit shows how close a file system is to reaching the I/O limit of the
General Purpose performance mode.

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
An implication of per-operation
latency: I/O size impacts throughput
of serialized operations
4 KB 32 KB 256 KB 2 MB 16 MB
I/O size
Throughput

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How to take advantage of EFS’s
distributed architecture: Parallelize
0
5000
10000
15000
20000
25000
30000
0 20 40 60 80 100 120 140 160
IOPS
# of Total Threads

43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kernel version and mount options
Recommended
Kernel
version
• Use Linux kernel 4.0+ (e.g., Amazon Linux 17.09.0—use latest
versions, Ubuntu 16.04 or 17.10)
Mount
options
• Mount via NFSv4.1
• Specify 1MB read/write buffers (“rsize”/”wsize”)
• Ensure operations are asynchronous
Recommend the following mount options:
-o nfsvers=4.1, rsize=1048576, wsize=1048576,
hard, timeo=600, retrans=2, async

44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Bursting model: Throughput you can
drive scales as your file system grows
50 MiB/s per TiB continuously
100 MiB/s per TiB in bursts

45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Key Recommendations
• Test your application!
• Use General Purpose mode for lowest latency, Max-I/O for scale-out
• Use Linux kernel version 4.0 or newer, mount via NFSv4.1
• To optimize, look for opportunities to:
• Aggregate I/O
• Perform async operations
• Parallelize
• Cache
• Don’t forget to check your burst credit earn/spend rate when testing
– ensure sufficient amount of storage

46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Wrapping up

47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How to learn more: aws.amazon.com/efs
Feature blogs
White papers
Ref architectures
TCO calculator
10-minute tutorials
Documentation

48. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you
Poll and Q&A next