Cloud Storage with AWS:
EFS vs EBS vs S3
Cloud Storage with AWS
Cloud storage is a critical component of cloud computing, holding the information
used by applications. Big data analytics, data warehouses, Internet of Things,
databases, and backup and archive applications all rely on some form of data
Amazon Simple Storage Service (Amazon
Object storage, to make data available through an API that can be accessed
It is secure, durable, and highly scalable, and applications can access storage
directly. Amazon S3 is used for content storage and distribution, large-scale web
development, data lakes for analytics workloads, and cloud-native applications
going directly across the web.
Amazon Elastic File System (Amazon EFS)
File storage, to make data available to one or more EC2 instances via a file system
interface and file system access semantics.
It delivers a file system interface with standard file system access semantics for
Amazon EC2 instances. Amazon EFS grows and shrinks capacity automatically, and
provides high throughput with consistently low latencies. Amazon EFS is ideal for
Big Data analytics, media processing workflows, content management, web
serving, and home directories.
Amazon Elastic Block Storage (Amazon
Block storage, to make data available as a virtual volume attached to a single
Amazon EC2 compute instance.
Amazon EBS is designed for workloads that require persistent storage accessible
by single EC2 instances. Typical use cases include boot volumes, transactional and
NoSQL databases (like Microsoft Exchange, Cassandra and MongoDB), Big Data
analytics platforms (like Hadoop, Amazon EMR, and HortonWorks), stream and log
processing applications (like Kafka and Splunk), and data warehousing
applications (like Vertica and Cassandra).
Highly affordable long-term storage that can replace tape for archive and
Amazon Glacier is a solution for customers who want low-cost storage for
infrequently accessed data. It can replace tape while assisting with compliance in
highly regulated organizations like healthcare, life science, and financial services.
AWS Storage Gateway
A software appliance that links your on-premises storage environment to Amazon
It provides low-latency access to frequently accessed data and maintains durable
remote volumes, to help you match the value of your storage with the value of
your data. Use it with existing backup and recovery applications to replace tape
automation without disrupting existing processes, or supplement “bursty” on-
premises workloads with storage capacity on demand.
Cloud Data Migration Services
A portfolio of services to help simplify and accelerate moving data of all types and
sizes into and out of the AWS cloud.
These services help you do things like securely and quickly move multi-petabyte
archives, capture continuous streaming data from multiple sources, and accelerate
network transfers on existing infrastructure.
Differences: EFS vs EBS vs S3 vs Glacier
Why would I use EBS over EFS? They have the same use cases but minor semantic
Why choose S3 over EFS? They both store files, scale and are replicated.
Generally Available (out of preview), but may not yet be available in your region
Network filesystem (that means it may have bigger latency but it can be shared
across several instances; even between regions)
It is expensive compared to EBS (~10x more) but it gives extra features.
It's a highly available service.
It's a managed service
You need an EC2 instance to attach it to
Can be accessed by multiple EC2 instances simultaneously
A block storage (so you need to format it). This means you are able to choose which type of file
system you want.
As it's a block storage, you can use Raid 1 (or 0 or 10) with multiple block storages
It is really fast
It is relatively cheap
With the new announcements from Amazon, you can store up to 16TB data per storage on SSD-s.
You can snapshot an EBS (while it's still running) for backup reasons
But it only exists in a particular region. Although you can migrate it to another region, you cannot
just access it across regions (only if you share it via the EC2; but that means you have a file server)
You need an EC2 instance to attach it to
An object store (not a file system).
You can store files and "folders" but can't have locks, permissions etc like you would with a traditional file system
This means, by default you can't just mount S3 and use it as your webserver
But it's perfect for storing your images and videos for your website
Great for short term archiving (e.g. a few weeks). It's good for long term archiving too, but Glacier is more cost efficient.
Great for storing logs
You can access the data from every region (extra costs may apply)
Highly Available, Redundant. Basically data loss is not possible (99.999999999% durability, 99.9 uptime SLA)
Much cheaper than EBS.
You can serve the content directly to the internet, you can even have a full (static) website working direct from S3, without an EC2
Long term archive storage
Extremely cheap to store
Potentially very expensive to retrieve
Takes up to 4 hours to "read back" your data (so only store items you know you
won't need to retrieve for a long time)
Comparison of Cost
S3-IA (announced in 2015.09): $0.0125/Month (+$0.01/gig retrieval charge)
EBS: $0.05-0.1/Month (depends on speed - SSD or not) + IOPS costs