This document provides an overview and comparison of different types of cloud storage: - Object storage is good for large, immutable files like backups and media but more expensive for frequent access. Block storage provides consistent performance for databases and VMs. File storage can migrate legacy systems but is more expensive and slower than block storage. - Performance of block storage varies by provider but Google generally offers the best performance for the price. File storage is not recommended for databases due to access times. Managed databases simplify maintenance but have limited scalability and control. - Kubernetes can automatically provision different storage types using StorageClasses and PersistentVolumeClaims, integrating cloud storage with containers.

1. A GUIDE TO
CLOUD STORAGE
Matej Čapkovič

2. Agenda
1. Overview of all types of cloud storage
2. Performance comparison
3. Usage in Kubernetes

3. 1. Storage Types
● Object storage
● Block storage
● File storage
● Managed databases and others

4. Object Storage
● Good for web content, archives, big data analytics, backups, etc.
● Almost unlimited capacity
● High availability
● Immutable files, versioning
● Accessible via REST api, libraries, tooling
● Access management, metadata, encryption
● Web serving (+upload)
● Cheaper options for infrequent access
● Pay for size and operations

5. Block Storage
● Used as persistent disks in virtual machines (or kubernetes)
● Usually network-attached to single VM
● Ideal for databases
● Can be zonal, regional or local
● HDD or SSD
● Consistent I/O performance and low-latency
● Encryption
● Snapshots
● Pay for size, not for operations

6. File Storage
● Network storage
● Attached via NFS, SMB…
● ReadWriteMany - accessible from multiple machines
● Lower performance compared to block storage
● Can be really slow in some scenarios
● In some clouds also regional for higher availability
● Good for migrating legacy systems to the cloud
● Pricing varies, more expensive than block storage

7. Managed databases
● Cloud providers can manage many types of DBs for you
○ key-value
○ document
○ relational
○ graph, queues, time-series...
● Higher costs
● Zero maintenance
● Vendor databases with awesome scalability and high availability
● Popular open-source databases in HA setup

8. 2. Performance Comparison

9. Object Storage

10. Price for standard storage (EU, single region, per GB, per month):
● Google Cloud Storage: $0.020, $0.023 or $0.026 (depending on region)
● AWS S3: $0.023
● Azure Blob: $0.0196 or $0.022 (depending on region)
Price for egress traffic (per GB, >10+ TB* / <10TB)
● Google: $0.08 / $0.11
● AWS: $0.085 / $0.09
● Azure: $0.083 / $0.087
* can be even cheaper for larger amounts
Price

11. Reliability
Durability (can you lose the data?):
● AWS: 99.999999999%
● Google: 99.999999999%
● Azure: 99.999999999%
Availability SLA (can you read the data?):
● AWS: 99.9%
● Google: 99.9%*
● Azure: 99.9%
* 99.95% for multi-regional

12. Performance
http://blog.zachbjornson.com/2015/12/29/cloud-storage-performance.html

13. Differences?
● Incompatible protocols
● S3 has no multi-region buckets, you need to pay for multiple buckets and
synchronization traffic
Verdict: Basically no difference between
providers. I wouldn’t focus on
object storage differentiation if
I’m about to pick the best
provider.

14. Block Storage

15. Price
● Amazon Elastic Block Store (EBS): 3 types, HDD, SSD and faster SSD, NVMe
● Azure block storage (called Managed Disks): standard on HDD or 2 levels of SSD and ultra SSD
● Google Persistent Disks (PDs): standard, SSD option or local SSD
Prices (EU, per GB / month in USD):
AWS Azure GCP
HDD $0.0475 ~ $0.054 $0.04 (2) $0.040 ~ $0.052
SSD $0.1311 (1) $0.1379 $0.170 ~ $0.221
SSD (low perf.) $0.1045 ~ $0.119 $0.075 (2)
Local / Ultra / NVMe $0.1559 (3) $0.080 ~ $0.096
1) +$0.0684 per iops/month 2) +$0.0005 per 10,000 ops 3) +0,0644 per iops/month + $1,3 per MB/s/month

16. Performance - Google Persistent Disk
Linear performance increase with disk size:
Standard: iops: 0.75/1.5 read/write IOPS/GB
throughput: 0.12 MB/s per GB
SSD: iops: 30 IOPS/GB
throughput: 0.48 MB/s per GB
NVMe: iops: 453.3/240 read/write IOPS/GB
throughput: 1.77/0.94 MB/s per GB

17. Performance - Amazon Elastic Block Store
General Purpose SSD (gp2) Volumes:
3 IOPS* per GiB of volume size, 0.75 MB/s* per GB
Provisioned IOPS SSD (io1) Volumes:
Configurable IOPS / throughput
Throughput Optimized HDD (st1) Volumes:
Throughput 0,04 MB/s* per GB
Cold HDD (sc1) Volumes:
Throughput 0,012 MB/s* per GB
* burst credits can increase performance

18. Performance - Azure Managed Disks
Non-linear performance increase:
IOPS / GB 4 GB 8 16 32 64 128 256 512 1024 2048
Premium SSD 120 120 120 120 240 500 1100 2300 5000 7500
Standard SSD 120 120 120 120 240 500 500 500 500 500
Standard HDD 500 500 500 500 500 500 500 500 500 500
Ultra 1200 2400 4800 9600 19200 38400 76800 160000
Premium SSD 1TB: 5 IOPS per GB, 0.2 MB/s per GB
Standard SSD 1TB: 0.5 IOPS per GB, 0.6 MB/s per GB
Standard HDD 1TB: 0.5 IOPS per GB, 0.6 MB/s per GB
Ultra 512GB: 312 IOPS per GB, 4 MB/s per GB

19. Findings
● All providers offer similar products
● Some cloud providers monetize throughput, this could be expensive for
high IO disks
● Google has probably the best performance for price
● VM size can be limiting factor for storage performance

20. File storage

21. Pricing
GCP Filestore:
● Premium: $0.33/GB, Standard: $0.22/GB
Amazon Elastic File System (Amazon EFS):
● Standard: $0.30/GB, Infrequent: $0.025/GB, Infrequent Access: $0.01/GB
● Provisioned Throughput: $6 MB/s/Month
Azure Files:
● Premium: $0.288/GB, Standard: $0.06/GB
* pricing can vary in different zones

22. Findings
● Amazon and Google utilizes NFS protocol, Azure SMB
● Google Filestores:
○ Only single zone
○ Storage limit is 63.9TB (similar to single persistent disk)
○ Minimum storage size 1TB or 2.5TB for premium
○ Don't confuse Filestore with Firestore
● Amazon iops scale with capacity, google is static
● Amazon performance:
○ access time being about ~3ms
○ large files work well, but if you have many smaller files it's killer
● Probably not good for databases

23. Managed databases

24. PostgreSQL, MySQL, and SQL Server
● Fully managed
● No OS and software
maintenance
● High availability, failover
● Backups
● Security updates
● Vertical scalability
● More expensive than VMs
● Limited or no horizontal scalability
● No full control
● No performance tuning

25. Redis
● Fully managed
● No OS and software maintenance
● HA setup
● Monitoring
● AWS also supports memcached
● Higher price

26. Cloud native NoSQL / SQL databases
● Massively scalable
● Low and predictable latency
● Integrations with cloud services
● Backend storage for other databases
● Used by many big companies
● Minimal instance can be expensive

27. Kubernetes

28. It works out of the box
1. Create storage class
2. Create persistent volume claim
3. Your storage will be automatically provisioned

29. Examples
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-volume
spec:
storageClassName: faster
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 30Gi
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: faster
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-ssd

30. Questions?