Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

(SDD416) Amazon EBS Deep Dive | AWS re:Invent 2014

31,183 views

Published on

Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.

Published in: Technology

(SDD416) Amazon EBS Deep Dive | AWS re:Invent 2014

  1. 1. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. Amazon Elastic Block Store Deep Dive •Larger and faster volumes
  3. 3. EBS !=
  4. 4. A few definitions… •IOPS: Input/output operations per second (#) •Throughput: Read/write rate to storage (MB/s) •Latency: Delay between request and completion (ms) •Capacity: Volume of data that can be stored (GB) •Block size: Size of each I/O (k)
  5. 5. When performance matters, use SSD-backed volumes
  6. 6. •New default volume type for Amazon EBS •Every volume can burst up to 3,000 IOPS -Larger volumes can burst for longer periods •3 IOPS per GB baseline performance -Maximum of 3,000 on a 1TB volume •99% performance consistency
  7. 7. (2) Max IO credit per bucket is 5.4M (1) Always accumulating 3 IOPS per GB per second (3) You can spend up to 3000 IOPS per second 13
  8. 8. 14
  9. 9. 15
  10. 10. 16
  11. 11. VolumeType BootTime AccessTime OS GP2 3:31 4:33 WindowsServer 2012 Standard 4:30 7:16 WindowsServer 2012 GP2 0:36 0:45 CentOS6 Standard 0:57 1:16 CentOS6 40% Reduction in boot times by using General Purpose (SSD)
  12. 12. 80% Cost Savings. 50% more peak I/O with General Purpose (SSD)
  13. 13. 19
  14. 14. •Best for I/O intensive databases that require highest consistency •Provision up to 4,000 IOPS per volume •Supports IOPS to GB ratio of 30 •Designed to offer 99.9% consistency
  15. 15. •Best for cold workloads •Rarely accessed data that needs always on access •IOPS: ~100 IOPS steady-state, with best-effort bursts •Throughput: variable by workload, best effort to 10s of MB/s •Latency: Varies, reads typically ~20-40 ms, writes typically ~2-10ms
  16. 16. General Purpose (SSD) Provisioned IOPS (SSD) Magnetic Recommend use cases Boot volumes Small to med DBs Dev and test I/O intensive Large DBs Cold storage Storage media SSD-backed SSD-backed Magnetic-backed Volume size 1GB-1TB 1GB-1TB 1GB-1TB Max IOPS per volume 3 IOPS/GB Burst up to 3000 IOPS 4,000 IOPS ~100 IOPS Read and write peak throughput 128 MB/s 128 MB/s ~50-90 MBps Max I/O per node(16k) 48,000 48,000 48,000 Peak throughput node 800 MB/s 800 MB/s 800MB/s Latency (random read) 1-2ms 1-2 ms 20-40 ms API name gp2 io1 standard Price* $.10/GB-Month $.125/GB-Month $.065/provisioned IOPS $.05/GB-Month $.05/ 1M I/O
  17. 17. Always use General Purpose (SSD) for boot volumes
  18. 18. •LAW –L –A –W
  19. 19. 1.IOPS:Read/write I/O rate (IOPS) 2.Latency:Time between I/O submission and completion (ms) 3.Throughput:Read/write transfer rate (MB/s) –Throughput = IOPS X I/O Size
  20. 20. (1) Amazon EC2 (2) I/O (4) Amazon EBS (3) Network link
  21. 21. 1.EC2 instance:Network bandwidth(Mbps) 2.EBS-optimized:EC2 instance option (On/Off) 3.Workload:Block size, read/write ratio, serialization 4.Queue depth:The number of outstanding I/Os 5.RAID:Stripe volumes to maximize performance 6.Pre-warm:Eliminates first touch penalty
  22. 22. 1. Amazon EC2 instance • Compute optimized– C3 • Memory optimized- R3 • General Purpose – M3 Amazon EBS Amazon EC2 Select Amazon EC2 instance that has the right network, RAM, and CPU resources for your applications.
  23. 23. 2. Amazon EBS-optimized b
  24. 24. Use EBS-optimized instances for consistent EBS performance
  25. 25. 3. Workload EBS SSD-backed volumes measure I/O size up to 256KB EBS SSD-backed delivers same performance for read and write NEW
  26. 26. 4,000 IOPS PIOPS volume 4,000 IOPS 128MB/s throughput You can achieve 4,000 IOPS when driving smaller I/O You can achieve up to 128MB/s when driving larger I/O
  27. 27. 4,000 IOPS PIOPS volume 4,000 IOPS 128MB/s throughput 4,000 X 4KB = 16MB/s 4,000 X 32KB =128MB/s 8,000 x 8KB = 64MB/s 4,000 x 8KB = 32MB/s 2,000 x 128KB=256MB/s 1,000 x 128KB =128MB/s
  28. 28. Block (I/O) size determines whether your application is IOPS bound or throughput bound
  29. 29. 4. Queue depth An I/O Amazon EBS After it’s gone, it’s gone Amazon EC2 Queue depth is the pending I/O in flight
  30. 30. 41
  31. 31. I/O Latency
  32. 32. 0.075 35.1 0 5 10 15 20 25 30 35 40 1 4 8 12 16 20 24 28 32 Latency TP90 ( ms) Queue Depth Random READ latency across various QDs Latency (TP90) Read latency linearly increases with increase in queue depth
  33. 33. 0.075 35.1 2.09 1,865 4,152 3,851 - 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 0 5 10 15 20 25 30 35 1 4 8 12 16 20 24 28 32 Latency TP90 ( ms) Queue Depth 16 KB random READ IOPS, Latency across various QDs Latency (TP90) Avg Read IOPS IOPS Queue depth of one has the lowest latency, but has the lowest IOPS
  34. 34. 0.075 35.1 2.09 1,865 4,152 3,851 - 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 0 5 10 15 20 25 30 35 1 4 8 12 16 20 24 28 32 Latency TP90 ( ms) Queue Depth 16 KB random READ IOPS, Latency across various QDs Latency (TP90) Avg Read IOPS IOPS Queue depth between 4-8 has the optimal IOPS and latency performance
  35. 35. 0.075 35.1 2.09 1,865 4,152 3,851 - 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 0 5 10 15 20 25 30 35 1 4 8 12 16 20 24 28 32 Latency TP90 ( ms) Queue Depth 16 KB random READ IOPS, Latency across various QDs Latency (TP90) Avg Read IOPS IOPS Higher queue depths have negative impact on IOPS and latency
  36. 36. 0.08 7.71 845 4,152 0 500 1,000 1,500 2,000 2,500 3,000 3,500 4,000 4,500 0 1 2 3 4 5 6 7 8 9 10 1 4 8 12 16 20 24 28 32 Latency TP90 ( ms) Queue Depth 16 KB random WRITE IOPS, Latency across various QDs Latency (TP90) AvgIOPS IOPS Write latency, queue depth and IOPS interaction is similar to that of read latency
  37. 37. Optimal queue depth to achieve lower latency and highest IOPS is typically between 4-8; ~1 QD per 500 IOPS Amazon EBS-optimized offers consistent latency experience
  38. 38. 5. RAID • Increase performance or capacity or both • Stripe multiple volumes to achieve 10’s of TB and up-to 48,000 IOPS per Instance • Don’t mix volume types • Typically RAID 0 or LVM stripe • Avoid RAID for redundancy Amazon EBS Amazon EC2
  39. 39. Maximum performance per instance 12×400 GB PIOPS, pre-warmed, RAID 0 LVM, Stripe size 128 KB, attached to CR1 instance
  40. 40. Use stripe size of 128KB or 256KB
  41. 41. 6. Pre-warminghttp://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-prewarm.html
  42. 42. Use large block size to speed up your pre-warming. ex: sudoddif=/dev/xvdfof=/dev/xvdfconv=notruncbs=1M
  43. 43. Final tips •Ext4 or XFS (understand journal impact!) •Amazon EBS encryption has no visible IOPS performance impact for many Amazon EC2 instances •Alignment can matter, check tools use 4k
  44. 44. Workload/ Software Typical block size Random/ Seq? Max EBS @ 500 Mb/s Max EBS @ 1 Gb/s MaxEBS @ 10Gb/s instances Oracle DB Configurable:2 KB–16 KB Default 8 KB random ~7800 IOPS ~15,600 IOPS ~96,000 IOPS Microsoft SQL Server 8 KB w/ 64 KB extents random ~7800 IOPS ~15,600 IOPS ~80,000 IOPS MySQL 16 KB random ~4000 IOPS ~7,800 IOPS ~48,000 IOPS PostgreSQL 8 KB random ~7,800 IOPS ~15,600 IOPS ~96,000 IOPS MongoDB 4 KB serialized ~15,600 IOPS ~31,000 IOPS ~96,000 IOPS Cassandra 4 KB random ~15,600 IOPS ~31,000 IOPS ~96,000 IOPS GlusterFS 128 KB sequential ~500 IOPS ~1000 IOPS ~6,000 IOPS
  45. 45. EBS-optimized Oh, YEAH!! Amazon EC2 A “boatload” of I/O Right sized EBS
  46. 46. 1.EC2 instance:Network transfer rate (Mbps) 2.EBS-optimized:EC2 instance option (On/Off) 3.Workload:Block size, read/write ratio, serialization 4.Queue depth:The number of outstanding I/Os(#) 5.RAID:Stripe volumes to maximize performance 6.Pre-warm:Eliminates first touch penalty
  47. 47. Performance (per volume) GP2 PIOPS GP2 PIOPS Max Size 1 TB 1 TB 16 TB 16 TB Max IOPS 3,000 4,000 10,000 20,000 Max Throughput 128 Mbps 128 Mbps 160 MBps 320 MBps PREVIOUSLY W/ LARGER & FASTER VOLUMES
  48. 48. Amazon EBS larger and faster volume types General Purpose (SSD) Provisioned IOPS (SSD) Magnetic Recommend use cases Boot volumes Small to med DBs Dev and test I/O intensive Large DBs Cold storage Storage media SSD-backed SSD-backed Magnetic-backed Volume size 1GB-16TB 4GB-16TB 1GB-1TB Max IOPS per volume 10,000 IOPS 20,000 IOPS ~100 IOPS burst < 1TB to3000 IOPS baseline baseline Read and write peak throughput 160 MB/s 320 MB/s ~50-90 MBps Max IOPS per node (16k) 48,000 48,000 48,000 Peak throughput node 800 MB/s 800 MB/s 800MB/s Latency (random read) 1-2ms 1-2 ms 20-40 ms API name gp2 io1 standard Price* $.10/GB-Month $.125/GB-Month $.065/provisioned IOPS $.05/GB-Month $.05/ 1M I/O
  49. 49. 64
  50. 50. 66
  51. 51. • 2-tiered key hierarchy using envelope encryption • Unique data key encrypt customer data • AWS KMS master keys encrypt data keys • Benefits of envelope encryption: • Limits risk of a compromised data key • Better performance for encrypting large data • Easier to manage a small number of master keys than millions of data keys Master Key(s) Data Key 1 S3 object EBS volume Redshift cluster Data Key 2 Data Key 3 Data Key 4 Custom application AWS KMS
  52. 52. Select the right volume for your workload Select the right instance for your workload Take snapshots Use encryption if you need it
  53. 53. Please give us your feedback on this session. Complete session evaluations and earn re:Invent swag. http://bit.ly/awsevals

×