Maximizing Amazon EC2 and Amazon EBS performance
•
14 likes•8,160 views
Learn tips and techniques that will improve the performance of your applications and databases running on Amazon EC2 instance storage and/or Amazon Elastic Block Store (EBS). This advanced session discusses when to use HI1, HS1, and Amazon EBS. We will share an "under the hood" view to tune the performance of your Elastic Block Store and best practices for running workloads on Amazon EBS, such as relational databases (MySQL, Oracle, SQL Server, Postgres) and NoSQL data stores, such as MongoDB and Riak.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Maximizing Amazon EC2 and Amazon EBS performance
Similar to Maximizing Amazon EC2 and Amazon EBS performance (20)
More from Amazon Web Services
More from Amazon Web Services (20)
Recently uploaded
Recently uploaded (20)
Maximizing Amazon EC2 and Amazon EBS performance
- 1. Maximizing EC2 and Elastic Block Store Disk Performance Martin Elwin, Manager, Solutions Architecture, AWS
- 2. For most builders AWS is get in and go!
- 3. What is Amazon EBS? • Network block storage • Attach to EC2 within the same AZ • Point-in-time snapshots to S3 • Optional managed encryption • Provision and pay just for what you need
- 4. EBS Volume Types • General Purpose (SSD) • Provisioned IOPS (SSD) • Magnetic When performance matters, use SSD-backed volumes
- 5. Amazon EBS General Purpose (SSD) • New default volume type for EBS • Every volume can burst up to 3,000 IOPS • Larger volumes can burst for longer periods • Token bucket burst algorithm • 3 IOPS per GB baseline performance • Maximum of 3,000 on a 1TB volume • Throughput up to 128MB/s per volume • Designed to deliver within 10% of performance 99% of the time • Latency: Single digit milliseconds
- 6. Token Bucket Bucket Size 3000 IOPS in 30 mins = 5.4M tokens Fill rate 3 IOPS * volume size in GB Bucket starts full!
- 7. Amazon EBS PIOPS (SSD) • Provision up to 4,000 IOPS per volume • Supports IOPS to GB ratio of 30 (>133GB to achieve 4000 IOPS) • Designed to deliver within 10% of performance 99.9% of the time • Throughput: up to 128MB/s per volume • Latency: Single digit millisecond NEW • Best for IO intensive databases that require highest consistency
- 8. Amazon EBS Magnetic • IOPS: ~100 IOPS steady-state, with best-effort bursts • Throughput: variable by workload, best effort to 10s of MB/s. • Latency: Varies, reads typically ~20-40 ms, writes typically ~2-10ms • Best for cold workloads
- 9. EBS Volume Types (Cheat Sheet) General Purpose (SSD) Provisioned IOPS (SSD) Magnetic Recommend Use Cases Boot Volumes Low to Med Perf DBs Dev and test I/O intensive High Perf DBs Cold Storage Storage Media SSD-backed SSD-backed Magnetic-backed Volume Size 1GB-1TB 1GB-1TB 1GB-1TB Max IOPS per volume 3 IOPS/GB Burst up to 3000 IOPS 4,000 IOPS ~100 IOPS Read and Write Peak Throughput 128 MB/s 128 MB/s ~50-90 MB/s Max I/O per node 48,000 48,000 ~1000-5000 Peak Throughput Node 800 MB/s 800 MB/s ~800 MB/s Latency (random read) 1-2ms 1-2 ms 20-40 ms API Name gp2 io1 standard Price* $.10/GB-Month $.125/GB-Month $.065/provisioned IOPS $.05/GB-Month $.05/ 1M I/O
- 10. Always use General Purpose (SSD) for boot volumes
- 11. Performance Optimization is Measured by 1. IOPS: 2. Latency: 3. Throughput:
- 12. Four Key Pieces (2) I/O (1) Amazon EC2 (4) Amazon EBS (3) Network link
- 13. 5 areas influencing performance 1. EC2 Instance: 2. EBS Optimized: 3. Workload: 4. Queue Depth: 5. RAID:
- 14. 1. EC2 Instance Amazon EBS Amazon EC2 Select EC2 instance that has the right Network, RAM and CPU resources for your applications.
- 15. 2. EBS-Optimized • Offers “SAN like” experience • All new instance family supports EBS-optimized flag • EBS-optimized now supports up-to 2Gb/s – Drive 16,000 16K IOPS or 256MB/s • EC2 8XL instances support 10Gb/s network • Max IOPS per node supported is ~100,000 IOPS @ 4K IO Instance Type Dedicated EBS (Mbps) Max 16K IOPS* Max MB/s c1.xlarge 1,000 8,000 128 c3.xlarge 500 4,000 64 c3.2xlarge 1,000 8,000 128 c3.4xlarge 2,000 16,000 256 g2.2xlarge 1,000 8,000 128 i2.xlarge 500 4,000 64 i2.2xlarge 1,000 8,000 128 i2.4xlarge 2,000 16,000 256 m1.large 500 4,000 64 m1.xlarge 1,000 8,000 128 m2.2xlarge 500 4,000 64 m2.4xlarge 1,000 8,000 128 m3.xlarge 500 4,000 64 m3.2xlarge 1,000 8,000 128 r3.xlarge 500 4,000 64 r3.2xlarge 1,000 8,000 128 r3.4xlarge 2,000 16,000 256 r3.8xlarge NA 48,000 800 c3.8xlarge NA 48,000 800 i2.8xlarge NA 48,000 800 cc2.8xlarge NA 48,000 800 * Max 2X for 8KB IO & * Max 4X for 4KB IO
- 16. Use EBS-Optimized instances for consistent EBS performance
- 17. 3. Workload • I/O size – Data size of read/write • I/O pattern – Sequential and random • I/O type – Read and write • I/O concurrency – Number of concurrent IO EBS SSD-backed volumes measure IO size up to 256KB EBS SSD-backed delivers same performance for read and write NEW
- 18. IOPS and Throughput limits on EBS 4,000 IOPS PIOPS volume 4,000 IOPS You can achieve 4,000 IOPS 128MB/s throughput when driving smaller IO 4000 IOPS x 32 KB = 128 MB/s You can achieve up to 128MB/s when driving larger IO 500 IOPS x 256 KB = 128 MB/s
- 19. Block (IO) size determines whether your application is IOPS bound or throughput bound
- 20. 4. Queue Depth Queue depth is the pending IO in flight. An I/O Amazon EBS After it’s gone, it’s gone Amazon EC2
- 21. Random Read Latency 0.075 35.1 16 KB random READ IOPS, Latency across various QDs 2.09 1,865 4,152 3,851 4,500 4,000 3,500 3,000 2,500 2,000 1,500 1,000 500 - 35 30 25 20 15 10 5 0 1 4 8 12 16 20 24 28 32 Latency TP90 ( ms) Queue Depth Latency (TP90) Avg Read IOPS IOPS Queue depth of one has the lowest latency, but has the lowest IOPS
- 22. Random Read Latency 0.075 35.1 16 KB random READ IOPS, Latency across various QDs 2.09 1,865 4,152 3,851 4,500 4,000 3,500 3,000 2,500 2,000 1,500 1,000 500 - 35 30 25 20 15 10 5 0 1 4 8 12 16 20 24 28 32 Latency TP90 ( ms) Queue Depth Latency (TP90) Avg Read IOPS IOPS Queue depth between 4-8 has the optimal IOPS and latency performance
- 23. Random Read Latency 0.075 35.1 16 KB random READ IOPS, Latency across various QDs 2.09 1,865 4,152 3,851 4,500 4,000 3,500 3,000 2,500 2,000 1,500 1,000 500 - 35 30 25 20 15 10 5 0 1 4 8 12 16 20 24 28 32 Latency TP90 ( ms) Queue Depth Latency (TP90) Avg Read IOPS IOPS Higher queue depths have negative impact on IOPS and latency
- 24. Optimal queue depth to achieve lower latency and highest IOPS is between 4-8; 1 QD per 500 IOPS EBS-Optimized offers consistent latency experience
- 25. 5. RAID Amazon EBS Amazon EC2
- 26. Maximum performance per instance 12×400 GB PIOPS, RAID 0 LVM, Stripe size 64 KB, attached to CR1 instance • How should you think about taking snapshots on a striped volume? – Quiesce file systems and take snapshot – Unmount file system and take snapshot – Use OS-specific tools
- 27. Use stripe size of 128KB or 256KB
- 28. Cheat Sheet Sample! Storage Workloads on AWS Workload/ Software Typical block size Random /Seq? Typical correct EBS type Max EBS @ 500 Mb/s Max EBS @ 1 Gb/s Max EBS @ 10Gb/s instances Oracle DB Configurable:2 KB –16 KB Default 8 KB random PIOPS/GP2 RAID 0, 2 × 4000 PIOPS: ~7800 IOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS Microsoft SQL Server 8 KB w/ 64 KB extents random PIOPS/GP2 RAID 0, 2 × 4000 PIOPS: ~7800 IOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 20 × 4000 PIOPS: ~80,000 IOPS MySQL 16 KB random PIOPS/GP2 1 x 4000 PIOPS: ~4000 IOPS RAID 0, 2 × 4000 PIOPS: ~7,800 IOPS RAID 0, 12 × 4000 PIOPS: ~48,000 IOPS PostgreSQL 8 KB random PIOPS/GP2 RAID 0, 2 × 4000 PIOPS: ~7,800 IOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS MongoDB 4 KB serialized PIOPS/GP2 RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 8 × 4000 PIOPS: ~31,000 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS Cassandra 4 KB random PIOPS/GP2 RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 8 × 4000 PIOPS: ~31,000 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS GlusterFS 128 KB sequential PIOPS/GP2 1 × 4000 PIOPS: ~500 IOPS RAID 0, 2 × 4000 PIOPS: ~1000 IOPS RAID 0, 12 × 4000 PIOPS: ~6,000 IOPS (768MB/s)
- 29. 5 areas influencing performance 1. EC2 Instance: 2. EBS Optimized: 3. Workload: 4. Queue Depth: 5. RAID:
- 30. Four Key Pieces: Balanced Oh, YEAH!! EBS-optimized A “boatload” of I/O Amazon EC2 Right sized EBS
- 31. Thank you! Maximizing EC2 and Elastic Block Store Disk Performance Martin Elwin, Manager, Solutions Architecture, AWS
Editor's Notes
- 1 Welcome and thank you for taking the time to attend the AWS NYC Summit and this session. 2. I’m Vinay Kumar, Product Manager for Amazon Elastic Block Storage. 3 For the next 50 minutes or so we will be discussing – How to extract the maximum amount of performance from the combination of EC2 and EBS.
- For most builders – AWS is get in and go. We have made it even easier with our recent announcement of General Purpose (SSD) storage. Attach General purpose (SSD) volumes ..you should be just fine. (What is it) the rest, let’s talk about optimizing for performance.
- So – what is EBS? Persistent block level storage - storage volumes that persist independently from the life of the instance. Automatically replicated within the same AZ ..designed for five 9s Do things such as take snapshots (more on that later) Has certain reliability design goals Follows the general AWS approach of choice and pay as you go
- EBS now has 3 volume types – You need to carefully choose the volume type to optimize performance and cost for your applications. Magnetic Standard Guidanace is true for both GP(SSD) and PIOPS
- Newest volume type added Making SSD compelling use case for general purpose use cases. $0.10 per GB …4 months Every volume ..irrespevie of the size can burst to 3000 IOPS for smaller IO size; or 128MB?s for larger block size use cases Smaller volumes can burst up to 30 mins …larger volumes can burst for longer durations. Baseline performance of 3 IOPS per GB. For example, 500GB ..1500 but burst to 3000 99% consitency Latency expectionai is in single digit millit seconds. One of the design goal is to make boot faster. ~6000 Ios linux For database, if your database requires Take away: IF you need a cost effective Smaller volumes burst up to 30 mins ..larger volumes burst for longer periods. For example ..750GB volumes can burst …2Hours.
- The second is EBS PIOPS – Made available in Mid-2012 Backed by SSD Used for heavy-use workloads needing very consistent performance DATABASES! What can I expect from EBS PIOPS ? Specify up front amount from 100 to 4K Provisioned IOPS (PIOPS) 30 IOPS per GB (4K PIOP volume as small as 134 GB) Can achieve single digit millisecond latencies when paired with EBS-Optimized Designed to deliver within 10% of the provisioned IOPS performance 99.9% of the time. Throughput occurs in 16K blocks in PIOPS as a service – that’s a throttle used to constrain the performance you will receive which allows us to maintain the consistency. Which means up to 64 MB/s, as provisioned 4K (PIOP/second) x 16K (Bytes) = 64,000,000 or 64 Mega (million) Bytes/second Latency(much lower than EBS Standard) 1 over the PIOPS or 1 Second/The number of Provisioned –IOPS = Latency For example 1K PIOP volume would be 1/1K or one ms (1/1,000) Or, for example a 4K PIOP volume would be ¼ of a ms latency Price Again, from a technical perspective, request exactly what you need. In TCO discussions – want to make to finance folks head spin – because it is hard to quantify. When we don’t need it, we’ll make it go away and we’ll stop paying for it. When we need it again we’ll make it re-appear and we’ll begin paying for it. If we need it to have different characteristics we’ll tell it to change.
- There are two “Product” choices when it comes to EBS. The first we will review is EBS Standard - Our original block storage as a service Behaviors are: Bursty I/O – swap to disk, load OS and App Heavy sequential READ – FAST! Original design consideration was as AMI image for EC2 What can I expect from Standard EBS? ~100 IOPS steady state Best effort bursts to hundreds of IOPS (maybe 700 up to 1,100) Provided on a “best effort” basis (steady state loads will normalize to the ~100 number) You’ll observe short-term bursts of accepted work but then will return to the wall rate of ~100 IOPS Throughput As a result of variable IOPS Standard EBS has a variable throughput Latency Reads < 20 ms Writes < 10 ms Capacity As provisioned, up to 1 TB Use case for EBS Standard? Well-suited for use as boot volumes or applications where I/O can be bursty. - it is designed as a root volume service designed for EC2 to start and stop instances.
- Putting it all together – Use SSD-backed for performance GP2 offers the best price performance Always use gp2 volumes for boot.
- Customers generally use
- Breeze through .. Level set ..so that we are talking the same there. I’m
- To achieve end-to-end performance optimization, you need to look at these 4 components. For many of you this is review – but let’s take a moment and get everyone on the same page. Key pieces are: 1-TheEC2 instance 2-The EBS Volume 3-The network link that connects the EBS volume and the EC2 instance 4-The Input and Output - The individual transits back and forth between EBS and EC2 These four key components work together. So, to maximize performance it is important to arrange them into a balanced system.
- There are 6 available tools to balance the systems and we are going ..
- EBS is a network attached block storage. Network connectivity is super important for performance Bandwidth: Instances are important: Reminder that EBS is a network block store Customers generally use memory optimized insances R3 Network resources available per instance is very very important Low – moderate – high Higher the resource ..higher the IOPS.
- First – Choosing the right instance is important. EC2 is classified into memory, compute and general purpose Instance network resource = low, medium, and high 1,000 Mbits/sec = 128MB/s Key take-aways from this slide: (This table is a sample) 1-Maximum throughput delivered by EBS is limited by EC2 bandwidth (see this fairly regularly as an SA) 2-Different EC2 instances have different levels of bandwidth capacity (review the AWS site details) 3-You can compute the max MB/s for a given configuration 4-There is a CPU factor 6-Cluster compute instances (list) are on the “Cluster Compute Network” which means: 10 Gigabit Full bi-section Non-blocking Not linear in relationship to cost Does not include option to select “EBS Optimized” – that’s already done.
- You’ll hear this many times from me and my AWS colleagues - Use PIOPS and EBS Optimized for your databases Key Messages: Use EBS SSD backed for consistency of latency and throughput Use EBS Optimized to ensure that network traffic isn’t interfering with storage traffic
- Every application IO workload is different. They are characterized by Consistency is the keyword PIOPS, because of its “throttled” say computer-generated behavior is going to provide a very consistent behavior – unlike many physical devices it has the same: 1- performance for random and sequential workloads. (Physical devices don’t do that). 2- The same number of IOPS for reads and writes 3- The same I/O size – 16K – All the time Oracle, MySQL, mongo PIOPS is Optimized for database workloads.
- MySQL default block size is 16k, SQL Server default block size is 8k, MongoDB default block size is 4K, and Oracle default block size is 8k Cloudera and MapR has a default block size of 64MB, DWH has block size in 1MB-2MB range
- Queue depth is the pending IO in flight. Another simple way to think about it is ..you need to keep EBS volume busy all the time. Hence, you need to have IO in queue to be processed by EBS. You can’t put lost IOPS back on the clock. Discuss the CloudWatch metrics for Queue Depth
- RAID is a useful tool It allows you to aggregate the performance of multiple EBS volumes to deliver whatever IOPS number you need to get to. Again, think end-to-end and avoiding constraints. Considerations: If your application (such as MS SQL Server) has the ability to manage storage – typically will do a better job at this. Just give your software EBS volumes. RAID 5 and RAID 6 are not recommended (strong statement) – please do not use. Parity information is distributed among the drives. Which means a write to every disk. A write occurs for every volume in RAID 5 and RAID 6. And contention potentially occurs with the other writes increasing latency. RAID 0, RAID 1 and in extreme cases RAID 10 make sense. Both Standard EBS and PIOPS EBS both operate at much better Annual Failure Rate (AFR) than standard physical disk at point 1 percent (0.1%) to point 4 percent (0.4%) per year. Which compares very favorably to physical disk at 4.0%. Certainly the design consideration should be that “things fail”. RAID in AWS is on top of an already existing improved redundancy level. We have team using LVM direct to disk. No problem there. More users are using MDADM. Lower volume of support requests on that side. We also have teams using MDADM to manage the storage and LVM above that to manage the volume you construct above that storage. For example – MongoDB: You create one big RAID so that anyone any one of the logical storage volumes you construct can use the IOPS you have available. You can also stripe multiple volumes together to achieve up to 48,000 IOPS when attached to larger EC2 instances.
- Key Messages 1-On the larger instances you can drive pretty awesome read performance – in the range of 50K READ IOPS. The volume need not be attached to a running instance in order to take a snapshot. These snapshots can be used to create multiple new Amazon EBS volumes, expand the size of a volume, or move volumes across Availability Zones. The snapshots can be shared with specific AWS accounts or made public. Considerations: When you do RAID the single biggest operational consideration is executing snapshots. Because its possible to have a write that’s happening to multiple different EBS volumes at slightly different latencies, then taking a snapshot from those volumes has be done in such a way that is more consistent. Which means that the file system cannot be in a state of change when a snapshot is taken.
- Discussion points 1-This is a preview of work that is underway and planned for publishing 2-Different workloads with different default characteristics 3-Optimized configuration for each workload Zeroing in on just one – Mongo 1-Typical block size – 4 KB 2-Typical I/O Type – serialized (interesting, we’ll get to that shortly) 3-Recommended EBS Product – EBS PIOPS 4-Throughput on different instance link capacity (linear with PIOPS 4 volumes and 8 volumes both at RAID 0 CC though jumps off the chart at 3x the IOPS
- So to re-cap 1-EC2 instances have different workload focus and network config – select one that fits your goals 2-EBS Optimized, in the context of this discussion is almost always a “yes please” 3-PIOPS is built for database workloads – enough said 4-Manage your queue depth 5-Understand and match your workload blocksize to PIOPS 6-Yor workload has its own tuning options and behavios – those matter too
- Now we have everything optimized and working as a “system” Which results in Maximized EC2 and Elastic Block Store Disk Performance Oh, YEAH!
- 1 Welcome and thank you for taking the time to attend the AWS NYC Summit and this session. 2. I’m Vinay Kumar, Product Manager for Amazon Elastic Block Storage. 3 For the next 50 minutes or so we will be discussing – How to extract the maximum amount of performance from the combination of EC2 and EBS.