© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or i...
A “Normal” Hard Drive
A few definitions…
So.. what storage workloads?
What is Amazon EBS?
Very flexible service with lots of choice
–  Used with Amazon EC2 instances
–  Attach/detach/copy/dele...
EBS =
EC2 EBS
Amazon EBS Standard
Amazon Elastic
Block Storage
(EBS)
IOPS: ~100 IOPS steady-state, with best-effort bursts
Throughput: v...
EBS STANDARD
≠
EBS PIOPS
EBS PIOPS
Amazon Elastic
Block Storage
(EBS)
IOPS: Within 10% of up to 4000 IOPS,
99.9% of a given year, as provisioned.
T...
❶ Select a new type of Provisioned IOPS volume
❸ Specify the number of I/O operations per
second your application needs, u...
I/O Characteristics
•  I/O size
–  4 KB to 64 MB
•  I/O pattern
–  Sequential and random
•  I/O type
–  Read and write
•  ...
EC2 EBS
16 KBytes
A bigger I/O
EBS
>16 KB? = more than one I/O
EC2
EC2
A smaller I/O
EBS
< 16 KB? = no “bonus” random I/Os
EC2
An I/O
EBS
Once it’s gone, it’s gone.
EC2
an I/O
EBS
Just because EC2 sends more work
doesn’t mean there’s enough
IOPS to handle it!
EC2 EBS
Just because Amazon EC2 sends more
work doesn’t mean there’s enough
bandwidth to handle it!
EC2
An I/O
Without more bandwidth,
more EBS volumes or higher PIOPS won’t help!
EC2
An I/O
EBS
Adding bandwidth alone
or increasing instance size won’t help!
EBS-Optimized
USE EBS PIOPS
WITH
EBS-OPTIMIZED EC2 INSTANCES
BALANCE
IOPS / BANDWIDTH / THROUGHPUT
EBS-Optimized
EC2
EBS-Optimized
Network interference tests:
No impact on IOPS or
Amazon EBS throughput
Row Labels AvgBW AvgIOPs
m3.2xlarge (...
Architecture – What Else Uses Network?
Architecting for Performance
IOPS consistency requires
EBS-optimized instances
Maximum throughput delivered by
Amazon EBS ...
Smaller I/O (4 KB, 16 KB)
•  Why are 4 KB I/O size in
sequential operations
driving greater than 4000
IOPS?
•  Why is m1.l...
Larger I/O (128 KB, 512 KB)
•  Why am I seeing only 462
IOPS on a volume?
•  Why there is no difference
in performance for...
Write Latency
•  Database applications care
about latency as much as IOPS
delivered
•  There is an Interdependency
among I...
Read Latency
•  Reads can take advantage of a
deeper queue
•  Current guidance is queue
depth of 1 for every 250 IOPS
•  E...
•  Performance requirements may be driven by IOPS or latency or both
•  Recommendation is to start with queue depth of 4 a...
•  Typically 5%, extreme worst case of 50% performance
reduction in IOPS and latency when volumes are used without
pre-war...
What about RAID?
Amazon Elastic
Block Storage
(EBS)
Amazon Elastic
Compute Cloud
(EC2)
•  Don’t do
RAID 5, 6
Amazon Elasti...
Architecting for Performance: RAID
Stripe number of volumes to
drive higher IOPS and
throughput
–  RAID 0 or RAID 10
Takin...
Workload/
Software
Typical block
size
Random
/Seq?
Typical
correct
EBS type
Max EBS @
500 MB/s
Max EBS @
1 GB/s
Max EBS @ ...
Workload/
Software
Typical
block
size
Random
/Seq?
Typical
correct
EBS type
Max EBS @
500 MB/s
Max EBS @ 1 GB/s Max EBS @ ...
Cheat Sheet! Storage Workloads on AWS
Workload/
Software
Typical
block
size
Random
/Seq?
Typical
correct
EBS type
Max EBS ...
Performance – Extra-large Production Scale
•  Leverage SSD instance type
(hi1.4xlarge)
o  2 × 1 TB SSD storage (ephemeral
...
STABLE
Testing Random 4 KB Reads
EBS PIOPS+ SSD
What about Capacity Cost?
cc2.8xlarge
48x
1TB
EBS
VS.
hs1.8xlarge hs1.8xlarge
$7312 on-demand,
$6128 effective 3 YR reserv...
Performance / Stability Tips
•  Ext4 or XFS (understand journal impact!)
•  nobarrier, noatime, noexec, nodiratime
•  Rais...
From 1000 to 98,000: What does that mean?
So.. all storage workloads!
© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or i...
Upcoming SlideShare
Loading in...5
×

AWS Summit London 2014 | Maximising EC2 and EBC Performance (400)

1,170

Published on

This advanced technical session is ideal for customers that are looking to maximise the performance of AWS Elastic Block Store (EBS) storage to support workloads with demanding IO performance requirements. If you need to run high IO workloads on EBS such as NoSQL or RBDMS systems then attend this session to find out how to optimise your EBS configuration to enable this.

Published in: Technology, Business
1 Comment
7 Likes
Statistics
Notes
  • Why do u keep disabling Save!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
1,170
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
2
Comments
1
Likes
7
Embeds 0
No embeds

No notes for slide

Transcript of "AWS Summit London 2014 | Maximising EC2 and EBC Performance (400)"

  1. 1. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Maximizing Amazon EC2 & Amazon EBS Performance Carlos Conde – Head of AWS Evangelism EMEA
  2. 2. A “Normal” Hard Drive
  3. 3. A few definitions…
  4. 4. So.. what storage workloads?
  5. 5. What is Amazon EBS? Very flexible service with lots of choice –  Used with Amazon EC2 instances –  Attach/detach/copy/delete volumes –  Point-in-time snapshots of volumes -> Amazon S3 –  Automatically replicated within its Availability Zone to protect from component failure –  Paying a low price for only what you provision
  6. 6. EBS =
  7. 7. EC2 EBS
  8. 8. Amazon EBS Standard Amazon Elastic Block Storage (EBS) IOPS: ~100 IOPS steady-state, with best-effort bursts Throughput: variable by workload, best effort to 10s of MB/s. Latency: Varies, reads typically <20 ms writes typically <10 ms Capacity: As provisioned, up to 1 TB
  9. 9. EBS STANDARD ≠ EBS PIOPS
  10. 10. EBS PIOPS Amazon Elastic Block Storage (EBS) IOPS: Within 10% of up to 4000 IOPS, 99.9% of a given year, as provisioned. Throughput: 16 KB per I/O = up to 64 MB/s, as provisioned. Latency: low and consistent. Second / IOPS Capacity: As provisioned, up to 1 TB *
  11. 11. ❶ Select a new type of Provisioned IOPS volume ❸ Specify the number of I/O operations per second your application needs, up to 4000 IOPS per volume. The volume will deliver the specified I/O operations per second. ❷ Specify the volume capacity
  12. 12. I/O Characteristics •  I/O size –  4 KB to 64 MB •  I/O pattern –  Sequential and random •  I/O type –  Read and write •  PIOPS always measures I/O in terms of 16 KB or smaller •  PIOPS delivers same number of IOPS for sequential and random I/O •  PIOPS delivers same number of IOPS for reads or writes PIOPS is optimized for database workloads
  13. 13. EC2 EBS 16 KBytes
  14. 14. A bigger I/O EBS >16 KB? = more than one I/O EC2
  15. 15. EC2 A smaller I/O EBS < 16 KB? = no “bonus” random I/Os
  16. 16. EC2 An I/O EBS Once it’s gone, it’s gone.
  17. 17. EC2 an I/O EBS Just because EC2 sends more work doesn’t mean there’s enough IOPS to handle it!
  18. 18. EC2 EBS Just because Amazon EC2 sends more work doesn’t mean there’s enough bandwidth to handle it!
  19. 19. EC2 An I/O Without more bandwidth, more EBS volumes or higher PIOPS won’t help!
  20. 20. EC2 An I/O EBS Adding bandwidth alone or increasing instance size won’t help! EBS-Optimized
  21. 21. USE EBS PIOPS WITH EBS-OPTIMIZED EC2 INSTANCES
  22. 22. BALANCE IOPS / BANDWIDTH / THROUGHPUT
  23. 23. EBS-Optimized EC2
  24. 24. EBS-Optimized Network interference tests: No impact on IOPS or Amazon EBS throughput Row Labels AvgBW AvgIOPs m3.2xlarge (EBS-optimized) no network load random     read 57,542 3,596 write 61,713 3,857 rw (70/30) 66,997 4,186 sequential     read 61,708 3,856 write 61,651 3,853 rw (70/30) 66,996 4,187 with network load-test1 random     read 59,835 3,739 write 63,407 3,962 rw (70/30) 68,859 4,303 sequential     read 61,736 3,858 write 63,360 3,959 rw (70/30) 68,859 4,302
  25. 25. Architecture – What Else Uses Network?
  26. 26. Architecting for Performance IOPS consistency requires EBS-optimized instances Maximum throughput delivered by Amazon EBS is limited by Amazon EC2 bandwidth EBS throughput = EBS IOPS × Block size Ex: 64 MB/s = 4000 IOPS × 16 KB Instance   vCPU   EBS Optimized  Max MB/s   Max 16k IOPS   t1 micro   1   No   32MB/s   2000   m1.small   1   No   64MB/s   4000   m1.medium   1   No   64MB/s   4000   m1.large   2   Yes   64MB/s   4000   m1.xlarge   4   Yes   128MB/s   8000   m3.xlarge   4   Yes   64MB/s   4000   m3.2xlarge   8   Yes   128MB/s   8000   c1.medium   2   No   32MB/s   2000   c1.xlarge   8   Yes   128MB/s   8000   cc2.8xlarge   32   NA   800MB/s   50,000   m2.xlarge   2   No   64MB/s   4000   m2.2xlarge   4   Yes   64MB/s   4000   m2.4xlarge   8   Yes   128MB/s   8000   cr1.8xlarge   32   NA   800MB/s   50,000   hi1.4xlarge   16   NA   800MB/s   50,000   cg1.4xlarge   16   NA   800MB/s   50,000  
  27. 27. Smaller I/O (4 KB, 16 KB) •  Why are 4 KB I/O size in sequential operations driving greater than 4000 IOPS? •  Why is m1.large and m3.xlarge IOPS at 16 KB less than 4000 IOPS? •  Database needs 5000 ops/ second. How many IOPS do I need to provision? •  What happens when customers want to burst beyond provisioned IOPS? IOPS and BW performance at QD 8 m1.large   M3.xlarge   m3.2xlarge AvgIOPs Avg BW ( KB)   AvgIOPs Avg BW ( KB)   AvgIOPs Avg BW ( KB) Write sequential                 4K 4146 16,587  5997 23,990  7767 31,068 16K 3712 59,402  4157 55,461  4153 60,332 Write random                 4K 4082 16,329  4433 17,733  4178 16,712 16K 3713 59,422  3743 53,813  4153 60,332                   Read sequential                 4K 5301 21,205  9232 36,929  13450 53,802 16K 3533 56,535  4796 56,824  4153 60,332 Read random                 4K 4538 18,154  5864 23,457  4177 16,711 16K 3510 56,168  3583 51,246  4153 60,332 Results for 400 GB volume with 4000 IOPS at QD 8; EBS-optimized instances
  28. 28. Larger I/O (128 KB, 512 KB) •  Why am I seeing only 462 IOPS on a volume? •  Why there is no difference in performance for random and sequential workloads? •  How should I configure 500 MB/s read or write throughput using PIOPS volumes IOPS and BW performance at QD 8 m1.large   M3.xlarge   m3.2xlarge AvgIOPs Avg BW(KB)   AvgIOPs Avg BW(KB)   AvgIOPs Avg BW(KB) Write sequential                 128K 462 59,268  462 59,145  522 66,843 512K 115 59,292  115 59,278  130 66,804 Write random                 128K 462 59,265  462 59,241  522 66,843 512K 115 59,291  115 59,272  130 66,843 Read sequential                 128K 455 58,240  454 58,225  522 66,843 512K 113 58,003  114 58,589  130 66,843 Read random                 128K 455 58,236  454 58,215  522 66,843 512K 113 57,960  114 58,496  130 66,805 4000, 16 KB read/write per second, or 2000 32 KB read/write per second, or 1000 64 KB read/ write per second… Results for 400 GB volume with 4000 IOPS at QD 8
  29. 29. Write Latency •  Database applications care about latency as much as IOPS delivered •  There is an Interdependency among IOPS, queue depth, and latency •  Current guidance is queue depth of 1 for every 200 IOPS, but if latency-bound and write- heavy, 1:500 – 1:1000 is better. 1 4 8 12 16 20 24 28 32 AvgIOPS ( Count) 845 4152 4153 4177 4152 4176 4177 4177 4151 AvgTP90 ( ms) 3.13 1.47 2.03 3.56 3.62 5.54 6.18 7.48 7.71 845 4152 3.13 1.47 2.03 3.56 3.62 5.54 6.18 7.48 7.71 0 1 2 3 4 5 6 7 8 9 0 500 1000 1500 2000 2500 3000 3500 4000 4500 WriteIOPS 16 KBk random WRITE- M3.2Xlarge EBS- optimized L a t e n c y QD
  30. 30. Read Latency •  Reads can take advantage of a deeper queue •  Current guidance is queue depth of 1 for every 250 IOPS •  EBS-optimized provides predictable latency 1 4 8 12 16 20 24 28 32 AvgIOPS ( Count) 1864 4153 4153 4177 4120 2800 1965 1213 1089 AvgTP90 ( ms) 0.68 1.46 2.15 3.43 3.88 5.18 91.14 93.18 93.70 1864 4153 4120 1965 0.68 1.46 2.15 3.43 3.88 5.18 91.14 93.18 93.70 0 10 20 30 40 50 60 70 80 90 100 0 500 1000 1500 2000 2500 3000 3500 4000 4500 ReadIOPS 16 KB random READ - M3.2Xlarge EBS-optimized L a t e n c y QD
  31. 31. •  Performance requirements may be driven by IOPS or latency or both •  Recommendation is to start with queue depth of 4 and tune based on IOPS and latency requirement –  Some customers may need lowest possible latency; this can be achieved at queue depth of 1 or 2 •  Very high queue depths ( >24) may decrease IOPS count as well as increase latency Architecting for Performance: Latency
  32. 32. •  Typically 5%, extreme worst case of 50% performance reduction in IOPS and latency when volumes are used without pre-warming –  Performance is as provisioned when all the chunks are accessed •  Recommendation if testing or you have spare setup time: –  Write to every 4 MB block before using new volumes •  Linux: DD •  Windows: NTFS Full format –  Takes roughly an hour to pre-warm 1TB 4KB PIOPS volume –  Be warned, can take up to a day for a 1 TB standard EBS volume Pre-warming EBS volumes
  33. 33. What about RAID? Amazon Elastic Block Storage (EBS) Amazon Elastic Compute Cloud (EC2) •  Don’t do RAID 5, 6 Amazon Elastic Block Storage (EBS) Amazon Elastic Compute Cloud (EC2)
  34. 34. Architecting for Performance: RAID Stripe number of volumes to drive higher IOPS and throughput –  RAID 0 or RAID 10 Taking snapshots on a striped volume: –  Quiesce file systems and take snapshot –  Unmount file system and take snapshot –  Use OS-specific tools 12×400 GB PIOPS volumes, pre-warmed, RAID 0 LVM, Stripe size 64 KB, attached to CR1 instance IO  Pa&ern   Block  Sizes   Thread   Write  IOPS    Write  BW  (MB)   Read  IOPS   Read  BW  (MB)   Sequen>al     4K   8   33,500   134   48,250   193   16K   8   13,875   222   48,063   769   1M   1   247   247   823   823       Random   4K   8   35,250   141   48,250   193   16K   8   13,875   222   42,125   674   1M   1   496   496   795   795  
  35. 35. Workload/ Software Typical block size Random /Seq? Typical correct EBS type Max EBS @ 500 MB/s Max EBS @ 1 GB/s Max EBS @ CC/H Oracle DB Configurable:2 KB – 16 KB Default 8 KB random PIOPS RAID 0, 2 × 4000 PIOPS: ~7800 IOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS Microsoft SQL Server 8 KB w/ 64 KB extents random PIOPS RAID 0, 2 × 4000 PIOPS: ~7800 IOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 20 × 4000 PIOPS: ~80,000 IOPS MySQL 16 KB random PIOPS 1 x 4000 PIOPS: ~4000 IOPS RAID 0, 2 × 4000 PIOPS: ~7,800 IOPS RAID 0, 12 × 4000 PIOPS: ~48,000 IOPS PostgreSQL 8 KB random PIOPS RAID 0, 2 × 4000 PIOPS: ~7,800 IOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS IBM DB/2 Configurable:2 KB –-256 KB Default 32 KB random PIOPS 1 x 4000 PIOPS: ~2000 IOPS RAID 0, 4 × 4000 PIOPS: ~8,000 IOPS RAID 0, 12 × 4000 PIOPS: ~24,000 IOPS NuoDB 8 KB random PIOPS RAID 0, 2 × 4000 PIOPS: ~7800 IOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS Clustrix 4 KB random PIOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 8 × 4000 PIOPS: ~31,000 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS GenieDB 16 KB random PIOPS 1 × 4000 PIOPS: ~4000 IOPS RAID 0, 2 × 4000 PIOPS: ~7,800 IOPS RAID 0, 12 × 4000 PIOPS: ~48,000 IOPS NewSQLRDBMS Cheat Sheet! Storage Workloads on AWS
  36. 36. Workload/ Software Typical block size Random /Seq? Typical correct EBS type Max EBS @ 500 MB/s Max EBS @ 1 GB/s Max EBS @ CC/H MongoDB 4 KB serialized PIOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 8 × 4000 PIOPS: ~31,000 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS Cassandra 4 KB random PIOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 8 × 4000 PIOPS: ~31,000 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS CouchBase 4 KB random PIOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 8 × 4000 PIOPS: ~31,000 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS Hadoop / Hbase 64 KB random PIOPS 1 × 4000 PIOPS: ~1000 IOPS RAID 0, 2 × 4000 PIOPS: ~2000 IOPS RAID 0, 12 × 4000 PIOPS: ~12,000 IOPS Riak / LevelDB 4 KB random PIOPS RAID 0, 4 × 4000 PIOPS: ~15,600 IOPS RAID 0, 8 × 4000 PIOPS: ~31,000 IOPS RAID 0, 24 × 4000 PIOPS: ~96,000 IOPS NoSQL Cheat Sheet! Storage Workloads on AWS
  37. 37. Cheat Sheet! Storage Workloads on AWS Workload/ Software Typical block size Random /Seq? Typical correct EBS type Max EBS @ 500 MB/s Max EBS @ 1 GB/s Max EBS @ CC/H AeroSpike 128 KB random PIOPS 1 × 4000 PIOPS: ~500 IOPS RAID 0, 2 x 4000 PIOPS: ~1000 IOPS RAID 0, 12 x 4000 PIOPS: ~6,000 IOPS (768MB/s) Vertica 1 MB sequential PIOPS 1 × 4000 PIOPS: ~68 IOPS RAID 0, 2 × 4000 PIOPS: ~125 IOPS RAID 0, 12 × 4000 PIOPS: ~750 IOPS (768MB/s) TeraData 128 KB sequential PIOPS 1 × 4000 PIOPS: ~500 IOPS RAID 0, 2 × 4000 PIOPS: ~1000 IOPS RAID 0, 12 × 4000 PIOPS: ~6,000 IOPS (768MB/s) GlusterFS 128 KB sequential PIOPS 1 × 4000 PIOPS: ~500 IOPS RAID 0, 2 × 4000 PIOPS: ~1000 IOPS RAID 0, 12 × 4000 PIOPS: ~6,000 IOPS (768MB/s) OrangeFS 256 KB sequential PIOPS 1 × 4000 PIOPS: ~250 IOPS RAID 0, 2 × 4000 PIOPS: ~500 IOPS RAID 0, 12 × 4000 PIOPS: ~3,000 IOPS (768MB/s) Riak (CS) 1 MB sequential PIOPS 1 × 4000 PIOPS: ~68 IOPS RAID 0, 2 × 4000 PIOPS: ~125 IOPS RAID 0, 12 × 4000 PIOPS: ~750 IOPS (768MB/s) DNFileSystemWarehouse
  38. 38. Performance – Extra-large Production Scale •  Leverage SSD instance type (hi1.4xlarge) o  2 × 1 TB SSD storage (ephemeral storage) o  Perfect for replicas •  If replicas on SSD instance types, disable integrity features such as fsync and full_page_writes on those hosts to improve performance
  39. 39. STABLE Testing Random 4 KB Reads EBS PIOPS+ SSD
  40. 40. What about Capacity Cost? cc2.8xlarge 48x 1TB EBS VS. hs1.8xlarge hs1.8xlarge $7312 on-demand, $6128 effective 3 YR reserved $6734 on-demand, $2408 effective 3 YR reserved If >43TB, or > 800MB/s, choose hs1 If 3 year, and >18TB, choose hs1
  41. 41. Performance / Stability Tips •  Ext4 or XFS (understand journal impact!) •  nobarrier, noatime, noexec, nodiratime •  Raise file descriptor limits •  Set read-aheads low •  Amazon CloudWatch metrics •  SNAPSHOT SNAPSHOT SNAPSHOT
  42. 42. From 1000 to 98,000: What does that mean?
  43. 43. So.. all storage workloads!
  44. 44. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Maximizing Amazon EC2 & Amazon EBS Performance Carlos Conde – Head of AWS Evangelism EMEA Thank you!

×