Consistent High IO Performance with Amazon Elastic Block Store


Published on

Learn about Amazon Elastic Block Store (EBS) and how to get consistent High IO performance using Provisioned IOPS volumes. Also hear from CopperEgg about how they use EBS to get the consistent high IO for their service

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Consistent High IO Performance with Amazon Elastic Block Store

  1. 1. Consistent High I/O Performancewith Amazon Elastic Block Store Jafar Shameem Business Development Manager, Amazon Web Services Eric Anderson CTO and Co-Founder, CopperEgg
  2. 2. Agenda • AWS Storage Options and EBS • Introducing Provisioned IOPS • Maximize your application’s IO through Provisioned IOPS • How CopperEgg benefits from Provisioned IOPS
  3. 3. Storage Options on AWS Block Storage Object Storage (Elastic Block Store) (S3, Glacier) Use for: Use for: • Access to raw • Pictures, videos, hi unformatted block ghly durable media level storage storage • Persistent Storage • Cold storage for long-term archive
  4. 4. Amazon Elastic Block Store (EBS)Elastic Block Storage: Persistent Storage for EC2 Feature Details High Mount EBS as drives and performance format as required file system Flexible size Volumes from 1GB to 1TB in size Secure Private to your instances Available Replicated within an High performance block storage Availability Zone device Backups Volumes can be snapshotted Mount as drives to instances for point in time restore Persistent and independent of Monitoring Detailed metrics captured via instance lifecycle Cloud Watch
  5. 5. $ sudo mdadm --create /dev/md0 --level=0 --chunk=64 --raid-devices=3 /dev/sdf /dev/sdg/dev/sdh
  6. 6. Introducing ProvisionedIOPS Volumes
  7. 7. Introducing Provisioned IOPS Volumes ❶ Select a new type of Provisioned IOPS volumes ❷ Specify the volume capacity ❸ Specify the number of IOs per second your application needs, up to 2000 PIOPS per volume. The volume will deliver the specified IO per second.$ ec2-create-volume --size 500 --availability-zone us-east-1b --type io1 –iops 2000
  8. 8. Consistent IO performance Provisioned IOPS volumes deliver within 10% of the IOPS performance 99.9% of the time over a given year.
  9. 9. When should I use Provisioned IOPS?• Standard volumes are designed for boot volumes, running applications with moderate and bursty IO, sequential IO access• Provisioned IOPS volumes are designed for running transactional applications that require high and consistent IO: o Relational Databases o NoSQL Databases, e.g. MongoDB o Productivity applications, e.g. Microsoft Exchange o Enterprise Applications
  10. 10. Simple pricing
  11. 11. Getting more out ofProvisioned IOPS Volumes
  12. 12. Tip # 1: Use EBS-Optimized instances
  13. 13. Tip #2: Understanding Provisioned IOPSVolume Throughput
  14. 14. Tip #3: Stripe multiple volumes formore IOPS sudo mdadm --verbose --create /dev/md0 --level=10 --chunk=256 --raid- devices=4 /dev/sdh1 /dev/sdh2 /dev/sdh3 /dev/sdh4
  15. 15. Tip #4: Achieving consistent performance • There is a 5 to 50 percent reduction in IOPS when you first access the data on a volume. • $ dd if=/dev/md0 of=/dev/null • To minimize the impact of snapshots on performance of a master node, create snapshots from a read replica of your data. Ideally, create these snapshots during off-peak usage.
  16. 16. Tip #5: Understanding Queue DepthMaintain a number of pending I/O requests to get the most out of your ProvisionedIOPS volume. Queue depth of 5 per 1000 IOPS.
  17. 17. How CopperEgguses Provisioned IOPS
  18. 18. CopperEgg’s IO Use Case • Large metric capture volumes • Real-time dashboards • Historical data analysis • Durability/Availability • Real-time alerting
  19. 19. How CopperEgg uses Provisioned IOPS • Used in our main metric data store • Stores Egg-zillions of metrics • Rock-solid predictable IO that’s affordable • Snapshots for backups • XFS file system on top
  20. 20. CopperEgg’s IO Demands • Store many terabytes of data • Persist the data over long periods of time • Backups (use snapshots) • High IO: 10-15k+ ops/s per node minimum • Consistent IO behavior (non-spikey)
  21. 21. CopperEgg IO Sweet Spot Data Persistence Sweet spot Cost Stability IO Performance Predictability
  22. 22. Provisioned IOPS hits the spot • High Performance: get what you ask for o More than 20,000 IOPS on a single node • Extremely predictable IO characteristics • Cost is reasonable (and predictable!) • Same persistence as standard EBS • Data replicated in the same AZ just like Standard EBS
  23. 23. Cost Effective? • For steady IO use patterns, it is! • For bursty IO, do the math
  24. 24. CopperEgg’s Provisioned IOPS rule ofthumb • Need predictable performance? Provisioned IOPS • Less than 100 ops/s on average? Standard EBS • Need more than 100 ops/s per volume? Provisioned IOPS
  25. 25. Get started on Provisioned IOPS today! Questions: e-mail:• Amazon Provisioned IOPS o• Benchmarking EBS performance: o rmance.html• Stratalux: Putting Amazon’s Provisioned IOPS to the test o’s- provisioned-iops-to-the-test/