Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Consistent High IO Performance with Amazon Elastic Block Store


Published on

Learn about Amazon Elastic Block Store (EBS) and how to get consistent High IO performance using Provisioned IOPS volumes. Also hear from CopperEgg about how they use EBS to get the consistent high …

Learn about Amazon Elastic Block Store (EBS) and how to get consistent High IO performance using Provisioned IOPS volumes. Also hear from CopperEgg about how they use EBS to get the consistent high IO for their service

Published in: Technology

  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Consistent High I/O Performancewith Amazon Elastic Block Store Jafar Shameem Business Development Manager, Amazon Web Services Eric Anderson CTO and Co-Founder, CopperEgg
  • 2. Agenda • AWS Storage Options and EBS • Introducing Provisioned IOPS • Maximize your application’s IO through Provisioned IOPS • How CopperEgg benefits from Provisioned IOPS
  • 3. Storage Options on AWS Block Storage Object Storage (Elastic Block Store) (S3, Glacier) Use for: Use for: • Access to raw • Pictures, videos, hi unformatted block ghly durable media level storage storage • Persistent Storage • Cold storage for long-term archive
  • 4. Amazon Elastic Block Store (EBS)Elastic Block Storage: Persistent Storage for EC2 Feature Details High Mount EBS as drives and performance format as required file system Flexible size Volumes from 1GB to 1TB in size Secure Private to your instances Available Replicated within an High performance block storage Availability Zone device Backups Volumes can be snapshotted Mount as drives to instances for point in time restore Persistent and independent of Monitoring Detailed metrics captured via instance lifecycle Cloud Watch
  • 5. $ sudo mdadm --create /dev/md0 --level=0 --chunk=64 --raid-devices=3 /dev/sdf /dev/sdg/dev/sdh
  • 6. Introducing ProvisionedIOPS Volumes
  • 7. Introducing Provisioned IOPS Volumes ❶ Select a new type of Provisioned IOPS volumes ❷ Specify the volume capacity ❸ Specify the number of IOs per second your application needs, up to 2000 PIOPS per volume. The volume will deliver the specified IO per second.$ ec2-create-volume --size 500 --availability-zone us-east-1b --type io1 –iops 2000
  • 8. Consistent IO performance Provisioned IOPS volumes deliver within 10% of the IOPS performance 99.9% of the time over a given year.
  • 9. When should I use Provisioned IOPS?• Standard volumes are designed for boot volumes, running applications with moderate and bursty IO, sequential IO access• Provisioned IOPS volumes are designed for running transactional applications that require high and consistent IO: o Relational Databases o NoSQL Databases, e.g. MongoDB o Productivity applications, e.g. Microsoft Exchange o Enterprise Applications
  • 10. Simple pricing
  • 11. Getting more out ofProvisioned IOPS Volumes
  • 12. Tip # 1: Use EBS-Optimized instances
  • 13. Tip #2: Understanding Provisioned IOPSVolume Throughput
  • 14. Tip #3: Stripe multiple volumes formore IOPS sudo mdadm --verbose --create /dev/md0 --level=10 --chunk=256 --raid- devices=4 /dev/sdh1 /dev/sdh2 /dev/sdh3 /dev/sdh4
  • 15. Tip #4: Achieving consistent performance • There is a 5 to 50 percent reduction in IOPS when you first access the data on a volume. • $ dd if=/dev/md0 of=/dev/null • To minimize the impact of snapshots on performance of a master node, create snapshots from a read replica of your data. Ideally, create these snapshots during off-peak usage.
  • 16. Tip #5: Understanding Queue DepthMaintain a number of pending I/O requests to get the most out of your ProvisionedIOPS volume. Queue depth of 5 per 1000 IOPS.
  • 17. How CopperEgguses Provisioned IOPS
  • 18. CopperEgg’s IO Use Case • Large metric capture volumes • Real-time dashboards • Historical data analysis • Durability/Availability • Real-time alerting
  • 19. How CopperEgg uses Provisioned IOPS • Used in our main metric data store • Stores Egg-zillions of metrics • Rock-solid predictable IO that’s affordable • Snapshots for backups • XFS file system on top
  • 20. CopperEgg’s IO Demands • Store many terabytes of data • Persist the data over long periods of time • Backups (use snapshots) • High IO: 10-15k+ ops/s per node minimum • Consistent IO behavior (non-spikey)
  • 21. CopperEgg IO Sweet Spot Data Persistence Sweet spot Cost Stability IO Performance Predictability
  • 22. Provisioned IOPS hits the spot • High Performance: get what you ask for o More than 20,000 IOPS on a single node • Extremely predictable IO characteristics • Cost is reasonable (and predictable!) • Same persistence as standard EBS • Data replicated in the same AZ just like Standard EBS
  • 23. Cost Effective? • For steady IO use patterns, it is! • For bursty IO, do the math
  • 24. CopperEgg’s Provisioned IOPS rule ofthumb • Need predictable performance? Provisioned IOPS • Less than 100 ops/s on average? Standard EBS • Need more than 100 ops/s per volume? Provisioned IOPS
  • 25. Get started on Provisioned IOPS today! Questions: e-mail:• Amazon Provisioned IOPS o• Benchmarking EBS performance: o rmance.html• Stratalux: Putting Amazon’s Provisioned IOPS to the test o’s- provisioned-iops-to-the-test/