• Like
  • Save

Consistent High IO Performance with Amazon Elastic Block Store

  • 2,974 views
Uploaded on

Learn about Amazon Elastic Block Store (EBS) and how to get consistent High IO performance using Provisioned IOPS volumes. Also hear from CopperEgg about how they use EBS to get the consistent high …

Learn about Amazon Elastic Block Store (EBS) and how to get consistent High IO performance using Provisioned IOPS volumes. Also hear from CopperEgg about how they use EBS to get the consistent high IO for their service

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
2,974
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
10

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Consistent High I/O Performancewith Amazon Elastic Block Store Jafar Shameem Business Development Manager, Amazon Web Services Eric Anderson CTO and Co-Founder, CopperEgg
  • 2. Agenda • AWS Storage Options and EBS • Introducing Provisioned IOPS • Maximize your application’s IO through Provisioned IOPS • How CopperEgg benefits from Provisioned IOPS
  • 3. Storage Options on AWS Block Storage Object Storage (Elastic Block Store) (S3, Glacier) Use for: Use for: • Access to raw • Pictures, videos, hi unformatted block ghly durable media level storage storage • Persistent Storage • Cold storage for long-term archive
  • 4. Amazon Elastic Block Store (EBS)Elastic Block Storage: Persistent Storage for EC2 Feature Details High Mount EBS as drives and performance format as required file system Flexible size Volumes from 1GB to 1TB in size Secure Private to your instances Available Replicated within an High performance block storage Availability Zone device Backups Volumes can be snapshotted Mount as drives to instances for point in time restore Persistent and independent of Monitoring Detailed metrics captured via instance lifecycle Cloud Watch
  • 5. $ sudo mdadm --create /dev/md0 --level=0 --chunk=64 --raid-devices=3 /dev/sdf /dev/sdg/dev/sdh
  • 6. Introducing ProvisionedIOPS Volumes
  • 7. Introducing Provisioned IOPS Volumes ❶ Select a new type of Provisioned IOPS volumes ❷ Specify the volume capacity ❸ Specify the number of IOs per second your application needs, up to 2000 PIOPS per volume. The volume will deliver the specified IO per second.$ ec2-create-volume --size 500 --availability-zone us-east-1b --type io1 –iops 2000
  • 8. Consistent IO performance Provisioned IOPS volumes deliver within 10% of the IOPS performance 99.9% of the time over a given year.
  • 9. When should I use Provisioned IOPS?• Standard volumes are designed for boot volumes, running applications with moderate and bursty IO, sequential IO access• Provisioned IOPS volumes are designed for running transactional applications that require high and consistent IO: o Relational Databases o NoSQL Databases, e.g. MongoDB o Productivity applications, e.g. Microsoft Exchange o Enterprise Applications
  • 10. Simple pricing
  • 11. Getting more out ofProvisioned IOPS Volumes
  • 12. Tip # 1: Use EBS-Optimized instances
  • 13. Tip #2: Understanding Provisioned IOPSVolume Throughput
  • 14. Tip #3: Stripe multiple volumes formore IOPS sudo mdadm --verbose --create /dev/md0 --level=10 --chunk=256 --raid- devices=4 /dev/sdh1 /dev/sdh2 /dev/sdh3 /dev/sdh4
  • 15. Tip #4: Achieving consistent performance • There is a 5 to 50 percent reduction in IOPS when you first access the data on a volume. • $ dd if=/dev/md0 of=/dev/null • To minimize the impact of snapshots on performance of a master node, create snapshots from a read replica of your data. Ideally, create these snapshots during off-peak usage.
  • 16. Tip #5: Understanding Queue DepthMaintain a number of pending I/O requests to get the most out of your ProvisionedIOPS volume. Queue depth of 5 per 1000 IOPS.
  • 17. How CopperEgguses Provisioned IOPS
  • 18. CopperEgg’s IO Use Case • Large metric capture volumes • Real-time dashboards • Historical data analysis • Durability/Availability • Real-time alerting
  • 19. How CopperEgg uses Provisioned IOPS • Used in our main metric data store • Stores Egg-zillions of metrics • Rock-solid predictable IO that’s affordable • Snapshots for backups • XFS file system on top
  • 20. CopperEgg’s IO Demands • Store many terabytes of data • Persist the data over long periods of time • Backups (use snapshots) • High IO: 10-15k+ ops/s per node minimum • Consistent IO behavior (non-spikey)
  • 21. CopperEgg IO Sweet Spot Data Persistence Sweet spot Cost Stability IO Performance Predictability
  • 22. Provisioned IOPS hits the spot • High Performance: get what you ask for o More than 20,000 IOPS on a single node • Extremely predictable IO characteristics • Cost is reasonable (and predictable!) • Same persistence as standard EBS • Data replicated in the same AZ just like Standard EBS
  • 23. Cost Effective? • For steady IO use patterns, it is! • For bursty IO, do the math
  • 24. CopperEgg’s Provisioned IOPS rule ofthumb • Need predictable performance? Provisioned IOPS • Less than 100 ops/s on average? Standard EBS • Need more than 100 ops/s per volume? Provisioned IOPS
  • 25. Get started on Provisioned IOPS today! aws.amazon.com/ebs Questions: e-mail: shameemj@amazon.com• Amazon Provisioned IOPS o http://copperegg.com/amazon-provisioned-iops-ebs/• Benchmarking EBS performance: o http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerfo rmance.html• Stratalux: Putting Amazon’s Provisioned IOPS to the test o http://www.stratalux.com/2012/08/09/putting-amazon’s- provisioned-iops-to-the-test/