Learn about Amazon Elastic Block Store (EBS) and how to get consistent High IO performance using Provisioned IOPS volumes. Also hear from CopperEgg about how they use EBS to get the consistent high
Learn about Amazon Elastic Block Store (EBS) and how to get consistent High IO performance using Provisioned IOPS volumes. Also hear from CopperEgg about how they use EBS to get the consistent high IO for their service
Consistent High I/O Performancewith Amazon Elastic Block Store Jafar Shameem Business Development Manager, Amazon Web Services Eric Anderson CTO and Co-Founder, CopperEgg
Agenda • AWS Storage Options and EBS • Introducing Provisioned IOPS • Maximize your application’s IO through Provisioned IOPS • How CopperEgg benefits from Provisioned IOPS
Storage Options on AWS Block Storage Object Storage (Elastic Block Store) (S3, Glacier) Use for: Use for: • Access to raw • Pictures, videos, hi unformatted block ghly durable media level storage storage • Persistent Storage • Cold storage for long-term archive
Amazon Elastic Block Store (EBS)Elastic Block Storage: Persistent Storage for EC2 Feature Details High Mount EBS as drives and performance format as required file system Flexible size Volumes from 1GB to 1TB in size Secure Private to your instances Available Replicated within an High performance block storage Availability Zone device Backups Volumes can be snapshotted Mount as drives to instances for point in time restore Persistent and independent of Monitoring Detailed metrics captured via instance lifecycle Cloud Watch
Introducing Provisioned IOPS Volumes ❶ Select a new type of Provisioned IOPS volumes ❷ Specify the volume capacity ❸ Specify the number of IOs per second your application needs, up to 2000 PIOPS per volume. The volume will deliver the specified IO per second.$ ec2-create-volume --size 500 --availability-zone us-east-1b --type io1 –iops 2000
Consistent IO performance Provisioned IOPS volumes deliver within 10% of the IOPS performance 99.9% of the time over a given year.
When should I use Provisioned IOPS?• Standard volumes are designed for boot volumes, running applications with moderate and bursty IO, sequential IO access• Provisioned IOPS volumes are designed for running transactional applications that require high and consistent IO: o Relational Databases o NoSQL Databases, e.g. MongoDB o Productivity applications, e.g. Microsoft Exchange o Enterprise Applications
Tip #4: Achieving consistent performance • There is a 5 to 50 percent reduction in IOPS when you first access the data on a volume. • $ dd if=/dev/md0 of=/dev/null • To minimize the impact of snapshots on performance of a master node, create snapshots from a read replica of your data. Ideally, create these snapshots during off-peak usage.
Tip #5: Understanding Queue DepthMaintain a number of pending I/O requests to get the most out of your ProvisionedIOPS volume. Queue depth of 5 per 1000 IOPS.
CopperEgg’s IO Use Case • Large metric capture volumes • Real-time dashboards • Historical data analysis • Durability/Availability • Real-time alerting
How CopperEgg uses Provisioned IOPS • Used in our main metric data store • Stores Egg-zillions of metrics • Rock-solid predictable IO that’s affordable • Snapshots for backups • XFS file system on top
CopperEgg’s IO Demands • Store many terabytes of data • Persist the data over long periods of time • Backups (use snapshots) • High IO: 10-15k+ ops/s per node minimum • Consistent IO behavior (non-spikey)
Provisioned IOPS hits the spot • High Performance: get what you ask for o More than 20,000 IOPS on a single node • Extremely predictable IO characteristics • Cost is reasonable (and predictable!) • Same persistence as standard EBS • Data replicated in the same AZ just like Standard EBS
Cost Effective? • For steady IO use patterns, it is! • For bursty IO, do the math
CopperEgg’s Provisioned IOPS rule ofthumb • Need predictable performance? Provisioned IOPS • Less than 100 ops/s on average? Standard EBS • Need more than 100 ops/s per volume? Provisioned IOPS
Get started on Provisioned IOPS today! aws.amazon.com/ebs Questions: e-mail: firstname.lastname@example.org• Amazon Provisioned IOPS o http://copperegg.com/amazon-provisioned-iops-ebs/• Benchmarking EBS performance: o http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerfo rmance.html• Stratalux: Putting Amazon’s Provisioned IOPS to the test o http://www.stratalux.com/2012/08/09/putting-amazon’s- provisioned-iops-to-the-test/