AWS Webcast - Introduction to EBS


Published on

Elastic Block Store is one of the fundamental components of a best-practices cloud architecture. Join Senior Solutions Architect Miles Ward for a detailed review of the service, a clear breakdown on cost and approaches for price estimates, easy methods for extracting maximum performance from both Standard and Provisioned-IOPS EBS volumes, configuration nuances for both Windows and Linux users, RAID guidance, RDBMS and NoSQL storage configuration tuning, as well as several examples of EBS delivering amazing value at scale.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

AWS Webcast - Introduction to EBS

  1. 1. Presenter Name Presenter Title Month Day, Year Understanding Amazon EBS Availability and Performance Miles Ward Senior Manager, Solutions Architecture
  2. 2. Agenda • Overview of Elastic Block Store • Some key concepts: block size, IOPS, throughput • Performance • Availability
  3. 3. Storage Options on AWS Block Storage (Elastic Block Store) Object Storage (S3, Glacier) Use for: • Access to raw unformatted block level storage • Persistent Storage Use for: • Pictures, videos, highly durable media storage • Cold storage for long-term archive
  4. 4. Amazon Elastic Block Store (EBS) Elastic Block Storage: Persistent Storage for EC2 Feature Details High performance file system Mount EBS as drives and format as required Flexible size Volumes from 1GB to 1TB in size Secure Private to your instances Available Replicated within an Availability Zone Backups Volumes can be snapshotted for point in time restore Monitoring Detailed metrics captured via Cloud Watch
  5. 5. What are some of our customers doing with EBS? Enterprises Enterprise workloads are built on block storage Oracle, SAP, Microsoft Applications Convenient, cost-effective, reliable file server Gaming/Social/ Mobile/Education Very high performance and consistent IO for NoSQL and relational DBs Marketing / Analytics Fast sequential IO access
  6. 6. Key EBS concepts • Standard EBS Volumes • Provisioned IOPS EBS Volumes • Block Size • Queue Depth • Snapshots
  7. 7. Standard and Provisioned IOPS Volume Types Standard Volumes Provisioned IOPS Volumes Optimized for Workloads with low or moderate IOPS needs and occasional bursts. Transactional workloads requiring consistent IOPS. Volume Attributes Up to 1 TB, average 100 IOPS per volume. Best effort performance. Can be striped together for larger size and higher IOPS. Up to 1TB, 2,000 IOPS per volume. Consistent IOPS. Can be striped together for larger size and higher IOPS. Workloads File server, Log processing, Websites, Analytics, Boot, etc. Business applications, MongoDB, SQL server, MySQL, Postgres, Oracle, etc.
  8. 8. Introducing Provisioned IOPS Volumes ❶ Select a new type of Provisioned IOPS volumes ❸ Specify the number of IOs per second your application needs, up to 4000 PIOPS per volume. The volume will deliver the specified IO per second. ❷ Specify the volume capacity $ ec2-create-volume --size 500 --availability-zone us-east-1b --type io1 –iops 2000
  9. 9. Consistent IO performance Provisioned IOPS volumes deliver within 10% of the IOPS performance 99.9% of the time over a given year.
  10. 10. Stable Testing random 4K reads EBS PIOPS+ SSD
  11. 11. Before PIOPS: After PIOPS: End-to-end latency
  12. 12. When should I use Provisioned IOPS? • Provisioned IOPS volumes are designed for running transactional applications that require high and consistent IO: o Relational Databases o NoSQL Databases, e.g. MongoDB o Productivity applications, e.g. Microsoft Exchange o Enterprise Applications • Standard volumes are designed for boot volumes, running applications with moderate and bursty IO, and sequential IO access
  13. 13. IOPS vs. Throughput One input operation or “block” The network throughput available on the connection between EC2 and EBS The IOPS provisioned
  14. 14. IOPS vs. Throughput Bigger than 16k… Greater than the mbps limit… Greater than the IOPS provisioned…
  15. 15. A little math • Networks are in mbit/s, disks tend to be in MB/s. • 1000mbit/s = 125 MB/s • 500mbit/s = 67.5 MB/s • Blocks/ IOs are in KB • ~7,500 16KB blocks fit in 1000mbit/s • ~30,000 4KB blocks fit in 1000mbit/s
  16. 16. Block Size
  17. 17. Queue Depth Maintain a number of pending I/O requests to get the most out of your Provisioned IOPS volume. The volumes must maintain an average queue length of 1 (rounded up to the nearest whole number) for every 200 provisioned IOPS in a minute
  18. 18. Performance • Architecting for Performance – Avoid IO saturation – EBS Optimized – Avoid throughput saturation – RAID • Achieving Consistent Performance – Pre-warm PIOPS volumes – Plan for snapshot • Snapshot Performance
  19. 19. Architecting for Performance: Use EBS Optimized Instances
  20. 20. USING Provisioned IOPS Volumes ❶ Select a new type of Provisioned IOPS volumes ❸ SPECIFY ENOUGH IOPS!! ❷ Specify the volume capacity $ ec2-create-volume –size 400 --availability-zone us-east-1b --type io1 –iops 4000
  21. 21. Architecting for Performance: Avoid Throughput Saturation • Example: – Cluster Compute instance types have 2Gb/s bandwidth to EBS, more than 8 PIOPS volumes at 2000 IOPS each will saturate 2 Gb/s network – EBS Optimized M3.2Xlarge instance has 1 Gb/s bandwidth dedicated to EBS, more than 16 PIOPS volumes at 500 IOPS each will saturate the 1 GB/s network
  22. 22. RAID • RAID 10: provides increased redundancy – Replace EBS volume without application downtime – Increases read throughput – However, 50% reduction of provisioned aggregate write performance – E.g., MongoDB optimized around the benefits of RAID 10 • RAID 0: – All EBS volumes are replicated in the same AZ – Increased throughput
  23. 23. Achieving Consistent Performance: Pre-warm PIOPS volumes • There is a 5 to 50 percent performance reduction in IOPS when you first access the data on a PIOPS volume. • Write to all blocks on volumes before first use – $ dd of=/dev/md0 if=/dev/null
  24. 24. Snapshots • Create snapshots (backups) of any Amazon EBS volume. • The volume need not be attached to a running instance in order to take a snapshot. • These snapshots can be used to create multiple new Amazon EBS volumes, expand the size of a volume, or move volumes across Availability Zones. • The snapshots can be shared with specific AWS accounts or made public. • You can use this functionality to increase the size of an existing volume, rapidly replicate development and testing environments, or use Snapshot Copy to copy snapshots to another region for disaster recovery or regional expansion.
  25. 25. Snapshot Performance • To improve snapshot performance – Increase the frequency of snapshots
  26. 26. Achieving Consistent Performance: Plan for Snapshot To minimize the impact of snapshots on performance of a master node: – create snapshots from a read replica of your data – Plan snapshots during off-peak usage – Use cross-region snapshot copy to keep distant regional copies for HA / low latency local access
  27. 27. Performance – Minimum production DB scale • Always use Elastic Block Store (EBS) o Significant write cache o Great random IO performance o Enhanced durability compared to instance stores
  28. 28. Performance – production scale • Use Provisioned IOPS for all workloads targeting above 100 IOPS. • Move up to higher bandwidth instance types (m1.xlarge, c1.xlarge, m2.4xlarge) • Increase EBS volume size to >= 400 GB • Increase number of volumes in RAID set
  29. 29. Performance – Extra-large scale • Leverage Cluster Compute instance types o More bandwidth to EBS o Ex. CC2 will make excellent primary nodes, particularly when paired with a large number of EBS volumes (= 8) • Improve RAID configuration with: o effective_io_concurrency = # of stripes in RAID set
  30. 30. • Amazon Provisioned IOPS o • Benchmarking EBS performance: o • Stratalux: Putting Amazon’s Provisioned IOPS to the test o’s-provisioned-iops-to-the-test/ Get started on EBS today! Questions: e-mail: @milesward