Amazon RDS for Performance-Intensive Production Applications (DAT301) | AWS re:Invent 2013

3,593 views

Published on

Learn how to take advantage of Amazon RDS to run highly-available and performance-intensive production applications on AWS. We show you what you can do to achieve the highest levels of availability and performance for your relational databases. You learn how easy it is to architect for these requirements using several Amazon RDS features, such as Multi-AZ deployments, read replicas, and Provisioned IOPS storage. In addition, you learn how to quickly architect for the level of disaster recovery required by your business. Finally, some of our customers share how they built very high performing web and enterprise applications on Amazon RDS.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
3,593
On SlideShare
0
From Embeds
0
Number of Embeds
23
Actions
Shares
0
Downloads
89
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Amazon RDS for Performance-Intensive Production Applications (DAT301) | AWS re:Invent 2013

  1. 1. DAT301 - Running Highly-Available and Performance-Intensive Production Applications on Amazon RDS Grant McAlister – AWS Database Services Brennan Saeta – Coursera November 14, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. Why High Availability and Scalability DB1a DB1 DB1b DB1c
  3. 3. High Availability and Disaster Recovery
  4. 4. Availability – Read and Write – Multi-AZ cname update DNS Primary Update AZ1 Physical Synchronous Replication AZ2
  5. 5. Read Availability – Read Replicas Sync Replication Async Replication
  6. 6. Logical Failover – Replicas of Replicas Sync Replication Async Replication
  7. 7. Backup and Restore Amazon S3 Log 1 Log 2 AZ1 Log 5 AZ2
  8. 8. Cross Region – Snapshot Copy Amazon S3 Amazon S3 AZ1 AZ3 Region 1 Region 2
  9. 9. Cross Region – Read Replicas AZ1 Region 1 AZ3 Region 2
  10. 10. Performance and Scale
  11. 11. Provisioned IOPS - Scale db.m2.4xlarge - 130GB Data - Partial Random Read Workload 20,000 17,500 15,000 333GB - Regular 12,500 T P 10,000 S 7,500 5,000 2,500 0 333GB - 1000 PIOPS 300GB - 3000 PIOPS
  12. 12. Provisioned IOPS - Latency db.m2.4xlarge - 130GB Data - Partial Random Read Workload - Max Rate 9% Percentage in Latency Bucket 8% 7.69% 7.89% 7% 6% 333GB - Regular 333GB - 1000 PIOPS 300GB - 3000 PIOPS 5% 4% 3.83% 3% 2% 1.38% 1% 0.42% 0.00% 0.01% 0.00% 0.00% 0% 3-20 ms 20-500ms >500ms
  13. 13. Provisioned IOPS - Scale db.m2.4xlarge - 130GB Data - Partial Random Read Workload 20,000 17,500 15,000 333GB - Regular 333GB - 1000 PIOPS 12,500 T P 10,000 S 7,500 5,000 Target = 5000 2,500 0 300GB - 3000 PIOPS
  14. 14. Provisioned IOPS– Latency @ 5000 TPS db.m2.4xlarge - 130GB Data - Partial Random Read Workload - 5000 TPS 9% Percentage in Latency Bucket 8% 7.89% 7% 6% 5% 333GB - Regular 333GB - 1000 PIOPS 300GB - 3000 PIOPS 4% 3% 2% 1.17% 1% 0.35% 0.27% 0.00% 0.00% 0.04% 0.00% 0.00% 0% 3-20 ms 20-500ms >500ms
  15. 15. Provisioned IOPS– 10,000 IOPS db.m2.4xlarge - 130GB Data - Partial Random Read Workload 40,000 35,000 30,000 25,000 T P 20,000 S 15,000 10,000 5,000 0 333GB - Regular 300GB - 3000 PIOPS 333GB - 1000 PIOPS 1000GB-10000 PIOPS
  16. 16. What Limit? – Amazon CloudWatch Metrics
  17. 17. Bottlenecks and Scaling Throughput • • • • EBS Optimized (0.5 -1Gbit) db.m1.large,db.m1.xlarge,m2.2xlarge,m2.4xlarge db.cr1.8xlarge – 10Gbit Regular + non optimized – shared DB Instance Class IOPS • • • • db.t1.micro – db.cr1.8xlarge 1-88 ECU 0.6 – 244 GB RAM Memory to ECU Ratio • 1.7 to 2.8 (exclude micro) Regular or 1,000-30,000 Storage • Ratio 3:1 to 10:1 5GB - 100GB - 3TB DB Engine • • Block Size 8 – 16K Ability to utilize resources
  18. 18. Read/Write Benchmark 130GB Data - Partial Random 90R/10W Workload 10K IOPS 80,000 70,000 cpu 95% r/s 6800 w/s 2600 rMB/s 104 60,000 TPS 50,000 40,000 30,000 20,000 10,000 cpu 85% r/s 3450 w/s 600 rMB/s 52 cpu 100% r/s 6800 w/s 950 rMB/s 104 42,000 cpu 40% r/s 0 w/s 3200 rMB/s 0 47,000 13,200 5,850 - db.m1.large db.m1.xlarge db.m2.4xlarge db.cr1.8xlarge
  19. 19. Bottlenecks and Scaling – Read Replicas
  20. 20. Read/Write Benchmark – Using RR 130GB Data - Partial Random 90R/10W Workload – 10K IOPS 120,000 100,000 TPS 80,000 126,000 60,000 40,000 42,000 20,000 5,850 47,000 db.m2.4xlarge db.cr1.8xlarge 13,200 - db.m1.large db.m1.xlarge cr1 + 3 x 4xlarge
  21. 21. Our Amazon RDS Journey Tips, tricks, and war stories from our pervasive use of Amazon RDS
  22. 22. re:Coursera
  23. 23. re:Brennan Saeta Infrastructure Engineer saeta@coursera.org @bsaeta http://betacs.pro
  24. 24. Coursera’s Architecture Journey
  25. 25. Baby coursera.org Webserver Webserver Webserver Frontend Webserver Frontend Frontend nginx ELB nginx nginx nginx php class.coursera php class.coursera php class.coursera php class.coursera memcache memcache memcache memcache Database
  26. 26. Simplified, modern coursera.org Legacy Frontend nginx notifications python www. Cloud Front courservice pdf Cloud Search other services SES SES other services REST/HTTP coursera S3 S3 SQS AWS IaaS launcher REST/HTTP ELB kangxi subtitle php class.coursera online services maestro maestrorr memcache Zoo Keeper class class-nrr Kafka ELB ELB Edge arrrrgh api.coursera DBs: MySQL on RDS Auth accounts.coursera Cascade workbench REST/HTTP ELB Eventing eventing.coursera arrrrgh ELB Wiki share.coursera ELB other ELB services near-line services
  27. 27. Every website [almost]
  28. 28. It’s always the database!
  29. 29. Scaling – Every app is different!
  30. 30. Why Amazon RDS? – The Beginning • MySQL – Familiar tool, quick to work with • Amazon RDS – Automatic backups, point-in-time restore – Click to scale, click to read-replica – Multi-AZ
  31. 31. Why Amazon RDS? – Now • MySQL  Single node reliability and proven persistence engine • Amazon RDS  Automatic backups, point-in-time restore  Click to scale, click to read-replica ? Multi-AZ
  32. 32. Lessons Learned Along the Way It was the worst of times It was the best of times
  33. 33. If I could do one thing over… Provisioned IOPS
  34. 34. Hostile Networks
  35. 35. War Stories • Opaque ORMs • UTF-8 • Snapshots vs Perf • Practice failover
  36. 36. Please give us your feedback on this presentation DAT301 As a thank you, we will select prize winners daily for completed surveys!

×