AWS Webcast - Amazon RDS for MySQL: Best Practices and Migration

  • 3,187 views
Uploaded on

Amazon RDS makes it easy to set up, operate, and scale, relational databases in the cloud. Amazon RDS for MySQL supports applications that require up to tens of thousands of IOPS, and allows you to …

Amazon RDS makes it easy to set up, operate, and scale, relational databases in the cloud. Amazon RDS for MySQL supports applications that require up to tens of thousands of IOPS, and allows you to scale on demand without administrative complexity. In this webinar, we will discuss best practices for getting the most out of Amazon RDS for MySQL, as well as techniques for migrating data to and from the service.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,187
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
34
Comments
0
Likes
9

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Amazon RDS for MySQL: Best Practices and Data Migration Manish Dalwadi Sr. Product Manager Amazon Web Services
  • 2. Agenda 1. Introduction to Amazon RDS 2. Best Practices for High Availability 3. Best Practices for Disaster Recovery 4. Best Practices for High Performance 5. Best Practices for Data Migration
  • 3. 1: Introduction to Amazon RDS
  • 4. What is Amazon RDS? Amazon RDS is a web service that makes it easy to set up, operate, and scale a relational database in the cloud. Compatible with your existing apps. Easy to set up, operate, and scale. • Ease of deployment and patching • Automated backups and disaster recovery • Push button scalability • Failure detection and automatic recovery • Database monitoring and alerts
  • 5. 2: Best Practices for High Availability
  • 6. Multi-AZ deployments Synchronous Replication AZ1 AZ2 DNS cname update Primary Update Less than 2 minutes for failover New! New!
  • 7. Read Replicas – Read Availability Sync Replication Async Replication
  • 8. Logical Failover – Replicas of Replicas Sync Replication Async Replication Upgrade from MySQL 5.5 to MySQL 5.6 New!
  • 9. Things to consider – Read Replicas • MySQL replication is single-threaded • MySQL 5.1 and 5.5 – Replication may stop after crash recovery on the master – sync_binlog = 0 by default – Includes Multi-AZ failover – mysql.rds_next_master_log if potential missing info. is acceptable. • MySQL 5.6 – Crash safe slaves – sync_binlog = 1 by default – Less performance impact but more replica reliability
  • 10. 3: Best Practices for Disaster Recovery
  • 11. Backup and Restore Amazon S3 AZ1 AZ2 Log 2 Log 1 Log 5
  • 12. Cross Region – Snapshot Copy Amazon S3 AZ1 Region 1 Region 2 Amazon S3 AZ3
  • 13. Cross Region – Read Replicas AZ1 Region 1 Region 2 AZ3
  • 14. 4: Best Practices for High Performance
  • 15. Provisioned IOPS - Scale 0 2,500 5,000 7,500 10,000 12,500 15,000 17,500 20,000 T P S db.m2.4xlarge - 130GB Data - Partial Random Read Workload 333GB - Regular 333GB - 1000 PIOPS 300GB - 3000 PIOPS
  • 16. Provisioned IOPS - Latency 7.69% 1.38% 0.01% 7.89% 0.42% 0.00% 3.83% 0.00% 0.00% 0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 3-20 ms 20-500ms >500ms PercentageinLatencyBucket db.m2.4xlarge - 130GB Data - Partial Random Read Workload - Max Rate 333GB - Regular 333GB - 1000 PIOPS 300GB - 3000 PIOPS
  • 17. Provisioned IOPS - Scale 0 2,500 5,000 7,500 10,000 12,500 15,000 17,500 20,000 T P S db.m2.4xlarge - 130GB Data - Partial Random Read Workload 333GB - Regular 333GB - 1000 PIOPS 300GB - 3000 PIOPS Target = 5000
  • 18. Provisioned IOPS– Latency @ 5000 TPS 7.89% 1.17% 0.04% 0.35% 0.00% 0.00% 0.27% 0.00% 0.00% 0% 1% 2% 3% 4% 5% 6% 7% 8% 9% 3-20 ms 20-500ms >500ms PercentageinLatencyBucket db.m2.4xlarge - 130GB Data - Partial Random Read Workload - 5000 TPS 333GB - Regular 333GB - 1000 PIOPS 300GB - 3000 PIOPS
  • 19. Provisioned IOPS– 10,000 IOPS 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 T P S db.m2.4xlarge - 130GB Data - Partial Random Read Workload 333GB - Regular 333GB - 1000 PIOPS 300GB - 3000 PIOPS 1000GB-10000 PIOPS
  • 20. What’s the real limit? – CloudWatch Metrics
  • 21. Bottlenecks and Scaling DB Instance Class • db.t1.micro – db.cr1.8xlarge • 1-88 ECU • 0.6 – 244 GB RAM IOPS • Regular or 1,000-30,000 Storage • 5GB - 100GB - 3TB Ratio 3:1 to 10:1 Throughput • EBS Optimized (0.5 -1Gbit) • db.m1.large,db.m1.xlarge,m2.2xlarge,m2.4xlarge • db.cr1.8xlarge – 10Gbit • Regular + non optimized – shared Memory to ECU Ratio • 1.7 to 2.8 (exclude micro) DB Engine • Block Size 8 – 16K • Ability to utilize resources
  • 22. Read/Write Benchmark 5,850 13,200 42,000 47,000 - 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 TPS 130GB Data - Partial Random 90R/10W Workload 10K IOPS db.m1.large db.m1.xlarge db.m2.4xlarge db.cr1.8xlarge cpu 85% r/s 3450 w/s 600 rMB/s 52 cpu 100% r/s 6800 w/s 950 rMB/s 104 cpu 95% r/s 6800 w/s 2600 rMB/s 104 cpu 40% r/s 0 w/s 3200 rMB/s 0
  • 23. Bottlenecks and Scaling – Read Replicas
  • 24. Read/Write Benchmark – Using RR 5,850 13,200 42,000 47,000 126,000 - 20,000 40,000 60,000 80,000 100,000 120,000 TPS 130GB Data - Partial Random 90R/10W Workload – 10K IOPS db.m1.large db.m1.xlarge db.m2.4xlarge db.cr1.8xlarge cr1 + 3 x 4xlarge
  • 25. Things to consider – Provisioned IOPS • Use Provisioned IOPS optimized instances – m1.L, m3.XL, m2.2XL (500Mbps) – m1.XL, m3.2XL, m2.4XL (1000Mbps) – cr1.8XL (>1000 Mbps) • Understand Channel Bandwidth – Full duplex – 1000 Mbps ~ 100MBps (with protocol overhead) or – 100 MBps ~ 6250 16KB IOPS
  • 26. Things to consider – Provisioned IOPS • Max realizable IOPS – Workload dependent – 1:1 R/W -> Max realizable IOPS ~12.5K 16KB IOPS for M2.4XL – 1:1 R/W -> 20K 16KB IOPS for CR1.8XL • Provisioning more than Max can help lower latency • IO Size – IO Sizes <= 16KB is same – IO Sizes > 16KB consumes more IO – 6250 16KB IOPS = 3125 32KB IOPS
  • 27. Things to consider – Provisioned IOPS • Not able to realize IOPS provisioned? – Using Provisioned IOPS optimized instances? – Running automated backups, snapshots, scale storage? – Reviewed Queue Depth? – Database contention? Locking? Deadlocks?
  • 28. Other Best Practices • Storage Engine – Avoid MyISAM – not transactional • Cloudwatch alarms – CPU, Memory, Storage, Latency, Replica Lag • SMS/email notifications – Failover, Replication status • Number of Tables – Not more than 1000 tables (standard) and 10,000 tables (PIOPS)
  • 29. 5: Best Practices for Data Migration
  • 30. RDS Pre-Migration Steps • Take a snapshot • Disable backups • Use Single-AZ instances • Configure security for cross-DB traffic
  • 31. Importing from a MySQL DB Instance DB Application Staging area AWS Region Replication Application mysqldump scp Tsunami UDP Load data Staging server
  • 32. Create a DB Instance for MySQL and EC2 PROMPT>rds-create-db-instance mydbinstance -s 1024 -c db.m3.2xlarge -e MySQL - u <masterawsuser> -p <secretpassword> --backup-retention-period 0 Create DB instance for MySQL using AWS Management Console or CLI aws ec2 run-instances --image-id ami-xxxxxxxx --count 1 --instance-type m3.2xlarge --key-name MyKeyPair --security-groups MySecurityGroup Create Amazon EC2 (Staging server) using AWS Management Console or CLI mysql> GRANT SELECT,REPLICATION USER,REPLICATION CLIENT ON *.* TO repluser@‘<RDS Endpoint>' IDENTIFIED BY ‘<password>'; Create replication user on the master
  • 33. Configure the Master Database $ mysql -h localhost -u root -p mysql> show master statusG *************************** 1. row *************************** File: mysql-bin.000023 Position: 107 Binlog_Do_DB: mytest Binlog_Ignore_DB: 1 row in set (0.00 sec) Record the “File” and the “Position” values.
  • 34. Importing from a MySQL DB Instance
  • 35. Upload Files to Amazon EC2 using UDP • Tar and compress MySQL dump file preparation to ship to Amazon EC2 staging server. • Update the Amazon EC2 security group to allow UDP connection from the server where the dump file is being created to your new MySQL client server. • On the Amazon EC2 staging instance, untar the tar.tgz file.
  • 36. Configure the Amazon RDS database mysql> create database bench; Create the database Mysql> load data local infile '/reinvent/tables/customer_address.txt' into table customer_address fields terminated by ','; Mysql> load data local infile '/reinvent/tables/customer.txt' into table customer fields terminated by ','; Import the database that you previously exported from the master database mysql> call mysql.rds_set_external_master(‘<master server>',3306,‘<replicationuser>',‘<password>','mysql-bin.000013',107,0); mysql> call mysql.rds_start_replication; Configure the slave DB instance for MySQL, and start the slave server
  • 37. Make Amazon RDS instance the Master Switch over to the RDS instance – Stop the service/application that is pointing at the Master Database – Once all changes have been applied to New RDS Database. Stop replication with “call mysql.rds_stop_replication” – Point the service/application at the New RDS Database. – Once Migration is complete. “call mysql. rds_reset_external_master”
  • 38. RDS Post-migration Steps • Turn on backups • Turn on Multi-AZ • Tighten down security • Set up alarms for key metrics • Turn on notifications for Database Events
  • 39. References • Data Import Guide for MySQL • Best practices/operational guidelines • Using Amazon RDS Notifications
  • 40. Any questions?
  • 41. Thank you.