SlideShare a Scribd company logo
1 of 33
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vivek R – AWS Solutions Architect
Ricky Chang – AWS Professional Services
February 23, 2017
Optimize MySQL Workloads with
Amazon Elastic Block Store
Agenda
• MySQL Deployment Options on AWS
• Why use Elastic Block Store (EBS)?
• Optimizing MySQL with EBS
• Testing for Performance
• Best Practices
MySQL Deployment Options
RDS and MySQL on EC2
MySQL on
EC2 instance
• Flexible
• Deployment Options
- Infrastructure
- Storage
• Access to the OS
• MySQL plugins support
RDS for MySQL
• No Infrastructure Management
• Instant provisioning
• Application Compatibility
• HA and Read Replica
• Managed Patching, Maintenance,
Backup and Restore
Why Storage?
Durability and Monitoring
Monitoring
CloudWatch
• 5 min or 1 min polling interval
• Setup alarms and actions
• Integrate with MySQL Monitoring
Tools
Durability
• Data Persistence vs. Instance
Store
• Decouple Storage from Compute
• Redundancy within the AZ
Provisioned IOPS SSD
Highest Performance
Volume Size: 4 GiB to 16 TiB
Max IOPS/Volume: 20,000
Max Throughput/Volume: 320 MiB/s
General Purpose SSD
Price and Performance
Volume Size: 1 GiB to 16 TiB
Max IOPS/Volume: 10,000
Max Throughput/Volume: 160 MiB/s
Primer - Storage Options
Snapshots and Security
Security
• IAM Resource Level Permissions
• Data at Rest Encryption
Snapshots
• Entire MySQL Database Backup
• Scheduled snapshots using
CloudWatch Events
• Use for DR, Storage Resize,
Sharing Environment
Elastic Volumes
aws ec2 modify-volume --region us-east-1 --volume-id vol-11111111111111111 --size 200 --volume-type io1 --iops 10000
{
"VolumeModification": {
"TargetSize": 200,
"TargetVolumeType": "io1",
"ModificationState": "modifying",
"VolumeId": "vol-11111111111111111",
"TargetIops": 10000,
"StartTime": "2017-01-19T22:21:02.959Z",
"Progress": 0,
"OriginalVolumeType": "gp2",
"OriginalIops": 300,
"OriginalSize": 100
}
}
Elastic Volumes
• Ease of Administration
• Change while online
• Change Volume Type
• Change Provisioned
IOPS
• Increase volume size
Optimization
© 2016, Amazon Web Services, Inc. or its Affiliates. | Amazon Confidential
Making the most out of (GP2)
Max I/O credit per bucket is 5.4M
You can spend up to
3000 IOPS per second
Baseline performance = 3 IOPS per GiB or 100 IOPS
Always accumulating
3 IOPS per GiB per second
gp2
© 2016, Amazon Web Services, Inc. or its Affiliates. | Amazon Confidential
Burst & Baseline: General Purpose SSD (GP2)IOPS
0 1 16
1,000
2,000
3,000
8,000
10,000
BASELINE IOPS
(Baseline of 3 IOPS/GiB)
Burstable to
3,000 IOPS
3 90.5
Volume Size (TiB)
~ 3334 GB
© 2016, Amazon Web Services, Inc. or its Affiliates. | Amazon Confidential
Time to Deplete a Full GP2 Credit Bucket
0
100
200
300
400
500
600
700
1 8 30 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950
MinutesofBurst
Volume Size in GiB
43 Min 1 Hour
10 Hours
Monitor your Burst usage
Typically, Burst IOPS are consumed by:
 Performing MySQL Backups
 Restarting MySQL or EC2
 Large import and export of MySQL data
 Additional traffic to the MySQL database
coming from the application
Making the most out of io1
EBS Optimized Instances
• Dedicated Bandwidth to EBS 500 Mbps to 12,000 Mbps
• Increase Throughput and IOPS
Making the most out of Replication
Need more IOPS and Throughput?
RAID 0 - Considerations
• Use EBS optimized instance
• Fault tolerance, MySQL Slave
• Use the same EBS volume type
and size
• Ensure all volumes are quiesced
when performing EBS snapshots
Caching and Writes
MySQL Configuration
Parameters Default Tuned* Description
innodb_buffer_pool_size 128MB 70-80% % of Memory or large enough for “Hot” data
set
innodb_flush_neighbors 1 0 Spreads write operations over your EBS
volumes
innodb_log_file_size 50MB * Large enough to spread-out the checkpoint
flush activity
* Based on your write workloads
MySQL Master/Slave
Switching from a Physical Server to EC2
sync_binlog
sync_master_info
sync_relay_log
sync_relay_log_info
Improve IO and Throughput
• Separate the logs and data files
into different EBS volumes
• Use io1 for bin and relay logs
• RAID 0 – Stripe EBS Volumes
• Extreme cases – use local SSD
for Slave or read replica
Backup Server as target
EFS and S3 as Target
EBS Snapshots – best practices
1. Quiesce data before taking snapshots
flush tables with read lock
sync and fsfreeze
2. Warm up data during the restore
Parallel dd, fio, robocopy
Scan largest table
3. Automate and Manage Snapshots
http://docs.aws.amazon.com/AmazonCloudWatch/latest/events/TakeScheduledSnapshot.html
https://github.com/awslabs/aws-ebs-custodian
Performance Testing
Latency
• Check for available
IOPS
• Verify burst credits
for gp2
• Watch out for high
disk queue length
Throughput
Issues:
• Database replication
• Backup and Restore
• Import/Export
• Index Creation
Increase Throughput by:
• RAID 0
• Amazon Provisioned IOPS SSD
320 MiB/s
• EBS Optimized Instance
Benchmarking Tools
• Tools from MySQL such as
MySQL Enterprise Monitor
MySQL Workbench Performance
MySQL Query Analyzer
• MySQL Monitoring tools at the AWS Marketplace
• Amazon CloudWatch
• MySQL performance Schema
Demo
Best Practices
Best Practices
• Evaluate use case for Amazon RDS or MySQL on EC2
• Monitor burst metrics for gp2
• Use EBS optimized EC2 instances for io1
• Use EC2 placement groups for replication
• Test your MySQL workloads
• Verify MySQL parameters
• Monitor service limits
Thank you!

More Related Content

What's hot

What's hot (20)

Best Practices running SQL Server on AWS
Best Practices running SQL Server on AWSBest Practices running SQL Server on AWS
Best Practices running SQL Server on AWS
 
Deep Dive on Amazon RDS
Deep Dive on Amazon RDSDeep Dive on Amazon RDS
Deep Dive on Amazon RDS
 
Deep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance PerformanceDeep Dive on Delivering Amazon EC2 Instance Performance
Deep Dive on Delivering Amazon EC2 Instance Performance
 
Deep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Deep Dive on Elastic File System - February 2017 AWS Online Tech TalksDeep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Deep Dive on Elastic File System - February 2017 AWS Online Tech Talks
 
Getting Started with Amazon Aurora
Getting Started with Amazon AuroraGetting Started with Amazon Aurora
Getting Started with Amazon Aurora
 
NEW LAUNCH! Introducing PostgreSQL compatibility for Amazon Aurora
NEW LAUNCH! Introducing PostgreSQL compatibility for Amazon AuroraNEW LAUNCH! Introducing PostgreSQL compatibility for Amazon Aurora
NEW LAUNCH! Introducing PostgreSQL compatibility for Amazon Aurora
 
Getting started with amazon aurora - Toronto
Getting started with amazon aurora - TorontoGetting started with amazon aurora - Toronto
Getting started with amazon aurora - Toronto
 
Deep Dive on Elastic Load Balancing
Deep Dive on Elastic Load BalancingDeep Dive on Elastic Load Balancing
Deep Dive on Elastic Load Balancing
 
Amazon Web Services - Relational Database Service Meetup
Amazon Web Services - Relational Database Service MeetupAmazon Web Services - Relational Database Service Meetup
Amazon Web Services - Relational Database Service Meetup
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
Introduction to Block and File storage on AWS
Introduction to Block and File storage on AWSIntroduction to Block and File storage on AWS
Introduction to Block and File storage on AWS
 
Simple, Scalable and Highly Durable NAS in the Cloud – Amazon EFS
Simple, Scalable and Highly Durable NAS in the Cloud – Amazon EFSSimple, Scalable and Highly Durable NAS in the Cloud – Amazon EFS
Simple, Scalable and Highly Durable NAS in the Cloud – Amazon EFS
 
Getting Started with Amazon EC2 and Compute Services
Getting Started with Amazon EC2 and Compute ServicesGetting Started with Amazon EC2 and Compute Services
Getting Started with Amazon EC2 and Compute Services
 
SRV401 Deep Dive on Amazon Elastic File System (Amazon EFS)
SRV401 Deep Dive on Amazon Elastic File System (Amazon EFS)SRV401 Deep Dive on Amazon Elastic File System (Amazon EFS)
SRV401 Deep Dive on Amazon Elastic File System (Amazon EFS)
 
SRV401 Deep Dive on Amazon Elastic File System (Amazon EFS)
SRV401 Deep Dive on Amazon Elastic File System (Amazon EFS)SRV401 Deep Dive on Amazon Elastic File System (Amazon EFS)
SRV401 Deep Dive on Amazon Elastic File System (Amazon EFS)
 
Database Migration – Simple, Cross-Engine and Cross-Platform Migration
Database Migration – Simple, Cross-Engine and Cross-Platform MigrationDatabase Migration – Simple, Cross-Engine and Cross-Platform Migration
Database Migration – Simple, Cross-Engine and Cross-Platform Migration
 
Consolidate MySQL Shards Into Amazon Aurora Using AWS Database Migration Serv...
Consolidate MySQL Shards Into Amazon Aurora Using AWS Database Migration Serv...Consolidate MySQL Shards Into Amazon Aurora Using AWS Database Migration Serv...
Consolidate MySQL Shards Into Amazon Aurora Using AWS Database Migration Serv...
 
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech TalksDeep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
Deep Dive on Amazon EC2 Instances - January 2017 AWS Online Tech Talks
 
What's New in Amazon RDS for Open Source and Commercial Databases
What's New in Amazon RDS for Open Source and Commercial DatabasesWhat's New in Amazon RDS for Open Source and Commercial Databases
What's New in Amazon RDS for Open Source and Commercial Databases
 
Amazon RDS for Microsoft SQL: Performance, Security, Best Practices (DAT303) ...
Amazon RDS for Microsoft SQL: Performance, Security, Best Practices (DAT303) ...Amazon RDS for Microsoft SQL: Performance, Security, Best Practices (DAT303) ...
Amazon RDS for Microsoft SQL: Performance, Security, Best Practices (DAT303) ...
 

Viewers also liked

AWS CLOUD 2017 - AWS 코어팀과 함께하는 고객 성공 전략 (황인철 상무 & 박성훈 테크니컬 어카운트 매니저 & 김소희 컨설턴트)
AWS CLOUD 2017 - AWS 코어팀과 함께하는 고객 성공 전략 (황인철 상무 & 박성훈 테크니컬 어카운트 매니저 & 김소희 컨설턴트)AWS CLOUD 2017 - AWS 코어팀과 함께하는 고객 성공 전략 (황인철 상무 & 박성훈 테크니컬 어카운트 매니저 & 김소희 컨설턴트)
AWS CLOUD 2017 - AWS 코어팀과 함께하는 고객 성공 전략 (황인철 상무 & 박성훈 테크니컬 어카운트 매니저 & 김소희 컨설턴트)
Amazon Web Services Korea
 
Amazon 인공 지능(AI) 서비스 및 AWS 기반 딥러닝 활용 방법 - 윤석찬 (AWS, 테크에반젤리스트)
Amazon 인공 지능(AI) 서비스 및 AWS 기반 딥러닝 활용 방법 - 윤석찬 (AWS, 테크에반젤리스트)Amazon 인공 지능(AI) 서비스 및 AWS 기반 딥러닝 활용 방법 - 윤석찬 (AWS, 테크에반젤리스트)
Amazon 인공 지능(AI) 서비스 및 AWS 기반 딥러닝 활용 방법 - 윤석찬 (AWS, 테크에반젤리스트)
Amazon Web Services Korea
 

Viewers also liked (20)

Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...
Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...
Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...
 
AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...
AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...
AWS re:Invent 2016: [JK REPEAT] Deep Dive on Amazon EC2 Instances, Featuring ...
 
Building A Modern Data Analytics Architecture on AWS
Building A Modern Data Analytics Architecture on AWSBuilding A Modern Data Analytics Architecture on AWS
Building A Modern Data Analytics Architecture on AWS
 
Introduction to DevSecOps on AWS
Introduction to DevSecOps on AWSIntroduction to DevSecOps on AWS
Introduction to DevSecOps on AWS
 
AWS re:Invent 2016: AWS Database State of the Union (DAT320)
AWS re:Invent 2016: AWS Database State of the Union (DAT320)AWS re:Invent 2016: AWS Database State of the Union (DAT320)
AWS re:Invent 2016: AWS Database State of the Union (DAT320)
 
AWS CLOUD 2017 - AWS 코어팀과 함께하는 고객 성공 전략 (황인철 상무 & 박성훈 테크니컬 어카운트 매니저 & 김소희 컨설턴트)
AWS CLOUD 2017 - AWS 코어팀과 함께하는 고객 성공 전략 (황인철 상무 & 박성훈 테크니컬 어카운트 매니저 & 김소희 컨설턴트)AWS CLOUD 2017 - AWS 코어팀과 함께하는 고객 성공 전략 (황인철 상무 & 박성훈 테크니컬 어카운트 매니저 & 김소희 컨설턴트)
AWS CLOUD 2017 - AWS 코어팀과 함께하는 고객 성공 전략 (황인철 상무 & 박성훈 테크니컬 어카운트 매니저 & 김소희 컨설턴트)
 
AWS re:Invent 2016: Design Patterns for High Availability: Lessons from Amazo...
AWS re:Invent 2016: Design Patterns for High Availability: Lessons from Amazo...AWS re:Invent 2016: Design Patterns for High Availability: Lessons from Amazo...
AWS re:Invent 2016: Design Patterns for High Availability: Lessons from Amazo...
 
Accelerating the Transition to Broadcast and OTT Infrastructure in the Cloud
Accelerating the Transition to Broadcast and OTT Infrastructure in the CloudAccelerating the Transition to Broadcast and OTT Infrastructure in the Cloud
Accelerating the Transition to Broadcast and OTT Infrastructure in the Cloud
 
Best Practices with IoT Security - February Online Tech Talks
Best Practices with IoT Security - February Online Tech TalksBest Practices with IoT Security - February Online Tech Talks
Best Practices with IoT Security - February Online Tech Talks
 
Bases de datos en la nube con AWS
Bases de datos en la nube con AWSBases de datos en la nube con AWS
Bases de datos en la nube con AWS
 
천만 사용자를 위한 AWS 아키텍처 보안 모범 사례 (윤석찬, 테크에반젤리스트)
천만 사용자를 위한 AWS 아키텍처 보안 모범 사례 (윤석찬, 테크에반젤리스트)천만 사용자를 위한 AWS 아키텍처 보안 모범 사례 (윤석찬, 테크에반젤리스트)
천만 사용자를 위한 AWS 아키텍처 보안 모범 사례 (윤석찬, 테크에반젤리스트)
 
Amazon 인공 지능(AI) 서비스 및 AWS 기반 딥러닝 활용 방법 - 윤석찬 (AWS, 테크에반젤리스트)
Amazon 인공 지능(AI) 서비스 및 AWS 기반 딥러닝 활용 방법 - 윤석찬 (AWS, 테크에반젤리스트)Amazon 인공 지능(AI) 서비스 및 AWS 기반 딥러닝 활용 방법 - 윤석찬 (AWS, 테크에반젤리스트)
Amazon 인공 지능(AI) 서비스 및 AWS 기반 딥러닝 활용 방법 - 윤석찬 (AWS, 테크에반젤리스트)
 
Introducing Amazon Lex – A Service for Building Voice or Text Chatbots - Marc...
Introducing Amazon Lex – A Service for Building Voice or Text Chatbots - Marc...Introducing Amazon Lex – A Service for Building Voice or Text Chatbots - Marc...
Introducing Amazon Lex – A Service for Building Voice or Text Chatbots - Marc...
 
Towards Full Stack Security
Towards Full Stack Security Towards Full Stack Security
Towards Full Stack Security
 
AWS Services for Content Production
AWS Services for Content ProductionAWS Services for Content Production
AWS Services for Content Production
 
Deliver and monetize your content with video center operations on aws
Deliver and monetize your content with video center operations on awsDeliver and monetize your content with video center operations on aws
Deliver and monetize your content with video center operations on aws
 
AWS re:Invent 2016: Extending Datacenters to the Cloud: Connectivity Options ...
AWS re:Invent 2016: Extending Datacenters to the Cloud: Connectivity Options ...AWS re:Invent 2016: Extending Datacenters to the Cloud: Connectivity Options ...
AWS re:Invent 2016: Extending Datacenters to the Cloud: Connectivity Options ...
 
DDoS Resiliency
DDoS ResiliencyDDoS Resiliency
DDoS Resiliency
 
AWS re:Invent 2016: Become an AWS IAM Policy Ninja in 60 Minutes or Less (SAC...
AWS re:Invent 2016: Become an AWS IAM Policy Ninja in 60 Minutes or Less (SAC...AWS re:Invent 2016: Become an AWS IAM Policy Ninja in 60 Minutes or Less (SAC...
AWS re:Invent 2016: Become an AWS IAM Policy Ninja in 60 Minutes or Less (SAC...
 
The Best of re:invent 2016
The Best of re:invent 2016The Best of re:invent 2016
The Best of re:invent 2016
 

Similar to Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS Online Tech Talks

Similar to Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS Online Tech Talks (20)

AWS Summit Seoul 2015 - EBS 성능 향상 및 EC2 비용 최적화 기법
AWS Summit Seoul 2015 - EBS 성능 향상 및 EC2 비용 최적화 기법AWS Summit Seoul 2015 - EBS 성능 향상 및 EC2 비용 최적화 기법
AWS Summit Seoul 2015 - EBS 성능 향상 및 EC2 비용 최적화 기법
 
(STG402) Amazon EBS Deep Dive
(STG402) Amazon EBS Deep Dive(STG402) Amazon EBS Deep Dive
(STG402) Amazon EBS Deep Dive
 
Deep Dive: Maximizing EC2 and EBS Performance
Deep Dive: Maximizing EC2 and EBS PerformanceDeep Dive: Maximizing EC2 and EBS Performance
Deep Dive: Maximizing EC2 and EBS Performance
 
What's New in Amazon Relational Database Service (DAT203) - AWS re:Invent 2018
What's New in Amazon Relational Database Service (DAT203) - AWS re:Invent 2018What's New in Amazon Relational Database Service (DAT203) - AWS re:Invent 2018
What's New in Amazon Relational Database Service (DAT203) - AWS re:Invent 2018
 
Deep Dive: Maximizing Amazon EC2 and Amazon Elastic Block Store Performance
Deep Dive: Maximizing Amazon EC2 and Amazon Elastic Block Store PerformanceDeep Dive: Maximizing Amazon EC2 and Amazon Elastic Block Store Performance
Deep Dive: Maximizing Amazon EC2 and Amazon Elastic Block Store Performance
 
Deep Dive: Maximizing Amazon EC2 and Amazon Elastic Block Store Performance
Deep Dive: Maximizing Amazon EC2 and Amazon Elastic Block Store PerformanceDeep Dive: Maximizing Amazon EC2 and Amazon Elastic Block Store Performance
Deep Dive: Maximizing Amazon EC2 and Amazon Elastic Block Store Performance
 
SRV413 Deep Dive on Elastic Block Storage (Amazon EBS)
SRV413 Deep Dive on Elastic Block Storage (Amazon EBS)SRV413 Deep Dive on Elastic Block Storage (Amazon EBS)
SRV413 Deep Dive on Elastic Block Storage (Amazon EBS)
 
AWS Activate webinar - Scalable databases for fast growing startups
AWS Activate webinar - Scalable databases for fast growing startupsAWS Activate webinar - Scalable databases for fast growing startups
AWS Activate webinar - Scalable databases for fast growing startups
 
Maximizing EC2 and Elastic Block Store Disk Performance
Maximizing EC2 and Elastic Block Store Disk PerformanceMaximizing EC2 and Elastic Block Store Disk Performance
Maximizing EC2 and Elastic Block Store Disk Performance
 
AWS Webcast - Introduction to EBS
AWS Webcast - Introduction to EBS AWS Webcast - Introduction to EBS
AWS Webcast - Introduction to EBS
 
Amazon RDS_Deep Dive - SRV310
Amazon RDS_Deep Dive - SRV310 Amazon RDS_Deep Dive - SRV310
Amazon RDS_Deep Dive - SRV310
 
AWS Webcast - Cost and Performance Optimization in Amazon RDS
AWS Webcast - Cost and Performance Optimization in Amazon RDSAWS Webcast - Cost and Performance Optimization in Amazon RDS
AWS Webcast - Cost and Performance Optimization in Amazon RDS
 
SRV310 Optimizing Relational Databases on AWS: Deep Dive on Amazon RDS
 SRV310 Optimizing Relational Databases on AWS: Deep Dive on Amazon RDS SRV310 Optimizing Relational Databases on AWS: Deep Dive on Amazon RDS
SRV310 Optimizing Relational Databases on AWS: Deep Dive on Amazon RDS
 
Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018
 
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
 
DAT302_Deep Dive on Amazon Relational Database Service (RDS)
DAT302_Deep Dive on Amazon Relational Database Service (RDS)DAT302_Deep Dive on Amazon Relational Database Service (RDS)
DAT302_Deep Dive on Amazon Relational Database Service (RDS)
 
Amazon Elastic Block Store for Application Storage
Amazon Elastic Block Store for Application StorageAmazon Elastic Block Store for Application Storage
Amazon Elastic Block Store for Application Storage
 
SRV413 Deep Dive on Elastic Block Storage (Amazon EBS)
SRV413 Deep Dive on Elastic Block Storage (Amazon EBS)SRV413 Deep Dive on Elastic Block Storage (Amazon EBS)
SRV413 Deep Dive on Elastic Block Storage (Amazon EBS)
 
Amazon RDS: Deep Dive - SRV310 - Chicago AWS Summit
Amazon RDS: Deep Dive - SRV310 - Chicago AWS SummitAmazon RDS: Deep Dive - SRV310 - Chicago AWS Summit
Amazon RDS: Deep Dive - SRV310 - Chicago AWS Summit
 
Understanding Elastic Block Store Availability and Performance
Understanding Elastic Block Store Availability and PerformanceUnderstanding Elastic Block Store Availability and Performance
Understanding Elastic Block Store Availability and Performance
 

More from Amazon Web Services

Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
Amazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Recently uploaded

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 

Optimize MySQL Workloads with Amazon Elastic Block Store - February 2017 AWS Online Tech Talks

  • 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Vivek R – AWS Solutions Architect Ricky Chang – AWS Professional Services February 23, 2017 Optimize MySQL Workloads with Amazon Elastic Block Store
  • 2. Agenda • MySQL Deployment Options on AWS • Why use Elastic Block Store (EBS)? • Optimizing MySQL with EBS • Testing for Performance • Best Practices
  • 4. RDS and MySQL on EC2 MySQL on EC2 instance • Flexible • Deployment Options - Infrastructure - Storage • Access to the OS • MySQL plugins support RDS for MySQL • No Infrastructure Management • Instant provisioning • Application Compatibility • HA and Read Replica • Managed Patching, Maintenance, Backup and Restore
  • 6. Durability and Monitoring Monitoring CloudWatch • 5 min or 1 min polling interval • Setup alarms and actions • Integrate with MySQL Monitoring Tools Durability • Data Persistence vs. Instance Store • Decouple Storage from Compute • Redundancy within the AZ
  • 7. Provisioned IOPS SSD Highest Performance Volume Size: 4 GiB to 16 TiB Max IOPS/Volume: 20,000 Max Throughput/Volume: 320 MiB/s General Purpose SSD Price and Performance Volume Size: 1 GiB to 16 TiB Max IOPS/Volume: 10,000 Max Throughput/Volume: 160 MiB/s Primer - Storage Options
  • 8. Snapshots and Security Security • IAM Resource Level Permissions • Data at Rest Encryption Snapshots • Entire MySQL Database Backup • Scheduled snapshots using CloudWatch Events • Use for DR, Storage Resize, Sharing Environment
  • 9. Elastic Volumes aws ec2 modify-volume --region us-east-1 --volume-id vol-11111111111111111 --size 200 --volume-type io1 --iops 10000 { "VolumeModification": { "TargetSize": 200, "TargetVolumeType": "io1", "ModificationState": "modifying", "VolumeId": "vol-11111111111111111", "TargetIops": 10000, "StartTime": "2017-01-19T22:21:02.959Z", "Progress": 0, "OriginalVolumeType": "gp2", "OriginalIops": 300, "OriginalSize": 100 } } Elastic Volumes • Ease of Administration • Change while online • Change Volume Type • Change Provisioned IOPS • Increase volume size
  • 11. © 2016, Amazon Web Services, Inc. or its Affiliates. | Amazon Confidential Making the most out of (GP2) Max I/O credit per bucket is 5.4M You can spend up to 3000 IOPS per second Baseline performance = 3 IOPS per GiB or 100 IOPS Always accumulating 3 IOPS per GiB per second gp2
  • 12. © 2016, Amazon Web Services, Inc. or its Affiliates. | Amazon Confidential Burst & Baseline: General Purpose SSD (GP2)IOPS 0 1 16 1,000 2,000 3,000 8,000 10,000 BASELINE IOPS (Baseline of 3 IOPS/GiB) Burstable to 3,000 IOPS 3 90.5 Volume Size (TiB) ~ 3334 GB
  • 13. © 2016, Amazon Web Services, Inc. or its Affiliates. | Amazon Confidential Time to Deplete a Full GP2 Credit Bucket 0 100 200 300 400 500 600 700 1 8 30 100 150 200 250 300 350 400 450 500 550 600 650 700 750 800 850 900 950 MinutesofBurst Volume Size in GiB 43 Min 1 Hour 10 Hours
  • 14. Monitor your Burst usage Typically, Burst IOPS are consumed by:  Performing MySQL Backups  Restarting MySQL or EC2  Large import and export of MySQL data  Additional traffic to the MySQL database coming from the application
  • 15. Making the most out of io1 EBS Optimized Instances • Dedicated Bandwidth to EBS 500 Mbps to 12,000 Mbps • Increase Throughput and IOPS
  • 16. Making the most out of Replication
  • 17. Need more IOPS and Throughput? RAID 0 - Considerations • Use EBS optimized instance • Fault tolerance, MySQL Slave • Use the same EBS volume type and size • Ensure all volumes are quiesced when performing EBS snapshots
  • 19. MySQL Configuration Parameters Default Tuned* Description innodb_buffer_pool_size 128MB 70-80% % of Memory or large enough for “Hot” data set innodb_flush_neighbors 1 0 Spreads write operations over your EBS volumes innodb_log_file_size 50MB * Large enough to spread-out the checkpoint flush activity * Based on your write workloads
  • 21. Switching from a Physical Server to EC2 sync_binlog sync_master_info sync_relay_log sync_relay_log_info
  • 22. Improve IO and Throughput • Separate the logs and data files into different EBS volumes • Use io1 for bin and relay logs • RAID 0 – Stripe EBS Volumes • Extreme cases – use local SSD for Slave or read replica
  • 24. EFS and S3 as Target
  • 25. EBS Snapshots – best practices 1. Quiesce data before taking snapshots flush tables with read lock sync and fsfreeze 2. Warm up data during the restore Parallel dd, fio, robocopy Scan largest table 3. Automate and Manage Snapshots http://docs.aws.amazon.com/AmazonCloudWatch/latest/events/TakeScheduledSnapshot.html https://github.com/awslabs/aws-ebs-custodian
  • 27. Latency • Check for available IOPS • Verify burst credits for gp2 • Watch out for high disk queue length
  • 28. Throughput Issues: • Database replication • Backup and Restore • Import/Export • Index Creation Increase Throughput by: • RAID 0 • Amazon Provisioned IOPS SSD 320 MiB/s • EBS Optimized Instance
  • 29. Benchmarking Tools • Tools from MySQL such as MySQL Enterprise Monitor MySQL Workbench Performance MySQL Query Analyzer • MySQL Monitoring tools at the AWS Marketplace • Amazon CloudWatch • MySQL performance Schema
  • 30. Demo
  • 32. Best Practices • Evaluate use case for Amazon RDS or MySQL on EC2 • Monitor burst metrics for gp2 • Use EBS optimized EC2 instances for io1 • Use EC2 placement groups for replication • Test your MySQL workloads • Verify MySQL parameters • Monitor service limits