Monitoring MySQL at scale
•1 like•2,101 views
Report
Share
Download to read offline
Percona Live 2016 presentation on best practices to monitor large scale MySQL deployments.
Recommended
More Related Content
What's hot
What's hot(20)
Viewers also liked
Viewers also liked(19)
Similar to Monitoring MySQL at scale
Similar to Monitoring MySQL at scale(20)
Recently uploaded
Recently uploaded(20)
Monitoring MySQL at scale
- 1. Monitoring MySQL at SCALE
- 2. Who We Are Ilan Rabinovitch Dir. Technical Community Datadog Ovais Tariq Storage SRE Uber (formerly at Lithium & Percona)
- 3. Agenda 1. About Lithium and MySQL 2. Background: Monitoring Challenges in a Dynamic World 3. Theory: Monitoring 101 4. Practical: Triaging a Real Incident at Lithium
- 4. About Lithium Technologies Lithium’s platform helps brands connect, engage and understand their customers
- 5. MySQL Architecture / Data Flow •Multi-Tenant SaaS applications •Typical Master-slave replication setup •MySQL running ○ On bare metal ○ In AWS public cloud ○ In OpenStack
- 7. Culture Automation Metrics Sharing Damon Edwards and John Willis DevOps Day LA
- 8. Culture Automation Metrics Sharing Damon Edwards and John Willis DevOps Day LA
- 10. You’re in the cloud and it's everything you dreamed of! Autoscaling Infinite StorageManaged Databases Container Orchestration Private Clouds
- 12. Collecting data is cheap; not having it when you need it can be expensive
- 14. Instrument all the things!
- 15. Operational Complexity Increases with.. • Number of things to measure • Velocity of change
- 16. How much we measure? 1 instance • 10 metrics from CloudWatch 1 operating system (e.g., Linux) • 100 metrics MySQL Instance • 350~ metrics
- 25. •Earlier - typical Nagios and Cacti setup •Static config and lack of context •No correlation between alerts and graphs •No self-service for developers •In-house tooling has high cost
- 29. When to let a sleeping engineer lie?
- 31. Recurse until you find root cause
- 32. • Query Time • Queries Per Second Data Sources • Performance Schema • MySQL Status Variables
- 33. • Query Time • Queries Per Second Sources: • Performance Schema
- 34. • Disk Space Usage • Threads_connected • Threads_running • Connection_errors_ internal • Aborted_connects • Connection_errors_ max_connections Sources: ● Server Status Variables
- 35. • Configuration Change • Code Deployment • Service Started / Stopped • MySQL Upgrades • Failovers • etc
- 45. Change in workload without an increase in workload affected the schema ‘groupecasino’ • Workload characteristics change to make it more CPU bound • No increase in IO activity • Increase in number of read operations • No change in types of read operations • Similar number of range queries reading more rows
- 47. Monitoring 101: Alerting https://www.datadoghq.com/blog/monitoring-101-alerting/ Monitoring 101: Collecting the Right Data https://www.datadoghq.com/blog/monitoring-101-collecting-data/ Monitoring 101: Investigating performance issues https://www.datadoghq.com/blog/monitoring-101-investigation/ Monitoring MySQL Performance Metrics https://www.datadoghq.com/blog/monitoring-mysql-performance-metrics/ Collecting MySQL Metrics https://www.datadoghq.com/blog/collecting-mysql-statistics-and-metrics/