Getting to Know MySQL Enterprise Monitor

Getting To Know MySQL Enterprise Monitor Mark Leith (@MarkLeith)

Agenda
The MySQL Enterprise Monitor Vision
Architecture
A Tour Around The UI
Key Configuration
Questions

The MySQL Enterprise Monitor Team Vision
Continually monitor all aspects of MySQL environments
Analyze for problems, alerting when necessary
Streamline common “database problem” resolution
Simplify database management at scale

The MySQL Enterprise Monitor Vision
“ MySQL is simple to use ” ... but … there are many complex interactions happening within the database
Much of what is happening within MySQL has historically been termed as “ a black-box ” … but … that is changing rapidly
No one person can continually monitor what a single database is doing, let alone 10's, or 100's, of instances

The Obligatory Black Boxes

The MySQL Enterprise Monitor Way

Agenda
Architecture
Key Configuration
Questions

MEM Installation
A minimal installation consists of:
MEM Server (Tomcat app server, MySQL DB)
Ideally on dedicated resources
App server and DB can have separate hardware
MEM Agent (C based, multiple plugins)
Ideally installed on each database server
Can also monitor remotely (less OS stats)

MEM Architecture Tomcat App Server MySQL Repository Admins MEM Server MEM Agent Monitored DB Instance

MEM Installation – Statement Collection
MySQL Proxy integrated in to the Agent process
Funnel “legacy” app traffic for analysis
Data sent via the agent plugin
Connector Plugins for app side statement monitoring
Allows client side monitoring of statement latency
Allow tracing statement origination
Also spreads the load of statement analysis
Data sent directly to the MEM server via REST

MEM Architecture Java Web App MySQL Repository Admins Java App Server (or .Net) MEM Traffic App Traffic “ Legacy” App Users MEM Server Connector Plugins MEM Agent/Proxy

NEW – PHP and the Aggregator Admins MEM Server MEM Agent Monitored Instance PHP App MEM Traffic App Traffic MEM Aggregator

Agenda
Key Configuration
Questions

The Monitor Tab – Server Overview Key Graphs Current Critical Events Selected Server Overview Key Indicators

The Monitor Tab – Support Issues / Heat Chart
Waiting support issues
MOS account in settings
Status of agent and MySQL
Open event counts & drill down
Visual status of key statistics
See Heatchart Advisor to tune

MEM Advisors
A collection (>160) of simple expression based “ Rules ”
100-((%Threads_created% / %Connections%)*100) < THRESHOLD
All check standard best practices for MySQL
Scheduled to run on a set frequency basis
“ Counters” evaluate deltas, otherwise, evaluate totals
Rules trigger “Events” on three levels of “ Threshold ”
Info -> Warning -> Critical

The Advisors Tab – Current Schedule
Edit current settings
Per Group / Server
Quick enable/disable

Advisors Tab - Manage Rules Overview
>1000 MySQL metrics
SHOW STATUS
SHOW VARIABLES
SHOW SLAVE STATUS
SHOW INNODB STATUS
INFORMATION_SCHEMA
PERFORMANCE_SCHEMA
~100 OS metrics
Custom SQL possible
Create Custom Edit defaults Even app specific

Advisors Tab - Edit Rule
The Rule is the Expression
Certified rules allow threshold and frequency changes
Default values are generally conservative
Gather some graph data before refining

Events Tab Current / Worst Event states Where What When Drill In

Event Results Advice on the problem, and how to go about drilling in to it Recommended action to resolve the problem after investigation

Event Results The actual evaluation Advisor Config

Closing Events Whether to Continue Monitoring this rule Track a history of changes made for events Choose Auto-Close wisely: Flapping events notify, a lot

The Graphs Tab
Stacked for easy correlation
Covering key subsystems
InnoDB, Cluster, SQL layer activity, Query Cache, Thread Pool, Connection usage, Statement, Transaction and Row throughput, OS stats etc.
~75 Graphs
~182 series ~220 data points

Graphs and Filtering
When groups are selected graphs summarize (avg/min/max)
Drill in to servers to see raw stats
Filter by name with “Contains” matching or regex – i.e “^row”
..or just select specific graphs
Time range filtering for history
Keep defaults minimal
Data points every minute

Graphs – Query Correlation Select any portion of any graph to take you to a filtered Query Analyzer view for that time period

The Query Analyzer Tab Correlated Graph Advanced Filtering Normalized Statements (drill down) Key Aggregated Statistics Table Scan

Statement Normalization
Normalization does the following:
Strip all literals, replacing each with “?”
Strip pure comments, leading/trailing spaces
Make keywords UPPER CASE
Roll multi-row INSERT up to “(?,?) /*, … */”
Roll IN() lists to “IN (? /*, … */)”

Query Analysis – Example Queries Non-normalized Example statement Who it was executed by Where in the application it originated from (with connector plugins) When, and how long it took

Query Analysis – Explain / History How the query executed over time (rows, bytes, time, execs) Execution plan if captured

The Replication Tab

“ Your replication page shows our replication topology incorrectly, that server isn't a slave any more... ” Actually... Your topology probably is configured like that : CHANGE MASTER TO MASTER _HOST = '' RESET SLAVE ALL (now in 5.5.16)

The Replication Slave Details

Agenda
Key Configuration
Questions

Key Configuration Of course, the first thing to get right is Advisors..
Let the monitor run for a while with no rules enabled
Be selective on which rules you enable
When enabling rules, use graph info for thresholds
Set notifications wisely (do you need an email alert at 5 am that a user exists without a password?)

General Settings Notification mechanisms Data retention MOS credentials

Notification Groups Short Messages MEM Server Events Custom Event subjects

Query Analysis Configuration Collect Normalized Data Also get non-normalized example Threshold for how Long a statement Should run, before Generating an EXPLAIN

Key Configuration Files $mem_server_dir/apache-tomcat/bin/setenv.sh
Configure Java options (heap space)
$mem_server_dir/mysql/my.cnf
Database options – For larger installs, tune InnoDB
$mem_agent_dir/share/items/items-mysql-monitor.xml
Tweak SQL based data collection items

Tweaking / Adding SQL Collections <class> <namespace>mysql</namespace> <classname>foo_bar</classname> <query><![CDATA[SELECT foo, bar, FROM … ]]></query> <attributes> <attribute name="foo" counter="false" type="STRING" /> <attribute name="bar" counter="true" type="INTEGER" /> </attributes> </class>

Some resources..
Download - https://edelivery.oracle.com/
http://dev.mysql.com/doc/mysql-monitor/2.3/en/index.html
http://www.slideshare.net/Leithal/extending-mysql-enterprise-monitor?type=powerpoint

http://www.markleith.co.uk	527
http://blogs.oracle.com	168
https://blogs.oracle.com	151
http://us-w1.rockmelt.com	7
http://s386513550.websitehome.co.uk	6
http://localhost	4
http://www.hanrss.com	3
http://sqldump.cool-tools.co.uk	2
http://feeds.iti.bz	1
http://rscat.com	1
http://www.netvibes.com	1
http://webcache.googleusercontent.com	1
https://blogs.oraclegha.com	1

Getting to Know MySQL Enterprise Monitor

by Mark Leith

Accessibility

Categories

Upload Details

Usage Rights

13 Embeds 873

Statistics