Extending MySQL Enterprise Monitor


Published on

Talk at the MySQL User Conference 2009 about extending MEM with new data collections, graphs and rules

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Extending MySQL Enterprise Monitor

  1. 1. Extending MEM, Custom Advisors, Graphs and Data Collections Mark Leith Sun Microsystems (leith@sun.com)
  2. 2. Agenda  Quick MEM Agent Architecture  Creating Data Collections  Viewing Data Collection Information  Creating Custom Graphs  Creating and Modifying Advisors  Questions / Input
  3. 3. Quick MEM Agent Architecture
  4. 4. Architecture – MEM Agent
  5. 5. Creating Data Collections
  6. 6. Data Collection Overview  All Data Collections defined on each Agent installation locally  Defaults are within the “./agent/share/mysql-proxy” directory  SQL and Lua based collections supported  SQL statements defined in XML format  QUAN based collections are all Lua via the Proxy
  7. 7. Internal Data Collection Format  Defined in XML files in the Advisor bundles  Here's our definition of the innodb_buffer_pool_size variable: <itemList> <attribName>innodb_buffer_pool_size</attribName> <nameSpace>mysql</nameSpace> <attribType>INTEGER</attribType> <className>variables</className> <categoryName>Memory</categoryName> <subCategoryName>Buffers</subCategoryName> <isCounter>false</isCounter> </itemList>
  8. 8. Internal Data Collection Format  Types: nameSpace::className::attribName i.e: mysql::variables::innodb_buffer_pool_size or os::cpu::cpu_user  attribType – INTEGER | VARCHAR | DECIMAL  isCounter – Whether the variable is a counter (true) or point in time value (false)  Counter based variables are always evaluated using the delta between “now” and “now – 1” collections  Multiple instances of attributes supported internally (for things such as CPUs, disks, etc.)  Single instance collections use a “local” instance
  9. 9. SQL Based Data Collections  Defaults in: ./agent/share/mysql-proxy/items/items-mysql-monitor.xml <class> <namespace>mysql</namespace> <classname>classTitle</classname> <query><![CDATA[SELECT ...]]></query> </class>  SELECT must return one row (others ignored with warnings in the log file)  Can return multiple columns in 2.x ● Assign <classname> in column order  Note: <isCounter> is not available until 2.1 here
  10. 10. SQL Based Data Collections  Collections can be added to the default, or a new file  New files should be added to the agent-item-files variable in the mysql-monitor-agent.ini file  Let's try an example with the new PERFORMANCE_SCHEMA! <class> <namespace>mysql</namespace> <classname>wait_sum_open_mutex</classname> <query><![CDATA[ SELECT sum_timer_wait FROM PERFORMANCE_SCHEMA.EVENTS_WAITS_SUMMARY_BY_EVENT_NAME WHERE EVENT_NAME = 'wait/synch/mutex/sql/LOCK_open' ]]></query> </class>
  11. 11. Lua Based Data Collections  Lua is a powerful embedded language – many possibilities exist to collect data (running external commands, gathering external data by reading files, etc.)  Lua collections also go to: ./agent/share/mysql-proxy/items/  They should also be added to the agent-items- files variable in the mysql-monitor-agent.ini file  We build up the items, attributes and instances as associative arrays within Lua
  12. 12. Basic Lua Collections if not items then items = { } end if not items.oldag then items[quot;oldagquot;] = { } end items[quot;oldagquot;][quot;childrenquot;] = { instances = { { name = quot;ethanquot; } }, attributes = { [quot;couch-divesquot;] = { returns = quot;intquot; } }, collector = function() return { [quot;couch-divesquot;] = math.random(100) + 120 } end }
  13. 13. Matching Lua Arrays to Types  The namespace is the top level of the items array ● “oldag”  The classname / type name comes next ● “children”  We then define the instances of the namespace::classname by name ● “ethan”  And the attributes and their return types ● “couchdives” returning an int  Finally we define a collector function that pushes the values we want to collect by attribute
  14. 14. Data Collections in the Dashboard  Data collection metadata is stored in the “inventory_*” tables  inventory_namespaces (namespace info)  inventory_types (maps to “classname” information)  inventory_attributes (attributes for namespace::type)  inventory_instances (the instance names of each attribute)  inventory_instance_attributes (pivot like table for attributes to instances mappings)
  15. 15. Data Collections in the Dashboard  A useful SQL statement to get DC information: SELECT DISTINCT namespace, type_name, attribute_name, instance_name FROM inventory_namespaces iin, inventory_types iit, inventory_instance_attributes iia, inventory_instances ii, inventory_attributes ia WHERE iia.instance_id = ii.instance_id AND ii.type_id = iit.type_id AND iin.namespace_id = iit.namespace_id AND iia.attribute_id = ia.attribute_id; *************************** 1. row *************************** namespace: os type_name: fs attribute_name: fs_avail instance_name: mac:{0025003d17370000}./
  16. 16. Data Collections in the Dashboard  Actual data is stored within the “dc_ng_*” tables  The collected data is stored in it's raw form, whether it is a counter based variable or not  Currently data is only purged, the infrastructure is ready for aggregation and aging for historical data in 2.x (we want to test more!)  dc_ng_long_now (integer based data)  dc_ng_double_now (doubles)  dc_ng_string_now (strings)  dc_ng_(long|double)_age(0|1|2) (the rollup tables)  String data obviously can not be aggregated
  17. 17. Data Collections in the Dashboard  A useful SQL statement to get the collected data: SELECT namespace, type_name, attribute_name, instance_name, value, from_unixtime(end_time/1000) FROM inventory_namespaces iin, inventory_types iit, dc_ng_long_now dcn, inventory_instance_attributes iia, inventory_instances ii, inventory_attributes ia WHERE dcn.instance_attribute_id = iia.instance_attribute_id AND iia.instance_id = ii.instance_id AND ii.type_id = iit.type_id AND iin.namespace_id = iit.namespace_id AND iia.attribute_id = ia.attribute_id AND namespace = '...' AND type_name = '...' AND attribute_name = '...';
  18. 18. Creating Custom Graphs
  19. 19. Creating Custom Graphs  Graphs also use XML for their structure  Graphs can deal with both counter and non-counter variables  2.0 now allows you to import your own graphs:
  20. 20. Graph XML Template <com_mysql_merlin_server_graph_Design> <version>...</version> /* Internal graph version numbering */ <uuid>...</uuid> /* Internal graph unique identifier */ <tag>...</tag> /* Tags for the graph (will be exposed soon) */ <name>...</name> /* Visible graph name */ <rangeLabel>...</rangeLabel> /* Visible Y axis range label */ <series> <label>...</label> /* Visible Series name / label */ <expression>...</expression> /* The expression used for the series */ </series> <variables> <name>...</name> /* Arbitrary name for variable in expression */ <dcItem> <nameSpace>...</nameSpace> <className>...</className> <attribName>...</attribName> </dcItem> <instance>...</instance> /* The instance of the dc item to be used */ </variables> </com_mysql_merlin_server_graph_Design>
  21. 21. Building a Graph - Metadata  So let's build one up! Who wants to monitor disks?  Start with the basic info <com_mysql_merlin_server_graph_Design> <version>1.0</version> <uuid>a57c2bba-ea9b-102b-b396-94aca32bee28</uuid> <name>filesystem usage - /</name> <rangeLabel>MB</rangeLabel> <frequency>00:05:00</frequency>  Note <frequency>, this is not required, and defaults to 1 minute  We use 5 minutes here to share with current collection
  22. 22. Building a Graph - Variables  Next, skip to the end, and define the variables <variables> <name>used_fs</name> <dcItem> <nameSpace>os</nameSpace> <className>fs</className> <attribName>fs_used</attribName> </dcItem> <instance>/</instance> </variables> <variables> <name>total_fs</name> <dcItem> <nameSpace>os</nameSpace> <className>fs</className> <attribName>fs_total</attribName> </dcItem> <instance>/</instance> </variables> </com_mysql_merlin_server_graph_Design>
  23. 23. Building a Graph - Series  Now finally define each series <series> <label>used</label> <expression>used_fs/1024/1024</expression> </series> <series> <label>total size</label> <expression>total_fs/1024/1024</expression> </series>
  24. 24. View the Graph!
  25. 25. Graph PERFORMANCE_SCHEMA  Viewing Global Mutex Information
  26. 26. Graph PERFORMANCE_SCHEMA  Tracking Temporary File IO!
  27. 27. Graphs – Current Caveats  Currently only line graphs are supported  Graphs are shared across all servers – so they will show up even if the instances/collections are not available on a specific server  Graphs sort alphabetically – if you want custom graphs to sort together, keep that in mind  There is no “Delete” for graphs
  28. 28. Creating and Modifying Advisors
  29. 29. Modifying Current Rules  All of the current rules are conservative – they should certainly be modified to your environment  You can edit current rues on the “Manage Rules” page within the Advisors tab  Editing only allows you to modify the THRESHOLD and frequency values for the rule  Copying allows you to alter the expression, and advice or problem descriptions etc., as well as add new variables
  30. 30. Modifying Current Rules
  31. 31. Creating New Rules  Also done on the Manage Rules page  Any data collection can be used in a rule, with no limit on the number of data collections that can be used  We still only support the default 3 THRESHOLD values – Info / Warning / Critical  Wrapping the variable names in %percents% allows us to do variable value substitution within the Advice and Recommended Action sections  We can also format numerical output nicely with '{'formatNumber:%variable_name%'}'  You can make parts bold with two underscores on either side of a section of the advice etc. too
  32. 32. Creating New Rules
  33. 33. Creating New Rules
  34. 34. Creating New Rules  Once saved, you will be able to schedule the rule within the Advisors->Add to Schedule page
  35. 35. Questions?  Ask me now!  Join us at the “Stump the DBA” BoF tonight  Ask a MySQL Support Engineer http://support.mysql.com