Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MySQL Monitoring Shoot Out

19,575 views

Published on

My Talk about Monitoring MySQL as a component in your environment in the
MySQL Devroom at Fosdem 2009

Published in: Technology

MySQL Monitoring Shoot Out

  1. 1. Monitoring MySQL the open source way
  2. 2. Kris Buytaert <ul><li>Senior Linux and Open Source Consultant @inuits.be </li></ul>(sorry no P) <ul><li>„ Infrastructure Architect“
  3. 3. Linux since 0.98
  4. 4. OpenMosix, openQRM, ...
  5. 5. Early Adopter (Xen, MySQL Cluster)
  6. 6. Automating Large Scale Deployment , High Availability
  7. 7. Surviving the 10 th floor test
  8. 8. http://www.krisbuytaert.be/blog/
  9. 9. http://www.virtualization.com/ </li></ul>
  10. 10. Why this talk <ul><li>Monitoring ++
  11. 11. 2008 OLS Talk/Study about Monitoring in General
  12. 12. Beyond Geert's Nagiostalk :)
  13. 13. 2009 Refocus Study on MySQL
  14. 14. DBA vs Devministrator </li></ul>
  15. 15. In This Talk <ul><li>Monitoring ?
  16. 16. Monitoring MySQL </li><ul><li>as a part of your Infrastructure
  17. 17. with Open Source tools </li></ul></ul>In This Talk Not in This Talk <ul><li>In depth Monitoring MySQL Cluster
  18. 18. Monitoring MySQL With commercial tools
  19. 19. A zillion standalone MySQL tools compared
  20. 20. The solution to all your problems </li></ul>
  21. 21. Do you know what your children do at 5 am in the morning ? <ul><li>Are they asleep
  22. 22. Or Crashing at a party ?
  23. 23. Did something happen to them ?
  24. 24. How long have they been gone already ?
  25. 25. Why are there cops at your front door ? </li></ul>
  26. 26. Do you know what your servers are doing at 5 am in the morning ? <ul><li>You can't afford to be down
  27. 27. You can't afford to be slow
  28. 28. Systems grow and scale beyond manual/human capacity
  29. 29. Plan for growth
  30. 30. Good admins know how their systems behave
  31. 31. And what's abnormal systems behaviour </li></ul>
  32. 32. Monitoring <ul><li>Check status </li><ul><li>Define Limits
  33. 33. Running ? </li></ul><li>How to check ? </li><ul><li>Script
  34. 34. Status File
  35. 35. Agent
  36. 36. SNMP </li></ul></ul>
  37. 37. Alerts / Notifications <ul><li>Send a Warning Signal </li><ul><li>Email, SMS , xmpp , other </li></ul><li>Choose based on situation </li><ul><li>Based on time
  38. 38. Based on service
  39. 39. Based on state of system </li></ul><li>Escalation
  40. 40. SLA </li></ul>
  41. 41. Reporting <ul><li>Up / down
  42. 42. Since
  43. 43. Graphical Overview
  44. 44. Summary
  45. 45. Lies, damn lies and statistics </li></ul>
  46. 46. Trending <ul><li>Chart the data
  47. 47. A Visionary approach
  48. 48. Find Anomalies
  49. 49. Plan for Growth </li></ul>
  50. 50. Agent(less) <ul><li>Agent Based </li><ul><li>Impact on Measurement
  51. 51. More detailed information
  52. 52. Often Big performance penalty </li></ul><li>Agent Less </li><ul><li>Non intrusive
  53. 53. Less detail </li></ul><li>SNMP </li></ul>
  54. 54. Agent(less) In MySQL <ul><li>Through Proxy
  55. 55. By Sniffing Queries
  56. 56. On the Host
  57. 57. On a “modified” mysqld </li></ul>
  58. 58. What do you want from a tool ? <ul><li>Easy to configure
  59. 59. Autodetection
  60. 60. Supporting Gui
  61. 61. Automatable
  62. 62. Consistent
  63. 63. SNMP Integration
  64. 64. Trending Included ? </li></ul><ul><li>Agentless
  65. 65. Templates
  66. 66. Non Intrusive
  67. 67. Plenty of notification
  68. 68. Active community
  69. 69. Hackable </li></ul>
  70. 70. What do you want from a tool ? Regarding MySQL <ul><li>QPM/QPS </li><ul><li>Insert
  71. 71. Select
  72. 72. Update
  73. 73. Delete </li></ul><li>Replication State
  74. 74. Replication Lag </li></ul><ul><li>Queries not using IDX
  75. 75. Slow Queries
  76. 76. Cache hit Ratio
  77. 77. Data Length
  78. 78. Idx Length
  79. 79. Nr Rows
  80. 80. Nr Threads </li></ul>
  81. 81. Isolated vs Integrated tool <ul><li>Isolated : </li><ul><li>Quick Debugging
  82. 82. Development platforms </li></ul><li>Integrated : </li><ul><li>The Bigger Picture
  83. 83. Trends
  84. 84. Find the Real Cause </li></ul></ul>
  85. 85. A Zillion MySQL Focussed Tools <ul><li>MySQL Activity Report
  86. 86. MyTop
  87. 87. Innotop
  88. 88. Monolith
  89. 89. MonYOG
  90. 90. mtop
  91. 91. mysqlreport
  92. 92. Mysqlsla </li></ul>
  93. 93. One tool to Monitor Cluster <ul><li>All dump 1000 ?
  94. 94. Ugly Line wraps ? </li></ul><ul><li>CMON </li></ul><ul><li>Afaik none of the current general tools has native support for Monitoring Cluster </li></ul>
  95. 95. Integrated Monitoring : The Initial Contenders <ul><li>Hyperic HQ
  96. 96. Zabbix
  97. 97. Zenoss
  98. 98. OpenNMS
  99. 99. Nagios
  100. 100. GroundWorks
  101. 101. Hobbit
  102. 102. ... </li></ul>
  103. 103. Initial Experience <ul><li>First Phase
  104. 104. Setup Different Tools/Platforms
  105. 105. Initial Feeling
  106. 106. Installation Experience </li></ul>
  107. 107. Who made the Cut ? <ul><li>Nagios
  108. 108. Hyperic HQ Open Source
  109. 109. Zabbix
  110. 110. Zenoss Core </li></ul>
  111. 111. Nagios <ul><li>The “Standard”
  112. 112. A zillion tools based on it
  113. 113. Awkward config for the newbie
  114. 114. Very configurable
  115. 115. Very Pluggable
  116. 116. Great ecosystem
  117. 117. Often integrated with Cacti </li></ul>
  118. 118. Nagios : MySQL <ul><li>Nothing by default
  119. 119. NagiosExchange lists 69 plugins </li><ul><li>Replication checks
  120. 120. Nagios Stats
  121. 121. MySQL Perf ( http://www.consol.com/opensource/nagios/check-mysql-perf/ ) </li></ul><li>Ask Geert for more :) </li></ul>
  122. 122. Cacti : MySQL <ul><li>Baron Schwartz: http://code.google.com/p/mysql-cacti-templates/ </li></ul>
  123. 123. Nagios : Conclusion <ul><li>Con: </li><ul><li>“ steep” learning curve
  124. 124. No Default mysql support
  125. 125. Different unmaintained contributions to choose from </li></ul><li>Pro: </li><ul><li>The Standard
  126. 126. Giant Community (nagiosexchange, ...) to choose from :) </li></ul></ul>
  127. 127. Zenoss <ul><li>Integrated package featuring </li><ul><li>Availability
  128. 128. Performance
  129. 129. Events handling
  130. 130. Reporting </li></ul><li>Zope Based
  131. 131. SNMP for Autodetection
  132. 132. Based on standard protocols
  133. 133. Open Core </li></ul>
  134. 134. Zenoss <ul><li>3 part Architecture </li><ul><li>Web Console / Portal : visualizes data
  135. 135. Process Layer : daemons collect data </li><ul><ul><li>ZenPing, ZenProcess, ZenSyslog, ZenEventlog ... </li></ul></ul><li>Data Layer : stores data </li></ul><li>Data is stored in 3 places </li><ul><li>CMDB (Configuration Management DB) : Zope
  136. 136. Historical data : RRD
  137. 137. Events : MySQL </li></ul></ul>
  138. 139. Zenoss Monitoring methods/tools <ul><li>SNMP
  139. 140. Nagios plugins
  140. 141. Custom commands
  141. 142. ZenPacks: User commands, Perf templates, Graphs ... </li></ul>
  142. 143. Zenoss : MySQL <ul><li>“Nothing” by default
  143. 144. Nagios Plugins can be used </li></ul>
  144. 145. Zenoss MySQLMonitor ZenPack <ul><li>Command Statistics (SELECT, INSERT, UPDATE, DELETE)
  145. 146. Select Statistics (Scan, Range Check, Range Join, Full Join)
  146. 147. Handler Statistics (Keyed & Unkeyed Reads, Writes, Updates, Deletes)
  147. 148. Network Traffic (Received & Sent) </li></ul>
  148. 149. Zenoss MySQLMonitor ZenPack
  149. 150. Zenoss MySQLMonitor ZenPack
  150. 151. Zenoss Conclusion <ul><li>Con: </li><ul><li>Help I`m lost
  151. 152. Commercial features missing </li></ul><li>Pro: </li><ul><li>Scalabilty: multiple collectors
  152. 153. MySQL Zenpack </li></ul></ul>
  153. 154. HypericHQ <ul><li>Heavy Weight
  154. 155. Agent Based (Heavy)
  155. 156. Java
  156. 157. Autodiscovery (of services)
  157. 158. SIGAR (System Information Gatherer and Reporter) </li></ul>
  158. 159. HypericHQ <ul><li>Quick setup
  159. 160. Inside the applications </li><ul><ul><li>Real focus towards application monitoring
  160. 161. Focus on State
  161. 162. Focus on functionality </li></ul></ul><li>Great to do debugging </li></ul>
  162. 163. Hyperic Overview <ul><li>Server/Agent method
  163. 164. Focusses strongly on application/db/ performance
  164. 165. Grouping of servers/services
  165. 166. Very nice Dashboard! </li></ul>
  166. 167. Hyperic Inside the Apps <ul><li>Default installed plugins : </li><ul><li>MySQL </li><ul><li>Table level
  167. 168. Row count, qps, table size </li></ul><li>PostgresQL </li><ul><li>same </li></ul><li>Jboss </li><ul><li>Inside the JMX
  168. 169. Deployed WARS </li></ul></ul></ul>
  169. 170. Hyperic Inside the Apps
  170. 171. Hyperic Inside the Apps
  171. 172. Hyperic Inside the Apps
  172. 173. Hyperic Inside the Apps
  173. 174. Hyperic Conclusion <ul><li>Con: </li><ul><li>Help , I'm lost !
  174. 175. Lots of NTH features in Commercial Version
  175. 176. Not for your typical LAMP shop </li></ul><li>Pro: </li><ul><li>Very nice/simple/straight forward
  176. 177. “ Low” on java-memory, very responsive webfrontend, not 'sluggish' at all
  177. 178. Goes DEEP Inside the Application
  178. 179. MySQL / PgSQL out of the Box </li></ul></ul>
  179. 180. Zabbix <ul><li>“LightWeight”
  180. 181. Multi Tier </li><ul><li>Agents
  181. 182. Database + Daemon
  182. 183. Web Interface </li></ul><li>Template based </li></ul>
  183. 184. Zabbix <ul><li>Find the right package for your distro = smooth installation
  184. 185. “Auto detects” agents
  185. 186. Create your own screens </li></ul>
  186. 187. Zabbix Monitoring methods/tools <ul><li>Simple checks
  187. 188. Agent (availability of params depending OS)
  188. 189. SNMP
  189. 190. Other </li><ul><li>External checks
  190. 191. Internal checks
  191. 192. Aggregated checks </li></ul></ul>
  192. 193. Zabbix Features <ul><li>Alerting </li><ul><li>Harder to configure notifications
  193. 194. No sign of escalation (planned) </li></ul><li>Reporting </li><ul><li>Customizable layouts </li></ul><li>Trending </li><ul><li>Slideshow mode
  194. 195. Correlation of different graphs </li></ul></ul>
  195. 196. Zabbix MySQL <ul><li>Agent has checks disabled
  196. 197. Needs uncommenting </li></ul>### Set of parameter for monitoring MySQL server (v3.23.42 and later) ### Change -u<username> and add -p<password> if required UserParameter=mysql.ping,mysqladmin -uroot ping|grep alive|wc -l UserParameter=mysql.uptime,mysqladmin -uroot status|cut -f2 -d&quot;:&quot;|cut -f1 -d&quot;T&quot; UserParameter=mysql.threads,mysqladmin -uroot status|cut -f3 -d&quot;:&quot;|cut -f1 -d&quot;Q&quot; UserParameter=mysql.questions,mysqladmin -uroot status|cut -f4 -d&quot;:&quot;|cut -f1 -d&quot; S&quot; UserParameter=mysql.slowqueries,mysqladmin -uroot status|cut -f5 -d&quot;:&quot;|cut -f1 -d&quot;O&quot; UserParameter=mysql.qps,mysqladmin -uroot status|cut -f9 -d&quot;:&quot; UserParameter=mysql.version,mysql -V
  197. 198. Beyond the Default Zabbix <ul><li>Zabbix Wiki : Heavy /Extensive MySQL Monitoring </li><ul><li>Consolidates zillion tools in 1
  198. 199. Sends those checks to Zabbix
  199. 200. mysql.php script </li><ul><li>Any value
  200. 201. Replication Slave/Master </li></ul></ul><li>Others exist, but aren't published </li></ul>
  201. 202. Zabbix MySQL
  202. 203. Zabbix MySQL
  203. 204. Zabbix MySQL
  204. 205. Zabbix Conclusion <ul><li>Con: </li><ul><li>Pretty cumbersome to configure
  205. 206. Important features missing ( but planned in next version ): escalation, better reporting ,.... </li></ul><li>Pro: </li><ul><li>Lightweight both server and agents
  206. 207. Fully Integrated
  207. 208. Screens : Correlation of graphs </li></ul></ul>
  208. 209. Conclusion <ul><li>Choose yourself </li><ul><li>Java Shop ?
  209. 210. LAMP Shop ? </li></ul><li>We are using mostly </li><ul><li>Zabbix
  210. 211. Hyperic </li></ul></ul>
  211. 212. ` Kris Buytaert < [email_address] > Further Reading http://www.krisbuytaert.be/blog/ http://www.inuits.be/ http://www.virtualization.com/ http://www.oreillygmt.com/ ? !

×