MySQL Monitoring Shoot Out

19,326 views

Published on

My Talk about Monitoring MySQL as a component in your environment in the
MySQL Devroom at Fosdem 2009

Published in: Technology
1 Comment
10 Likes
Statistics
Notes
No Downloads
Views
Total views
19,326
On SlideShare
0
From Embeds
0
Number of Embeds
2,270
Actions
Shares
0
Downloads
311
Comments
1
Likes
10
Embeds 0
No embeds

No notes for slide
  • MySQL Monitoring Shoot Out

    1. 1. Monitoring MySQL the open source way
    2. 2. Kris Buytaert <ul><li>Senior Linux and Open Source Consultant @inuits.be </li></ul>(sorry no P) <ul><li>„ Infrastructure Architect“
    3. 3. Linux since 0.98
    4. 4. OpenMosix, openQRM, ...
    5. 5. Early Adopter (Xen, MySQL Cluster)
    6. 6. Automating Large Scale Deployment , High Availability
    7. 7. Surviving the 10 th floor test
    8. 8. http://www.krisbuytaert.be/blog/
    9. 9. http://www.virtualization.com/ </li></ul>
    10. 10. Why this talk <ul><li>Monitoring ++
    11. 11. 2008 OLS Talk/Study about Monitoring in General
    12. 12. Beyond Geert's Nagiostalk :)
    13. 13. 2009 Refocus Study on MySQL
    14. 14. DBA vs Devministrator </li></ul>
    15. 15. In This Talk <ul><li>Monitoring ?
    16. 16. Monitoring MySQL </li><ul><li>as a part of your Infrastructure
    17. 17. with Open Source tools </li></ul></ul>In This Talk Not in This Talk <ul><li>In depth Monitoring MySQL Cluster
    18. 18. Monitoring MySQL With commercial tools
    19. 19. A zillion standalone MySQL tools compared
    20. 20. The solution to all your problems </li></ul>
    21. 21. Do you know what your children do at 5 am in the morning ? <ul><li>Are they asleep
    22. 22. Or Crashing at a party ?
    23. 23. Did something happen to them ?
    24. 24. How long have they been gone already ?
    25. 25. Why are there cops at your front door ? </li></ul>
    26. 26. Do you know what your servers are doing at 5 am in the morning ? <ul><li>You can't afford to be down
    27. 27. You can't afford to be slow
    28. 28. Systems grow and scale beyond manual/human capacity
    29. 29. Plan for growth
    30. 30. Good admins know how their systems behave
    31. 31. And what's abnormal systems behaviour </li></ul>
    32. 32. Monitoring <ul><li>Check status </li><ul><li>Define Limits
    33. 33. Running ? </li></ul><li>How to check ? </li><ul><li>Script
    34. 34. Status File
    35. 35. Agent
    36. 36. SNMP </li></ul></ul>
    37. 37. Alerts / Notifications <ul><li>Send a Warning Signal </li><ul><li>Email, SMS , xmpp , other </li></ul><li>Choose based on situation </li><ul><li>Based on time
    38. 38. Based on service
    39. 39. Based on state of system </li></ul><li>Escalation
    40. 40. SLA </li></ul>
    41. 41. Reporting <ul><li>Up / down
    42. 42. Since
    43. 43. Graphical Overview
    44. 44. Summary
    45. 45. Lies, damn lies and statistics </li></ul>
    46. 46. Trending <ul><li>Chart the data
    47. 47. A Visionary approach
    48. 48. Find Anomalies
    49. 49. Plan for Growth </li></ul>
    50. 50. Agent(less) <ul><li>Agent Based </li><ul><li>Impact on Measurement
    51. 51. More detailed information
    52. 52. Often Big performance penalty </li></ul><li>Agent Less </li><ul><li>Non intrusive
    53. 53. Less detail </li></ul><li>SNMP </li></ul>
    54. 54. Agent(less) In MySQL <ul><li>Through Proxy
    55. 55. By Sniffing Queries
    56. 56. On the Host
    57. 57. On a “modified” mysqld </li></ul>
    58. 58. What do you want from a tool ? <ul><li>Easy to configure
    59. 59. Autodetection
    60. 60. Supporting Gui
    61. 61. Automatable
    62. 62. Consistent
    63. 63. SNMP Integration
    64. 64. Trending Included ? </li></ul><ul><li>Agentless
    65. 65. Templates
    66. 66. Non Intrusive
    67. 67. Plenty of notification
    68. 68. Active community
    69. 69. Hackable </li></ul>
    70. 70. What do you want from a tool ? Regarding MySQL <ul><li>QPM/QPS </li><ul><li>Insert
    71. 71. Select
    72. 72. Update
    73. 73. Delete </li></ul><li>Replication State
    74. 74. Replication Lag </li></ul><ul><li>Queries not using IDX
    75. 75. Slow Queries
    76. 76. Cache hit Ratio
    77. 77. Data Length
    78. 78. Idx Length
    79. 79. Nr Rows
    80. 80. Nr Threads </li></ul>
    81. 81. Isolated vs Integrated tool <ul><li>Isolated : </li><ul><li>Quick Debugging
    82. 82. Development platforms </li></ul><li>Integrated : </li><ul><li>The Bigger Picture
    83. 83. Trends
    84. 84. Find the Real Cause </li></ul></ul>
    85. 85. A Zillion MySQL Focussed Tools <ul><li>MySQL Activity Report
    86. 86. MyTop
    87. 87. Innotop
    88. 88. Monolith
    89. 89. MonYOG
    90. 90. mtop
    91. 91. mysqlreport
    92. 92. Mysqlsla </li></ul>
    93. 93. One tool to Monitor Cluster <ul><li>All dump 1000 ?
    94. 94. Ugly Line wraps ? </li></ul><ul><li>CMON </li></ul><ul><li>Afaik none of the current general tools has native support for Monitoring Cluster </li></ul>
    95. 95. Integrated Monitoring : The Initial Contenders <ul><li>Hyperic HQ
    96. 96. Zabbix
    97. 97. Zenoss
    98. 98. OpenNMS
    99. 99. Nagios
    100. 100. GroundWorks
    101. 101. Hobbit
    102. 102. ... </li></ul>
    103. 103. Initial Experience <ul><li>First Phase
    104. 104. Setup Different Tools/Platforms
    105. 105. Initial Feeling
    106. 106. Installation Experience </li></ul>
    107. 107. Who made the Cut ? <ul><li>Nagios
    108. 108. Hyperic HQ Open Source
    109. 109. Zabbix
    110. 110. Zenoss Core </li></ul>
    111. 111. Nagios <ul><li>The “Standard”
    112. 112. A zillion tools based on it
    113. 113. Awkward config for the newbie
    114. 114. Very configurable
    115. 115. Very Pluggable
    116. 116. Great ecosystem
    117. 117. Often integrated with Cacti </li></ul>
    118. 118. Nagios : MySQL <ul><li>Nothing by default
    119. 119. NagiosExchange lists 69 plugins </li><ul><li>Replication checks
    120. 120. Nagios Stats
    121. 121. MySQL Perf ( http://www.consol.com/opensource/nagios/check-mysql-perf/ ) </li></ul><li>Ask Geert for more :) </li></ul>
    122. 122. Cacti : MySQL <ul><li>Baron Schwartz: http://code.google.com/p/mysql-cacti-templates/ </li></ul>
    123. 123. Nagios : Conclusion <ul><li>Con: </li><ul><li>“ steep” learning curve
    124. 124. No Default mysql support
    125. 125. Different unmaintained contributions to choose from </li></ul><li>Pro: </li><ul><li>The Standard
    126. 126. Giant Community (nagiosexchange, ...) to choose from :) </li></ul></ul>
    127. 127. Zenoss <ul><li>Integrated package featuring </li><ul><li>Availability
    128. 128. Performance
    129. 129. Events handling
    130. 130. Reporting </li></ul><li>Zope Based
    131. 131. SNMP for Autodetection
    132. 132. Based on standard protocols
    133. 133. Open Core </li></ul>
    134. 134. Zenoss <ul><li>3 part Architecture </li><ul><li>Web Console / Portal : visualizes data
    135. 135. Process Layer : daemons collect data </li><ul><ul><li>ZenPing, ZenProcess, ZenSyslog, ZenEventlog ... </li></ul></ul><li>Data Layer : stores data </li></ul><li>Data is stored in 3 places </li><ul><li>CMDB (Configuration Management DB) : Zope
    136. 136. Historical data : RRD
    137. 137. Events : MySQL </li></ul></ul>
    138. 139. Zenoss Monitoring methods/tools <ul><li>SNMP
    139. 140. Nagios plugins
    140. 141. Custom commands
    141. 142. ZenPacks: User commands, Perf templates, Graphs ... </li></ul>
    142. 143. Zenoss : MySQL <ul><li>“Nothing” by default
    143. 144. Nagios Plugins can be used </li></ul>
    144. 145. Zenoss MySQLMonitor ZenPack <ul><li>Command Statistics (SELECT, INSERT, UPDATE, DELETE)
    145. 146. Select Statistics (Scan, Range Check, Range Join, Full Join)
    146. 147. Handler Statistics (Keyed & Unkeyed Reads, Writes, Updates, Deletes)
    147. 148. Network Traffic (Received & Sent) </li></ul>
    148. 149. Zenoss MySQLMonitor ZenPack
    149. 150. Zenoss MySQLMonitor ZenPack
    150. 151. Zenoss Conclusion <ul><li>Con: </li><ul><li>Help I`m lost
    151. 152. Commercial features missing </li></ul><li>Pro: </li><ul><li>Scalabilty: multiple collectors
    152. 153. MySQL Zenpack </li></ul></ul>
    153. 154. HypericHQ <ul><li>Heavy Weight
    154. 155. Agent Based (Heavy)
    155. 156. Java
    156. 157. Autodiscovery (of services)
    157. 158. SIGAR (System Information Gatherer and Reporter) </li></ul>
    158. 159. HypericHQ <ul><li>Quick setup
    159. 160. Inside the applications </li><ul><ul><li>Real focus towards application monitoring
    160. 161. Focus on State
    161. 162. Focus on functionality </li></ul></ul><li>Great to do debugging </li></ul>
    162. 163. Hyperic Overview <ul><li>Server/Agent method
    163. 164. Focusses strongly on application/db/ performance
    164. 165. Grouping of servers/services
    165. 166. Very nice Dashboard! </li></ul>
    166. 167. Hyperic Inside the Apps <ul><li>Default installed plugins : </li><ul><li>MySQL </li><ul><li>Table level
    167. 168. Row count, qps, table size </li></ul><li>PostgresQL </li><ul><li>same </li></ul><li>Jboss </li><ul><li>Inside the JMX
    168. 169. Deployed WARS </li></ul></ul></ul>
    169. 170. Hyperic Inside the Apps
    170. 171. Hyperic Inside the Apps
    171. 172. Hyperic Inside the Apps
    172. 173. Hyperic Inside the Apps
    173. 174. Hyperic Conclusion <ul><li>Con: </li><ul><li>Help , I'm lost !
    174. 175. Lots of NTH features in Commercial Version
    175. 176. Not for your typical LAMP shop </li></ul><li>Pro: </li><ul><li>Very nice/simple/straight forward
    176. 177. “ Low” on java-memory, very responsive webfrontend, not 'sluggish' at all
    177. 178. Goes DEEP Inside the Application
    178. 179. MySQL / PgSQL out of the Box </li></ul></ul>
    179. 180. Zabbix <ul><li>“LightWeight”
    180. 181. Multi Tier </li><ul><li>Agents
    181. 182. Database + Daemon
    182. 183. Web Interface </li></ul><li>Template based </li></ul>
    183. 184. Zabbix <ul><li>Find the right package for your distro = smooth installation
    184. 185. “Auto detects” agents
    185. 186. Create your own screens </li></ul>
    186. 187. Zabbix Monitoring methods/tools <ul><li>Simple checks
    187. 188. Agent (availability of params depending OS)
    188. 189. SNMP
    189. 190. Other </li><ul><li>External checks
    190. 191. Internal checks
    191. 192. Aggregated checks </li></ul></ul>
    192. 193. Zabbix Features <ul><li>Alerting </li><ul><li>Harder to configure notifications
    193. 194. No sign of escalation (planned) </li></ul><li>Reporting </li><ul><li>Customizable layouts </li></ul><li>Trending </li><ul><li>Slideshow mode
    194. 195. Correlation of different graphs </li></ul></ul>
    195. 196. Zabbix MySQL <ul><li>Agent has checks disabled
    196. 197. Needs uncommenting </li></ul>### Set of parameter for monitoring MySQL server (v3.23.42 and later) ### Change -u<username> and add -p<password> if required UserParameter=mysql.ping,mysqladmin -uroot ping|grep alive|wc -l UserParameter=mysql.uptime,mysqladmin -uroot status|cut -f2 -d&quot;:&quot;|cut -f1 -d&quot;T&quot; UserParameter=mysql.threads,mysqladmin -uroot status|cut -f3 -d&quot;:&quot;|cut -f1 -d&quot;Q&quot; UserParameter=mysql.questions,mysqladmin -uroot status|cut -f4 -d&quot;:&quot;|cut -f1 -d&quot; S&quot; UserParameter=mysql.slowqueries,mysqladmin -uroot status|cut -f5 -d&quot;:&quot;|cut -f1 -d&quot;O&quot; UserParameter=mysql.qps,mysqladmin -uroot status|cut -f9 -d&quot;:&quot; UserParameter=mysql.version,mysql -V
    197. 198. Beyond the Default Zabbix <ul><li>Zabbix Wiki : Heavy /Extensive MySQL Monitoring </li><ul><li>Consolidates zillion tools in 1
    198. 199. Sends those checks to Zabbix
    199. 200. mysql.php script </li><ul><li>Any value
    200. 201. Replication Slave/Master </li></ul></ul><li>Others exist, but aren't published </li></ul>
    201. 202. Zabbix MySQL
    202. 203. Zabbix MySQL
    203. 204. Zabbix MySQL
    204. 205. Zabbix Conclusion <ul><li>Con: </li><ul><li>Pretty cumbersome to configure
    205. 206. Important features missing ( but planned in next version ): escalation, better reporting ,.... </li></ul><li>Pro: </li><ul><li>Lightweight both server and agents
    206. 207. Fully Integrated
    207. 208. Screens : Correlation of graphs </li></ul></ul>
    208. 209. Conclusion <ul><li>Choose yourself </li><ul><li>Java Shop ?
    209. 210. LAMP Shop ? </li></ul><li>We are using mostly </li><ul><li>Zabbix
    210. 211. Hyperic </li></ul></ul>
    211. 212. ` Kris Buytaert < [email_address] > Further Reading http://www.krisbuytaert.be/blog/ http://www.inuits.be/ http://www.virtualization.com/ http://www.oreillygmt.com/ ? !

    ×