Monitoring your technology stack with New Relic

  • 4,483 views
Uploaded on

There is no excuse to not have monitoring of your LAMP stack, NoSQL database like MongoDB/Redis/Cassandra/Memcache, Cloud services and much more when you can use the popular New Relic tool for free. …

There is no excuse to not have monitoring of your LAMP stack, NoSQL database like MongoDB/Redis/Cassandra/Memcache, Cloud services and much more when you can use the popular New Relic tool for free. As the MySQL plugin author I can offer the following link will give you access to free monitoring http://j.mp/newrelic-mysql There can never be an excuse to not know how your application is performing, from 1 server to 100+ servers.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,483
On Slideshare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
39
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. MySQL/NoSQL/Cloud Monitoring with New Relic Ronald Bradford http://ronaldbradford.com Buenos Aires, Argentina 2013-10 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 2. ABOUT AUTHOR Ronald Bradford 24 years of RDBMS experience,14 years with MySQL MySQL Inc (2006-2008) Oracle Corporation (1996-1999) Published Author (4 books) Oracle ACE Director MySQL community member of the year (2009 & 2013) All time top MySQL blogger Provide independent MySQL/ LAMP/Cloud consulting EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 3. Why Monitor? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 4. Text Text
  • 5. Zabbix OpenTSDB Zenoss Hyperic Cacti Nagios Munin MySQL Enterprise Monitor Monitoring Options Ganglia CollectD StatsD http://en.wikipedia.org/wiki/Comparison_of_network_monitoring_systems New Relic Boundary DataDog EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 6. New Relic EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 7. New Relic Application Monitoring Historically Server Monitoring Software as a Service (SaaS) Agent-like EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 8. The cloud - SaaS Minimal installation Minimal administration Cloud based management Web Mobile Integrations to other products EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 9. New Relic Application Monitoring Introducing Server Monitoring NOW - New Relic Platform <insert product here> EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 10. NEw Relic Subscription per server Standard ($24 p.s. p.m.) Pro ($149 p.s. p.m.) YOU CAN GET UNLIMITED STANDARD FREE BECAUSE BY THE MySQL PLUGIN AUTHOR http://j.mp/newrelic-mysql EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 11. 1 2 3 No Credit Card Details required 4 http://j.mp/newrelic-mysql
  • 12. NAVIGATION EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 13. LICENSE EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 14. ALERTs EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 15. Server Ubuntu/Debian/RedHat/CentOS/Linux/Windows EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 16. SERVER INSTALL LICENSE_KEY="<enter key here>" sudo rpm -Uvh http://download.newrelic.com/pub/newrelic/ el5/x86_64/newrelic-repo-5-3.noarch.rpm sudo yum install -y newrelic-sysmond sudo nrsysmond-config --set license_key=${LICENSE_KEY} sudo /etc/init.d/newrelic-sysmond start Helps to use good hostnames EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 17. Application PHP/Ruby/.NET/Java/Python/Node.js EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 18. PHP, Ruby, .NET, Java, Python, Node.js
  • 19. PHP INSTALL LICENSE_KEY="<enter key here>" sudo rpm -Uvh http://download.newrelic.com/pub/newrelic/ el5/x86_64/newrelic-repo-5-3.noarch.rpm sudo yum install -y newrelic-php5 sudo newrelic-install install # prompted for LICENSE_KEY sudo service httpd restart EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 20. Platform MySQL/Memcached/Apache/Redis/MongoDB/AWS/... EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 21. JRE (on some server) https://github.com/newrelic-platform/newrelic_mysql_java_plugin/
  • 22. MYSQL INSTALL Version 1 - Needs streamlining sudo su LICENSE_KEY="<enter key here>" wget -O /tmp/newrelic.tar.gz https://raw.github.com/newrelic-platform/ newrelic_mysql_java_plugin/master/dist/newrelic_mysql_plugin-1.0.6.tar.gz cd /usr/local tar xvfz /tmp/newrelic.tar.gz mv newrelic_mysql_plugin-1.* newrelic_mysql_plugin cd /usr/local/newrelic_mysql_plugin echo "licenseKey=${LICENSE_KEY}" > config/newrelic.properties echo "java.util.logging.ConsoleHandler.level = INFO com.newrelic.metrics.publish.handlers = java.util.logging.ConsoleHandler" > config/logging.properties echo '[ { "name" : "'`hostname -s`'", "host" : "localhost", "metrics" : "status,newrelic,innodb,innodb_status,innodb_mutex,slave" } ]' > config/mysql.instance.json #mysql < mysql_user.sql nohup java -jar newrelic_mysql_plugin-*.jar & https://github.com/newrelic-platform/newrelic_mysql_java_plugin/ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 23. General Views EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 24. navigation EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 25. Summary View
  • 26. More
  • 27. More
  • 28. MYSQL Sources SHOW GLOBAL STATUS SHOW MASTER STATUS SHOW SLAVE STATUS SHOW ENGINE INNODB STATUS SHOW ENGINE INNODB MUTEX <Any SQL statement> EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 29. Coding Features SHOW ENGINE INNODB MUTEX Dynamic metric names Rollup, supporting different formats in 5.5, 5.6, Percona etc EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 30. Coding Features SHOW ENGINE INNODB STATUS EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 31. Version 1 Your input and feedback See limitations EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 32. Default is “PHP Application” /etc/php.d/newrelic.inc newrelic.appname = "PHP Application"
  • 33. Other Products EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 34. MEMCACHE INSTALL LICENSE_KEY="<enter key here>" sudo pip install newrelic-plugin-agent sudo cp /opt/newrelic_plugin_agent/newrelic_plugin_agent.cfg /etc/newrelic/newrelic_plugin_agent.cfg sudo sudo sudo sudo sudo useradd -m -s /bin/bash newrelic mkdir -p /var/log/newrelic /var/run/newrelic chown newrelic:newrelic /var/*/newrelic vi /etc/newrelic/newrelic_plugin_agent.cfg newrelic_plugin_agent -c /etc/newrelic/newrelic_plugin_agent.cfg -f https://rpm.newrelic.com/accounts/389698/plugins/directory/53 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 35. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 36. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 37. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 38. meetme plugin PHP Cache HTTP Server CouchDB Edgecast CDN API Memcached MongoDB Nginx pgBouncer PHP FPM PostgreSQL RabbitMQ Redis Riak uWSGI EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 39. Other Plugins Logwatcher F5 Varnish AWS EC2/RDS/ELB Sendgrid Rabbit MQ Cassandra Sphinx Haproxy <write your own> Hadoop EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 40. User Interface EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 41. History EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 42. Click and drag for subset of time
  • 43. Configuration EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 44. MySQL Plugin Plugin has more features than UI can support at present Configurable namespaces Row/Set SQL statements Configurable metrics EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 45. MySQL config $ cat /path/to/config/mysql.instance.json [ { "name" : "db1", "host" : "localhost", "metrics" : "status,newrelic,innodb_status, innodb_mutex,master" }, One agent can collect data from { many MySQL Instances "name" : "db2", "host" : "db2.hostname.com", "user" : "monitor", "password" : "password", "metrics" : "status,newrelic,slave" } ] EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 46. namespaces $ cat /path/to/config/metric.category.json [ { "category" : "status", "SQL" : "SHOW GLOBAL STATUS", "result" : "set", "value_metrics":"Max_used_connections,....", "counter_metrics" :"Bytes_received,Bytes_sent,..." }, { "category" : "slave", "SQL" : "SHOW SLAVE STATUS", "result" : "row", "value_metrics" : "Read_Master_Log_Pos, Slave_IO_Running, Slave_SQL_Running, EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 47. namespaces $ cat /path/to/config/metric.category.json [ { "category" : "innodb_metrics", "SQL" : "SELECT name, count FROM information_schema.innodb_metrics", "result" : "set", "comment" : "MySQL 5.6 or later", "value_metrics" : "...", "counter_metrics" : "..." }, ... Current UI does not support with default MySQL plugin, but EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 48. Modifying plugin https://github.com/newrelic-platform/ newrelic_mysql_java_plugin com.newrelic.plugins.mysql.instance.G UID="new value" Request to overload via parameter (to help with current UI limitations) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 49. Limitations (Also exist with most monitoring products) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 50. sampling 60 seconds (fixed) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 51. CopperEgg Example (15 second samples)
  • 52. averaging Deceiving No maximum support EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 53. alerting limits Positive range only Summary metrics only No low limits EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 54. Custom Dashboards Enables you to group graphs together Does not work with plugins YET! EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 55. Presentation Likes lots of moving/changing data Uninteresting values not well supported replication lag replication running YES/NO Important summary alert[able] metric EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 56. Servers Likes to work on per server basis Default reporting namespace Current com.newrelic.plugins.mysql.instance [agent] /mysql/status/com_select [agent] /mysql/status/com* One Goal com.newrelic.plugins.mysql.topology /mysql/status/com_select/<instance> EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 57. What is monitoring the monitoring?
  • 58. Usage Patterns EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 59. tmp disk tables High disk I/O - Investigate SQL EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 60. query cache Query Cache NOT working EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 61. Query CAche Enabling Query Cache benefit EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 62. Yes, this time the database (servers) are on fire!
  • 63. If only I could trigger an alert on this?
  • 64. A Developers View EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 65. SQL for developers Drill down to running queries Look at ALL thread SQL statements Aggregations EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 66. Standard Feature
  • 67. Standard Feature
  • 68. Pro Feature
  • 69. Standard Feature
  • 70. Pro Feature
  • 71. Pro Feature
  • 72. Pro Feature
  • 73. A better waterfall EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 74. Standard Feature Still not a DB problem, but, I need to identify and fix
  • 75. Conclusion EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 76. Conclusion There is no excuse to have NO monitoring Visualization can generally help show issues Historical reporting can help review issues NEITHER WILL SOLVE THE PROBLEM Hire Ronald Bradford EffectiveMySQL.com - Performance, Scalability & Business Continuity
  • 77. Ronald Bradford ronald@effectivemysql.com