MySQL/NoSQL/Cloud
Monitoring with New
Relic
Ronald Bradford
http://ronaldbradford.com
Buenos Aires, Argentina
2013-10
Effe...
ABOUT AUTHOR
Ronald Bradford
24 years of RDBMS experience,14 years with MySQL
MySQL Inc (2006-2008)
Oracle Corporation (19...
Why Monitor?

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Text
Text
Zabbix
OpenTSDB

Zenoss
Hyperic

Cacti
Nagios
Munin

MySQL Enterprise Monitor

Monitoring
Options

Ganglia
CollectD
StatsD...
New Relic

EffectiveMySQL.com - Performance, Scalability & Business Continuity
New Relic
Application Monitoring

Historically

Server Monitoring
Software as a Service (SaaS)
Agent-like

EffectiveMySQL....
The cloud - SaaS
Minimal installation
Minimal administration
Cloud based management
Web
Mobile
Integrations to other produ...
New Relic
Application Monitoring

Introducing

Server Monitoring
NOW - New Relic Platform
<insert product here>

Effective...
NEw Relic
Subscription per server
Standard ($24 p.s. p.m.)
Pro

($149 p.s. p.m.)

YOU CAN GET UNLIMITED
STANDARD FREE BECA...
1
2
3

No Credit Card Details required

4

http://j.mp/newrelic-mysql
NAVIGATION

EffectiveMySQL.com - Performance, Scalability & Business Continuity
LICENSE

EffectiveMySQL.com - Performance, Scalability & Business Continuity
ALERTs

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Server
Ubuntu/Debian/RedHat/CentOS/Linux/Windows

EffectiveMySQL.com - Performance, Scalability & Business Continuity
SERVER INSTALL
LICENSE_KEY="<enter key here>"
sudo rpm -Uvh http://download.newrelic.com/pub/newrelic/
el5/x86_64/newrelic...
Application
PHP/Ruby/.NET/Java/Python/Node.js

EffectiveMySQL.com - Performance, Scalability & Business Continuity
PHP, Ruby, .NET, Java, Python, Node.js
PHP INSTALL
LICENSE_KEY="<enter key here>"
sudo rpm -Uvh http://download.newrelic.com/pub/newrelic/
el5/x86_64/newrelic-re...
Platform
MySQL/Memcached/Apache/Redis/MongoDB/AWS/...

EffectiveMySQL.com - Performance, Scalability & Business Continuity
JRE (on some server)

https://github.com/newrelic-platform/newrelic_mysql_java_plugin/
MYSQL INSTALL
Version 1 - Needs streamlining

sudo su LICENSE_KEY="<enter key here>"
wget -O /tmp/newrelic.tar.gz https://...
General Views

EffectiveMySQL.com - Performance, Scalability & Business Continuity
navigation

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Summary View
More
More
MYSQL Sources
SHOW GLOBAL STATUS
SHOW MASTER STATUS
SHOW SLAVE STATUS
SHOW ENGINE INNODB STATUS
SHOW ENGINE INNODB MUTEX
<...
Coding Features
SHOW ENGINE INNODB MUTEX
Dynamic metric names

Rollup, supporting different formats
in 5.5, 5.6, Percona e...
Coding Features
SHOW ENGINE INNODB STATUS

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Version 1
Your input and feedback
See limitations

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Default is “PHP Application”
/etc/php.d/newrelic.inc
newrelic.appname = "PHP Application"
Other Products

EffectiveMySQL.com - Performance, Scalability & Business Continuity
MEMCACHE INSTALL
LICENSE_KEY="<enter key here>"
sudo pip install newrelic-plugin-agent
sudo cp /opt/newrelic_plugin_agent/...
EffectiveMySQL.com - Performance, Scalability & Business Continuity
EffectiveMySQL.com - Performance, Scalability & Business Continuity
EffectiveMySQL.com - Performance, Scalability & Business Continuity
meetme plugin
PHP Cache
HTTP Server
CouchDB
Edgecast CDN API
Memcached
MongoDB
Nginx
pgBouncer
PHP FPM

PostgreSQL
RabbitM...
Other Plugins
Logwatcher

F5

Varnish

AWS EC2/RDS/ELB

Sendgrid

Rabbit MQ

Cassandra

Sphinx

Haproxy

<write your own>
...
User Interface

EffectiveMySQL.com - Performance, Scalability & Business Continuity
History

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Click and drag for subset of time
Configuration

EffectiveMySQL.com - Performance, Scalability & Business Continuity
MySQL Plugin
Plugin has more features than UI can
support at present
Configurable namespaces
Row/Set SQL statements
Configur...
MySQL config
$ cat /path/to/config/mysql.instance.json
[
{
"name" : "db1",
"host" : "localhost",
"metrics" : "status,newre...
namespaces
$ cat /path/to/config/metric.category.json
[
{ "category" : "status",
"SQL" : "SHOW GLOBAL STATUS",
"result" : ...
namespaces
$ cat /path/to/config/metric.category.json
[
{
"category" : "innodb_metrics",
"SQL" : "SELECT name, count FROM
...
Modifying plugin
https://github.com/newrelic-platform/
newrelic_mysql_java_plugin
com.newrelic.plugins.mysql.instance.G
UI...
Limitations
(Also exist with most monitoring
products)

EffectiveMySQL.com - Performance, Scalability & Business Continuit...
sampling
60 seconds (fixed)

EffectiveMySQL.com - Performance, Scalability & Business Continuity
CopperEgg Example
(15 second samples)
averaging
Deceiving

No maximum
support
EffectiveMySQL.com - Performance, Scalability & Business Continuity
alerting limits
Positive range only
Summary metrics
only
No low limits

EffectiveMySQL.com - Performance, Scalability & Bu...
Custom Dashboards
Enables you to group graphs together
Does not work with plugins YET!

EffectiveMySQL.com - Performance, ...
Presentation
Likes lots of moving/changing data
Uninteresting values not well supported
replication lag
replication runnin...
Servers
Likes to work on per server basis
Default reporting namespace
Current
com.newrelic.plugins.mysql.instance
[agent] ...
What is monitoring the monitoring?
Usage Patterns

EffectiveMySQL.com - Performance, Scalability & Business Continuity
tmp disk tables
High disk I/O - Investigate SQL

EffectiveMySQL.com - Performance, Scalability & Business Continuity
query cache
Query Cache NOT working

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Query CAche
Enabling Query Cache benefit

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Yes, this time the
database (servers)
are on fire!
If only I could trigger an
alert on this?
A Developers View

EffectiveMySQL.com - Performance, Scalability & Business Continuity
SQL for developers
Drill down to running queries
Look at ALL thread SQL statements
Aggregations

EffectiveMySQL.com - Perf...
Standard Feature
Standard Feature
Pro Feature
Standard Feature
Pro Feature
Pro Feature
Pro Feature
A better waterfall

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Standard Feature

Still not a DB problem, but,
I need to identify and fix
Conclusion

EffectiveMySQL.com - Performance, Scalability & Business Continuity
Conclusion
There is no excuse to have NO monitoring
Visualization can generally help show issues
Historical reporting can ...
Ronald Bradford
ronald@effectivemysql.com
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Monitoring your technology stack with New Relic
Upcoming SlideShare
Loading in...5
×

Monitoring your technology stack with New Relic

6,409

Published on

There is no excuse to not have monitoring of your LAMP stack, NoSQL database like MongoDB/Redis/Cassandra/Memcache, Cloud services and much more when you can use the popular New Relic tool for free. As the MySQL plugin author I can offer the following link will give you access to free monitoring http://j.mp/newrelic-mysql There can never be an excuse to not know how your application is performing, from 1 server to 100+ servers.

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,409
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
57
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Monitoring your technology stack with New Relic

  1. 1. MySQL/NoSQL/Cloud Monitoring with New Relic Ronald Bradford http://ronaldbradford.com Buenos Aires, Argentina 2013-10 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  2. 2. ABOUT AUTHOR Ronald Bradford 24 years of RDBMS experience,14 years with MySQL MySQL Inc (2006-2008) Oracle Corporation (1996-1999) Published Author (4 books) Oracle ACE Director MySQL community member of the year (2009 & 2013) All time top MySQL blogger Provide independent MySQL/ LAMP/Cloud consulting EffectiveMySQL.com - Performance, Scalability & Business Continuity
  3. 3. Why Monitor? EffectiveMySQL.com - Performance, Scalability & Business Continuity
  4. 4. Text Text
  5. 5. Zabbix OpenTSDB Zenoss Hyperic Cacti Nagios Munin MySQL Enterprise Monitor Monitoring Options Ganglia CollectD StatsD http://en.wikipedia.org/wiki/Comparison_of_network_monitoring_systems New Relic Boundary DataDog EffectiveMySQL.com - Performance, Scalability & Business Continuity
  6. 6. New Relic EffectiveMySQL.com - Performance, Scalability & Business Continuity
  7. 7. New Relic Application Monitoring Historically Server Monitoring Software as a Service (SaaS) Agent-like EffectiveMySQL.com - Performance, Scalability & Business Continuity
  8. 8. The cloud - SaaS Minimal installation Minimal administration Cloud based management Web Mobile Integrations to other products EffectiveMySQL.com - Performance, Scalability & Business Continuity
  9. 9. New Relic Application Monitoring Introducing Server Monitoring NOW - New Relic Platform <insert product here> EffectiveMySQL.com - Performance, Scalability & Business Continuity
  10. 10. NEw Relic Subscription per server Standard ($24 p.s. p.m.) Pro ($149 p.s. p.m.) YOU CAN GET UNLIMITED STANDARD FREE BECAUSE BY THE MySQL PLUGIN AUTHOR http://j.mp/newrelic-mysql EffectiveMySQL.com - Performance, Scalability & Business Continuity
  11. 11. 1 2 3 No Credit Card Details required 4 http://j.mp/newrelic-mysql
  12. 12. NAVIGATION EffectiveMySQL.com - Performance, Scalability & Business Continuity
  13. 13. LICENSE EffectiveMySQL.com - Performance, Scalability & Business Continuity
  14. 14. ALERTs EffectiveMySQL.com - Performance, Scalability & Business Continuity
  15. 15. Server Ubuntu/Debian/RedHat/CentOS/Linux/Windows EffectiveMySQL.com - Performance, Scalability & Business Continuity
  16. 16. SERVER INSTALL LICENSE_KEY="<enter key here>" sudo rpm -Uvh http://download.newrelic.com/pub/newrelic/ el5/x86_64/newrelic-repo-5-3.noarch.rpm sudo yum install -y newrelic-sysmond sudo nrsysmond-config --set license_key=${LICENSE_KEY} sudo /etc/init.d/newrelic-sysmond start Helps to use good hostnames EffectiveMySQL.com - Performance, Scalability & Business Continuity
  17. 17. Application PHP/Ruby/.NET/Java/Python/Node.js EffectiveMySQL.com - Performance, Scalability & Business Continuity
  18. 18. PHP, Ruby, .NET, Java, Python, Node.js
  19. 19. PHP INSTALL LICENSE_KEY="<enter key here>" sudo rpm -Uvh http://download.newrelic.com/pub/newrelic/ el5/x86_64/newrelic-repo-5-3.noarch.rpm sudo yum install -y newrelic-php5 sudo newrelic-install install # prompted for LICENSE_KEY sudo service httpd restart EffectiveMySQL.com - Performance, Scalability & Business Continuity
  20. 20. Platform MySQL/Memcached/Apache/Redis/MongoDB/AWS/... EffectiveMySQL.com - Performance, Scalability & Business Continuity
  21. 21. JRE (on some server) https://github.com/newrelic-platform/newrelic_mysql_java_plugin/
  22. 22. MYSQL INSTALL Version 1 - Needs streamlining sudo su LICENSE_KEY="<enter key here>" wget -O /tmp/newrelic.tar.gz https://raw.github.com/newrelic-platform/ newrelic_mysql_java_plugin/master/dist/newrelic_mysql_plugin-1.0.6.tar.gz cd /usr/local tar xvfz /tmp/newrelic.tar.gz mv newrelic_mysql_plugin-1.* newrelic_mysql_plugin cd /usr/local/newrelic_mysql_plugin echo "licenseKey=${LICENSE_KEY}" > config/newrelic.properties echo "java.util.logging.ConsoleHandler.level = INFO com.newrelic.metrics.publish.handlers = java.util.logging.ConsoleHandler" > config/logging.properties echo '[ { "name" : "'`hostname -s`'", "host" : "localhost", "metrics" : "status,newrelic,innodb,innodb_status,innodb_mutex,slave" } ]' > config/mysql.instance.json #mysql < mysql_user.sql nohup java -jar newrelic_mysql_plugin-*.jar & https://github.com/newrelic-platform/newrelic_mysql_java_plugin/ EffectiveMySQL.com - Performance, Scalability & Business Continuity
  23. 23. General Views EffectiveMySQL.com - Performance, Scalability & Business Continuity
  24. 24. navigation EffectiveMySQL.com - Performance, Scalability & Business Continuity
  25. 25. Summary View
  26. 26. More
  27. 27. More
  28. 28. MYSQL Sources SHOW GLOBAL STATUS SHOW MASTER STATUS SHOW SLAVE STATUS SHOW ENGINE INNODB STATUS SHOW ENGINE INNODB MUTEX <Any SQL statement> EffectiveMySQL.com - Performance, Scalability & Business Continuity
  29. 29. Coding Features SHOW ENGINE INNODB MUTEX Dynamic metric names Rollup, supporting different formats in 5.5, 5.6, Percona etc EffectiveMySQL.com - Performance, Scalability & Business Continuity
  30. 30. Coding Features SHOW ENGINE INNODB STATUS EffectiveMySQL.com - Performance, Scalability & Business Continuity
  31. 31. Version 1 Your input and feedback See limitations EffectiveMySQL.com - Performance, Scalability & Business Continuity
  32. 32. Default is “PHP Application” /etc/php.d/newrelic.inc newrelic.appname = "PHP Application"
  33. 33. Other Products EffectiveMySQL.com - Performance, Scalability & Business Continuity
  34. 34. MEMCACHE INSTALL LICENSE_KEY="<enter key here>" sudo pip install newrelic-plugin-agent sudo cp /opt/newrelic_plugin_agent/newrelic_plugin_agent.cfg /etc/newrelic/newrelic_plugin_agent.cfg sudo sudo sudo sudo sudo useradd -m -s /bin/bash newrelic mkdir -p /var/log/newrelic /var/run/newrelic chown newrelic:newrelic /var/*/newrelic vi /etc/newrelic/newrelic_plugin_agent.cfg newrelic_plugin_agent -c /etc/newrelic/newrelic_plugin_agent.cfg -f https://rpm.newrelic.com/accounts/389698/plugins/directory/53 EffectiveMySQL.com - Performance, Scalability & Business Continuity
  35. 35. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  36. 36. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  37. 37. EffectiveMySQL.com - Performance, Scalability & Business Continuity
  38. 38. meetme plugin PHP Cache HTTP Server CouchDB Edgecast CDN API Memcached MongoDB Nginx pgBouncer PHP FPM PostgreSQL RabbitMQ Redis Riak uWSGI EffectiveMySQL.com - Performance, Scalability & Business Continuity
  39. 39. Other Plugins Logwatcher F5 Varnish AWS EC2/RDS/ELB Sendgrid Rabbit MQ Cassandra Sphinx Haproxy <write your own> Hadoop EffectiveMySQL.com - Performance, Scalability & Business Continuity
  40. 40. User Interface EffectiveMySQL.com - Performance, Scalability & Business Continuity
  41. 41. History EffectiveMySQL.com - Performance, Scalability & Business Continuity
  42. 42. Click and drag for subset of time
  43. 43. Configuration EffectiveMySQL.com - Performance, Scalability & Business Continuity
  44. 44. MySQL Plugin Plugin has more features than UI can support at present Configurable namespaces Row/Set SQL statements Configurable metrics EffectiveMySQL.com - Performance, Scalability & Business Continuity
  45. 45. MySQL config $ cat /path/to/config/mysql.instance.json [ { "name" : "db1", "host" : "localhost", "metrics" : "status,newrelic,innodb_status, innodb_mutex,master" }, One agent can collect data from { many MySQL Instances "name" : "db2", "host" : "db2.hostname.com", "user" : "monitor", "password" : "password", "metrics" : "status,newrelic,slave" } ] EffectiveMySQL.com - Performance, Scalability & Business Continuity
  46. 46. namespaces $ cat /path/to/config/metric.category.json [ { "category" : "status", "SQL" : "SHOW GLOBAL STATUS", "result" : "set", "value_metrics":"Max_used_connections,....", "counter_metrics" :"Bytes_received,Bytes_sent,..." }, { "category" : "slave", "SQL" : "SHOW SLAVE STATUS", "result" : "row", "value_metrics" : "Read_Master_Log_Pos, Slave_IO_Running, Slave_SQL_Running, EffectiveMySQL.com - Performance, Scalability & Business Continuity
  47. 47. namespaces $ cat /path/to/config/metric.category.json [ { "category" : "innodb_metrics", "SQL" : "SELECT name, count FROM information_schema.innodb_metrics", "result" : "set", "comment" : "MySQL 5.6 or later", "value_metrics" : "...", "counter_metrics" : "..." }, ... Current UI does not support with default MySQL plugin, but EffectiveMySQL.com - Performance, Scalability & Business Continuity
  48. 48. Modifying plugin https://github.com/newrelic-platform/ newrelic_mysql_java_plugin com.newrelic.plugins.mysql.instance.G UID="new value" Request to overload via parameter (to help with current UI limitations) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  49. 49. Limitations (Also exist with most monitoring products) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  50. 50. sampling 60 seconds (fixed) EffectiveMySQL.com - Performance, Scalability & Business Continuity
  51. 51. CopperEgg Example (15 second samples)
  52. 52. averaging Deceiving No maximum support EffectiveMySQL.com - Performance, Scalability & Business Continuity
  53. 53. alerting limits Positive range only Summary metrics only No low limits EffectiveMySQL.com - Performance, Scalability & Business Continuity
  54. 54. Custom Dashboards Enables you to group graphs together Does not work with plugins YET! EffectiveMySQL.com - Performance, Scalability & Business Continuity
  55. 55. Presentation Likes lots of moving/changing data Uninteresting values not well supported replication lag replication running YES/NO Important summary alert[able] metric EffectiveMySQL.com - Performance, Scalability & Business Continuity
  56. 56. Servers Likes to work on per server basis Default reporting namespace Current com.newrelic.plugins.mysql.instance [agent] /mysql/status/com_select [agent] /mysql/status/com* One Goal com.newrelic.plugins.mysql.topology /mysql/status/com_select/<instance> EffectiveMySQL.com - Performance, Scalability & Business Continuity
  57. 57. What is monitoring the monitoring?
  58. 58. Usage Patterns EffectiveMySQL.com - Performance, Scalability & Business Continuity
  59. 59. tmp disk tables High disk I/O - Investigate SQL EffectiveMySQL.com - Performance, Scalability & Business Continuity
  60. 60. query cache Query Cache NOT working EffectiveMySQL.com - Performance, Scalability & Business Continuity
  61. 61. Query CAche Enabling Query Cache benefit EffectiveMySQL.com - Performance, Scalability & Business Continuity
  62. 62. Yes, this time the database (servers) are on fire!
  63. 63. If only I could trigger an alert on this?
  64. 64. A Developers View EffectiveMySQL.com - Performance, Scalability & Business Continuity
  65. 65. SQL for developers Drill down to running queries Look at ALL thread SQL statements Aggregations EffectiveMySQL.com - Performance, Scalability & Business Continuity
  66. 66. Standard Feature
  67. 67. Standard Feature
  68. 68. Pro Feature
  69. 69. Standard Feature
  70. 70. Pro Feature
  71. 71. Pro Feature
  72. 72. Pro Feature
  73. 73. A better waterfall EffectiveMySQL.com - Performance, Scalability & Business Continuity
  74. 74. Standard Feature Still not a DB problem, but, I need to identify and fix
  75. 75. Conclusion EffectiveMySQL.com - Performance, Scalability & Business Continuity
  76. 76. Conclusion There is no excuse to have NO monitoring Visualization can generally help show issues Historical reporting can help review issues NEITHER WILL SOLVE THE PROBLEM Hire Ronald Bradford EffectiveMySQL.com - Performance, Scalability & Business Continuity
  77. 77. Ronald Bradford ronald@effectivemysql.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×