Nagios Conference 2011 - Mike Weber - Training: Reducing Nagios Server Load With Mod-Gearman

  • 2,150 views
Uploaded on

Mike Weber's training class on using mod-german to reduce the load on a Nagios monitoring server. The training session was held during the Nagios World Conference North America held Sept 27-29th, 2011 …

Mike Weber's training class on using mod-german to reduce the load on a Nagios monitoring server. The training session was held during the Nagios World Conference North America held Sept 27-29th, 2011 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,150
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
36
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Mod-Gearman Mike Weber mweber@spidertools.com
  • 2. Mod-Gearman Basic Functionality What does Mod-Gearman do? How does it work? Self-Load Balancing Using Mod-Gearman to reduce load on XI. Load Balancing Use workers to reduce load on Nagios. Distributed Monitoring Specify load on numerous networks and workers. Passive Monitoring Receiving passive checks. 2011 2
  • 3. Mod-Gearman: Basic Functionality Distributes tasks to multiple workers Mutiple Servers Multiple Workers Supports Multiple Programming Languages: C Java Perl PHP Python Shell 2011 3
  • 4. Mod-Gearman 2011 4
  • 5. NEB: Nagios Event Broker 2011 5
  • 6. Mod-Gearman Advantages Reduce Overall Load on Nagios Move Problem Processes Off Nagios Eventhandler waiting/hung Plugin Delay – some plugins capture resources for up to 10 seconds Reduce Overhead when Forking Nagios Core When Nagios has to fork a new process and the core is large, the system can take a major hit. 2011 6
  • 7. Mod-Gearman Queues 2011 7
  • 8. Mod-Gearman Worker Configuration Worker Identifier Unique identifier for worker, hostname min-worker Minimum number of total workers max-worker Maximum number of total workers idle-timeout Time in seconds before idle worker exits max-jobs Maximum number of jobs before worker exits 2011 8
  • 9. Install Process Install Nagios Event Broker broker_module=/usr/local/lib/mod_gearman/mod_gearman.o  config=/usr/local/etc/mod_gearman.conf  Install Server: gearmand /etc/init.d/gearmand start Install Worker: mod_gearman_worker /etc/init.d/mod_gearman_worker start Configuration File /usr/local/etc/mod_gearman.conf 2011 9
  • 10. Self-Load Balancing
  • 11. Avoiding Falling Dominos Mod-Gearman Server Monitoring and Distributing the Load Mod-Gearman Worker Using a Nagios Event Broker to Lighten the Load Using a Nagios Event Broker to Distribute the Load Reduce Latency Latency reduced by up to 37% Extend Life in Crisis Events Preparation for Load Balancing 2011 11
  • 12. Self-Load Balancing: Before - After 2011 12
  • 13. Mod-Gearman: Server and WorkerPID PPID CPU RAM13480 1 0.0 1328 03:14:05 /usr/local/bin/mod_gearman_worker -d –config=/usr/local/etc/mod_gearman.conf –pidfile=/usr/local/var/mod_gearman/mod_gearman_worker.pid13481 13480 0.0 1328 03:14:05 /usr/local/bin/mod_gearman_worker -d--config=/usr/local/etc/mod_gearman.conf –pidfile=/usr/local/var/mod_gearman/mod_gearman_worker.pid19734 1 0.0 114580 01:07:50 /usr/local/sbin/gearmand -p 4730 -P/usr/local/var/mod_gearman/gearmand.pid -d -j 0 -t 10 --log-file=/usr/local/var/mod_gearman/gearmand.log -v -v --lAs you can see the worker uses very little in the way of resources (1328 KB of Memory)compared to the server which uses over 111 MB of Memory. When you look at these figures,you can see adding a worker to help shift the load on the local Nagios server is not a far fetchedas you might originally think. 2011 13
  • 14. Load Balancing
  • 15. Load Balancing 2011 15
  • 16. Load Balancing: Server and WorkerServer Configuration: /usr/local/etc/mod_gearman.confserver=localhost:4730eventhandler=yesservices=yeshosts=yesencryption=yeskey=linux23_Qg549KWorker Configuration: /usr/local/etc/mod_gearman.confserver=192.168.5.4:4730eventhandler=yesservices=yeshosts=yesencryption=yeskey=linux23_Qg549K 2011 16
  • 17. Load Balancing: 2 Workers 2011 17
  • 18. Distributed Monitoring
  • 19. Distributed Monitoring 2011 19
  • 20. Distributed Monitoring: HostgroupsServer Configuration: /usr/local/etc/mod_gearman.confserver=localhost:4730eventhandler=yesservices=yeshosts=yesencryption=yeskey=linux23_Qg549KWorker Configuration: /usr/local/etc/mod_gearman.confserver=192.168.5.4:4730eventhandler=yesservices=yeshosts=yesencryption=yeskey=linux23_Qg549Khostgroups=debian­servers 2011 20
  • 21. Passive Monitoring
  • 22. Passive Monitoring 2011 22
  • 23. Passive Monitoring: send_gearman#!/bin/bash# Mod­Gearman Passive User Checkusers=`/usr/local/nagios/libexec/check_users ­w 1 ­c 5`/usr/local/bin/send_gearman ­­server=192.168.5.4:4730 ­­encryption=yes ­­key=linux23_Qg549K ­­host=db ­­service=GearUsers ­­message="$users" ­­returncode=1­­server=192.168.5.4:4730­­encryption=yes­­key=linux23_Qg549K­­host=db­­service=GearUsers­­message="$users"­­returncode=1 2011 23
  • 24. Passive Monitoring: Service Check 2011 24