NRPE
Nagios Remote Plugin Executor


        Mike Weber

    mweber@spidertools.com
NRPE Concepts


  Agent Required on Client
  * plugin or script executes locally
  * daemon runs on client (port 5666)

  Nagios
  * check_nrpe connects to client


  Direct Checks
  * connect and collect data from client

  Indirect Checks
  * connect to client, execute plugin to connect to second client




                                    2012                            2
Basic Concepts




                 2012   3
NRPE: Active and Indirect Checks




                      2012         4
NRPE Components: Client


   NRPE Daemon
   * daemon managed by xinetd
   * security based on allowed_hosts
   * runs on port 5666
   * open firewall

   NRPE Commands File (nrpe.cfg)
   * check_nrpe connects to client
   * command definitions

   Plugins
   * connect and collect data from client




                                     2012   5
NRPE Components: Server


  One Plugin
  * check_nrpe (installed by default in XI)

  Host and Service Definitions
  * define the host
  * define each service




                                   2012       6
NRPE Variables
NRPE Variables: Client


   Agent Install
   * repository (CentOS, SUSE,Ubuntu, Debian, etc.)
   * XI agent from wizard
   * compile

   Daemon Install
   * repository (CentOS, SUSE,Ubuntu, Debian, etc.)
   * compile

   nrpe.cfg
   * repository (CentOS, SUSE,Ubuntu, Debian, etc.)
   * compile

   Command Definitions


                                 2012                 8
Client Variables: Daemon and Config


   Program
   /usr/local/nagios/bin/nrpe
   /usr/sbin/nrpe

   Daemon Config
   /etc/xinet.d/nrpe
   /etc/nagios/nrpe
   /etc/nagios3/nrpe (Ubuntu)




                                2012   9
NRPE: Daemon Config - nrpe


# default: on
# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
     flags       = REUSE
     socket_type = stream
     port        = 5666
     wait        = no
     user        = nagios
     group        = nagios
     server       = /usr/local/nagios/bin/nrpe
     server_args = -c /usr/local/nagios/etc/nrpe.cfg --inetd
     log_on_failure += USERID
     disable      = no
     only_from      = 127.0.0.1 192.168.5.99
}




                                            2012               10
Client Variables: Commands File


 Location
 /usr/local/nagios/etc/nrpe.cfg (compile)
 /usr/local/nagios/etc/nrpe/common.cfg (XI Agent)
 /etc/nagios/nrpe.cfg (SUSE)
 /etc/nagios3/nrpe.cfg (Ubuntu)

 Definitions
 * able to use arguments
 * dont_blame_nrpe=1

 command[check_users]=/usr/local/nagios/libexec/check_users -w $ARG1$ -c $ARG2$

 * unable to use arguments
 * dont_blame_nrpe=0

 command[check_users]=/usr/local/nagios/libexec/check_users -w 2 -c 4



                                         2012                                     11
Client Variables: Plugins Location


 /usr/local/nagios/libexec

 /usr/lib/nagios/plugins

 /usr/lib64/nagios/plugins




                           2012      12
NRPE Compile
Why Compile NRPE and Plugins?


 Eliminate Variables Across Distribution
 * same daemon settings and location
 * same nrpe definitions
 * same commands locations and configuration (nrpe.cfg)
 * same location for plugins
 * same versions

 Uniform Troubleshooting
 * standards across distributions
 * easier for multiple administrators

 Uniform Upgrades
 * method of upgrade the same
 * upgrades will be the latest




                                        2012              14
NRPE: Install Script

#!/bin/bash
if [[ $EUID -ne 0 ]]; then
       echo "This script must be run as root" 1>&2
       exit 1
fi

cd /tmp
wget http://sourceforge.net/projects/nagios/files/nrpe-2.x/nrpe-2.13/nrpe-2.13.tar.gz/download
tar zxvf nrpe-2.13.tar.gz
cd nrpe-2.13

yum install -y mod_ssl openssl-devel xinetd gcc make
./configure --with-ssl=/usr/bin/openssl --with-ssl-lib=/usr/lib

make
make install
make install-daemon
make install-daemon-config
make install-xinetd

cp sample-config/nrpe.cfg /usr/local/nagios/etc/


                                                 2012                                        15
Plugins: Install Script
#!/bin/bash
if [[ $EUID -ne 0 ]]; then
       echo "This script must be run as root" 1>&2
       exit 1
fi

cd /tmp
wget http://sourceforge.net/projects/nagiosplug/files/nagiosplug/1.4.16/nagios-plugins-
1.4.16.tar.gz/download

yum install -y gcc make mysql-devel mysql net-snmp

useradd nagios
groupadd nagcmd
usermod -a -G nagcmd nagios

tar zxvf nagios-plugins-1.4.16.tar.gz
cd nagios-plugins-1.4.16
./configure --with-nagios-user=nagios --with-nagios-group=nagios

make
make install
exit 0

                                               2012                                       16
NRPE: Locations When Compiled


 Daemon Configuration
 /etc/xinetd.d/nrpe


 Daemon Binary
 /usr/local/nagios/bin/nrpe


 Configuration File
 /usr/local/nagios/etc/nrpe.cfg


 Plugins
 /usr/local/nagios/libexec



                                  2012   17
NRPE: XI Agent
CentOS Agent: XI


cd /tmp
wget http://assets.nagios.com/downloads/nagiosxi/agents/linux-nrpe-agent.tar.gz
tar zxvf linux-nrpe-agent.tar.gz
cd linux-nrpe-agent

./fullinstall

Allow from: 192.168.5.184

Stopping xinetd:                          [ OK ]
Starting xinetd:                          [ OK ]
Subcomponents installed OK
RESULT=0

##########################################################
###                                              ###
### Nagios XI Linux Agent Installation Complete! ###
###                                              ###
##########################################################


                                            2012                                  19
NRPE: Locations XI Agent

 Daemon Configuration
 /etc/xinetd.d/nrpe


 Daemon Binary
 /usr/local/nagios/bin/nrpe


 Configuration File
 /usr/local/nagios/etc/nrpe.cfg
     include_dir=/usr/local/nagios/etc/nrpe
    dont_blame_nrpe=1

 /usr/local/nagios/etc/nrpe/common.cfg

 Plugins
 /usr/local/nagios/libexec

                                    2012      20
common.cfg
### GENERIC SERVICES ###
command[check_init_service]=sudo /usr/local/nagios/libexec/check_init_service $ARG1$
command[check_services]=/usr/local/nagios/libexec/check_services -p $ARG1$

### MISC SYSTEM METRICS ###
#command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_users]=/usr/local/nagios/libexec/check_users $ARG1$
command[check_load]=/usr/local/nagios/libexec/check_load $ARG1$
command[check_swap]=/usr/local/nagios/libexec/check_swap $ARG1$
command[check_cpu_stats]=/usr/local/nagios/libexec/check_cpu_stats.sh $ARG1$
command[check_mem]=/usr/local/nagios/libexec/custom_check_mem $ARG1$

### YUM UPDATES ###
command[check_yum]=/usr/local/nagios/libexec/check_yum

### DISK ###
command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$
command[check_ide_smart]=/usr/local/nagios/libexec/check_ide_smart $ARG1$

### PROCESSES ###
command[check_all_procs]=/usr/local/nagios/libexec/custom_check_procs
command[check_procs]=/usr/local/nagios/libexec/check_procs $ARG1$

### OPEN FILES ###
command[check_open_files]=/usr/local/nagios/libexec/check_open_files.pl $ARG1$

### NETWORK CONNECTIONS ###
command[check_netstat]=/usr/local/nagios/libexec/check_netstat.pl -p $ARG1$ $ARG2$

                                                   2012                                21
Commands: nrpe.cfg
No Variables Allowed

dont_blame_nrpe=0

command <service_check> <path_to_plugin> <plugin> <arguments>

command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1



Using Variables

dont_blame_nrpe=1

command <service_check> <path_to_plugin> <plugin> <arguments>

command[check_hda1]=/usr/local/nagios/libexec/check_disk -w $ARG1$ -c $ARG2$ -p
$ARG3$




                                          2012                                        22
Downside of Arguments


 CentOS
 check_cpu_stats!-a '-w 85 -c 95'


 SUSE Enterprise
 check_cpu_stats!-a -w 80 -a -c 90




                                     2012   23
NRPE: Nagios Core
NRPE: Active Check




                     2012   25
Nagios Core: Service Checks

No Arguments
define service{
     use                   generic-service
     host_name             centos
     service_description   Users
     check_command          check_nrpe!check_users
     }
define service{
     use                   generic-service,srv-pnp
     host_name             centos
     service_description   Mailq
     check_command         check_nrpe!check_mailq
   }

Arguments
define service{
     use                   generic-service
     host_name             centos
     service_description   Disk
     check_command         check_nrpe_arg!check_disk!80!90!/
  }

                                           2012                26
Nagios Core: Command Definition


No Arguments
define command{
     command_name      check_nrpe
     command_line      $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}

define command{
     command_name      check_nrpe_60
     command_line      $USER1$/check_nrpe -H $HOSTADDRESS$ -t 60 -c $ARG1$
}

Arguments
define command{
     command_name      check_nrpe_arg
     command_line      $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a
$ARG2$ $ARG3$ $ARG4$
}



                                  2012                                    27
Summary
No Arguments
define command{
     command_name          check_nrpe
     command_line          $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}
define service{
     use                   generic-service
     host_name             centos
     service_description   Users
     check_command         check_nrpe!check_users
     }
command[check_users]=/usr/local/nagios/libexec/check_users -w 2 -c 4

Arguments
define command{
     command_name           check_nrpe_arg
     command_line           $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$
$ARG3$ $ARG4$
}
define service{
     use                    generic-service
     host_name              centos
     service_description    Disk
     check_command          check_nrpe_arg!check_disk!80!90!/
  }
command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$

                                         2012                                         28
AIDE: Advanced Intrusion Detection Environment

Script
!/bin/bash
# Create 4 Hour Cron Job With AIDE
logfile=/tmp/aide
x=$(grep "Looks okay" $logfile | wc -l)
if [ $x -eq 1 ]
then
       echo "All Systems Look OK"
       stateid=0
else
       echo "$(egrep "added|changed" /tmp/aide)"
       stateid=2
fi
exit $stateid

Command in nrpe.cfg
command[check_aide]=/usr/local/nagios/libexec/check_aide

Service Definition
define service{
     use              generic-service
     host_name           bash
     service_description AIDE
     check_command           check_nrpe!check_aide
     }

                                                   2012    29
NRPE: Nagios XI
NRPE: Linux Server




                     2012   31
NRPE: Linux Server




                     2012   32
NRPE: Linux Server




                     2012   33
NRPE: Linux Server




                     2012   34
NRPE: Linux Server




                     2012   35
/etc/init.d




ls /etc/init.d
crond            gpm         iscsi        lvm2-monitor   named       nscd        rawdevices
snmpd             sysstat    dc_client    haldaemon      iscsid      mcstrans     netconsole
ntpd             rdisc       snmptrapd    vsftpd         dc_server   halt         killall
messagebus       netfs       portmap      restorecond    snmptt      winbind      dovecot
httpd            kudzu       multipathd   netplugd        postfix    saslauthd   sshd
xinetd           functions   iptables     lm_sensors     mysqld      network     postgresql-9.0
single           syslog




                                              2012                                                36
NRPE: Linux Server




                     2012   37
NRPE: Linux Server




                     2012   38
NRPE: Linux Server




                     2012   39
NRPE: Linux Server




                     2012   40
NRPE: Additional Checks




                     2012   41
NRPE: Additional Checks




                     2012   42
NRPE: Additional Checks




                     2012   43
NRPE: Questions?

Nagios Conference 2012 - Mike Weber - NRPE

  • 1.
    NRPE Nagios Remote PluginExecutor Mike Weber mweber@spidertools.com
  • 2.
    NRPE Concepts Agent Required on Client * plugin or script executes locally * daemon runs on client (port 5666) Nagios * check_nrpe connects to client Direct Checks * connect and collect data from client Indirect Checks * connect to client, execute plugin to connect to second client 2012 2
  • 3.
  • 4.
    NRPE: Active andIndirect Checks 2012 4
  • 5.
    NRPE Components: Client NRPE Daemon * daemon managed by xinetd * security based on allowed_hosts * runs on port 5666 * open firewall NRPE Commands File (nrpe.cfg) * check_nrpe connects to client * command definitions Plugins * connect and collect data from client 2012 5
  • 6.
    NRPE Components: Server One Plugin * check_nrpe (installed by default in XI) Host and Service Definitions * define the host * define each service 2012 6
  • 7.
  • 8.
    NRPE Variables: Client Agent Install * repository (CentOS, SUSE,Ubuntu, Debian, etc.) * XI agent from wizard * compile Daemon Install * repository (CentOS, SUSE,Ubuntu, Debian, etc.) * compile nrpe.cfg * repository (CentOS, SUSE,Ubuntu, Debian, etc.) * compile Command Definitions 2012 8
  • 9.
    Client Variables: Daemonand Config Program /usr/local/nagios/bin/nrpe /usr/sbin/nrpe Daemon Config /etc/xinet.d/nrpe /etc/nagios/nrpe /etc/nagios3/nrpe (Ubuntu) 2012 9
  • 10.
    NRPE: Daemon Config- nrpe # default: on # description: NRPE (Nagios Remote Plugin Executor) service nrpe { flags = REUSE socket_type = stream port = 5666 wait = no user = nagios group = nagios server = /usr/local/nagios/bin/nrpe server_args = -c /usr/local/nagios/etc/nrpe.cfg --inetd log_on_failure += USERID disable = no only_from = 127.0.0.1 192.168.5.99 } 2012 10
  • 11.
    Client Variables: CommandsFile Location /usr/local/nagios/etc/nrpe.cfg (compile) /usr/local/nagios/etc/nrpe/common.cfg (XI Agent) /etc/nagios/nrpe.cfg (SUSE) /etc/nagios3/nrpe.cfg (Ubuntu) Definitions * able to use arguments * dont_blame_nrpe=1 command[check_users]=/usr/local/nagios/libexec/check_users -w $ARG1$ -c $ARG2$ * unable to use arguments * dont_blame_nrpe=0 command[check_users]=/usr/local/nagios/libexec/check_users -w 2 -c 4 2012 11
  • 12.
    Client Variables: PluginsLocation /usr/local/nagios/libexec /usr/lib/nagios/plugins /usr/lib64/nagios/plugins 2012 12
  • 13.
  • 14.
    Why Compile NRPEand Plugins? Eliminate Variables Across Distribution * same daemon settings and location * same nrpe definitions * same commands locations and configuration (nrpe.cfg) * same location for plugins * same versions Uniform Troubleshooting * standards across distributions * easier for multiple administrators Uniform Upgrades * method of upgrade the same * upgrades will be the latest 2012 14
  • 15.
    NRPE: Install Script #!/bin/bash if[[ $EUID -ne 0 ]]; then echo "This script must be run as root" 1>&2 exit 1 fi cd /tmp wget http://sourceforge.net/projects/nagios/files/nrpe-2.x/nrpe-2.13/nrpe-2.13.tar.gz/download tar zxvf nrpe-2.13.tar.gz cd nrpe-2.13 yum install -y mod_ssl openssl-devel xinetd gcc make ./configure --with-ssl=/usr/bin/openssl --with-ssl-lib=/usr/lib make make install make install-daemon make install-daemon-config make install-xinetd cp sample-config/nrpe.cfg /usr/local/nagios/etc/ 2012 15
  • 16.
    Plugins: Install Script #!/bin/bash if[[ $EUID -ne 0 ]]; then echo "This script must be run as root" 1>&2 exit 1 fi cd /tmp wget http://sourceforge.net/projects/nagiosplug/files/nagiosplug/1.4.16/nagios-plugins- 1.4.16.tar.gz/download yum install -y gcc make mysql-devel mysql net-snmp useradd nagios groupadd nagcmd usermod -a -G nagcmd nagios tar zxvf nagios-plugins-1.4.16.tar.gz cd nagios-plugins-1.4.16 ./configure --with-nagios-user=nagios --with-nagios-group=nagios make make install exit 0 2012 16
  • 17.
    NRPE: Locations WhenCompiled Daemon Configuration /etc/xinetd.d/nrpe Daemon Binary /usr/local/nagios/bin/nrpe Configuration File /usr/local/nagios/etc/nrpe.cfg Plugins /usr/local/nagios/libexec 2012 17
  • 18.
  • 19.
    CentOS Agent: XI cd/tmp wget http://assets.nagios.com/downloads/nagiosxi/agents/linux-nrpe-agent.tar.gz tar zxvf linux-nrpe-agent.tar.gz cd linux-nrpe-agent ./fullinstall Allow from: 192.168.5.184 Stopping xinetd: [ OK ] Starting xinetd: [ OK ] Subcomponents installed OK RESULT=0 ########################################################## ### ### ### Nagios XI Linux Agent Installation Complete! ### ### ### ########################################################## 2012 19
  • 20.
    NRPE: Locations XIAgent Daemon Configuration /etc/xinetd.d/nrpe Daemon Binary /usr/local/nagios/bin/nrpe Configuration File /usr/local/nagios/etc/nrpe.cfg include_dir=/usr/local/nagios/etc/nrpe dont_blame_nrpe=1 /usr/local/nagios/etc/nrpe/common.cfg Plugins /usr/local/nagios/libexec 2012 20
  • 21.
    common.cfg ### GENERIC SERVICES### command[check_init_service]=sudo /usr/local/nagios/libexec/check_init_service $ARG1$ command[check_services]=/usr/local/nagios/libexec/check_services -p $ARG1$ ### MISC SYSTEM METRICS ### #command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10 command[check_users]=/usr/local/nagios/libexec/check_users $ARG1$ command[check_load]=/usr/local/nagios/libexec/check_load $ARG1$ command[check_swap]=/usr/local/nagios/libexec/check_swap $ARG1$ command[check_cpu_stats]=/usr/local/nagios/libexec/check_cpu_stats.sh $ARG1$ command[check_mem]=/usr/local/nagios/libexec/custom_check_mem $ARG1$ ### YUM UPDATES ### command[check_yum]=/usr/local/nagios/libexec/check_yum ### DISK ### command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$ command[check_ide_smart]=/usr/local/nagios/libexec/check_ide_smart $ARG1$ ### PROCESSES ### command[check_all_procs]=/usr/local/nagios/libexec/custom_check_procs command[check_procs]=/usr/local/nagios/libexec/check_procs $ARG1$ ### OPEN FILES ### command[check_open_files]=/usr/local/nagios/libexec/check_open_files.pl $ARG1$ ### NETWORK CONNECTIONS ### command[check_netstat]=/usr/local/nagios/libexec/check_netstat.pl -p $ARG1$ $ARG2$ 2012 21
  • 22.
    Commands: nrpe.cfg No VariablesAllowed dont_blame_nrpe=0 command <service_check> <path_to_plugin> <plugin> <arguments> command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1 Using Variables dont_blame_nrpe=1 command <service_check> <path_to_plugin> <plugin> <arguments> command[check_hda1]=/usr/local/nagios/libexec/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$ 2012 22
  • 23.
    Downside of Arguments CentOS check_cpu_stats!-a '-w 85 -c 95' SUSE Enterprise check_cpu_stats!-a -w 80 -a -c 90 2012 23
  • 24.
  • 25.
  • 26.
    Nagios Core: ServiceChecks No Arguments define service{ use generic-service host_name centos service_description Users check_command check_nrpe!check_users } define service{ use generic-service,srv-pnp host_name centos service_description Mailq check_command check_nrpe!check_mailq } Arguments define service{ use generic-service host_name centos service_description Disk check_command check_nrpe_arg!check_disk!80!90!/ } 2012 26
  • 27.
    Nagios Core: CommandDefinition No Arguments define command{ command_name check_nrpe command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ } define command{ command_name check_nrpe_60 command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -t 60 -c $ARG1$ } Arguments define command{ command_name check_nrpe_arg command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$ $ARG3$ $ARG4$ } 2012 27
  • 28.
    Summary No Arguments define command{ command_name check_nrpe command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ } define service{ use generic-service host_name centos service_description Users check_command check_nrpe!check_users } command[check_users]=/usr/local/nagios/libexec/check_users -w 2 -c 4 Arguments define command{ command_name check_nrpe_arg command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$ $ARG3$ $ARG4$ } define service{ use generic-service host_name centos service_description Disk check_command check_nrpe_arg!check_disk!80!90!/ } command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$ 2012 28
  • 29.
    AIDE: Advanced IntrusionDetection Environment Script !/bin/bash # Create 4 Hour Cron Job With AIDE logfile=/tmp/aide x=$(grep "Looks okay" $logfile | wc -l) if [ $x -eq 1 ] then echo "All Systems Look OK" stateid=0 else echo "$(egrep "added|changed" /tmp/aide)" stateid=2 fi exit $stateid Command in nrpe.cfg command[check_aide]=/usr/local/nagios/libexec/check_aide Service Definition define service{ use generic-service host_name bash service_description AIDE check_command check_nrpe!check_aide } 2012 29
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
    /etc/init.d ls /etc/init.d crond gpm iscsi lvm2-monitor named nscd rawdevices snmpd sysstat dc_client haldaemon iscsid mcstrans netconsole ntpd rdisc snmptrapd vsftpd dc_server halt killall messagebus netfs portmap restorecond snmptt winbind dovecot httpd kudzu multipathd netplugd postfix saslauthd sshd xinetd functions iptables lm_sensors mysqld network postgresql-9.0 single syslog 2012 36
  • 37.
  • 38.
  • 39.
  • 40.
  • 41.
  • 42.
  • 43.
  • 44.