Icinga Camp Amsterdam 2016
Icinga 2 and puppet
automatic monitoring.. ?
Icinga Camp Amsterdam 2016
Who am I?
• Walter Heck, Software engineer turned DBA, turned
Sysadmin, turned entrepreneur, promoted to CTO
• CTO/Founder of OlinData (http://www.olindata.com)
o Icinga partner for Holland, India and Southeast Asia
o Puppet Labs training partner for most of Asia and part of
Europe (Not NL!)
o Linux Foundation training partner
o MySQL consulting
Icinga Camp Amsterdam 2016
Overview
• What is puppet?
• Basic icinga setup with puppet
• Zones, hosts, objects
• Puppet’s exported resources
• What’s next?
• Questions
Icinga Camp Amsterdam 2016
What is Puppet and why do we care?
• Configuration management software
- http://www.olindata.com/blog/2014/08/puppet-master-agent-setup
- http://olindata.com/blog/2015/03/setup-puppet-server-centos-70
• Scales very well (from 1 to 200k+ nodes)
• Multi-platform (windows, *nix, Mac OS, BSD)
• Commercially supported Open Source
• Infrastructure as code
Icinga Camp Amsterdam 2016
Typical Puppet Architecture
Puppet Master
Puppet Code
(.git repository)
web01.olindata.com icinga.olindata.comdb01.olindata.com
Puppet Agent Puppet AgentPuppet Agent
Icinga Camp Amsterdam 2016
● No official icinga2 module on the
forge
● Github repo in strange state
○ last commit to ‘master’ branch
for icinga2 module: 30 Jan 2015
(!)
○ use ‘develop’ branch instead: 261
(!) commits ahead of master
check out: https://github.com/Icinga/puppet-
icinga2/pull/94
please help me bug @lazyfrosch for this ;)
Puppet module status: ‘up for improvement’
Icinga Camp Amsterdam 2016
Puppet modules
● Your best bet for now: use github repos directly:
○ mod 'icingaweb2',
■ :git => 'https://github.com/icinga/puppet-icingaweb2.git'
○ mod 'icinga2',
■ :git => 'https://github.com/icinga/puppet-icinga2.git',
■ :branch => 'develop'
○ mod 'puppetlabs/mysql', '3.6.2'
Icinga Camp Amsterdam 2016
Prerequisite: mysql (or postgres)
class profile::icinga::db {
$icinga2_webdb_password = ‘mypw’
$icinga2_ido_password = ‘mypw’
$mysql_whitelist_range = ‘192.168.%’
mysql::db { 'icinga2_web':
user => 'icinga2_web',
password => $icinga2_webdb_password,
host => $mysql_whitelist_range,
grant => ['ALL'],
}
mysql::db { 'icinga2_data':
user => 'icinga2',
password => $icinga2_ido_password,
host => $mysql_whitelist_range,
grant => ['ALL'],
}
}
Icinga Camp Amsterdam 2016
Setting up the icinga server
$icinga2_db_ipaddress = hiera('icinga::mysql_ipaddress')
$icinga2_web_fqdn = hiera('icingaweb::fqdn')
$icinga2_ido_password = hiera('icinga::ido_password')
class { 'icinga2':
db_type => 'mysql',
db_host => $icinga2_db_ipaddress,
db_port => '3306',
db_name => 'icinga2_data',
db_user => 'icinga2',
db_pass => $icinga2_ido_password,
manage_database => true,
}
Icinga Camp Amsterdam 2016
Set up icingaweb2
● requires webserver + php
● please use https!
● populate the database
class profile::icinga::web {
class { 'icingaweb2':
manage_repo => true,
install_method => 'package',
manage_apache_vhost => true,
apache_vhost_name => $icinga2_web_fqdn,
ido_db => 'mysql',
ido_db_host => $icinga2_db_ip,
ido_db_name => 'icinga2_data',
ido_db_user => 'icinga2',
ido_db_pass => $icinga2_ido_pass,
web_db => 'mysql',
web_db_name => 'icinga2_web',
web_db_host => $icinga2_db_ip,
web_db_user => 'icinga2_web',
web_db_pass => $icinga2_webdb_pass,
web_db_port => '3306',
}
}
Icinga Camp Amsterdam 2016
Setting up an icinga native client
class profile::icinga::client {
include ::icinga2
include ::icinga2::feature::command
class { '::icinga2::feature::api':
accept_commands => true,
accept_config => true,
manage_zone => false,
}
# icinga2::pki::puppet class needs to be declared
# after the icinga2::feature::api class in order
# to avoid resource duplication
contain ::icinga2::pki::puppet
}
Icinga Camp Amsterdam 2016
Exported resources
PuppetDB
Puppet
Master
ICINGAWEB01
@@icinga2::object::host { $::fqdn:
ipv4_address => $::ipaddress,
}
Icinga2::Object::Host <<| |>>
1. Puppet agent run
6. Send to node
5. Retrieve from PuppetDB3. Store in PuppetDB
2. Export to PM
4. Collect on ICINGA
Icinga Camp Amsterdam 2016
Exported resources
PuppetDB
Puppet
Master
ICINGAWEB01
@@icinga2::object::host { $::fqdn:
ipv4_address => $::ipaddress,
}
Icinga2::Object::Host <<| |>>
1. Puppet agent run
6. Send to node
5. Retrieve from PuppetDB3. Store in PuppetDB
2. Export to PM
4. Collect on ICINGA
Icinga Camp Amsterdam 2016
Exported resources
PuppetDB
Puppet
Master
ICINGAWEB01
@@icinga2::object::host { $::fqdn:
ipv4_address => $::ipaddress,
}
Icinga2::Object::Host <<| |>>
1. Puppet agent run
6. Send to node
5. Retrieve from PuppetDB3. Store in PuppetDB
2. Export to PM
4. Collect on ICINGA
Icinga Camp Amsterdam 2016
Exported resources
PuppetDB
Puppet
Master
ICINGAWEB01
@@icinga2::object::host { $::fqdn:
ipv4_address => $::ipaddress,
}
Icinga2::Object::Host <<| |>>
1. Puppet agent run
6. Send to node
5. Retrieve from PuppetDB3. Store in PuppetDB
2. Export to PM
4. Collect on ICINGA
Icinga Camp Amsterdam 2016
Exported resources
PuppetDB
Puppet
Master
ICINGAWEB01
@@icinga2::object::host { $::fqdn:
ipv4_address => $::ipaddress,
}
Icinga2::Object::Host <<| |>>
1. Puppet agent run
6. Send to node
5. Retrieve from PuppetDB3. Store in PuppetDB
2. Export to PM
4. Collect on ICINGA
Icinga Camp Amsterdam 2016
Exported resources
PuppetDB
Puppet
Master
ICINGAWEB01
@@icinga2::object::host { $::fqdn:
ipv4_address => $::ipaddress,
}
Icinga2::Object::Host <<| |>>
1. Puppet agent run
6. Send to node
5. Retrieve from PuppetDB3. Store in PuppetDB
2. Export to PM
4. Collect on ICINGA
Icinga Camp Amsterdam 2016
Exported resources
node ‘icinga.olindata.com’ {
# Collect all exported host objects
Icinga2::Object::Host <<| |>>
# Collect all exported service objects
Icinga2::Object::Service <<| |>>
# Collect all exported zone objects
Icinga2::Object::Zone <<| |>>
}
node ‘web01.olindata.com’ {
@@icinga2::object::host { $::fqdn:
ipv4_address => $::ipaddress,
}
}
Icinga Camp Amsterdam 2016
# This is a resource default
Icinga2::Object::Apply_service {
assign_where => 'host.address && host.vars.remote == true && host.vars.remote_endpoint',
command_endpoint => 'host.vars.remote_endpoint',
}
icinga2::object::apply_service { 'user':
check_command => 'users',
}
icinga2::object::apply_service { 'load':
check_command => 'load',
}
icinga2::object::apply_service { 'process':
check_command => 'procs',
}
Adding some initial things
Icinga Camp Amsterdam 2016
What’s next?
● create profiles for each type of application, eg profile::
icinga::apache, profile::icinga::gitlab, etc.
● check out https://github.com/Icinga/icinga-vagrant/ for nice
vagrant examples (not so nice puppet though ;) )
● add grafana, business process monitoring, etc.
● create users, usergroups, hostgroups, etc.
Icinga Camp Amsterdam 2016
Icinga Training NL
● Official icinga training in the Netherlands on July 18th-21st
● Tickets only 1350 EUR(!)
http://olindata.com/training/netherlands/amsterdam/icinga-
fundamentals-training-amsterdam-july-2016
Icinga Camp Amsterdam 2016
Questions?
@walterheck / @olindata
http://www.olindata.com
walterheck@olindata.com
http://github.com/olindata
Icinga Camp Amsterdam 2016
We’re hiring!
NL based consultants
jobs@olindata.com

Icinga Camp Amsterdam - Icinga2 and Puppet

  • 1.
    Icinga Camp Amsterdam2016 Icinga 2 and puppet automatic monitoring.. ?
  • 2.
    Icinga Camp Amsterdam2016 Who am I? • Walter Heck, Software engineer turned DBA, turned Sysadmin, turned entrepreneur, promoted to CTO • CTO/Founder of OlinData (http://www.olindata.com) o Icinga partner for Holland, India and Southeast Asia o Puppet Labs training partner for most of Asia and part of Europe (Not NL!) o Linux Foundation training partner o MySQL consulting
  • 3.
    Icinga Camp Amsterdam2016 Overview • What is puppet? • Basic icinga setup with puppet • Zones, hosts, objects • Puppet’s exported resources • What’s next? • Questions
  • 4.
    Icinga Camp Amsterdam2016 What is Puppet and why do we care? • Configuration management software - http://www.olindata.com/blog/2014/08/puppet-master-agent-setup - http://olindata.com/blog/2015/03/setup-puppet-server-centos-70 • Scales very well (from 1 to 200k+ nodes) • Multi-platform (windows, *nix, Mac OS, BSD) • Commercially supported Open Source • Infrastructure as code
  • 5.
    Icinga Camp Amsterdam2016 Typical Puppet Architecture Puppet Master Puppet Code (.git repository) web01.olindata.com icinga.olindata.comdb01.olindata.com Puppet Agent Puppet AgentPuppet Agent
  • 6.
    Icinga Camp Amsterdam2016 ● No official icinga2 module on the forge ● Github repo in strange state ○ last commit to ‘master’ branch for icinga2 module: 30 Jan 2015 (!) ○ use ‘develop’ branch instead: 261 (!) commits ahead of master check out: https://github.com/Icinga/puppet- icinga2/pull/94 please help me bug @lazyfrosch for this ;) Puppet module status: ‘up for improvement’
  • 7.
    Icinga Camp Amsterdam2016 Puppet modules ● Your best bet for now: use github repos directly: ○ mod 'icingaweb2', ■ :git => 'https://github.com/icinga/puppet-icingaweb2.git' ○ mod 'icinga2', ■ :git => 'https://github.com/icinga/puppet-icinga2.git', ■ :branch => 'develop' ○ mod 'puppetlabs/mysql', '3.6.2'
  • 8.
    Icinga Camp Amsterdam2016 Prerequisite: mysql (or postgres) class profile::icinga::db { $icinga2_webdb_password = ‘mypw’ $icinga2_ido_password = ‘mypw’ $mysql_whitelist_range = ‘192.168.%’ mysql::db { 'icinga2_web': user => 'icinga2_web', password => $icinga2_webdb_password, host => $mysql_whitelist_range, grant => ['ALL'], } mysql::db { 'icinga2_data': user => 'icinga2', password => $icinga2_ido_password, host => $mysql_whitelist_range, grant => ['ALL'], } }
  • 9.
    Icinga Camp Amsterdam2016 Setting up the icinga server $icinga2_db_ipaddress = hiera('icinga::mysql_ipaddress') $icinga2_web_fqdn = hiera('icingaweb::fqdn') $icinga2_ido_password = hiera('icinga::ido_password') class { 'icinga2': db_type => 'mysql', db_host => $icinga2_db_ipaddress, db_port => '3306', db_name => 'icinga2_data', db_user => 'icinga2', db_pass => $icinga2_ido_password, manage_database => true, }
  • 10.
    Icinga Camp Amsterdam2016 Set up icingaweb2 ● requires webserver + php ● please use https! ● populate the database class profile::icinga::web { class { 'icingaweb2': manage_repo => true, install_method => 'package', manage_apache_vhost => true, apache_vhost_name => $icinga2_web_fqdn, ido_db => 'mysql', ido_db_host => $icinga2_db_ip, ido_db_name => 'icinga2_data', ido_db_user => 'icinga2', ido_db_pass => $icinga2_ido_pass, web_db => 'mysql', web_db_name => 'icinga2_web', web_db_host => $icinga2_db_ip, web_db_user => 'icinga2_web', web_db_pass => $icinga2_webdb_pass, web_db_port => '3306', } }
  • 11.
    Icinga Camp Amsterdam2016 Setting up an icinga native client class profile::icinga::client { include ::icinga2 include ::icinga2::feature::command class { '::icinga2::feature::api': accept_commands => true, accept_config => true, manage_zone => false, } # icinga2::pki::puppet class needs to be declared # after the icinga2::feature::api class in order # to avoid resource duplication contain ::icinga2::pki::puppet }
  • 12.
    Icinga Camp Amsterdam2016 Exported resources PuppetDB Puppet Master ICINGAWEB01 @@icinga2::object::host { $::fqdn: ipv4_address => $::ipaddress, } Icinga2::Object::Host <<| |>> 1. Puppet agent run 6. Send to node 5. Retrieve from PuppetDB3. Store in PuppetDB 2. Export to PM 4. Collect on ICINGA
  • 13.
    Icinga Camp Amsterdam2016 Exported resources PuppetDB Puppet Master ICINGAWEB01 @@icinga2::object::host { $::fqdn: ipv4_address => $::ipaddress, } Icinga2::Object::Host <<| |>> 1. Puppet agent run 6. Send to node 5. Retrieve from PuppetDB3. Store in PuppetDB 2. Export to PM 4. Collect on ICINGA
  • 14.
    Icinga Camp Amsterdam2016 Exported resources PuppetDB Puppet Master ICINGAWEB01 @@icinga2::object::host { $::fqdn: ipv4_address => $::ipaddress, } Icinga2::Object::Host <<| |>> 1. Puppet agent run 6. Send to node 5. Retrieve from PuppetDB3. Store in PuppetDB 2. Export to PM 4. Collect on ICINGA
  • 15.
    Icinga Camp Amsterdam2016 Exported resources PuppetDB Puppet Master ICINGAWEB01 @@icinga2::object::host { $::fqdn: ipv4_address => $::ipaddress, } Icinga2::Object::Host <<| |>> 1. Puppet agent run 6. Send to node 5. Retrieve from PuppetDB3. Store in PuppetDB 2. Export to PM 4. Collect on ICINGA
  • 16.
    Icinga Camp Amsterdam2016 Exported resources PuppetDB Puppet Master ICINGAWEB01 @@icinga2::object::host { $::fqdn: ipv4_address => $::ipaddress, } Icinga2::Object::Host <<| |>> 1. Puppet agent run 6. Send to node 5. Retrieve from PuppetDB3. Store in PuppetDB 2. Export to PM 4. Collect on ICINGA
  • 17.
    Icinga Camp Amsterdam2016 Exported resources PuppetDB Puppet Master ICINGAWEB01 @@icinga2::object::host { $::fqdn: ipv4_address => $::ipaddress, } Icinga2::Object::Host <<| |>> 1. Puppet agent run 6. Send to node 5. Retrieve from PuppetDB3. Store in PuppetDB 2. Export to PM 4. Collect on ICINGA
  • 18.
    Icinga Camp Amsterdam2016 Exported resources node ‘icinga.olindata.com’ { # Collect all exported host objects Icinga2::Object::Host <<| |>> # Collect all exported service objects Icinga2::Object::Service <<| |>> # Collect all exported zone objects Icinga2::Object::Zone <<| |>> } node ‘web01.olindata.com’ { @@icinga2::object::host { $::fqdn: ipv4_address => $::ipaddress, } }
  • 19.
    Icinga Camp Amsterdam2016 # This is a resource default Icinga2::Object::Apply_service { assign_where => 'host.address && host.vars.remote == true && host.vars.remote_endpoint', command_endpoint => 'host.vars.remote_endpoint', } icinga2::object::apply_service { 'user': check_command => 'users', } icinga2::object::apply_service { 'load': check_command => 'load', } icinga2::object::apply_service { 'process': check_command => 'procs', } Adding some initial things
  • 20.
    Icinga Camp Amsterdam2016 What’s next? ● create profiles for each type of application, eg profile:: icinga::apache, profile::icinga::gitlab, etc. ● check out https://github.com/Icinga/icinga-vagrant/ for nice vagrant examples (not so nice puppet though ;) ) ● add grafana, business process monitoring, etc. ● create users, usergroups, hostgroups, etc.
  • 21.
    Icinga Camp Amsterdam2016 Icinga Training NL ● Official icinga training in the Netherlands on July 18th-21st ● Tickets only 1350 EUR(!) http://olindata.com/training/netherlands/amsterdam/icinga- fundamentals-training-amsterdam-july-2016
  • 22.
    Icinga Camp Amsterdam2016 Questions? @walterheck / @olindata http://www.olindata.com walterheck@olindata.com http://github.com/olindata
  • 23.
    Icinga Camp Amsterdam2016 We’re hiring! NL based consultants jobs@olindata.com