Puppet and the
HashiCorp Suite
Bram Vogelaar
Amsterdam HashiCorp User Group
~$ whoami~$ whoami
● I used to be a Molecular Biologist,I used to be a Molecular Biologist,
● Then became a Dev,Then became a Dev,
● Now an Ops.Now an Ops.
● Open Source Consultant @Open Source Consultant @inuits.euinuits.eu
PuppetPuppet
● Open Source configuration management toolOpen Source configuration management tool
since 2005since 2005
● Desired State DSL on top of rubyDesired State DSL on top of ruby
● Client – Server architectureClient – Server architecture
● Runs on pretty much any OSRuns on pretty much any OS
● Combines node info (Facter), node configCombines node info (Facter), node config
(Hiera), node model (DSL) into a catalogue that(Hiera), node model (DSL) into a catalogue that
is applies at regular intervalsis applies at regular intervals
Puppet WorkflowPuppet Workflow
PackerPacker
● Open Source tool to make OS imagesOpen Source tool to make OS images
● Supports Cloud Providers, Docker, Vbox, …Supports Cloud Providers, Docker, Vbox, …
(builders)(builders)
● Has hooks to provision the base imagesHas hooks to provision the base images
(provisioners)(provisioners)
● Create artifacts (Post-Processors)Create artifacts (Post-Processors)
PackerPacker
{{
"builders": ["builders": [
{{
"type": "azure-arm","type": "azure-arm",
......
}}
],],
"provisioners": ["provisioners": [
{{
"scripts": ["scripts": [
"scripts/common/consul_install.sh","scripts/common/consul_install.sh",
"scripts/common/puppet_install.sh""scripts/common/puppet_install.sh"
],],
"type": "shell","type": "shell",
}}
]]
}}
$ packer build template.json$ packer build template.json
Packer & PuppetPacker & Puppet
https://github.com/petems/puppet-install-shellhttps://github.com/petems/puppet-install-shell
$ ./puppet_install.sh [-p] [-v version] …$ ./puppet_install.sh [-p] [-v version] …
VagrantVagrant
● Open Source tool to bootstrap local vmsOpen Source tool to bootstrap local vms
– Build by packer!Build by packer!
● Supports many vm Providers, Docker, Vbox, …Supports many vm Providers, Docker, Vbox, …
● Has hooks to provision the base imagesHas hooks to provision the base images
(provisioners), Puppet, Chef, Ansible, Bash(provisioners), Puppet, Chef, Ansible, Bash
VagrantfileVagrantfile
# -*- mode: ruby -*-# -*- mode: ruby -*-
# vi: set ft=ruby :# vi: set ft=ruby :
Vagrant.configure("2") do |config|Vagrant.configure("2") do |config|
config.vm.box = "base"config.vm.box = "base"
# config.vm.box_check_update = false# config.vm.box_check_update = false
# config.vm.network "forwarded_port", guest: 80, host: 8080# config.vm.network "forwarded_port", guest: 80, host: 8080
# config.vm.network "private_network", ip: "192.168.33.10"# config.vm.network "private_network", ip: "192.168.33.10"
# config.vm.network "public_network"# config.vm.network "public_network"
# config.vm.synced_folder "../data", "/vagrant_data"# config.vm.synced_folder "../data", "/vagrant_data"
# config.vm.provider "virtualbox" do |vb|# config.vm.provider "virtualbox" do |vb|
# vb.gui = true# vb.gui = true
# vb.memory = "1024"# vb.memory = "1024"
# end# end
# config.vm.provision "shell", inline: <<-SHELL# config.vm.provision "shell", inline: <<-SHELL
# apt-get update# apt-get update
# apt-get install -y apache2# apt-get install -y apache2
# SHELL# SHELL
endend
Vagrant & PuppetVagrant & Puppet
https://github.com/petems/vagrant-puppet-installhttps://github.com/petems/vagrant-puppet-install
$ vagrant plugin install vagrant-puppet-install$ vagrant plugin install vagrant-puppet-install
Vagrant & PuppetVagrant & Puppet
cat Vagrantfilecat Vagrantfile
casecase nodenode[["provision_type""provision_type"]]
whenwhen 'masterless''masterless'
srvsrv.vm.synced_folder.vm.synced_folder ""#{environment}#{environment}/hieradata"/hieradata",,
"/etc/puppetlabs/code/environments/"/etc/puppetlabs/code/environments/#{environment}#{environment}/hieradata"/hieradata"
srvsrv.vm.provision :puppet.vm.provision :puppet dodo ||puppetpuppet||
puppetpuppet.environment_path =.environment_path = ".""."
puppetpuppet.hiera_config_path =.hiera_config_path = ""#{environment}#{environment}/hiera.yaml"/hiera.yaml"
endend
$ vagrant (up,halt,destroy)$ vagrant (up,halt,destroy)
Vagrant & PuppetVagrant & Puppet
cat Vagrantfilecat Vagrantfile
casecase nodenode[["provision_type""provision_type"]]
whenwhen 'puppet_agent''puppet_agent'
srvsrv.vm.provision.vm.provision "puppet_server" do"puppet_server" do ||puppetpuppet||
puppetpuppet.options =.options = "-t --environment"-t --environment #{environment}#{environment}""
puppetpuppet.puppet_server =.puppet_server = "puppetmaster."puppetmaster.#{environment}#{environment}.vagrant".vagrant"
endend
srv.trigger.after :destroy do |trigger|srv.trigger.after :destroy do |trigger|
trigger.name = "Cleaning puppet certificate"trigger.name = "Cleaning puppet certificate"
trigger.run =trigger.run = {inline: "vagrant ssh puppetmaster -c 'sudo{inline: "vagrant ssh puppetmaster -c 'sudo
/opt/puppetlabs/bin/puppetserver ca clean --certname #{node["hostname"]}'"}/opt/puppetlabs/bin/puppetserver ca clean --certname #{node["hostname"]}'"}
endend
$ vagrant (up,halt,destroy)$ vagrant (up,halt,destroy)
TerraformTerraform
● Open Source Automation ToolOpen Source Automation Tool
● ““cloud” orientedcloud” oriented
● Cloud are API’sCloud are API’s
● API’s orientedAPI’s oriented
Terraform is an open source automation toolTerraform is an open source automation tool
which can deal with any kind of CRUD api’s –which can deal with any kind of CRUD api’s –
including major cloud providersincluding major cloud providers
HCLHCL
● Hashicorp Configuration LanguageHashicorp Configuration Language
● Yet another cfgmgmt DSLYet another cfgmgmt DSL
● Desired stateDesired state
● Used by multiple hashicorp tools but also 3rdUsed by multiple hashicorp tools but also 3rd
party toolsparty tools
The Terraform modelThe Terraform model
● You model your infrastructureYou model your infrastructure
● You make a planYou make a plan
● If ok, you apply that planIf ok, you apply that plan
● Current state is saved for future changesCurrent state is saved for future changes
$ terraform (fmt, validate, plan, apply)$ terraform (fmt, validate, plan, apply)
Lets magic a nodeLets magic a node
resourceresource "azurerm_virtual_machine" "puppetmaster" {"azurerm_virtual_machine" "puppetmaster" {
name = "vm-${var.node_name}"name = "vm-${var.node_name}"
location = "${var.azure_region_name}"location = "${var.azure_region_name}"
resource_group_name = "${var.resource_group}"resource_group_name = "${var.resource_group}"
network_interface_ids = ["$network_interface_ids = ["$
{azurerm_network_interface.puppetmaster.id}"]{azurerm_network_interface.puppetmaster.id}"]
vm_size = "${var.vm_size}"vm_size = "${var.vm_size}"
storage_image_referencestorage_image_reference {{
id = "${var.packerimage_id}" # ← your packer image idid = "${var.packerimage_id}" # ← your packer image id
}}
storage_os_diskstorage_os_disk {{
......
}}
os_profileos_profile {{
......
}}
}}
Lets magic a nodeLets magic a node
resourceresource "azurerm_virtual_machine" "puppetmaster" {"azurerm_virtual_machine" "puppetmaster" {
......
os_profileos_profile {{
custom_data = "${data.template_file.bootstrap_sh.rendered}"custom_data = "${data.template_file.bootstrap_sh.rendered}"
}}
}}
output "private_ip" {output "private_ip" {
value = "${azurerm_network_interface.network_interface.private_ip_address}"value = "${azurerm_network_interface.network_interface.private_ip_address}"
}}
Bootstrap yourBootstrap your
puppetmasterpuppetmaster
#!/usr/bin/env bash#!/usr/bin/env bash
apt-get install puppetserverapt-get install puppetserver
puppet config set --section agent environment ${environment}puppet config set --section agent environment ${environment}
systemctl start puppetserversystemctl start puppetserver
Move it all to a moduleMove it all to a module
module "web" {module "web" {
source = "../modules/azure/instance"source = "../modules/azure/instance"
domain = "${var.domain}"domain = "${var.domain}"
environment = "${var.environment}"environment = "${var.environment}"
node_name = "web"node_name = "web"
private_subnet = "${module.inuits_play_bootstrap.subnet_id}"private_subnet = "${module.inuits_play_bootstrap.subnet_id}"
puppet_master = "${var.puppet_master}"puppet_master = "${var.puppet_master}"
}}
Its a DNS problemIts a DNS problem
ConsulConsul
● Open Source Service Discovery ToolOpen Source Service Discovery Tool
● Build-in KV storeBuild-in KV store
● Service Mesh toolService Mesh tool
ConsulConsul
classclass { '::consul':{ '::consul':
config_hash => $config,config_hash => $config,
extra_options => $options,extra_options => $options,
version => $version,version => $version,
}}
::consul::service { 'puppetmaster':::consul::service { 'puppetmaster':
port => 8140,port => 8140,
}}
::consul::check { 'puppetmaster_tcp':::consul::check { 'puppetmaster_tcp':
interval => '10s',interval => '10s',
tcp => 'localhost:8140',tcp => 'localhost:8140',
notes => 'Puppetmasters listen on port 8140',notes => 'Puppetmasters listen on port 8140',
service_id => 'puppetmaster',service_id => 'puppetmaster',
}}
https://forge.puppet.com/KyleAnderson/consulhttps://forge.puppet.com/KyleAnderson/consul
Consul~Icinga(2) ExitConsul~Icinga(2) Exit
CodesCodes
::consul::check { 'puppetmaster':::consul::check { 'puppetmaster':
interval => '10s',interval => '10s',
script =>script => '/usr/lib64/nagios/plugins/puppetmaster''/usr/lib64/nagios/plugins/puppetmaster',,
notes => 'Puppetmasters listen on port 8140',notes => 'Puppetmasters listen on port 8140',
service_id => 'puppetmaster',service_id => 'puppetmaster',
}}
ConsulConsul
dig @127.0.0.1 -p 8600 puppetmaster.service.consul ANY
Geolocation StuffGeolocation Stuff
consul_prepared_query { 'consul_prepared_query { 'puppetmasterpuppetmaster':':
ensure => 'present',ensure => 'present',
service_name => 'service_name => 'puppetmasterpuppetmaster',',
service_failover_n => 1,service_failover_n => 1,
service_failover_dcs => [ 'failover', 'dr' ],service_failover_dcs => [ 'failover', 'dr' ],
service_only_passing => true,service_only_passing => true,
service_tags => [ 'tag1', 'tag2' ],service_tags => [ 'tag1', 'tag2' ],
ttl => 10,ttl => 10,
}}
dig @127.0.0.1 -p 8600 puppetmaster.query.consul ANYdig @127.0.0.1 -p 8600 puppetmaster.query.consul ANY
X509 is hardX509 is hard
#!/usr/bin/env bash#!/usr/bin/env bash
apt-get install puppetserverapt-get install puppetserver
puppet config set --section agent environment ${environment}puppet config set --section agent environment ${environment}
puppet config set --section master dns_alt_names puppet config set --section master dns_alt_names 
puppet,puppetmaster.service.consul,puppetmaster.query.consulpuppet,puppetmaster.service.consul,puppetmaster.query.consul
systemctl start puppetserversystemctl start puppetserver
What about thatWhat about that
chicken?chicken?
#!/usr/bin/env bash#!/usr/bin/env bash
consul agent -retry-join "provider=azure consul agent -retry-join "provider=azure 
tag_name=consul tag_name=consul 
tenant_id=${tid} tenant_id=${tid} 
client_id=${cid} client_id=${cid} 
subscription_id=$cid} subscription_id=$cid} 
secret_access_key=${ro_key}"secret_access_key=${ro_key}"
puppet config set --section agent environment ${environment}puppet config set --section agent environment ${environment}
puppet config set --section agent server puppet.service.consulpuppet config set --section agent server puppet.service.consul
/opt/puppetlabs/bin/puppet agent -t/opt/puppetlabs/bin/puppet agent -t
Fast ExportedFast Exported
ResourcesResources
::consul::watch { 'detect_backend_changes':::consul::watch { 'detect_backend_changes':
type => 'service',type => 'service',
handler => '/usr/bin/runpuppet.sh',handler => '/usr/bin/runpuppet.sh',
service => 'node_exporter',service => 'node_exporter',
passingonly => true,passingonly => true,
require => File['/usr/bin/runpuppet.sh'],require => File['/usr/bin/runpuppet.sh'],
}}
VaultVault
● Open Source tool to do secrets managementOpen Source tool to do secrets management
● Secure, store and tightly control access toSecure, store and tightly control access to
tokens, passwords, certificates, encryption keystokens, passwords, certificates, encryption keys
for protecting secrets and other sensitive datafor protecting secrets and other sensitive data
using a UI, CLI, or HTTP API.using a UI, CLI, or HTTP API.
● Certificate managementCertificate management
● Password rotatationPassword rotatation
Pesky PasswordsPesky Passwords
class profiles::security::vault (class profiles::security::vault (
$config_dir = '/etc/vault.d',$config_dir = '/etc/vault.d',
$enable_ui = true,$enable_ui = true,
Variant[Hash, Array[Hash]] $listener = {Variant[Hash, Array[Hash]] $listener = {
'tcp' => {'tcp' => {
'address' => '127.0.0.1:8200','address' => '127.0.0.1:8200',
'cluster_address' => '127.0.0.1:8201','cluster_address' => '127.0.0.1:8201',
'tls_disable' => 1,'tls_disable' => 1,
},},
},},
Hash $storage = { 'consul' => { 'address' => '127.0.0.1:8500', 'path' => 'vault/' }},Hash $storage = { 'consul' => { 'address' => '127.0.0.1:8500', 'path' => 'vault/' }},
String $version = '1.1.3',String $version = '1.1.3',
){){
class {'vault':class {'vault':
config_dir => $config_dir,config_dir => $config_dir,
enable_ui => $enable_ui,enable_ui => $enable_ui,
listener => $listener,listener => $listener,
storage => $storage,storage => $storage,
version => $version,version => $version,
}}
https://forge.puppet.com/jsok/vaulthttps://forge.puppet.com/jsok/vault
Pesky PasswordsPesky Passwords
classclass { '::vault':{ '::vault':
install_method => 'repo',install_method => 'repo',
}}
$ vault unseal$ vault unseal
$ vault write kv/my-secret value="s3c(eT"$ vault write kv/my-secret value="s3c(eT"
$ vault read kv/mysecret$ vault read kv/mysecret
Key ValueKey Value
--- -------- -----
refresh_interval 768hrefresh_interval 768h
mysecret s3c(eTmysecret s3c(eT
https://forge.puppet.com/jsok/vaulthttps://forge.puppet.com/jsok/vault
Pesky PasswordsPesky Passwords
$ vault unseal$ vault unseal
$ vault write kv/my-secret value="s3c(eT"$ vault write kv/my-secret value="s3c(eT"
$ vault read kv/mysecret$ vault read kv/mysecret
Key ValueKey Value
--- -------- -----
refresh_interval 768hrefresh_interval 768h
mysecret s3c(eTmysecret s3c(eT
HieraHiera
cat hiera.yamlcat hiera.yaml
------
version: 5version: 5
hierarchy:hierarchy:
- name: "Hiera-vault lookup"- name: "Hiera-vault lookup"
lookup_key: hiera_vaultlookup_key: hiera_vault
options:options:
confine_to_keys:confine_to_keys:
- '^sensitive_.*'- '^sensitive_.*'
- '^password.*'- '^password.*'
address: http://active.vault.service.consul:8200address: http://active.vault.service.consul:8200
mounts:mounts:
generic:generic:
- secret/puppet/%{::trusted.certname}/- secret/puppet/%{::trusted.certname}/
- secret/puppet/common/- secret/puppet/common/
https://github.com/petems/hiera_backend_vaulthttps://github.com/petems/hiera_backend_vault
Puppet5Puppet5
classclass profiles::database::mysql (profiles::database::mysql (
Sensitive $root_password,Sensitive $root_password,
) {) {
classclass { '::mysql::server':{ '::mysql::server':
root_password => $root_password.unwrap,root_password => $root_password.unwrap,
}}
notice($root_password)notice($root_password)
}}
Notice: Scope(Class[main]): Sensitive [value redacted]Notice: Scope(Class[main]): Sensitive [value redacted]
Puppet 6Puppet 6
$root_password = Deferred($root_password = Deferred(
'vault_lookup::lookup','vault_lookup::lookup',
["password/mysql"],["password/mysql"],
''https://active.vault.service.consul:8200https://active.vault.service.consul:8200',',
))
classclass { '::mysql::server':{ '::mysql::server':
root_password => $root_password,root_password => $root_password,
}}
NomadNomad
● Open Source tool to do dynamic workloadOpen Source tool to do dynamic workload
schedulingscheduling
● Batch, containerized, and non-containerizedBatch, containerized, and non-containerized
applications.applications.
● Has native Consul and Vault integrations.Has native Consul and Vault integrations.
NomadNomad
class { '::nomad':class { '::nomad':
config_hash = {config_hash = {
'client' => {'client' => {
'enabled' => true,'enabled' => true,
},},
'consul' => {'consul' => {
'address' => '127.0.0.1:8500','address' => '127.0.0.1:8500',
},},
'datacenter' => 'services','datacenter' => 'services',
'data_dir' => '/var/lib/nomad','data_dir' => '/var/lib/nomad',
'server' => {'server' => {
'enabled' => true'enabled' => true
'bootstrap_expect' => 3,'bootstrap_expect' => 3,
},},
'vault' => {'vault' => {
'enabled' => true,'enabled' => true,
'address' => ''address' => 'https://active.vault.service.consul:8200https://active.vault.service.consul:8200',',
''create_from_rolecreate_from_role' => '' => 'nomad-clusternomad-cluster',',
''tokentoken' => '' => 's.krHYYcd8PRzpSBO59AE6sawOs.krHYYcd8PRzpSBO59AE6sawO',',
},},
}}
}}
https://forge.puppet.com/dudemcbacon/nomadhttps://forge.puppet.com/dudemcbacon/nomad
https://github.com/attachmentgenie/puppet-nomadhttps://github.com/attachmentgenie/puppet-nomad
ContactContact
Bram VogelaarBram Vogelaar
bram.vogelaar@inuits.eubram.vogelaar@inuits.eu
@attachmentgenie@attachmentgenie
https://www.slideshare.net/attachmentgeniehttps://www.slideshare.net/attachmentgenie
https://github.com/attachmentgenie/vagrant-schedulerhttps://github.com/attachmentgenie/vagrant-scheduler
Inuits BEInuits BE
Essensteenweg 31Essensteenweg 31
2930 Brasschaat2930 Brasschaat
BelgiumBelgium
Inuits NLInuits NL
Maashaven Zuidzijde 2Maashaven Zuidzijde 2
3081 AE Rotterdam3081 AE Rotterdam
NetherlandsNetherlands
BoltBolt
● Open Source tool to do task orchestrationOpen Source tool to do task orchestration
● Aimed at fire and forget 2nd day operationsAimed at fire and forget 2nd day operations
Bolt && TerraformBolt && Terraform
~/terraform_provision/inventory.yaml~/terraform_provision/inventory.yaml
groups:groups:
- name: terraform- name: terraform
nodes: [] # This will be populated by the Bolt plannodes: [] # This will be populated by the Bolt plan
config:config:
transport: sshtransport: ssh
ssh:ssh:
private-key: ~/.ssh/id_rsa-phraselessprivate-key: ~/.ssh/id_rsa-phraseless
user: ubuntuuser: ubuntu
host-key-check: falsehost-key-check: false
Bolt && TerraformBolt && Terraform
~/terraform_provision/plans/init.pp~/terraform_provision/plans/init.pp
plan terraform_provision(String $tf_path) {plan terraform_provision(String $tf_path) {
$localhost = get_targets('localhost')$localhost = get_targets('localhost')
$ip_string = run_command("cd ${$tf_path} && terraform output interal_ips"$ip_string = run_command("cd ${$tf_path} && terraform output interal_ips"
$localhost).map |$r| { $r['stdout'] }$localhost).map |$r| { $r['stdout'] }
$ips = Array($ip_string).map |$ip| { $ip.strip }$ips = Array($ip_string).map |$ip| { $ip.strip }
# Turn IPs into Bolt targets, and add to inventory# Turn IPs into Bolt targets, and add to inventory
$targets = $ips.map |$ip| {$targets = $ips.map |$ip| {
Target.new("${$ip}").add_to_group('terraform')Target.new("${$ip}").add_to_group('terraform')
}}
# Deploy website# Deploy website
apply_prep($targets)apply_prep($targets)
apply($targets, _run_as => 'root') {}apply($targets, _run_as => 'root') {}
return $ipsreturn $ips
}}
Bolt && TerraformBolt && Terraform
$ bolt plan run terraform_provision $ bolt plan run terraform_provision 
-i ~/terraform_provision/inventory.yaml -i ~/terraform_provision/inventory.yaml 
tf_path=~/terraform-playgroundtf_path=~/terraform-playground
Icinga ProviderIcinga Provider
providerprovider "icinga2" {"icinga2" {
api_url = "https://icinga.alerting.vagrant:5665/v1"api_url = "https://icinga.alerting.vagrant:5665/v1"
api_user = "root"api_user = "root"
api_password = "icinga"api_password = "icinga"
insecure_skip_tls_verify =insecure_skip_tls_verify = truetrue
}}
Icinga HostIcinga Host
resource "aws_instance" "node1" {resource "aws_instance" "node1" {
ami = "${var.ami_id}"ami = "${var.ami_id}"
instance_type = "${var.ami_size}"instance_type = "${var.ami_size}"
key_name = "${var.key_pair_name}"key_name = "${var.key_pair_name}"
subnet_id = "${var.private_subnet}"subnet_id = "${var.private_subnet}"
vpc_security_group_ids = ["${var.security_group_id}"]vpc_security_group_ids = ["${var.security_group_id}"]
}}
resourceresource "icinga2_host" "node1" {"icinga2_host" "node1" {
hostname = "hostname = "node1"node1"
address = "address = "${aws_instance.instance.private_ip}${aws_instance.instance.private_ip}""
check_command = "hostalive"check_command = "hostalive"
varsvars {{
os = "linux"os = "linux"
}}
}}
Check CommandCheck Command
resourceresource "icinga2_checkcommand" "apache_status" {"icinga2_checkcommand" "apache_status" {
name = "apache_status"name = "apache_status"
templates = ["apache-status", "plugin-check-command",templates = ["apache-status", "plugin-check-command",
"plugin-check-command", "ipv4-or-ipv6"]"plugin-check-command", "ipv4-or-ipv6"]
command =command =
"/usr/lib64/nagios/plugins/check_apache_status.pl""/usr/lib64/nagios/plugins/check_apache_status.pl"
arguments = {arguments = {
"-H" = "$apache_status_address$""-H" = "$apache_status_address$"
"-c" = "$apache_status_critical$""-c" = "$apache_status_critical$"
"-p" = "$apache_status_port$""-p" = "$apache_status_port$"
}}
}}
resourceresource "icinga2_service" "my-service" {"icinga2_service" "my-service" {
name = "ssh"name = "ssh"
hostname = "c1-mysql-1"hostname = "c1-mysql-1"
check_command = "ssh"check_command = "ssh"
}}
NotificationsNotifications
resource "icinga2_user" "user" {resource "icinga2_user" "user" {
name = "terraform"name = "terraform"
email = "terraform@dev.null"email = "terraform@dev.null"
}}
resourceresource "icinga2_notification" "host-notification" {"icinga2_notification" "host-notification" {
hostname = "docker-icinga2"hostname = "docker-icinga2"
command = "mail-host-notification"command = "mail-host-notification"
users = ["user"]users = ["user"]
}}
resourceresource "icinga2_notification" "ping-service-notification" {"icinga2_notification" "ping-service-notification" {
hostname = "docker-icinga2"hostname = "docker-icinga2"
command = "mail-service-notification"command = "mail-service-notification"
users = ["user"]users = ["user"]
servicename = "ping"servicename = "ping"
}}
Other ResourcesOther Resources
resourceresource "icinga2_checkcommand" "apache_status" {"icinga2_checkcommand" "apache_status" {
name = "apache_status"name = "apache_status"
templates = ["apache-status", "plugin-check-command",templates = ["apache-status", "plugin-check-command",
"plugin-check-command", "ipv4-or-ipv6"]"plugin-check-command", "ipv4-or-ipv6"]
command =command =
"/usr/lib64/nagios/plugins/check_apache_status.pl""/usr/lib64/nagios/plugins/check_apache_status.pl"
arguments = {arguments = {
"-H" = "$apache_status_address$""-H" = "$apache_status_address$"
"-c" = "$apache_status_critical$""-c" = "$apache_status_critical$"
"-p" = "$apache_status_port$""-p" = "$apache_status_port$"
}}
}}
Icinga Director & ConsulIcinga Director & Consul
github.com/attachmentgenie/icingaweb2-module-consul.gitgithub.com/attachmentgenie/icingaweb2-module-consul.git
Query ConsulQuery Consul
use SensioLabsConsulServiceFactory;use SensioLabsConsulServiceFactory;
public functionpublic function fetchData()fetchData()
{{
$sf = new ServiceFactory($sf = new ServiceFactory(
arrayarray('base_uri' => $this->getSetting('consul_url'))('base_uri' => $this->getSetting('consul_url'))
););
$agent = $sf->get('agent');$agent = $sf->get('agent');
return json_decode($agent->members()->getBody());return json_decode($agent->members()->getBody());
}}
public functionpublic function listColumns()listColumns()
{{
return array_keys((array) current($this->fetchData()));return array_keys((array) current($this->fetchData()));
}}
Adding a sync sourceAdding a sync source
Adding a sync ruleAdding a sync rule
Add sync propertiesAdd sync properties
Fast FeedbackFast Feedback
file { 'Director Sync and Deploy':file { 'Director Sync and Deploy':
path => '/usr/local/bin/director_sync_and_deploy.sh',path => '/usr/local/bin/director_sync_and_deploy.sh',
source =>source =>
'puppet:///modules/profiles/monitoring/icinga2/director_sync_'puppet:///modules/profiles/monitoring/icinga2/director_sync_
and_deploy.sh',and_deploy.sh',
owner => 'consul',owner => 'consul',
group => 'consul',group => 'consul',
mode => '0744',mode => '0744',
}}
::consul::watch { 'director_import':::consul::watch { 'director_import':
type => 'service',type => 'service',
handler =>handler =>
'/usr/local/bin/director_sync_and_deploy.sh','/usr/local/bin/director_sync_and_deploy.sh',
service => 'node_exporter',service => 'node_exporter',
passingonly =>passingonly => truetrue,,
require => File['Director Sync and Deploy'],require => File['Director Sync and Deploy'],
}}
director_sync_and_deploy.shdirector_sync_and_deploy.sh
#!/usr/bin/env bash#!/usr/bin/env bash
setset -x-x
icingacliicingacli director importsource run --id 1director importsource run --id 1
icingacliicingacli director syncrule run --id 1director syncrule run --id 1
icingacliicingacli director config deploydirector config deploy

Puppet and the HashiCorp Suite

  • 1.
    Puppet and the HashiCorpSuite Bram Vogelaar Amsterdam HashiCorp User Group
  • 2.
    ~$ whoami~$ whoami ●I used to be a Molecular Biologist,I used to be a Molecular Biologist, ● Then became a Dev,Then became a Dev, ● Now an Ops.Now an Ops. ● Open Source Consultant @Open Source Consultant @inuits.euinuits.eu
  • 4.
    PuppetPuppet ● Open Sourceconfiguration management toolOpen Source configuration management tool since 2005since 2005 ● Desired State DSL on top of rubyDesired State DSL on top of ruby ● Client – Server architectureClient – Server architecture ● Runs on pretty much any OSRuns on pretty much any OS ● Combines node info (Facter), node configCombines node info (Facter), node config (Hiera), node model (DSL) into a catalogue that(Hiera), node model (DSL) into a catalogue that is applies at regular intervalsis applies at regular intervals
  • 5.
  • 6.
    PackerPacker ● Open Sourcetool to make OS imagesOpen Source tool to make OS images ● Supports Cloud Providers, Docker, Vbox, …Supports Cloud Providers, Docker, Vbox, … (builders)(builders) ● Has hooks to provision the base imagesHas hooks to provision the base images (provisioners)(provisioners) ● Create artifacts (Post-Processors)Create artifacts (Post-Processors)
  • 7.
    PackerPacker {{ "builders": ["builders": [ {{ "type":"azure-arm","type": "azure-arm", ...... }} ],], "provisioners": ["provisioners": [ {{ "scripts": ["scripts": [ "scripts/common/consul_install.sh","scripts/common/consul_install.sh", "scripts/common/puppet_install.sh""scripts/common/puppet_install.sh" ],], "type": "shell","type": "shell", }} ]] }} $ packer build template.json$ packer build template.json
  • 8.
    Packer & PuppetPacker& Puppet https://github.com/petems/puppet-install-shellhttps://github.com/petems/puppet-install-shell $ ./puppet_install.sh [-p] [-v version] …$ ./puppet_install.sh [-p] [-v version] …
  • 9.
    VagrantVagrant ● Open Sourcetool to bootstrap local vmsOpen Source tool to bootstrap local vms – Build by packer!Build by packer! ● Supports many vm Providers, Docker, Vbox, …Supports many vm Providers, Docker, Vbox, … ● Has hooks to provision the base imagesHas hooks to provision the base images (provisioners), Puppet, Chef, Ansible, Bash(provisioners), Puppet, Chef, Ansible, Bash
  • 10.
    VagrantfileVagrantfile # -*- mode:ruby -*-# -*- mode: ruby -*- # vi: set ft=ruby :# vi: set ft=ruby : Vagrant.configure("2") do |config|Vagrant.configure("2") do |config| config.vm.box = "base"config.vm.box = "base" # config.vm.box_check_update = false# config.vm.box_check_update = false # config.vm.network "forwarded_port", guest: 80, host: 8080# config.vm.network "forwarded_port", guest: 80, host: 8080 # config.vm.network "private_network", ip: "192.168.33.10"# config.vm.network "private_network", ip: "192.168.33.10" # config.vm.network "public_network"# config.vm.network "public_network" # config.vm.synced_folder "../data", "/vagrant_data"# config.vm.synced_folder "../data", "/vagrant_data" # config.vm.provider "virtualbox" do |vb|# config.vm.provider "virtualbox" do |vb| # vb.gui = true# vb.gui = true # vb.memory = "1024"# vb.memory = "1024" # end# end # config.vm.provision "shell", inline: <<-SHELL# config.vm.provision "shell", inline: <<-SHELL # apt-get update# apt-get update # apt-get install -y apache2# apt-get install -y apache2 # SHELL# SHELL endend
  • 11.
    Vagrant & PuppetVagrant& Puppet https://github.com/petems/vagrant-puppet-installhttps://github.com/petems/vagrant-puppet-install $ vagrant plugin install vagrant-puppet-install$ vagrant plugin install vagrant-puppet-install
  • 12.
    Vagrant & PuppetVagrant& Puppet cat Vagrantfilecat Vagrantfile casecase nodenode[["provision_type""provision_type"]] whenwhen 'masterless''masterless' srvsrv.vm.synced_folder.vm.synced_folder ""#{environment}#{environment}/hieradata"/hieradata",, "/etc/puppetlabs/code/environments/"/etc/puppetlabs/code/environments/#{environment}#{environment}/hieradata"/hieradata" srvsrv.vm.provision :puppet.vm.provision :puppet dodo ||puppetpuppet|| puppetpuppet.environment_path =.environment_path = ".""." puppetpuppet.hiera_config_path =.hiera_config_path = ""#{environment}#{environment}/hiera.yaml"/hiera.yaml" endend $ vagrant (up,halt,destroy)$ vagrant (up,halt,destroy)
  • 13.
    Vagrant & PuppetVagrant& Puppet cat Vagrantfilecat Vagrantfile casecase nodenode[["provision_type""provision_type"]] whenwhen 'puppet_agent''puppet_agent' srvsrv.vm.provision.vm.provision "puppet_server" do"puppet_server" do ||puppetpuppet|| puppetpuppet.options =.options = "-t --environment"-t --environment #{environment}#{environment}"" puppetpuppet.puppet_server =.puppet_server = "puppetmaster."puppetmaster.#{environment}#{environment}.vagrant".vagrant" endend srv.trigger.after :destroy do |trigger|srv.trigger.after :destroy do |trigger| trigger.name = "Cleaning puppet certificate"trigger.name = "Cleaning puppet certificate" trigger.run =trigger.run = {inline: "vagrant ssh puppetmaster -c 'sudo{inline: "vagrant ssh puppetmaster -c 'sudo /opt/puppetlabs/bin/puppetserver ca clean --certname #{node["hostname"]}'"}/opt/puppetlabs/bin/puppetserver ca clean --certname #{node["hostname"]}'"} endend $ vagrant (up,halt,destroy)$ vagrant (up,halt,destroy)
  • 14.
    TerraformTerraform ● Open SourceAutomation ToolOpen Source Automation Tool ● ““cloud” orientedcloud” oriented ● Cloud are API’sCloud are API’s ● API’s orientedAPI’s oriented Terraform is an open source automation toolTerraform is an open source automation tool which can deal with any kind of CRUD api’s –which can deal with any kind of CRUD api’s – including major cloud providersincluding major cloud providers
  • 15.
    HCLHCL ● Hashicorp ConfigurationLanguageHashicorp Configuration Language ● Yet another cfgmgmt DSLYet another cfgmgmt DSL ● Desired stateDesired state ● Used by multiple hashicorp tools but also 3rdUsed by multiple hashicorp tools but also 3rd party toolsparty tools
  • 16.
    The Terraform modelTheTerraform model ● You model your infrastructureYou model your infrastructure ● You make a planYou make a plan ● If ok, you apply that planIf ok, you apply that plan ● Current state is saved for future changesCurrent state is saved for future changes $ terraform (fmt, validate, plan, apply)$ terraform (fmt, validate, plan, apply)
  • 17.
    Lets magic anodeLets magic a node resourceresource "azurerm_virtual_machine" "puppetmaster" {"azurerm_virtual_machine" "puppetmaster" { name = "vm-${var.node_name}"name = "vm-${var.node_name}" location = "${var.azure_region_name}"location = "${var.azure_region_name}" resource_group_name = "${var.resource_group}"resource_group_name = "${var.resource_group}" network_interface_ids = ["$network_interface_ids = ["$ {azurerm_network_interface.puppetmaster.id}"]{azurerm_network_interface.puppetmaster.id}"] vm_size = "${var.vm_size}"vm_size = "${var.vm_size}" storage_image_referencestorage_image_reference {{ id = "${var.packerimage_id}" # ← your packer image idid = "${var.packerimage_id}" # ← your packer image id }} storage_os_diskstorage_os_disk {{ ...... }} os_profileos_profile {{ ...... }} }}
  • 18.
    Lets magic anodeLets magic a node resourceresource "azurerm_virtual_machine" "puppetmaster" {"azurerm_virtual_machine" "puppetmaster" { ...... os_profileos_profile {{ custom_data = "${data.template_file.bootstrap_sh.rendered}"custom_data = "${data.template_file.bootstrap_sh.rendered}" }} }} output "private_ip" {output "private_ip" { value = "${azurerm_network_interface.network_interface.private_ip_address}"value = "${azurerm_network_interface.network_interface.private_ip_address}" }}
  • 19.
    Bootstrap yourBootstrap your puppetmasterpuppetmaster #!/usr/bin/envbash#!/usr/bin/env bash apt-get install puppetserverapt-get install puppetserver puppet config set --section agent environment ${environment}puppet config set --section agent environment ${environment} systemctl start puppetserversystemctl start puppetserver
  • 20.
    Move it allto a moduleMove it all to a module module "web" {module "web" { source = "../modules/azure/instance"source = "../modules/azure/instance" domain = "${var.domain}"domain = "${var.domain}" environment = "${var.environment}"environment = "${var.environment}" node_name = "web"node_name = "web" private_subnet = "${module.inuits_play_bootstrap.subnet_id}"private_subnet = "${module.inuits_play_bootstrap.subnet_id}" puppet_master = "${var.puppet_master}"puppet_master = "${var.puppet_master}" }}
  • 21.
    Its a DNSproblemIts a DNS problem
  • 22.
    ConsulConsul ● Open SourceService Discovery ToolOpen Source Service Discovery Tool ● Build-in KV storeBuild-in KV store ● Service Mesh toolService Mesh tool
  • 23.
    ConsulConsul classclass { '::consul':{'::consul': config_hash => $config,config_hash => $config, extra_options => $options,extra_options => $options, version => $version,version => $version, }} ::consul::service { 'puppetmaster':::consul::service { 'puppetmaster': port => 8140,port => 8140, }} ::consul::check { 'puppetmaster_tcp':::consul::check { 'puppetmaster_tcp': interval => '10s',interval => '10s', tcp => 'localhost:8140',tcp => 'localhost:8140', notes => 'Puppetmasters listen on port 8140',notes => 'Puppetmasters listen on port 8140', service_id => 'puppetmaster',service_id => 'puppetmaster', }} https://forge.puppet.com/KyleAnderson/consulhttps://forge.puppet.com/KyleAnderson/consul
  • 24.
    Consul~Icinga(2) ExitConsul~Icinga(2) Exit CodesCodes ::consul::check{ 'puppetmaster':::consul::check { 'puppetmaster': interval => '10s',interval => '10s', script =>script => '/usr/lib64/nagios/plugins/puppetmaster''/usr/lib64/nagios/plugins/puppetmaster',, notes => 'Puppetmasters listen on port 8140',notes => 'Puppetmasters listen on port 8140', service_id => 'puppetmaster',service_id => 'puppetmaster', }}
  • 25.
    ConsulConsul dig @127.0.0.1 -p8600 puppetmaster.service.consul ANY
  • 26.
    Geolocation StuffGeolocation Stuff consul_prepared_query{ 'consul_prepared_query { 'puppetmasterpuppetmaster':': ensure => 'present',ensure => 'present', service_name => 'service_name => 'puppetmasterpuppetmaster',', service_failover_n => 1,service_failover_n => 1, service_failover_dcs => [ 'failover', 'dr' ],service_failover_dcs => [ 'failover', 'dr' ], service_only_passing => true,service_only_passing => true, service_tags => [ 'tag1', 'tag2' ],service_tags => [ 'tag1', 'tag2' ], ttl => 10,ttl => 10, }} dig @127.0.0.1 -p 8600 puppetmaster.query.consul ANYdig @127.0.0.1 -p 8600 puppetmaster.query.consul ANY
  • 27.
    X509 is hardX509is hard #!/usr/bin/env bash#!/usr/bin/env bash apt-get install puppetserverapt-get install puppetserver puppet config set --section agent environment ${environment}puppet config set --section agent environment ${environment} puppet config set --section master dns_alt_names puppet config set --section master dns_alt_names puppet,puppetmaster.service.consul,puppetmaster.query.consulpuppet,puppetmaster.service.consul,puppetmaster.query.consul systemctl start puppetserversystemctl start puppetserver
  • 28.
    What about thatWhatabout that chicken?chicken? #!/usr/bin/env bash#!/usr/bin/env bash consul agent -retry-join "provider=azure consul agent -retry-join "provider=azure tag_name=consul tag_name=consul tenant_id=${tid} tenant_id=${tid} client_id=${cid} client_id=${cid} subscription_id=$cid} subscription_id=$cid} secret_access_key=${ro_key}"secret_access_key=${ro_key}" puppet config set --section agent environment ${environment}puppet config set --section agent environment ${environment} puppet config set --section agent server puppet.service.consulpuppet config set --section agent server puppet.service.consul /opt/puppetlabs/bin/puppet agent -t/opt/puppetlabs/bin/puppet agent -t
  • 29.
    Fast ExportedFast Exported ResourcesResources ::consul::watch{ 'detect_backend_changes':::consul::watch { 'detect_backend_changes': type => 'service',type => 'service', handler => '/usr/bin/runpuppet.sh',handler => '/usr/bin/runpuppet.sh', service => 'node_exporter',service => 'node_exporter', passingonly => true,passingonly => true, require => File['/usr/bin/runpuppet.sh'],require => File['/usr/bin/runpuppet.sh'], }}
  • 30.
    VaultVault ● Open Sourcetool to do secrets managementOpen Source tool to do secrets management ● Secure, store and tightly control access toSecure, store and tightly control access to tokens, passwords, certificates, encryption keystokens, passwords, certificates, encryption keys for protecting secrets and other sensitive datafor protecting secrets and other sensitive data using a UI, CLI, or HTTP API.using a UI, CLI, or HTTP API. ● Certificate managementCertificate management ● Password rotatationPassword rotatation
  • 31.
    Pesky PasswordsPesky Passwords classprofiles::security::vault (class profiles::security::vault ( $config_dir = '/etc/vault.d',$config_dir = '/etc/vault.d', $enable_ui = true,$enable_ui = true, Variant[Hash, Array[Hash]] $listener = {Variant[Hash, Array[Hash]] $listener = { 'tcp' => {'tcp' => { 'address' => '127.0.0.1:8200','address' => '127.0.0.1:8200', 'cluster_address' => '127.0.0.1:8201','cluster_address' => '127.0.0.1:8201', 'tls_disable' => 1,'tls_disable' => 1, },}, },}, Hash $storage = { 'consul' => { 'address' => '127.0.0.1:8500', 'path' => 'vault/' }},Hash $storage = { 'consul' => { 'address' => '127.0.0.1:8500', 'path' => 'vault/' }}, String $version = '1.1.3',String $version = '1.1.3', ){){ class {'vault':class {'vault': config_dir => $config_dir,config_dir => $config_dir, enable_ui => $enable_ui,enable_ui => $enable_ui, listener => $listener,listener => $listener, storage => $storage,storage => $storage, version => $version,version => $version, }} https://forge.puppet.com/jsok/vaulthttps://forge.puppet.com/jsok/vault
  • 32.
    Pesky PasswordsPesky Passwords classclass{ '::vault':{ '::vault': install_method => 'repo',install_method => 'repo', }} $ vault unseal$ vault unseal $ vault write kv/my-secret value="s3c(eT"$ vault write kv/my-secret value="s3c(eT" $ vault read kv/mysecret$ vault read kv/mysecret Key ValueKey Value --- -------- ----- refresh_interval 768hrefresh_interval 768h mysecret s3c(eTmysecret s3c(eT https://forge.puppet.com/jsok/vaulthttps://forge.puppet.com/jsok/vault
  • 33.
    Pesky PasswordsPesky Passwords $vault unseal$ vault unseal $ vault write kv/my-secret value="s3c(eT"$ vault write kv/my-secret value="s3c(eT" $ vault read kv/mysecret$ vault read kv/mysecret Key ValueKey Value --- -------- ----- refresh_interval 768hrefresh_interval 768h mysecret s3c(eTmysecret s3c(eT
  • 34.
    HieraHiera cat hiera.yamlcat hiera.yaml ------ version:5version: 5 hierarchy:hierarchy: - name: "Hiera-vault lookup"- name: "Hiera-vault lookup" lookup_key: hiera_vaultlookup_key: hiera_vault options:options: confine_to_keys:confine_to_keys: - '^sensitive_.*'- '^sensitive_.*' - '^password.*'- '^password.*' address: http://active.vault.service.consul:8200address: http://active.vault.service.consul:8200 mounts:mounts: generic:generic: - secret/puppet/%{::trusted.certname}/- secret/puppet/%{::trusted.certname}/ - secret/puppet/common/- secret/puppet/common/ https://github.com/petems/hiera_backend_vaulthttps://github.com/petems/hiera_backend_vault
  • 35.
    Puppet5Puppet5 classclass profiles::database::mysql (profiles::database::mysql( Sensitive $root_password,Sensitive $root_password, ) {) { classclass { '::mysql::server':{ '::mysql::server': root_password => $root_password.unwrap,root_password => $root_password.unwrap, }} notice($root_password)notice($root_password) }} Notice: Scope(Class[main]): Sensitive [value redacted]Notice: Scope(Class[main]): Sensitive [value redacted]
  • 36.
    Puppet 6Puppet 6 $root_password= Deferred($root_password = Deferred( 'vault_lookup::lookup','vault_lookup::lookup', ["password/mysql"],["password/mysql"], ''https://active.vault.service.consul:8200https://active.vault.service.consul:8200',', )) classclass { '::mysql::server':{ '::mysql::server': root_password => $root_password,root_password => $root_password, }}
  • 37.
    NomadNomad ● Open Sourcetool to do dynamic workloadOpen Source tool to do dynamic workload schedulingscheduling ● Batch, containerized, and non-containerizedBatch, containerized, and non-containerized applications.applications. ● Has native Consul and Vault integrations.Has native Consul and Vault integrations.
  • 38.
    NomadNomad class { '::nomad':class{ '::nomad': config_hash = {config_hash = { 'client' => {'client' => { 'enabled' => true,'enabled' => true, },}, 'consul' => {'consul' => { 'address' => '127.0.0.1:8500','address' => '127.0.0.1:8500', },}, 'datacenter' => 'services','datacenter' => 'services', 'data_dir' => '/var/lib/nomad','data_dir' => '/var/lib/nomad', 'server' => {'server' => { 'enabled' => true'enabled' => true 'bootstrap_expect' => 3,'bootstrap_expect' => 3, },}, 'vault' => {'vault' => { 'enabled' => true,'enabled' => true, 'address' => ''address' => 'https://active.vault.service.consul:8200https://active.vault.service.consul:8200',', ''create_from_rolecreate_from_role' => '' => 'nomad-clusternomad-cluster',', ''tokentoken' => '' => 's.krHYYcd8PRzpSBO59AE6sawOs.krHYYcd8PRzpSBO59AE6sawO',', },}, }} }} https://forge.puppet.com/dudemcbacon/nomadhttps://forge.puppet.com/dudemcbacon/nomad https://github.com/attachmentgenie/puppet-nomadhttps://github.com/attachmentgenie/puppet-nomad
  • 39.
  • 40.
    BoltBolt ● Open Sourcetool to do task orchestrationOpen Source tool to do task orchestration ● Aimed at fire and forget 2nd day operationsAimed at fire and forget 2nd day operations
  • 41.
    Bolt && TerraformBolt&& Terraform ~/terraform_provision/inventory.yaml~/terraform_provision/inventory.yaml groups:groups: - name: terraform- name: terraform nodes: [] # This will be populated by the Bolt plannodes: [] # This will be populated by the Bolt plan config:config: transport: sshtransport: ssh ssh:ssh: private-key: ~/.ssh/id_rsa-phraselessprivate-key: ~/.ssh/id_rsa-phraseless user: ubuntuuser: ubuntu host-key-check: falsehost-key-check: false
  • 42.
    Bolt && TerraformBolt&& Terraform ~/terraform_provision/plans/init.pp~/terraform_provision/plans/init.pp plan terraform_provision(String $tf_path) {plan terraform_provision(String $tf_path) { $localhost = get_targets('localhost')$localhost = get_targets('localhost') $ip_string = run_command("cd ${$tf_path} && terraform output interal_ips"$ip_string = run_command("cd ${$tf_path} && terraform output interal_ips" $localhost).map |$r| { $r['stdout'] }$localhost).map |$r| { $r['stdout'] } $ips = Array($ip_string).map |$ip| { $ip.strip }$ips = Array($ip_string).map |$ip| { $ip.strip } # Turn IPs into Bolt targets, and add to inventory# Turn IPs into Bolt targets, and add to inventory $targets = $ips.map |$ip| {$targets = $ips.map |$ip| { Target.new("${$ip}").add_to_group('terraform')Target.new("${$ip}").add_to_group('terraform') }} # Deploy website# Deploy website apply_prep($targets)apply_prep($targets) apply($targets, _run_as => 'root') {}apply($targets, _run_as => 'root') {} return $ipsreturn $ips }}
  • 43.
    Bolt && TerraformBolt&& Terraform $ bolt plan run terraform_provision $ bolt plan run terraform_provision -i ~/terraform_provision/inventory.yaml -i ~/terraform_provision/inventory.yaml tf_path=~/terraform-playgroundtf_path=~/terraform-playground
  • 44.
    Icinga ProviderIcinga Provider providerprovider"icinga2" {"icinga2" { api_url = "https://icinga.alerting.vagrant:5665/v1"api_url = "https://icinga.alerting.vagrant:5665/v1" api_user = "root"api_user = "root" api_password = "icinga"api_password = "icinga" insecure_skip_tls_verify =insecure_skip_tls_verify = truetrue }}
  • 45.
    Icinga HostIcinga Host resource"aws_instance" "node1" {resource "aws_instance" "node1" { ami = "${var.ami_id}"ami = "${var.ami_id}" instance_type = "${var.ami_size}"instance_type = "${var.ami_size}" key_name = "${var.key_pair_name}"key_name = "${var.key_pair_name}" subnet_id = "${var.private_subnet}"subnet_id = "${var.private_subnet}" vpc_security_group_ids = ["${var.security_group_id}"]vpc_security_group_ids = ["${var.security_group_id}"] }} resourceresource "icinga2_host" "node1" {"icinga2_host" "node1" { hostname = "hostname = "node1"node1" address = "address = "${aws_instance.instance.private_ip}${aws_instance.instance.private_ip}"" check_command = "hostalive"check_command = "hostalive" varsvars {{ os = "linux"os = "linux" }} }}
  • 46.
    Check CommandCheck Command resourceresource"icinga2_checkcommand" "apache_status" {"icinga2_checkcommand" "apache_status" { name = "apache_status"name = "apache_status" templates = ["apache-status", "plugin-check-command",templates = ["apache-status", "plugin-check-command", "plugin-check-command", "ipv4-or-ipv6"]"plugin-check-command", "ipv4-or-ipv6"] command =command = "/usr/lib64/nagios/plugins/check_apache_status.pl""/usr/lib64/nagios/plugins/check_apache_status.pl" arguments = {arguments = { "-H" = "$apache_status_address$""-H" = "$apache_status_address$" "-c" = "$apache_status_critical$""-c" = "$apache_status_critical$" "-p" = "$apache_status_port$""-p" = "$apache_status_port$" }} }} resourceresource "icinga2_service" "my-service" {"icinga2_service" "my-service" { name = "ssh"name = "ssh" hostname = "c1-mysql-1"hostname = "c1-mysql-1" check_command = "ssh"check_command = "ssh" }}
  • 47.
    NotificationsNotifications resource "icinga2_user" "user"{resource "icinga2_user" "user" { name = "terraform"name = "terraform" email = "terraform@dev.null"email = "terraform@dev.null" }} resourceresource "icinga2_notification" "host-notification" {"icinga2_notification" "host-notification" { hostname = "docker-icinga2"hostname = "docker-icinga2" command = "mail-host-notification"command = "mail-host-notification" users = ["user"]users = ["user"] }} resourceresource "icinga2_notification" "ping-service-notification" {"icinga2_notification" "ping-service-notification" { hostname = "docker-icinga2"hostname = "docker-icinga2" command = "mail-service-notification"command = "mail-service-notification" users = ["user"]users = ["user"] servicename = "ping"servicename = "ping" }}
  • 48.
    Other ResourcesOther Resources resourceresource"icinga2_checkcommand" "apache_status" {"icinga2_checkcommand" "apache_status" { name = "apache_status"name = "apache_status" templates = ["apache-status", "plugin-check-command",templates = ["apache-status", "plugin-check-command", "plugin-check-command", "ipv4-or-ipv6"]"plugin-check-command", "ipv4-or-ipv6"] command =command = "/usr/lib64/nagios/plugins/check_apache_status.pl""/usr/lib64/nagios/plugins/check_apache_status.pl" arguments = {arguments = { "-H" = "$apache_status_address$""-H" = "$apache_status_address$" "-c" = "$apache_status_critical$""-c" = "$apache_status_critical$" "-p" = "$apache_status_port$""-p" = "$apache_status_port$" }} }}
  • 49.
    Icinga Director &ConsulIcinga Director & Consul github.com/attachmentgenie/icingaweb2-module-consul.gitgithub.com/attachmentgenie/icingaweb2-module-consul.git
  • 50.
    Query ConsulQuery Consul useSensioLabsConsulServiceFactory;use SensioLabsConsulServiceFactory; public functionpublic function fetchData()fetchData() {{ $sf = new ServiceFactory($sf = new ServiceFactory( arrayarray('base_uri' => $this->getSetting('consul_url'))('base_uri' => $this->getSetting('consul_url')) );); $agent = $sf->get('agent');$agent = $sf->get('agent'); return json_decode($agent->members()->getBody());return json_decode($agent->members()->getBody()); }} public functionpublic function listColumns()listColumns() {{ return array_keys((array) current($this->fetchData()));return array_keys((array) current($this->fetchData())); }}
  • 51.
    Adding a syncsourceAdding a sync source
  • 52.
    Adding a syncruleAdding a sync rule
  • 53.
    Add sync propertiesAddsync properties
  • 54.
    Fast FeedbackFast Feedback file{ 'Director Sync and Deploy':file { 'Director Sync and Deploy': path => '/usr/local/bin/director_sync_and_deploy.sh',path => '/usr/local/bin/director_sync_and_deploy.sh', source =>source => 'puppet:///modules/profiles/monitoring/icinga2/director_sync_'puppet:///modules/profiles/monitoring/icinga2/director_sync_ and_deploy.sh',and_deploy.sh', owner => 'consul',owner => 'consul', group => 'consul',group => 'consul', mode => '0744',mode => '0744', }} ::consul::watch { 'director_import':::consul::watch { 'director_import': type => 'service',type => 'service', handler =>handler => '/usr/local/bin/director_sync_and_deploy.sh','/usr/local/bin/director_sync_and_deploy.sh', service => 'node_exporter',service => 'node_exporter', passingonly =>passingonly => truetrue,, require => File['Director Sync and Deploy'],require => File['Director Sync and Deploy'], }}
  • 55.
    director_sync_and_deploy.shdirector_sync_and_deploy.sh #!/usr/bin/env bash#!/usr/bin/env bash setset-x-x icingacliicingacli director importsource run --id 1director importsource run --id 1 icingacliicingacli director syncrule run --id 1director syncrule run --id 1 icingacliicingacli director config deploydirector config deploy