SlideShare a Scribd company logo
1 of 44
Download to read offline
Icinga2Icinga2
A highly available cloud setup deployed at DigitalOceanA highly available cloud setup deployed at DigitalOcean
Who am I? | Short tales from the past
Solution | Result - know your why!
incl. the database & Galera cluster configuration
Highlights and critical sections
Don't do the same mistakes as I did
Agenda
> Quick Introduction
> High level overview
> Installation of all components
> Configuration deep-dive
> Pitfalls & Experienced limitations
Who am I?
Marcel Weinberg
Living in Hamburg
Working at CoreMedia AG (since 09/2019)
Doing O(o)ps things for more then ten years now
With a great passion for
Automation, Monitoring, Open Source
& Mountainbiking
Quick Introduction
@winem_
winem
+ extensive documentation
+ well defined & tested DR* procedure
- single instance with many SPOFs
- limited redundancy on the uplink
Quick Introducation - Tales from the past
Previous Setup
It was a single instance in the server room:
* DR = Disaster Recovery
High Level Overview
Solution
And we
still have
alarming!
High Level Overview
Solution
Result
High Level Overview
Solution
Result - Know your why
vs.
Installation Of All Components
Always refer to the public available documentation for up-to-date installation and upgrade
instructions.
> PHP7.3 - ppa:ondrej/php
> icinga2, icingacli, icingaweb2 - packages.icinga.com
apt install icinga2 icinga2-ido-mysql icingaweb2 icingacli vim-icinga2 vim-addon-manager php7.3-gd
> haproxy - ppa:vbernat/haproxy-1.9
> keepalived from the standard repositories
Installation Of All Components
Always refer to the public available documentation for up-to-date installation and upgrade
instructions.
> PHP7.3 - ppa:ondrej/php
> icinga2, icingacli, icingaweb2 - packages.icinga.com
apt install icinga2 icinga2-ido-mysql icingaweb2 icingacli vim-icinga2 vim-addon-manager php7.3-gd
> haproxy - ppa:vbernat/haproxy-1.9
> keepalived from the standard repositories
keepalived_script - a dedicated user to run check and notification scripts:
sudo adduser --no-create-home --disabled-login keepalived_script
Installation Of All Components
MariaDB
Secure your installation and don't start mariadb automatically:
sudo mysql_secure_installation
sudo systemctl disable mariadb
Link: https://downloads.mariadb.org/mariadb/repositories/
Installation of all components
MariaDB Con guration Files
Use mysqld --help --verbose to see the loaded configuration files and sections:
agent@m-do-ffm-galera-01:~$ mysql --help --verbose | head -n 10
mysql Ver 15.1 Distrib 10.4.8-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Usage: mysql [OPTIONS] [database]
Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf
The following groups are read: mysql mariadb-client client client-server client-mariadb
Installation of all components
MariaDB Con guration Files
Galera Cluster Con guration
/etc/mysql/mariadb.conf.d/galera.cnf
[mysqld]
bind-address=10.135.209.4
default_storage_engine=InnoDB
binlog_format=row
innodb_autoinc_lock_mode=2
innodb_flush_log_at_trx_commit=0
# Galera cluster configuration
wsrep_on=ON
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://10.135.209.4,10.135.207.79,10.135.207.111"
wsrep_cluster_name="mariadb-galera-backend"
wsrep_slave_threads=8
wsrep_sst_method=rsync
# Cluster node configuration
wsrep_node_address="10.135.209.4"
wsrep_node_name="galera-01"
> Highlighted lines show those values that are unique per node.
Installation of all components
MariaDB Con guration Files
Galera Cluster Con guration
/etc/mysql/mariadb.conf.d/galera.cnf
[mysqld]
bind-address=10.135.209.4
default_storage_engine=InnoDB
binlog_format=row
innodb_autoinc_lock_mode=2
innodb_flush_log_at_trx_commit=0
# Galera cluster configuration
wsrep_on=ON
wsrep_provider=/usr/lib/galera/libgalera_smm.so
wsrep_cluster_address="gcomm://10.135.209.4,10.135.207.79,10.135.207.111"
wsrep_cluster_name="mariadb-galera-backend"
wsrep_slave_threads=8
wsrep_sst_method=rsync
# Cluster node configuration
wsrep_node_address="10.135.209.4"
wsrep_node_name="galera-01"
> Set wsrep_slave_threads to the number of CPUs * 2.
> innodb_flush_log_at_trx_commit = 0 means to write the caches to disk every second.
Start the first cluster node:
sudo galera_new_cluster
Start mariadb on the remaining nodes:
sudo service mariadb start
Installation of all components
MariaDB Con guration Files |GaleraClusterCon guration
Bootstrap the cluster
Wait until all nodes joined the cluster and restart mariadb on the first node
MariaDB [(none)]> show global status where Variable_name in 
('wsrep_cluster_size', 'wsrep_local_state_comment', 'wsrep_ready', 'wsrep_connected');
+---------------------------+--------+
| Variable_name | Value |
+---------------------------+--------+
| wsrep_local_state_comment | Synced |
| wsrep_cluster_size | 3 |
| wsrep_connected | ON |
| wsrep_ready | ON |
+---------------------------+--------+
4 rows in set (0.002 sec)
Database cluster, configuration & accounts
icinga2
CREATE DATABASE icinga2;
GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE VIEW, INDEX, EXECUTE ON 
icinga2.* TO 'icinga2'@'10.135.201.194' IDENTIFIED BY 'maa3EeHah0Ea';
GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE VIEW, INDEX, EXECUTE ON 
icinga2.* TO 'icinga2'@'10.135.221.140' IDENTIFIED BY 'maa3EeHah0Ea';
Copy the mysql.sql file to any of the galera nodes and import it:
mysql -u root icinga2 < /tmp/mysql.sql
Icingaweb2
CREATE DATABASE icingaweb2;
GRANT ALL ON icingaweb2.* TO icingaweb2@10.135.201.194 IDENTIFIED BY 'ievachiYie4ooz5a';
GRANT ALL ON icingaweb2.* TO icingaweb2@10.135.221.140 IDENTIFIED BY 'ievachiYie4ooz5a';
GRANT SELECT, INSERT, UPDATE, DELETE, DROP, INDEX, EXECUTE, CREATE VIEW 
ON icinga2.* TO 'icingaweb2'@'10.135.201.194';
GRANT SELECT, INSERT, UPDATE, DELETE, DROP, INDEX, EXECUTE, CREATE VIEW 
ON icinga2.* TO 'icingaweb2'@'10.135.221.140';
Database cluster, configuration & accounts
HAProxy
CREATE USER 'haproxy_checks'@'10.135.201.194';
CREATE USER 'haproxy_checks'@'10.135.221.140';
Database cluster, configuration & accounts
Keepalived
GRANT SELECT ON icingaweb2.* TO 'keepalived_check'@'10.135.221.140' 
IDENTIFIED BY 'iiw8ahthe1Ch';
GRANT SELECT ON icingaweb2.* TO 'keepalived_check'@'10.135.221.249' 
IDENTIFIED BY 'iiw8ahthe1Ch';
my.cnf file for the keepalived healthches from the icinga2 nodes:
sudo cat > /etc/keepalived/.my.cnf << EOF
[client]
host=127.0.0.1
password=iiw8ahthe1Ch
user=keepalived_check
EOF
Set proper permissions:
sudo chown keepalived_script:root /etc/keepalived/.my.cnf
sudo chmod 600 /etc/keepalived/.my.cnf
agent@m-do-ffm-plugins-01:~$ sudo icinga2 node setup 
--master 
--cn m-do-ffm-plugins-01.mydomain.com 
--disable-confd 
--accept-config 
--accept-commands 
--zone do-ffm-masters
agent@m-do-ffm-plugins-02:~$ sudo icinga2 node setup 
--master 
--cn m-do-ffm-plugins-02.mydomain.com 
--disable-confd 
--accept-config 
--accept-commands 
--zone do-ffm-masters
Deep-dive into the configuration
Icinga2 - Node Setup
Icinga2 configuration using the node setup utility
agent@m-do-ffm-plugins-01:~$ sudo icinga2 node setup 
--master 
--cn m-do-ffm-plugins-01.mydomain.com 
--disable-confd 
--accept-config 
--accept-commands 
--zone do-ffm-masters
agent@m-do-ffm-plugins-02:~$ sudo icinga2 node setup 
--master 
--cn m-do-ffm-plugins-02.mydomain.com 
--disable-confd 
--accept-config 
--accept-commands 
--zone do-ffm-masters
Deep-dive into the configuration
Icinga2 - Node Setup
Icinga2 configuration using the node setup utility
/etc/icinga2/constants.conf
[...]
const NodeName = "m-do-ffm-plugins-01.mydomain.com"
/* Our local zone name. */
const ZoneName = "do-ffm-masters"
/* Secret key for remote node tickets */
const TicketSalt = "689eb691e3d5cbf8e3a11fcb50089610"
[...]
Deep-dive into the configuration
Icinga2 - Zone & endpoint con guration
/etc/icinga2/zones.conf
object Endpoint "m-do-ffm-plugins-01.mydomain.com" {
host = "10.135.201.194"
}
object Endpoint "m-do-ffm-plugins-02.mydomain.com" {
host = "10.135.221.140"
}
object Zone "do-ffm-masters" {
endpoints = [ "m-do-ffm-plugins-01.mydomain.com", "m-do-ffm-plugins-02.mydomain.com" ]
}
object Zone "global-templates" {
global = true
}
object Zone "director-global" {
global = true
}
Deep-dive into the configuration
Icinga2 - ido-mysql feature
The configuration is on both nodes the same. /etc/icinga2/features-available/ido-mysql.conf
library "db_ido_mysql"
object IdoMysqlConnection "ido-mysql" {
user = "icinga2",
password = "maa3EeHah0Ea",
host = "127.0.0.1",
database = "icinga2"
enable_ha = true
cleanup = {
contactnotifications_age = 30d
downtimehistory_age = 60d
statehistory_age = 60d
externalcommands_age = 2d
}
}
agent@m-do-ffm-plugins-01:~$ icinga2 feature enable ido-mysql
agent@m-do-ffm-plugins-01:~$ sudo icinga2 feature list
Disabled features: command compatlog debuglog elasticsearch gelf graphite 
influxdb livestatus opentsdb perfdata statusdata syslog
Enabled features: api checker ido-mysql mainlog notification
Deep-dive into the configuration
Icinga2 - API users
/etc/icinga2/zones.d/do-ffm-masters/api-users.conf
/**
* The ApiUser objects are used for authentication against the API.
*/
object ApiUser "api-healthchecks" {
password = "eiy8aiGh0aog"
permissions = [ "actions/process-check-result" ]
}
object ApiUser "icingaweb2" {
password = "Eila5iRaed2o"
permissions = [ "status/query", "actions/*", "objects/modify/*", "objects/query/*" ]
}
object ApiUser "keepalived" {
password = "paigohLe8soh"
permissions = [ "status/query" ]
}
Deep-dive into the configuration
Icinga2 - Valdiate & apply the adjusted con guration
Validate the new configuration
agent@m-do-ffm-plugins-01:~$ sudo icinga2 daemon -C
[2019-09-28 23:29:05 +0000] information/cli: Icinga application loader (version: r2.11.0-1)
[2019-09-28 23:29:05 +0000] information/cli: Loading configuration file(s).
[2019-09-28 23:29:06 +0000] information/ConfigItem: Committing config item(s).
[2019-09-28 23:29:06 +0000] information/ApiListener: My API identity: m-do-ffm-plugins-01.mydomain.com
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 FileLogger.
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 NotificationComponent.
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 IcingaApplication.
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 CheckerComponent.
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 3 Zones.
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 2 Endpoints.
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 3 ApiUser.
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 ApiListener.
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 235 CheckCommands.
[2019-09-28 23:29:06 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2019-09-28 23:29:06 +0000] information/cli: Finished validating the configuration file(s).
Deep-dive into the configuration
Icinga2
HAProxy - Global and default con guration for both nodes
global
log /dev/log
local0
log /dev/log
local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd
defaults
log global
option httplog
option dontlognulltimeout
timeout connect 5000
timeout client 5000
timeout server 5000
retries 2
Deep-dive into the configuration
Icinga2
HAProxy - General listener con guration & statistics endpoint
listeners
stats timeout 30s
user haproxy
group haproxy
daemon
listen ha-stats
bind 127.0.0.1:8026
mode http
stats enable
stats uri /
stats realm Strictly Privatestats auth icinga_monitor:chong4gohCe8
> The ha-stats endpoint exposes statistics about haproxy, the configured listeners and backends.
> Useful for debugging, testing and monitoring.
> Keep it secure! (see the last line and setup firewall rules)
Deep-dive into the configuration
Icinga2
HAProxy
Special thanks to Carsten!
@Mikeschova
https://community.icinga.com/t/galera-mysql-cluster-with-vips-and-haproxy-for-ido-mysql-and-more/
Deep-dive into the configuration
Icinga2
HAProxy - 3 di erent listeners for the MariaDB backend
> use galera-01 and galera-02 as prefered endpoint for write-intensive access (port 3306)
> balance read queries round robin across all 3 nodes; individual weight of each galera node varies
between the two icinga2 nodes (port 3307)
> the director has probably an equal distribution of read and write queries (port 3308)
Deep-dive into the configuration
Icinga2
HAProxy
Listener to provide database access to for icinga2
# 1st node
listen icinga2_ido
bind 127.0.0.1:3306
mode tcp
option mysql-check user haproxy_checks
balance source
server do-ffm-galera01 10.135.209.4:3306 check weight 1 inter 5s rise 2 fall 2
server do-ffm-galera02 10.135.207.79:3306 check weight 2 inter 5s rise 2 fall 2 backup
server do-ffm-galera03 10.135.207.11:3306 check weight 3 inter 5s rise 2 fall 2 backup
# 2nd node
listen icinga2_ido
bind 127.0.0.1:3306
mode tcp
option mysql-check user haproxy_checks
balance source
server do-ffm-galera01 10.135.209.4:3306 check weight 2 inter 5s rise 2 fall 2 backup
server do-ffm-galera02 10.135.207.79:3306 check weight 1 inter 5s rise 2 fall 2
server do-ffm-galera03 10.135.207.11:3306 check weight 3 inter 5s rise 2 fall 2 backup
Deep-dive into the configuration
Icinga2
HAProxy
Listener to provide database access for icingaweb2 and handle mostly read queries
# 1st node
listen mysqlread
bind 127.0.0.1:3308
mode tcp
option mysql-check user haproxy_checks
balance roundrobin
server do-ffm-galera01 10.135.209.4:3306 check weight 2 inter 5s rise 2 fall 2
server do-ffm-galera02 10.135.207.79:3306 check weight 3 inter 5s rise 2 fall 2
server do-ffm-galera03 10.135.207.11:3306 check weight 1 inter 5s rise 2 fall 2
# 2nd node
listen mysqlread
bind 127.0.0.1:3308
mode tcp
option mysql-check user haproxy_checks
balance roundrobin
server do-ffm-galera01 10.135.209.4:3306 check weight 1 inter 5s rise 2 fall 2
server do-ffm-galera02 10.135.207.79:3306 check weight 3 inter 5s rise 2 fall 2
server do-ffm-galera03 10.135.207.11:3306 check weight 2 inter 5s rise 2 fall 2
Deep-dive into the configuration
Icinga2
HAProxy
Listener to provide access to the icingaweb2 director database backend
listen director
bind 127.0.0.1:3307
mode tcp
option mysql-check user haproxy_checks
balance source
server do-ffm-galera01 10.135.209.4:3306 check weight 3 inter 5s rise 2 fall 2 backup
server do-ffm-galera02 10.135.207.79:3306 check weight 2 inter 5s rise 2 fall 2 backup
server do-ffm-galera03 10.135.207.11:3306 check weight 1 inter 5s rise 2 fall 2
> This is the same for both nodes again.
Deep-dive into the configuration
Icinga2|HAProxy
Apache2 & PHP
Virtual host configuration:
> set proper date.timezone in php.ini
> Enforce encryption and redirect HTTP requests to HTTPS:
Redirect permanent / https://moni.mydomain.com
> Rewrite empty URIs to /icingaweb2:
RewriteRule ^(.*) http://%{HTTP_HOST}/icingaweb2
> Enable the required modules and the new site:
sudo a2enmod proxy_http rewrite ssl
sudo a2ensite moni.mydomain.com
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP
Icingaweb2 - https://moni.mydomain.com/Setup
> Protected custom variables (pw, pass, password, hmac, community)
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP
Icingaweb2
Add a dedicated user to sync /etc/icingaweb2/ between both icinga2 nodes.
agent@m-do-ffm-plugins-01:~$ sudo adduser --disabled-password icingaweb_sync
agent@m-do-ffm-plugins-01:~$ sudo su - icingaweb_sync
icingaweb_sync@m-do-ffm-plugins-01:~$ ssh-keygen -t rsa -b 4096
Add the public key to the authorized_keys file.
Setup the crontab on both nodes
agent@m-do-ffm-plugins-01:~$ sudo cat /etc/cron.d/icingaweb_sync
MAILTO=""
* * * * * icingaweb_sync <ip-of-the-peer-node>:/etc/icingaweb2 rsync -rugopl /etc/icingaweb2/
> the system user keepalived_script
> the .my.cnf file in /etc/keepalived
> the icinga2 API user keepalived to perform healthchecks
> the icinga2 API user api_healthchecks to send the check results to icinga
> the database user keepalived_checks to test the connection to the DB
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP|Icingaweb2
Keepalived
Remember what we already have:
vrrp_script check_apache2 {
script "/usr/bin/killall -0 apache2"
interval 2
fall 1
rise 5
user root
}
vrrp_script check_icinga2_api {
script "/usr/local/bin/check_icinga_api"
interval 2
fall 3
rise 3
}
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP|Icingaweb2
Keepalived - /etc/keepalived/keepalived.conf (Healthchecks)
vrrp_script check_mysql_conn {
script "mysql --defaults-file=/etc/keepalived/.my.cnf -e 'select count(*) from icingaweb2.icingaweb_user'"
interval 2
fall 1
rise 3
}
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP|Icingaweb2
Keepalived - Icinga2 API Healthcheck
/usr/local/bin/check_icinga_api - a simple healthcheck
#!/usr/bin/env bash
icinga_api_user='keepalived'
icinga_api_pass='paigohLe8soh'
icinga_api_timeout=1
api_rc=$(curl -k -s -o /dev/null -w %{http_code} --max-time $icinga_api_timeout 
-u $icinga_api_user:$icinga_api_pass https://localhost:5665/v1/status/ApiListener)
if [ $api_rc -eq 200 ]; then
exit 0
else
exit 1
fi
global_defs {
enable_script_security
}
vrrp_instance icinga_fe {
state BACKUP
nopreempt
interface eth1
virtual_router_id 10
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass CaemooXah1Ke
}
unicast_src_ip 10.135.201.194
unicast_peer {
10.135.221.140
}
global_defs {
enable_script_security
}
vrrp_instance icinga_fe {
state BACKUP
nopreempt
interface eth1
virtual_router_id 10
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass CaemooXah1Ke
}
unicast_src_ip 10.135.201.140
unicast_peer {
10.135.221.194
}
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP|Icingaweb2
Keepalived - /etc/keepalived/keepalived.conf (global_defs & vrrp_instance)
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP|Icingaweb2
Keepalived - /etc/keepalived/keepalived.conf (track & notify scripts)
track_script {
check_apache2
check_mysql_conn
check_icinga2_api
}
notify_backup "/usr/local/bin/keepalived-alarming BACKUP"
notify_fault "/usr/local/bin/keepalived-alarming FAULT"
notify_master /etc/keepalived/master.sh
> /etc/keepalived/master.sh triggers keepalived-alarming MASTER
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP|Icingaweb2
Keepalived - Script to assign the oating IP to the new master
#!/usr/bin/env bash
floating_ip='68.183.240.112'
droplet_id=$(curl -s http://169.254.169.254/metadata/v1/id)
has_floating_ip=$(curl -s http://169.254.169.254/metadata/v1/floating_ip/ipv4/active)
export DO_TOKEN='ohjajeelaex8oongiequuxiethae4ing1jooch2ezieP2Shil6ohei8ierei3oizi'
if [ $has_floating_ip = "false" ]; then
n=0
while [ $n -lt 5 ]; do
python3 /usr/local/bin/assign-ip $floating_ip $droplet_id && break
n=$((n+1))
sleep 3
done
fi
keepalived-alarming MASTER
Monitor floating IP action via https://api.digitalocean.com/v2/floating_ips/<floating-ip>/actions?page=1&per_page=1 .
Possible states: completed, in-progress or failed.
MASTER & BACKUP = 0
FAULT = 2
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP|Icingaweb2
Keepalived - Noti cation Script (Snippets)
/usr/local/bin/keepalived-alarming
#/usr/bin/env bash
# $1 = target state of transition ("MASTER", "BACKUP", "FAULT")
t_state=$1
chk_message="Keepalived entered $t_state state."
icinga_chk_host=$(hostname)
# 1st node endpoints
icinga_endpoints=("127.0.0.1" "10.135.221.140")
Use a case statement to translate t_state as following:
Deep-dive into the configuration
Icinga2|HAProxy|Apache2&PHP|Icingaweb2
Keepalived - Noti cation Script (Snippets)
Construct the JSON for the icinga2 API call:
icinga_post_data()
{
cat <<EOF
{
"service": "${icinga_chk_host}!${icinga_service}",
"exit_status": "$e_status",
"plugin_output": "$chk_message",
"performance_data": [
"entered_keepalived_state=$new_state;;2;0;2",
]
}
EOF
}
Pitfalls & Don't Dos
> Don't stop all galera nodes at the same time! If that happens, you have to bootstrap the cluster
again.
> Don't forget to set enable_ha = true for the ido-mysql feature
> Don't create the rsync user without --disabled-password
> Don't rely on the volatile instance / droplet IPs. Use floating IPs
> Don't forget backups / snapshots: droplets, database content, configuration & setup
> Don't miss to migrate to the director if you have the chance to
> Setup a local firewall if the cloud providers firewall does not support VRRP2
Firewall Rules
> Communication between the Icinga2 masters
TCP/5665, VRRP2
> Communication from the satellites and agents to the master nodes
TCP/5665
> Icinga2 nodes to the galera nodes
TCP/3306 , TCP/3307 , TCP/3308
> Communication between the galera cluster nodes
TCP/4567, TCP/4568, UDP/4567
> Restrictive ipset & iptables How-To:
https://community.icinga.com/t/firewall-setup-at-a-icinga2-cluster-with-iptables-ipset/2156
THANK YOU!THANK YOU!
@winem_@winem_

More Related Content

What's hot

Tested install-isp config3-ubuntu-16-04
Tested install-isp config3-ubuntu-16-04Tested install-isp config3-ubuntu-16-04
Tested install-isp config3-ubuntu-16-04SANTIAGO HERNÁNDEZ
 
图文详解安装Net backup 6.5备份恢复oracle 10g rac 数据库
图文详解安装Net backup 6.5备份恢复oracle 10g rac 数据库图文详解安装Net backup 6.5备份恢复oracle 10g rac 数据库
图文详解安装Net backup 6.5备份恢复oracle 10g rac 数据库maclean liu
 
Hardening Linux and introducing Securix Linux
Hardening Linux and introducing Securix LinuxHardening Linux and introducing Securix Linux
Hardening Linux and introducing Securix LinuxSecurity Session
 
CentOS Linux Server Hardening
CentOS Linux Server HardeningCentOS Linux Server Hardening
CentOS Linux Server HardeningMyOwn Telco
 
Clouldera Implementation Guide for Production Deployments
Clouldera Implementation Guide for Production DeploymentsClouldera Implementation Guide for Production Deployments
Clouldera Implementation Guide for Production DeploymentsAhmed Mekawy
 
Server hardening
Server hardeningServer hardening
Server hardeningTeja Babu
 
Trouble shooting apachecloudstack
Trouble shooting apachecloudstackTrouble shooting apachecloudstack
Trouble shooting apachecloudstackSailaja Sunil
 
第6讲 操作与配置Cisco Ios
第6讲 操作与配置Cisco Ios第6讲 操作与配置Cisco Ios
第6讲 操作与配置Cisco IosF.l. Yu
 
Linux Server Hardening - Steps by Steps
Linux Server Hardening - Steps by StepsLinux Server Hardening - Steps by Steps
Linux Server Hardening - Steps by StepsSunil Paudel
 
Passwordless login with unix auth_socket
Passwordless login with unix auth_socketPasswordless login with unix auth_socket
Passwordless login with unix auth_socketOtto Kekäläinen
 
Software Packaging with RPM
Software Packaging with RPMSoftware Packaging with RPM
Software Packaging with RPMSchalk Cronjé
 
How to configure a hive high availability connection with zeppelin
How to configure a hive high availability connection with zeppelinHow to configure a hive high availability connection with zeppelin
How to configure a hive high availability connection with zeppelinTiago Simões
 
Introduction to Stacki at Atlanta Meetup February 2016
Introduction to Stacki at Atlanta Meetup February 2016Introduction to Stacki at Atlanta Meetup February 2016
Introduction to Stacki at Atlanta Meetup February 2016StackIQ
 
Helpful Juniper Tips and Tricks for New Network Engineers
Helpful Juniper Tips and Tricks for New Network EngineersHelpful Juniper Tips and Tricks for New Network Engineers
Helpful Juniper Tips and Tricks for New Network EngineersLizbeth E. Melendez
 
Less passwords, more security: unix socket authentication and other MariaDB h...
Less passwords, more security: unix socket authentication and other MariaDB h...Less passwords, more security: unix socket authentication and other MariaDB h...
Less passwords, more security: unix socket authentication and other MariaDB h...Otto Kekäläinen
 
How to create a secured cloudera cluster
How to create a secured cloudera clusterHow to create a secured cloudera cluster
How to create a secured cloudera clusterTiago Simões
 
Configure, Pack and Distribute: An RPM Creation Workshop
Configure, Pack and Distribute: An RPM Creation WorkshopConfigure, Pack and Distribute: An RPM Creation Workshop
Configure, Pack and Distribute: An RPM Creation WorkshopNovell
 

What's hot (20)

Tested install-isp config3-ubuntu-16-04
Tested install-isp config3-ubuntu-16-04Tested install-isp config3-ubuntu-16-04
Tested install-isp config3-ubuntu-16-04
 
OTRS
OTRSOTRS
OTRS
 
图文详解安装Net backup 6.5备份恢复oracle 10g rac 数据库
图文详解安装Net backup 6.5备份恢复oracle 10g rac 数据库图文详解安装Net backup 6.5备份恢复oracle 10g rac 数据库
图文详解安装Net backup 6.5备份恢复oracle 10g rac 数据库
 
Hardening Linux and introducing Securix Linux
Hardening Linux and introducing Securix LinuxHardening Linux and introducing Securix Linux
Hardening Linux and introducing Securix Linux
 
CentOS Linux Server Hardening
CentOS Linux Server HardeningCentOS Linux Server Hardening
CentOS Linux Server Hardening
 
Clouldera Implementation Guide for Production Deployments
Clouldera Implementation Guide for Production DeploymentsClouldera Implementation Guide for Production Deployments
Clouldera Implementation Guide for Production Deployments
 
Server hardening
Server hardeningServer hardening
Server hardening
 
Trouble shooting apachecloudstack
Trouble shooting apachecloudstackTrouble shooting apachecloudstack
Trouble shooting apachecloudstack
 
Freeradius edir
Freeradius edirFreeradius edir
Freeradius edir
 
第6讲 操作与配置Cisco Ios
第6讲 操作与配置Cisco Ios第6讲 操作与配置Cisco Ios
第6讲 操作与配置Cisco Ios
 
Linux Server Hardening - Steps by Steps
Linux Server Hardening - Steps by StepsLinux Server Hardening - Steps by Steps
Linux Server Hardening - Steps by Steps
 
Passwordless login with unix auth_socket
Passwordless login with unix auth_socketPasswordless login with unix auth_socket
Passwordless login with unix auth_socket
 
Software Packaging with RPM
Software Packaging with RPMSoftware Packaging with RPM
Software Packaging with RPM
 
MySQL Monitoring 101
MySQL Monitoring 101MySQL Monitoring 101
MySQL Monitoring 101
 
How to configure a hive high availability connection with zeppelin
How to configure a hive high availability connection with zeppelinHow to configure a hive high availability connection with zeppelin
How to configure a hive high availability connection with zeppelin
 
Introduction to Stacki at Atlanta Meetup February 2016
Introduction to Stacki at Atlanta Meetup February 2016Introduction to Stacki at Atlanta Meetup February 2016
Introduction to Stacki at Atlanta Meetup February 2016
 
Helpful Juniper Tips and Tricks for New Network Engineers
Helpful Juniper Tips and Tricks for New Network EngineersHelpful Juniper Tips and Tricks for New Network Engineers
Helpful Juniper Tips and Tricks for New Network Engineers
 
Less passwords, more security: unix socket authentication and other MariaDB h...
Less passwords, more security: unix socket authentication and other MariaDB h...Less passwords, more security: unix socket authentication and other MariaDB h...
Less passwords, more security: unix socket authentication and other MariaDB h...
 
How to create a secured cloudera cluster
How to create a secured cloudera clusterHow to create a secured cloudera cluster
How to create a secured cloudera cluster
 
Configure, Pack and Distribute: An RPM Creation Workshop
Configure, Pack and Distribute: An RPM Creation WorkshopConfigure, Pack and Distribute: An RPM Creation Workshop
Configure, Pack and Distribute: An RPM Creation Workshop
 

Similar to OSMC 2019 | Use Cloud services & features in your redundant Icinga2 Environment by Marcel Weinberg

RAC-Installing your First Cluster and Database
RAC-Installing your First Cluster and DatabaseRAC-Installing your First Cluster and Database
RAC-Installing your First Cluster and DatabaseNikhil Kumar
 
Installing oracle grid infrastructure and database 12c r1
Installing oracle grid infrastructure and database 12c r1Installing oracle grid infrastructure and database 12c r1
Installing oracle grid infrastructure and database 12c r1Voeurng Sovann
 
How to install squid proxy on server or how to install squid proxy on centos o
How to install squid proxy on server  or how to install squid proxy on centos oHow to install squid proxy on server  or how to install squid proxy on centos o
How to install squid proxy on server or how to install squid proxy on centos oProxiesforrent
 
OSMC 2021 | Icinga-Installer – the easy way to your Icinga
OSMC 2021 | Icinga-Installer – the easy way to your IcingaOSMC 2021 | Icinga-Installer – the easy way to your Icinga
OSMC 2021 | Icinga-Installer – the easy way to your IcingaNETWAYS
 
Asian Spirit 3 Day Dba On Ubl
Asian Spirit 3 Day Dba On UblAsian Spirit 3 Day Dba On Ubl
Asian Spirit 3 Day Dba On Ublnewrforce
 
MySQL Spider Architecture
MySQL Spider ArchitectureMySQL Spider Architecture
MySQL Spider ArchitectureI Goo Lee
 
Spider Setup with AWS/sandbox
Spider Setup with AWS/sandboxSpider Setup with AWS/sandbox
Spider Setup with AWS/sandboxI Goo Lee
 
MySQL Audit using Percona audit plugin and ELK
MySQL Audit using Percona audit plugin and ELKMySQL Audit using Percona audit plugin and ELK
MySQL Audit using Percona audit plugin and ELKYoungHeon (Roy) Kim
 
26.1.7 lab snort and firewall rules
26.1.7 lab   snort and firewall rules26.1.7 lab   snort and firewall rules
26.1.7 lab snort and firewall rulesFreddy Buenaño
 
Configuration Management with Saltstack
Configuration Management with SaltstackConfiguration Management with Saltstack
Configuration Management with Saltstackinovex GmbH
 
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...SaltStack
 
How to deploy docker container inside ikoula's cloud
How to deploy docker container inside ikoula's cloudHow to deploy docker container inside ikoula's cloud
How to deploy docker container inside ikoula's cloudNicolas Trauwaen
 
X64服务器 lnmp服务器部署标准 new
X64服务器 lnmp服务器部署标准 newX64服务器 lnmp服务器部署标准 new
X64服务器 lnmp服务器部署标准 newYiwei Ma
 
Kl 031.30 eng_class_setup_guide_1.2
Kl 031.30 eng_class_setup_guide_1.2Kl 031.30 eng_class_setup_guide_1.2
Kl 031.30 eng_class_setup_guide_1.2Freddy Ortiz
 
Architecting cloud
Architecting cloudArchitecting cloud
Architecting cloudTahsin Hasan
 
Oracle 11g R2 RAC setup on rhel 5.0
Oracle 11g R2 RAC setup on rhel 5.0Oracle 11g R2 RAC setup on rhel 5.0
Oracle 11g R2 RAC setup on rhel 5.0Santosh Kangane
 
Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1
Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1
Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1Raheel Syed
 

Similar to OSMC 2019 | Use Cloud services & features in your redundant Icinga2 Environment by Marcel Weinberg (20)

RAC-Installing your First Cluster and Database
RAC-Installing your First Cluster and DatabaseRAC-Installing your First Cluster and Database
RAC-Installing your First Cluster and Database
 
Installing oracle grid infrastructure and database 12c r1
Installing oracle grid infrastructure and database 12c r1Installing oracle grid infrastructure and database 12c r1
Installing oracle grid infrastructure and database 12c r1
 
How to install squid proxy on server or how to install squid proxy on centos o
How to install squid proxy on server  or how to install squid proxy on centos oHow to install squid proxy on server  or how to install squid proxy on centos o
How to install squid proxy on server or how to install squid proxy on centos o
 
OSMC 2021 | Icinga-Installer – the easy way to your Icinga
OSMC 2021 | Icinga-Installer – the easy way to your IcingaOSMC 2021 | Icinga-Installer – the easy way to your Icinga
OSMC 2021 | Icinga-Installer – the easy way to your Icinga
 
Asian Spirit 3 Day Dba On Ubl
Asian Spirit 3 Day Dba On UblAsian Spirit 3 Day Dba On Ubl
Asian Spirit 3 Day Dba On Ubl
 
MySQL Spider Architecture
MySQL Spider ArchitectureMySQL Spider Architecture
MySQL Spider Architecture
 
Spider Setup with AWS/sandbox
Spider Setup with AWS/sandboxSpider Setup with AWS/sandbox
Spider Setup with AWS/sandbox
 
MySQL Audit using Percona audit plugin and ELK
MySQL Audit using Percona audit plugin and ELKMySQL Audit using Percona audit plugin and ELK
MySQL Audit using Percona audit plugin and ELK
 
26.1.7 lab snort and firewall rules
26.1.7 lab   snort and firewall rules26.1.7 lab   snort and firewall rules
26.1.7 lab snort and firewall rules
 
Configuration Management with Saltstack
Configuration Management with SaltstackConfiguration Management with Saltstack
Configuration Management with Saltstack
 
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...
Arnold Bechtoldt, Inovex GmbH Linux systems engineer - Configuration Manageme...
 
How to deploy docker container inside ikoula's cloud
How to deploy docker container inside ikoula's cloudHow to deploy docker container inside ikoula's cloud
How to deploy docker container inside ikoula's cloud
 
Commcon 2018
Commcon 2018Commcon 2018
Commcon 2018
 
How to configure esx to pass an audit
How to configure esx to pass an auditHow to configure esx to pass an audit
How to configure esx to pass an audit
 
X64服务器 lnmp服务器部署标准 new
X64服务器 lnmp服务器部署标准 newX64服务器 lnmp服务器部署标准 new
X64服务器 lnmp服务器部署标准 new
 
Mysql
Mysql Mysql
Mysql
 
Kl 031.30 eng_class_setup_guide_1.2
Kl 031.30 eng_class_setup_guide_1.2Kl 031.30 eng_class_setup_guide_1.2
Kl 031.30 eng_class_setup_guide_1.2
 
Architecting cloud
Architecting cloudArchitecting cloud
Architecting cloud
 
Oracle 11g R2 RAC setup on rhel 5.0
Oracle 11g R2 RAC setup on rhel 5.0Oracle 11g R2 RAC setup on rhel 5.0
Oracle 11g R2 RAC setup on rhel 5.0
 
Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1
Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1
Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1
 

Recently uploaded

Transformer Neural Network Use Cases with Links
Transformer Neural Network Use Cases with LinksTransformer Neural Network Use Cases with Links
Transformer Neural Network Use Cases with LinksJinanKordab
 
GraphSummit Milan - Neo4j: The Art of the Possible with Graph
GraphSummit Milan - Neo4j: The Art of the Possible with GraphGraphSummit Milan - Neo4j: The Art of the Possible with Graph
GraphSummit Milan - Neo4j: The Art of the Possible with GraphNeo4j
 
Evolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI EraEvolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI Eraconfluent
 
Your Ultimate Web Studio for Streaming Anywhere | Evmux
Your Ultimate Web Studio for Streaming Anywhere | EvmuxYour Ultimate Web Studio for Streaming Anywhere | Evmux
Your Ultimate Web Studio for Streaming Anywhere | Evmuxevmux96
 
Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Maxim Salnikov
 
BusinessGPT - Security and Governance for Generative AI
BusinessGPT  - Security and Governance for Generative AIBusinessGPT  - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AIAGATSoftware
 
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfThe Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfkalichargn70th171
 
Effective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeConEffective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeConNatan Silnitsky
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMarkus Moeller
 
The Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationThe Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationElement34
 
Rapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and InsightsRapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and Insightsrapidoform
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024MulesoftMunichMeetup
 
CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...
CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...
CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...Neo4j
 
Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...
Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...
Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...Abortion Clinic
 
Community is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletCommunity is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletAndrea Goulet
 
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Andreas Granig
 
Encryption Recap: A Refresher on Key Concepts
Encryption Recap: A Refresher on Key ConceptsEncryption Recap: A Refresher on Key Concepts
Encryption Recap: A Refresher on Key Conceptsthomashtkim
 

Recently uploaded (20)

Transformer Neural Network Use Cases with Links
Transformer Neural Network Use Cases with LinksTransformer Neural Network Use Cases with Links
Transformer Neural Network Use Cases with Links
 
GraphSummit Milan - Neo4j: The Art of the Possible with Graph
GraphSummit Milan - Neo4j: The Art of the Possible with GraphGraphSummit Milan - Neo4j: The Art of the Possible with Graph
GraphSummit Milan - Neo4j: The Art of the Possible with Graph
 
Evolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI EraEvolving Data Governance for the Real-time Streaming and AI Era
Evolving Data Governance for the Real-time Streaming and AI Era
 
Abortion Clinic In Springs ](+27832195400*)[ 🏥 Safe Abortion Pills in Springs...
Abortion Clinic In Springs ](+27832195400*)[ 🏥 Safe Abortion Pills in Springs...Abortion Clinic In Springs ](+27832195400*)[ 🏥 Safe Abortion Pills in Springs...
Abortion Clinic In Springs ](+27832195400*)[ 🏥 Safe Abortion Pills in Springs...
 
Your Ultimate Web Studio for Streaming Anywhere | Evmux
Your Ultimate Web Studio for Streaming Anywhere | EvmuxYour Ultimate Web Studio for Streaming Anywhere | Evmux
Your Ultimate Web Studio for Streaming Anywhere | Evmux
 
Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?
 
BusinessGPT - Security and Governance for Generative AI
BusinessGPT  - Security and Governance for Generative AIBusinessGPT  - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AI
 
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfThe Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
 
Effective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeConEffective Strategies for Wix's Scaling challenges - GeeCon
Effective Strategies for Wix's Scaling challenges - GeeCon
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdf
 
Abortion Pill Prices Germiston ](+27832195400*)[ 🏥 Women's Abortion Clinic in...
Abortion Pill Prices Germiston ](+27832195400*)[ 🏥 Women's Abortion Clinic in...Abortion Pill Prices Germiston ](+27832195400*)[ 🏥 Women's Abortion Clinic in...
Abortion Pill Prices Germiston ](+27832195400*)[ 🏥 Women's Abortion Clinic in...
 
The Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationThe Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test Automation
 
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
 
Rapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and InsightsRapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and Insights
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
 
CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...
CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...
CERVED e Neo4j su una nuvola, migrazione ed evoluzione di un grafo mission cr...
 
Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...
Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...
Abortion Pill Prices Jane Furse ](+27832195400*)[ 🏥 Women's Abortion Clinic i...
 
Community is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletCommunity is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea Goulet
 
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024Automate your OpenSIPS config tests - OpenSIPS Summit 2024
Automate your OpenSIPS config tests - OpenSIPS Summit 2024
 
Encryption Recap: A Refresher on Key Concepts
Encryption Recap: A Refresher on Key ConceptsEncryption Recap: A Refresher on Key Concepts
Encryption Recap: A Refresher on Key Concepts
 

OSMC 2019 | Use Cloud services & features in your redundant Icinga2 Environment by Marcel Weinberg

  • 1. Icinga2Icinga2 A highly available cloud setup deployed at DigitalOceanA highly available cloud setup deployed at DigitalOcean
  • 2. Who am I? | Short tales from the past Solution | Result - know your why! incl. the database & Galera cluster configuration Highlights and critical sections Don't do the same mistakes as I did Agenda > Quick Introduction > High level overview > Installation of all components > Configuration deep-dive > Pitfalls & Experienced limitations
  • 3. Who am I? Marcel Weinberg Living in Hamburg Working at CoreMedia AG (since 09/2019) Doing O(o)ps things for more then ten years now With a great passion for Automation, Monitoring, Open Source & Mountainbiking Quick Introduction @winem_ winem
  • 4. + extensive documentation + well defined & tested DR* procedure - single instance with many SPOFs - limited redundancy on the uplink Quick Introducation - Tales from the past Previous Setup It was a single instance in the server room: * DR = Disaster Recovery
  • 6. And we still have alarming! High Level Overview Solution Result
  • 8. Installation Of All Components Always refer to the public available documentation for up-to-date installation and upgrade instructions. > PHP7.3 - ppa:ondrej/php > icinga2, icingacli, icingaweb2 - packages.icinga.com apt install icinga2 icinga2-ido-mysql icingaweb2 icingacli vim-icinga2 vim-addon-manager php7.3-gd > haproxy - ppa:vbernat/haproxy-1.9 > keepalived from the standard repositories
  • 9. Installation Of All Components Always refer to the public available documentation for up-to-date installation and upgrade instructions. > PHP7.3 - ppa:ondrej/php > icinga2, icingacli, icingaweb2 - packages.icinga.com apt install icinga2 icinga2-ido-mysql icingaweb2 icingacli vim-icinga2 vim-addon-manager php7.3-gd > haproxy - ppa:vbernat/haproxy-1.9 > keepalived from the standard repositories keepalived_script - a dedicated user to run check and notification scripts: sudo adduser --no-create-home --disabled-login keepalived_script
  • 10. Installation Of All Components MariaDB Secure your installation and don't start mariadb automatically: sudo mysql_secure_installation sudo systemctl disable mariadb Link: https://downloads.mariadb.org/mariadb/repositories/
  • 11. Installation of all components MariaDB Con guration Files Use mysqld --help --verbose to see the loaded configuration files and sections: agent@m-do-ffm-galera-01:~$ mysql --help --verbose | head -n 10 mysql Ver 15.1 Distrib 10.4.8-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2 Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Usage: mysql [OPTIONS] [database] Default options are read from the following files in the given order: /etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf The following groups are read: mysql mariadb-client client client-server client-mariadb
  • 12. Installation of all components MariaDB Con guration Files Galera Cluster Con guration /etc/mysql/mariadb.conf.d/galera.cnf [mysqld] bind-address=10.135.209.4 default_storage_engine=InnoDB binlog_format=row innodb_autoinc_lock_mode=2 innodb_flush_log_at_trx_commit=0 # Galera cluster configuration wsrep_on=ON wsrep_provider=/usr/lib/galera/libgalera_smm.so wsrep_cluster_address="gcomm://10.135.209.4,10.135.207.79,10.135.207.111" wsrep_cluster_name="mariadb-galera-backend" wsrep_slave_threads=8 wsrep_sst_method=rsync # Cluster node configuration wsrep_node_address="10.135.209.4" wsrep_node_name="galera-01" > Highlighted lines show those values that are unique per node.
  • 13. Installation of all components MariaDB Con guration Files Galera Cluster Con guration /etc/mysql/mariadb.conf.d/galera.cnf [mysqld] bind-address=10.135.209.4 default_storage_engine=InnoDB binlog_format=row innodb_autoinc_lock_mode=2 innodb_flush_log_at_trx_commit=0 # Galera cluster configuration wsrep_on=ON wsrep_provider=/usr/lib/galera/libgalera_smm.so wsrep_cluster_address="gcomm://10.135.209.4,10.135.207.79,10.135.207.111" wsrep_cluster_name="mariadb-galera-backend" wsrep_slave_threads=8 wsrep_sst_method=rsync # Cluster node configuration wsrep_node_address="10.135.209.4" wsrep_node_name="galera-01" > Set wsrep_slave_threads to the number of CPUs * 2. > innodb_flush_log_at_trx_commit = 0 means to write the caches to disk every second.
  • 14. Start the first cluster node: sudo galera_new_cluster Start mariadb on the remaining nodes: sudo service mariadb start Installation of all components MariaDB Con guration Files |GaleraClusterCon guration Bootstrap the cluster Wait until all nodes joined the cluster and restart mariadb on the first node MariaDB [(none)]> show global status where Variable_name in ('wsrep_cluster_size', 'wsrep_local_state_comment', 'wsrep_ready', 'wsrep_connected'); +---------------------------+--------+ | Variable_name | Value | +---------------------------+--------+ | wsrep_local_state_comment | Synced | | wsrep_cluster_size | 3 | | wsrep_connected | ON | | wsrep_ready | ON | +---------------------------+--------+ 4 rows in set (0.002 sec)
  • 15. Database cluster, configuration & accounts icinga2 CREATE DATABASE icinga2; GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE VIEW, INDEX, EXECUTE ON icinga2.* TO 'icinga2'@'10.135.201.194' IDENTIFIED BY 'maa3EeHah0Ea'; GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE VIEW, INDEX, EXECUTE ON icinga2.* TO 'icinga2'@'10.135.221.140' IDENTIFIED BY 'maa3EeHah0Ea'; Copy the mysql.sql file to any of the galera nodes and import it: mysql -u root icinga2 < /tmp/mysql.sql Icingaweb2 CREATE DATABASE icingaweb2; GRANT ALL ON icingaweb2.* TO icingaweb2@10.135.201.194 IDENTIFIED BY 'ievachiYie4ooz5a'; GRANT ALL ON icingaweb2.* TO icingaweb2@10.135.221.140 IDENTIFIED BY 'ievachiYie4ooz5a'; GRANT SELECT, INSERT, UPDATE, DELETE, DROP, INDEX, EXECUTE, CREATE VIEW ON icinga2.* TO 'icingaweb2'@'10.135.201.194'; GRANT SELECT, INSERT, UPDATE, DELETE, DROP, INDEX, EXECUTE, CREATE VIEW ON icinga2.* TO 'icingaweb2'@'10.135.221.140';
  • 16. Database cluster, configuration & accounts HAProxy CREATE USER 'haproxy_checks'@'10.135.201.194'; CREATE USER 'haproxy_checks'@'10.135.221.140';
  • 17. Database cluster, configuration & accounts Keepalived GRANT SELECT ON icingaweb2.* TO 'keepalived_check'@'10.135.221.140' IDENTIFIED BY 'iiw8ahthe1Ch'; GRANT SELECT ON icingaweb2.* TO 'keepalived_check'@'10.135.221.249' IDENTIFIED BY 'iiw8ahthe1Ch'; my.cnf file for the keepalived healthches from the icinga2 nodes: sudo cat > /etc/keepalived/.my.cnf << EOF [client] host=127.0.0.1 password=iiw8ahthe1Ch user=keepalived_check EOF Set proper permissions: sudo chown keepalived_script:root /etc/keepalived/.my.cnf sudo chmod 600 /etc/keepalived/.my.cnf
  • 18. agent@m-do-ffm-plugins-01:~$ sudo icinga2 node setup --master --cn m-do-ffm-plugins-01.mydomain.com --disable-confd --accept-config --accept-commands --zone do-ffm-masters agent@m-do-ffm-plugins-02:~$ sudo icinga2 node setup --master --cn m-do-ffm-plugins-02.mydomain.com --disable-confd --accept-config --accept-commands --zone do-ffm-masters Deep-dive into the configuration Icinga2 - Node Setup Icinga2 configuration using the node setup utility
  • 19. agent@m-do-ffm-plugins-01:~$ sudo icinga2 node setup --master --cn m-do-ffm-plugins-01.mydomain.com --disable-confd --accept-config --accept-commands --zone do-ffm-masters agent@m-do-ffm-plugins-02:~$ sudo icinga2 node setup --master --cn m-do-ffm-plugins-02.mydomain.com --disable-confd --accept-config --accept-commands --zone do-ffm-masters Deep-dive into the configuration Icinga2 - Node Setup Icinga2 configuration using the node setup utility /etc/icinga2/constants.conf [...] const NodeName = "m-do-ffm-plugins-01.mydomain.com" /* Our local zone name. */ const ZoneName = "do-ffm-masters" /* Secret key for remote node tickets */ const TicketSalt = "689eb691e3d5cbf8e3a11fcb50089610" [...]
  • 20. Deep-dive into the configuration Icinga2 - Zone & endpoint con guration /etc/icinga2/zones.conf object Endpoint "m-do-ffm-plugins-01.mydomain.com" { host = "10.135.201.194" } object Endpoint "m-do-ffm-plugins-02.mydomain.com" { host = "10.135.221.140" } object Zone "do-ffm-masters" { endpoints = [ "m-do-ffm-plugins-01.mydomain.com", "m-do-ffm-plugins-02.mydomain.com" ] } object Zone "global-templates" { global = true } object Zone "director-global" { global = true }
  • 21. Deep-dive into the configuration Icinga2 - ido-mysql feature The configuration is on both nodes the same. /etc/icinga2/features-available/ido-mysql.conf library "db_ido_mysql" object IdoMysqlConnection "ido-mysql" { user = "icinga2", password = "maa3EeHah0Ea", host = "127.0.0.1", database = "icinga2" enable_ha = true cleanup = { contactnotifications_age = 30d downtimehistory_age = 60d statehistory_age = 60d externalcommands_age = 2d } } agent@m-do-ffm-plugins-01:~$ icinga2 feature enable ido-mysql agent@m-do-ffm-plugins-01:~$ sudo icinga2 feature list Disabled features: command compatlog debuglog elasticsearch gelf graphite influxdb livestatus opentsdb perfdata statusdata syslog Enabled features: api checker ido-mysql mainlog notification
  • 22. Deep-dive into the configuration Icinga2 - API users /etc/icinga2/zones.d/do-ffm-masters/api-users.conf /** * The ApiUser objects are used for authentication against the API. */ object ApiUser "api-healthchecks" { password = "eiy8aiGh0aog" permissions = [ "actions/process-check-result" ] } object ApiUser "icingaweb2" { password = "Eila5iRaed2o" permissions = [ "status/query", "actions/*", "objects/modify/*", "objects/query/*" ] } object ApiUser "keepalived" { password = "paigohLe8soh" permissions = [ "status/query" ] }
  • 23. Deep-dive into the configuration Icinga2 - Valdiate & apply the adjusted con guration Validate the new configuration agent@m-do-ffm-plugins-01:~$ sudo icinga2 daemon -C [2019-09-28 23:29:05 +0000] information/cli: Icinga application loader (version: r2.11.0-1) [2019-09-28 23:29:05 +0000] information/cli: Loading configuration file(s). [2019-09-28 23:29:06 +0000] information/ConfigItem: Committing config item(s). [2019-09-28 23:29:06 +0000] information/ApiListener: My API identity: m-do-ffm-plugins-01.mydomain.com [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 FileLogger. [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 NotificationComponent. [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 IcingaApplication. [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 CheckerComponent. [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 3 Zones. [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 2 Endpoints. [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 3 ApiUser. [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 ApiListener. [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 1 IdoMysqlConnection. [2019-09-28 23:29:06 +0000] information/ConfigItem: Instantiated 235 CheckCommands. [2019-09-28 23:29:06 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars' [2019-09-28 23:29:06 +0000] information/cli: Finished validating the configuration file(s).
  • 24. Deep-dive into the configuration Icinga2 HAProxy - Global and default con guration for both nodes global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd defaults log global option httplog option dontlognulltimeout timeout connect 5000 timeout client 5000 timeout server 5000 retries 2
  • 25. Deep-dive into the configuration Icinga2 HAProxy - General listener con guration & statistics endpoint listeners stats timeout 30s user haproxy group haproxy daemon listen ha-stats bind 127.0.0.1:8026 mode http stats enable stats uri / stats realm Strictly Privatestats auth icinga_monitor:chong4gohCe8 > The ha-stats endpoint exposes statistics about haproxy, the configured listeners and backends. > Useful for debugging, testing and monitoring. > Keep it secure! (see the last line and setup firewall rules)
  • 26. Deep-dive into the configuration Icinga2 HAProxy Special thanks to Carsten! @Mikeschova https://community.icinga.com/t/galera-mysql-cluster-with-vips-and-haproxy-for-ido-mysql-and-more/
  • 27. Deep-dive into the configuration Icinga2 HAProxy - 3 di erent listeners for the MariaDB backend > use galera-01 and galera-02 as prefered endpoint for write-intensive access (port 3306) > balance read queries round robin across all 3 nodes; individual weight of each galera node varies between the two icinga2 nodes (port 3307) > the director has probably an equal distribution of read and write queries (port 3308)
  • 28. Deep-dive into the configuration Icinga2 HAProxy Listener to provide database access to for icinga2 # 1st node listen icinga2_ido bind 127.0.0.1:3306 mode tcp option mysql-check user haproxy_checks balance source server do-ffm-galera01 10.135.209.4:3306 check weight 1 inter 5s rise 2 fall 2 server do-ffm-galera02 10.135.207.79:3306 check weight 2 inter 5s rise 2 fall 2 backup server do-ffm-galera03 10.135.207.11:3306 check weight 3 inter 5s rise 2 fall 2 backup # 2nd node listen icinga2_ido bind 127.0.0.1:3306 mode tcp option mysql-check user haproxy_checks balance source server do-ffm-galera01 10.135.209.4:3306 check weight 2 inter 5s rise 2 fall 2 backup server do-ffm-galera02 10.135.207.79:3306 check weight 1 inter 5s rise 2 fall 2 server do-ffm-galera03 10.135.207.11:3306 check weight 3 inter 5s rise 2 fall 2 backup
  • 29. Deep-dive into the configuration Icinga2 HAProxy Listener to provide database access for icingaweb2 and handle mostly read queries # 1st node listen mysqlread bind 127.0.0.1:3308 mode tcp option mysql-check user haproxy_checks balance roundrobin server do-ffm-galera01 10.135.209.4:3306 check weight 2 inter 5s rise 2 fall 2 server do-ffm-galera02 10.135.207.79:3306 check weight 3 inter 5s rise 2 fall 2 server do-ffm-galera03 10.135.207.11:3306 check weight 1 inter 5s rise 2 fall 2 # 2nd node listen mysqlread bind 127.0.0.1:3308 mode tcp option mysql-check user haproxy_checks balance roundrobin server do-ffm-galera01 10.135.209.4:3306 check weight 1 inter 5s rise 2 fall 2 server do-ffm-galera02 10.135.207.79:3306 check weight 3 inter 5s rise 2 fall 2 server do-ffm-galera03 10.135.207.11:3306 check weight 2 inter 5s rise 2 fall 2
  • 30. Deep-dive into the configuration Icinga2 HAProxy Listener to provide access to the icingaweb2 director database backend listen director bind 127.0.0.1:3307 mode tcp option mysql-check user haproxy_checks balance source server do-ffm-galera01 10.135.209.4:3306 check weight 3 inter 5s rise 2 fall 2 backup server do-ffm-galera02 10.135.207.79:3306 check weight 2 inter 5s rise 2 fall 2 backup server do-ffm-galera03 10.135.207.11:3306 check weight 1 inter 5s rise 2 fall 2 > This is the same for both nodes again.
  • 31. Deep-dive into the configuration Icinga2|HAProxy Apache2 & PHP Virtual host configuration: > set proper date.timezone in php.ini > Enforce encryption and redirect HTTP requests to HTTPS: Redirect permanent / https://moni.mydomain.com > Rewrite empty URIs to /icingaweb2: RewriteRule ^(.*) http://%{HTTP_HOST}/icingaweb2 > Enable the required modules and the new site: sudo a2enmod proxy_http rewrite ssl sudo a2ensite moni.mydomain.com
  • 32. Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP Icingaweb2 - https://moni.mydomain.com/Setup > Protected custom variables (pw, pass, password, hmac, community)
  • 33. Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP Icingaweb2 Add a dedicated user to sync /etc/icingaweb2/ between both icinga2 nodes. agent@m-do-ffm-plugins-01:~$ sudo adduser --disabled-password icingaweb_sync agent@m-do-ffm-plugins-01:~$ sudo su - icingaweb_sync icingaweb_sync@m-do-ffm-plugins-01:~$ ssh-keygen -t rsa -b 4096 Add the public key to the authorized_keys file. Setup the crontab on both nodes agent@m-do-ffm-plugins-01:~$ sudo cat /etc/cron.d/icingaweb_sync MAILTO="" * * * * * icingaweb_sync <ip-of-the-peer-node>:/etc/icingaweb2 rsync -rugopl /etc/icingaweb2/
  • 34. > the system user keepalived_script > the .my.cnf file in /etc/keepalived > the icinga2 API user keepalived to perform healthchecks > the icinga2 API user api_healthchecks to send the check results to icinga > the database user keepalived_checks to test the connection to the DB Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP|Icingaweb2 Keepalived Remember what we already have:
  • 35. vrrp_script check_apache2 { script "/usr/bin/killall -0 apache2" interval 2 fall 1 rise 5 user root } vrrp_script check_icinga2_api { script "/usr/local/bin/check_icinga_api" interval 2 fall 3 rise 3 } Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP|Icingaweb2 Keepalived - /etc/keepalived/keepalived.conf (Healthchecks) vrrp_script check_mysql_conn { script "mysql --defaults-file=/etc/keepalived/.my.cnf -e 'select count(*) from icingaweb2.icingaweb_user'" interval 2 fall 1 rise 3 }
  • 36. Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP|Icingaweb2 Keepalived - Icinga2 API Healthcheck /usr/local/bin/check_icinga_api - a simple healthcheck #!/usr/bin/env bash icinga_api_user='keepalived' icinga_api_pass='paigohLe8soh' icinga_api_timeout=1 api_rc=$(curl -k -s -o /dev/null -w %{http_code} --max-time $icinga_api_timeout -u $icinga_api_user:$icinga_api_pass https://localhost:5665/v1/status/ApiListener) if [ $api_rc -eq 200 ]; then exit 0 else exit 1 fi
  • 37. global_defs { enable_script_security } vrrp_instance icinga_fe { state BACKUP nopreempt interface eth1 virtual_router_id 10 priority 100 advert_int 1 authentication { auth_type PASS auth_pass CaemooXah1Ke } unicast_src_ip 10.135.201.194 unicast_peer { 10.135.221.140 } global_defs { enable_script_security } vrrp_instance icinga_fe { state BACKUP nopreempt interface eth1 virtual_router_id 10 priority 100 advert_int 1 authentication { auth_type PASS auth_pass CaemooXah1Ke } unicast_src_ip 10.135.201.140 unicast_peer { 10.135.221.194 } Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP|Icingaweb2 Keepalived - /etc/keepalived/keepalived.conf (global_defs & vrrp_instance)
  • 38. Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP|Icingaweb2 Keepalived - /etc/keepalived/keepalived.conf (track & notify scripts) track_script { check_apache2 check_mysql_conn check_icinga2_api } notify_backup "/usr/local/bin/keepalived-alarming BACKUP" notify_fault "/usr/local/bin/keepalived-alarming FAULT" notify_master /etc/keepalived/master.sh > /etc/keepalived/master.sh triggers keepalived-alarming MASTER
  • 39. Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP|Icingaweb2 Keepalived - Script to assign the oating IP to the new master #!/usr/bin/env bash floating_ip='68.183.240.112' droplet_id=$(curl -s http://169.254.169.254/metadata/v1/id) has_floating_ip=$(curl -s http://169.254.169.254/metadata/v1/floating_ip/ipv4/active) export DO_TOKEN='ohjajeelaex8oongiequuxiethae4ing1jooch2ezieP2Shil6ohei8ierei3oizi' if [ $has_floating_ip = "false" ]; then n=0 while [ $n -lt 5 ]; do python3 /usr/local/bin/assign-ip $floating_ip $droplet_id && break n=$((n+1)) sleep 3 done fi keepalived-alarming MASTER Monitor floating IP action via https://api.digitalocean.com/v2/floating_ips/<floating-ip>/actions?page=1&per_page=1 . Possible states: completed, in-progress or failed.
  • 40. MASTER & BACKUP = 0 FAULT = 2 Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP|Icingaweb2 Keepalived - Noti cation Script (Snippets) /usr/local/bin/keepalived-alarming #/usr/bin/env bash # $1 = target state of transition ("MASTER", "BACKUP", "FAULT") t_state=$1 chk_message="Keepalived entered $t_state state." icinga_chk_host=$(hostname) # 1st node endpoints icinga_endpoints=("127.0.0.1" "10.135.221.140") Use a case statement to translate t_state as following:
  • 41. Deep-dive into the configuration Icinga2|HAProxy|Apache2&PHP|Icingaweb2 Keepalived - Noti cation Script (Snippets) Construct the JSON for the icinga2 API call: icinga_post_data() { cat <<EOF { "service": "${icinga_chk_host}!${icinga_service}", "exit_status": "$e_status", "plugin_output": "$chk_message", "performance_data": [ "entered_keepalived_state=$new_state;;2;0;2", ] } EOF }
  • 42. Pitfalls & Don't Dos > Don't stop all galera nodes at the same time! If that happens, you have to bootstrap the cluster again. > Don't forget to set enable_ha = true for the ido-mysql feature > Don't create the rsync user without --disabled-password > Don't rely on the volatile instance / droplet IPs. Use floating IPs > Don't forget backups / snapshots: droplets, database content, configuration & setup > Don't miss to migrate to the director if you have the chance to > Setup a local firewall if the cloud providers firewall does not support VRRP2
  • 43. Firewall Rules > Communication between the Icinga2 masters TCP/5665, VRRP2 > Communication from the satellites and agents to the master nodes TCP/5665 > Icinga2 nodes to the galera nodes TCP/3306 , TCP/3307 , TCP/3308 > Communication between the galera cluster nodes TCP/4567, TCP/4568, UDP/4567 > Restrictive ipset & iptables How-To: https://community.icinga.com/t/firewall-setup-at-a-icinga2-cluster-with-iptables-ipset/2156