Successfully reported this slideshow.
Your SlideShare is downloading. ×

OSMC 2021 | Still directing the director… and more!

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 22 Ad

OSMC 2021 | Still directing the director… and more!

Download to read offline

For the monitoring of our systems, we make extensive use of Icinga, its director, and the business process monitoring module. We also make broad use of automation (at least we try to!). In this talk we would like to tell you how we automated the monitoring of our services using our self-written Ansible collections. We will cover how we developed the Ansible components and how we use them. We’ll also show you what we plan to do with them in the future.

For the monitoring of our systems, we make extensive use of Icinga, its director, and the business process monitoring module. We also make broad use of automation (at least we try to!). In this talk we would like to tell you how we automated the monitoring of our services using our self-written Ansible collections. We will cover how we developed the Ansible components and how we use them. We’ll also show you what we plan to do with them in the future.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to OSMC 2021 | Still directing the director… and more! (20)

Advertisement

Recently uploaded (20)

OSMC 2021 | Still directing the director… and more!

  1. 1. Still directing the Director and more! Page 1 / 22
  2. 2. Contents The Monitoring History The Director-module The Business Process-module The Icinga-Collection Q&A Page 2 / 22
  3. 3. About us Daniel Uhlmann Sebastian Gumprich Page 3 / 22
  4. 4. Directing the Director in 2019 Anyone remember this guy? Page 4 / 22
  5. 5. What we did with our architecture Central kubernetes cluster with icinga instances at least 45 Icinga instances Page 5 / 22
  6. 6. What problems we faced with our code No exsisting upstream modules instead use Ansible uri-Tasks Page 6 / 22
  7. 7. What we did with our Ansible code - name: create service template if it does not exist   uri:     headers:       Accept: application/json     url: "{{ icinga_host }}/director/service"     method: POST     user: "{{ icinga_user }}"     password: "{{ icinga_pass }}"     body: '{"check_command":"{{ item.item.check_command }}", "object_name":"{{ item.item.name }}", "object_type": "template", "use_agent":{{ item.item.use_agent }}}'     body_format: json     return_content: yes   register: service_template_created   with_items: "{{ service_template_presence.results }}"   when: "'error' in item.content"   changed_when: service_template_created.status == 201   failed_when: service_template_created.status != 201 Page 7 / 22
  8. 8. What we're doing now with the code - name: Create servicetemplate with event command t_systems_mms.icinga_director.icinga_service_template: state: present url: "{{ icinga_url }}" url_username: "{{ icinga_user }}" url_password: "{{ icinga_pass }}" object_name: apache_check_servicetemplate use_agent: false event_command: restart_httpd notes: "example note" notes_url: "'http://url1' 'http://url2'" Page 8 / 22
  9. 9. What we accomplished Date Feature 04/2020 internal development started 05/2020 Modules on Github in v1.0.0 05/20 - 12/20 command_template, host_template, zones and endpoints, service 01/21 inclusion into Ansible 02/21 facts/info modules for all objects 03/21 usergroups, timeperiod_template 07/21 inventory-plugin Page 9 / 22
  10. 10. Facts-/Info-modules > ansible localhost -m t_systems_mms.icinga_director.icinga_host_info -a "query=customer-test-web01" localhost | SUCCESS => { "changed": false, "objects": [ { "address": "172.29.23.203", "display_name": "customer-test-web01", "imports": [ "StandardServer" ], "object_name": "customer-test-web01", "object_type": "object" } ] } Page 10 / 22
  11. 11. Inventory Plugin > ansible-inventory -i test.icinga_director_inventory.yaml --graph @all: |--@customer_abn: | |--customer-abn-internet | |--customer-abn-web01 |--@customer_dev: | |--customer-dev-app01 | |--customer-dev-internet | |--customer-dev-web01 |--@customer_live: | |--customer-live-app01 | |--customer-live-db01 | |--customer-live-db02 | |--customer-live-internet Page 11 / 22
  12. 12. Inventory Plugin object Host "customer-live-web01" { display_name = "customer-live-web01" address = "172.29.23.204" check_command = "hostalive" max_check_attempts = "3" check_period = "24/7" retry_interval = 1m enable_notifications = true enable_active_checks = true enable_passive_checks = false enable_event_handler = false enable_perfdata = true volatile = false vars.HostOS = "Linux" } Page 12 / 22
  13. 13. Inventory Plugin > ansible customer-live-web01 -m debug -a "var=hostvars[inventory_hostname]['vars']['HostOS']" customer-live-web01 | SUCCESS => { "hostvars[inventory_hostname]['vars']['HostOS']": "Linux" } Page 13 / 22
  14. 14. Icinga Director Stats Page 14 / 22
  15. 15. Let's talk about the Business Processes Plugin (BP) Optional Icinga2 module Build custom process-based dashboards Get visualized Overview over your monitored objects Page 15 / 22
  16. 16. Example Business Process Node Page 16 / 22
  17. 17. Collection development First things first: manually play around with every feature Check the requests made by the extension (we need those for the upload or check part f.e) Our first draft was a Ansible role: {% if node.operator == 'or' -%} {{ node.name }} = {{ checks|join(' | ') }} {% elif node.operator == 'not' -%} {{ node.name }} = {{ checks|join(' ! ') }} {% elif node.operator == 'and' -%} {{ node.name }} = {{ checks|join(' & ') }} {% elif node.operator is defined -%} {{ node.name }} = {{ node.operator }}: {{ checks|join(' + ') }} {% endif %} Page 17 / 22
  18. 18. Configuration overview --- - hosts: localhost roles: - t_systems_mms.icinga_business_process.ansible_icinga_business_process vars: icinga_business_processes: - title: project-live description: Project Live Environment owner: "{{ icinga_user }}" menu: yes statetype: soft nodes: - name: loadbalancer displayname: Loadbalancer Pools operator: and visible: false checks: - name: lb01 type: service service: lb_pool_project-live - name: lb02 type: service service: lb_pool_project-live Page 18 / 22
  19. 19. The future At this time all functionalities are already implemented Create service templates as check commands to integrate monitoring-alerts - service_apply_object: - "check_bpm" imports: - domon-business-process assign_filter: 'host.name="pseudo-host"' vars: config: $bpm_servive_name process: $bpm_process Page 19 / 22
  20. 20. Yet another Ansible Icinga collection? Page 20 / 22
  21. 21. Yet another Ansible Icinga collection? One Collection to rule them all! https://github.com/T-Systems-MMS/ansible-collection-icinga Page 21 / 22
  22. 22. Ending + Questions Sebastian Gumprich Sebastian.Gumprich@t-systems.com https://github.com/rndmh3ro Daniel Uhlmann Daniel.Uhlmann@t-systems.com https://github.com/xFuture603 https://twitter.com/xFuturecs Page 22 / 22

×