CfgMgmtCamp 2023 - Puppet is YAML.pdf

1. © betadots GmbH 2023 © betadots GmbH 2023 Puppet is YAML or The power of Hiera 5 ConfigManagementCamp 2023 Martin Alfke ma@betadots.de

2. © betadots GmbH 2023 Martin Alfke CEO/Consultant/Trainer at betadots GmbH Berlin, Germany • Puppet Trainer and Puppet Solution Engineer • Platform Engineering, Consulting and Training • Agile methods, Scrum • tuxmea (Twitter, GitHub, Slack) Puppet is YAML - cfgmgmtcamp 2023

3. © betadots GmbH 2023 Puppet is YAML - Topics - Starting with Puppet is hard - YAML is simple - Hiera is YAML - YAML node classification - YAML resource declaration - YAML limitations of resource declaration - Usage of Library Modules in YAML - Puppet Plans in YAML - Summary Puppet is YAML - cfgmgmtcamp 2023

4. © betadots GmbH 2023 © betadots GmbH 2023 Why Puppet is hard

5. © betadots GmbH 2023 Starting with Puppet is hard People new to Puppet IT Automation must learn many new things: - GIT and Control-Repo - Facter - Node classification - Puppet DSL - Hiera - Modules - PDK and Onceover How much can you learn in 3 days? Puppet is YAML - cfgmgmtcamp 2023

6. © betadots GmbH 2023 Starting with Puppet is hard using YAML How can we reduce learning time needed? Is there something we can skip in the beginning? - GIT and Control-Repo - Facter - Node classification (partly) - Puppet DSL - Hiera (YAML) - Modules (partly) - PDK and Onceover Puppet is YAML - cfgmgmtcamp 2023

7. © betadots GmbH 2023 Hiera is YAML - Hiera was introduced to separate code and data - Layers of hierarchies provide different options for different configs - Hierarchies are based on facts - OS - Datacenter/Network zone - Application and Service - Stage (Dev, Test, Prod) - Most people use YAML in Hiera Puppet is YAML - cfgmgmtcamp 2023

8. © betadots GmbH 2023 Puppet uses Hiera - Puppet can query Hiera for data - explicit lookup - automatic data binding (from classes) - Puppet can query ANY data from Hiera Puppet is YAML - cfgmgmtcamp 2023

9. © betadots GmbH 2023 © betadots GmbH 2023 Node classification in YAML

10. © betadots GmbH 2023 Node classification - Many people still pray the roles and profile pattern for node classification - Profiles use Library Modules to implement technical settings - Roles reflect the system business use case But: - Roles make sense only if one has many similar systems or - If you insist in static node classification - Roles and Profiles need an understanding of Puppet DSL and Modules Puppet is YAML - cfgmgmtcamp 2023

11. © betadots GmbH 2023 Node classification using Hiera Array Query Hiera for classes to add to the nodes catalog: # manifests/site.pp lookup( { 'name' => 'classes', 'value_type' => Array, 'default_value' => [], } ).include Puppet is YAML - cfgmgmtcamp 2023

12. © betadots GmbH 2023 Node classification using Hiera Array Hierarchy structure allows one to separate base from OS and from application classes: # Classes which are needed on all systems data/common.yaml # OS specific classes data/os/%{facts.os.name}-%{facts.os.release.major}.yaml # Application specific classes data/app/%{trusted.extension.pp_application}-%{trusted.ext ension.pp_service}-%{trusted.extension.pp_stage}.yaml Puppet is YAML - cfgmgmtcamp 2023

13. © betadots GmbH 2023 Node classification using Hiera Array Configure Hiera to lookup classes from ALL hierarchies: # data/common.yaml --- lookup_options: 'classes': merge: 'unique' classes: - 'class_a' - 'class_b' Puppet is YAML - cfgmgmtcamp 2023

14. © betadots GmbH 2023 Node classification using Hiera Array Downside of Hiera Array node classification - Classes can only be added - Classes can not be overwritten - There is no possibility to remove a class in higher hierarchy - One only can set the merge behavior on a higher Hierarchy (first), omitting all other classes arrays Solution: Hiera Hash node classification Puppet is YAML - cfgmgmtcamp 2023

15. © betadots GmbH 2023 Node classification using Hiera Hash Query Hiera for classes to add to the nodes catalog: # manifests/site.pp 1 lookup( 'classes_hash', 2 { 3 'value_type' => Hash, 4 'default_value' => {}, 5 } 6 ).each |$name, $c| { 7 unless $c.empty { 8 contain $c 9 } else { 10 # needs ipcrm/echo module 11 echo { "Class ${name} on ${facts['networking']['fqdn']} is disabled": 12 withpath => false, 13 } 14 } 15 } Puppet is YAML - cfgmgmtcamp 2023

16. © betadots GmbH 2023 Node classification using Hiera Hash Configure Hiera to lookup classes Hash from ALL hierarchies: # data/common.yaml --- lookup_options: 'classes_hash': merge: 'deep' classes_hash: 'description of class_a': 'class_a' 'description of class_b': 'class_b' Puppet is YAML - cfgmgmtcamp 2023

17. © betadots GmbH 2023 Node classification using Hiera Hash Disabling a class on a higher Hierarchy: # data/node/rz12pw5jz.domain.tld.yaml --- classes_hash: 'description of class_a': '' # ← An empty string 'description of class_c': 'class_c' Puppet is YAML - cfgmgmtcamp 2023

18. © betadots GmbH 2023 Node classification using Hiera Hash This is a flexible solution where you can even query OS based classes: # manifests/site.pp lookup( "${facts['kernel'].downcase}_classes_hash", { 'value_type' => Hash, 'default_value' => {} } ).each |$name, $c| { unless $c.empty { contain $c } else { echo { "Class ${name} on ${facts['networking']['fqdn']} is disabled": } # ← needs ipcrm/echo module } } Puppet is YAML - cfgmgmtcamp 2023

19. © betadots GmbH 2023 Node classification using Hiera Hash Configure Hiera to lookup classes Hash from ALL hierarchies: # data/common.yaml --- lookup_options: "(.*)_classes_hash": merge: 'deep' linux_classes_hash: 'description of class_a': 'class_a' 'description of class_b': 'class_b' windows_classes_hash: 'description of class_a': 'class_a' 'description of class_c': 'class_c' Puppet is YAML - cfgmgmtcamp 2023

20. © betadots GmbH 2023 © betadots GmbH 2023 Resource declaration in YAML

21. © betadots GmbH 2023 Resource declaration in YAML Resource types are a core concept of Puppet. Every resource type describes a small portion to be configured on a system. Core resource types are part of Puppet Agent installation. Resource Type declaration via Hiera is possible using the stdlib::manage class since version 8.2.0. The stdlib Library Module must be added to Puppetfile # Puppetfile forge: baseurl: https://forgeapi.puppetlabs.com/ mod 'ipcrm-echo', '0.1.7' mod 'puppetlabs/stdlib', '8.5.0' # needs 8.2.0 or newer Puppet is YAML - cfgmgmtcamp 2023

22. © betadots GmbH 2023 Resource declaration in YAML Configure Hiera to use the stdlib::manage class and to fetch data from all hierarchies # data/common.yaml --- lookup_options: 'classes_hash': merge: 'deep' 'stdlib::manage::create_resources': merge: 'deep' classes_hash: 'hiera_yaml_resources': 'stdlib::manage' Puppet is YAML - cfgmgmtcamp 2023

23. © betadots GmbH 2023 Resource declaration in YAML The stdlib::manage class has a parameter (create_resources). Any resource will be created programmatically from this data hash. The data hash has the following syntax: stdlib::manage::create_resources: '<resource type>': '<title or name>': '<parameters of the type>': '<value>' stdlib::manage::create_resources: 'package': 'htop': 'ensure': 'installed' Puppet is YAML - cfgmgmtcamp 2023

24. © betadots GmbH 2023 Resource declaration in YAML Now we can add core resources to YAML data # data/os/RedHat-8.yaml --- stdlib::manage::create_resources: 'package': 'chrony': 'ensure': 'installed' 'file': '/etc/chrony.conf': 'ensure': 'file' 'source': 'http://server/path/file' 'service': 'chrony': 'ensure': 'running' 'enable': true Puppet is YAML - cfgmgmtcamp 2023

25. © betadots GmbH 2023 Resource declaration in YAML Static files should be part of the control-repo so they are also under version control. Files can be added to a module and Puppet must know where to find modules: 1. configure modulepath # environment.conf modulepath=site:modules:$basemodulepath 2. add profile module and files directory mkdir -p site/profile/files 3. add config file vi site/profile/files/chrony.conf Puppet is YAML - cfgmgmtcamp 2023

26. © betadots GmbH 2023 Resource declaration in YAML Switch file to Puppet code location # data/os/RedHat-8.yaml --- stdlib::manage::create_resources: 'package': 'chrony': 'ensure': 'installed' 'file': '/etc/chrony.conf': 'ensure': 'file' 'source': 'puppet:///modules/profile/chrony.conf' 'service': 'chrony': 'ensure': 'running' 'enable': true Puppet is YAML - cfgmgmtcamp 2023

27. © betadots GmbH 2023 Resource declaration in YAML Resource defaults are possible using anchors and aliases (but only within the same YAML file) First we set the anchor: # data/app/zoofoo-web-dev.yaml --- file_defaults: @file_defaults 'owner': 'zoofoo' 'group': 'zoofoo' 'mode': '0644' Puppet is YAML - cfgmgmtcamp 2023

28. © betadots GmbH 2023 Resource declaration in YAML Now we can use the alias: # data/app/zoofoo-web-dev.yaml --- stdlib::manage::create_resources: 'file': '/etc/zoofoo': << : *file_defaults 'ensure': 'directory' '/etc/zoofoo/app.cfg': << : *file_defaults 'ensure': 'file' Puppet is YAML - cfgmgmtcamp 2023

29. © betadots GmbH 2023 Limitations of resource declaration in YAML 1. it is not possible to execute Puppet functions within Hiera data e.g. content: "%{epp('profile/chrony.conf.epp')}" https://tickets.puppetlabs.com/browse/HI-638 https://github.com/voxpupuli/hiera-eyaml/issues/336 Puppet is YAML - cfgmgmtcamp 2023

30. © betadots GmbH 2023 Limitations of resource declaration in YAML 2. it is not possible to set a hash key to an array in Hiera data (yes, this is valid YAML) e.g. stdlib::manage::create_resources: 'package': ['htop', 'less', vim']: 'ensure': 'installed' https://tickets.puppetlabs.com/browse/HI-637 https://github.com/voxpupuli/hiera-eyaml/issues/337 Puppet is YAML - cfgmgmtcamp 2023

32. © betadots GmbH 2023 Usage of Library Modules in YAML Library Modules offer the possibility to configure technical components and are made available on Puppet Forge. Carefully check which module you want to use (badges, author reputation, last release date, open issues, active development, code review). Modern modules allow settings to be configured via Hiera data # data/node/rz12pw5jz.domain.tld.yaml --- classes_hash: 'webserver': 'nginx' nginx::port: 8080 Puppet is YAML - cfgmgmtcamp 2023

33. © betadots GmbH 2023 Usage of Library Modules in YAML Defined types (based on existing resource types) are used to configure component specific settings and can be added using stdlib::manage: # data/node/rz12pw5jz.domain.tld.yaml --- classes_hash: 'webserver': 'nginx' stdlib::manage::create_resources: 'nginx::resource::server': # ← Defined Type 'www.domain.tld': 'listen_port': 80 'proxy': 'http://localhost:8088' Puppet is YAML - cfgmgmtcamp 2023

35. © betadots GmbH 2023 Puppet Plans in YAML Puppet/Bolt Plans can be written in PuppetDSL or YAML # site/profile/plans/zoofoo/install.yaml --- parameters: version: type: 'String' description: 'Version of ZooFoo to deploy' frontends: type: 'TargetSpec' description: 'The frontend web servers' backends: type: 'TargetSpec' description: 'The backend servers' Puppet is YAML - cfgmgmtcamp 2023

36. © betadots GmbH 2023 Puppet Plans in YAML Puppet/Bolt Plans can be written in PuppetDSL or YAML # site/profile/plans/zoofoo/install.yaml - continued --- steps: - name: 'zoofoo_fe' task: 'profile::zoofoo_install' targets: $frontends description: 'Install zoofoo frontends' parameters: version: $version return: $zoofoo_fe.map |result| { result['stdout']} Puppet is YAML - cfgmgmtcamp 2023

38. © betadots GmbH 2023 Summary YAML hash based node classification is flexible, extensible and mature and should be considered best practice. Puppet YAML resources allows an easy start but has its limitations. Keep your YAML data - simple - obvious - flexible Don't be afraid of long YAML files and use an IDE Validate your YAML data in CI/CD Visualize your Hiera Data in a web interface (Hiera Data Manager) Puppet is YAML - cfgmgmtcamp 2023

39. © betadots GmbH 2023 Summary YAML HELL Check proper YAML syntax! Quote Strings (especially if the string starts with digits)!!!!!!! Quote Regexp (especially when starting with & or *) Quote no as it is a Boolean. Quote keys, see all of the above Quote Version String. Unquoted can lead to unintentional numbers https://ruudvanasseldonk.com/2023/01/11/the-yaml-document-from-hell Puppet is YAML - cfgmgmtcamp 2023

40. © betadots GmbH 2023 Summary Module Authors: - please provide Hiera YAML data examples - please add code documentation and use puppet-strings Puppet and Voxpupuli - please allow Hiera hash key as array (HI-637 and hiera-eyaml#337) - please allow Puppet function class from Hiera (HI-638 and hiera-eyaml#336) Puppet is YAML - cfgmgmtcamp 2023

41. © betadots GmbH 2023 Summary More complex configurations must be done in Puppet DSL modules or classes. e.g. - you need to manage an application on several OS - there is more to do than only add some resource types While using YAML, one can start exploring and learning on how to write Puppet DSL code. Puppet is YAML - cfgmgmtcamp 2023

42. © betadots GmbH 2023 Summary Use Hiera Data Manager to visualize your YAML data https://github.com/betadots/hdm Puppet is YAML - cfgmgmtcamp 2023

CfgMgmtCamp 2023 - Puppet is YAML.pdf

You are reading a preview.

Check these out next

CfgMgmtCamp 2023 - Puppet is YAML.pdf

More Related Content

Similar to CfgMgmtCamp 2023 - Puppet is YAML.pdf (20)

More from Martin Alfke (17)

Recently uploaded (20)