Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Puppet Camp Paris 2016 Data in Modules

703 views

Published on

Data in Modules and Environments
A guide for Hiera and Data Functions in Puppet Modules and Environments - providing code and configuration examples.

Published in: Internet
  • Be the first to comment

Puppet Camp Paris 2016 Data in Modules

  1. 1. Copyright example42 GmbH - 2016 Puppet 4 - Data in Modules PuppetCamp Paris 2016 Martin Alfke - ma@example42.com Image: Tatlin - tatlin.net Copyright example42 GmbH - 2016
  2. 2. Copyright example42 GmbH - 2016 Martin Alfke ! Berlin/Germany ! CEO example42 GmbH Freelance Puppet Expert Network ! Puppet since 2007 ! Puppet Trainer, Consultant ! Co-Author of “Puppet 4 Essentials”
  3. 3. Copyright example42 GmbH - 2016 Puppet 4 Data in Modules • Separation of Code and Data • Data in Modules • Lookup Priority • Data in Component Modules • Data in Environments
  4. 4. Copyright example42 GmbH - 2016 Separation of Code and Data Image: Tatlin - tatlin.net
  5. 5. Copyright example42 GmbH - 2016 Separation of Code and Data • data in code class my_ntp {! !if $::environment == ‘dev’ {! $ntp_server = [‘pool.ntp.org’]! } else {! if $::facts[‘fqdn’] == ‘ntp1.example42.com’ {! # switch back to ntp1 when issue is solved! $ntp_server = [‘ntp2.example42.com’]! } else {! $ntp_server = [‘127.0.0.1’]! }! }!
  6. 6. Copyright example42 GmbH - 2016 Separation of Code and Data • explicit lookup • hiera(‘key’[, ‘default’][, ‘override hierarchy’])
  7. 7. Copyright example42 GmbH - 2016 Separation of Code and Data • implicit lookup class my_ntp (! !Array $ntp_server,! ) {! !# …! }! ! contain my_ntp! ! # hiera data! my_ntp::ntp_server:! - ‘pool.ntp.org’!
  8. 8. Copyright example42 GmbH - 2016 Separation of Code and Data • hiera.yaml # version 1! :backends:! - yaml! :yaml:! :datadir: “/etc/puppetlabs/code/ environments/%{environment}/hieradata”! :hierarchy:! - “nodes/%{::trusted.certname}”! - “os/%{::facts[‘os’][‘osfamily’]}”! - common!
  9. 9. Copyright example42 GmbH - 2016 Separation of Code and Data • hieradata os/Debian.yaml apache::pkgname:! - ‘apache2’! - ‘apache2-ssl’! ! os/RedHat.yaml apache::pkgname:! - ‘httpd’! ! common.yaml apache::purge_configs: true!
  10. 10. Copyright example42 GmbH - 2016 Separation of Code and Data • hieradata os/FreeBSD.yaml apache::pkgname:! - ‘apache’!
  11. 11. Copyright example42 GmbH - 2016 Separation of Code and Data • puppet code # apache/manifests/params.pp class apache::params {! case $::operatingsystem {! ‘Debian’: { # … }! ‘RedHat’: { # … }! default: {! fail(‘OS not supported’)! }! }! }!
  12. 12. Copyright example42 GmbH - 2016 Data in Modules Image: Tatlin - tatlin.net
  13. 13. Copyright example42 GmbH - 2016 Lookup Priority Image: Tatlin - tatlin.net
  14. 14. Copyright example42 GmbH - 2016 Lookup Priority • Hiera -> Global Overrides ! ! !
  15. 15. Copyright example42 GmbH - 2016 Lookup Priority • Hiera -> Global Overrides ! • Environment Data -> Core Puppet Lookup !
  16. 16. Copyright example42 GmbH - 2016 Lookup Priority • Hiera -> Global Overrides ! • Environment Data -> Core Puppet Lookup ! • Module Data -> Defaults
  17. 17. Copyright example42 GmbH - 2016 Lookup Functions • Explicit lookup: lookup(‘key’)! ! • CLI lookup: puppet lookup ‘key’ ! • Automatic lookup: ‘<namespace>::<key>’
  18. 18. Copyright example42 GmbH - 2016 Lookup Functions • lookup(‘key’, <Type>, <merge_behavior>, <default>)! • e.g. lookup(‘ntp_servers’, Array) • Merge behavior: • first! • unique (array merge)! • hash! • deep!
  19. 19. Copyright example42 GmbH - 2016 Data Provider Configuration • Global Environment Provider: puppet.conf • environment_data_provider = <data provider>! ! ! !
  20. 20. Copyright example42 GmbH - 2016 Data Provider Configuration • Global Environment Provider: puppet.conf • environment_data_provider = <data provider>! • Environment Provider: environment.conf • environment_data_provider = <data provider>! !
  21. 21. Copyright example42 GmbH - 2016 Data Provider Configuration • Global Environment Provider: puppet.conf • environment_data_provider = <data provider>! • Environment Provider: environment.conf • environment_data_provider = <data provider>! • Module Provider: metadata.json • “data_provider”: “<data provider>”
  22. 22. Copyright example42 GmbH - 2016 Data Provider • none -> standard hiera lookup ! • hiera -> hiera lookup (hiera v4) ! • function -> data function lookup
  23. 23. Copyright example42 GmbH - 2016 Data Provider Hiera • replace hiera, hiera_array, hiera_hash with ‘lookup’ • needs hiera.yaml v4 configuration file • set data_provider to ‘hiera’ in puppet.conf, environment.conf or metadata.json • modify global hiera.yaml to use datadir outside environment
  24. 24. Copyright example42 GmbH - 2016 Data Provider Hiera # /etc/puppetlabs/code/environments/production/hiera.yaml # /etc/puppetlabs/code/environments/production/modules/<module>/ hiera.yaml # - - -! version: 4! datadir: hieradata! hierarchy:! - name: “Nodes”! backend: yaml! path: “nodes/%{trusted.certname}”! - name: “OS”! backend: json! path: “os/%{facts.os.family}”! - name: “common”! backend: yaml
  25. 25. Copyright example42 GmbH - 2016 Data Provider Function • write data function • Puppet 4 Function • <module>/functions/<module>/data.pp • <env>/functions/<env>/data.pp • set data_provider to function in puppet.conf, environment.conf or metadata.json
  26. 26. Copyright example42 GmbH - 2016 Data Provider Function - Puppet # ntp/functions/ntp/data.pp function ntp::data() {! $params = {! ‘ntp::ntpservers’ => [‘pool.ntp.org’],! }! $os_params = case $facts[‘os’][‘family’] {! ‘Debian’: {! { ‘ntp::ntpackage’ => ‘ntpd’, }! },! default: {! {}! }! }! $params + $os_params! }
  27. 27. Copyright example42 GmbH - 2016 Data Provider Function • write data function • Ruby Function (Puppet 4 function API) • <module>/lib/puppet/functions/<module>/ data.rb • <env>/lib/puppet/functions/<env>/data.rb • set data_provider to function in puppet.conf, environment.conf or metadata.json
  28. 28. Copyright example42 GmbH - 2016 Data Provider Function - Ruby # ntp/lib/puppet/functions/ntp/data.rb Puppet::Functions.create_function(:’ntp::data’) do! def base_data()! { ‘ntp::ntpservers’ => [‘pool.ntp.org’], }! end! def os_data()! case Facter.value(:os)[‘family’]! when ‘Debian’! { ‘ntp::pkgname’ => ‘ntpd’, }! else! {}! end! def data()! self.base_data.merge!(self.os_data)! end! end
  29. 29. Copyright example42 GmbH - 2016 Data in Component Modules Image: Tatlin - tatlin.net
  30. 30. Copyright example42 GmbH - 2016 Data in Component Modules • add data provider to metadata.json • provide OS defaults • remove params.pp / remove inheritance • allow users to overwrite any data
  31. 31. Copyright example42 GmbH - 2016 Data Provider Function - Ruby # my_ntp/manifests/init.pp class my_ntp (! $server = $my_ntp::params::server,! $pkgname = $my_ntp::params::pkgname,! $secure = $my_ntp::params::secure,! ) inherits my_ntp::params {! # ...! }!
  32. 32. Copyright example42 GmbH - 2016 Data in Environments Image: Tatlin - tatlin.net
  33. 33. Copyright example42 GmbH - 2016 Data in Environments • old hiera replacement • add hiera.yaml to environment base path • overwrite data from modules, roles & profiles
  34. 34. Copyright example42 GmbH - 2016 Summary Image: Tatlin - tatlin.net
  35. 35. Copyright example42 GmbH - 2016 Summary # /etc/puppetlabs/code/ ! ! hiera.yaml! ! ! hieradata/! ! ! environments/production/ ! ! ! ! ! ! environment.conf! ! ! ! ! ! ! hiera.yaml! ! ! ! ! ! ! hieradata/! modules/my_module/ ! ! ! ! ! ! ! ! ! ! metadata.json! ! ! ! ! ! ! ! ! ! ! hiera.yaml! ! ! ! ! ! ! ! ! ! ! hieradata/
  36. 36. Copyright example42 GmbH - 2016 Summary # /etc/puppetlabs/code/ ! ! hiera.yaml! ! ! hieradata/! ! ! environments/production/ ! ! ! ! ! environment.conf! ! ! ! ! ! lib/functions/data.pp! ! ! ! ! ! lib/puppet/functions/ ! ! ! ! ! ! ! ! ! production/data.rb! modules/my_module/ ! ! ! ! ! ! !! ! metadata.json! ! ! ! ! ! ! ! ! lib/functions/data.pp! ! ! ! ! ! ! ! ! lib/puppet/functions/ ! ! ! ! ! ! ! ! ! ! ! ! my_module/data.rb
  37. 37. Copyright example42 GmbH - 2016 Summary - Pro • Per hierarchy Hiera Data backend possible • Data Function lookups without need for hiera backend (e.g. Cloud Management API data) • No more inheritance required
  38. 38. Copyright example42 GmbH - 2016 Summary - Con • No single Source of Authority? • Debugging can be complex when iterating over many data providers and hierarchies
  39. 39. Copyright example42 GmbH - 2016 Module Developers • switch to data in modules • give users the possibility to provide own data • allow users to overwrite any data • allow users to know their data for missing OS support
  40. 40. Copyright example42 GmbH - 2016 Module Users • switch to hieradata in modules then • switch to data in environments • keep data simple and readable • don’t overcomplicate !
  41. 41. Copyright example42 GmbH - 2016 Module Users • hieradata common.yaml my_ntp: ‘pool.ntp.org’! apache::default_mods: false! apache::purge_configs: true! mysql::remove_default_accounts: true! mysql::root_password: ‘puppet’! oradb::database::version: ’12.1’! oradb::shout: ‘MISSING DATA’
  42. 42. Copyright example42 GmbH - 2016 References • http://docs.puppetlabs.com/puppet/4.3/reference/ lookup_quick.html • http://docs.puppetlabs.com/puppet/4.3/reference/ lookup_quick_module.html • http://puppet-on-the-edge.blogspot.de/2015/01/ puppet-40-data-in-modules-and.html
  43. 43. Copyright example42 GmbH - 2016 Puppet 4 - Data in modules PuppetCamp Paris 2016 Martin Alfke - ma@example42.com Image: Tatlin - tatlin.net Copyright example42 GmbH - 2016

×