Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PuppetConf. 2016: External Data in Puppet 4 – R.I. Pienaar

132 views

Published on

Here are the slides from R.I. Pienaar's PuppetConf 2016 presentation called External Data in Puppet 4. Watch the videos at https://www.youtube.com/playlist?list=PLV86BgbREluVjwwt-9UL8u2Uy8xnzpIqa

Published in: Technology
  • Be the first to comment

  • Be the first to like this

PuppetConf. 2016: External Data in Puppet 4 – R.I. Pienaar

  1. 1. R.I.Pienaar PuppetConf 2016 External Data in Puppet 4
  2. 2. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Who am I? • Puppet User since 0.22 • Really into external data • Author facts.d, extlookup, Hiera, MCollective • Consultant - DevOps, Automation, Architect, Development • Open Source @ github.com/ripienaar • Volcane on IRC
  3. 3. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar External Data Is Configuration For Your Code
  4. 4. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Hierarchical Data class monitor ( Array[Monitor::Plugin] $plugins = [] ) { … }
  5. 5. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Hierarchical Data class{“monitor”: plugins => [ “cpu", “write_graphite”, “df”, … ] }
  6. 6. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Automatic Param Lookup include monitor # hieradb/nodes/host1.yaml monitor::plugins: - my_app class{“monitor”: plugins => [ “my_app”, “cpu”, “df”, … ] } + =
  7. 7. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Hierarchical Data host1 host2 host3 host4 host5 host6 prod stage dev dc1 dc2 Site wide common data
  8. 8. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Hierarchical Data host1 host2 host3 host4 host5 host6 prod stage dev dc1 dc2 Site wide common data
  9. 9. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Hierarchical Data host1 host2 host3 host4 host5 host6 prod stage dev dc1 dc2 Site wide common data
  10. 10. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Hierarchical Data host1 host2 host3 host4 host5 host6 prod stage dev dc1 dc2 Site wide common data $trusted[“certname”] $environment $facts[“dc”]
  11. 11. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Automatic Param Lookup include monitor # hieradb/nodes/host1.yaml monitor::plugins: - my_app class{“monitor”: plugins => [ “my_app”, “cpu”, “df”, … ] } + =
  12. 12. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup • lookup() function replaces hiera*() • Automatic Param Lookup more useful • environment and module aware • Great CLI • YAML/JSON out of the box, more to come • Data compatible with Hiera, cohabits with Hiera • Pluggable
  13. 13. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - CLI $ puppet lookup monitor::plugins —merge unique —— - my_app - write_graphite - cpu
  14. 14. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar $ puppet lookup monitor::plugins —merge unique —explain Merge strategy unique … Data Provider "Nodes" Path “/etc/…/production/data/nodes/dev3.devco.net.yaml” Original path: "nodes/%{trusted.certname}" Found key: "monitor::plugins" value: [ "my_app" ] … Data Provider "common" Path “/etc/…/data/common.yaml” Original path: "common" Found key: "monitor::plugins" value: [ "cpu", "write_graphite" ] … Merged result: [ "my_app", "cpu", "write_graphite" ]
  15. 15. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar $ puppet lookup monitor::plugins —merge unique —explain Merge strategy unique … Data Provider "Nodes" Path “/etc/…/production/data/nodes/dev3.devco.net.yaml” Original path: "nodes/%{trusted.certname}" Found key: "monitor::plugins" value: [ "my_app" ] … Data Provider "common" Path “/etc/…/data/common.yaml” Original path: "common" Found key: "monitor::plugins" value: [ "cpu", "write_graphite" ] … Merged result: [ "my_app", "cpu", "write_graphite" ] Every tier Final result
  16. 16. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - Config # environments/production/hiera.yaml --- version: 4 datadir: data hierarchy: - name: "Nodes" backend: yaml path: "nodes/%{trusted.certname}" - name: "common" backend: yaml
  17. 17. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function $plugins = lookup(“monitor::plugins”)
  18. 18. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function $plugins = lookup(“monitor::plugins”, { “value_type” => Array[Monitor::Plugin], “default_value” => [“cpu”, …] })
  19. 19. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function function monitor::params() { $result = { “monitor::plugins” => [ “cpu”, … ] } } $plugins = lookup(“monitor::plugins”, { “default_value_hash” => monitor::params() })
  20. 20. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function $servers = lookup(“monitor::plugins”, { “merge” => “first” }) Default behaviour
  21. 21. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function $servers = lookup(“monitor::plugins”, { “merge” => “unique” }) Array Merge (hiera_array())
  22. 22. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function $servers = lookup(“monitor::plugins”, { “merge” => “hash” }) Shallow Hash Merge (hiera_hash())
  23. 23. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function $servers = lookup(“monitor::plugins”, { “merge” => “deep”, }) Deep Hash Merge
  24. 24. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function # common.yaml users::local: rip: email: “rip@devco.net" comment: “R.I.Pienaar” sudoer: true shell: “/bin/zsh”
  25. 25. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function # nodes/dev3.devco.net users::local: rip: email: “arri.pienaar@client.com”
  26. 26. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function $ puppet lookup users::local —merge deep --- rip: email: arri.pienaar@client.com comment: R.I.Pienaar sudoer: true shell: "/bin/zsh"
  27. 27. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Automatic Param Lookup include monitor # hieradb/nodes/host1.yaml monitor::plugins: - my_app class{“monitor”: plugins => [ “my_app”, “cpu”, “df”, … ] } + =
  28. 28. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function $lookup_options = { “merge” => { “strategy” => “unique” } } $plugins = lookup(“monitor::plugins”, $lookup_options)
  29. 29. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - function $plugins = lookup(“monitor::plugins”) # hieradata/common.yaml lookup_options: monitor::plugins: merge: strategy: unique
  30. 30. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar include monitor # …/production/hieradata/common.yaml lookup_options: monitor::plugins: strategy: unique monitor::plugins: - cpu - … class monitor ( Array[Monitor::Plugin] $plugins = [] ) {} Lookup - function
  31. 31. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Lookup - Data In Modules # modules/monitor/hiera.yaml --- version: 4 datadir: "data" hierarchy: - name: "common" backend: "yaml" # modules/monitor/data/common.yaml lookup_options: monitor::plugins: strategy: unique
  32. 32. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar include monitor # hieradata/common.yaml monitor::plugins: - cpu - write_graphite Lookup - Data In Modules # hieradata/nodes/node1.yaml monitor::plugins: - my_app
  33. 33. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Tips and Tricks
  34. 34. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar lookup(“classes”, {“merge” => “unique”}).include hiera_include()
  35. 35. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar hiera_include() # common.yaml lookup_options: classes: merge: strategy: “deep” knockout_prefix: “-“ classes: - nagios # nodes/node1.yaml classes: - -nagios - sensu
  36. 36. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar lookup(“classes”).include hiera_include() # common.yaml lookup_options: classes: merge: strategy: “deep” knockout_prefix: “-“
  37. 37. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar params.pp # modules/ntp/hiera.yaml --- version: 4 datadir: data hierarchy: - name: "Operating System Family" backend: yaml path: "%{facts.os.family}" - name: "common" backend: yaml
  38. 38. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar params.pp # modules/ntp/data/AIX.yaml --- ntp::driftfile: '/etc/ntp.drift' ntp::package_name: - 'bos.net.tcp.client' ntp::restrict: - 'default nomodify notrap nopeer noquery' - '127.0.0.1' ntp::service_name: 'xntpd'
  39. 39. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar params.pp # modules/ntp/data/Debian.yaml ntp::restrict: - '-4 default kod nomodify notrap nopeer noquery' - '-6 default kod nomodify notrap nopeer noquery' - '127.0.0.1' - '::1' ntp::service_name: 'ntp' ntp::servers: - '0.debian.pool.ntp.org' - '1.debian.pool.ntp.org'
  40. 40. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Future • Hiera deprecated • No more site wide hiera, only environment and modules • Pluggable via functions in modules - native or ruby, much much simpler than old backends • lookup.yaml or lookup.conf
  41. 41. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar Questions? twitter: @ripienaar email: rip@devco.net blog: www.devco.net github: ripienaar freenode: Volcane slack.puppet.com: ripienaar https://www.devco.net/

×