Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Puppet and your Metadata - PuppetCamp London 2015

1,480 views

Published on

How do you organise your metadata in Puppet, it's important to know all the different options, choices and steps you can do to make your metadata rock!

Published in: Technology
  • Be the first to comment

Puppet and your Metadata - PuppetCamp London 2015

  1. 1. Marc  Cluet https://www.flickr.com/photos/jsprig/5106529710/
  2. 2. Checklist https://www.flickr.com/photos/timothytsuihin/2743062701/ https://www.flickr.com/photos/zachinglis/5507648594/ https://www.flickr.com/photos/epsos/8270256961/ https://www.flickr.com/photos/mini82/1361976303/
  3. 3. Who am I? Marc  Cluet  (@lynxman) Grumpy  Engineer  based  in  London Co-­‐Founder  of  Ukon Cherry 17  years  of  experience  as  a  SysAdmin Founding  member  of  Juju  and  MAAS   while  at  Canonical Built  a  DevOps Engineering  Team  at   Rackspace Been  DevOps’in for  the  last  6 years
  4. 4. Who am I? http://www.meetup.com/London-­‐DevOps/
  5. 5. What is Metadata? https://www.flickr.com/photos/annarbor/4349876203/
  6. 6. What is Metadata? Metadata  is  “data  about  data” Structural  Metadata Descriptive  Metadata
  7. 7. What is Metadata? Structural  Metadata architecture  =>  amd64 ipaddress =>  10.0.115.197 Descriptive  Metadata $puppetversion =  3.7.5 $apacheversion =  2.4
  8. 8. Metadata purpose? Abstract Definitions Unique Data
  9. 9. Metadata purpose? Abstract Definitions Unique Data
  10. 10. Metadata in Puppet https://www.flickr.com/photos/jimmcd/4859841581/
  11. 11. Metadata in Puppet Variables Facts Hiera PuppetDB
  12. 12. Variables class  apples { $apples  =  5 $string  =  “I  have  ${apples}  apples” } $apples::apples $apples::string global
  13. 13. Scope web1 mail web2 apache global
  14. 14. Exported resources Class  newclient { @@sshkey {  $::hostname: type  =>  dsa, key    =>  $mykey, } }
  15. 15. Exported resources Server  SSH  bastion class  bastion { Sshkey <<|  |>> }
  16. 16. Facts Facter Puppet
  17. 17. Facts $  facter interfaces  =>  eth0,lo ipaddress =>  10.0.115.197 ipaddress_eth0  =>  10.0.115.197 ipaddress_lo =>  127.0.0.1 is_virtual =>  true kernel  =>  Linux kernelmajversion =>  3.13
  18. 18. Custom Facts Where /etc/facter/facts.d What Text  File Script Output role=webserver
  19. 19. Custom Facts $  facter role webserver
  20. 20. Scope web1 facts web2 apache mail global
  21. 21. Metadata Distribution https://www.flickr.com/photos/wallyg/6271443142/
  22. 22. PuppetDB Facter Puppet PuppetDB
  23. 23. PuppetDB Install Debian/Ubuntu $  apt-­‐get  install  puppetdb RedHat $  yum  install  puppetdb
  24. 24. PuppetDB Data Keeps Reports Facts Exported  Resources  (searchable) Used  by Puppet  Client  (Exp Resources) Reporting
  25. 25. Hiera Facter Puppet Hiera PuppetDB
  26. 26. Hiera Install $  gem  install  hiera
  27. 27. Hiera Config /etc/hiera.yaml -­‐-­‐-­‐ :backends: -­‐ yaml :hierarchy: -­‐ "%{environment}/nodes/%{fqdn}” -­‐ "%{environment}/roles/%{role}” -­‐ "%{environment}/common” -­‐ common :yaml: :datadir:  /etc/puppet/hieradata
  28. 28. Hiera Data /etc/puppet/hieradata/common.yaml #  Define  our  variable example:  foo
  29. 29. Hiera Data in Puppet class  test  { $hieravar =  hiera(‘example’) }
  30. 30. Hiera Data in Puppet class  test  { $hieravar =  hiera(‘example’,’bar’) }
  31. 31. Hiera Data Hierarchy /etc/puppet/hieradata/common.yaml #  Apache  variables apache::rootdir:  /www apache::user:  www-­‐data
  32. 32. Hiera Loops /etc/puppet/hieradata/roles/apache.yaml #  Apache  variables vhost:  site rootdir:  "%{hiera(’apache::rootdir')}/site" user:  "%{hiera(’apache::user')}"
  33. 33. Modules consume Hiera https://forge.puppetlabs.com/mthibaut/users /etc/puppet/hieradata/common.yaml #  Define  our  users users_sysadmins: marc: ensure:  present uid:  10001 gid:  staff groups:  sysadmin comment:  Marc  Cluet shell:  /bin/bash
  34. 34. Hiera Security
  35. 35. Hiera Security Hiera Security  Projects Hiera GPG https://github.com/crayfishx/hiera-­‐gpg Hiera eyaml https://github.com/TomPoulton/hiera-­‐eyaml
  36. 36. Hiera eyaml Install $  gem  install  hiera_eyaml eyaml {createkeys decrypt edit encrypt recrypt}
  37. 37. Hiera eyaml /etc/hiera.yaml -­‐-­‐-­‐ :backends: -­‐ eyaml :hierarchy: -­‐ "%{environment}/nodes/%{fqdn}” -­‐ "%{environment}/roles/%{role}” -­‐ "%{environment}/common” -­‐ common :eyaml: :datadir:  /etc/puppet/hieradata :pkcs7_private_key:  "/secure/private.pkcs7.pem” :pkcs7_public_key:  "/secure/public.pkcs7.pem"
  38. 38. Hiera eyaml secretpassword:  DEC::PCSK7[potato]!
  39. 39. Hiera eyaml secretpassword:  ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoII BejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNA QEBBQAEggEAGPLCXn8goh27KO3msvCz6GJydTHTNrplQ23dR/d fV0MuSmYwnASFa9RDpqe9K5fhs5XO5TfyQ5Uf4IAZxhnuDx+3Z SiVY1lbXY6C3x6XeXUBN0jfB5FkdrR+mMYCzGRbVB3gPlM0I8g 2Wq2397h5zMHRkFizPr16vhKuQDxMeGlq8yfoZ4FqwhUteeYxP MnCV7lx0I6Z/e8I4UZek4FQ7dMfuXNerdDAxx+UMZjRhK3trQl R/x2TLS4vgh8m0Ml3F5q851W4s5O4ZE7/wvq//nvlr1RoH5EnP pPQ6shv6R6I7lBfViqIRJalRdyTWVLpFgUknXKLRzoxH8S7IlD 5PzBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBA66G5+tBaaF 9g8Lq4dTrwNgCCVP77OWTPoSWGyfxNkuTrmoOANAL98kVWXioO re612Vw==]
  40. 40. Connecting Metadata https://www.flickr.com/photos/lightspectrals/1447079898/
  41. 41. Automation metadata Provisioner Puppet Monitoring Services
  42. 42. Automation metadata Provisioner Puppet Monitoring Services ?
  43. 43. Metadata Partitioning Service Node  A Load   Balancer Service Node  B Service Node  C
  44. 44. Metadata Partitioning Service Node  A Load   Balancer Service Node  C
  45. 45. Metadata partitioning https://www.flickr.com/photos/genista/4346973713/
  46. 46. Detour! https://www.flickr.com/photos/bionicteaching/14586204543/
  47. 47. Service Discovery
  48. 48. What is Service Discovery? Service Node  A Health   Check Discovery   Agent Discovery   Agent Service Node  B Health   Check Service   Publication Node  A Node  B
  49. 49. Metadata Unification https://www.flickr.com/photos/bangorfuji9500/7152182001/
  50. 50. Metadata Unification hiera_mysql https://forge.puppetlabs.com/crayfishx/hiera_mysql hiera_redis https://forge.puppetlabs.com/rubyisbeautiful/hiera_redis hiera_http https://forge.puppetlabs.com/crayfishx/hiera_http hiera_etcd https://forge.puppetlabs.com/garethr/hiera_etcd hiera_consul https://forge.puppetlabs.com/lynxman/hiera_consul
  51. 51. Metadata Unification Puppet yaml Hiera consul consul k/v 1 2 3 2
  52. 52. Hiera Consul /etc/hiera.yaml -­‐-­‐-­‐ :backends: -­‐ consul :consul: :host:  127.0.0.1 :port:  8500 :failure:  graceful :paths: -­‐ /v1/catalog/service -­‐ /v1/catalog/node
  53. 53. Hiera Consul http://localhost:8500/v1/catalog/service/sensu [ { "ServicePort":  0, "ServiceTags":  [], "ServiceName":  "sensu", "ServiceID":  "sensu", "Address":  "10.0.115.197", "Node":  "ip-­‐10-­‐0-­‐115-­‐197“ } ]
  54. 54. Hiera Consul http://localhost:8500/v1/catalog/service/sensu [ { "ServicePort":  0, "ServiceTags":  [], "ServiceName":  "sensu", "ServiceID":  "sensu", "Address":  "10.0.115.197", "Node":  "ip-­‐10-­‐0-­‐115-­‐197“ } ]
  55. 55. Hiera Consul $sensu_service =  hiera('sensu',[]) $sensu_service[ServicePort] $sensu_service[ServiceTags] $sensu_service[ServiceName] $sensu_service[ServiceID] $sensu_service[Address] $sensu_service[Node]
  56. 56. Hiera Consul $sensu_service =  hiera('sensu',[]) $s_address =  consul_info($sensu_service,  'Address')
  57. 57. Warnings & Advice https://www.flickr.com/photos/catherinekolodziej/8866489274/
  58. 58. Facter Facter needs  to  be  FAST Don’t  put  here  long  running  scripts Cron it  up! Text  always  works  better
  59. 59. Hiera Hiera is  awesome! Be  careful  of  long  query  loops
  60. 60. Who’s the brain? Provisioner Puppet Orchestration
  61. 61. Who’s the brain? Provisioner Puppet Orchestration
  62. 62. Who’s the brain? Provisioner Puppet Orchestration
  63. 63. Who’s the brain? Provisioner Puppet Orchestration
  64. 64. Who’s the brain? It’s  your  decision! Keep  coherency Unified  metadata  is  powerful
  65. 65. https://www.flickr.com/photos/mikko_luntiala/12691267935 https://www.flickr.com/photos/dullhunk/202872717 @lynxman http://slideshare.net/lynxmanuk/ https://github.com/lynxman/ https://devroot.org/

×