Puppet as data_chicago

1,029 views

Published on

1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total views
1,029
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
19
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Puppet as data_chicago

  1. 1. Puppet as Data Transformations Dan Bode| Puppet Labs dan@puppetlabs.combodepd <on> [twitter,freenode]
  2. 2. What is it about? • Deconstructing Puppet to data • Why you should care # puppetcamp # puppetize @ puppetlabs
  3. 3. Part One Dissecting a Puppet Run
  4. 4. Facter, who am I? Hi! your facts are: kernel=linux ipaddress=10.0.0.3 Agent macaddress=… # puppetconf # puppetize @ puppetlabs
  5. 5. facts Hi Mr. Master, I need a catalog. Here Agent are my facts http://www.dgcomputers.org/testimonials.php# puppetconf # puppetize @ puppetlabs
  6. 6. facts Thanks for you facts. Agent store them in I’ll just PuppetDB PuppetDB# puppetconf # puppetize @ puppetlabs
  7. 7. Mr. ENC, is this host defined as an external Yep, he should be an node? apache server. Here is the definition nodes ENC ENC Agent PuppetDB PuppetDB facts# puppetconf # puppetize @ puppetlabs
  8. 8. catalog Just compiled your Agent One sec while catalog. PuppetDB PuppetDB I store it in PuppetDB. facts# puppetconf # puppetize @ puppetlabs
  9. 9. Here is your catalog. Send me a report and let me know how it went! catalog Agent PuppetDB PuppetDB catalog facts# puppetconf # puppetize @ puppetlabs
  10. 10. I hate to be a bother, but can you compute the md5sums of this file? catalog Agent PuppetDB PuppetDB catalog facts# puppetconf # puppetize @ puppetlabs
  11. 11. Oh, I need that one! Can I get the file. catalog Agent PuppetDB PuppetDB catalog facts# puppetconf # puppetize @ puppetlabs
  12. 12. Thanks, now let’s just do that 1000 more times. catalog Agent PuppetDB PuppetDB catalog facts# puppetconf # puppetize @ puppetlabs
  13. 13. You know ‘content’ will embed the contents in the catalog? catalog Agent PuppetDB PuppetDB catalog facts# puppetconf # puppetize @ puppetlabs
  14. 14. report Just finished applying. Agent are the results. Here PuppetDB PuppetDB catalog facts# puppetconf # puppetize @ puppetlabs
  15. 15. Part Two Puppet’s internal data language (indirectory/termini)
  16. 16. facts find fromterminus facter Agent # puppetconf # puppetize @ puppetlabs
  17. 17. facts catalog find from terminus rest Agent http://www.dgcomputers.org/testimonials.php# puppetconf # puppetize @ puppetlabs
  18. 18. facts facts save to terminus Agent puppetdb PuppetDB# puppetconf # puppetize @ puppetlabs
  19. 19. node find from terminus exec (or ldap) nodes ENC ENC Agent PuppetDB PuppetDB facts# puppetconf # puppetize @ puppetlabs
  20. 20. catalog catalog find from Agent terminus compiler PuppetDB PuppetDB facts# puppetconf # puppetize @ puppetlabs
  21. 21. catalog catalog save to Agent puppetdb terminus PuppetDB PuppetDB facts# puppetconf # puppetize @ puppetlabs
  22. 22. Disecting a Puppet RunFacterFacter Facts ENC Nodes/ ENC Manifest Compiler Compiler Reports Config Catalogs # puppetconf # puppetize @ puppetlabs
  23. 23. CLI commands# puppetconf # puppetize @ puppetlabs
  24. 24. CLI Puppet Facts# mkdir –p /tmp/yaml/facts# puppet facts find node_name --render-as yaml > /tmp/yaml/facts/node_name.yaml# puppetcamp # puppetize @ puppetlabs
  25. 25. Creating a node (optional): # mkdir –p /tmp/yaml/nodes # puppet node find node_name --node_terminus=exec --external_nodes=/etc/puppet/nodes.sh --facts_terminus=yaml --clientyamldir=/tmp/yaml/ --render-as=yaml > /tmp/yaml/nodes/node_name.yaml # puppetcamp # puppetize @ puppetlabs
  26. 26. Applying a catalog:# puppet catalog find node_name /tmp/catalog.yaml# puppetcamp # puppetize @ puppetlabs
  27. 27. Creating a catalog:# puppet apply –catalog /tmp/catalog.yaml(its easy assuming you are not using file sources)# puppetcamp # puppetize @ puppetlabs
  28. 28. Fun with IRB# puppetconf # puppetize @ puppetlabs
  29. 29. IRB Factsirb:> require ‘puppet/face’> Puppet.parse_config # required Puppet > 3.0> facts=Puppet::Face[:facts, :current].find(node_name) # puppetcamp # puppetize @ puppetlabs
  30. 30. Access a Fact value (irb): … > facts.values[ipaddress] => "10.0.2.15" # puppetcamp # puppetize @ puppetlabs
  31. 31. Creating a node (from irb): … > node=Puppet::Node.new(node_name, {:classes => {:foo => {:bar => :baz}}}) >node.merge(facts.values) # puppetcamp # puppetize @ puppetlabs
  32. 32. Creating a catalog:…irb> catalog=Puppet::Face[:catalog, :current]. find(node_name, :extra => { :node => node}) # puppetcamp # puppetize @ puppetlabs
  33. 33. Use CasesInteracting with Puppet’s Data
  34. 34. Inspecting the catalog: • What types are in the catalog? irb> catalog.resources.collect {|r| r.type }.uniq • Gimme a resource: irb>catalog.resource(‘Package[httpd]’)# puppetcamp # puppetize @ puppetlabs
  35. 35. Rspec Puppet: let :facts do {:operatingsystem => ‘Redhat’} end let :params do {:bind_address => ‘0.0.0.0’ end it { should contain_file(‘/tmp/foo.conf’) }# puppetcamp # puppetize @ puppetlabs
  36. 36. Thundering HerdPre-compile catalogs for faster auto-scaling# puppetcamp # puppetize @ puppetlabs
  37. 37. Applying pre-compiled catalogs: • Gather facts from an ec2 instance • Generate a single catalog • Apply that catalog to all hosts puppet apply --catalog /tmp/catalog.json –server puppet-fileserver # puppetcamp # puppetize @ puppetlabs
  38. 38. DMZtcp over USB# puppetcamp # puppetize @ puppetlabs
  39. 39. Use CasesInteracting with Puppet’s Data
  40. 40. Hacking reports Everything in Puppet is a state transition User[‘dan’] : absent -> present User[‘dan’][‘shell’] -> ‘/sbin/nologin’ -> /bin/bash# puppetcamp # puppetize @ puppetlabs
  41. 41. Setting up the agent: [agent] report=true# puppetcamp # puppetize @ puppetlabs
  42. 42. Archive reports in youryamldir [master] reports = store# puppetcamp # puppetize @ puppetlabs
  43. 43. Puppet reports $ cd `puppet config print reportdir` $ ls node1 node2 node3 $ ls node1# puppetcamp # puppetize @ puppetlabs
  44. 44. Every report from every runever $ ls node1 201206060256.yaml 201206060303.yaml 201206060519.yaml 201206122349.yaml 201206122354.yaml 201206130002.yaml# puppetcamp # puppetize @ puppetlabs
  45. 45. Lets crack one open! Irb > require ‘yaml’ >reports=YAML.load_file(201206130002.yaml)# puppetcamp # puppetize @ puppetlabs
  46. 46. Have a look >(reports.methods - Object.methods).sort Notice the following methods:# puppetcamp # puppetize @ puppetlabs
  47. 47. High level data > reports.exit_status ⇒0 > reports.status => "unchanged" > reports.host ⇒”node1”# puppetcamp # puppetize @ puppetlabs
  48. 48. metrics > reports.metrics.keys ⇒["resources", "events", "changes", "time"] > reports.metrics[resources] ⇒[‘failed’, 0],[ ‘changed’, ‘7’]# puppetcamp # puppetize @ puppetlabs
  49. 49. And the awesome sauce > reports.resource_statuses.keys => ["Package[xinetd]", "File[/srv/node/1]", "Package[swift]", "Exec[compile fragments]", "Package[swift-container]", "File[/var/opt/lib/pe- puppet/concat/_etc_swift_object-server.conf]", "File[/etc/rsync.d/frag-account]”]# puppetcamp # puppetize @ puppetlabs
  50. 50. And the awesome sauce > status = reports.resource_statuses > status.keys => ["Package[xinetd]", "File[/srv/node/1]", "Package[swift]", "Exec[compile fragments]", "Package[swift-container]", "File[/var/opt/lib/pe- puppet/concat/_etc_swift_object-server.conf]", "File[/etc/rsync.d/frag-account]”]# puppetcamp # puppetize @ puppetlabs
  51. 51. And the awesome sauce >events = status["File[/etc/swift/swift.conf]"].events > events.first.status ⇒"success” > events.first.desired_value ⇒:present > events.first.previous_value => :absent# puppetcamp # puppetize @ puppetlabs
  52. 52. ampThank YouDan Bode| Puppet Labsdan@puppetlabs.com

×