Hacking The Data out of Puppet - PuppetConf '12

17,046 views

Published on

"Hacking The Data out of Puppet" by Dan Bode, Business Development at Puppet Labs
Watch videos from PuppetConf 2012 here: www.puppetlabs.com/community/videos/puppetconf

Speaker Bio: Dan has worked in the technology industry as a consultant and software developer for almost a decade. He has spent most of the last 4 years building infrastructure automation solutions and teaching people how to use Puppet. He currently works in the Business Development at PuppetLabs where he spends his time researching technologies and figuring out how they can be integrated with Puppet to provide value.

Learn more about puppet: http://bit.ly/QQoAP1

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
17,046
On SlideShare
0
From Embeds
0
Number of Embeds
4,121
Actions
Shares
0
Downloads
99
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Hacking The Data out of Puppet - PuppetConf '12

  1. 1. HackingThe Dataout of Puppet Dan Bode| Puppet Labs dan@puppetlabs.com
  2. 2. Who is this talk for? •  People who already understand Puppet •  Developers or people who are dev-curious # puppetconf # puppetize @ puppetlabs
  3. 3. Shameless plug # puppetconf # puppetize @ puppetlabs
  4. 4. What is it about? •  Deconstructing Puppet to data # puppetconf # puppetize @ puppetlabs
  5. 5. Puppet as Data Dissecting a Puppet Run
  6. 6. Facter, who am I? Hi! your facts are: kernel=linux ipaddress=10.0.0.3 Agent macaddress=… # puppetconf # puppetize @ puppetlabs
  7. 7. facts Hi Mr. Master, I need a catalog. Here Agent are my facts http://www.dgcomputers.org/testimonials.php # puppetconf # puppetize @ puppetlabs
  8. 8. facts Thanks for you facts. Agent store them in I’ll just PuppetDB PuppetDB # puppetconf # puppetize @ puppetlabs
  9. 9. Mr. ENC, is this host defined as an external Yep, he should be an node? apache server. Here is the definition nodes ENC Agent PuppetDB facts # puppetconf # puppetize @ puppetlabs
  10. 10. catalog Just compiled your Agent One sec while catalog. PuppetDB I store it in PuppetDB. facts # puppetconf # puppetize @ puppetlabs
  11. 11. Here is your catalog. Send me a report and let me know how it went! catalog Agent PuppetDB catalog facts # puppetconf # puppetize @ puppetlabs
  12. 12. I hate to be a bother, but can you compute the md5sums of a few files? catalog Agent PuppetDB catalog facts # puppetconf # puppetize @ puppetlabs
  13. 13. report Just finished applying. Agent are the results. Here PuppetDB catalog facts # puppetconf # puppetize @ puppetlabs
  14. 14. Termini and the indirector Interacting with Puppet’s Data
  15. 15. facts find fromterminus facter Agent # puppetconf # puppetize @ puppetlabs
  16. 16. facts catalog find from terminus rest Agent http://www.dgcomputers.org/testimonials.php # puppetconf # puppetize @ puppetlabs
  17. 17. facts facts save to terminus Agent puppetdb PuppetDB # puppetconf # puppetize @ puppetlabs
  18. 18. node find from terminus exec (or ldap) nodes ENC Agent PuppetDB facts # puppetconf # puppetize @ puppetlabs
  19. 19. catalog catalog find from Agent compiler terminus PuppetDB facts # puppetconf # puppetize @ puppetlabs
  20. 20. catalog catalog save to Agent puppetdb terminus PuppetDB facts # puppetconf # puppetize @ puppetlabs
  21. 21. Disecting a Puppet RunFacter Facts ENC Nodes/ Manifest Compiler Reports Config Catalogs # puppetconf # puppetize @ puppetlabs
  22. 22. CLI commands puppet facts find puppet node find puppet catalog find # puppetconf # puppetize @ puppetlabs
  23. 23. CLI Puppet Facts# mkdir –p /tmp/yaml/facts# puppet facts find node_name --render-as yaml > /tmp/yaml/facts/node_name.yaml# puppetconf # puppetize @ puppetlabs
  24. 24. Creating a node (optional): # puppet node find node_name --node_terminus=exec --external_nodes=/etc/puppet/nodes.sh --facts_terminus=yaml --clientyamldir=/tmp/yaml/ --render-as=yaml > /tmp/yaml/nodes/node_name.yaml # puppetconf # puppetize @ puppetlabs
  25. 25. Creating a catalog:# puppet catalog find node_name --facts_terminus=yaml # puppet catalog find node_name --node_terminus=yaml --manifest=/etc/puppet/manifest/site.pp --modulepath=/etc/puppet/modules/--clientyamldir=/tmp/yaml/ > /tmp/catalog.yaml# puppetconf # puppetize @ puppetlabs
  26. 26. Fun with IRB Puppet::Node::Facts.indirection.find facts Puppet::Node.new nodes Puppet::Catalog.indirection.find catalog # puppetconf # puppetize @ puppetlabs
  27. 27. IRB Factsirb:> require ‘puppet/face’ > facts=Puppet::Face[:facts, :current].find(node’) # puppetconf # puppetize @ puppetlabs
  28. 28. Access a Fact value (irb): … > facts.values[ipaddress] => "10.0.2.15" # puppetconf # puppetize @ puppetlabs
  29. 29. Creating a node (from irb): … > node=Puppet::Node.new(node_name, {:classes => {:foo => {:bar => :baz}}}) >node.merge(facts.values) # puppetconf # puppetize @ puppetlabs
  30. 30. Creating a catalog:…irb> catalog=Puppet::Resource::Catalog.indirection. find(node_name’, :node => node) # puppetconf # puppetize @ puppetlabs
  31. 31. Use Cases Interacting with Puppet’s Data
  32. 32. Inspecting the catalog: •  What types are in the catalog? irb> catalog.resources.collect do |r| r.type end.uniq •  Gimme a resource: irb>catalog.resource(‘Package[httpd]’) # puppetconf # puppetize @ puppetlabs
  33. 33. Rspec Puppet: let :facts do {:operatingsystem => ‘Redhat’} end let :params do {:bind_address => ‘0.0.0.0’ end it { should contain_file(‘/tmp/foo.conf’) }# puppetconf # puppetize @ puppetlabs
  34. 34. Thundering HerdPre-compile catalogs for faster auto-scaling# puppetconf # puppetize @ puppetlabs
  35. 35. Applying pre-compiled catalogs:puppet apply --catalog /tmp/catalog.json –serverpuppet-fileserver# puppetconf # puppetize @ puppetlabs
  36. 36. DMZtcp over USB# puppetconf # puppetize @ puppetlabs
  37. 37. Use Cases Interacting with Puppet’s Data
  38. 38. Hacking reports Everything in Puppet is a state transition User[‘dan’] : absent -> present User[‘dan’][‘shell’] -> ‘/sbin/nologin’ -> /bin/bash# puppetconf # puppetize @ puppetlabs
  39. 39. Setting up the agent: [agent] report=true# puppetconf # puppetize @ puppetlabs
  40. 40. Archive reports in youryamldir [master] reports = store# puppetconf # puppetize @ puppetlabs
  41. 41. Puppet reports $ cd `puppet config print reportdir` $ ls node1 node2 node3 $ ls node1# puppetconf # puppetize @ puppetlabs
  42. 42. Every report from every runever $ ls node1 201206060256.yaml 201206060303.yaml 201206060519.yaml 201206122349.yaml 201206122354.yaml 201206130002.yaml# puppetconf # puppetize @ puppetlabs
  43. 43. Lets crack one open! Irb > require ‘yaml’ >reports=YAML.load_file(201206130002.yaml)# puppetconf # puppetize @ puppetlabs
  44. 44. Have a look >(reports.methods - Object.methods).sort Notice the following methods:# puppetconf # puppetize @ puppetlabs
  45. 45. High level data > reports.exit_status ⇒ 0 > reports.status => "unchanged" > reports.host ⇒ ”node1”# puppetconf # puppetize @ puppetlabs
  46. 46. metrics > reports.metrics.keys ⇒ ["resources", "events", "changes", "time"] > reports.metrics[resources] ⇒ [‘failed’, 0],[ ‘changed’, ‘7’]# puppetconf # puppetize @ puppetlabs
  47. 47. And the awesome sauce > reports.resource_statuses.keys => ["Package[xinetd]", "File[/srv/node/1]", "Package[swift]", "Exec[compile fragments]", "Package[swift-container]", "File[/var/opt/lib/pe- puppet/concat/_etc_swift_object-server.conf]", "File[/etc/rsync.d/frag-account]”]# puppetconf # puppetize @ puppetlabs
  48. 48. And the awesome sauce > status = reports.resource_statuses > status.keys => ["Package[xinetd]", "File[/srv/node/1]", "Package[swift]", "Exec[compile fragments]", "Package[swift-container]", "File[/var/opt/lib/pe- puppet/concat/_etc_swift_object-server.conf]", "File[/etc/rsync.d/frag-account]”]# puppetconf # puppetize @ puppetlabs
  49. 49. And the awesome sauce >events = status["File[/etc/swift/swift.conf]"].events > events.first.status ⇒ "success” > events.first.desired_value ⇒ :present > events.first.previous_value => :absent# puppetconf # puppetize @ puppetlabs
  50. 50. Thank YouDan Bode| Puppet Labsdan@puppetlabs.com

×