![Exported Resources
Nagios_host <<||>> ~> Service[‘nagios’]
Nagios_service <<||>> ~> Service[‘nagios’]
16](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-16-2048.jpg)
![Exported Resources - Removing
# puppet node deactivate <node name> [<another node>, ...]
17](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-17-2048.jpg)
![Querying Facts
• /v2/fact_names
[ "architecture", "augeasversion",
"boardmanufacturer",
"boardproductname",...]
63](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-63-2048.jpg)
![Querying Facts
[ {
"certname" : "my_host.example.com",
"name" : "architecture",
"value" : "x86_64"
}, {
"certname" : "my_host.example.com",
"name" : "augeasversion",
"value" : "0.10.0"
}, ...
]
65](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-65-2048.jpg)
![Querying Resources
[ {
"parameters" : {
"ensure" : "running",
"enable" : true
},
"sourceline" : 139,
"sourcefile" : "/etc/puppetlabs/puppet/modules/puppetdb/
manifests/server.pp",
...
"title" : "pe-puppetdb",
"type" : "Service",
"certname" : "puppetdb"
}, ... ]
67](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-67-2048.jpg)
![Query Format
http://localhost:8080
/v2
/endpoint
query=[“=”, ”certname”, “my_host.com”]
70](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-70-2048.jpg)
![Query Format
[
"and",
["=", "type", "Service"],
["=", "title", "httpd"]
]
71](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-71-2048.jpg)
![Query Example
curl -G
-H ‘Accept: application/json’
http://localhost:8080/v2/resources
--data-urlencode
‘query=[“=”,“sourcefile”,“/etc/puppet/manifests/site.pp”]’
73](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-73-2048.jpg)
![Helpful Filters
/v2/resources/User
["~","sourcefile","/opt/puppet/share/puppet/modules"]
/v2/resources
[
"and",
["=", "exported", true],
["=", "certname", "exporter.example.com"]
]
76](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-76-2048.jpg)
![dalen/puppetdbquery
puppet query nodes '(Package["mysql-server"])'
puppet query nodes '(architecture=amd64)'
77](https://image.slidesharecdn.com/puppetdbpugnyc-130604171846-phpapp01/75/PuppetDB-A-Single-Source-for-Storing-Your-Puppet-Data-PUG-NY-77-2048.jpg)
Uploaded byPuppet
PuppetDB: A Single Source for Storing Your Puppet Data - PUG NY
This document provides an introduction to PuppetDB, Puppet's datastore for information generated during Puppet runs, outlining its components and architecture. It discusses data handling, including facts, catalogs, and exported resources, as well as installation and querying methods. Additionally, it highlights the scalability options and upcoming features for users managing complex infrastructures.
More Related Content
![[JAM 1.2] Design & Multitasking (Andrew Solovey)](https://cdn.slidesharecdn.com/ss_thumbnails/1-110415015750-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to PuppetDB: A Single Source for Storing Your Puppet Data - PUG NY
PuppetDB: A Single Source for Storing Your Puppet Data - PUG NY
- 1. IT Automation Softwarefor System Administrators
- 2. James Sweeny james.sweeny@puppetlabs.com Puppet Labs, Inc. Professional Services A User’s Introduction to PuppetDB 2
- 3. What is PuppetDB *The* datastore for informaBon generated by Puppet 3
- 4. PuppetDB Components • HTTP Server • Middleware Queue • Worker Threads • Database 4
- 5. PuppetDB Components • HTTP Server • Middleware Queue • Worker Threads • Database 5
- 6. Yes,it runs ina JVM • WriMen in Clojure • And yes, it’s fast. 6
- 7.
- 8. Lots of Data Small environment: ~100 nodes ~800KiB catalog 30 minute interval >3GiB/day 8
- 9. Lots of Data Large environment: Start adding zeros. Some users generate over 750GiB/day 9
- 10. Puppet Data • Facts •Catalogs • Reports • Exported Resources 10
- 11. Facts • Collected with *every* puppet run • Contain lots of data about your systems • Available from the inventory service 11 architecture => i386 augeasversion => 0.9.0 boardmanufacturer => Intel CorporaBon boardproductname => 440BX Desktop Reference Plaeorm boardserialnumber => None domain => localdomain facterversion => 1.6.7 fqdn => pe-‐centos6.localdomain hardwareisa => i686 hardwaremodel => i686 hostname => pe-‐centos6 id => root interfaces => eth1,lo ipaddress => 172.16.199.144 ipaddress_eth1 => 172.16.199.144 ipaddress_lo => 127.0.0.1 is_virtual => true kernel => Linux kernelmajversion => 2.6 kernelrelease => 2.6.32-‐220.13.1.el6.i686 kernelversion => 2.6.32 macaddress => 00:0C: 29:19:A2:33 macaddress_eth1 => 00:0C: 29:19:A2:33 manufacturer => VMware, Inc. memoryfree => 382.88 MB memorysize => 499.15 MB memorytotal => 499.15 MB netmask => 255.255.255.0 netmask_eth1 => 255.255.255.0 netmask_lo => 255.0.0.0 network_eth1 => 172.16.199.0 network_lo => 127.0.0.0 operaBngsystem => CentOS operaBngsystemrelease => 6.2 osfamily => RedHat physicalprocessorcount => 1 processor0 => Intel(R) Core(TM) i7-‐2620M CPU @ 2.70GHz processorcount => 1
- 12. Catalogs • StaBc configuraBon of every Puppet run • What the node *should* be 12
- 13. Exported Resources # Exportthe host file entry @@host { $::fqdn: host_aliases => $::hostname, ip => $::ipaddress, } # Collect and apply all the host file entries Host <<| |>> 13
- 14.
- 15. Exported Resources @@nagios_host {$::fqdn: ensure => present, alias => $::hostname, address => $::ipaddress, use => ‘generic-host’, } @@nagios_service { “check_ping_${::hostname}”: check_command => “check_ping!100.0,20%!500.0,60%”, use => ‘generic-service’, notification_period => ‘24x7’, service_description => “${hostname}_check_ping”, } 15
- 16. Exported Resources Nagios_host <<||>>~> Service[‘nagios’] Nagios_service <<||>> ~> Service[‘nagios’] 16
- 17. Exported Resources -Removing # puppet node deactivate <node name> [<another node>, ...] 17
- 18.
- 19. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc
- 20. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc
- 21. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc F
- 22. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc CF C
- 23. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc F C C
- 24. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 25. R Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 26. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C R
- 27.
- 28. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc R
- 29. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc R
- 30. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc F R F
- 31. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc ?F R
- 32. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc ? R F
- 33. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc ? R F
- 34. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc R F
- 35. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc RF
- 36. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc RF
- 37. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc CC
- 38. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc C C
- 39. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc C
- 40. Failure and theDLO 40
- 41. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts Catalo g Resrc
- 42. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc
- 43. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc F
- 44. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc CF C
- 45. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc F C C
- 46. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 47. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 48. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 49. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 50. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 51. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 52. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 53. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 54. Agent Master PuppetDB Server HTTPMQ DB Workers DLO Facts F Catalo g Resrc C C
- 55. Dead Letter Office Contains complete content of failed transacBons 55
- 56. Installation 1.Install Puppet 2.Enable PuppetLabs repos 3. puppet module install puppetlabs/puppetdb 4.Classify your servers 56
- 57. Installation node puppetmaster { class{ 'puppetdb': } -> class { 'puppetdb::master::config': } } 57
- 58. Scaling 1.Move HSQLDB to Postgres (at > 100 nodes) 2.Separate Postgres 3.MulBple PuppetDB servers 4.Clustered Postgres 5.HTTP(s) proxy 58
- 59.
- 60. REST Interface • By default listens on localhost:8080 • Can easily set public listener class { ‘puppetdb’: ssl_listen_address => $::ipaddress_eth0, ssl_listen_port => ‘8081’, } 60
- 61. REST Interface • Queries are processed immediately • Commands are queued 61
- 62. REST Interface • Responses are JSON • Differ by query type 62
- 63. Querying Facts • /v2/fact_names ["architecture", "augeasversion", "boardmanufacturer", "boardproductname",...] 63
- 64. Querying Facts • /v2/facts •/v2/facts/<fact_name> • /v2/facts/<fact_name>/<fact_value> • Fact Name • Fact Value • Node Name 64
- 65. Querying Facts [ { "certname": "my_host.example.com", "name" : "architecture", "value" : "x86_64" }, { "certname" : "my_host.example.com", "name" : "augeasversion", "value" : "0.10.0" }, ... ] 65
- 66. Querying Resources • Query all resources in your infrastructure • /v2/resources/ • /v2/resources/<type> • /v2/resources/<type>/<9tle> 66
- 67. Querying Resources [ { "parameters": { "ensure" : "running", "enable" : true }, "sourceline" : 139, "sourcefile" : "/etc/puppetlabs/puppet/modules/puppetdb/ manifests/server.pp", ... "title" : "pe-puppetdb", "type" : "Service", "certname" : "puppetdb" }, ... ] 67
- 68. Querying Nodes • Query all or specific nodes for facts and resources • /v2/nodes • /v2/nodes/<node name> • /v2/nodes/<node name>/resources • /v2/nodes/<node name>/facts 68
- 69. Query Format • HTTP GET • Endpoint is the URL • Query is the data 69
- 70.
- 71. Query Format [ "and", ["=", "type","Service"], ["=", "title", "httpd"] ] 71
- 72. Query Operators • Binary Operators • = (equality) • > (greater than) • < (less than) • >= (less than or equal to) • <= (greater than or equal to) • ~ (regexp match) • Boolean Operators • and • or • not 72
- 73. Query Example curl -G -H ‘Accept: application/json’ http://localhost:8080/v2/resources --data-urlencode ‘query=[“=”,“sourcefile”,“/etc/puppet/manifests/site.pp”]’ 73
- 74. Helpful Curl Switches -G -H‘Accept: application/json’ --data-urlencode <data> --cacert /etc/puppet/ssl/certs/ca.pem --cert /etc/puppet/ssl/certs/<node>.pem --key /etc/puppet/ssl/private_keys/<node>.pem 74
- 75. Putting your Queriesin Files --data-urlencode query@<filename> 75
- 76. Helpful Filters /v2/resources/User ["~","sourcefile","/opt/puppet/share/puppet/modules"] /v2/resources [ "and", ["=", "exported",true], ["=", "certname", "exporter.example.com"] ] 76
- 77. dalen/puppetdbquery puppet query nodes'(Package["mysql-server"])' puppet query nodes '(architecture=amd64)' 77
- 78. Upcoming Features • Report and Event endpoints – available now under /experimental/ • MCollecBve discovery plugin 78
- 79.
- 80. Other Resources • hMp://docs.puppetlabs.com/puppetdb •hMp://github.com/puppetlabs/puppetdb • hMp://projects.puppetlabs.com/projects/ puppetdb • #puppet on freenode 80
- 81.