Your SlideShare is downloading. ×
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

PuppetDB: New Adventures in Higher-Order Automation - PuppetConf 2013

4,801

Published on

"PuppetDB: New Adventures in Higher-Order Automation" by …

"PuppetDB: New Adventures in Higher-Order Automation" by
Deepak Giridharagopal, Director of Engineering, Puppet Labs.

Presentation Overview: PuppetDB gives users fast, robust, centralized storage for Puppet-produced data. The 1.0 version landed at Puppetconf 2012, and now we're one year older and one year wiser. It's been deployed in thousands of sites, people have written libraries and tools on top of it, and there's been plenty of activity in the past year. We've tightly integrated it into Puppet Enterprise. We've added new features like report storage, event querying, import/export, better HTTP endpoints, and unified querying. And though we've added features, we've also made PuppetDB faster and consume less disk space. This talk will cover what's happened in the PuppetDB world between Puppetconf 2012 and now. We'll go into the new features, talk about performance and correctness, and discuss lessons learned.

Speaker Bio: Deepak is Director of Engineering at Puppet Labs, one of the authors of PuppetDB, and a many-times-over Puppetconf veteran. Prior to joining Puppet Labs, he was Principal Engineer at Dell/MessageOne, using Puppet to manage thousands of production systems.

Published in: Technology, Spiritual
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,801
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
110
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. The life and times of PuppetDB Friday, August 23, 13
  • 2. DEEPAK GIRIDHARAGOPAL deepak@puppetlabs.com @grim_radical Friday, August 23, 13
  • 3. We need to talk! Friday, August 23, 13
  • 4. Friday, August 23, 13
  • 5. Friday, August 23, 13
  • 6. Puppet agent Puppet master Friday, August 23, 13
  • 7. Puppet agent Puppet master facts Friday, August 23, 13
  • 8. Puppet agent Puppet master facts netmask_lo: 255.0.0.0 augeasversion: 0.10.0 fqdn: pe-debian6.localdomain manufacturer: "VMware, Inc." processorcount: "1" productname: VMware Virtual Platform physicalprocessorcount: 1 facterversion: 1.6.7 boardproductname: 440BX Desktop Reference Platform kernelmajversion: "2.6" hardwareisa: unknown timezone: PDT puppetversion: 2.7.12 (Puppet Enterprise 2.5.1) lsbdistcodename: squeeze is_virtual: "true" operatingsystemrelease: 6.0.2 virtual: vmware type: Other domain: localdomain hostname: pe-debian6 selinux: "false" kernel: Linux kernelrelease: 2.6.32-5-686 ipaddress: 172.16.245.128 processor0: Intel(R) Core(TM) i7-2635QM CPU @ 2.00GHz lsbdistrelease: 6.0.2 uniqueid: 007f0101 hardwaremodel: i686 kernelversion: 2.6.32 operatingsystem: Debian architecture: i386 lsbdistdescription: Debian GNU/Linux 6.0.2 (squeeze) lsbmajdistrelease: "6" interfaces: "eth0,lo" ipaddress_lo: 127.0.0.1 uptime_days: 0 lsbdistid: Debian rubysitedir: /opt/puppet/lib/site_ruby/ 1.8 rubyversion: 1.8.7 osfamily: Debian memorytotal: &id001 502.57 MB memorysize: *id001 boardmanufacturer: Intel Corporation path: /usr/local/sbin:/usr/local/bin:/ Friday, August 23, 13
  • 9. Puppet agent Puppet master facts Friday, August 23, 13
  • 10. Puppet agent Puppet master facts Friday, August 23, 13
  • 11. Puppet agent Puppet master catalog Friday, August 23, 13
  • 12. Puppet agent Puppet master catalog Friday, August 23, 13
  • 13. file {“/tmp/foo”: content => “This is a test”} Friday, August 23, 13
  • 14. target: &id063 !ruby/object:Puppet::Resource catalog: *id001 exported: false file: /etc/puppetlabs/puppet/manifests/site.pp line: 44 parameters: !ruby/sym content: This is a test !ruby/sym backup: main reference: "File[/tmp/foo]" tags: - file - node - default - class title: /tmp/foo type: File file {“/tmp/foo”: content => “This is a test”} Friday, August 23, 13
  • 15. File[/var/lib/peadmin/.vim] le[/var/lib/peadmin/.mcollective.d/peadmin-cert.pem] File[/var/lib/peadmin] File[/var/lib/peadmin/.bashrc.custom] File[/var/lib/peadmin/.bashrc] Group[peadmin] User[peadmin] Friday, August 23, 13
  • 16. Relationships File[/var/lib/peadmin/.mcollective.d/peadmin-public.pem] File[puppet-dashboard-public.pem] File[/var/lib/peadmin/.mcollective] File[/opt/puppet/shapeadmin/.mcollective.d/peadmin-private.pem] File[/var/lib/peadmin/.vim]Exec[mcollective-client-cert] File[/var/lib/peadmin/.mcollective.d/peadmin-cert.pem] Pe_accounts::Home_dir[/var/lib/peadmin]File[/var/lib/peadmin] File[/var/lib/peadmin/.bashrc.custom] File[/var/lib/peadmin/.bash_profile]File[/var/lib/peadmin/.bashrc]File[/var/lib/peadmin/.mcollective.d] File[/var/lib/peadmin/.ssh] File[/var/lib/peadmin/.ssh/authorized_keys] Pe_accounts::User[peadmin] Group[peadmin] User[peadmin] Friday, August 23, 13
  • 17. Relationships File[/opt/puppet/libexec/mcollective/mcollective/agent/service.rb] Service[mcollective] File[/opt/puppet/libexec/mcollective/mcollective/agent/service.ddl] File[/var/lib/peadmin/.mcollective.d/peadmin-public.pem] File[/opt/puppet/share/puppet-dashboard/.bashrc] File[/etc/puppetlabs/mcollective/ssl] File[/etc/puppetlabs/mcollective/ssl/clients]File[mcollective-cert.pem] File[mcollective-public.pem]File[mcollective-private.pem] File[peadmin-public.pem]File[/etc/puppetlabs/mcollective/ssl/clients/mcollective-public.pem] File[puppet-dashboard-public.pem] File[/var/lib/peadmin/.mcollective] File[/opt/puppet/share/puppet-dashboard/.mcollective] Class[Pe_accounts::Data] File[/opt/puppet/share/puppet-dashboard/.ssh/File[/opt/puppet/share/puppet-dashboard/.mcollective.d/puppet-dashboard-cert.pem] Pe_accounts::Home_dir[/opt/puppet/share/puppet-dashboard] File[/opt/puppet/share/puppet-dashbo File[/var/lib/peadmin/.mcollective.d/peadmin-private.pem] File[/var/lib/peadmin/.vim] File[/etc/puppetlabs/mcollective/server.cfg] File[/opt/puppet/share/puppet-dashboard/.mcollective.d] File[/opt/puppet/share/puppet-dashboard/.mcollective.d/puppet-dashboard-public.pem] File[/opt/puppet/share/puppet-dashboard/.mcollective.d/puppet-dashboard-private.pem]File[/opt/puppet/libexec/mcollective/mcollective/security/aespe_security.rb] Exec[mcollective-client-cert] File[/var/lib/peadmin/.mcollective.d/peadmin-cert.pem] File[/opt/puppet/libexec/mcollective/mcollective/agent] File[/opt/puppet/libexec/mcollective/mcollective/agent/puppetd.rb] File[/opt/puppet/libexec/mcollective/mcollective/agent/package.rb] File[/opt/puppet/libexec/mcollective/mcollective/agent/puppetd.ddl] File[/opt/puppet/libexec/mcollective/mcollective/agent/puppetral.ddl]/mcollective/mcollective/agent/puppetral.rb] File[/opt/puppet/libexec/mcollective/mcollective/agent/package.ddl] File[/opt/puppet/libexec/mcollective/mcollective/security/sshkey.rb] File[/opt/puppet/libexec/mcollective/mcollective/util] File[/opt/puppet/libexec/mcollective/mcollective/util/actionpolicy.rb] Pe_accounts::Home_dir[/var/lib/peadmin] Group[puppet-dashboard] File[/opt/puppet/share/puppet-dashboard] File[/opt/puppet/share/puppet-dashboard/.bash_profile] File[/opt/puppet/share/puppet-dashboard/.vim]File[/opt/puppet/share/puppet-dashboard/.bashrc.custom] User[puppet-dashboard] Exec[mcollective-server-cert] File[/var/lib/peadmin] File[/var/lib/peadmin/.bashrc.custom] File[/var/lib/peadmin/.bash_profile]File[/var/lib/peadmin/.bashrc]File[/var/lib/peadmin/.mcollective.d] File[/var/lib/peadmin/.ssh] File[/var/lib/peadmin/.ssh/authorized_keys] Exec[puppet-dashboard-client-cert]File[/opt/puppet/libexec/mcollective/mcollective/application/package.rb] Pe_accounts::User[peadmin] Pe_accounts::User[puppet-dashboard] File[/opt/puppet/libexec/mcollective/mcollective/application/service.rb] File[/opt/puppet/libexec/mcollective/mcollective/security] Group[peadmin] User[peadmin] File[/opt/puppet/libexec/mcollective/mcollective/registration/meta.rb] File[/opt/puppet/libexec/mcollective/mcollective/registration] File[/opt/puppet/libexec/mcollective/mcollective/application/puppetd.rb] Friday, August 23, 13
  • 18. Puppet agent Puppet master catalog Friday, August 23, 13
  • 19. Puppet agent Puppet master catalog Friday, August 23, 13
  • 20. Puppet agent Puppet master catalog Friday, August 23, 13
  • 21. Puppet agent Puppet master report Friday, August 23, 13
  • 22. Puppet agent Puppet master report "File[/tmp/foo]": !ruby/object:Puppet::Resource::Status change_count: 1 changed: true evaluation_time: 0.001869 events: - !ruby/object:Puppet::Transaction::Event audited: false desired_value: !ruby/sym file historical_value: message: *id006 name: !ruby/sym file_created previous_value: !ruby/sym absent property: ensure status: success time: 2011-10-25 18:51:37.143970 -07:00 failed: false file: *id007 line: 44 out_of_sync: true out_of_sync_count: 1 resource: "File[/tmp/foo]" resource_type: File skipped: false tags: - file - node - default - class time: 2011-10-25 18:51:37.143396 -07:00 title: /tmp/foo Friday, August 23, 13
  • 23. Puppet agent Puppet master report Friday, August 23, 13
  • 24. Puppet agent Puppet master report Friday, August 23, 13
  • 25. Puppet agent Puppet master Friday, August 23, 13
  • 26. Puppet agent Puppet master Friday, August 23, 13
  • 27. Friday, August 23, 13
  • 28. Friday, August 23, 13
  • 29. Puppet agent Puppet master PuppetDB Friday, August 23, 13
  • 30. Puppet agent Puppet master PuppetDB facts Friday, August 23, 13
  • 31. Puppet agent Puppet master PuppetDB facts Friday, August 23, 13
  • 32. Puppet agent Puppet master PuppetDB facts Friday, August 23, 13
  • 33. Puppet agent Puppet master PuppetDB catalog facts catalog Friday, August 23, 13
  • 34. Puppet agent Puppet master PuppetDB catalog catalog facts Friday, August 23, 13
  • 35. Puppet agent Puppet master PuppetDB catalog facts Friday, August 23, 13
  • 36. Puppet agent Puppet master PuppetDB catalog facts Friday, August 23, 13
  • 37. Puppet agent Puppet master PuppetDB report catalog facts Friday, August 23, 13
  • 38. Puppet agent Puppet master PuppetDB report catalog facts Friday, August 23, 13
  • 39. Puppet agent Puppet master PuppetDB report catalog facts Friday, August 23, 13
  • 40. Puppet agent Puppet master PuppetDB report catalog facts Friday, August 23, 13
  • 41. Friday, August 23, 13
  • 42. Friday, August 23, 13
  • 43. Active Record Puppet master catalog Friday, August 23, 13
  • 44. Active Record Puppet master catalogcatalogcatalogcatalogcatalogcatalog Friday, August 23, 13
  • 45. Active Record Puppet master catalogcatalogcatalogcatalogcatalog catalog Friday, August 23, 13
  • 46. Active Record Puppet master catalogcatalogcatalogcatalog catalogcatalog Friday, August 23, 13
  • 47. Active Record Puppet master catalogcatalogcatalog catalogcatalogcatalog Friday, August 23, 13
  • 48. Active Record Puppet master catalogcatalog catalogcatalogcatalogcatalog Friday, August 23, 13
  • 49. Active Record Puppet master catalog catalogcatalogcatalogcatalogcatalog Friday, August 23, 13
  • 50. Active Record Puppet master catalog catalog Friday, August 23, 13
  • 51. Puppet master catalog Friday, August 23, 13
  • 52. Friday, August 23, 13
  • 53. Puppet agent Puppet agent Puppet agent Puppet agent Puppet agent Puppet agent Puppet agent Puppet agent Puppet agent agent agent agent Puppet agent Puppet agent Puppet agent Puppet agent Puppet agent Puppet agent Puppet agent Puppet agent Puppet agentFriday, August 23, 13
  • 54. Friday, August 23, 13
  • 55. Active Record Friday, August 23, 13
  • 56. Active Record Which boxes are running nginx? Friday, August 23, 13
  • 57. Active Record Friday, August 23, 13
  • 58. Active Record How many servers are running a vulnerable version of rails? Friday, August 23, 13
  • 59. Active Record Friday, August 23, 13
  • 60. Active Record What are the IP addresses of my webservers? Friday, August 23, 13
  • 61. Active Record Friday, August 23, 13
  • 62. Active Record Which users have sudo access? Friday, August 23, 13
  • 63. Active Record Friday, August 23, 13
  • 64. Active Record Friday, August 23, 13
  • 65. Active Record Friday, August 23, 13
  • 66. Active Record Friday, August 23, 13
  • 67. Active Record Friday, August 23, 13
  • 68. Active Record Friday, August 23, 13
  • 69. Active Record Friday, August 23, 13
  • 70. Active Record LOLWUT Friday, August 23, 13
  • 71. Active Record LOLWUT ಠ ಠ _ Friday, August 23, 13
  • 72. And now for something completely different Friday, August 23, 13
  • 73. PuppetDB Friday, August 23, 13
  • 74. /resources/Service/ nginx PuppetDB Friday, August 23, 13
  • 75. resources /resources/Service/ nginx PuppetDB) O O Friday, August 23, 13
  • 76. PuppetDB Friday, August 23, 13
  • 77. /resources/Package/ rails PuppetDB Friday, August 23, 13
  • 78. resources /resources/Package/ rails PuppetDB) O O Friday, August 23, 13
  • 79. PuppetDB Friday, August 23, 13
  • 80. /nodes/foo.com/ resources/User/ deepak PuppetDB Friday, August 23, 13
  • 81. resources /nodes/foo.com/ resources/User/ deepak PuppetDB) O O Friday, August 23, 13
  • 82. (demo) Friday, August 23, 13
  • 83. We built something quite different Friday, August 23, 13
  • 84. 1. Asynchrony Friday, August 23, 13
  • 85. Storage & Querying Friday, August 23, 13
  • 86. CQRS Friday, August 23, 13
  • 87. Command Query Responsibility Separation use a different model to update information than the model you use to read information Friday, August 23, 13
  • 88. CQRS write pipeline async, parallel, MQ-based, with automatic retry Friday, August 23, 13
  • 89. { :command "replace catalog" :version 2 :payload {...} } Friday, August 23, 13
  • 90. /commands MQ Parse Delayed Dead Letter Office Process UUID Friday, August 23, 13
  • 91. Command processors must be retry-aware expect failure, because it *will* happen. Friday, August 23, 13
  • 92. Failures like, oh I don't know, a database crash? Friday, August 23, 13
  • 93. 2. New runtime Friday, August 23, 13
  • 94. Fast, Free, Portable, Multi-core, Popular, The JVM is all these things Friday, August 23, 13
  • 95. Haters gonna hate! Friday, August 23, 13
  • 96. Tons and tons of high quality libraries Web servers, concurrency frameworks, databases, fast parsing/lexing, clustering, debugging, profiling, etc. Friday, August 23, 13
  • 97. Can ship an uberjar, makes deployment straightforward with few moving pieces Friday, August 23, 13
  • 98. And it's fast. Friday, August 23, 13
  • 99. Nobody cares what runtime we use. Users just want stuff to work. Friday, August 23, 13
  • 100. 3. AST querying Friday, August 23, 13
  • 101. Queries are expressed in their own“language” domain specific,AST-based query language Friday, August 23, 13
  • 102. ["and", ["=", "type", "User"], ["=", "title", "deepak"]] Friday, August 23, 13
  • 103. ["and", ["=", ["fact", "operatingsystem"], "Debian"], ["<", ["fact", "uptime_seconds"], 10000]] Friday, August 23, 13
  • 104. ["and", ["=", "name", "ipaddress"], ["in", "certname", ["extract", "certname", ["select-resources", ["and", ["=", "type", "Class"], ["=", "title", "Apache"]]]] Friday, August 23, 13
  • 105. ["or", ["=", "certname", "foo.com"], ["=", "certname", "bar.com"], ["=", "certname", "baz.com"]] Friday, August 23, 13
  • 106. We walk the tree, compiling it to efficient SQL Friday, August 23, 13
  • 107. Haters gonna hate! Friday, August 23, 13
  • 108. AST-based API lets users write their own languages ah, you’ve got to love open source! Friday, August 23, 13
  • 109. (Package[httpd] and country=fr) or country=us Package["mysql-server"] and architecture=amd64 Erik Dalén, Spotify https://github.com/dalen/puppet-puppetdbquery Friday, August 23, 13
  • 110. AST-based API lets us more safely manipulate queries Friday, August 23, 13
  • 111. daenny, Puppetboard https://github.com/nedap/puppetboard Friday, August 23, 13
  • 112. Puppet Enterprise, Event Inspector https://puppetlabs.com Friday, August 23, 13
  • 113. Foreman Integration (CERN) https://github.com/cernops/puppetdb_foreman Web UI https://github.com/dima-exe/puppetdb-db Web UI https://github.com/gbougeard/puppetdb-frontend Friday, August 23, 13
  • 114. Ruby https://github.com/dalen/puppet-puppetdbquery Ruby (DataMapper) https://github.com/dalen/dm-puppetdb-adapter Ruby https://github.com/ripienaar/ruby-puppetdb Friday, August 23, 13
  • 115. Python https://github.com/nedap/pypuppetdb Python https://github.com/arcus-io/puppetdb-python Python https://github.com/JHaals/puppetdb-grep Friday, August 23, 13
  • 116. Java https://github.com/thallgren/puppetdb-javaclient Go https://github.com/nightlyone/puppetquery Scala https://github.com/gbougeard/puppetdb-frontend CoffeeScript https://gist.github.com/pmuellr/5591686 Node.js https://github.com/nightfly19/minidb Friday, August 23, 13
  • 117. MCollective https://github.com/ploubser/mcollective-puppetdb- discovery Rundeck https://github.com/sirhopcount/puppetdb-rundeck Rundeck https://github.com/martin2110/puppetdb-rundeck Friday, August 23, 13
  • 118. OpenStack https://github.com/bodepd/puppet- openstack_puppetdb Vagrant https://github.com/grahamgilbert/vagrant- puppetmaster PowerDNS https://github.com/evenup/evenup-pdns Friday, August 23, 13
  • 119. 4. Boring technology Friday, August 23, 13
  • 120. Relational Database, embedded or PostgreSQL because they’re actually pretty fantastic at ad-hoc queries, aggregation, windowing, etc. while maintaining safety Friday, August 23, 13
  • 121. Friday, August 23, 13
  • 122. Relational Database, embedded or PostgreSQL we use arrays, recursive queries, indexing inside complex structures Friday, August 23, 13
  • 123. 5. Weird alien technology Friday, August 23, 13
  • 124. Friday, August 23, 13
  • 125. --Jeff Gagliardi Friday, August 23, 13
  • 126. Thousands of deployments, Hundreds of threads per install, Zero deadlocks, Zero bugs involving mutable state companion Ruby code has ~10x the defect rate Friday, August 23, 13
  • 127. All with a pretty tiny codebase Friday, August 23, 13
  • 128. 6. Conjectures about performance Friday, August 23, 13
  • 129. Posit: A resource often exists across multiple hosts Friday, August 23, 13
  • 130. Feature: Single-instance resource storage Friday, August 23, 13
  • 131. Posit: We’ll often receive the same catalog for a host Friday, August 23, 13
  • 132. Feature: Single-instance catalog storage Friday, August 23, 13
  • 133. In the field,we almost always see Resource and catalog duplication rates of over 85%. Friday, August 23, 13
  • 134. Monitoring and instrumentation is a big deal.Users want easy ways to consume metrics and analyze performance. Friday, August 23, 13
  • 135. Friday, August 23, 13
  • 136. Nagios https://github.com/jasonhancock/nagios-puppetdb Nagios https://github.com/favoretti/puppetdb-external-naginator Munin https://github.com/vpetersson/munin_puppetdb Munin https://github.com/dalen/puppetdb-muninplugins Collectd https://gist.github.com/mfournier/5615125 Friday, August 23, 13
  • 137. Turns out, people appreciate these efforts Friday, August 23, 13
  • 138. (how many?) Friday, August 23, 13
  • 139. Thousands of production deployments Small shops with a dozen hosts, large shops with thousands of hosts,standalone,clustered... Friday, August 23, 13
  • 140. There is a new deployment of PuppetDB every 15 minutes. Friday, August 23, 13
  • 141. So...long time since we last spoke Friday, August 23, 13
  • 142. Availability Friday, August 23, 13
  • 143. Available in PE3 On by default,fully supported, and the basis for upcoming reporting and analytics features. Friday, August 23, 13
  • 144. Performance Friday, August 23, 13
  • 145. 20% faster storage Improvements to memoization and caching,eliminate double- serialization,nuked superfluous indexes Friday, August 23, 13
  • 146. Much faster terminus Better caching and data structures.For a catalog with 10k resources,drops serialization time from ~80s to ~6s. Friday, August 23, 13
  • 147. Resiliance Friday, August 23, 13
  • 148. Death to keystores Can now use PEM certificates directly,eliminating one of the largest sources of configuration problems. Friday, August 23, 13
  • 149. Configurable HTTPS Can customize the set of cipher suites and SSL protocols you'd like to use,to match your security needs. Friday, August 23, 13
  • 150. Automatic: -Recovery from MQ corruption -Compression of the DLO -Purging of inactive node data -DB connection recycling Friday, August 23, 13
  • 151. Backup and restore Now integrated into the daemon,can restore while PuppetDB is running. Friday, August 23, 13
  • 152. Query changes Friday, August 23, 13
  • 153. V2 API -No need to ask for only active nodes -Full fact queries (instead of just a list of facts for a node) -Node metadata Friday, August 23, 13
  • 154. Wildcard Accept Headers curl localhost:8080/v2/nodes Friday, August 23, 13
  • 155. Subqueries You can now correlate data from resource queries with fact queries with node queries. "Give me the IP address of all machines with the Nginx service configured" Friday, August 23, 13
  • 156. Report storage -Comes with a report processing plugin -Store report-level metadata -Can do queries on events that span reports -Basis for PE's Event Inspector Friday, August 23, 13
  • 157. Streaming queries! Friday, August 23, 13
  • 158. Streaming queries Stream results to clients on-the- fly,as they come in from the database. Massively lower latency for first response! Friday, August 23, 13
  • 159. resourceresourceresourceresourceresourceresourceresource PuppetDB Friday, August 23, 13
  • 160. resourceresourceresourceresourceresourceresourceresource /v2/resources PuppetDB Friday, August 23, 13
  • 161. resourceresourceresourceresourceresourceresourceresource /v2/resources PuppetDB) O O Friday, August 23, 13
  • 162. (demo) Friday, August 23, 13
  • 163. Coming up! Friday, August 23, 13
  • 164. We will be developing tools to replicate data from one PuppetDB daemon to another. This will help with HA and DR. PuppetDB Diff & Mirror PuppetDB Friday, August 23, 13
  • 165. By initially developing an out-of-band mirroring tool, we can create more interesting replication topologies: PuppetDB Diff & Mirror PuppetDB Diff & Mirror Friday, August 23, 13
  • 166. We can also later optimize the process to lower latency, but preserve eventual consistency: PuppetDB Diff & Mirror PuppetDBDirect MQ connection Friday, August 23, 13
  • 167. More flexible routing is coming, allowing for soft failures and read/write splits: PuppetDB Puppetmaster PuppetDB Replication Catalogs,Facts, Reports Collection queries Log error and continue Friday, August 23, 13
  • 168. So anyways, Friday, August 23, 13
  • 169. Documented at http:// docs.puppetlabs.com /puppetdb install, config, upkeep, specs, the works! Friday, August 23, 13
  • 170. Packaged as deb and rpm for open source,part of Puppet Enterprise available in the Puppet Labs package repositories Friday, August 23, 13
  • 171. Puppetized using the puppetlabs/puppetdb module available now, on the Module Forge! Friday, August 23, 13
  • 172. Open source http://github.com/ puppetlabs/puppetdb same license as Puppet itself! Friday, August 23, 13
  • 173. deepak giridharagopal deepak@puppetlabs.com @grim_radical [github twitter freenode] Friday, August 23, 13

×