Node collaboration - Exported Resources and PuppetDB

3,347 views

Published on

Node Collaboration - How can your servers share information with each other. Exploring Exported Resources, PuppetDB and other methods.

This talk was given at Sydney Puppet Users Meetup on 14/08/2014.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,347
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
28
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • The Datastore for information generated by Puppet
  • Large inventory of metadata about every node in your infrastructure and a searchable database of every single resource being managed on any node.
  • Large inventory of metadata about every node in your infrastructure and a searchable database of every single resource being managed on any node.
  • Return all nodes matching the given queryprocessorcount
  • Return all nodes matching the given queryprocessorcount
  • Return all nodes matching the given queryprocessorcount
  • Return all nodes matching the given queryprocessorcount
  • Query based upon resources and facts
    Returns a list of nodes
  • Returns a hash of facts
  • Returns a hash of facts
  • Returns a hash of facts
  • Query_node - Accepts 2 arguments
    Query to discover nodes
    Fact that should be returned (optional)
    Returns array of certnames or fact values

    Query_facts – requires 2 arguments
    query
    list of facts to return in a nested hash
  • Accepts 2 arguments
    Query to discover nodes
    Fact that should be returned (optional)
    Returns array of certnames or fact values
  • Problems with PuppetDB – puppetdbquery generates results based on compiled catalogs by the puppet master. We are not actually sure whether resources have been applied successfully

    How frequently do you run Puppet?
    Too slowly and infrastructure is slow to adapt to change
    Too fast and there maybe race conditions.
  • Node collaboration - Exported Resources and PuppetDB

    1. 1. Michael Richardson @m_richo
    2. 2. This talk contains both Facts and Opinions*. *don’t believe everything I say.
    3. 3. Puppet’s answer
    4. 4. Puppet’s answer Exported Resources!
    5. 5. Exported Resources * https://docs.puppetlabs.com/puppet/latest/reference/lang_exported.html Definition: An exported resource declaration specifies a desired state for a resource, does not manage the resource on the target system, and publishes the resource for use by other nodes. Any node (including the node that exported it) can then collect the exported resource and manage its own copy of it. *
    6. 6. Exported Resources
    7. 7. Exported Resources For those that can’t afford DNS servers…
    8. 8. Exported Resources Common Examples • Backend servers configured in a load balancer • Monitoring Servers updated with monitoring clients (nagios_host / nagios_service) • Distribute public keys (ssh and openssl)
    9. 9. Exported Resources Question for the audience
    10. 10. Exported Resources Question for the audience What do you think of Exported Resources?
    11. 11. Exported Resources Question for the audience What do you think of Exported Resources? Great. Alright. So so. Let’s explore alternatives
    12. 12. What about our friends @
    13. 13. Ohai Node data (think facter) is automatically added, indexed and searchable in the Chef Server. No need to export facts or resources.
    14. 14. How to use Chef Search Query the the Chef server data via • Method within recipe (think puppet manifest) • Search subcommand of ‘knife’ • /search or /search/INDEX endpoints of Chef Server API
    15. 15. How to use Chef Search 1. Method within recipe (think puppet manifest)
    16. 16. How to use Chef Search 1. Method within recipe (think puppet manifest)
    17. 17. Chef Search What do people think?
    18. 18. Chef Search What do people think? Personally I feel this approach can sometimes be better than exported resources approach. * * Remember this is an opinion 
    19. 19. But what about Puppet?
    20. 20. Introducing PuppetDB
    21. 21. PuppetDB Definition: collects data generated by Puppet. It enables advanced Puppet features like the inventory service and exported resources, and can be the foundation for other applications that use Puppet’s data. * https://docs.puppetlabs.com/puppetdb/2.1/index.html
    22. 22. PuppetDB • Written in Clojure. • Runs on the JVM • Backed by PostgreSQL • Scales horizontally
    23. 23. PuppetDB PuppetDB stores: • The most recent facts from every node • The most recent catalog for every node • Optionally, 14 days of event reports for every node • Exported Resources
    24. 24. PuppetDB PuppetDB REST API V3 API includes the following endpoints: • Facts • Resources • Nodes • Fact-names, • Metrics • Reports • Events • And more
    25. 25. PuppetDB GET /v3/nodes/
    26. 26. PuppetDB GET /v3/nodes/ QUERY = (JSON array of queries)
    27. 27. PuppetDB GET /v3/nodes/<NODE>/facts Return all facts for given node
    28. 28. PuppetDB GET /v3/nodes/<NODE>/facts/<NAME> Return facts by their name
    29. 29. PuppetDB I know what you’re thinking So much awesome information How can I make use of it?
    30. 30. PuppetDB checkout Puppetdbquery module by Erik Dalén https://forge.puppetlabs.com/dalen/puppetdbquery https://github.com/dalen/puppet-puppetdbquery
    31. 31. PuppetDB Puppetdbquery module • Command line tools • Puppet functions to query PuppetDB • Hiera backend to return query results from PuppetDB
    32. 32. PuppetDB Puppetdbquery CLI - nodes
    33. 33. PuppetDB Puppetdbquery CLI – facts
    34. 34. PuppetDB Puppetdbquery functions in manifests • query_nodes • query_facts
    35. 35. PuppetDB Puppetdbquery functions
    36. 36. PuppetDB Puppetdbquery functions note: Output is abbreviated
    37. 37. Where to from here? Exported Resources vs PuppetDB + puppetdbquery
    38. 38. Where to from here? What about something completely different
    39. 39. Where to from here? What about something completely different • Confd • Etcd • Consul • Zookeeper • Doozer • Skydns
    40. 40. Where to from here? What about something completely different • Confd • Etcd • Consul • Zookeeper • Doozer • Skydns Really interesting space… but that’s for another talk. 
    41. 41. Summary How can you share information between your nodes/servers?
    42. 42. Summary How can you share information between your nodes/servers? Plenty of ways • Exported resources • PuppetDB • Many, many others.
    43. 43. Summary How can you share information between your nodes/servers? Plenty of ways • Exported resources • PuppetDB • Many, many others. Pick the method that works best for you
    44. 44. Thank you
    45. 45. Thank you #protip go buy this book now By Alessandro Franceschi It is excellent!
    46. 46. IS HIRING
    47. 47. QUESTIONS?

    ×