Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Intro to open source telemetry linux con 2016
Next
Download to read offline and view in fullscreen.

1

Share

Download to read offline

Snap Telemetry Framework & Plugin Architecture at GrafanaCon 2016

Download to read offline

Joel Cooklin and I cover the usage and design of Snap, an open telemetry framework. I also refer to Jason Dixon a few times because I'm a fan of his presentation that influenced my view of monitoring. If you love monitoring and don't mind profanity, check this out first: https://speakerdeck.com/obfuscurity/the-state-of-open-source-monitoring

Related Books

Free with a 30 day trial from Scribd

See all

Snap Telemetry Framework & Plugin Architecture at GrafanaCon 2016

  1. 1. Snap A story of telemetry, data, python And references to Jason Dixon at GrafanaCon 2016
  2. 2. Hey
  3. 3. The Snap Telemetry Framework
  4. 4. 4 what my friends think telemetry is what my parents think telemetry is what society thinks telemetry is what my boss thinks telemetry is what I think telemetry is what telemetry actually is
  5. 5. (Do I get bonus points for quoting Jason at GrafanaCon?) “Telemetry refers to the collection of measurements, typically of remote instruments, for the purposes of monitoring.” ~ Jason Dixon in The Graphite Book
  6. 6. Telemetry Framework ™
  7. 7. Client libraries for: ● Prometheus ● Sensu ● Statsd ● & more ● beats ● tcollector ● scollector ● Nagios ● Snap ● collectd ● telegraf ● diamond
  8. 8. Requirements Composable Declarative Extensible
  9. 9. Composable Monitoring Snap
  10. 10. Composable Monitoring Snap
  11. 11. Declarative Explicit definition of - Interval - Collected metrics - Published endpoints Allows for - Collect from multiple sensors - Publish to multiple endpoints - Multiple workflows at different intervals - Metrics available upon plugin update Let’s look at a Task Manifest “I’ll make you a workflow you can’t refuse”
  12. 12. Extensible Collect Process Publish
  13. 13. Extensible Available in the Plugin Catalog Collectors written: ● Nginx ● Couchbase ● MongoDB ● Netstat ● Procstat Publishers written: ● Blueflood ● Couchbase
  14. 14. Extensible Available in the Plugin Catalog Collectors written: ● Nginx ● Couchbase ● MongoDB ● Netstat ● Procstat Publishers written: ● Blueflood ● Couchbase
  15. 15. Extensible Available in the Plugin Catalog Collectors written: ● Nginx ● Couchbase ● MongoDB ● Netstat ● Procstat Publishers written: ● Blueflood ● Couchbase
  16. 16. Extensible Available in the Plugin Catalog Collectors written: ● Nginx ● Couchbase ● MongoDB ● Netstat ● Procstat Publishers written: ● Blueflood ● Couchbase Grafana Labs
  17. 17. The Anatomy of a Snap Plugin
  18. 18. Plugin Authoring - Plugin Types Collectors Processors Publishers
  19. 19. Plugin Authoring - Lifecycle Loading a plugin
  20. 20. Plugin Authoring - Lifecycle Loading a plugin 1. Snap starts the plugin
  21. 21. Plugin Authoring - Lifecycle Loading a plugin 1. Snap starts the plugin 2. Snap negotiates with the plugin over stdout (handshake)
  22. 22. Plugin Authoring - Lifecycle Loading a plugin 1. Snap starts the plugin 2. Snap negotiates with the plugin over stdout (handshake) 3. Snap calls the plugin a. To get its ConfigPolicy
  23. 23. Plugin Authoring - Lifecycle Loading a plugin 1. Snap starts the plugin 2. Snap negotiates with the plugin over stdout (handshake) 3. Snap calls the plugin a. To get its config policy b. To get what metrics it collects
  24. 24. Plugin Authoring - Lifecycle Loading a plugin 1. Snap starts the plugin 2. Snap negotiates with the plugin over stdout (handshake) 3. Snap calls the plugin a. To get its config policy b. To get what metrics it exposes 4. Snap stops the plugin
  25. 25. Plugin Authoring - Lifecycle Starting a task
  26. 26. Plugin Authoring - Lifecycle Starting a task 1. Snap ensures plugins required by the task are running
  27. 27. Plugin Authoring - Lifecycle Starting a task 1. Snap ensures plugins required by the task are running a. Plugins that are used have their subscriptions increased
  28. 28. Starting a task 1. Snap ensures needed plugins required by the task are running a. Plugins that are used have their subscriptions increased Plugin subscriptions drive the logic related to plugin routing and concurrency Plugin Authoring - Lifecycle
  29. 29. Plugin Authoring - Lifecycle Stopping a task
  30. 30. Plugin Authoring - Lifecycle Stopping a task Causes plugins to stop When the plugin’s subscription count falls below zero
  31. 31. Plugin Authoring - Meta Plugin Meta
  32. 32. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts
  33. 33. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts {"Meta":{"Type":2,"Name":"graphite","Version":5,"RPCType":2," RPCVersion":1,"ConcurrencyCount":5,"Exclusive":false,"Cach eTTL":0,"RoutingStrategy":0},"ListenAddress":"127.0.0.1:37166"," PprofAddress":"0"}
  34. 34. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - Plugin name as it will appear in the plugin catalog
  35. 35. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher
  36. 36. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin
  37. 37. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used
  38. 38. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ○ The default is ‘gRPC’ ○ We will maintain legacy support for GORPC based plugins
  39. 39. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ● RPCVersion - Defines the version of the service the plugin implements
  40. 40. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ● RPCVersion - Defines the version of the service the plugin implements service Collector { rpc CollectMetrics(MetricsArg) returns (MetricsReply) {} rpc GetMetricTypes(GetMetricTypesArg) returns (MetricsReply) {} rpc Ping(Empty) returns (ErrReply) {} rpc Kill(KillArg) returns (ErrReply) {} rpc GetConfigPolicy(Empty) returns (GetConfigPolicyReply) {} }
  41. 41. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ● RPCVersion - Defines the version of the service the plugin implements service Processor { rpc Process(PubProcArg) returns (MetricsReply) {} rpc Ping(Empty) returns (ErrReply) {} rpc Kill(KillArg) returns (ErrReply) {} rpc GetConfigPolicy(Empty) returns (GetConfigPolicyReply) {} }
  42. 42. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ● RPCVersion - Defines the version of the service the plugin implements service Publisher { rpc Publish(PubProcArg) returns (ErrReply) {} rpc Ping(Empty) returns (ErrReply) {} rpc Kill(KillArg) returns (ErrReply) {} rpc GetConfigPolicy(Empty) returns (GetConfigPolicyReply) {} }
  43. 43. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ● RPCVersion - Defines the version of the service the plugin implements ● RoutingStrategy - Least Recently Used Sticky (task) Config based
  44. 44. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ● RPCVersion - Defines the version of the service the plugin implements ● RoutingStrategy - Least recently used, Sticky (task) or Config based ● ConcurrencyCount - Defines the maximum number of subscriptions the plugin can have before starting another instance of the plugin
  45. 45. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ● RPCVersion - Defines the version of the service the plugin implements ● RoutingStrategy - Least recently used, Sticky (task) or Config based ● ConcurrencyCount - Defines the maximum number of subscriptions the plugin can have before starting another instance of the plugin ● Exclusive - Results in a single instance of the plugin running for any number of subscriptions (ConcurrencyCount is ignored)
  46. 46. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ● RPCVersion - Defines the version of the service the plugin implements ● RoutingStrategy - Least recently used, Sticky (task) or Config based ● ConcurrencyCount - Defines the maximum number of subscriptions the plugin can have before starting another instance of the plugin ● Exclusive - Results in a single instance of the plugin running for any number of subscriptions ● CacheTTL - Overrides the default CacheTTL of 500ms
  47. 47. Plugin Authoring - Meta Plugin Meta Is communicated over STDOUT between the plugin and Snap when a plugin starts ● Name - name as it will appear in the plugin catalog ● Type - collector, process, publisher ● Version - version of the plugin ● RPCType - RPC mechanism to be used ● RPCVersion - Defines the version of the service the plugin implements ● RoutingStrategy - Least recently used, Sticky (task) or Config based ● ConcurrencyCount - Defines the maximum number of subscriptions the plugin can have before starting another instance of the plugin ● Exclusive - Results in a single instance of the plugin running for any number of subscriptions ● CacheTTL - Overrides the default CacheTTL of 500ms If the framework attempts to call a plugin for a metric before it’s TTL has expired it will be retrieved from the cache instead
  48. 48. Plugin Authoring - Plugin libs Want to write a plugin?
  49. 49. Plugin Authoring - Plugin libs Want to write a plugin? Start with one of the Snap plugin libs
  50. 50. Plugin Authoring - Plugin libs Want to write a plugin? Start with one of the Snap plugin libs
  51. 51. Plugin Authoring - Plugin libs Want to write a plugin? Start with one of the Snap plugin libs
  52. 52. Plugin Authoring - Plugin libs Want to write a plugin? Start with one of the Snap plugin libs
  53. 53. Plugin Authoring Leveraging existing investments ● Using the plugin-libs wrap existing tools ○ Example: snap-plugin-collector-diamond
  54. 54. The Snap Telemetry Framework To Review
  55. 55. The Snap Telemetry Framework Snap is an Open Source telemetry framework that allows you to collect, process and publish measurements. Collectors Processors Publishers
  56. 56. The Snap Telemetry Framework Snap 1.0 is a general availability of a rock solid daemon with a collection of 67 plugins from multiple companies.
  57. 57. The Snap Telemetry Framework Snap 1.0 is the beginning. And we want you involved. snap-telemetry.io
  58. 58. Thank you!
  59. 59. Backup Notes
  60. 60. Everything is Challenging At Scale 60
  61. 61. Add new task 61
  62. 62. Add new task 62
  63. 63. define as a tribe Scaling with Tribe 63
  64. 64. Scaling with Tribe Add new task 64
  65. 65. Snap | Distributed Workflow Physical/Virtual Host Scheduler Processing Publishing Collection 65
  66. 66. Snap | Distributed Workflow 66 Physical/VM Host Physical/VM Host Physical/VM Host Physical/VM Host Physical/VM Host Physical/VM Host Collection Collection Collection Scheduler Processing Publishing
  67. 67. Customizable definition of task and related workflow: Collect Publish Publish Collect Publish ProcessCollect Publish Collect Process Publish Process Publish Snap | Overview – Example Workflows 67
  • mitsuhirotanda

    Dec. 4, 2016

Joel Cooklin and I cover the usage and design of Snap, an open telemetry framework. I also refer to Jason Dixon a few times because I'm a fan of his presentation that influenced my view of monitoring. If you love monitoring and don't mind profanity, check this out first: https://speakerdeck.com/obfuscurity/the-state-of-open-source-monitoring

Views

Total views

1,264

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

33

Shares

0

Comments

0

Likes

1

×