Building a Telegraf Plugin by Noah Crowly | Developer Advocate | InfluxData

Noah Crowley / Developer Advocate
How to Build a Telegraf
Plugin

© 2018 InfluxData. All rights reserved.
What we’ll be covering
✔ Telegraf overview

✔ Examples of Telegraf plugins

✔ Telegraf plugin architecture

✔ How to a write a Telegraf plugin

Telegraf Overview
• Telegraf is an open source, plugin-driven agent for collecting
and reporting metrics

• “Push” or “Pull”

• Large collection of plugins: 192 (as of 3/13/2019)

• Inputs: 149

• Outputs: 30

• Aggregators: 9

• Processors: 4

Telegraf Overview
• Can act as:

• Agent

• Collector

• Part of an Ingest Pipeline

Telegraf Plugins
• Outputs

• Allows you to send metrics to a variety of datastores

• Plugins for both InfluxDB 1.x and 2.0

• Supports Kafka, MQTT, NSQ, OpenMetrics, and more

• Aggregators and Processors allow you to manipulate data as it
flows through Telegraf

• Transform tags, convert types, calculate histograms

Telegraf Input Plugins
• Metrics ingestion from host system: CPU, I/O, Network…

• Common Applications like NGINX, Postgres, Redis…

• Third Party APIs: Mailchimp, Cloudwatch, Google Analytics…

• General-Purpose Protocols: HTTP, Socket, MQTT…

• …and more!

Telegraf Benefits
• Minimal memory footprint

• Tagging of metrics

• Batching of metrics to reduce the number of atomic writes

• Easy contribution of functionality thanks to Go

• Static Binary

• Strong Community Contributions

statsd
• Listens for incoming data
• Acts as a statsd instance
• Outputs to any configured output

postgresql
• Collects performance data
• Uses data from built-in views:
• pg_stat_database
• pg_stat_bgwriter

apache
• Connects over HTTP
• Reads data from:
• /server-status?auto

win_perf_counters
• Collects performance data from
Windows machines

Input Plugin Considerations
• Where does the data exist?

• How can it be collected?

• What shape is the data?

prometheus_client
• Output plugin
• Exposes metrics in Prometheus
(now OpenMetrics) format.

mqtt
• Output half of the MQTT plugins
• Allows you to write messages to
an MQTT broker.

Output Plugin Considerations
• How should the data be shaped?

• How is the data output?

• Where is the data output to?

Plugin Architecture
• Built to be extensible from the beginning

• Make use of interfaces in Go

• Each plugin type has an interface

package simple
// simple.go
import (
"github.com/influxdata/telegraf"
"github.com/influxdata/telegraf/plugins/inputs"
)
type Simple struct {
Ok bool
}
func (s *Simple) Description() string {
return "a demo plugin"
}
func (s *Simple) SampleConfig() string {
return `
## Indicate if everything is fine
ok = true
`
}
func (s *Simple) Gather(acc telegraf.Accumulator) error {
if s.Ok {
acc.AddFields("state", map[string]interface{}{"value":
"pretty good"}, nil)
} else {
acc.AddFields("state", map[string]interface{}{"value":
"not great"}, nil)
}
return nil
}
func init() {
inputs.Add("simple", func() telegraf.Input { return
&Simple{} })
}

Getting started…
• Recommend using Go 1.11

• Understanding of Git & Pull Requests

• What is our plugin going to do?

• Generate data using “sine” and “cosine” trigonometric
functions

• Read the CONTRIBUTING.md file to ensure you can build & test
Telegraf before continuing

Getting started…
• Set up your workspace, get the code, and create a branch:

• Create the required directories and files:

• Add boilerplate…
$ go get github.com/influxdata/telegraf 
$ cd $GOPATH/github.com/influxdata/telegraf 
$ git checkout -b trig-demo
$ cd plugins/inputs
$ mkdir trig
$ touch trig/trig.go

package trig 
 
import ( 
    "github.com/influxdata/telegraf" 
    "github.com/influxdata/telegraf/plugins/inputs" 
) 
 
type Trig struct { 
} 
 
func (s *Trig) SampleConfig() string { 
    return "" 
} 
 
func (s *Trig) Description() string { 
    return "" 
} 
 
func (s *Trig) Gather(acc telegraf.Accumulator) error { 
    return nil 
} 
 
func init() { 
    inputs.Add("trig", func() telegraf.Input { return &Trig{} }) 
}
trig.go

Import your plugin
• Add the following to telegraf/plugins/inputs/all/all.go

• This will your plugin into the main Telegraf package and ensure
that it can run.
_ "github.com/influxdata/telegraf/plugins/inputs/trig"

Add configuration variables
• Each plugin has a struct

• There must be a property on the struct to hold any
configuration variables

• Any other variables should be unexported

• We’ll add two variables:

• “x” will hold the state of the plugin between collection intervals

• “Amplitude” will hold the amplitude value for the sin wave

package trig 
 
import ( 
) 
 
type Trig struct {
    x         float64
    Amplitude float64 
} 
 
    return "" 
} 
 
    return "" 
} 
 
trig.go

Add sample configuration
• Telegraf can dynamically construct configuration files

• Aggregates sample configs for all plugins

• The CLI allows you to provide arguments to filter which plugins
are included

• In 2.0, InfluxDB will use this info to generate Telegraf configs

package trig 
 
import ( 
) 
 
type Trig struct {
    x         float64
} 
 
var TrigConfig = ` 
## Set the amplitude 
amplitude = 10.0 
` 
 
    return TrigConfig 
}
 
trig.go

Add a simple description
• This description is included above the plugin configuration and
should briefly describe what your plugin does

type Trig struct {
    x         float64
} 
 
var TrigConfig = ` 
## Set the amplitude 
amplitude = 10.0 
` 
 
    return TrigConfig 
}
 
    return "Inserts sine and cosine waves for demonstration
purposes" 
} 
 
trig.go

Test your config!
• First, build Telegraf

• Generate a config:
$ make telegraf
$ telegraf -sample-config -input-filter trig
-output-filter influxdb -debug

... 
###############################################################################
# INPUT PLUGINS #
###############################################################################
# Inserts sine and cosine waves for demonstration purposes
[[inputs.trig]]
## Set the amplitude
amplitude = 10.0

The “Gather” function
• The heart of the plugin, it is run every time telegraf collects
metrics

• This is where you’ll do the majority of the work

• We’ll generate sine and cosine values

• At the end of the function, we add any data we’ve collected
and to the Telegraf “Accumulator” by
calling .AddFields(measurement, tags, fields).

• This defines a new point in InfluxDB with the timestamp being
the collection time.

    return "Inserts sine and cosine waves for demonstration
purposes" 
} 
 
    sinner := math.Sin((s.x*math.Pi)/5.0) * s.Amplitude 
    cosinner := math.Cos((s.x*math.Pi)/5.0) * s.Amplitude 
 
    fields := make(map[string]interface{}) 
    fields["sine"] = sinner 
    fields["cosine"] = cosinner 
 
    tags := make(map[string]string) 
 
    s.x += 1.0 
    acc.AddFields("trig", fields, tags) 
 
    return nil 
}
trig.go

Add initial state
• In our boilerplate, we added an init() function

• Called when the plugin is first initialized

• Makes the plugin available to the Telegraf agent

• We can define starting state as part of this function

• We need to initialize our “x” variable

    cosinner := math.Cos((s.x*math.Pi)/5.0) * s.Amplitude 
 
    fields["sine"] = sinner 
    fields["cosine"] = cosinner 
 
    tags := make(map[string]string) 
 
    s.x += 1.0 
    acc.AddFields("trig", fields, tags) 
 
    return nil 
}
 
 
func init() {
    inputs.Add("trig", func() telegraf.Input { return &Trig{x: 0.0} })
}
trig.go

Compile and test
• Once again, build Telegraf:

• Generate a config and write it to a file:
$ make telegraf
$ ./telegraf -sample-config -input-filter
trig -output-filter influxdb -debug >>
telegraf.conf.test
• Run Telegraf with the new config:
$ ./telegraf -config telegraf.conf.test
-debug

$ ./telegraf -config telegraf.conf.test -debug
2019-03-14T06:32:12Z I! Starting Telegraf
2019-03-14T06:32:12Z I! Loaded inputs: trig
2019-03-14T06:32:12Z I! Loaded aggregators:
2019-03-14T06:32:12Z I! Loaded processors:
2019-03-14T06:32:12Z I! Loaded outputs: influxdb
2019-03-14T06:32:12Z I! Tags enabled: host=noah-mbp.local
2019-03-14T06:32:12Z I! [agent] Config: Interval:10s, Quiet:false,
Hostname:"noah-mbp.local", Flush Interval:10s
2019-03-14T06:32:12Z D! [agent] Connecting outputs
2019-03-14T06:32:12Z D! [agent] Attempting connection to output: influxdb
2019-03-14T06:32:12Z D! [agent] Successfully connected to output: influxdb
2019-03-14T06:32:12Z D! [agent] Starting service inputs
2019-03-14T06:32:30Z D! [outputs.influxdb] wrote batch of 1 metrics in
23.857525ms
2019-03-14T06:32:30Z D! [outputs.influxdb] buffer fullness: 1 / 10000 metrics.

Final steps before submitting
• Add a few things to your plugin…

• a README.md

• a LICENSE

• A sample of the input/output format

• Tests!

package trig 
 
import ( 
    "math" 
    "testing" 
    "github.com/influxdata/telegraf/testutil" 
) 
 
func TestTrig(t *testing.T) { 
 
    s := &Trig{ 
 
        Amplitude: 10.0, 
    } 
 
    for i := 0.0; i < 10.0; i++ { 
        var acc testutil.Accumulator 
        sine := math.Sin((i*math.Pi)/5.0) * s.Amplitude 
        cosine := math.Cos((i*math.Pi)/5.0) * s.Amplitude 
        s.Gather(&acc) 
        fields["sine"] = sine 
        fields["cosine"] = cosine 
 
        acc.AssertContainsFields(t, "trig", fields) 
 
    } 
 
}
trig_test.go

Final steps before submitting
• Add a few things to your plugin…

• a README.md

• a LICENSE

• A sample of the input/output format

• Tests!

• …and submit!

• github.com/influxdata/telegraf/pulls

Building a Telegraf Plugin by Noah Crowly | Developer Advocate | InfluxData

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Building a Telegraf Plugin by Noah Crowly | Developer Advocate | InfluxData

Similar to Building a Telegraf Plugin by Noah Crowly | Developer Advocate | InfluxData (20)

More from InfluxData

More from InfluxData (20)

Recently uploaded

Recently uploaded (20)

Building a Telegraf Plugin by Noah Crowly | Developer Advocate | InfluxData