Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Steven Soroka and Anais Dotis-Georgiou
Machine Learning
with Telegraf Execd
Processor Plugin
Introduction and Overview
Telegraf
© 2020 InfluxData. All rights reserved. 3
Introduction to Telegraf
● Active community
● Over 200 plugins
○ 170+ Input
○ 40...
© 2020 InfluxData. All rights reserved. 4
Simple example of a Telegraf Configuration
[agent]
interval = "1s"
flush_interva...
Overview
Execd Processor Plugin
© 2020 InfluxData. All rights reserved. 6
Introduction to the Execd Processor Plugin
● Runs an external program as a separ...
© 2020 InfluxData. All rights reserved. 7
Execd Processor Plugin Diagram
© 2020 InfluxData. All rights reserved. 8
Influx Line Protocol
© 2020 InfluxData. All rights reserved. 9
Simple example of Execd Processor Plugin
# A simple processor. It reads stdin an...
© 2020 InfluxData. All rights reserved. 10
Context: Monitoring temperatures of two beers (haze_v5 and bv_1). The ideal
tem...
© 2020 InfluxData. All rights reserved. 12
Brewing Setup
© 2020 InfluxData. All rights reserved. 13
Seven Segment OCR
ssocr -b black -f white -t 85 make_mono remove_isolated crop ...
© 2020 InfluxData. All rights reserved. 14
Execd Processor Plugin for Forecasting and
Anomaly Detection
Using the Execd Pr...
© 2020 InfluxData. All rights reserved. 15
haze_v5
© 2020 InfluxData. All rights reserved. 16
bv_1
© 2020 InfluxData. All rights reserved. 17
● Double exponential smoothing employs a level component
and a trend component ...
© 2020 InfluxData. All rights reserved. 18
● Statsmodels is a Python package that allows users to explore
data, estimate s...
Code Explained
© 2020 InfluxData. All rights reserved. 20
FAQ about the Execd Processor Plugin
● What are the advantages to running the E...
Resources
• github.com/lukebond/homebrew-webcam-temperature
• github.com/influxdata/tg-brew-anomaly
• github.com/auerswal/...
© 2020 InfluxData. All rights reserved. 22
Thanks!
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

Download to read offline

Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

  1. 1. Steven Soroka and Anais Dotis-Georgiou Machine Learning with Telegraf Execd Processor Plugin
  2. 2. Introduction and Overview Telegraf
  3. 3. © 2020 InfluxData. All rights reserved. 3 Introduction to Telegraf ● Active community ● Over 200 plugins ○ 170+ Input ○ 40+ Output ○ 15+ Processors ○ 5+ Aggregators ○ 5+ Serializers ● Single Binary ● 9.2k Stars ● Extendable in any language
  4. 4. © 2020 InfluxData. All rights reserved. 4 Simple example of a Telegraf Configuration [agent] interval = "1s" flush_interval = "10s" omit_hostname = true [[inputs.cpu]] [[outputs.influxdb_v2]] urls = ["http://127.0.0.1:9999"] token = "$INFLUX_TOKEN" organization = "$INFLUX_ORG" bucket = "brew"
  5. 5. Overview Execd Processor Plugin
  6. 6. © 2020 InfluxData. All rights reserved. 6 Introduction to the Execd Processor Plugin ● Runs an external program as a separate process, passes metrics back and forth. ● Your program must read Influx Line Protocol on STDIN and write metrics in Influx Line Protocol to STDOUT. ● Your program runs continuously in STDIN read loop, so you only pay startup cost once. ● Can be any language, not just Go
  7. 7. © 2020 InfluxData. All rights reserved. 7 Execd Processor Plugin Diagram
  8. 8. © 2020 InfluxData. All rights reserved. 8 Influx Line Protocol
  9. 9. © 2020 InfluxData. All rights reserved. 9 Simple example of Execd Processor Plugin # A simple processor. It reads stdin and writes it to stdout. import sys def main(): for line in sys.stdin: print(line.rstrip()) sys.stdout.flush() if __name__ == '__main__': main() [[processors.execd]] command = ["python", "./processors/forecasting.py"]
  10. 10. © 2020 InfluxData. All rights reserved. 10 Context: Monitoring temperatures of two beers (haze_v5 and bv_1). The ideal temperature for the batch of beer is between 18-21. Objectives: ● Forecast temperatures for the next 10 minutes ● Detect anomalous spikes in temperature that may indicate attention is needed or equipment failure. Monitoring a home brew
  11. 11. © 2020 InfluxData. All rights reserved. 12 Brewing Setup
  12. 12. © 2020 InfluxData. All rights reserved. 13 Seven Segment OCR ssocr -b black -f white -t 85 make_mono remove_isolated crop 30 95 460 220 shear 22 photo.png
  13. 13. © 2020 InfluxData. All rights reserved. 14 Execd Processor Plugin for Forecasting and Anomaly Detection Using the Execd Processor Plugin to achieve our goal and keep the beer at a steady temperature.
  14. 14. © 2020 InfluxData. All rights reserved. 15 haze_v5
  15. 15. © 2020 InfluxData. All rights reserved. 16 bv_1
  16. 16. © 2020 InfluxData. All rights reserved. 17 ● Double exponential smoothing employs a level component and a trend component at each period. Double exponential smoothing uses two weights, (also called smoothing parameters), to update the components at each period. ● In other words, an exponentially weighted average of the trend and values of the data together. What is Exponential Smoothing?
  17. 17. © 2020 InfluxData. All rights reserved. 18 ● Statsmodels is a Python package that allows users to explore data, estimate statistical models, and perform statistical tests. ● Dataset considerations–small and sparse ● Easy to use ● Statsmodels enables more model configuration options than Flux holtWitners() function ● Using the damped version of Holt helps eliminate false positives. ● Algorithm works with small, irregular, and sparse datasets Algorithm Selection: Holt’s Exponential Smoothing
  18. 18. Code Explained
  19. 19. © 2020 InfluxData. All rights reserved. 20 FAQ about the Execd Processor Plugin ● What are the advantages to running the Execd plugin as opposed to a cron job? ● Can I use the Execd Processor Plugin in combination with a Neural Net? ● When shouldn’t I use the Execd Plugin?
  20. 20. Resources • github.com/lukebond/homebrew-webcam-temperature • github.com/influxdata/tg-brew-anomaly • github.com/auerswal/ssocr • Influx Line Protocol
  21. 21. © 2020 InfluxData. All rights reserved. 22 Thanks!

Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

Views

Total views

176

On Slideshare

0

From embeds

0

Number of embeds

1

Actions

Downloads

9

Shares

0

Comments

0

Likes

0

×