Successfully reported this slideshow.
Your SlideShare is downloading. ×

Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 21 Ad

Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

Download to read offline

Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020 (20)

Advertisement

More from InfluxData (20)

Recently uploaded (20)

Advertisement

Anais Dotis-Georgiou & Steven Soroka [InfluxData] | Machine Learning with Telegraf Execd Processor Plugin | InfluxDays Virtual Experience NA 2020

  1. 1. Steven Soroka and Anais Dotis-Georgiou Machine Learning with Telegraf Execd Processor Plugin
  2. 2. Introduction and Overview Telegraf
  3. 3. © 2020 InfluxData. All rights reserved. 3 Introduction to Telegraf ● Active community ● Over 200 plugins ○ 170+ Input ○ 40+ Output ○ 15+ Processors ○ 5+ Aggregators ○ 5+ Serializers ● Single Binary ● 9.2k Stars ● Extendable in any language
  4. 4. © 2020 InfluxData. All rights reserved. 4 Simple example of a Telegraf Configuration [agent] interval = "1s" flush_interval = "10s" omit_hostname = true [[inputs.cpu]] [[outputs.influxdb_v2]] urls = ["http://127.0.0.1:9999"] token = "$INFLUX_TOKEN" organization = "$INFLUX_ORG" bucket = "brew"
  5. 5. Overview Execd Processor Plugin
  6. 6. © 2020 InfluxData. All rights reserved. 6 Introduction to the Execd Processor Plugin ● Runs an external program as a separate process, passes metrics back and forth. ● Your program must read Influx Line Protocol on STDIN and write metrics in Influx Line Protocol to STDOUT. ● Your program runs continuously in STDIN read loop, so you only pay startup cost once. ● Can be any language, not just Go
  7. 7. © 2020 InfluxData. All rights reserved. 7 Execd Processor Plugin Diagram
  8. 8. © 2020 InfluxData. All rights reserved. 8 Influx Line Protocol
  9. 9. © 2020 InfluxData. All rights reserved. 9 Simple example of Execd Processor Plugin # A simple processor. It reads stdin and writes it to stdout. import sys def main(): for line in sys.stdin: print(line.rstrip()) sys.stdout.flush() if __name__ == '__main__': main() [[processors.execd]] command = ["python", "./processors/forecasting.py"]
  10. 10. © 2020 InfluxData. All rights reserved. 10 Context: Monitoring temperatures of two beers (haze_v5 and bv_1). The ideal temperature for the batch of beer is between 18-21. Objectives: ● Forecast temperatures for the next 10 minutes ● Detect anomalous spikes in temperature that may indicate attention is needed or equipment failure. Monitoring a home brew
  11. 11. © 2020 InfluxData. All rights reserved. 12 Brewing Setup
  12. 12. © 2020 InfluxData. All rights reserved. 13 Seven Segment OCR ssocr -b black -f white -t 85 make_mono remove_isolated crop 30 95 460 220 shear 22 photo.png
  13. 13. © 2020 InfluxData. All rights reserved. 14 Execd Processor Plugin for Forecasting and Anomaly Detection Using the Execd Processor Plugin to achieve our goal and keep the beer at a steady temperature.
  14. 14. © 2020 InfluxData. All rights reserved. 15 haze_v5
  15. 15. © 2020 InfluxData. All rights reserved. 16 bv_1
  16. 16. © 2020 InfluxData. All rights reserved. 17 ● Double exponential smoothing employs a level component and a trend component at each period. Double exponential smoothing uses two weights, (also called smoothing parameters), to update the components at each period. ● In other words, an exponentially weighted average of the trend and values of the data together. What is Exponential Smoothing?
  17. 17. © 2020 InfluxData. All rights reserved. 18 ● Statsmodels is a Python package that allows users to explore data, estimate statistical models, and perform statistical tests. ● Dataset considerations–small and sparse ● Easy to use ● Statsmodels enables more model configuration options than Flux holtWitners() function ● Using the damped version of Holt helps eliminate false positives. ● Algorithm works with small, irregular, and sparse datasets Algorithm Selection: Holt’s Exponential Smoothing
  18. 18. Code Explained
  19. 19. © 2020 InfluxData. All rights reserved. 20 FAQ about the Execd Processor Plugin ● What are the advantages to running the Execd plugin as opposed to a cron job? ● Can I use the Execd Processor Plugin in combination with a Neural Net? ● When shouldn’t I use the Execd Plugin?
  20. 20. Resources • github.com/lukebond/homebrew-webcam-temperature • github.com/influxdata/tg-brew-anomaly • github.com/auerswal/ssocr • Influx Line Protocol
  21. 21. © 2020 InfluxData. All rights reserved. 22 Thanks!

×