Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to InfluxDB and TICK Stack


Published on

A quick walk through InfluxDB and TICK Stack.
Telegraf (Collect), InfluxDB (Store), Chrongraf (Visualize), and Kapacitor (Process).

- What is time series data?
- Why TICK Stack?
- Where could TICK Stack be used?

Published in: Data & Analytics
  • Be the first to comment

Introduction to InfluxDB and TICK Stack

  1. 1. Introduction to InfluxDB and TICK Stack Ahmed AbouZaid @aabouzaid
  2. 2. Ahmed AbouZaid DevOps @ CrossEngage Author and Free/Open source geek who loves the community. Automation, data, and metrics are my preferred areas. I have a built-in monitoring chip, and too lazy to do anything manually :D Blog | Github | Twitter Nope, I’m not Mexican, but Egyptian! So I could actually be anything!
  3. 3. SaaS CDP (Customer Data Platform) that easily combines all customer data sources and manages cross-channel marketing campaigns with your existing infrastructure.
  4. 4. Overview ● What is time series data? ● Why TICK Stack? ● Where could it be used?
  5. 5. Time Series Data “A time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data”. Properties of Time Series Data ● Billions of individual data points. ● High read and write throughput. ● Large deletes (data expiration). ● Mostly an insert/append workload, very few updates.
  6. 6. Time Series Data
  7. 7. TICK Stack
  8. 8. TICK Stack TICK = Telegraf (Collect), InfluxDB (Store), Chrongraf (Visualize), and Kapacitor (Process). ● The Open Source Time Series Stack provides services and functionality to accumulate, analyze, and act on time series data. ● Has a big community and ecosystem. ● Written entirely in Go. It compiles into a single binary with no external dependencies. ● TOML configuration,Tom's Obvious, Minimal Language.
  9. 9. Telegraf
  10. 10. Telegraf The plugin-driven agent for collecting & reporting metrics. ● Minimal memory footprint. ● +100 plugs with a wide number of plugins for many popular services already exist for well known services and APIs. ● Plugin system allows new inputs and outputs to be easily added. ● Can work with any external scripts.
  11. 11. Telegraf ● Understand Telegraf output: $ telegraf --config telegraf.conf --input-filter cpu --test [measurement],[tag1,tag2] [field1,field2,field3] [timestamp] system,host=tux,env=prod load1=1.25,load5=1.27,load15=1.29 1509997632000000000
  12. 12. InfluxDB
  13. 13. InfluxDB Scalable time series datastore for metrics, events, and real-time analytics. ● High performance datastore written specifically for time series data. ● Simple, high performing write and query HTTP(S) APIs. ● Plugins support for other data sources such as Graphite, and collectd. ● SQL-like query language to easily query aggregated data. ● Tags allow series to be indexed for fast and efficient queries. ● Retention policies efficiently auto-expire stale data. ● Continuous queries automatically compute aggregate data to make frequent queries more efficient.
  14. 14. InfluxDB ● Query example: > SELECT "host", "env", "load1" as "load" FROM "cpu" WHERE "host" = 'tux’ LIMIT 1 name: cpu --------- time host env load 2017-11-01T01:11:00.000000000Z tux prod 1.25
  15. 15. Chronograf
  16. 16. Chronograf TICK stack web interface for monitoring, visualization, and management. ● Data Visualization. ● Database Management. ● Infrastructure overview. ● Alert Management.
  17. 17. Chronograf
  18. 18. Chronograf
  19. 19. Kapacitor
  20. 20. Kapacitor Framework for processing, monitoring, and alerting on time series data. ● Process both streaming data and batch data. ● Query data from InfluxDB on a schedule, and receive data via the line protocol and any other method InfluxDB supports. ● Perform any transformation currently possible in InfluxQL. ● Store transformed data back in InfluxDB. ● Support custom user defined functions to detect anomalies. ● Has an easy DSL to define data processing pipelines. ● Integrate with HipChat, OpsGenie, Alerta, Sensu, Slack, and more.
  21. 21. Kapacitor ● TICKscript DSL example: stream |from() .measurement('app') |eval(lambda: "errors" / "total") .as('error_percent') // Write the transformed data to InfluxDB. |influxDBOut() .database('app') .retentionPolicy('15D') .measurement('errors') .tag('kapacitor', 'true') .tag('version', '0.2')
  22. 22. Use cases ● Infrastructure monitoring. ● Work with sensors (i.e. interacting with IoT). ● Anomaly detection.
  23. 23. Resources ● ● ● perties-of-time-series-data ●
  24. 24. Questions? Thanks :-)