InfluxDB and TICK Stack
DevOps @ CrossEngage
Author and Free/Open source geek
who loves the community.
Automation, data, and metrics are my preferred
areas. I have a built-in monitoring chip, and
too lazy to do anything manually :D
Blog | Github | Twitter
Nope, I’m not Mexican, but Egyptian!
So I could actually be anything!
SaaS CDP (Customer Data Platform) that easily combines all customer
data sources and manages cross-channel marketing campaigns
with your existing infrastructure.
● What is time series data?
● Why TICK Stack?
● Where could it be used?
Time Series Data
“A time series is a series of data points indexed (or listed or graphed) in time
order. Most commonly, a time series is a sequence taken at successive equally
spaced points in time. Thus it is a sequence of discrete-time data”.
Properties of Time Series Data
● Billions of individual data points.
● High read and write throughput.
● Large deletes (data expiration).
● Mostly an insert/append workload, very few updates.
TICK = Telegraf (Collect), InfluxDB (Store),
Chrongraf (Visualize), and Kapacitor (Process).
● The Open Source Time Series Stack provides services
and functionality to accumulate, analyze, and act on time series data.
● Has a big community and ecosystem.
● Written entirely in Go. It compiles into a single binary with
no external dependencies.
● TOML configuration,Tom's Obvious, Minimal Language.
The plugin-driven agent for collecting & reporting metrics.
● Minimal memory footprint.
● +100 plugs with a wide number of plugins for many popular services
already exist for well known services and APIs.
● Plugin system allows new inputs and outputs to be easily added.
● Can work with any external scripts.
Scalable time series datastore for metrics, events, and real-time analytics.
● High performance datastore written specifically for time series data.
● Simple, high performing write and query HTTP(S) APIs.
● Plugins support for other data sources such as Graphite, and collectd.
● SQL-like query language to easily query aggregated data.
● Tags allow series to be indexed for fast and efficient queries.
● Retention policies efficiently auto-expire stale data.
● Continuous queries automatically compute aggregate data to make
frequent queries more efficient.
● Query example:
> SELECT "host", "env", "load1" as "load" FROM "cpu" WHERE "host" = 'tux’ LIMIT 1
time host env load
2017-11-01T01:11:00.000000000Z tux prod 1.25
Framework for processing, monitoring, and alerting on time series data.
● Process both streaming data and batch data.
● Query data from InfluxDB on a schedule, and receive data via the line
protocol and any other method InfluxDB supports.
● Perform any transformation currently possible in InfluxQL.
● Store transformed data back in InfluxDB.
● Support custom user defined functions to detect anomalies.
● Has an easy DSL to define data processing pipelines.
● Integrate with HipChat, OpsGenie, Alerta, Sensu, Slack, and more.
● TICKscript DSL example:
|eval(lambda: "errors" / "total")
// Write the transformed data to InfluxDB.
● Infrastructure monitoring.
● Work with sensors (i.e. interacting with IoT).
● Anomaly detection.