Flux and InfluxDB 2.0

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Flux and InﬂuxDB 2.0
Paul Dix

@pauldix

paul@inﬂuxdata.com

• Data-scripting language

• Functional

• MIT Licensed

• Language & Runtime/Engine

Language + Query Engine

2.0

Biggest Change Since 0.9

Clean Migration Path

Compatibility Layer

• MIT Licensed

• Multi-tenanted

• Telegraf, InﬂuxDB, Chronograf, Kapacitor rolled into 1

• OSS single server

• Cloud u...

• MIT Licensed

• Multi-tenanted

• Telegraf, InﬂuxDB, Chronograf, Kapacitor rolled into 1

• OSS single server

• Cloud u...

TICK is dead

Long Live InﬂuxDB 2.0
(and Telegraf)

Consistent Documented API
Collection, Write/Query, Streaming & Batch Processing, Dashboards

Ofﬁcially Supported Client
Libraries
Go, Node.js, Ruby, Python, PHP, Java, C#, C, Kotlin

Visualization Libraries

Ways to run Flux - (interpreter,
InﬂuxDB 1.7 & 2.0)

Flux Language Elements

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:-1h)
/...

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:-1h)
/...

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:-1h)
/...

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:-1h)
/...

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:-1h)
/...

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:-1h)
/...

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:2018-1...

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:-1h)
/...

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:-1h)
/...

// get all data from the telegraf db
from(bucket:”telegraf/autogen”)
// filter that by the last hour
|> range(start:-1h)
/...

// variables
some_int = 23

// variables
some_int = 23
some_float = 23.2

// variables
some_int = 23
some_float = 23.2
some_string = “cpu"

// variables
some_int = 23
some_float = 23.2
some_string = “cpu"
some_duration = 1h

// variables
some_int = 23
some_float = 23.2
some_string = “cpu"
some_duration = 1h
some_time = 2018-10-10T19:00:00

// variables
some_int = 23
some_float = 23.2
some_string = “cpu"
some_duration = 1h
some_time = 2018-10-10T19:00:00
some_a...

// variables
some_int = 23
some_float = 23.2
some_string = “cpu"
some_duration = 1h
some_time = 2018-10-10T19:00:00
some_a...

Data Model & Working with
Tables

Example Series
_measurement=mem,host=A,region=west,_ﬁeld=free
_measurement=mem,host=B,region=west,_ﬁeld=free
_measurement=...

Example Series
_measurement=mem,host=A,region=west,_ﬁeld=free
_measurement=mem,host=B,region=west,_ﬁeld=free
_measurement=...

Example Series
_measurement=mem,host=A,region=west,_ﬁeld=free
_measurement=mem,host=B,region=west,_ﬁeld=free
_measurement=...

Table
_measurement host region _ﬁeld _time _value
mem A west free 2018-06-14T09:15:00 10
mem A west free 2018-06-14T09:14:...

_measurement host region _ﬁeld _time _value
mem A west free 2018-06-14T09:15:00 10
mem A west free 2018-06-14T09:14:50 10
...

_measurement host region _ﬁeld _time _value
mem A west free 2018-06-14T09:15:00 10
mem A west free 2018-06-14T09:14:50 10
...

_measurement host region _ﬁeld _time _value
mem A west free 2018-06-14T09:15:00 10
mem A west free 2018-06-14T09:14:50 10
...

_measurement host region _ﬁeld _time _value
mem A west free 2018-06-14T09:15:00 10
mem A west free 2018-06-14T09:14:50 10
...

Table Per Series
_measurement host region _ﬁeld _time _value
mem A west free 2018-06-14T09:15:00 10
mem A west free 2018-0...

input tables -> function -> output tables

input tables -> function -> output tables
// example query
from(db:"telegraf")
|> range(start:2018-06-14T09:14:50, start:2...

input tables -> function -> output tables
What to sum on?
// example query
from(db:"telegraf")
|> range(start:2018-06-14T0...

input tables -> function -> output tables
Default columns argument
// example query
from(db:"telegraf")
|> range(start:201...

input tables -> function -> output tables
_meas
ureme
host region _ﬁeld _time _valu
e
mem A west free 2018-06-
14T09:1
10
...

input tables -> function -> output tables
_meas
ureme
host region _ﬁeld _time _valu
e
mem A west free 2018-06-
14T09:1
10
...

input tables -> function -> output tables
// example query
from(db:"telegraf")
|> range(start:2018-06-14T09:14:50, start:2...

N to N table mapping
(1 to 1 mapping)

N to M table mapping

window
// example query
from(db:"telegraf")
|> range(start:2018-06-14T09:14:30, end:2018-06-14T09:15:01)
|> filter(fn: r =...

window
// example query
from(db:"telegraf")
|> range(start:2018-06-14T09:14:30, end:2018-06-14T09:15:01)
|> filter(fn: r =...

window
_meas host region _ﬁeld _time _valu
mem A west free …14:30 10
mem A west free …14:40 11
mem A west free …14:50 12
m...

window
_meas host region _ﬁeld _time _valu
mem A west free …14:30 10
mem A west free …14:40 11
mem A west free …14:50 12
m...

window
_meas host region _ﬁeld _time _valu
mem A west free …14:30 10
mem A west free …14:40 11
mem A west free …14:50 12
m...

window
_meas host region _ﬁeld _time _valu
mem A west free …14:30 10
mem A west free …14:40 11
mem A west free …14:50 12
m...

Window based on time
_start and _stop columns

group
// example query
from(db:"telegraf")
|> range(start:2018-06-14T09:14:30, end:2018-06-14T09:15:01)
|> filter(fn: r =>...

group
// example query
from(db:"telegraf")
|> range(start:2018-06-14T09:14:30, end:2018-06-14T09:15:01)
|> filter(fn: r =>...

group
_meas host region _ﬁeld _time _valu
mem A west free …14:30 10
mem A west free …14:40 11
mem A west free …14:50 12
me...

group
_meas host region _ﬁeld _time _valu
mem A west free …14:30 10
mem A west free …14:40 11
mem A west free …14:50 12
me...

Group based on columns

Flux Design Principles

Useable

Make Everyone a Data
Programmer!

Readable

Flexible

Composable

Testable

Contributable

Shareable

Functions Overview

Inputs
from, fromKafka, fromFile, fromS3, fromPrometheus, fromMySQL, etc.

Flux != InﬂuxDB

Follow Telegraf Model

import "mysql"
customers = mysql.from(connect: loadSecret(name:”mysql_prod"),
query: "select id, name from customers")
dat...

import "mysql"
customers = mysql.from(connect: loadSecret(name:"mysql_prod"),
query: "select id, name from customers")
dat...

import "mysql"
customers = mysql.from(connect: loadSecret(name:"mysql_prod"),
query: "select id, name from customers")
dat...

import "mysql"
customers = mysql.from(connect: loadSecret(name:"mysql_prod"),
query: "select id, name from customers")
dat...

import "mysql"
customers = mysql.from(connect: loadSecret(name:"mysql_prod"),
query: "select id, name from customers")
dat...

import "mysql"
customers = mysql.from(connect: loadSecret(name:"mysql_prod"),
query: "select id, name from customers")
dat...

import "mysql"
customers = mysql.from(connect: loadSecret(name:"mysql_prod"),
query: "select id, name from customers")
dat...

Outputs
to, toKafka, toFile, toS3, toPrometheus, toMySQL, etc.

Outputs are for Tasks

option task = {
name: “Alert on disk",
every: 5m,
}
crit = 90 // alert at this percentage
warn = 80 // warn at this percen...

option task = {
name: “Alert on disk",
every: 5m,
}
crit = 90 // alert at this percentage
warn = 80 // warn at this percen...

option task = {
name: “Alert on disk",
every: 5m,
}
crit = 90 // alert at this percentage
warn = 80 // warn at this percen...

option task = {
name: “Alert on disk",
every: 5m,
}
crit = 90 // alert at this percentage
warn = 80 // warn at this percen...

option task = {
name: “Alert on disk",
every: 5m,
}
crit = 90 // alert at this percentage
warn = 80 // warn at this percen...

Separate Alerts From
Notiﬁcations!

option task = {name: "slack critical alerts", every: 1m}
import "slack"
lastNotificationTime = from(bucket: "notificatons"...

option task = {name: "slack critical alerts", every: 1m}
import "slack"
lastNotificationTime = from(bucket: "notificatons"...

option task = {name: "slack critical alerts", every: 1m}
import "slack"
lastNotificationTime = from(bucket: "notificatons"...

option task = {name: "slack critical alerts", every: 1m}
import "slack"
lastNotificationTime = from(bucket: "notificatons"...

option task = {name: "slack critical alerts", every: 1m}
import "slack"
lastNotificationTime = from(bucket: "notificatons"...

option task = {name: "slack critical alerts", every: 1m}
import "slack"
lastNotificationTime = from(bucket: "notificatons"...

option task = {name: "slack critical alerts", every: 1m}
import "slack"
lastNotificationTime = from(bucket: "notificatons"...

option task = {
name: "email alert digest",
cron: "0 5 * * 0"
}
import "smtp"
body = ""
from(bucket: "alerts")
|> range(st...

option task = {
name: "email alert digest",
cron: "0 5 * * 0"
}
import "smtp"
body = ""
from(bucket: "alerts")
|> range(st...

option task = {
name: "email alert digest",
cron: "0 5 * * 0"
}
import "smtp"
body = ""
from(bucket: "alerts")
|> range(st...

Tasks run logs
(just another time series)

UI will hide complexity

Built on top of primitives

API for Deﬁning Dashboards

Bulk Import & Export
Specify bucket, range, predicate

Same API in OSS, Cloud, and
Enterprise

CLI & UI

2.0

Thank you.
Paul Dix

@pauldix

paul@inﬂuxdata.com

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1011447 views
Storytelling For The Web: Integrate... by Chiara Aliotta 919244 views
Artificial Intelligence, Data and C... by OECD Directorate ... 820562 views
How to Leverage AI to Boost Employe... by SocialHRCamp 342424 views
2024 State of Marketing Report – by... by Marius Sescu 195380 views
Everything You Need To Know About C... by Expeed Software 218594 views

View on Slideshare

1 of 125 Ad

View on Slideshare

1 of 125 Ad