SlideShare a Scribd company logo
© 2017 InfluxData. All rights reserved.1
Agenda: New Practitioners Track
WORKSHOPAGENDA
8:30 AM – 9:00 AM Coffee & Pastries
9:00 AM – 9:15 AM Welcome
9:15 AM – 10:15 AM Introduction to the TICK Stack Katy Farmer
10:15 AM – 10:30 AM Break
10:30 AM – 11:30 AM Optimizing the TICK Stack Sam Dillard
11:30 AM – 11:45 AM Break
11:45 AM – 12:45 PM Chronograf and Dashboarding Russ Savage
12:45 PM – 1:45 PM Lunch
1:45 PM – 2:45 PM InfluxEnterprise Architectural Patterns Craig Hobbs
2:45 PM – 3:00 PM Break
3:00 PM – 4:00 PM InfluxQL & TICKscript Michael DeSa
4:00 PM – 4:15PM Closing
Michael Desa
Software Engineer,
InfluxData
InfluxQL & TICKscript
Michael Desa is a Software Engineer at InfluxData who
works on the 2.0 API for the InfluxData Platform. He has
led the InfluxDB training course across the US, providing
students with an in depth understanding of how InfluxDB
works as well as sharing best practices. He has a degree
in Math from the University of California, at Berkeley.
© 2017 InfluxData. All rights reserved.3
© 2017 InfluxData. All rights reserved.3
✓ What is InfluxQL
• What types of questions can I ask
about my data with InfluxQL
✓ What is TICKscript
• What types of questions can I solve
with TICKscript
✓ What is Flux
• Why does it matter
Agenda
© 2017 InfluxData. All rights reserved.4
© 2018 InfluxData. All rights reserved.4
InfluxQL
© 2017 InfluxData. All rights reserved.5
What is InfluxQL?
● A SQL-like query language for InfluxDB
○ Extends SQL to express queries that range over time
○ Limited support for relational features of SQL
● Easy to use
○ Should feel familiar to anyone with SQL experience
■ Not SQL compliant (which can bother those that want full access to SQL)
● Data definition and meta queries
○ Used to define and view database structures
Basic Select Statement
SELECT <field> FROM <measurement>
SELECT * FROM cpu
SELECT free FROM mem
SELECT x + y FROM vars
SELECT x,y FROM nums
Basic Select Statement
> SELECT * FROM h2o_quality LIMIT 10
name: h2o_quality
-----------------
time index location id
2015-08-18T00:00:00Z 41 coyote_creek 1
2015-08-18T00:00:00Z 99 santa_monica 2
2015-08-18T00:00:00Z 41 coyote_creek 1
2015-08-18T00:06:00Z 56 santa_monica 2
2015-08-18T00:06:00Z 11 coyote_creek 3
2015-08-18T00:06:00Z 11 coyote_creek 3
2015-08-18T00:12:00Z 65 santa_monica 3
2015-08-18T00:12:00Z 38 coyote_creek 1
2015-08-18T00:12:00Z 38 coyote_creek 1
2015-08-18T00:18:00Z 57 santa_monica 3
Select Statement with
WHERE clause
SELECT <field> FROM <measurement> WHERE <conditions>
SELECT * FROM cpu WHERE busy > 50
SELECT free FROM mem WHERE host = 'server1'
SELECT x + y FROM vars WHERE some_tag = 'some_key'
SELECT x,y FROM nums WHERE domain =~ /.*/
Select Statement with
Relative Time
SELECT <field> FROM <measurement> WHERE <time>
SELECT * FROM cpu WHERE time > now() - 1h
SELECT * FROM cpu WHERE time > now() - 10s
SELECT free FROM mem WHERE time > now() - 4d
SELECT x + y FROM vars WHERE time > now() - 10w
SELECT x,y FROM nums WHERE time > now() + 15m
Select Statement with
GROUP BY clause
[SELECT STATEMENT] GROUP BY <tag>
SELECT * FROM cpu GROUP BY host
SELECT * FROM cpu GROUP BY *
SELECT free FROM mem GROUP BY location, host
Select Statement with
GROUP BY clause
> SELECT * FROM h2o_quality GROUP BY location
name: h2o_quality
tags: location = coyote_creek
time index id
---- ----- ---
2015-08-18T00:00:00Z 41 1
2015-08-18T00:00:00Z 41 1
name: h2o_quality
tags: location = santa_monica
time index id
---- ----- ---
2015-08-18T00:00:00Z 99 2
2015-08-18T00:06:00Z 56 2
Select Statement with a
function
SELECT <function>(<field>) FROM <measurement>
SELECT count(value) FROM cpu
SELECT mean(free) FROM mem WHERE time > now() - 1h
SELECT sum(x) FROM vars WHERE x > 100
SELECT median(y) FROM nums WHERE domain = 'Z'
Select Statement with a
function
> SELECT count(index) FROM h2o_quality
WHERE location = 'coyote_creek'
name: h2o_quality
-----------------
time count
1970-01-01T00:00:00Z 12777
Select Statement with a
function
> SELECT max(usage_user) FROM cpu
WHERE time > now() - 10d
name: cpu
time max
---- ---
2018-11-05T22:12:05Z 54
Types of
Functions
● Aggregators
○ count
○ distinct
○ integral
○ mean
○ median
○ spread
○ sum
○ stddev
● Selectors
○ bottom
○ first
○ last
○ max
○ min
○ percentile
○ top
● Transformers
○ derivative
○ difference
○ moving_average
○ elapsed
Select Statement with
GROUP BY time clause
[SELECT STATEMENT] WHERE <time condition>
GROUP BY time(<period>)
SELECT max(busy) FROM cpu WHERE time > now() - 1h
GROUP BY time(10m)
SELECT mean(free) FROM free WHERE time > now() - 1d
GROUP BY time(1h)
Select Statement with
GROUP BY time clause
Invalid queries
SELECT busy FROM cpu WHERE time > now() - 1h
GROUP BY time(10m)
SELECT mean(busy) FROM cpu GROUP BY time(10m)
Select Statement with
GROUP BY time clause
> SELECT mean(degrees) FROM average_temperature
WHERE time < '2015-09-19'
AND time > '2015-09-18'
GROUP BY time(12h)
name: average_temperature
-------------------------
time mean
2015-09-18T00:00:00Z 79.83613445378151
2015-09-18T12:00:00Z 79.65034965034965
Select Statement with
GROUP BY time and
tag
> SELECT mean(degrees) FROM average_temperature
WHERE time < '2015-09-19'
AND time > '2015-09-18'
GROUP BY time(12h), location
name: average_temperature
tags: location = coyote_creek
-------------------------
time mean
2015-09-18T00:00:00Z 79.83613445378151
2015-09-18T12:00:00Z 79.65034965034965
name: average_temperature
tags: location = santa_monica
-------------------------
time mean
2015-09-18T00:00:00Z
2015-09-18T12:00:00Z 79.95033445378151
Select with fill
> SELECT mean(degrees) FROM average_temperature
WHERE time < '2015-09-19'
AND time > '2015-09-18'
GROUP BY time(12h), location fill(<fill>)
name: average_temperature
tags: location = coyote_creek
-------------------------
time mean
2015-09-18T00:00:00Z 79.83613445378151
2015-09-18T12:00:00Z 79.65034965034965
name: average_temperature
tags: location = santa_monica
-------------------------
time mean
2015-09-18T00:00:00Z <fill>
2015-09-18T12:00:00Z 79.95033445378151
Select with fill 10
> SELECT mean(degrees) FROM average_temperature
WHERE time < '2015-09-19'
AND time > '2015-09-18'
GROUP BY time(12h), location fill(10)
name: average_temperature
tags: location = coyote_creek
-------------------------
time mean
2015-09-18T00:00:00Z 79.83613445378151
2015-09-18T12:00:00Z 79.65034965034965
name: average_temperature
tags: location = santa_monica
-------------------------
time mean
2015-09-18T00:00:00Z 10
2015-09-18T12:00:00Z 79.95033445378151
Select with fill next
> SELECT mean(degrees) FROM average_temperature
WHERE time < '2015-09-19'
AND time > '2015-09-18'
GROUP BY time(12h), location fill(next)
name: average_temperature
tags: location = coyote_creek
-------------------------
time mean
2015-09-18T00:00:00Z 79.83613445378151
2015-09-18T12:00:00Z 79.65034965034965
name: average_temperature
tags: location = santa_monica
-------------------------
time mean
2015-09-18T00:00:00Z 79.95033445378151
2015-09-18T12:00:00Z 79.95033445378151
Select with fill none
> SELECT mean(degrees) FROM average_temperature
WHERE time < '2015-09-19'
AND time > '2015-09-18'
GROUP BY time(12h), location fill(none)
name: average_temperature
tags: location = coyote_creek
-------------------------
time mean
2015-09-18T00:00:00Z 79.83613445378151
2015-09-18T12:00:00Z 79.65034965034965
name: average_temperature
tags: location = santa_monica
-------------------------
time mean
2015-09-18T12:00:00Z 79.95033445378151
Select with fill null
> SELECT mean(degrees) FROM average_temperature
WHERE time < '2015-09-19'
AND time > '2015-09-18'
GROUP BY time(12h), location fill(null)
name: average_temperature
tags: location = coyote_creek
-------------------------
time mean
2015-09-18T00:00:00Z 79.83613445378151
2015-09-18T12:00:00Z 79.65034965034965
name: average_temperature
tags: location = santa_monica
-------------------------
time mean
2015-09-18T00:00:00Z
2015-09-18T12:00:00Z 79.95033445378151
Sub-query
SELECT … FROM (
SELECT …
FROM …
WHERE …
GROUP BY …
)WHERE …
GROUP BY …
Sub-query - Having
SELECT mean
FROM (SELECT mean(usage_user) FROM cpu
WHERE time > now() - 10m
GROUP BY time(1m))
WHERE mean > 10
Sub-query - Counting
the distinct tag values
SELECT distinct(count(host))
FROM (SELECT usage_user, host FROM cpu
WHERE time > now() - 10m)
Question
What happens?
> SELECT max(usage_user) FROM cpu
WHERE time > now() - 1d GROUP BY time(1d)
Answer
> SELECT max(usage_user) FROM cpu
WHERE time > now() - 1d GROUP BY time(1d)
name: cpu
time max
---- ---
2018-11-07T00:00:00Z 37.37373737373738
2018-11-08T00:00:00Z 72
How to get
one value back
> SELECT max(usage_user) FROM cpu
WHERE time > now() - 1d
GROUP BY time(1d,now())
name: cpu
time max
---- ---
2018-11-07T15:50:38.560319Z 72
2018-11-08T15:50:38.560319Z
I only want one!!
> SELECT max(usage_user) FROM cpu
WHERE time > now() - 1d
GROUP BY time(1d,now()) fill(none)
name: cpu
time max
---- ---
2018-11-07T15:51:37.466919Z 72
Question
What happens?
// No data in the time range
> SELECT count(usage_user) FROM cpu
WHERE time > now() - 10s
Answer
> SELECT count(usage_user) FROM cpu
WHERE time > now() - 1s
…
😞
Question
Write a query that
computes the average
of the usage_user and
usage_system fields
grouped by host in 20
second intervals
cpu,host=A usage_user=10,usage_system=70 10s
cpu,host=A usage_user=20,usage_system=67 20s
cpu,host=A usage_user=30,usage_system=72 30s
cpu,host=A usage_user=50,usage_system=74 40s
cpu,host=B usage_user=14,usage_system=40 10s
cpu,host=B usage_user=43,usage_system=56 20s
cpu,host=B usage_user=64,usage_system=77 30s
cpu,host=B usage_user=69,usage_system=74 40s
cpu,host=C usage_user=10,usage_system=12 10s
cpu,host=C usage_user=12,usage_system=23 20s
cpu,host=C usage_user=13,usage_system=78 30s
cpu,host=C usage_user=15,usage_system=88 40s
Answer
SELECT mean(usage_user), mean(usage_system)
FROM cpu
WHERE time > now() - 40s
GROUP BY time(20s), host
Answer
SELECT mean(*)
FROM cpu
WHERE time > now() - 40s
GROUP BY time(20s), host
Question
Write a query that
computes the mean of
free field for memory
and the mean of the
usage_user field for
cpu in 20s windows
cpu,host=A usage_user=10 10s
cpu,host=A usage_user=20 20s
cpu,host=A usage_user=30 30s
cpu,host=A usage_user=50 40s
mem,host=A free=10 10s
mem,host=A free=20 20s
mem,host=A free=123 30s
mem,host=A free=43 40s
Answer
SELECT mean(usage_user)
FROM cpu
WHERE time > now() - 40s
GROUP BY time(20s);
SELECT mean(free)
FROM mem
WHERE time > now() - 40s
GROUP BY time(20s)
Question
Write a query that
computes total rate of
change across all of
the counters grouped
by path
http_req,host=A,path=/ counter=10 10s
http_req,host=A,path=/ counter=11 20s
http_req,host=A,path=/ counter=0 30s
http_req,host=A,path=/ counter=9 40s
http_req,host=A,path=/home counter=20 10s
http_req,host=A,path=/home counter=21 20s
http_req,host=A,path=/home counter=24 30s
http_req,host=A,path=/home counter=25 40s
http_req,host=B,path=/admin counter=24 10s
http_req,host=B,path=/admin counter=30 20s
http_req,host=B,path=/admin counter=34 30s
http_req,host=B,path=/admin counter=34 40s
http_req,host=B,path=/home counter=20 10s
http_req,host=B,path=/home counter=25 20s
http_req,host=B,path=/home counter=28 30s
http_req,host=B,path=/home counter=30 40s
Answer
SELECT sum(rate) FROM (
SELECT non_negative_derivative(counter)
FROM http_req
WHERE <time range>
GROUP BY *
) GROUP BY time(10s), path
Answer
SELECT sum(rate) FROM (
SELECT non_negative_derivative(max(counter))
FROM http_req
WHERE <time range>
GROUP BY time(10s), *
) GROUP BY time(10s), path
Question
Write a query that
computes the ratio of
system load5 to cpu
usage_user
cpu,host=A usage_user=10 10s
cpu,host=A usage_user=20 20s
cpu,host=A usage_user=30 30s
cpu,host=A usage_user=50 40s
sys,host=A load5=10 10s
sys,host=A load5=20 20s
sys,host=A load5=34 30s
sys,host=A load5=43 40s
Answer
😞
© 2017 InfluxData. All rights reserved.44
© 2018 InfluxData. All rights reserved.44
TICKscript
© 2017 InfluxData. All rights reserved.45
What is TICKscript?
● Domain Specific Language for Kapacitor
○ Used to define Kapacitor tasks
● Made for processing streams of data
○ No support for ad-hoc execution
● Data model based off on InfluxDB line protocol
● Great for alerting
○ Expressing complex queries can be difficult and repetitive
© 2018 InfluxData. All rights reserved.46
TICK Script
• Chain invocation language
– | chains together different nodes
– . refers to specific attributes on a
node
• Variables refer to values
– Strings
– Ints, Floats
– Durations
– Pipelines
var measurement = 'requests'
var data = stream
|from()
.measurement(measurement)
|where(lambda: "is_up" == TRUE)
|where(lambda: "my_field" > 10)
|window()
.period(5m)
.every(5m)
// Count number of points in window
data
|count('value')
.as('the_count')
// Compute mean of data window
data
|mean('value')
.as('the_average')
© 2018 InfluxData. All rights reserved.47
TICKScript Syntax - Quoting Rules
• Double Quotes
– References data in lambda
expression
• Single Quotes
– Literal String value
// ' means the use the literal string value
var measurement = 'requests'
var data = stream
|from()
.measurement(measurement)
// " means use the reference value
|where(lambda: "is_up" == TRUE)
|where(lambda: "my_field" > 10)
|window()
.period(5m)
.every(5m)
// ' means to use the literal string value
data
|count('value')
.as('the_count')
© 2018 InfluxData. All rights reserved.48
Create a More Interesting Stream
TICKscript
• Create 5m windows of data that emit
every 1m
• Compute the average of the field
usage_user
• Log the result
// cpu.tick
stream
|from()
.measurement('cpu')
|window()
.period(5m)
.every(1m)
|mean('usage_user')
.as('mean_usage_user')
|log()
© 2018 InfluxData. All rights reserved.49
An even more interesting TICKscript
• Filter on the tag cpu=cpu-total
• Create 5m windows of data that emit every
1m
• Compute the average of the field
usage_user
• Log the result
// cpu.tick
stream
|from()
.measurement('cpu')
|where(lambda: "cpu" == 'cpu-total')
|window()
.period(5m)
.every(1m)
|mean('usage_user')
.as('mean_usage_user')
|log()
© 2018 InfluxData. All rights reserved.50
Create a Batch TICKscript
• Query 5m windows of data every 1m
• Compute the average of the field
usage_user
• Log the result
// batch_cpu.tick
batch
|query('''
SELECT mean("usage_user") AS
mean_usage_user
FROM "telegraf"."autogen"."cpu"
''')
.period(5m)
.every(1m)
|log()
Types of Nodes
● BatchNode
● StreamNode
● AlertNode
● BarrierNode
● ChangeDetect
● CombineNode
● DefaultNode
● DeleteNode
● DerivativeNode
● EC2AutoscaleNode
● EvalNode
● FlattenNode
● FromNode
● GroupByNode
● HTTPOutNode
● HTTPPostNode
● InfluxDBOutNode
● InfluxQLNode
● JoinNode
● K8sAutoscaleNode
● KapacitorLoopback
● LogNode
● NoOpNode
● QueryNode
● SampleNode
● ShiftNode
● SideloadNode
● StateCountNode
● StateDurationNode
● StatsNode
● SwarmAutoscaleNode
● UDFNode
● UnionNode
● WhereNode
● WindowNode
© 2017 InfluxData. All rights reserved.52
Batch Stream
● Issues an InfluxQL query to
InfluxDB on a schedule
● Yields the resulting data to the
rest of the Kapacitor pipeline
● InfluxDB writes data to Kapacitor
as it receives writes
● Each individual point is yielded to
the Kapacitor pipeline
Question
Write a query that
computes the average
of the usage_user and
usage_system fields
grouped by host in 20
second intervals
cpu,host=A usage_user=10,usage_system=70 10s
cpu,host=A usage_user=20,usage_system=67 20s
cpu,host=A usage_user=30,usage_system=72 30s
cpu,host=A usage_user=50,usage_system=74 40s
cpu,host=B usage_user=14,usage_system=40 10s
cpu,host=B usage_user=43,usage_system=56 20s
cpu,host=B usage_user=64,usage_system=77 30s
cpu,host=B usage_user=69,usage_system=74 40s
cpu,host=C usage_user=10,usage_system=12 10s
cpu,host=C usage_user=12,usage_system=23 20s
cpu,host=C usage_user=13,usage_system=78 30s
cpu,host=C usage_user=15,usage_system=88 40s
Answer
var data = stream
|from()
.measurement('cpu')
.groupBy('host')
|window()
.period(20s)
.every(20s)
data
|mean('usage_user')
data
|mean('usage_system')
Not Possible
var data = stream
|from()
.measurement('cpu')
|window()
.period(20s)
.every(20s)
data
|mean(*) // Cant do this
Question
Write a query that
computes the mean of
free field for memory
and the mean of the
usage_user field for
cpu in 20s windows
cpu,host=A usage_user=10 10s
cpu,host=A usage_user=20 20s
cpu,host=A usage_user=30 30s
cpu,host=A usage_user=50 40s
mem,host=A free=10 10s
mem,host=A free=20 20s
mem,host=A free=123 30s
mem,host=A free=43 40s
Answer
stream
|from()
.measurement('cpu')
|window()
.period(20s)
.every(20s)
|mean('usage_user')
stream
|from()
.measurement('mem')
|window()
.period(20s)
.every(20s)
|mean('free')
Question
Write a query that
computes total rate of
change across all of
the counters grouped
by path
http_req,host=A,path=/ counter=10 10s
http_req,host=A,path=/ counter=11 20s
http_req,host=A,path=/ counter=0 30s
http_req,host=A,path=/ counter=9 40s
http_req,host=A,path=/home counter=20 10s
http_req,host=A,path=/home counter=21 20s
http_req,host=A,path=/home counter=24 30s
http_req,host=A,path=/home counter=25 40s
http_req,host=B,path=/admin counter=24 10s
http_req,host=B,path=/admin counter=30 20s
http_req,host=B,path=/admin counter=34 30s
http_req,host=B,path=/admin counter=34 40s
http_req,host=B,path=/home counter=20 10s
http_req,host=B,path=/home counter=25 20s
http_req,host=B,path=/home counter=28 30s
http_req,host=B,path=/home counter=30 40s
Answer
stream
|from()
.measurement('http_req')
.groupBy(*)
|derivative('counter')
.nonNegative()
.as('rate')
|groupBy('path')
|sum('rate')
Question
Write a query that
computes the ratio of
system load5 to cpu
usage_user
cpu,host=A usage_user=10 10s
cpu,host=A usage_user=20 20s
cpu,host=A usage_user=30 30s
cpu,host=A usage_user=50 40s
sys,host=A load5=10 10s
sys,host=A load5=20 20s
sys,host=A load5=34 30s
sys,host=A load5=43 40s
Answer
var cpu = stream
|from()
.measurement('cpu')
var sys = stream
|from()
.measurement('sys')
cpu
|join('sys')
.as('cpu', 'sys')
|eval(lambda: "cpu.usage_user" / "sys.load5")
© 2017 InfluxData. All rights reserved.62
© 2018 InfluxData. All rights reserved.62
Flux
© 2017 InfluxData. All rights reserved.63
What is Flux?
● “Data Scripting Language”
○ Idea is to allow for more features than you would expect from a pure
query language
● Made for processing streams of data
○ Supports ad-hoc queries
● Rich data model that is distinct from the InfluxDB data model
○ Makes it easier to reason about the internals of what the language is
doing to your data
Question
Write a query that
computes the average
of the usage_user and
usage_system fields
grouped by host in 20
second intervals
cpu,host=A usage_user=10,usage_system=70 10s
cpu,host=A usage_user=20,usage_system=67 20s
cpu,host=A usage_user=30,usage_system=72 30s
cpu,host=A usage_user=50,usage_system=74 40s
cpu,host=B usage_user=14,usage_system=40 10s
cpu,host=B usage_user=43,usage_system=56 20s
cpu,host=B usage_user=64,usage_system=77 30s
cpu,host=B usage_user=69,usage_system=74 40s
cpu,host=C usage_user=10,usage_system=12 10s
cpu,host=C usage_user=12,usage_system=23 20s
cpu,host=C usage_user=13,usage_system=78 30s
cpu,host=C usage_user=15,usage_system=88 40s
Answer
from(bucket: "mybucket")
|> range(start: -40s)
|> filter(fn: (r) => r._measurement == "cpu")
|> filter(fn: (r) => r._field == "usage_user" OR
r._field == "usage_system")
|> group(by: ["host"])
|> window(period: 20s, every: 20s)
|> mean()
More General
from(bucket: "mybucket")
|> range(start: -40s)
|> filter(fn: (r) => r._measurement == "cpu")
|> group(by: ["host"])
|> window(period: 20s, every: 20s)
|> mean()
Question
Write a query that
computes the mean of
free field for memory
and the mean of the
usage_user field for
cpu in 20s windows
cpu,host=A usage_user=10 10s
cpu,host=A usage_user=20 20s
cpu,host=A usage_user=30 30s
cpu,host=A usage_user=50 40s
mem,host=A free=10 10s
mem,host=A free=20 20s
mem,host=A free=123 30s
mem,host=A free=43 40s
Answer
from(bucket: "mybucket")
|> range(start: -40s)
|> filter(fn: (r) => r._measurement == "cpu" OR
r._measurement == "mem")
|> filter(fn: (r) => r._field == "usage_user" OR
r._field == "free")
|> group(by: ["host"])
|> window(period: 20s, every: 20s)
|> mean()
Question
Write a query that
computes total rate of
change across all of
the counters grouped
by path
http_req,host=A,path=/ counter=10 10s
http_req,host=A,path=/ counter=11 20s
http_req,host=A,path=/ counter=0 30s
http_req,host=A,path=/ counter=9 40s
http_req,host=A,path=/home counter=20 10s
http_req,host=A,path=/home counter=21 20s
http_req,host=A,path=/home counter=24 30s
http_req,host=A,path=/home counter=25 40s
http_req,host=B,path=/admin counter=24 10s
http_req,host=B,path=/admin counter=30 20s
http_req,host=B,path=/admin counter=34 30s
http_req,host=B,path=/admin counter=34 40s
http_req,host=B,path=/home counter=20 10s
http_req,host=B,path=/home counter=25 20s
http_req,host=B,path=/home counter=28 30s
http_req,host=B,path=/home counter=30 40s
Answer
from(bucket: "mybucket")
|> range(start: -40s)
|> filter(fn: (r) => r._measurement == "http_req")
|> filter(fn: (r) => r._field == "counter")
|> derivative(non_negative: true)
|> group(by: ["path"])
|> sum()
Making that over
time
from(bucket: "mybucket")
|> range(start: -40s)
|> filter(fn: (r) => r._measurement == "http_req")
|> filter(fn: (r) => r._field == "counter")
|> derivative(non_negative: true)
|> group(by: ["path"])
|> window(period: 20s, every:20s)
|> sum()
Turning it into a
function so I never
have to think
about it again
rate =(table=<-, m, field="counter", by, interval) =>
|> filter(fn: (r) => r._measurement == m)
|> filter(fn: (r) => r._field == field)
|> derivative(non_negative: true)
|> group(by: by)
|> window(period: interval, every: interval)
|> sum()
from(bucket: "mybucket")
|> range(start: -40s)
|> rate(m: "http_req",
by: ["path"],
interval: 20s)
Question
Write a query that
computes the ratio of
system load5 to cpu
usage_user
cpu,host=A usage_user=10 10s
cpu,host=A usage_user=20 20s
cpu,host=A usage_user=30 30s
cpu,host=A usage_user=50 40s
sys,host=A load5=10 10s
sys,host=A load5=20 20s
sys,host=A load5=34 30s
sys,host=A load5=43 40s
Answer
mybucket = from(bucket: "mybucket")
|> range(start: -40s)
cpu = mybucket
|> filter(fn: (r) => r._measurement == "cpu")
|> filter(fn: (r) => r._field == "usage_user")
sys = mybucket
|> filter(fn: (r) => r._measurement == "sys")
|> filter(fn: (r) => r._field == "load5")
join(tables: {cpu: cpu, sys: sys}, on: ["_time"])
|> map(fn: (r) => r._cpu_value / r._sys_value)

More Related Content

What's hot

Dynamo and BigTable in light of the CAP theorem
Dynamo and BigTable in light of the CAP theoremDynamo and BigTable in light of the CAP theorem
Dynamo and BigTable in light of the CAP theoremGrisha Weintraub
 
Sharding Methods for MongoDB
Sharding Methods for MongoDBSharding Methods for MongoDB
Sharding Methods for MongoDBMongoDB
 
Vitess VReplication: Standing on the Shoulders of a MySQL Giant
Vitess VReplication: Standing on the Shoulders of a MySQL GiantVitess VReplication: Standing on the Shoulders of a MySQL Giant
Vitess VReplication: Standing on the Shoulders of a MySQL GiantMatt Lord
 
The InfluxDB 2.0 Storage Engine | Jacob Marble | InfluxData
The InfluxDB 2.0 Storage Engine | Jacob Marble | InfluxDataThe InfluxDB 2.0 Storage Engine | Jacob Marble | InfluxData
The InfluxDB 2.0 Storage Engine | Jacob Marble | InfluxDataInfluxData
 
Optimizing Delta/Parquet Data Lakes for Apache Spark
Optimizing Delta/Parquet Data Lakes for Apache SparkOptimizing Delta/Parquet Data Lakes for Apache Spark
Optimizing Delta/Parquet Data Lakes for Apache SparkDatabricks
 
How to Structure the Data Organization
How to Structure the Data OrganizationHow to Structure the Data Organization
How to Structure the Data OrganizationRobyn Bollhorst
 
Data Catalog as a Business Enabler
Data Catalog as a Business EnablerData Catalog as a Business Enabler
Data Catalog as a Business EnablerSrinivasan Sankar
 
Data Governance Initiative
Data Governance InitiativeData Governance Initiative
Data Governance InitiativeDataWorks Summit
 
Facebook Messages & HBase
Facebook Messages & HBaseFacebook Messages & HBase
Facebook Messages & HBase强 王
 
New Analytic Uses of Master Data Management in the Enterprise
New Analytic Uses of Master Data Management in the EnterpriseNew Analytic Uses of Master Data Management in the Enterprise
New Analytic Uses of Master Data Management in the EnterpriseDATAVERSITY
 
Developing a Data Strategy
Developing a Data StrategyDeveloping a Data Strategy
Developing a Data StrategyMartha Horler
 
Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...
Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...
Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...DATAVERSITY
 
Data governance
Data governanceData governance
Data governanceMD Redaan
 
Impuls-Vortrag Data Strategy
Impuls-Vortrag Data StrategyImpuls-Vortrag Data Strategy
Impuls-Vortrag Data StrategyMarco Geuer
 
Enterprise Architecture vs. Data Architecture
Enterprise Architecture vs. Data ArchitectureEnterprise Architecture vs. Data Architecture
Enterprise Architecture vs. Data ArchitectureDATAVERSITY
 
CDO - Chief Data Officer Momentum and Trends
CDO - Chief Data Officer Momentum and TrendsCDO - Chief Data Officer Momentum and Trends
CDO - Chief Data Officer Momentum and TrendsJeffrey T. Pollock
 
Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...
Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...
Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...StampedeCon
 
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...HostedbyConfluent
 
Virtual Flink Forward 2020: Netflix Data Mesh: Composable Data Processing - J...
Virtual Flink Forward 2020: Netflix Data Mesh: Composable Data Processing - J...Virtual Flink Forward 2020: Netflix Data Mesh: Composable Data Processing - J...
Virtual Flink Forward 2020: Netflix Data Mesh: Composable Data Processing - J...Flink Forward
 

What's hot (20)

Dynamo and BigTable in light of the CAP theorem
Dynamo and BigTable in light of the CAP theoremDynamo and BigTable in light of the CAP theorem
Dynamo and BigTable in light of the CAP theorem
 
Sharding Methods for MongoDB
Sharding Methods for MongoDBSharding Methods for MongoDB
Sharding Methods for MongoDB
 
Vitess VReplication: Standing on the Shoulders of a MySQL Giant
Vitess VReplication: Standing on the Shoulders of a MySQL GiantVitess VReplication: Standing on the Shoulders of a MySQL Giant
Vitess VReplication: Standing on the Shoulders of a MySQL Giant
 
The InfluxDB 2.0 Storage Engine | Jacob Marble | InfluxData
The InfluxDB 2.0 Storage Engine | Jacob Marble | InfluxDataThe InfluxDB 2.0 Storage Engine | Jacob Marble | InfluxData
The InfluxDB 2.0 Storage Engine | Jacob Marble | InfluxData
 
Optimizing Delta/Parquet Data Lakes for Apache Spark
Optimizing Delta/Parquet Data Lakes for Apache SparkOptimizing Delta/Parquet Data Lakes for Apache Spark
Optimizing Delta/Parquet Data Lakes for Apache Spark
 
How to Structure the Data Organization
How to Structure the Data OrganizationHow to Structure the Data Organization
How to Structure the Data Organization
 
Data Catalog as a Business Enabler
Data Catalog as a Business EnablerData Catalog as a Business Enabler
Data Catalog as a Business Enabler
 
Data Governance Initiative
Data Governance InitiativeData Governance Initiative
Data Governance Initiative
 
Facebook Messages & HBase
Facebook Messages & HBaseFacebook Messages & HBase
Facebook Messages & HBase
 
New Analytic Uses of Master Data Management in the Enterprise
New Analytic Uses of Master Data Management in the EnterpriseNew Analytic Uses of Master Data Management in the Enterprise
New Analytic Uses of Master Data Management in the Enterprise
 
Developing a Data Strategy
Developing a Data StrategyDeveloping a Data Strategy
Developing a Data Strategy
 
Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...
Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...
Self-Service Data Analysis, Data Wrangling, Data Munging, and Data Modeling –...
 
Data governance
Data governanceData governance
Data governance
 
Impuls-Vortrag Data Strategy
Impuls-Vortrag Data StrategyImpuls-Vortrag Data Strategy
Impuls-Vortrag Data Strategy
 
Deep Dive on Amazon Redshift
Deep Dive on Amazon RedshiftDeep Dive on Amazon Redshift
Deep Dive on Amazon Redshift
 
Enterprise Architecture vs. Data Architecture
Enterprise Architecture vs. Data ArchitectureEnterprise Architecture vs. Data Architecture
Enterprise Architecture vs. Data Architecture
 
CDO - Chief Data Officer Momentum and Trends
CDO - Chief Data Officer Momentum and TrendsCDO - Chief Data Officer Momentum and Trends
CDO - Chief Data Officer Momentum and Trends
 
Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...
Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...
Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...
 
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
Standing on the Shoulders of Open-Source Giants: The Serverless Realtime Lake...
 
Virtual Flink Forward 2020: Netflix Data Mesh: Composable Data Processing - J...
Virtual Flink Forward 2020: Netflix Data Mesh: Composable Data Processing - J...Virtual Flink Forward 2020: Netflix Data Mesh: Composable Data Processing - J...
Virtual Flink Forward 2020: Netflix Data Mesh: Composable Data Processing - J...
 

Similar to INFLUXQL & TICKSCRIPT

Macy's: Changing Engines in Mid-Flight
Macy's: Changing Engines in Mid-FlightMacy's: Changing Engines in Mid-Flight
Macy's: Changing Engines in Mid-FlightDataStax Academy
 
Advanced Query Optimizer Tuning and Analysis
Advanced Query Optimizer Tuning and AnalysisAdvanced Query Optimizer Tuning and Analysis
Advanced Query Optimizer Tuning and AnalysisMYXPLAIN
 
LVOUG meetup #4 - Case Study 10g to 11g
LVOUG meetup #4 - Case Study 10g to 11gLVOUG meetup #4 - Case Study 10g to 11g
LVOUG meetup #4 - Case Study 10g to 11gMaris Elsins
 
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013Sergey Petrunya
 
MongoDB Days UK: Using MongoDB and Python for Data Analysis Pipelines
MongoDB Days UK: Using MongoDB and Python for Data Analysis PipelinesMongoDB Days UK: Using MongoDB and Python for Data Analysis Pipelines
MongoDB Days UK: Using MongoDB and Python for Data Analysis PipelinesMongoDB
 
OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...
OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...
OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...Altinity Ltd
 
OpenWorld Sep14 12c for_developers
OpenWorld Sep14 12c for_developersOpenWorld Sep14 12c for_developers
OpenWorld Sep14 12c for_developersConnor McDonald
 
Drizzle to MySQL, Stress Free Migration
Drizzle to MySQL, Stress Free MigrationDrizzle to MySQL, Stress Free Migration
Drizzle to MySQL, Stress Free MigrationAndrew Hutchings
 
Unit-5 Time series data Analysis.pptx
Unit-5 Time series data Analysis.pptxUnit-5 Time series data Analysis.pptx
Unit-5 Time series data Analysis.pptxSheba41
 
Oracle 12c Automatic Data Optimization (ADO) - ILM
Oracle 12c Automatic Data Optimization (ADO) - ILMOracle 12c Automatic Data Optimization (ADO) - ILM
Oracle 12c Automatic Data Optimization (ADO) - ILMMonowar Mukul
 
Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021
Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021
Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021InfluxData
 
Dangerous on ClickHouse in 30 minutes, by Robert Hodges, Altinity CEO
Dangerous on ClickHouse in 30 minutes, by Robert Hodges, Altinity CEODangerous on ClickHouse in 30 minutes, by Robert Hodges, Altinity CEO
Dangerous on ClickHouse in 30 minutes, by Robert Hodges, Altinity CEOAltinity Ltd
 
Apache Cassandra for Timeseries- and Graph-Data
Apache Cassandra for Timeseries- and Graph-DataApache Cassandra for Timeseries- and Graph-Data
Apache Cassandra for Timeseries- and Graph-DataGuido Schmutz
 
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOxInfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOxInfluxData
 
2021 10-13 i ox query processing
2021 10-13 i ox query processing2021 10-13 i ox query processing
2021 10-13 i ox query processingAndrew Lamb
 
Oracle trace data collection errors: the story about oceans, islands, and rivers
Oracle trace data collection errors: the story about oceans, islands, and riversOracle trace data collection errors: the story about oceans, islands, and rivers
Oracle trace data collection errors: the story about oceans, islands, and riversCary Millsap
 
OpenWorld 2018 - Common Application Developer Disasters
OpenWorld 2018 - Common Application Developer DisastersOpenWorld 2018 - Common Application Developer Disasters
OpenWorld 2018 - Common Application Developer DisastersConnor McDonald
 
Sangam 19 - PLSQL still the coolest
Sangam 19 - PLSQL still the coolestSangam 19 - PLSQL still the coolest
Sangam 19 - PLSQL still the coolestConnor McDonald
 

Similar to INFLUXQL & TICKSCRIPT (20)

Macy's: Changing Engines in Mid-Flight
Macy's: Changing Engines in Mid-FlightMacy's: Changing Engines in Mid-Flight
Macy's: Changing Engines in Mid-Flight
 
Advanced Query Optimizer Tuning and Analysis
Advanced Query Optimizer Tuning and AnalysisAdvanced Query Optimizer Tuning and Analysis
Advanced Query Optimizer Tuning and Analysis
 
Introduction to SLURM
Introduction to SLURMIntroduction to SLURM
Introduction to SLURM
 
LVOUG meetup #4 - Case Study 10g to 11g
LVOUG meetup #4 - Case Study 10g to 11gLVOUG meetup #4 - Case Study 10g to 11g
LVOUG meetup #4 - Case Study 10g to 11g
 
Flashback ITOUG
Flashback ITOUGFlashback ITOUG
Flashback ITOUG
 
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
MySQL/MariaDB query optimizer tuning tutorial from Percona Live 2013
 
MongoDB Days UK: Using MongoDB and Python for Data Analysis Pipelines
MongoDB Days UK: Using MongoDB and Python for Data Analysis PipelinesMongoDB Days UK: Using MongoDB and Python for Data Analysis Pipelines
MongoDB Days UK: Using MongoDB and Python for Data Analysis Pipelines
 
OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...
OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...
OSA Con 2022 - Specifics of data analysis in Time Series Databases - Roman Kh...
 
OpenWorld Sep14 12c for_developers
OpenWorld Sep14 12c for_developersOpenWorld Sep14 12c for_developers
OpenWorld Sep14 12c for_developers
 
Drizzle to MySQL, Stress Free Migration
Drizzle to MySQL, Stress Free MigrationDrizzle to MySQL, Stress Free Migration
Drizzle to MySQL, Stress Free Migration
 
Unit-5 Time series data Analysis.pptx
Unit-5 Time series data Analysis.pptxUnit-5 Time series data Analysis.pptx
Unit-5 Time series data Analysis.pptx
 
Oracle 12c Automatic Data Optimization (ADO) - ILM
Oracle 12c Automatic Data Optimization (ADO) - ILMOracle 12c Automatic Data Optimization (ADO) - ILM
Oracle 12c Automatic Data Optimization (ADO) - ILM
 
Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021
Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021
Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021
 
Dangerous on ClickHouse in 30 minutes, by Robert Hodges, Altinity CEO
Dangerous on ClickHouse in 30 minutes, by Robert Hodges, Altinity CEODangerous on ClickHouse in 30 minutes, by Robert Hodges, Altinity CEO
Dangerous on ClickHouse in 30 minutes, by Robert Hodges, Altinity CEO
 
Apache Cassandra for Timeseries- and Graph-Data
Apache Cassandra for Timeseries- and Graph-DataApache Cassandra for Timeseries- and Graph-Data
Apache Cassandra for Timeseries- and Graph-Data
 
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOxInfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
 
2021 10-13 i ox query processing
2021 10-13 i ox query processing2021 10-13 i ox query processing
2021 10-13 i ox query processing
 
Oracle trace data collection errors: the story about oceans, islands, and rivers
Oracle trace data collection errors: the story about oceans, islands, and riversOracle trace data collection errors: the story about oceans, islands, and rivers
Oracle trace data collection errors: the story about oceans, islands, and rivers
 
OpenWorld 2018 - Common Application Developer Disasters
OpenWorld 2018 - Common Application Developer DisastersOpenWorld 2018 - Common Application Developer Disasters
OpenWorld 2018 - Common Application Developer Disasters
 
Sangam 19 - PLSQL still the coolest
Sangam 19 - PLSQL still the coolestSangam 19 - PLSQL still the coolest
Sangam 19 - PLSQL still the coolest
 

More from InfluxData

Announcing InfluxDB Clustered
Announcing InfluxDB ClusteredAnnouncing InfluxDB Clustered
Announcing InfluxDB ClusteredInfluxData
 
Best Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow EcosystemBest Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow EcosystemInfluxData
 
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...InfluxData
 
Power Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDBPower Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDBInfluxData
 
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base InfluxData
 
Build an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING StackBuild an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING StackInfluxData
 
Meet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using RustMeet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using RustInfluxData
 
Introducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud DedicatedIntroducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud DedicatedInfluxData
 
Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB InfluxData
 
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...InfluxData
 
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...InfluxData
 
Introducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage EngineIntroducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage EngineInfluxData
 
Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena InfluxData
 
Understanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage EngineUnderstanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage EngineInfluxData
 
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDBStreamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDBInfluxData
 
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...InfluxData
 
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022InfluxData
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022InfluxData
 
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...InfluxData
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022InfluxData
 

More from InfluxData (20)

Announcing InfluxDB Clustered
Announcing InfluxDB ClusteredAnnouncing InfluxDB Clustered
Announcing InfluxDB Clustered
 
Best Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow EcosystemBest Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow Ecosystem
 
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
 
Power Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDBPower Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDB
 
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
 
Build an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING StackBuild an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING Stack
 
Meet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using RustMeet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using Rust
 
Introducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud DedicatedIntroducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud Dedicated
 
Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB
 
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
 
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
 
Introducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage EngineIntroducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage Engine
 
Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena
 
Understanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage EngineUnderstanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage Engine
 
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDBStreamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
 
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
 
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
 
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
 

Recently uploaded

1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
 
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesMulti-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesSanjeev Rampal
 
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理aagad
 
The AI Powered Organization-Intro to AI-LAN.pdf
The AI Powered Organization-Intro to AI-LAN.pdfThe AI Powered Organization-Intro to AI-LAN.pdf
The AI Powered Organization-Intro to AI-LAN.pdfSiskaFitrianingrum
 
How to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptxHow to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptxGal Baras
 
The+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptxThe+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptxlaozhuseo02
 
The Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case StudyThe Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case StudyDamar Juniarto
 
Article writing on excessive use of internet.pptx
Article writing on excessive use of internet.pptxArticle writing on excessive use of internet.pptx
Article writing on excessive use of internet.pptxabhinandnam9997
 
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shopHistory+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shoplaozhuseo02
 
ER(Entity Relationship) Diagram for online shopping - TAE
ER(Entity Relationship) Diagram for online shopping - TAEER(Entity Relationship) Diagram for online shopping - TAE
ER(Entity Relationship) Diagram for online shopping - TAEHimani415946
 

Recently uploaded (12)

The Best AI Powered Software - Intellivid AI Studio
The Best AI Powered Software - Intellivid AI StudioThe Best AI Powered Software - Intellivid AI Studio
The Best AI Powered Software - Intellivid AI Studio
 
1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...
 
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesMulti-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
 
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
一比一原版UTS毕业证悉尼科技大学毕业证成绩单如何办理
 
The AI Powered Organization-Intro to AI-LAN.pdf
The AI Powered Organization-Intro to AI-LAN.pdfThe AI Powered Organization-Intro to AI-LAN.pdf
The AI Powered Organization-Intro to AI-LAN.pdf
 
How to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptxHow to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptx
 
The+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptxThe+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptx
 
The Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case StudyThe Use of AI in Indonesia Election 2024: A Case Study
The Use of AI in Indonesia Election 2024: A Case Study
 
Article writing on excessive use of internet.pptx
Article writing on excessive use of internet.pptxArticle writing on excessive use of internet.pptx
Article writing on excessive use of internet.pptx
 
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shopHistory+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
 
Stay Ahead with 2024's Top Web Design Trends
Stay Ahead with 2024's Top Web Design TrendsStay Ahead with 2024's Top Web Design Trends
Stay Ahead with 2024's Top Web Design Trends
 
ER(Entity Relationship) Diagram for online shopping - TAE
ER(Entity Relationship) Diagram for online shopping - TAEER(Entity Relationship) Diagram for online shopping - TAE
ER(Entity Relationship) Diagram for online shopping - TAE
 

INFLUXQL & TICKSCRIPT

  • 1. © 2017 InfluxData. All rights reserved.1 Agenda: New Practitioners Track WORKSHOPAGENDA 8:30 AM – 9:00 AM Coffee & Pastries 9:00 AM – 9:15 AM Welcome 9:15 AM – 10:15 AM Introduction to the TICK Stack Katy Farmer 10:15 AM – 10:30 AM Break 10:30 AM – 11:30 AM Optimizing the TICK Stack Sam Dillard 11:30 AM – 11:45 AM Break 11:45 AM – 12:45 PM Chronograf and Dashboarding Russ Savage 12:45 PM – 1:45 PM Lunch 1:45 PM – 2:45 PM InfluxEnterprise Architectural Patterns Craig Hobbs 2:45 PM – 3:00 PM Break 3:00 PM – 4:00 PM InfluxQL & TICKscript Michael DeSa 4:00 PM – 4:15PM Closing
  • 2. Michael Desa Software Engineer, InfluxData InfluxQL & TICKscript Michael Desa is a Software Engineer at InfluxData who works on the 2.0 API for the InfluxData Platform. He has led the InfluxDB training course across the US, providing students with an in depth understanding of how InfluxDB works as well as sharing best practices. He has a degree in Math from the University of California, at Berkeley.
  • 3. © 2017 InfluxData. All rights reserved.3 © 2017 InfluxData. All rights reserved.3 ✓ What is InfluxQL • What types of questions can I ask about my data with InfluxQL ✓ What is TICKscript • What types of questions can I solve with TICKscript ✓ What is Flux • Why does it matter Agenda
  • 4. © 2017 InfluxData. All rights reserved.4 © 2018 InfluxData. All rights reserved.4 InfluxQL
  • 5. © 2017 InfluxData. All rights reserved.5 What is InfluxQL? ● A SQL-like query language for InfluxDB ○ Extends SQL to express queries that range over time ○ Limited support for relational features of SQL ● Easy to use ○ Should feel familiar to anyone with SQL experience ■ Not SQL compliant (which can bother those that want full access to SQL) ● Data definition and meta queries ○ Used to define and view database structures
  • 6. Basic Select Statement SELECT <field> FROM <measurement> SELECT * FROM cpu SELECT free FROM mem SELECT x + y FROM vars SELECT x,y FROM nums
  • 7. Basic Select Statement > SELECT * FROM h2o_quality LIMIT 10 name: h2o_quality ----------------- time index location id 2015-08-18T00:00:00Z 41 coyote_creek 1 2015-08-18T00:00:00Z 99 santa_monica 2 2015-08-18T00:00:00Z 41 coyote_creek 1 2015-08-18T00:06:00Z 56 santa_monica 2 2015-08-18T00:06:00Z 11 coyote_creek 3 2015-08-18T00:06:00Z 11 coyote_creek 3 2015-08-18T00:12:00Z 65 santa_monica 3 2015-08-18T00:12:00Z 38 coyote_creek 1 2015-08-18T00:12:00Z 38 coyote_creek 1 2015-08-18T00:18:00Z 57 santa_monica 3
  • 8. Select Statement with WHERE clause SELECT <field> FROM <measurement> WHERE <conditions> SELECT * FROM cpu WHERE busy > 50 SELECT free FROM mem WHERE host = 'server1' SELECT x + y FROM vars WHERE some_tag = 'some_key' SELECT x,y FROM nums WHERE domain =~ /.*/
  • 9. Select Statement with Relative Time SELECT <field> FROM <measurement> WHERE <time> SELECT * FROM cpu WHERE time > now() - 1h SELECT * FROM cpu WHERE time > now() - 10s SELECT free FROM mem WHERE time > now() - 4d SELECT x + y FROM vars WHERE time > now() - 10w SELECT x,y FROM nums WHERE time > now() + 15m
  • 10. Select Statement with GROUP BY clause [SELECT STATEMENT] GROUP BY <tag> SELECT * FROM cpu GROUP BY host SELECT * FROM cpu GROUP BY * SELECT free FROM mem GROUP BY location, host
  • 11. Select Statement with GROUP BY clause > SELECT * FROM h2o_quality GROUP BY location name: h2o_quality tags: location = coyote_creek time index id ---- ----- --- 2015-08-18T00:00:00Z 41 1 2015-08-18T00:00:00Z 41 1 name: h2o_quality tags: location = santa_monica time index id ---- ----- --- 2015-08-18T00:00:00Z 99 2 2015-08-18T00:06:00Z 56 2
  • 12. Select Statement with a function SELECT <function>(<field>) FROM <measurement> SELECT count(value) FROM cpu SELECT mean(free) FROM mem WHERE time > now() - 1h SELECT sum(x) FROM vars WHERE x > 100 SELECT median(y) FROM nums WHERE domain = 'Z'
  • 13. Select Statement with a function > SELECT count(index) FROM h2o_quality WHERE location = 'coyote_creek' name: h2o_quality ----------------- time count 1970-01-01T00:00:00Z 12777
  • 14. Select Statement with a function > SELECT max(usage_user) FROM cpu WHERE time > now() - 10d name: cpu time max ---- --- 2018-11-05T22:12:05Z 54
  • 15. Types of Functions ● Aggregators ○ count ○ distinct ○ integral ○ mean ○ median ○ spread ○ sum ○ stddev ● Selectors ○ bottom ○ first ○ last ○ max ○ min ○ percentile ○ top ● Transformers ○ derivative ○ difference ○ moving_average ○ elapsed
  • 16. Select Statement with GROUP BY time clause [SELECT STATEMENT] WHERE <time condition> GROUP BY time(<period>) SELECT max(busy) FROM cpu WHERE time > now() - 1h GROUP BY time(10m) SELECT mean(free) FROM free WHERE time > now() - 1d GROUP BY time(1h)
  • 17. Select Statement with GROUP BY time clause Invalid queries SELECT busy FROM cpu WHERE time > now() - 1h GROUP BY time(10m) SELECT mean(busy) FROM cpu GROUP BY time(10m)
  • 18. Select Statement with GROUP BY time clause > SELECT mean(degrees) FROM average_temperature WHERE time < '2015-09-19' AND time > '2015-09-18' GROUP BY time(12h) name: average_temperature ------------------------- time mean 2015-09-18T00:00:00Z 79.83613445378151 2015-09-18T12:00:00Z 79.65034965034965
  • 19. Select Statement with GROUP BY time and tag > SELECT mean(degrees) FROM average_temperature WHERE time < '2015-09-19' AND time > '2015-09-18' GROUP BY time(12h), location name: average_temperature tags: location = coyote_creek ------------------------- time mean 2015-09-18T00:00:00Z 79.83613445378151 2015-09-18T12:00:00Z 79.65034965034965 name: average_temperature tags: location = santa_monica ------------------------- time mean 2015-09-18T00:00:00Z 2015-09-18T12:00:00Z 79.95033445378151
  • 20. Select with fill > SELECT mean(degrees) FROM average_temperature WHERE time < '2015-09-19' AND time > '2015-09-18' GROUP BY time(12h), location fill(<fill>) name: average_temperature tags: location = coyote_creek ------------------------- time mean 2015-09-18T00:00:00Z 79.83613445378151 2015-09-18T12:00:00Z 79.65034965034965 name: average_temperature tags: location = santa_monica ------------------------- time mean 2015-09-18T00:00:00Z <fill> 2015-09-18T12:00:00Z 79.95033445378151
  • 21. Select with fill 10 > SELECT mean(degrees) FROM average_temperature WHERE time < '2015-09-19' AND time > '2015-09-18' GROUP BY time(12h), location fill(10) name: average_temperature tags: location = coyote_creek ------------------------- time mean 2015-09-18T00:00:00Z 79.83613445378151 2015-09-18T12:00:00Z 79.65034965034965 name: average_temperature tags: location = santa_monica ------------------------- time mean 2015-09-18T00:00:00Z 10 2015-09-18T12:00:00Z 79.95033445378151
  • 22. Select with fill next > SELECT mean(degrees) FROM average_temperature WHERE time < '2015-09-19' AND time > '2015-09-18' GROUP BY time(12h), location fill(next) name: average_temperature tags: location = coyote_creek ------------------------- time mean 2015-09-18T00:00:00Z 79.83613445378151 2015-09-18T12:00:00Z 79.65034965034965 name: average_temperature tags: location = santa_monica ------------------------- time mean 2015-09-18T00:00:00Z 79.95033445378151 2015-09-18T12:00:00Z 79.95033445378151
  • 23. Select with fill none > SELECT mean(degrees) FROM average_temperature WHERE time < '2015-09-19' AND time > '2015-09-18' GROUP BY time(12h), location fill(none) name: average_temperature tags: location = coyote_creek ------------------------- time mean 2015-09-18T00:00:00Z 79.83613445378151 2015-09-18T12:00:00Z 79.65034965034965 name: average_temperature tags: location = santa_monica ------------------------- time mean 2015-09-18T12:00:00Z 79.95033445378151
  • 24. Select with fill null > SELECT mean(degrees) FROM average_temperature WHERE time < '2015-09-19' AND time > '2015-09-18' GROUP BY time(12h), location fill(null) name: average_temperature tags: location = coyote_creek ------------------------- time mean 2015-09-18T00:00:00Z 79.83613445378151 2015-09-18T12:00:00Z 79.65034965034965 name: average_temperature tags: location = santa_monica ------------------------- time mean 2015-09-18T00:00:00Z 2015-09-18T12:00:00Z 79.95033445378151
  • 25. Sub-query SELECT … FROM ( SELECT … FROM … WHERE … GROUP BY … )WHERE … GROUP BY …
  • 26. Sub-query - Having SELECT mean FROM (SELECT mean(usage_user) FROM cpu WHERE time > now() - 10m GROUP BY time(1m)) WHERE mean > 10
  • 27. Sub-query - Counting the distinct tag values SELECT distinct(count(host)) FROM (SELECT usage_user, host FROM cpu WHERE time > now() - 10m)
  • 28. Question What happens? > SELECT max(usage_user) FROM cpu WHERE time > now() - 1d GROUP BY time(1d)
  • 29. Answer > SELECT max(usage_user) FROM cpu WHERE time > now() - 1d GROUP BY time(1d) name: cpu time max ---- --- 2018-11-07T00:00:00Z 37.37373737373738 2018-11-08T00:00:00Z 72
  • 30. How to get one value back > SELECT max(usage_user) FROM cpu WHERE time > now() - 1d GROUP BY time(1d,now()) name: cpu time max ---- --- 2018-11-07T15:50:38.560319Z 72 2018-11-08T15:50:38.560319Z
  • 31. I only want one!! > SELECT max(usage_user) FROM cpu WHERE time > now() - 1d GROUP BY time(1d,now()) fill(none) name: cpu time max ---- --- 2018-11-07T15:51:37.466919Z 72
  • 32. Question What happens? // No data in the time range > SELECT count(usage_user) FROM cpu WHERE time > now() - 10s
  • 33. Answer > SELECT count(usage_user) FROM cpu WHERE time > now() - 1s … 😞
  • 34. Question Write a query that computes the average of the usage_user and usage_system fields grouped by host in 20 second intervals cpu,host=A usage_user=10,usage_system=70 10s cpu,host=A usage_user=20,usage_system=67 20s cpu,host=A usage_user=30,usage_system=72 30s cpu,host=A usage_user=50,usage_system=74 40s cpu,host=B usage_user=14,usage_system=40 10s cpu,host=B usage_user=43,usage_system=56 20s cpu,host=B usage_user=64,usage_system=77 30s cpu,host=B usage_user=69,usage_system=74 40s cpu,host=C usage_user=10,usage_system=12 10s cpu,host=C usage_user=12,usage_system=23 20s cpu,host=C usage_user=13,usage_system=78 30s cpu,host=C usage_user=15,usage_system=88 40s
  • 35. Answer SELECT mean(usage_user), mean(usage_system) FROM cpu WHERE time > now() - 40s GROUP BY time(20s), host
  • 36. Answer SELECT mean(*) FROM cpu WHERE time > now() - 40s GROUP BY time(20s), host
  • 37. Question Write a query that computes the mean of free field for memory and the mean of the usage_user field for cpu in 20s windows cpu,host=A usage_user=10 10s cpu,host=A usage_user=20 20s cpu,host=A usage_user=30 30s cpu,host=A usage_user=50 40s mem,host=A free=10 10s mem,host=A free=20 20s mem,host=A free=123 30s mem,host=A free=43 40s
  • 38. Answer SELECT mean(usage_user) FROM cpu WHERE time > now() - 40s GROUP BY time(20s); SELECT mean(free) FROM mem WHERE time > now() - 40s GROUP BY time(20s)
  • 39. Question Write a query that computes total rate of change across all of the counters grouped by path http_req,host=A,path=/ counter=10 10s http_req,host=A,path=/ counter=11 20s http_req,host=A,path=/ counter=0 30s http_req,host=A,path=/ counter=9 40s http_req,host=A,path=/home counter=20 10s http_req,host=A,path=/home counter=21 20s http_req,host=A,path=/home counter=24 30s http_req,host=A,path=/home counter=25 40s http_req,host=B,path=/admin counter=24 10s http_req,host=B,path=/admin counter=30 20s http_req,host=B,path=/admin counter=34 30s http_req,host=B,path=/admin counter=34 40s http_req,host=B,path=/home counter=20 10s http_req,host=B,path=/home counter=25 20s http_req,host=B,path=/home counter=28 30s http_req,host=B,path=/home counter=30 40s
  • 40. Answer SELECT sum(rate) FROM ( SELECT non_negative_derivative(counter) FROM http_req WHERE <time range> GROUP BY * ) GROUP BY time(10s), path
  • 41. Answer SELECT sum(rate) FROM ( SELECT non_negative_derivative(max(counter)) FROM http_req WHERE <time range> GROUP BY time(10s), * ) GROUP BY time(10s), path
  • 42. Question Write a query that computes the ratio of system load5 to cpu usage_user cpu,host=A usage_user=10 10s cpu,host=A usage_user=20 20s cpu,host=A usage_user=30 30s cpu,host=A usage_user=50 40s sys,host=A load5=10 10s sys,host=A load5=20 20s sys,host=A load5=34 30s sys,host=A load5=43 40s
  • 44. © 2017 InfluxData. All rights reserved.44 © 2018 InfluxData. All rights reserved.44 TICKscript
  • 45. © 2017 InfluxData. All rights reserved.45 What is TICKscript? ● Domain Specific Language for Kapacitor ○ Used to define Kapacitor tasks ● Made for processing streams of data ○ No support for ad-hoc execution ● Data model based off on InfluxDB line protocol ● Great for alerting ○ Expressing complex queries can be difficult and repetitive
  • 46. © 2018 InfluxData. All rights reserved.46 TICK Script • Chain invocation language – | chains together different nodes – . refers to specific attributes on a node • Variables refer to values – Strings – Ints, Floats – Durations – Pipelines var measurement = 'requests' var data = stream |from() .measurement(measurement) |where(lambda: "is_up" == TRUE) |where(lambda: "my_field" > 10) |window() .period(5m) .every(5m) // Count number of points in window data |count('value') .as('the_count') // Compute mean of data window data |mean('value') .as('the_average')
  • 47. © 2018 InfluxData. All rights reserved.47 TICKScript Syntax - Quoting Rules • Double Quotes – References data in lambda expression • Single Quotes – Literal String value // ' means the use the literal string value var measurement = 'requests' var data = stream |from() .measurement(measurement) // " means use the reference value |where(lambda: "is_up" == TRUE) |where(lambda: "my_field" > 10) |window() .period(5m) .every(5m) // ' means to use the literal string value data |count('value') .as('the_count')
  • 48. © 2018 InfluxData. All rights reserved.48 Create a More Interesting Stream TICKscript • Create 5m windows of data that emit every 1m • Compute the average of the field usage_user • Log the result // cpu.tick stream |from() .measurement('cpu') |window() .period(5m) .every(1m) |mean('usage_user') .as('mean_usage_user') |log()
  • 49. © 2018 InfluxData. All rights reserved.49 An even more interesting TICKscript • Filter on the tag cpu=cpu-total • Create 5m windows of data that emit every 1m • Compute the average of the field usage_user • Log the result // cpu.tick stream |from() .measurement('cpu') |where(lambda: "cpu" == 'cpu-total') |window() .period(5m) .every(1m) |mean('usage_user') .as('mean_usage_user') |log()
  • 50. © 2018 InfluxData. All rights reserved.50 Create a Batch TICKscript • Query 5m windows of data every 1m • Compute the average of the field usage_user • Log the result // batch_cpu.tick batch |query(''' SELECT mean("usage_user") AS mean_usage_user FROM "telegraf"."autogen"."cpu" ''') .period(5m) .every(1m) |log()
  • 51. Types of Nodes ● BatchNode ● StreamNode ● AlertNode ● BarrierNode ● ChangeDetect ● CombineNode ● DefaultNode ● DeleteNode ● DerivativeNode ● EC2AutoscaleNode ● EvalNode ● FlattenNode ● FromNode ● GroupByNode ● HTTPOutNode ● HTTPPostNode ● InfluxDBOutNode ● InfluxQLNode ● JoinNode ● K8sAutoscaleNode ● KapacitorLoopback ● LogNode ● NoOpNode ● QueryNode ● SampleNode ● ShiftNode ● SideloadNode ● StateCountNode ● StateDurationNode ● StatsNode ● SwarmAutoscaleNode ● UDFNode ● UnionNode ● WhereNode ● WindowNode
  • 52. © 2017 InfluxData. All rights reserved.52 Batch Stream ● Issues an InfluxQL query to InfluxDB on a schedule ● Yields the resulting data to the rest of the Kapacitor pipeline ● InfluxDB writes data to Kapacitor as it receives writes ● Each individual point is yielded to the Kapacitor pipeline
  • 53. Question Write a query that computes the average of the usage_user and usage_system fields grouped by host in 20 second intervals cpu,host=A usage_user=10,usage_system=70 10s cpu,host=A usage_user=20,usage_system=67 20s cpu,host=A usage_user=30,usage_system=72 30s cpu,host=A usage_user=50,usage_system=74 40s cpu,host=B usage_user=14,usage_system=40 10s cpu,host=B usage_user=43,usage_system=56 20s cpu,host=B usage_user=64,usage_system=77 30s cpu,host=B usage_user=69,usage_system=74 40s cpu,host=C usage_user=10,usage_system=12 10s cpu,host=C usage_user=12,usage_system=23 20s cpu,host=C usage_user=13,usage_system=78 30s cpu,host=C usage_user=15,usage_system=88 40s
  • 54. Answer var data = stream |from() .measurement('cpu') .groupBy('host') |window() .period(20s) .every(20s) data |mean('usage_user') data |mean('usage_system')
  • 55. Not Possible var data = stream |from() .measurement('cpu') |window() .period(20s) .every(20s) data |mean(*) // Cant do this
  • 56. Question Write a query that computes the mean of free field for memory and the mean of the usage_user field for cpu in 20s windows cpu,host=A usage_user=10 10s cpu,host=A usage_user=20 20s cpu,host=A usage_user=30 30s cpu,host=A usage_user=50 40s mem,host=A free=10 10s mem,host=A free=20 20s mem,host=A free=123 30s mem,host=A free=43 40s
  • 58. Question Write a query that computes total rate of change across all of the counters grouped by path http_req,host=A,path=/ counter=10 10s http_req,host=A,path=/ counter=11 20s http_req,host=A,path=/ counter=0 30s http_req,host=A,path=/ counter=9 40s http_req,host=A,path=/home counter=20 10s http_req,host=A,path=/home counter=21 20s http_req,host=A,path=/home counter=24 30s http_req,host=A,path=/home counter=25 40s http_req,host=B,path=/admin counter=24 10s http_req,host=B,path=/admin counter=30 20s http_req,host=B,path=/admin counter=34 30s http_req,host=B,path=/admin counter=34 40s http_req,host=B,path=/home counter=20 10s http_req,host=B,path=/home counter=25 20s http_req,host=B,path=/home counter=28 30s http_req,host=B,path=/home counter=30 40s
  • 60. Question Write a query that computes the ratio of system load5 to cpu usage_user cpu,host=A usage_user=10 10s cpu,host=A usage_user=20 20s cpu,host=A usage_user=30 30s cpu,host=A usage_user=50 40s sys,host=A load5=10 10s sys,host=A load5=20 20s sys,host=A load5=34 30s sys,host=A load5=43 40s
  • 61. Answer var cpu = stream |from() .measurement('cpu') var sys = stream |from() .measurement('sys') cpu |join('sys') .as('cpu', 'sys') |eval(lambda: "cpu.usage_user" / "sys.load5")
  • 62. © 2017 InfluxData. All rights reserved.62 © 2018 InfluxData. All rights reserved.62 Flux
  • 63. © 2017 InfluxData. All rights reserved.63 What is Flux? ● “Data Scripting Language” ○ Idea is to allow for more features than you would expect from a pure query language ● Made for processing streams of data ○ Supports ad-hoc queries ● Rich data model that is distinct from the InfluxDB data model ○ Makes it easier to reason about the internals of what the language is doing to your data
  • 64. Question Write a query that computes the average of the usage_user and usage_system fields grouped by host in 20 second intervals cpu,host=A usage_user=10,usage_system=70 10s cpu,host=A usage_user=20,usage_system=67 20s cpu,host=A usage_user=30,usage_system=72 30s cpu,host=A usage_user=50,usage_system=74 40s cpu,host=B usage_user=14,usage_system=40 10s cpu,host=B usage_user=43,usage_system=56 20s cpu,host=B usage_user=64,usage_system=77 30s cpu,host=B usage_user=69,usage_system=74 40s cpu,host=C usage_user=10,usage_system=12 10s cpu,host=C usage_user=12,usage_system=23 20s cpu,host=C usage_user=13,usage_system=78 30s cpu,host=C usage_user=15,usage_system=88 40s
  • 65. Answer from(bucket: "mybucket") |> range(start: -40s) |> filter(fn: (r) => r._measurement == "cpu") |> filter(fn: (r) => r._field == "usage_user" OR r._field == "usage_system") |> group(by: ["host"]) |> window(period: 20s, every: 20s) |> mean()
  • 66. More General from(bucket: "mybucket") |> range(start: -40s) |> filter(fn: (r) => r._measurement == "cpu") |> group(by: ["host"]) |> window(period: 20s, every: 20s) |> mean()
  • 67. Question Write a query that computes the mean of free field for memory and the mean of the usage_user field for cpu in 20s windows cpu,host=A usage_user=10 10s cpu,host=A usage_user=20 20s cpu,host=A usage_user=30 30s cpu,host=A usage_user=50 40s mem,host=A free=10 10s mem,host=A free=20 20s mem,host=A free=123 30s mem,host=A free=43 40s
  • 68. Answer from(bucket: "mybucket") |> range(start: -40s) |> filter(fn: (r) => r._measurement == "cpu" OR r._measurement == "mem") |> filter(fn: (r) => r._field == "usage_user" OR r._field == "free") |> group(by: ["host"]) |> window(period: 20s, every: 20s) |> mean()
  • 69. Question Write a query that computes total rate of change across all of the counters grouped by path http_req,host=A,path=/ counter=10 10s http_req,host=A,path=/ counter=11 20s http_req,host=A,path=/ counter=0 30s http_req,host=A,path=/ counter=9 40s http_req,host=A,path=/home counter=20 10s http_req,host=A,path=/home counter=21 20s http_req,host=A,path=/home counter=24 30s http_req,host=A,path=/home counter=25 40s http_req,host=B,path=/admin counter=24 10s http_req,host=B,path=/admin counter=30 20s http_req,host=B,path=/admin counter=34 30s http_req,host=B,path=/admin counter=34 40s http_req,host=B,path=/home counter=20 10s http_req,host=B,path=/home counter=25 20s http_req,host=B,path=/home counter=28 30s http_req,host=B,path=/home counter=30 40s
  • 70. Answer from(bucket: "mybucket") |> range(start: -40s) |> filter(fn: (r) => r._measurement == "http_req") |> filter(fn: (r) => r._field == "counter") |> derivative(non_negative: true) |> group(by: ["path"]) |> sum()
  • 71. Making that over time from(bucket: "mybucket") |> range(start: -40s) |> filter(fn: (r) => r._measurement == "http_req") |> filter(fn: (r) => r._field == "counter") |> derivative(non_negative: true) |> group(by: ["path"]) |> window(period: 20s, every:20s) |> sum()
  • 72. Turning it into a function so I never have to think about it again rate =(table=<-, m, field="counter", by, interval) => |> filter(fn: (r) => r._measurement == m) |> filter(fn: (r) => r._field == field) |> derivative(non_negative: true) |> group(by: by) |> window(period: interval, every: interval) |> sum() from(bucket: "mybucket") |> range(start: -40s) |> rate(m: "http_req", by: ["path"], interval: 20s)
  • 73. Question Write a query that computes the ratio of system load5 to cpu usage_user cpu,host=A usage_user=10 10s cpu,host=A usage_user=20 20s cpu,host=A usage_user=30 30s cpu,host=A usage_user=50 40s sys,host=A load5=10 10s sys,host=A load5=20 20s sys,host=A load5=34 30s sys,host=A load5=43 40s
  • 74. Answer mybucket = from(bucket: "mybucket") |> range(start: -40s) cpu = mybucket |> filter(fn: (r) => r._measurement == "cpu") |> filter(fn: (r) => r._field == "usage_user") sys = mybucket |> filter(fn: (r) => r._measurement == "sys") |> filter(fn: (r) => r._field == "load5") join(tables: {cpu: cpu, sys: sys}, on: ["_time"]) |> map(fn: (r) => r._cpu_value / r._sys_value)