Scott Anderson
Technical Writer @ InfluxData
Map & Reduce
The Powerhouses of
Custom Flux Functions

© 2020 InfluxData. All rights reserved. 2
Disclaimer
This presentation is not about Hadoop MapReduce-like
functionality in Flux. This is about building custom
functions with map() and reduce().

© 2020 InfluxData. All rights reserved. 3
Introduction to Flux
www.influxdata.com/resources/introduction-to-
flux-and-functional-data-scripting/

A quick background
Flux

© 2020 InfluxData. All rights reserved. 5
“
Flux is a functional data scripting language
that lets you query, process, write, and
alert on data in a single syntax.
”

Design Goals
● Turing-complete
● Usable
● Readable
● Flexible
● Extensible
● Testable
● Contributable
● Shareable

Design Goals
● Turing-complete
● Usable
● Readable
● Flexible
● Extensible
● Testable
● Contributable
● Shareable

Design Goals
● Turing-complete
● Usable
● Readable
● Flexible
● Extensible
● Testable
● Contributable
● Shareable

© 2020 InfluxData. All rights reserved. 9
Extensibility
● InfluxQL was and is hard to extend.
● TICKscript was and is easier to extend, but depends on
external languages for true customization.

© 2020 InfluxData. All rights reserved. 10
Extensibility
● Early strategy for Flux has been to create “primitive” functions
that can be used to create other functions.
● Flux recently introduced architecture and guidelines for
community contributions.

If it doesn’t exist, create it
Custom Functions

© 2020 InfluxData. All rights reserved. 12
Custom Function Definition
funcName = (parameters) => funcOperations
Function
square = (x) => x * x
Transformation
somethingCool = (tables=<-) => tables |> ...

© 2020 InfluxData. All rights reserved. 13
Functions & Transformations
Function
bool(v: 1)
// Returns true
Transformation
toBool = (tables=<-) =>
tables
|> map(fn: (r) => ({
r with _value:
bool(v: r._value)
}))

© 2020 InfluxData. All rights reserved. 14
Functions & Transformations
from(bucket: "myData")
|> range(start: -1h)
|> filter(fn: (r) => r._field == "enabled")
|> toBool()

© 2020 InfluxData. All rights reserved. 15
map() & reduce()

Remap rows fer’days
map()

© 2020 InfluxData. All rights reserved. 17
_time sensorID _field _value
2020-06-01T00:12:00Z TM02001 tempC 21.29
2020-06-01T00:12:01Z TM02001 tempC 21.36
2020-06-01T00:12:02Z TM02001 tempC 21.34

© 2020 InfluxData. All rights reserved. 18
_time sensorID _field _value
2020-06-01T00:12:00Z TM02001 tempC 21.29
{
_time: 2020-06-01T00:12:00Z,
sensorID: "TM02001",
_field: "tempC",
_value: 21.29
}

© 2020 InfluxData. All rights reserved. 19
map(
fn:
)
(r) => ({ ... })
)

© 2020 InfluxData. All rights reserved. 20
(r) => ({ ... })
r = {
_time: 2020-06-01T00:12:00Z,
sensorID: "TM02001",
_field: "tempC",
_value: 21.29
}

© 2020 InfluxData. All rights reserved. 21
r = {
_time: 2020-06-01T00:12:00Z,
sensorID: "TM02001",
_field: "tempC",
_value: 21.29
}
(r) => ({ _value: r._value })

© 2020 InfluxData. All rights reserved. 22
r = {
_time: 2020-06-01T00:12:00Z,
sensorID: "TM02001",
_field: "tempC",
_value: 21.29
}
map(fn: (r) => ({
_field: "tempF", _value: (r._value * 9.0 / 5.0) + 32.0
}))

© 2020 InfluxData. All rights reserved. 23
{
_field: "tempF",
_value: 70.322
}
map(fn: (r) => ({
_field: "tempF", _value: (r._value * 9.0 / 5.0) + 32.0
}))
_field _value
tempF 70.322

© 2020 InfluxData. All rights reserved. 24
{
_time: 2020-06-01T00:12:00Z,
sensorID: "TM02001",
_field: "tempF",
_value: 70.322
}
map(fn: (r) => ({ r with
_field: "tempF", _value: (r._value * 9.0 / 5.0) + 32.0
}))

© 2020 InfluxData. All rights reserved. 25
map(fn: (r) => ({ r with
_field: "tempF", _value: (r._value * 9.0 / 5.0) + 32.0
}))
_time sensorID _field _value
2020-06-01T00:12:00Z TM02001 tempF 70.322

© 2020 InfluxData. All rights reserved. 26
|> map(fn: (r) => ({ r with _field: "tempF", _value: (r._value * 9.0 / 5.0) + 32.0 }))
_time sensorID _field _value
2020-06-01T00:12:00Z TM02001 tempC 21.29
2020-06-01T00:12:01Z TM02001 tempC 21.36
2020-06-01T00:12:02Z TM02001 tempC 21.34
_time sensorID _field _value
2020-06-01T00:12:00Z TM02001 tempF 70.322
2020-06-01T00:12:01Z TM02001 tempF 70.448
2020-06-01T00:12:02Z TM02001 tempF 70.412

© 2020 InfluxData. All rights reserved. 27
toF
Create a custom transformation
toF = (tables=<-)toF = (tables=<-) =>
tables
toF = (tables=<-) =>
tables
|> map(
fn: (r) => ({ r with
_field: "tempF",
_value: (r._value * 9.0 / 5.0) + 32.0
})
)

© 2020 InfluxData. All rights reserved. 28
|> map(fn: (r) => ({ r with _field: "tempF", _value: (r._value * 9.0 / 5.0) + 32.0 }))
_time sensorID _field _value
2020-06-01T00:12:00Z TM02001 tempC 21.29
2020-06-01T00:12:01Z TM02001 tempC 21.36
2020-06-01T00:12:02Z TM02001 tempC 21.34
_time sensorID _field _value
2020-06-01T00:12:00Z TM02001 tempF 70.322
2020-06-01T00:12:01Z TM02001 tempF 70.448
2020-06-01T00:12:02Z TM02001 tempF 70.412
|> toF()

© 2020 InfluxData. All rights reserved. 29
Temperature Range Temperature Text
below 0°C freezing
0°C - 15.5°C cold
15.5°C - 26.5°C mild
26.5°C - 37.5°C hot
37.5°C or higher very hot
Add text based on temperature range?

© 2020 InfluxData. All rights reserved. 30
(r) => ({
column: if r._value == "foo" then "bar" else "baz"
})
Use conditional logic!

© 2020 InfluxData. All rights reserved. 31
tempToText = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with
})
)
tempToText = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with
temp_text:
})
)
tempToText = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with
temp_text:
if r._value <= 0.0 then "freezing"
})
)
tempToText = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with
temp_text:
if r._value <= 0.0 then "freezing"
else if r._value > 0.0 and r._value <= 15.5 then "cold"
})
)
tempToText = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with
temp_text:
if r._value <= 0.0 then "freezing"
else if r._value > 0.0 and r._value <= 15.5 then "cold"
else if r._value > 15.5 and r._value <= 26.5 then "mild"
else if r._value > 26.5 and r._value <= 37.5 then "hot"
else if r._value > 37.5 then "very hot"
})
)
tempToText = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with
temp_text:
if r._value <= 0.0 then "freezing"
else if r._value > 0.0 and r._value <= 15.5 then "cold"
else if r._value > 15.5 and r._value <= 26.5 then "mild"
else if r._value > 26.5 and r._value <= 37.5 then "hot"
else if r._value > 37.5 then "very hot"
else "unknown"
})
)

© 2020 InfluxData. All rights reserved. 32
|> tempToText()
_time sensorID _field _value
2020-06-01T00:12:00Z TM02001 tempC -1.2
2020-06-01T00:12:01Z TM02001 tempC 22.1
2020-06-01T00:12:02Z TM02001 tempC 31.8
_time sensorID _field _value temp_text
2020-06-01T00:12:00Z TM02001 tempC -1.2 freezing
2020-06-01T00:12:01Z TM02001 tempC 22.1 mild
2020-06-01T00:12:02Z TM02001 tempC 31.8 hot

Example custom functions
using map()

© 2020 InfluxData. All rights reserved. 34
Basic mathematical operations
multiplyByX = (tables=<-, x) =>
tables
|> map(fn: (r) => ({ r with _value: r._value * x }))
square = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with _value: r._value * r._value }))
quarter = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with _value: r._value / 4.0 }))

© 2020 InfluxData. All rights reserved. 35
Other mathematical operations
import "math"
sin = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with _value: math.sin(x: r._value) }))
circumference = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with circumference: 2.0 * math.pi * r._value }))
circularArea = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with area: math.pi * r._value ^ 2.0 }))

© 2020 InfluxData. All rights reserved. 36
String manipulationimport "strings"
rmHostPrefix = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with
host: strings.trimPrefix(v: r.host, prefix: "host_") }))
replaceSpaces = (tables=<-, char="_") =>
tables
|> map(fn: (r) => ({ r with
_value: strings.replaceAll(v: r._value, t: " ", u: char) }))
humanReadableMessage = (tables=<-) =>
tables
|> map(fn: (r) => ({ r with
humanReadable: "The current value is ${string(v: r._value)}." }))

© 2020 InfluxData. All rights reserved. 37
Common gotchas
● The with operator is your friend.
● Mathematical operands must be of the same data type.
● map() does not add new columns to group keys.
● map() only operates on a single row at a time.
○ Can’t use values from previous or subsequent rows.
○ If your operation requires values from separate fields, use pivot()
or join() to align field values in rows.

Reduce all the things!!
reduce()

© 2020 InfluxData. All rights reserved. 39
|> reduce(...)

© 2020 InfluxData. All rights reserved. 40
_time sensorID _field _value
2020-06-01T00:12:00Z TM02001 tempC 21.29
2020-06-01T00:12:01Z TM02001 tempC 21.36
2020-06-01T00:12:02Z TM02001 tempC 21.34
sensorID _field _value
TM02001 tempC 21.33

© 2020 InfluxData. All rights reserved. 41
reduce(
fn:
identity:
)

© 2020 InfluxData. All rights reserved. 42
reduce(
fn: (r, accumulator) => ({ ... }),
identity:
)

© 2020 InfluxData. All rights reserved. 43
reduce(
fn: (r, accumulator) => ({ ... }),
identity: { ... }
)

© 2020 InfluxData. All rights reserved. 44
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0001, _field: "example", _value: 1.3}
group_key = [ _field ]
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })
accumulator = {sum: 0.0}
identity: {sum: 0.0}

© 2020 InfluxData. All rights reserved. 45
{ sum: r._value + accumulator.sum }
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0001, _field: "example", _value: 1.3}
group_key = [ _field ]
accumulator = {sum: 0.0}
identity: {sum: 0.0}
{ sum: 1.3 + 0.0 }{ sum: 1.3 }
Returns:
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })

© 2020 InfluxData. All rights reserved. 46
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0002, _field: "example", _value: 12.2}
group_key = [ _field ]
accumulator = {sum: 1.3}
identity: {sum: 0.0}
{ sum: 12.2 + 1.3 }{ sum: 13.5 }
Returns:
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })

© 2020 InfluxData. All rights reserved. 47
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0003, _field: "example", _value: 4.9}
group_key = [ _field ]
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })
accumulator = {sum: 13.5}
identity: {sum: 0.0}
{ sum: 18.4 }
Returns:

© 2020 InfluxData. All rights reserved. 48
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0003, _field: "example", _value: 3.2}
group_key = [ _field ]
accumulator = {sum: 18.4}
identity: {sum: 0.0}
{ sum: 21.6 }
Returns:
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })

© 2020 InfluxData. All rights reserved. 49
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0003, _field: "example", _value: 15.8}
group_key = [ _field ]
accumulator = {sum: 21.6}
identity: {sum: 0.0}
{ sum: 37.4 }
Returns:
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })

© 2020 InfluxData. All rights reserved. 50
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
group_key = [ _field ]
Final accumulator:
{ sum: 37.4 }
identity: {sum: 0.0}
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })

© 2020 InfluxData. All rights reserved. 51
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
group_key = [ _field ]
● Final accumulator
● Columns in the group key
● Drops all other columns
identity: {sum: 0.0}
Output table
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })

© 2020 InfluxData. All rights reserved. 52
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
group_key = [ _field ]
identity: {sum: 0.0}
_field sum
example 37.4
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })

© 2020 InfluxData. All rights reserved. 53
_start _stop _time _measurement tag _field _value
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-01T00:00:00Z foo bar example 1.3
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-02T00:00:00Z foo bar example 12.2
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-03T00:00:00Z foo bar example 4.9
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-04T00:00:00Z foo bar example 3.2
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-05T00:00:00Z foo bar example 15.8
group_key = [ _start,_stop,_measurement,tag,_field ]
identity: {sum: 0.0}
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })

© 2020 InfluxData. All rights reserved. 54
_start _stop _measurement tag _field sum
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z foo bar example 37.4
group_key = [ _start,_stop,_measurement,tag,_field ]
identity: {_value: 0.0}
fn: (r, accumulator) => ({ sum: r._value + accumulator.sum })

© 2020 InfluxData. All rights reserved. 55
_start _stop _time _measurement tag _field _value
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-01T00:00:00Z foo bar example 1.3
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-02T00:00:00Z foo bar example 12.2
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-03T00:00:00Z foo bar example 4.9
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-04T00:00:00Z foo bar example 3.2
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-05T00:00:00Z foo bar example 15.8
group_key = [ _start,_stop,_measurement,tag,_field ]
_start _stop _time _measurement tag _field _value
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-01T00:00:00Z foo baz example 8.1
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-02T00:00:00Z foo baz example 12.4
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-03T00:00:00Z foo baz example 13.7
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-04T00:00:00Z foo baz example 4.8
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z 2020-06-05T00:00:00Z foo baz example 6.5

© 2020 InfluxData. All rights reserved. 56
_start _stop _measurement tag _field sum
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z foo bar example 37.4
group_key = [ _start,_stop,_measurement,tag,_field ]
_start _stop _measurement tag _field sum
2020-06-01T00:00:00Z 2020-06-05T23:59:59Z foo baz example 45.5

© 2020 InfluxData. All rights reserved. 57
average

© 2020 InfluxData. All rights reserved. 58
average = (tables=<-) =>
tables
|> reduce(
)

© 2020 InfluxData. All rights reserved. 59
average = (tables=<-) =>
tables
|> reduce(
fn: (r, accumulator) => ({
}),
)

© 2020 InfluxData. All rights reserved. 60
average = (tables=<-) =>
tables
|> reduce(
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
}),
)

© 2020 InfluxData. All rights reserved. 61
average = (tables=<-) =>
tables
|> reduce(
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
}),
)

© 2020 InfluxData. All rights reserved. 62
average = (tables=<-) =>
tables
|> reduce(
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
}),
)

© 2020 InfluxData. All rights reserved. 63
average = (tables=<-) =>
tables
|> reduce(
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
}),
identity: { sum: 0.0, count: 1.0, avg: 0.0 }
)

© 2020 InfluxData. All rights reserved. 64
{
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
}
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0001, _field: "example", _value: 1.3}
group_key = [ _field ]
accumulator = {sum: 0.0, count: 1.0, avg: 0.0}
Returns:
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}

© 2020 InfluxData. All rights reserved. 65
{
sum: 1.3 + 0.0,
count: 1.0 + 1.0,
avg: (1.3 + 0.0) / 1.0
}
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0001, _field: "example", _value: 1.3}
group_key = [ _field ]
accumulator = {sum: 0.0, count: 1.0, avg: 0.0}
Returns:
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}

© 2020 InfluxData. All rights reserved. 66
{
sum: 1.3,
count: 2.0,
avg: 1.3
}
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0001, _field: "example", _value: 1.3}
group_key = [ _field ]
accumulator = {sum: 0.0, count: 1.0, avg: 0.0}
Returns:
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}

© 2020 InfluxData. All rights reserved. 67
{
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
}
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0002, _field: "example", _value: 12.2}
group_key = [ _field ]
accumulator = {sum: 1.3, count: 2.0, avg: 1.3}
Returns:
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}

© 2020 InfluxData. All rights reserved. 68
{
sum: 12.2 + 1.3,
count: 2.0 + 1.0,
avg: (12.2 + 1.3) / 2.0
}
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0002, _field: "example", _value: 12.2}
group_key = [ _field ]
accumulator = {sum: 1.3, count: 2.0, avg: 1.3}
Returns:
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}

© 2020 InfluxData. All rights reserved. 69
{
sum: 14.5,
count: 3.0,
avg: 6.75
}
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0002, _field: "example", _value: 12.2}
group_key = [ _field ]
accumulator = {sum: 1.3, count: 2.0, avg: 1.3}
Returns:
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}

© 2020 InfluxData. All rights reserved. 70
{
sum: 18.4,
count: 4.0,
avg: 6.13
}
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0003, _field: "example", _value: 4.9}
group_key = [ _field ]
accumulator = {sum: 14.5, count: 3.0, avg: 6.75}
Returns:
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}

© 2020 InfluxData. All rights reserved. 71
{
sum: 21.6,
count: 5.0,
avg: 5.4
}
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0004, _field: "example", _value: 3.2}
group_key = [ _field ]
accumulator = {sum: 18.4, count: 4.0, avg: 6.47}
Returns:
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}

© 2020 InfluxData. All rights reserved. 72
{
sum: 37.4,
count: 6.0,
avg: 7.48
}
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
r = {_time: 0005, _field: "example", _value: 15.8}
group_key = [ _field ]
accumulator = {sum: 21.6, count: 5.0, avg: 5.4}
Returns:
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}

© 2020 InfluxData. All rights reserved. 73
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
group_key = [ _field ]
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}
Final accumulator:
{ sum: 37.4, count: 6.0, avg: 7.48 }

© 2020 InfluxData. All rights reserved. 74
_time _field _value
0001 example 1.3
0002 example 12.2
0003 example 4.9
0004 example 3.2
0005 example 15.8
group_key = [ _field ]
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
})
identity: {
sum: 0.0,
count: 1.0,
avg: 0.0
}
_field sum count avg
example 37.4 6.0 7.48

© 2020 InfluxData. All rights reserved. 75
average = (tables=<-) =>
tables
|> reduce(
fn: (r, accumulator) => ({
sum: r._value + accumulator.sum,
count: accumulator.count + 1.0,
avg: (r._value + accumulator.sum) / accumulator.count
}),
identity: { sum: 0.0, count: 1.0, avg: 0.0 }
)
|> drop(columns: ["sum","count"])
|> rename(columns: {avg: "_value"})

© 2020 InfluxData. All rights reserved. 76
_time tag _field _value
0001 foo example 98.3
0002 foo example 89.1
0003 foo example 72.9
0004 foo example 100.6
0005 foo example 210.4
_time tag _field _value
0001 bar example 56.1
0002 bar example 49.3
0003 bar example 67.6
0004 bar example 69.6
0005 bar example 131.4
tag _field _value
foo example 114.26
tag _field _value
bar example 74.8
|> average()

Example custom functions
using reduce()

© 2020 InfluxData. All rights reserved. 78
minMaxMean = (tables=<-) =>
tables
|> reduce(
identity: {count: 0, sum: 0.0, min: 0.0, max: 0.0, mean:0.0},
fn: (r, accumulator) => ({
count: accumulator.count + 1,
sum: r._value + accumulator.sum,
min: if accumulator.count == 0 then r._value
else if r._value < accumulator.min then r._value
else accumulator.min,
max: if accumulator.count == 0 then r._value
else if r._value > accumulator.max then r._value
else accumulator.max,
mean: if accumulator.count == 0 then r._value
else (r._value + accumulator.sum) / float(v: accumulator.count + 1)
})
)
|> drop(columns: ["count", "sum"])

© 2020 InfluxData. All rights reserved. 79
candlestick = (tables=<-) =>
tables
|> reduce(
identity: { index: 0, first: 0.0, last: 0.0, max: 0.0, min: 0.0 },
fn: (r, accumulator) => ({
index: accumulator.index + 1,
first: if accumulator.index == 0 then r._value
else accumulator.first,
last: r._value,
max: if accumulator.index == 0 then r._value
else if r._value > accumulator.max then r._value
else accumulator.max,
min: if accumulator.index == 0 then r._value
else if r._value < accumulator.min then r._value
else accumulator.min
})
)
|> drop(columns: ["index"])

© 2020 InfluxData. All rights reserved. 80
import "strings"
commaSeparatedList = (tables=<-) =>
tables
|> reduce(
identity: {values: ""},
fn: (r, accumulator) => ({
values: accumulator.values + "${string(v: r._value)},"
})
)
|> map(fn: (r) => ({ r with
values: strings.trimRight(v: r.values, cutset: ",")
}))

© 2020 InfluxData. All rights reserved. 81
Common gotchas
● reduce() does not support the with operator.
● Columns not in the group key or explicitly mapped in the
reduce operation are dropped.
● Mathematical operands must be of the same data type.
● reduce() is “destructive” and will only output a single row
per table.

Contribute custom functions to Flux
Contribute & Share

© 2020 InfluxData. All rights reserved. 83
Contribute to Flux
1. Fork and clone the Flux repo – github.com/influxdata/flux
2. Navigate to flux/stdlib/contrib and create a new directory with
your GitHub username.
flux/stdlib/contrib/sanderson
1. Create directories for each sub-package.
flux/stdlib/contrib/sanderson/finance
4. Add a package definition file.
flux/stdlib/contrib/sanderson/finance/finance.flux

© 2020 InfluxData. All rights reserved. 84
package finance

© 2020 InfluxData. All rights reserved. 85
package finance
import "math"
import "regexp"

© 2020 InfluxData. All rights reserved. 86
package finance
import "math"
import "regexp"
option currency = "$"

© 2020 InfluxData. All rights reserved. 87
package finance
import "math"
import "regexp"
option currency = "$"
nyse = ["A","AA","AACG","AAL","AAMC","AAME" ... ]

© 2020 InfluxData. All rights reserved. 88
package finance
import "math"
import "regexp"
option currency = "$"
nyse = ["A","AA","AACG","AAL","AAMC","AAME" ... ]
// Comments that explain what functions do and how they work
candlestick = (tables=<-) => tables |> ...
// Calculate profit and loss
profitAndLoss = (tables=<-) => tables |> ...
// Calculate the days until retirement
retireIn = (tables=<-, goal) => tables |> ...

© 2020 InfluxData. All rights reserved. 89
5. Add tests for each function.
flux/stdlib/contrib/sanderson/finance/retireIn_test.f
lux
Contribute to Flux

© 2020 InfluxData. All rights reserved. 90
package retireIn_test
import "testing"
import "contrib/sanderson/finance"
inData= "...some annotated CSV..."
outData = "...some more annotated CSV..."
t_retireIn = (table=<-) =>
table
|> range(start: 2020-06-01T00:00:00Z, stop: 2020-06-05T00:00:00Z)
|> finance.retireIn(goal: 5000000)
test _retireIn = () => ({
input: testing.loadStorage(csv: inData),
want: testing.loadMem(csv: outData),
fn: t_retireIn
})

© 2020 InfluxData. All rights reserved. 91
6. Generate the standard library.
make or go generate ./stdlib
6. Push your changes to GitHub.
7. Submit a pull request on the Flux repo.
8. Use your contributed package.
Contribute to Flux
5. Add tests for each function.
flux/stdlib/contrib/sanderson/finance/retireIn_test.f
lux

© 2020 InfluxData. All rights reserved. 92
import "contrib/sanderson/finance"
from(bucket: "financials")
|> range(start: -2y)
|> filter(fn: (r) => r._measurement =~ /assets|liabilities|stocks/)
|> finance.retireIn(goal: 5000000)

© 2020 InfluxData. All rights reserved. 93
6. Generate the standard library.
make or go generate ./stdlib
6. Push your changes to GitHub.
7. Submit a pull request on the Flux repo.
8. Use your contributed package.
9. Share your contribution.
Contribute to Flux
5. Add tests for each function.
flux/stdlib/contrib/sanderson/finance/retireIn_test.f
lux

© 2020 InfluxData. All rights reserved. 94
Thank you!
Email: scott@influxdata.com
Github: @sanderson
InfluxData Community: @scott
InfluxDB Community Slack: @Scott Anderson