Successfully reported this slideshow.
Your SlideShare is downloading. ×

Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 69 Ad

More Related Content

Slideshows for you (20)

Similar to Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021 (20)

Advertisement

More from InfluxData (20)

Recently uploaded (20)

Advertisement

Anais Dotis-Georgiou [InfluxData] | Learn Flux by Example | InfluxDays NA 2021

  1. 1. Anais Dotis-Georgiou Developer Advocate, InfluxData Flux by Example
  2. 2. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. About Me ○ Anais Jackie Dotis on LinkedIn ○ @AnaisDotis Developer Advocate, InfluxData
  3. 3. © 2021  InfluxData Inc. All Rights Reserved. https:/ /docs.influxdata.com/influxdb/v2.0/reference/sample-data/
  4. 4. © 2021  InfluxData Inc. All Rights Reserved. https:/ /docs.influxdata.com/influxdb/v2.0/reference/sample-data/ • NOAA water sample dataset import "experimental/csv" csv.from(url: "https://influx-testdata.s3.amazonaws.com/noaa.csv") |> to(bucket: "noaa") • Air sensor sample dataset import "influxdata/influxdb/sample" sample.data(set: "airSensor") |>to(bucket: "airSensor")
  5. 5. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Sample datasets • NOAA water sample dataset The NOAA water sample dataset is static dataset extracted from NOAA Center for Operational Oceanographic Products and Services data. The sample dataset includes 15,258 observations of water levels (ft) collected every six minutes at two stations (Santa Monica, CA (ID 9410840) and Coyote Creek, CA (ID 9414575)) over the period from August 18, 2015 through September 18, 2015. • Air sensor sample dataset Air sensor sample data represents an “Internet of Things” (IoT) use case by simulating temperature, humidity, and carbon monoxide levels for multiple rooms in a building.
  6. 6. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. NOAA water sample dataset schema ● 5 measurement: ○ average_tempearture, h2o_feet, h2o_pH, h2o_quality, h2o_temperature ● 5 fields: ○ degrees, index, level description, pH, water_level ● 2 tags: location, randtag ○ 2 location tag values ■ coyote_creek, santa_monica ○ 3 randtag tag values ■ 1, 2, 3
  7. 7. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Air sensor sample dataset schema ● 1 measurement: airSensors ● 3 fields: ○ co ○ humidity ○ temperature ● 1 tag: sensor_id ○ 8 sensor_id tag values ■ TM0100, TM0101, TM0102, TM0103, TM0200, TM0201, TM0202, TM0203
  8. 8. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Agenda • from() |> range() |> filter() • conditional filtering • group() • aggregateWindow() vs window() • selectors and aggregators • yield() • map() • conditional mapping • pivot() and schema.fieldsAsCol() • findRecord()
  9. 9. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Agenda–the questions we’ll be answering 1. How do I return all temp values greater than 80 for coyote_creek? 2. How do I return all temp values greater than 80 for coyote_creek and all temp values less than 80 for santa_monica? 3. How do I calculate the mean() temp for coyote_creek and santa_monica? 4. How do I calculate the mean() temp across coyote_creek and santa_monica? 5. How do I return the values for 3. and 4. simultaneously? 6. How do I calculate the difference in temp across coyote_creek and santa_monica? 7. How do I calculate the difference between the mean() temp of coyote creek and the temp of coyote creek?
  10. 10. © 2021  InfluxData Inc. All Rights Reserved. 10 © 2021  InfluxData Inc. All Rights Reserved. 1. How do I return all temp values greater than 80 for coyote_creek?
  11. 11. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 1: Understanding from() |> range() |> filter() from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek") |> limit(n:3) |> yield(name: "last 3") where rfc3339startTime, rfc3339stopTime and rfc3339timeN = 2018-10-07T19:36:05.000Z use |> range(start: 0) to filter for all data. use relative durations as well |> range(start: -1y)
  12. 12. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 1: Understanding from() |> range() |> filter() Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table last 3 _measurement _field _value location _start _start _time 0 average_temper ature temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temper ature temperature 73.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temper ature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3
  13. 13. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 2: Filtering on thresholds from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek") |> limit(n:3) |> filter(fn: (r) => r._value > 80.0) |> yield(name: "thresholds")
  14. 14. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Answer 1 Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table thresholds _measurement _field _value location _start _start _time 0 average_temper ature temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temper ature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3
  15. 15. © 2021  InfluxData Inc. All Rights Reserved. 15 © 2021  InfluxData Inc. All Rights Reserved. 2. How do I return all temp values greater than 80 for coyote_creek and all temp values less than 80 for santa_monica?
  16. 16. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 1: Filter for both locations from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> yield(name: "before")
  17. 17. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table before _measurement _field _value location _start _start _time 0 average_temperature temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 73.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3 1 average_temperature temperature 85.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time1 1 average_temperature temperature 74.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time2 1 average_temperature temperature 80.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time3
  18. 18. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 2: Conditional filtering based on the location from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> filter(fn: (r) => if r["location"] == "coyote_creek" then r._value > 80.0 else r._value < 80.0 ) |> yield(name: "conditional filtering")
  19. 19. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table conditional filtering _measurement _field _value location _start _start _time 0 average_temperature temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3 1 average_temperature temperature 74.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time2 Answer 2
  20. 20. © 2021  InfluxData Inc. All Rights Reserved. 20 © 2021  InfluxData Inc. All Rights Reserved. 3. How do I calculate the mean() temp for coyote_creek and santa_monica?
  21. 21. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 1: Apply mean() from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> mean() |> yield(name: "mean")
  22. 22. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key table conditional filtering _measurement _field _value location _start _start 0 average_temperature temperature 80.3333 coyote_creek rfc3339startTime rfc3339stopTime 1 average_temperature temperature 79.6667 santa_monica rfc3339startTime rfc3339stopTime Solution 3
  23. 23. © 2021  InfluxData Inc. All Rights Reserved. 23 © 2021  InfluxData Inc. All Rights Reserved. 4. How do I calculate the mean() temp across coyote_creek and santa_monica?
  24. 24. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 1: group() data from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> group(columns:["_measurement"]) |> yield(name: "group on measurement")
  25. 25. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table before _measurement _field _value location _start _start _time 0 average_temperature temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 73.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3 1 average_temperature temperature 85.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time1 1 average_temperature temperature 74.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time2 1 average_temperature temperature 80.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time3
  26. 26. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key Not In Group Key Not in Group Key Not In Group Key Not In Group Key Not In Group Key Not in Group Key table group on measurement _measurement _field _value location _start _start _time 0 average_temperatur e temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperatur e temperature 73.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperatur e temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3 0 average_temperatur e temperature 85.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperatur e temperature 74.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperatur e temperature 80.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time3
  27. 27. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 2: Apply mean() from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime,) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> group(columns:["_measurement"]) |> mean() |> yield(name: "mean across")
  28. 28. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key Not in Group Key table conditional filtering _measurement _value 0 average_temperature 80 Solution 4
  29. 29. © 2021  InfluxData Inc. All Rights Reserved. 29 © 2021  InfluxData Inc. All Rights Reserved. 4. Bonus Q: How could I retain the other columns?
  30. 30. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Bonus Solution 4A: group() data and include other columns from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> group(columns:["_measurement", "_field", "_start", "_stop"]) |> yield(name: "group on many")
  31. 31. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Bonus Solution 4B: group() data and use the except mode from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> group(columns:["_time", "_value", "location"], mode: "except") |> yield(name: "group except")
  32. 32. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table before _measurement _field _value location _start _start _time 0 average_temperature temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 73.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3 1 average_temperature temperature 85.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time1 1 average_temperature temperature 74.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time2 1 average_temperature temperature 80.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time3
  33. 33. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key Not in Group Key In Group Key In Group Key Not in Group Key table before _measurement _field _value location _start _start _time 0 average_temperature temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 73.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3 0 average_temperature temperature 85.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 74.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperature temperature 80.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time3
  34. 34. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Bonus Solution 4C: group() to ungroup the data from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> group() |> yield(name: "ungroup")
  35. 35. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key Not in Group Key Not in Group Key Not in Group Key Not in Group Key Not in Group Key Not in Group Key Not in Group Key table before _measurement _field _value location _start _start _time 0 average_temperature temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 73.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3 0 average_temperature temperature 85.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 74.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperature temperature 80.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time3
  36. 36. © 2021  InfluxData Inc. All Rights Reserved. 36 © 2021  InfluxData Inc. All Rights Reserved. 5. How do I return the values for 3. and 4. simultaneously?
  37. 37. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Use multiple yield() functions data = from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) data |> mean() |> yield(name: "mean temp for each") data |> group(columns:["_measurement"]) |> mean() |> yield(name: "mean temp across both")
  38. 38. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key Not in Group Key table mean temp across both _measurement _value 0 average_temperature 80 Solution 5 Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key table mean for each _measurement _field _value location _start _start 0 average_temperature temperature 80.3333 coyote_creek rfc3339startTime rfc3339stopTime 1 average_temperature temperature 79.6667 santa_monica rfc3339startTime rfc3339stopTime
  39. 39. © 2021  InfluxData Inc. All Rights Reserved. 39 © 2021  InfluxData Inc. All Rights Reserved. 5. Bonus Q: how do I return the min(), max(), and mean() of the coyote_creek temp?
  40. 40. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Use multiple yield() functions data = from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek") |> limit(n:3) data |> mean() |> yield(name: "mean") data |> max() |> yield(name: "max") data |> min() |> yield(name: "min")
  41. 41. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Bonus Solution 5 Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key table mean _measurement _field _value location _start _start 0 average_temperature temperature 80.3333 coyote_creek rfc3339startTime rfc3339stopTime Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table min _measurement _field _value location _start _start _time 0 average_temperature temperature 73.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table max _measurement _field _value location _start _start _time 0 average_temperature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3
  42. 42. © 2021  InfluxData Inc. All Rights Reserved. 42 © 2021  InfluxData Inc. All Rights Reserved. 6. How do I calculate the difference in temp across coyote_creek and santa_monica?
  43. 43. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Solution 6A: group() on time and difference() from(bucket: "noaa") |> range(start: rfc3339startTime, stop: v.rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> group(columns:["_time”]) |> yield(name: "group on time") |> difference() |> yield(name: "difference")
  44. 44. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key Not in Group Key In Group Key In Group Key In Group Key table group on time _measurement _field _value location _start _start _time 0 average_temperature temperature 82.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 85.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time1 1 average_temperature temperature 73.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 1 average_temperature temperature 74.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time2 2 average_temperature temperature 86.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3 2 average_temperature temperature 80.0 santa_monica rfc3339startTime rfc3339stopTime rfc3339time3
  45. 45. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key Not in Group Key In Group Key In Group Key In Group Key table difference _measurement _field _value location _start _start _time 0 average_temperature temperature -3.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time1 1 average_temperature temperature -1.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time2 2 average_temperature temperature 6.0 coyote_creek rfc3339startTime rfc3339stopTime rfc3339time3
  46. 46. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. What do I do if my timestamps aren’t the same across tags or fields? • window() your data instead of grouping on time • Also it’s more performant! • truncate your timestamps to transform irregular time series into regular ones with date.truncate()
  47. 47. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Solution 6B: window() and difference() from(bucket: "noaa") |> range(start: rfc3339startTime, stop: v.rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> group() |> window(every: 6m) // |> group(columns:["_start"]) |> difference() |> yield(name: "window difference")
  48. 48. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key Not in Group Key In Group Key In Group Key In Group Key table window difference _measurement _field _value location _start _start _time 0 average_temperature temperature -3.0 coyote_creek rfc3339startTime rfc3339startTime + 6m rfc3339time1 1 average_temperature temperature -1.0 coyote_creek rfc3339startTime + 6m rfc3339stopTime + 12m rfc3339time2 2 average_temperature temperature 6.0 coyote_creek rfc3339startTime + 12m rfc3339stopTime + 18m rfc3339time3
  49. 49. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. aggregateWindow() vs window() • aggregateWindow() applies an aggregate or selector function (any function with a column parameter) to fixed windows of time. • The window() function groups records based on a time value. The function calculates time windows and stores window bounds in the _start and _stop columns. _start and _stop values are assigned to rows based on the _time value.
  50. 50. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Solution 6C: pivot() and map() from(bucket: "noaa") |> range(start: rfc3339startTime, stop: v.rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") |> limit(n:3) |> pivot(rowKey:["_time"], columnKey: ["location"], valueColumn: "_value") |> yield(name: "pivot") |> map(fn: (r) => ({ r with difference: r.coyote_creek - r.santa_monica })) |> yield(name: "after map"
  51. 51. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key Not In Group Key In Group Key In Group Key Not in Group Key table pivot _measurement _field coyote_creek santa_monica _start _start _time 0 average_temperature temperature 82.0 85.0 rfc3339startTime rfc3339stopTime rfc3339time1 0 average_temperature temperature 73.0 74.0 rfc3339startTime rfc3339stopTime rfc3339time2 0 average_temperature temperature 86.0 80.0 rfc3339startTime rfc3339stopTime rfc3339time3 Not in Group Key In Group Key Not in Group Key Not In Group Key Not In Group Key In Group Key In Group Key Not in Group Key table pivot _measurement coyote_creek santa_monica difference _start _start _time 0 average_temper ature 82.0 85.0 -3.0 rfc3339startTi me rfc3339stopTime rfc3339time1 0 average_temper ature 73.0 74.0 -1.0 rfc3339startTi me rfc3339stopTime rfc3339time2 0 average_temper ature 86.0 80.0 6.0 rfc3339startTi me rfc3339stopTime rfc3339time3
  52. 52. © 2021  InfluxData Inc. All Rights Reserved. 52 © 2021  InfluxData Inc. All Rights Reserved. 6. Bonus Topic: digging deeper into pivot() and map()
  53. 53. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Using map() to change data types data = from(bucket: "noaa") |> range(start: rfc3339startTime, stop: v.rfc3339stopTime) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek" or r["location"] == "santa_monica") data |> limit(n:3) |> map(fn: (r) => ({ r with int: float(v: r._value})) |> yield(name: "type conversion") Especially useful for when you want to: • perform math across columns with different data types • group data together with different data types
  54. 54. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Conditionally transform values Especially useful for when you want to: • create custom checks • handle null values • handle special mathematical exceptions i.e. when trying to perform transforms like dividing by two values and handling 0 values.
  55. 55. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Assigning a level to your data data |> map(fn: (r) => ({ r with level: if r._value >= 80.0 then "critical" else if r._value < 80.0 and r._value >= 70.0 then "warning" else "normal" }) )
  56. 56. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Handling diving by 0 or null values data |> map(fn: (r) => ({ r with diving: if not exists r.coyote_creek or r.coyote_creek == 0.0 then 0.0 else if not exists r.santa_monica or r.santa_monica == 0.0 then 0.0 else r.coyote_creek / r.coyote_creek }))
  57. 57. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Using pivot() on fields and schema.fieldsAsCol() data = from(bucket: "airsensor") |> range(start: rfc3339startTime, stop: rfc3339stop Time) |> filter(fn: (r) => r["_measurement"] == "airSensors") |> filter(fn: (r) => r["_field"] == "co" or r["_field"] == "humidity") |> filter(fn: (r) => r["sensor_id"] == "TLM0100") |> limit(n:2) |> yield(name: "before") data |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value") |> yield(name: "pivoted") data |> schema.fieldsAsCol() |> yield(name: "fiedsAsCol")
  58. 58. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table before _measurement _field _value sensor_id _start _start _time 0 airSensors co 0.483 TM0100 rfc3339startTime rfc3339stopTime rfc3339time1 0 airSensors co 0.478 TM0100 rfc3339startTime rfc3339stopTime rfc3339time2 Not in Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table pivoted/fieldsAsCol _measurement co sensor_id _start _start _time 0 airSensors 0.483 TM0100 rfc3339startTime rfc3339stopTime rfc3339time1 0 airSensors 0.478 TM0100 rfc3339startTime rfc3339stopTime rfc3339time2
  59. 59. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Using pivot() multiple columns from(bucket: "airsensor") |> range(start: rfc3339startTime, stop: rfc3339stop Time) |> filter(fn: (r) => r["_measurement"] == "airSensors") |> filter(fn: (r) => r["_field"] == "co" or r["_field"] == "humidity") |> filter(fn: (r) => r["sensor_id"] == "TLM0100" r["sensor_id"] == "TLM0101" ) |> pivot(rowKey:["_time"], columnKey: ["_field", "sensor_id"], valueColumn: "_value") |> yield(name: "mult col pivot")
  60. 60. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table before _measurement _field _value sensor_id _start _start _time 0 airSensors co 0.483 TM0100 rfc3339startTime rfc3339stopTime rfc3339time1 1 airSensors co 0.478 TM0101 rfc3339startTime rfc3339stopTime rfc3339time1 2 airSensors humidity 35.12 TM0100 rfc3339startTime rfc3339stopTime rfc3339time1 3 airSensors humidity 34.86 TM0101 rfc3339startTime rfc3339stopTime rfc3339time1 Not in Group Key In Group Key Not in Group Key Not in Group Key Not in Group Key Not in Group Key In Group Key In Group Key In Group Key Not in Group Key table mult col pivot _measure ment co_TM010 0 co_TM010 1 humidty_T M0100 humidty_TM0 101 sensor_id _start _start _time 0 airSensors 0.483 0.478 35.12 34.86 TM0100 rfc3339startTime rfc3339stopTime rfc3339time1
  61. 61. © 2021  InfluxData Inc. All Rights Reserved. 61 © 2021  InfluxData Inc. All Rights Reserved. 7. How do I calculate the difference between the mean() temp of coyote creek and the temp of coyote creek?
  62. 62. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 1: use findRecord() data = from(bucket: "noaa") |> range(start: rfc3339startTime, stop: rfc3339stopTime,) |> filter(fn: (r) => r["_measurement"] == "average_temperature") |> filter(fn: (r) => r["_field"] == "degrees") |> filter(fn: (r) => r["location"] == "coyote_creek") |> limit(n:3) mean_coyote = data |> mean() |> findRecord(fn: (key) => true, idx: 0) // |> findRecord(fn: (key) => key.location == "santa_monica", idx: 0) // where mean_coyote._value = 80.3
  63. 63. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Step 2: apply map() data |> map(fn: (r) => ({ r with difference: r._value - mean_coyote._value })) |> yield( name: "diff mean and coyote creek") import "array" array.from(rows: [{_time: now(), _value: float(v: mean_coyote._value)}]) Bonus: use array.from() to check mean value
  64. 64. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Not in Group Key In Group Key In Group Key Not in Group Key Not In Group Key In Group Key In Group Key Not in Group Key table diff mean and coyote creek _measurement _field _value difference _start _start _time 0 average_tempera ture temperature 82.0 1.667 rfc3339startTime rfc3339stopTime rfc3339time1 0 average_tempera ture temperature 73.0 -7.333 rfc3339startTime rfc3339stopTime rfc3339time2 0 average_tempera ture temperature 86.0 5.667 rfc3339startTime rfc3339stopTime rfc3339time3 Not in Group Key Not in Group Key Not in Group Key table _result _value _time 0 80.333 rfc3339timeNow
  65. 65. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Pushdown Patterns from() |> range() |> filter() Bare Aggregators and Selectors |> count() |> sum() |> first() |> last() |> min() |> max() |> mean() Group + Bare Aggregates and selectors |>group() |> count() |>group() |> sum() etc... Window + Bare aggregates and selectors |>window() |> count() |>window() |> sum() etc.. Window + Bare aggregates and selectors |>aggregateWindow(fn: count) |>aggregateWindow(fn: sum) etc...
  66. 66. © 2021  InfluxData Inc. All Rights Reserved. © 2021  InfluxData Inc. All Rights Reserved. Flux Profiler
  67. 67. © 2021  InfluxData Inc. All Rights Reserved. Resources ● Top 5 Hurdles for Flux Beginners and Resources for Learning to Use Flux ● Top 5 Hurdles for Intermediate Flux Users and Resources for Optimizing Flux ● TL;DR InfluxDB Tech Tips – Optimizing Flux Performance in InfluxDB Cloud
  68. 68. © 2021  InfluxData Inc. All Rights Reserved. Questions?
  69. 69. © 2021  InfluxData Inc. All Rights Reserved. Thank You

×