Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData

72 views

Published on

This talk introduces the SQL data source for Flux. It will start with examples of using data from MySQL or Postgres with time series data from InfluxDB. It will then go over the details of how the SQL data source was created.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData

  1. 1. Katy Farmer Developer Advocate @thekatertot Using the SQL Datasource in 2.0
  2. 2. InfluxDB . + Flux + SQL (+ you!) Multiple Datasources InfluxDB . + Flux + SQL (+ you!) Multiple Datasources
  3. 3. Why? • Connect your time series data with other types of data for enrichment (and for fun) • Single responsibility databases Why? • Connect your time series data with other types of data for enrichment (and for fun) • Single responsibility databases
  4. 4. © InfluxData. All rights reserved. Industrial IoT Relational Data All sensor metrics (temperature, pressure, water level, Time Series Data Hardware information (model, year, etc.), date deployed, region, id © InfluxData. All rights reserved. Industrial IoT Relational Data All sensor metrics (temperature, pressure, water level, Time Series Data Hardware information (model, year, etc.), date deployed, region, id © InfluxData. All rights reserved. Industrial IoT Relational Data All sensor metrics (temperature, pressure, water level, Time Series Data Hardware information (model, year, etc.), date deployed, region, id
  5. 5. © InfluxData. All rights reserved. User Tracking Relational Data Logins, page visits, clicks, duration on page Time Series Data Name, email, location, history (browsing, purchases, etc.) © InfluxData. All rights reserved. User Tracking Relational Data Logins, page visits, clicks, duration on page Time Series Data Name, email, location, history (browsing, purchases, etc.) © InfluxData. All rights reserved. User Tracking Relational Data Logins, page visits, clicks, duration on page Time Series Data Name, email, location, history (browsing, purchases, etc.)
  6. 6. Example: Robots (Industrial IoT) Each robot has: ❏ ID ❏ name ❏ model ❏ created_at (timestamp) Example: Robots (Industrial IoT) Each robot has: ❏ ID ❏ name ❏ model ❏ created_at (timestamp) Example: Robots (Industrial IoT) Each robot has: ❏ ID ❏ name ❏ model ❏ created_at (timestamp) Example: Robots (Industrial IoT) Each robot has: ❏ ID ❏ name ❏ model ❏ created_at (timestamp)
  7. 7. Example: Robots (Industrial IoT) Robot jobs vary. Some measure parts, some place parts on belts, and others rotate the parts for inspection. But not all robots perform the same. Some of them crash. What is the average CPU usage of each robot? Example: Robots (Industrial IoT) Robot jobs vary. Some measure parts, some place parts on belts, and others rotate the parts for inspection. But not all robots perform the same. Some of them crash. What is the average CPU usage of each robot? Example: Robots (Industrial IoT) Robot jobs vary. Some measure parts, some place parts on belts, and others rotate the parts for inspection. But not all robots perform the same. Some of them crash. What is the average CPU usage of each robot?
  8. 8. © InfluxData. All rights reserved. Step : Importing SQL Data with Flux import "sql" sql.from( driverName: "postgres", dataSourceName: "postgresql://user:password@localhost", query:"SELECT * FROM mydb;")
  9. 9. © InfluxData. All rights reserved. Step : Importing SQL Data with Flux import "sql" sql.from( driverName: "postgres", dataSourceName: "postgresql://localhost/robots?sslmode=disable", query:"SELECT * FROM robots;")
  10. 10. © InfluxData. All rights reserved.© InfluxData. All rights reserved.
  11. 11. © InfluxData. All rights reserved. Step : Write your first Flux query from(bucket: "robots") |> range(start: -10d) |> filter(fn: (r) => r._measurement == "cpu") |> filter(fn: (r) => r._field == "system_usage") |> mean()
  12. 12. © InfluxData. All rights reserved.© InfluxData. All rights reserved.
  13. 13. © InfluxData. All rights reserved. Step : Joining Data Streams import "sql" info = sql.from( driverName: "postgres", dataSourceName: "postgresql://localhost/robots?sslmode=disable", query:"SELECT * FROM robots;") cpu = from(bucket: "robots") |> range(start: -10d) |> filter(fn: (r) => r._measurement == "cpu") |> filter(fn: (r) => r._field == "system_usage") |> mean() join(tables: {cpu:cpu, info:info}, on: ["id"])
  14. 14. © InfluxData. All rights reserved.© InfluxData. All rights reserved.
  15. 15. © InfluxData. All rights reserved. Step : Storing Results sql.to() requirements: ❏ Data in the stream must have same column names as the SQL DB ❏ Drop any columns that won’t be stored ❏ Remember your schema! Any rules (not null, unique, etc.) must be followed or the write will not succeed.
  16. 16. © InfluxData. All rights reserved. import "sql" info = sql.from( driverName: "postgres", dataSourceName: "postgresql://localhost/robots?sslmode=disable", query:"SELECT * FROM robots;") cpu = from(bucket: "robots") |> range(start: -10d) |> filter(fn: (r) => r._measurement == "cpu") |> filter(fn: (r) => r._field == "system_usage") |> mean() join(tables: {cpu:cpu, info:info}, on: ["id"]) |> keep(columns: ["id", "_value", "created_at"]) |> sql.to( driverName: "postgres", dataSourceName: "postgresql://localhost/robots?sslmode=disable", table: "averages")
  17. 17. © InfluxData. All rights reserved. Things I did wrong © InfluxData. All rights reserved. Things I did wrong
  18. 18. © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot © InfluxData. All rights reserved. Questions Thank you! katy@influxdata.com @thekatertot

×