SlideShare a Scribd company logo
InfluxDB Tasks
Overview
Balaji Palani - Senior Director of Product Management,
InfluxData
Connect Learn Build
Hear from and meet developers
from the InfluxDB Community
Be inspired by use cases from
our partners and InfluxDB engineers
Learn best practices that will
help you build great experiences
for your projects
An InfluxDB task is a scheduled Flux script that takes a
stream of input data, modifies or analyzes it in some way.
This session breaks down how to use tasks, introduces
invokable scripts, and looks at the future of tasks.
Balaji Palani
Senior Director of Product
Management, InfluxData
Balaji Palani is the Director of Product Management focused on
InfluxDB Cloud. He is passionate about building powerful cloud
products that help Developers achieve the fastest time to awesome.
And with InfluxDB Cloud, his customers are able to collect and utilize
time series data to hit even the toughest SLAs. Previous to InfluxData,
Balaji has held several Product Management and Engineering
positions at companies like BMC, HP, and Mercury. Balaji holds an MS
degree in Computer Science from West Virginia University and a BS in
Electrical Engineering from Annamalai University.
InfluxDB Tasks Overview
Agenda
1. What Are Tasks?
2. Invokable Scripts
3. The Future of Tasks
What Are Tasks?
Tasks
(Automate your transformations)
Raw Data Transformed
or
Downsampled
Data
Tasks
(Automate your transformations)
Raw Data Transformed
or
Downsampled
Data
Flux
1. Query your raw data:
○ Bucket
○ Time range
○ Filters
Task structure
from(bucket: "raw_data")
|> range(start: -15m, stop: now())
|> filter(fn: (r) => (r._measurement
== "airSensors"))
|> filter(fn: (r) => (r._field ==
"temperature"))
Flux
1. Query your raw data:
○ Bucket
○ Time range
○ Filters
Task structure
from(bucket: "raw_data")
|> range(start: -15m, stop: now())
|> filter(fn: (r) => (r._measurement
== "airSensors"))
|> filter(fn: (r) => (r._field ==
"temperature"))
2. Aggregate the data
○ Use Flux math functions
○ Build your custom logic
Task structure
from(bucket: "raw_data")
|> range(start: -15m, stop: now())
|> filter(fn: (r) => (r._measurement
== "airSensors"))
|> filter(fn: (r) => (r._field ==
"temperature"))
|> mean(column: "_value")
Flux
3. Write the results to
downsampled bucket
Task structure
from(bucket: "raw_data")
|> range(start: -15m, stop: now())
|> filter(fn: (r) => (r._measurement
== "airSensors"))
|> filter(fn: (r) => (r._field ==
"temperature"))
|> mean(column: "_value")
|>rename (columns: {_stop:”_time”})
|> to(bucket: "downsampled_data")
Flux
4. Setup your task options:
■ Name of the task
■ Frequency
Task structure
option task = {name: “agg_1h_sum”,
every: 15m, offset: 30s}
from(bucket: "raw_data")
|> range(start: -15m, stop: now())
|> filter(fn: (r) => (r._measurement
== "airSensors"))
|> filter(fn: (r) => (r._field ==
"temperature"))
|> mean(column: "_value")
|>rename (columns: {_stop:”_time”})
|> to(bucket: "downsampled_data")
Flux
Flux has a vast array of
transformation functions to
build any custom task
An example of a custom downsampling task
option task = {name: "transform_10s_power",
every: 10s}
offset = 1h
step = 10s
ta1 = from(bucket: "raw_bucket_1")
|> range(start: -offset)
|> filter(fn: (r) => (r["_field"] == "power"))
|> aggregateWindow(every: step, fn: mean)
|> pivot(rowKey: ["_time"], columnKey:
["_field"], valueColumn: "_value")
ta2 = from(bucket: "raw_bucket_2)
|> range(start: -offset)
|> filter(fn: (r) => (r["_field"] ==
"parsed_value”))
|> aggregateWindow(every: step, fn: mean)
|> pivot(rowKey: ["_time"], columnKey:
["_field"], valueColumn: "_value")
join(tables: {t1: ta1, t2: ta2}, on: ["_time"])
|> to(bucket: "agg_bucket")
Flux
Flux has a vast array of
transformation functions to
build any custom task
An example of a custom downsampling task
option task = {name: "transform_10s_power",
every: 10s}
offset = 1h
step = 10s
ta1 = from(bucket: "raw_bucket_1")
|> range(start: -offset)
|> filter(fn: (r) => (r["_field"] == "power"))
|> aggregateWindow(every: step, fn: mean)
|> pivot(rowKey: ["_time"], columnKey:
["_field"], valueColumn: "_value")
ta2 = from(bucket: "raw_bucket_2)
|> range(start: -offset)
|> filter(fn: (r) => (r["_field"] ==
"parsed_value”))
|> aggregateWindow(every: step, fn: mean)
|> pivot(rowKey: ["_time"], columnKey:
["_field"], valueColumn: "_value")
join(tables: {t1: ta1, t2: ta2}, on: ["_time"])
|> to(bucket: "agg_bucket")
Flux
Alert Checks
(Built on the Tasks subsystem)
Monitored
Data
_monitoring
(System
Bucket)
monitoring()
An example of a deadman check task
Flux
import "influxdata/influxdb/monitor"
import "experimental"
import "influxdata/influxdb/v1"
data = from(bucket: "idping"
|> range(start: -10m)
|> filter(fn: (r) => (r["_measurement"] == "ctr"))
|> filter(fn: (r) => (r["_field"] == "n")))
option task = {name: "idping Deadman", every: 1m, offset: 0s}
check = {
_check_id: "CHECK_ID",
_check_name: "CHECK_NAME",
_type: "deadman",
tags: {idping: "deadman"}
}
crit = (r) => (r["dead"])
messageFn = (r) => ("Check: ${r._check_name} is: ${r._level}")
data
|> v1["fieldsAsCols"]()
|> monitor["deadman"](t: experimental["subDuration"](from: now(), d: 5m))
|> monitor["check"](data: check, messageFn: messageFn, crit: crit)
An example of a deadman check task
Flux
import "influxdata/influxdb/monitor"
import "experimental"
import "influxdata/influxdb/v1"
data = from(bucket: "idping"
|> range(start: -10m)
|> filter(fn: (r) => (r["_measurement"] == "ctr"))
|> filter(fn: (r) => (r["_field"] == "n")))
option task = {name: "idping Deadman", every: 1m, offset: 0s}
check = {
_check_id: "CHECK_ID",
_check_name: "CHECK_NAME",
_type: "deadman",
tags: {idping: "deadman"}
}
crit = (r) => (r["dead"])
messageFn = (r) => ("Check: ${r._check_name} is: ${r._level}")
data
|> v1["fieldsAsCols"]()
|> monitor["deadman"](t: experimental["subDuration"](from: now(), d: 5m))
|> monitor["check"](data: check, messageFn: messageFn, crit: crit)
Notifications
(Built on the Tasks subsystem)
Call
Notification
Endpoint
(e.g.
http.post)
_monitoring
(System
Bucket)
monitoring()
Notifications
(Built on the Tasks subsystem)
Call
Notification
Endpoint
(e.g.
http.post)
_monitoring
(System
Bucket)
monitoring()
An example of a notification task
import "influxdata/influxdb/monitor"
import "slack"
import "influxdata/influxdb/secrets"
import "experimental"
option task = {name: "Reads Deadman Notification", every: 1m, offset: 0s}
slack_endpoint = slack["endpoint"](url: "https://hooks.slack.com/services/SGSDFGER/HW36BWGDFEY/Slack_Token")
notification = {
_notification_rule_id: "NOTIF_RULE_ID",
_notification_rule_name: "NOTIF RULE NAME",
_notification_endpoint_id: "NOTIF_ENDPOINT",
_notification_endpoint_name: "ENDPOINT_NAME",
}
statuses = monitor["from"](start: -2m, fn: (r) => (r["reads"] == "deadman"))
crit = statuses
|> filter(fn: (r) => (r["_level"] == "crit"))
|> filter(fn: (r) => (r["_time"] >= experimental[
"subDuration"](from: now(), d: 1m)))
crit
|> monitor["notify"](data: notification, endpoint: slack_endpoint(mapFn: (r) =>
({channel: "", text: "Notification Rule: ${r._notification_rule_name} triggered by check:
${r._check_name}: ${r._message}",
color: if r["_level"] == "crit" then "danger" else "good"})))
Flux
Tasks and Invokable Scripts
API Invokable Script
App / Platform
/scripts/aggregate_1h_sum
Raw Data Transformed
Data
AWS Lambda
Node-Red
Web Connector
Azure Functions
API Invokable Script
App / Platform
/scripts/aggregate_1h_sum
Raw Data Transformed
Data
AWS Lambda
Node-Red
Web Connector
Azure Functions
Tasks leveraging Scripts
/scripts/aggregate_1h_sum
Raw Data Downsampled
Data
Developer value:
• Code reusability and
shareability
• Separation of roles
• Drastically improves
ease of use
Pass param
values
The Future of Tasks
Vision: Automate at scale
Schedule
Remote Invoke
Task Run
Flux
Script Language
Python
Javascript
Destination
Notification
Endpoint
Bucket
External
Datastore
Vision: Automate at scale
Schedule
Remote Invoke
Task Run
Flux
Script Language
Python
Javascript
Destination
Notification
Endpoint
Bucket
External
Datastore
Additional Resources
Free InfluxDB: OSS or Cloud - influxdata.com/cloud
Forums: community.influxdata.com
Slack: influxcommunity.slack.com
Reddit: r/InfluxData
Influx Community (GH): github.com/InfluxCommunity
Book: awesome.influxdata.com
Docs: docs.influxdata.com
Blogs: influxdata.com/blog
InfluxDB University: influxdata.com/university
How to guides: docs.influxdata.com/resources/how-to-guides/
T H A N K Y O U

More Related Content

What's hot

Building Grails applications with PostgreSQL
Building Grails applications with PostgreSQLBuilding Grails applications with PostgreSQL
Building Grails applications with PostgreSQL
Command Prompt., Inc
 
Web Development with Python and Django
Web Development with Python and DjangoWeb Development with Python and Django
Web Development with Python and Django
Michael Pirnat
 
Defending broken access control in .NET
Defending broken access control in .NETDefending broken access control in .NET
Defending broken access control in .NET
Supriya G
 
하둡 고가용성(HA) 설정
하둡 고가용성(HA) 설정하둡 고가용성(HA) 설정
하둡 고가용성(HA) 설정
NoahKIM36
 
Docker 활용법: dumpdocker
Docker 활용법: dumpdockerDocker 활용법: dumpdocker
Docker 활용법: dumpdocker
Jaehwa Park
 
Performance Analysis: The USE Method
Performance Analysis: The USE MethodPerformance Analysis: The USE Method
Performance Analysis: The USE Method
Brendan Gregg
 
Uncomplicated Nomad
Uncomplicated NomadUncomplicated Nomad
Uncomplicated Nomad
Bram Vogelaar
 
Redux Saga - Under the hood
Redux Saga - Under the hoodRedux Saga - Under the hood
Redux Saga - Under the hood
Waqqas Jabbar
 
Rediscovering Spring with Spring Boot(1)
Rediscovering Spring with Spring Boot(1)Rediscovering Spring with Spring Boot(1)
Rediscovering Spring with Spring Boot(1)
Gunith Devasurendra
 
Jmeter Performance Testing
Jmeter Performance TestingJmeter Performance Testing
Jmeter Performance Testing
Atul Pant
 
Spring Boot Tutorial
Spring Boot TutorialSpring Boot Tutorial
Spring Boot Tutorial
Naphachara Rattanawilai
 
Advanced mainframe hacking
Advanced mainframe hackingAdvanced mainframe hacking
Advanced mainframe hacking
Philip Young
 
Grails Spring Boot
Grails Spring BootGrails Spring Boot
Grails Spring Boot
TO THE NEW | Technology
 
Advanced Django ORM techniques
Advanced Django ORM techniquesAdvanced Django ORM techniques
Advanced Django ORM techniques
Daniel Roseman
 
Sinatra Rack And Middleware
Sinatra Rack And MiddlewareSinatra Rack And Middleware
Sinatra Rack And Middleware
Ben Schwarz
 
AV Evasion with the Veil Framework
AV Evasion with the Veil FrameworkAV Evasion with the Veil Framework
AV Evasion with the Veil Framework
VeilFramework
 
스프링 시큐리티 구조 이해
스프링 시큐리티 구조 이해스프링 시큐리티 구조 이해
스프링 시큐리티 구조 이해
beom kyun choi
 
Spring Batch Avance
Spring Batch AvanceSpring Batch Avance
Spring Batch Avance
Olivier BAZOUD
 
Logging with Elasticsearch, Logstash & Kibana
Logging with Elasticsearch, Logstash & KibanaLogging with Elasticsearch, Logstash & Kibana
Logging with Elasticsearch, Logstash & Kibana
Amazee Labs
 
Ansible
AnsibleAnsible
Ansible
Raul Leite
 

What's hot (20)

Building Grails applications with PostgreSQL
Building Grails applications with PostgreSQLBuilding Grails applications with PostgreSQL
Building Grails applications with PostgreSQL
 
Web Development with Python and Django
Web Development with Python and DjangoWeb Development with Python and Django
Web Development with Python and Django
 
Defending broken access control in .NET
Defending broken access control in .NETDefending broken access control in .NET
Defending broken access control in .NET
 
하둡 고가용성(HA) 설정
하둡 고가용성(HA) 설정하둡 고가용성(HA) 설정
하둡 고가용성(HA) 설정
 
Docker 활용법: dumpdocker
Docker 활용법: dumpdockerDocker 활용법: dumpdocker
Docker 활용법: dumpdocker
 
Performance Analysis: The USE Method
Performance Analysis: The USE MethodPerformance Analysis: The USE Method
Performance Analysis: The USE Method
 
Uncomplicated Nomad
Uncomplicated NomadUncomplicated Nomad
Uncomplicated Nomad
 
Redux Saga - Under the hood
Redux Saga - Under the hoodRedux Saga - Under the hood
Redux Saga - Under the hood
 
Rediscovering Spring with Spring Boot(1)
Rediscovering Spring with Spring Boot(1)Rediscovering Spring with Spring Boot(1)
Rediscovering Spring with Spring Boot(1)
 
Jmeter Performance Testing
Jmeter Performance TestingJmeter Performance Testing
Jmeter Performance Testing
 
Spring Boot Tutorial
Spring Boot TutorialSpring Boot Tutorial
Spring Boot Tutorial
 
Advanced mainframe hacking
Advanced mainframe hackingAdvanced mainframe hacking
Advanced mainframe hacking
 
Grails Spring Boot
Grails Spring BootGrails Spring Boot
Grails Spring Boot
 
Advanced Django ORM techniques
Advanced Django ORM techniquesAdvanced Django ORM techniques
Advanced Django ORM techniques
 
Sinatra Rack And Middleware
Sinatra Rack And MiddlewareSinatra Rack And Middleware
Sinatra Rack And Middleware
 
AV Evasion with the Veil Framework
AV Evasion with the Veil FrameworkAV Evasion with the Veil Framework
AV Evasion with the Veil Framework
 
스프링 시큐리티 구조 이해
스프링 시큐리티 구조 이해스프링 시큐리티 구조 이해
스프링 시큐리티 구조 이해
 
Spring Batch Avance
Spring Batch AvanceSpring Batch Avance
Spring Batch Avance
 
Logging with Elasticsearch, Logstash & Kibana
Logging with Elasticsearch, Logstash & KibanaLogging with Elasticsearch, Logstash & Kibana
Logging with Elasticsearch, Logstash & Kibana
 
Ansible
AnsibleAnsible
Ansible
 

Similar to Balaji Palani [InfluxData] | InfluxDB Tasks Overview | InfluxDays 2022

Scott Anderson [InfluxData] | InfluxDB Tasks – Beyond Downsampling | InfluxDa...
Scott Anderson [InfluxData] | InfluxDB Tasks – Beyond Downsampling | InfluxDa...Scott Anderson [InfluxData] | InfluxDB Tasks – Beyond Downsampling | InfluxDa...
Scott Anderson [InfluxData] | InfluxDB Tasks – Beyond Downsampling | InfluxDa...
InfluxData
 
InfluxData Platform Future and Vision
InfluxData Platform Future and VisionInfluxData Platform Future and Vision
InfluxData Platform Future and Vision
InfluxData
 
9:40 am InfluxDB 2.0 and Flux – The Road Ahead Paul Dix, Founder and CTO | ...
 9:40 am InfluxDB 2.0 and Flux – The Road Ahead  Paul Dix, Founder and CTO | ... 9:40 am InfluxDB 2.0 and Flux – The Road Ahead  Paul Dix, Founder and CTO | ...
9:40 am InfluxDB 2.0 and Flux – The Road Ahead Paul Dix, Founder and CTO | ...
InfluxData
 
Monitoring Your ISP Using InfluxDB Cloud and Raspberry Pi
Monitoring Your ISP Using InfluxDB Cloud and Raspberry PiMonitoring Your ISP Using InfluxDB Cloud and Raspberry Pi
Monitoring Your ISP Using InfluxDB Cloud and Raspberry Pi
InfluxData
 
Streaming Way to Webscale: How We Scale Bitly via Streaming
Streaming Way to Webscale: How We Scale Bitly via StreamingStreaming Way to Webscale: How We Scale Bitly via Streaming
Streaming Way to Webscale: How We Scale Bitly via Streaming
All Things Open
 
Flask patterns
Flask patternsFlask patterns
Flask patterns
it-people
 
Flux and InfluxDB 2.0 by Paul Dix
Flux and InfluxDB 2.0 by Paul DixFlux and InfluxDB 2.0 by Paul Dix
Flux and InfluxDB 2.0 by Paul Dix
InfluxData
 
Background Jobs - Com BackgrounDRb
Background Jobs - Com BackgrounDRbBackground Jobs - Com BackgrounDRb
Background Jobs - Com BackgrounDRb
Juan Maiz
 
112 portfpres.pdf
112 portfpres.pdf112 portfpres.pdf
112 portfpres.pdf
sash236
 
Time Series Analysis for Network Secruity
Time Series Analysis for Network SecruityTime Series Analysis for Network Secruity
Time Series Analysis for Network Secruity
mrphilroth
 
Emerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the HorizonEmerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the Horizon
Alex Payne
 
Monitoring InfluxEnterprise
Monitoring InfluxEnterpriseMonitoring InfluxEnterprise
Monitoring InfluxEnterprise
InfluxData
 
Advanced kapacitor
Advanced kapacitorAdvanced kapacitor
Advanced kapacitor
InfluxData
 
Deep Dive Into Swift
Deep Dive Into SwiftDeep Dive Into Swift
Deep Dive Into Swift
Sarath C
 
Tools for Making Machine Learning more Reactive
Tools for Making Machine Learning more ReactiveTools for Making Machine Learning more Reactive
Tools for Making Machine Learning more Reactive
Jeff Smith
 
Router Queue Simulation in C++ in MMNN and MM1 conditions
Router Queue Simulation in C++ in MMNN and MM1 conditionsRouter Queue Simulation in C++ in MMNN and MM1 conditions
Router Queue Simulation in C++ in MMNN and MM1 conditions
Morteza Mahdilar
 
Python Manuel-R2021.pdf
Python Manuel-R2021.pdfPython Manuel-R2021.pdf
Python Manuel-R2021.pdf
RamprakashSingaravel1
 
How I Built a Power Debugger Out of the Standard Library and Things I Found o...
How I Built a Power Debugger Out of the Standard Library and Things I Found o...How I Built a Power Debugger Out of the Standard Library and Things I Found o...
How I Built a Power Debugger Out of the Standard Library and Things I Found o...
doughellmann
 
"Angular.js Concepts in Depth" by Aleksandar Simović
"Angular.js Concepts in Depth" by Aleksandar Simović"Angular.js Concepts in Depth" by Aleksandar Simović
"Angular.js Concepts in Depth" by Aleksandar Simović
JS Belgrade
 
WattGo: Analyses temps-réél de series temporelles avec Spark et Solr (Français)
WattGo: Analyses temps-réél de series temporelles avec Spark et Solr (Français)WattGo: Analyses temps-réél de series temporelles avec Spark et Solr (Français)
WattGo: Analyses temps-réél de series temporelles avec Spark et Solr (Français)
DataStax Academy
 

Similar to Balaji Palani [InfluxData] | InfluxDB Tasks Overview | InfluxDays 2022 (20)

Scott Anderson [InfluxData] | InfluxDB Tasks – Beyond Downsampling | InfluxDa...
Scott Anderson [InfluxData] | InfluxDB Tasks – Beyond Downsampling | InfluxDa...Scott Anderson [InfluxData] | InfluxDB Tasks – Beyond Downsampling | InfluxDa...
Scott Anderson [InfluxData] | InfluxDB Tasks – Beyond Downsampling | InfluxDa...
 
InfluxData Platform Future and Vision
InfluxData Platform Future and VisionInfluxData Platform Future and Vision
InfluxData Platform Future and Vision
 
9:40 am InfluxDB 2.0 and Flux – The Road Ahead Paul Dix, Founder and CTO | ...
 9:40 am InfluxDB 2.0 and Flux – The Road Ahead  Paul Dix, Founder and CTO | ... 9:40 am InfluxDB 2.0 and Flux – The Road Ahead  Paul Dix, Founder and CTO | ...
9:40 am InfluxDB 2.0 and Flux – The Road Ahead Paul Dix, Founder and CTO | ...
 
Monitoring Your ISP Using InfluxDB Cloud and Raspberry Pi
Monitoring Your ISP Using InfluxDB Cloud and Raspberry PiMonitoring Your ISP Using InfluxDB Cloud and Raspberry Pi
Monitoring Your ISP Using InfluxDB Cloud and Raspberry Pi
 
Streaming Way to Webscale: How We Scale Bitly via Streaming
Streaming Way to Webscale: How We Scale Bitly via StreamingStreaming Way to Webscale: How We Scale Bitly via Streaming
Streaming Way to Webscale: How We Scale Bitly via Streaming
 
Flask patterns
Flask patternsFlask patterns
Flask patterns
 
Flux and InfluxDB 2.0 by Paul Dix
Flux and InfluxDB 2.0 by Paul DixFlux and InfluxDB 2.0 by Paul Dix
Flux and InfluxDB 2.0 by Paul Dix
 
Background Jobs - Com BackgrounDRb
Background Jobs - Com BackgrounDRbBackground Jobs - Com BackgrounDRb
Background Jobs - Com BackgrounDRb
 
112 portfpres.pdf
112 portfpres.pdf112 portfpres.pdf
112 portfpres.pdf
 
Time Series Analysis for Network Secruity
Time Series Analysis for Network SecruityTime Series Analysis for Network Secruity
Time Series Analysis for Network Secruity
 
Emerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the HorizonEmerging Languages: A Tour of the Horizon
Emerging Languages: A Tour of the Horizon
 
Monitoring InfluxEnterprise
Monitoring InfluxEnterpriseMonitoring InfluxEnterprise
Monitoring InfluxEnterprise
 
Advanced kapacitor
Advanced kapacitorAdvanced kapacitor
Advanced kapacitor
 
Deep Dive Into Swift
Deep Dive Into SwiftDeep Dive Into Swift
Deep Dive Into Swift
 
Tools for Making Machine Learning more Reactive
Tools for Making Machine Learning more ReactiveTools for Making Machine Learning more Reactive
Tools for Making Machine Learning more Reactive
 
Router Queue Simulation in C++ in MMNN and MM1 conditions
Router Queue Simulation in C++ in MMNN and MM1 conditionsRouter Queue Simulation in C++ in MMNN and MM1 conditions
Router Queue Simulation in C++ in MMNN and MM1 conditions
 
Python Manuel-R2021.pdf
Python Manuel-R2021.pdfPython Manuel-R2021.pdf
Python Manuel-R2021.pdf
 
How I Built a Power Debugger Out of the Standard Library and Things I Found o...
How I Built a Power Debugger Out of the Standard Library and Things I Found o...How I Built a Power Debugger Out of the Standard Library and Things I Found o...
How I Built a Power Debugger Out of the Standard Library and Things I Found o...
 
"Angular.js Concepts in Depth" by Aleksandar Simović
"Angular.js Concepts in Depth" by Aleksandar Simović"Angular.js Concepts in Depth" by Aleksandar Simović
"Angular.js Concepts in Depth" by Aleksandar Simović
 
WattGo: Analyses temps-réél de series temporelles avec Spark et Solr (Français)
WattGo: Analyses temps-réél de series temporelles avec Spark et Solr (Français)WattGo: Analyses temps-réél de series temporelles avec Spark et Solr (Français)
WattGo: Analyses temps-réél de series temporelles avec Spark et Solr (Français)
 

More from InfluxData

Announcing InfluxDB Clustered
Announcing InfluxDB ClusteredAnnouncing InfluxDB Clustered
Announcing InfluxDB Clustered
InfluxData
 
Best Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow EcosystemBest Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow Ecosystem
InfluxData
 
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
InfluxData
 
Power Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDBPower Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDB
InfluxData
 
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
InfluxData
 
Build an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING StackBuild an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING Stack
InfluxData
 
Meet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using RustMeet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using Rust
InfluxData
 
Introducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud DedicatedIntroducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud Dedicated
InfluxData
 
Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB
InfluxData
 
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
InfluxData
 
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
InfluxData
 
Introducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage EngineIntroducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage Engine
InfluxData
 
Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena
InfluxData
 
Understanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage EngineUnderstanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage Engine
InfluxData
 
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDBStreamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
InfluxData
 
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
InfluxData
 
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
InfluxData
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
InfluxData
 
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
InfluxData
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
InfluxData
 

More from InfluxData (20)

Announcing InfluxDB Clustered
Announcing InfluxDB ClusteredAnnouncing InfluxDB Clustered
Announcing InfluxDB Clustered
 
Best Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow EcosystemBest Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow Ecosystem
 
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
 
Power Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDBPower Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDB
 
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
 
Build an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING StackBuild an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING Stack
 
Meet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using RustMeet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using Rust
 
Introducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud DedicatedIntroducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud Dedicated
 
Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB
 
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
 
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
 
Introducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage EngineIntroducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage Engine
 
Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena
 
Understanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage EngineUnderstanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage Engine
 
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDBStreamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
 
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
 
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
 
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
 

Recently uploaded

TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 

Recently uploaded (20)

TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 

Balaji Palani [InfluxData] | InfluxDB Tasks Overview | InfluxDays 2022

  • 1.
  • 2. InfluxDB Tasks Overview Balaji Palani - Senior Director of Product Management, InfluxData
  • 3. Connect Learn Build Hear from and meet developers from the InfluxDB Community Be inspired by use cases from our partners and InfluxDB engineers Learn best practices that will help you build great experiences for your projects
  • 4. An InfluxDB task is a scheduled Flux script that takes a stream of input data, modifies or analyzes it in some way. This session breaks down how to use tasks, introduces invokable scripts, and looks at the future of tasks. Balaji Palani Senior Director of Product Management, InfluxData Balaji Palani is the Director of Product Management focused on InfluxDB Cloud. He is passionate about building powerful cloud products that help Developers achieve the fastest time to awesome. And with InfluxDB Cloud, his customers are able to collect and utilize time series data to hit even the toughest SLAs. Previous to InfluxData, Balaji has held several Product Management and Engineering positions at companies like BMC, HP, and Mercury. Balaji holds an MS degree in Computer Science from West Virginia University and a BS in Electrical Engineering from Annamalai University. InfluxDB Tasks Overview
  • 5. Agenda 1. What Are Tasks? 2. Invokable Scripts 3. The Future of Tasks
  • 7. Tasks (Automate your transformations) Raw Data Transformed or Downsampled Data
  • 8. Tasks (Automate your transformations) Raw Data Transformed or Downsampled Data
  • 9. Flux 1. Query your raw data: ○ Bucket ○ Time range ○ Filters Task structure from(bucket: "raw_data") |> range(start: -15m, stop: now()) |> filter(fn: (r) => (r._measurement == "airSensors")) |> filter(fn: (r) => (r._field == "temperature"))
  • 10. Flux 1. Query your raw data: ○ Bucket ○ Time range ○ Filters Task structure from(bucket: "raw_data") |> range(start: -15m, stop: now()) |> filter(fn: (r) => (r._measurement == "airSensors")) |> filter(fn: (r) => (r._field == "temperature"))
  • 11. 2. Aggregate the data ○ Use Flux math functions ○ Build your custom logic Task structure from(bucket: "raw_data") |> range(start: -15m, stop: now()) |> filter(fn: (r) => (r._measurement == "airSensors")) |> filter(fn: (r) => (r._field == "temperature")) |> mean(column: "_value") Flux
  • 12. 3. Write the results to downsampled bucket Task structure from(bucket: "raw_data") |> range(start: -15m, stop: now()) |> filter(fn: (r) => (r._measurement == "airSensors")) |> filter(fn: (r) => (r._field == "temperature")) |> mean(column: "_value") |>rename (columns: {_stop:”_time”}) |> to(bucket: "downsampled_data") Flux
  • 13. 4. Setup your task options: ■ Name of the task ■ Frequency Task structure option task = {name: “agg_1h_sum”, every: 15m, offset: 30s} from(bucket: "raw_data") |> range(start: -15m, stop: now()) |> filter(fn: (r) => (r._measurement == "airSensors")) |> filter(fn: (r) => (r._field == "temperature")) |> mean(column: "_value") |>rename (columns: {_stop:”_time”}) |> to(bucket: "downsampled_data") Flux
  • 14. Flux has a vast array of transformation functions to build any custom task An example of a custom downsampling task option task = {name: "transform_10s_power", every: 10s} offset = 1h step = 10s ta1 = from(bucket: "raw_bucket_1") |> range(start: -offset) |> filter(fn: (r) => (r["_field"] == "power")) |> aggregateWindow(every: step, fn: mean) |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") ta2 = from(bucket: "raw_bucket_2) |> range(start: -offset) |> filter(fn: (r) => (r["_field"] == "parsed_value”)) |> aggregateWindow(every: step, fn: mean) |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") join(tables: {t1: ta1, t2: ta2}, on: ["_time"]) |> to(bucket: "agg_bucket") Flux
  • 15. Flux has a vast array of transformation functions to build any custom task An example of a custom downsampling task option task = {name: "transform_10s_power", every: 10s} offset = 1h step = 10s ta1 = from(bucket: "raw_bucket_1") |> range(start: -offset) |> filter(fn: (r) => (r["_field"] == "power")) |> aggregateWindow(every: step, fn: mean) |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") ta2 = from(bucket: "raw_bucket_2) |> range(start: -offset) |> filter(fn: (r) => (r["_field"] == "parsed_value”)) |> aggregateWindow(every: step, fn: mean) |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value") join(tables: {t1: ta1, t2: ta2}, on: ["_time"]) |> to(bucket: "agg_bucket") Flux
  • 16. Alert Checks (Built on the Tasks subsystem) Monitored Data _monitoring (System Bucket) monitoring()
  • 17. An example of a deadman check task Flux import "influxdata/influxdb/monitor" import "experimental" import "influxdata/influxdb/v1" data = from(bucket: "idping" |> range(start: -10m) |> filter(fn: (r) => (r["_measurement"] == "ctr")) |> filter(fn: (r) => (r["_field"] == "n"))) option task = {name: "idping Deadman", every: 1m, offset: 0s} check = { _check_id: "CHECK_ID", _check_name: "CHECK_NAME", _type: "deadman", tags: {idping: "deadman"} } crit = (r) => (r["dead"]) messageFn = (r) => ("Check: ${r._check_name} is: ${r._level}") data |> v1["fieldsAsCols"]() |> monitor["deadman"](t: experimental["subDuration"](from: now(), d: 5m)) |> monitor["check"](data: check, messageFn: messageFn, crit: crit)
  • 18. An example of a deadman check task Flux import "influxdata/influxdb/monitor" import "experimental" import "influxdata/influxdb/v1" data = from(bucket: "idping" |> range(start: -10m) |> filter(fn: (r) => (r["_measurement"] == "ctr")) |> filter(fn: (r) => (r["_field"] == "n"))) option task = {name: "idping Deadman", every: 1m, offset: 0s} check = { _check_id: "CHECK_ID", _check_name: "CHECK_NAME", _type: "deadman", tags: {idping: "deadman"} } crit = (r) => (r["dead"]) messageFn = (r) => ("Check: ${r._check_name} is: ${r._level}") data |> v1["fieldsAsCols"]() |> monitor["deadman"](t: experimental["subDuration"](from: now(), d: 5m)) |> monitor["check"](data: check, messageFn: messageFn, crit: crit)
  • 19. Notifications (Built on the Tasks subsystem) Call Notification Endpoint (e.g. http.post) _monitoring (System Bucket) monitoring()
  • 20. Notifications (Built on the Tasks subsystem) Call Notification Endpoint (e.g. http.post) _monitoring (System Bucket) monitoring()
  • 21. An example of a notification task import "influxdata/influxdb/monitor" import "slack" import "influxdata/influxdb/secrets" import "experimental" option task = {name: "Reads Deadman Notification", every: 1m, offset: 0s} slack_endpoint = slack["endpoint"](url: "https://hooks.slack.com/services/SGSDFGER/HW36BWGDFEY/Slack_Token") notification = { _notification_rule_id: "NOTIF_RULE_ID", _notification_rule_name: "NOTIF RULE NAME", _notification_endpoint_id: "NOTIF_ENDPOINT", _notification_endpoint_name: "ENDPOINT_NAME", } statuses = monitor["from"](start: -2m, fn: (r) => (r["reads"] == "deadman")) crit = statuses |> filter(fn: (r) => (r["_level"] == "crit")) |> filter(fn: (r) => (r["_time"] >= experimental[ "subDuration"](from: now(), d: 1m))) crit |> monitor["notify"](data: notification, endpoint: slack_endpoint(mapFn: (r) => ({channel: "", text: "Notification Rule: ${r._notification_rule_name} triggered by check: ${r._check_name}: ${r._message}", color: if r["_level"] == "crit" then "danger" else "good"}))) Flux
  • 23. API Invokable Script App / Platform /scripts/aggregate_1h_sum Raw Data Transformed Data AWS Lambda Node-Red Web Connector Azure Functions
  • 24. API Invokable Script App / Platform /scripts/aggregate_1h_sum Raw Data Transformed Data AWS Lambda Node-Red Web Connector Azure Functions
  • 25. Tasks leveraging Scripts /scripts/aggregate_1h_sum Raw Data Downsampled Data Developer value: • Code reusability and shareability • Separation of roles • Drastically improves ease of use Pass param values
  • 26. The Future of Tasks
  • 27. Vision: Automate at scale Schedule Remote Invoke Task Run Flux Script Language Python Javascript Destination Notification Endpoint Bucket External Datastore
  • 28. Vision: Automate at scale Schedule Remote Invoke Task Run Flux Script Language Python Javascript Destination Notification Endpoint Bucket External Datastore
  • 29. Additional Resources Free InfluxDB: OSS or Cloud - influxdata.com/cloud Forums: community.influxdata.com Slack: influxcommunity.slack.com Reddit: r/InfluxData Influx Community (GH): github.com/InfluxCommunity Book: awesome.influxdata.com Docs: docs.influxdata.com Blogs: influxdata.com/blog InfluxDB University: influxdata.com/university How to guides: docs.influxdata.com/resources/how-to-guides/
  • 30. T H A N K Y O U