Paul will outline his vision around the platform and give the latest updates on IFQL ( a new query language), the decoupling of query and storage, the impact of hybrid cloud environments on architecture, cardinality, and discuss the technical directions of the platform. This talk will walk through the vision and architecture with demonstrations of working prototypes of the projects.
InfluxQL is a powerful query language for InfluxDB, and TICKScript is a domain specific language used by Kapacitor to define tasks involving the extraction, transformation and loading of data and also involving the tracking of arbitrary changes and detection of events within data. The combination of these two can make your monitoring apps powerful. During this session, InfluxData Engineer Michael DeSa will share best practices for using these powerful tools. Prerequisite: Intro To Kapacitor.
The document provides an agenda for a seasoned developers track workshop. The agenda includes sessions on InfluxDB query language (IFQL), writing Telegraf plugins, using InfluxDB for open tracing, advanced Kapacitor techniques, setting up InfluxData for IoT, and database orchestration. There will also be breakfast, lunch, breaks and pizza/beer.
Kapacitor - Real Time Data Processing EnginePrashant Vats
Kapacitor is a native data processing engine.Kapacitor is a native data processing engine.It can process both stream and batch data from InfluxDB.It lets you plug in your own custom logic or user-defined functions to process alerts with dynamic thresholds. Key Kapacitor Capabilities
-Alerting
-ETL (Extraction, Transformation and Loading)
-Action Oriented
-Streaming Analytics
-Anomaly Detection
Kapacitor uses a DSL (Domain Specific Language) called TICKscript to define tasks.
A detailed overview of Kapacitor, InfluxDB’s native data processing engine. How to install, configure and build custom TICKscripts enable alerting and anomaly detection
You use InfluxData to monitor the performance of your infrastructure and apps—so it is equally important to keep your InfluxEnterprise instance up and running. Tim Hall, InfluxData VP of Products, will outline why and how you can monitor InfluxEnterprise with InfluxDB.
InfluxQL is a powerful query language for InfluxDB, and TICKScript is a domain specific language used by Kapacitor to define tasks involving the extraction, transformation and loading of data and also involving the tracking of arbitrary changes and detection of events within data. The combination of these two can make your monitoring apps powerful. During this session, InfluxData Engineer Michael DeSa will share best practices for using these powerful tools. Prerequisite: Intro To Kapacitor.
The document provides an agenda for a seasoned developers track workshop. The agenda includes sessions on InfluxDB query language (IFQL), writing Telegraf plugins, using InfluxDB for open tracing, advanced Kapacitor techniques, setting up InfluxData for IoT, and database orchestration. There will also be breakfast, lunch, breaks and pizza/beer.
Kapacitor - Real Time Data Processing EnginePrashant Vats
Kapacitor is a native data processing engine.Kapacitor is a native data processing engine.It can process both stream and batch data from InfluxDB.It lets you plug in your own custom logic or user-defined functions to process alerts with dynamic thresholds. Key Kapacitor Capabilities
-Alerting
-ETL (Extraction, Transformation and Loading)
-Action Oriented
-Streaming Analytics
-Anomaly Detection
Kapacitor uses a DSL (Domain Specific Language) called TICKscript to define tasks.
A detailed overview of Kapacitor, InfluxDB’s native data processing engine. How to install, configure and build custom TICKscripts enable alerting and anomaly detection
You use InfluxData to monitor the performance of your infrastructure and apps—so it is equally important to keep your InfluxEnterprise instance up and running. Tim Hall, InfluxData VP of Products, will outline why and how you can monitor InfluxEnterprise with InfluxDB.
In this presentation, I take a deep dive into the InfluxDB open source storage engine. More than just a single storage engine, InfluxDB is two engines in one: the first for time series data and the second, an index for metadata. I'll delve into the optimizations for achieving high write throughput, compression and fast reads for both the raw time series data and the metadata.
InfluxDB 1.0 - Optimizing InfluxDB by Sam DillardInfluxData
Learn how to optimize InfluxDB 1.0 for performance including hardware and architecture choices, schema design, configuration setup, and running queries. In this InfluxDays NYC 2019 presentation, Sam Dillard provides numerous actionable tips and insights into InfluxDB optimization.
Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData InfluxData
This talk introduces the SQL data source for Flux. It will start with examples of using data from MySQL or Postgres with time series data from InfluxDB. It will then go over the details of how the SQL data source was created.
Extending Flux to Support Other Databases and Data Stores | Adam Anthony | In...InfluxData
Flux was designed to work across databases and data stores. In this talk, Adam will walk through the steps necessary for you to add your own database or custom data source to Flux.
Taming the Tiger: Tips and Tricks for Using TelegrafInfluxData
Taming the Tiger: Tips and Tricks for Using Telegraf
As part of InfluxDays North America 2020 Virtual Experience, the Technical Services team will be offering a free live InfluxDB training to the first 100 registered attendees.This will be hosted over Zoom and Slack with two main trainers and there will be assistants to help participants with the course work. The training will be recorded and made available on the InfluxDays website and the InfluxData YouTube channel.
The course provides an introduction to using Telegraf within a hands-on lab setting. Attendees will be presented a series of lab exercises and get the chance to work through them with the assistance of our remote proctors. After taking this class, attendants will be able to:
Articulate the purposes and value of Telegraf
Understand the basics of configuring and running Telegraf
Understand how to manipulate incoming data to optimize InfluxDB schema
Visualize the insertion results using InfluxDB Cloud UI
There have been plenty of “explaining EXPLAIN” type talks over the years, which provide a great introduction to it. They often also cover how to identify a few of the more common issues through it. EXPLAIN is a deep topic though, and to do a good introduction talk, you have to skip over a lot of the tricky bits. As such, this talk will not be a good introduction to EXPLAIN, but instead a deeper dive into some of the things most don’t cover. The idea is to start with some of the more complex and unintuitive calculations needed to work out the relationships between operations, rows, threads, loops, timings, buffers, CTEs and subplans. Most popular tools handle at least several of these well, but there are cases where they don’t that are worth being conscious of and alert to. For example, we’ll have a look at whether certain numbers are averaged per-loop or per-thread, or both. We’ll also cover a resulting rounding issue or two to be on the lookout for. Finally, some per-operation timing quirks are worth looking out for where CTEs and subqueries are concerned, for example CTEs that are referenced more than once. As time allows, we can also look at a few rarer issues that can be spotted via EXPLAIN, as well as a few more gotchas that we’ve picked up along the way. This includes things like spotting when the query is JIT, planning, or trigger time dominated, spotting the signs of table and index bloat, issues like lossy bitmap scans or index-only scans fetching from the heap, as well as some things to be aware of when using auto_explain.
This document discusses working with time series data using InfluxDB. It provides an overview of time series data and why InfluxDB is useful for storing and querying it. Key features of InfluxDB covered include its SQL-like query language, retention policies for managing data storage, continuous queries for aggregation, and tools for data collection, visualization and monitoring.
Finding OOMS in Legacy Systems with the Syslog Telegraf PluginInfluxData
Dylan Ferreira from FuseMail will share how they use the Syslog Telegraf plugin to help them troubleshoot their systems faster and with more success. Dylan will go over how to set up Rsyslog and Telegraf to filter logs then configure Kapacitor to help you look for interesting things in your raw logs to trigger alerts to your team. He will then bring this all together in a dashboard for your teams to use.
Learn How To Use The #1 DevOps Open Source Time Series DB Platform for Metrics & Events (Time Series Data).
Presentation used in Udemy training: https://www.udemy.com/course/influxdb-time-series-database/?referralCode=09D0B30F92258262D4C6
If you're looking to setup a system to store your metrics in (e.g. app/server metrics), or you need to store & manage other time series, then this course is for you! InfluxDB is currently the #1 time series database (according to db-engines). More and more companies are moving their time series data into a database that is really fit for this purpose, which makes it a really good skill to have.
InfluxDB is an open-source database optimized for fast, high-availability storage and retrieval of time series data. InfluxDB is great for operations monitoring, application metrics, and real-time analytics. InfluxDB is the Time Series Database in the TICK stack and this technology is rising and so is the need for this knowledge in the job market. Its a super useful tool to have on your toolbelt as a DevOps engineer or as a IT professional in general. In this course we will touch all important topics without the need for any prior knowledge.
Anais Dotis-Georgiou & Faith Chikwekwe [InfluxData] | Top 10 Hurdles for Flux...InfluxData
This document discusses top hurdles for Flux beginners and provides solutions. It covers 10 hurdles: 1) Overlooking UI tools for writing Flux, 2) Misunderstanding annotated CSVs, 3) Data layout design leading to high cardinality, 4) Storing wrong data in InfluxDB, 5) Confusion about projecting multiple aggregations, 6) Unfamiliarity with common Flux packages and functions, 7) Unawareness of performance optimizations, 8) Improper use of schema mutations, 9) Knowing when to write custom tasks, and 10) Not understanding when and why to downsample data. For each hurdle, it provides explanations and code examples to illustrate solutions.
Lessons Learned: Running InfluxDB Cloud and Other Cloud Services at Scale | T...InfluxData
In this session, Tim will cover principles, learnings, and practical advice from operating multiple cloud services at scale, including of course our InfluxDB Cloud service. What do we monitor, what do we alert on, and how did we architect it all? What are our underlying architectural and operational principles?
Obtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and TelegrafInfluxData
Did you know you can use InfluxDB to monitor your BBQ and to ensure the tastiest results? Join this meetup to learn two different approaches to using a time series database to monitor a BBQ or a smoker. Learn how Will Cooke uses Python, MQTT, Telegraf and InfluxDB 2.0 to monitor his smoker and to gain insight into temperature changes, the stall, and other important stats about his brisket. Scott Anderson will demonstrate how he uses a FireBoard wireless thermometer, Telegraf and InfluxDB 2.0 to continuously work towards the perfect smoke.
In this InfluxDays NYC 2019 talk, InfluxData Founder & CTO Paul Dix will outline his vision around the platform and its new data scripting and query language Flux, and he will give the latest updates on InfluxDB time series database. This talk will walk through the vision and architecture with demonstrations of working prototypes of the projects.
This document discusses using continuous queries and retention policies in InfluxDB to downsample and manage the retention of time series data. It provides examples of:
- Creating continuous queries to periodically aggregate high resolution data into lower resolution measurements
- Creating retention policies to configure how long raw and aggregated data is stored
- A case study combining continuous queries and retention policies to downsample 10-second telemetry data to 5-minute and store for different durations
9:40 am InfluxDB 2.0 and Flux – The Road Ahead Paul Dix, Founder and CTO | ...InfluxData
The document discusses the evolution of InfluxDB from versions 0.0.1 to 2.0. Key points include the introduction of the line protocol in version 0.9.0, optimizations for performance and queries in later versions, and the introduction of Flux as a query language and for building tasks and packages. Version 2.0 unifies the database, tasks, and UI capabilities into a single platform with a consistent API across languages. It also introduces user packages for sharing Flux code and tasks.
Telegraf is a plugin-driven server agent for collecting & reporting metrics and there are many plugins already written to source data from a variety of services and systems. However, there may be instances where you need to write your own plugin to source data from your particular systems. In this session, Noah will provide you with the steps on how to write your own Telelgraf plugin. This will require an understanding of the Go programming language.
Meet the Experts: Visualize Your Time-Stamped Data Using the React-Based Gira...InfluxData
This document discusses Giraffe, a React-based library for visualizing time-series data from InfluxData. It provides examples of using Giraffe to visualize data exported from Flux queries in InfluxData by converting the data to layers in Giraffe configurations. The document also contains code examples for connecting to InfluxData and executing Flux queries to export data to visualize in Giraffe.
Paul will outline his vision around the platform and give the latest updates on Flux (a new query language), the decoupling of query and storage, the impact of hybrid cloud environments on architecture, cardinality, and discuss the technical directions of the platform. This talk will walk through the vision and architecture with demonstrations of working prototypes of the projects.
Using Grafana with InfluxDB 2.0 and Flux Lang by Jacob LisiInfluxData
Flux, the new InfluxData data scripting and query language (formerly IFQL), super-charges queries both for analytics and data science. Jacob Lisi from Grafana Labs will give a quick overview of the language features as well as the moving parts for a working deployment. Grafana is an open source dashboard solution that shares Flux’s passion for analytics and data science. For that reason, they are very excited to showcase the new Flux support within Grafana, and a couple of common analytics use cases to get the most out of your data.
In this InfluxDays NYC 2019 talk, Jacob Lisi will share the latest updates they have made with their Flux builder in Grafana.
In this presentation, I take a deep dive into the InfluxDB open source storage engine. More than just a single storage engine, InfluxDB is two engines in one: the first for time series data and the second, an index for metadata. I'll delve into the optimizations for achieving high write throughput, compression and fast reads for both the raw time series data and the metadata.
InfluxDB 1.0 - Optimizing InfluxDB by Sam DillardInfluxData
Learn how to optimize InfluxDB 1.0 for performance including hardware and architecture choices, schema design, configuration setup, and running queries. In this InfluxDays NYC 2019 presentation, Sam Dillard provides numerous actionable tips and insights into InfluxDB optimization.
Creating and Using the Flux SQL Datasource | Katy Farmer | InfluxData InfluxData
This talk introduces the SQL data source for Flux. It will start with examples of using data from MySQL or Postgres with time series data from InfluxDB. It will then go over the details of how the SQL data source was created.
Extending Flux to Support Other Databases and Data Stores | Adam Anthony | In...InfluxData
Flux was designed to work across databases and data stores. In this talk, Adam will walk through the steps necessary for you to add your own database or custom data source to Flux.
Taming the Tiger: Tips and Tricks for Using TelegrafInfluxData
Taming the Tiger: Tips and Tricks for Using Telegraf
As part of InfluxDays North America 2020 Virtual Experience, the Technical Services team will be offering a free live InfluxDB training to the first 100 registered attendees.This will be hosted over Zoom and Slack with two main trainers and there will be assistants to help participants with the course work. The training will be recorded and made available on the InfluxDays website and the InfluxData YouTube channel.
The course provides an introduction to using Telegraf within a hands-on lab setting. Attendees will be presented a series of lab exercises and get the chance to work through them with the assistance of our remote proctors. After taking this class, attendants will be able to:
Articulate the purposes and value of Telegraf
Understand the basics of configuring and running Telegraf
Understand how to manipulate incoming data to optimize InfluxDB schema
Visualize the insertion results using InfluxDB Cloud UI
There have been plenty of “explaining EXPLAIN” type talks over the years, which provide a great introduction to it. They often also cover how to identify a few of the more common issues through it. EXPLAIN is a deep topic though, and to do a good introduction talk, you have to skip over a lot of the tricky bits. As such, this talk will not be a good introduction to EXPLAIN, but instead a deeper dive into some of the things most don’t cover. The idea is to start with some of the more complex and unintuitive calculations needed to work out the relationships between operations, rows, threads, loops, timings, buffers, CTEs and subplans. Most popular tools handle at least several of these well, but there are cases where they don’t that are worth being conscious of and alert to. For example, we’ll have a look at whether certain numbers are averaged per-loop or per-thread, or both. We’ll also cover a resulting rounding issue or two to be on the lookout for. Finally, some per-operation timing quirks are worth looking out for where CTEs and subqueries are concerned, for example CTEs that are referenced more than once. As time allows, we can also look at a few rarer issues that can be spotted via EXPLAIN, as well as a few more gotchas that we’ve picked up along the way. This includes things like spotting when the query is JIT, planning, or trigger time dominated, spotting the signs of table and index bloat, issues like lossy bitmap scans or index-only scans fetching from the heap, as well as some things to be aware of when using auto_explain.
This document discusses working with time series data using InfluxDB. It provides an overview of time series data and why InfluxDB is useful for storing and querying it. Key features of InfluxDB covered include its SQL-like query language, retention policies for managing data storage, continuous queries for aggregation, and tools for data collection, visualization and monitoring.
Finding OOMS in Legacy Systems with the Syslog Telegraf PluginInfluxData
Dylan Ferreira from FuseMail will share how they use the Syslog Telegraf plugin to help them troubleshoot their systems faster and with more success. Dylan will go over how to set up Rsyslog and Telegraf to filter logs then configure Kapacitor to help you look for interesting things in your raw logs to trigger alerts to your team. He will then bring this all together in a dashboard for your teams to use.
Learn How To Use The #1 DevOps Open Source Time Series DB Platform for Metrics & Events (Time Series Data).
Presentation used in Udemy training: https://www.udemy.com/course/influxdb-time-series-database/?referralCode=09D0B30F92258262D4C6
If you're looking to setup a system to store your metrics in (e.g. app/server metrics), or you need to store & manage other time series, then this course is for you! InfluxDB is currently the #1 time series database (according to db-engines). More and more companies are moving their time series data into a database that is really fit for this purpose, which makes it a really good skill to have.
InfluxDB is an open-source database optimized for fast, high-availability storage and retrieval of time series data. InfluxDB is great for operations monitoring, application metrics, and real-time analytics. InfluxDB is the Time Series Database in the TICK stack and this technology is rising and so is the need for this knowledge in the job market. Its a super useful tool to have on your toolbelt as a DevOps engineer or as a IT professional in general. In this course we will touch all important topics without the need for any prior knowledge.
Anais Dotis-Georgiou & Faith Chikwekwe [InfluxData] | Top 10 Hurdles for Flux...InfluxData
This document discusses top hurdles for Flux beginners and provides solutions. It covers 10 hurdles: 1) Overlooking UI tools for writing Flux, 2) Misunderstanding annotated CSVs, 3) Data layout design leading to high cardinality, 4) Storing wrong data in InfluxDB, 5) Confusion about projecting multiple aggregations, 6) Unfamiliarity with common Flux packages and functions, 7) Unawareness of performance optimizations, 8) Improper use of schema mutations, 9) Knowing when to write custom tasks, and 10) Not understanding when and why to downsample data. For each hurdle, it provides explanations and code examples to illustrate solutions.
Lessons Learned: Running InfluxDB Cloud and Other Cloud Services at Scale | T...InfluxData
In this session, Tim will cover principles, learnings, and practical advice from operating multiple cloud services at scale, including of course our InfluxDB Cloud service. What do we monitor, what do we alert on, and how did we architect it all? What are our underlying architectural and operational principles?
Obtaining the Perfect Smoke By Monitoring Your BBQ with InfluxDB and TelegrafInfluxData
Did you know you can use InfluxDB to monitor your BBQ and to ensure the tastiest results? Join this meetup to learn two different approaches to using a time series database to monitor a BBQ or a smoker. Learn how Will Cooke uses Python, MQTT, Telegraf and InfluxDB 2.0 to monitor his smoker and to gain insight into temperature changes, the stall, and other important stats about his brisket. Scott Anderson will demonstrate how he uses a FireBoard wireless thermometer, Telegraf and InfluxDB 2.0 to continuously work towards the perfect smoke.
In this InfluxDays NYC 2019 talk, InfluxData Founder & CTO Paul Dix will outline his vision around the platform and its new data scripting and query language Flux, and he will give the latest updates on InfluxDB time series database. This talk will walk through the vision and architecture with demonstrations of working prototypes of the projects.
This document discusses using continuous queries and retention policies in InfluxDB to downsample and manage the retention of time series data. It provides examples of:
- Creating continuous queries to periodically aggregate high resolution data into lower resolution measurements
- Creating retention policies to configure how long raw and aggregated data is stored
- A case study combining continuous queries and retention policies to downsample 10-second telemetry data to 5-minute and store for different durations
9:40 am InfluxDB 2.0 and Flux – The Road Ahead Paul Dix, Founder and CTO | ...InfluxData
The document discusses the evolution of InfluxDB from versions 0.0.1 to 2.0. Key points include the introduction of the line protocol in version 0.9.0, optimizations for performance and queries in later versions, and the introduction of Flux as a query language and for building tasks and packages. Version 2.0 unifies the database, tasks, and UI capabilities into a single platform with a consistent API across languages. It also introduces user packages for sharing Flux code and tasks.
Telegraf is a plugin-driven server agent for collecting & reporting metrics and there are many plugins already written to source data from a variety of services and systems. However, there may be instances where you need to write your own plugin to source data from your particular systems. In this session, Noah will provide you with the steps on how to write your own Telelgraf plugin. This will require an understanding of the Go programming language.
Meet the Experts: Visualize Your Time-Stamped Data Using the React-Based Gira...InfluxData
This document discusses Giraffe, a React-based library for visualizing time-series data from InfluxData. It provides examples of using Giraffe to visualize data exported from Flux queries in InfluxData by converting the data to layers in Giraffe configurations. The document also contains code examples for connecting to InfluxData and executing Flux queries to export data to visualize in Giraffe.
Paul will outline his vision around the platform and give the latest updates on Flux (a new query language), the decoupling of query and storage, the impact of hybrid cloud environments on architecture, cardinality, and discuss the technical directions of the platform. This talk will walk through the vision and architecture with demonstrations of working prototypes of the projects.
Using Grafana with InfluxDB 2.0 and Flux Lang by Jacob LisiInfluxData
Flux, the new InfluxData data scripting and query language (formerly IFQL), super-charges queries both for analytics and data science. Jacob Lisi from Grafana Labs will give a quick overview of the language features as well as the moving parts for a working deployment. Grafana is an open source dashboard solution that shares Flux’s passion for analytics and data science. For that reason, they are very excited to showcase the new Flux support within Grafana, and a couple of common analytics use cases to get the most out of your data.
In this InfluxDays NYC 2019 talk, Jacob Lisi will share the latest updates they have made with their Flux builder in Grafana.
An InfluxDB task is a scheduled Flux script that takes a stream of input data, modifies, or analyzes it in some way. This session breaks down how to use tasks, introduces invokable scripts, and looks at the future of tasks.
Optimizing the Grafana Platform for FluxInfluxData
Flux, the new InfluxData Data Scripting Language (formerly IFQL), super-charges queries both for analytics and data science. Matt will give a quick overview of the language features as well as the moving parts for a working deployment. Grafana is an open source dashboard solution that shares Flux’s passion for analytics and data science. For that reason, they are very excited to showcase the new Flux support within Grafana, and a couple of common analytics use cases to get the most out of your data.
In this talk, Matt Toback from Grafana Labs will share the latest updates they have made with their Flux builder in Grafana.
Functional programming with Ruby - can make you look smartChen Fisher
Functional programming can make you look smart and others feel stupid. Learn how with Ruby
code can be found here:
https://github.com/chenfisher/functional-programming-with-ruby
Youtube:
https://www.youtube.com/watch?v=Qzoh8w4OPtU
This document provides an overview of using Prometheus for monitoring and alerting. It discusses using Node Exporters and other exporters to collect metrics, storing metrics in Prometheus, querying metrics using PromQL, and configuring alert rules and the Alertmanager for notifications. Key aspects covered include scraping configs, common exporters, data types and selectors in PromQL, operations and functions, and setting up alerts and the Alertmanager for routing alerts.
Rhebok, High Performance Rack Handler / Rubykaigi 2015Masahiro Nagano
This document discusses Rhebok, a high performance Rack handler written in Ruby. Rhebok uses a prefork architecture for concurrency and achieves 1.5-2x better performance than Unicorn. It implements efficient network I/O using techniques like IO timeouts, TCP_NODELAY, and writev(). Rhebok also uses the ultra-fast PicoHTTPParser for HTTP request parsing. The document provides an overview of Rhebok, benchmarks showing its performance, and details on its internals and architecture.
This document provides an overview of building web applications with R Shiny, an open source package that allows developers to create interactive web apps using R. It discusses the key components of a Shiny app, including the user interface (UI) and server functions. The UI generates the webpage using R functions, while the server executes R code in response to user interactions. Apps can display interactive R analyses and share results without requiring users to have R installed. Basic apps contain a single app.R file with UI and server functions, but more complex apps can split components across multiple files. Input and output elements allow users to interact with widgets and view outputs. The reactive nature of Shiny causes outputs to update automatically in response to input changes.
Monitoring Your ISP Using InfluxDB Cloud and Raspberry PiInfluxData
When a large group of people change their habits, it can be tricky for infrastructures! Working from home and spending time indoor today means attending video calls and streaming movies and tv shows. This leads to increased internet traffic that can create congestion on the network infrastructure. So how do you get real-time visibility into your ISP connection? In this meetup, Mirko presents his setup based on a time series database and Raspberry Pi to better understand his ISP connection quality and speed — including upload and download speeds. Join us to discover how he does it using Telegraf, InfluxDB Cloud, Astro Pi, Telegram and Grafana! Finally, proof that your ISP connection is (or is not) as fast as it promises.
CloudForecast is a system monitoring and visualization tool that uses Perl and RRDTool to collect data from servers and generate graphs. It collects metrics like CPU usage, network traffic, and Gearman worker status. Data is stored in RRD files and a SQLite database. A radar component collects data and a web interface is used to view graphs generated from the collected data.
fog or: How I Learned to Stop Worrying and Love the CloudWesley Beary
Learn how to easily get started on cloud computing with fog. If you can control your infrastructure choices, you’ll make better choices in development and get what you need in production. You'll get an overview of fog and concrete examples to give you a head start on your provisioning workflow.
Influx/Days 2017 San Francisco | Paul DixInfluxData
IFQL: INFLUX FUNCTIONAL QUERY LANGUAGE – BRINGING DATA SCIENCE AND ANALYTICS INTO THE DATABASE
Introduction of the new query language that we’re building into InfluxDB and the InfluxData platform. Its design is functional and heavily inspired by projects like Pandas in Python and the Tidyverse projects in R. In addition to providing more complex query functionality, the language will facilitate more analytics and data science workloads within the database. Clustering on time series matrices, similarity metrics and k-nearest neighbors, forecasting models, and other data science tasks may become simple query operators within the query language. This talk will introduce the data model, some of the functions and walk through a working prototype implementation that showcases functionality unavailable in the language.
In this presentation, I am going to briefly talk about 'what cloud is' and highlight the various types of cloud (IaaS, PaaS, SaaS). The bulk of the talk will be about using the fog gem using IaaS. I will discuss fog concepts (collections, models, requests, services, providers) and supporting these with actual examples using fog
Router Queue Simulation in C++ in MMNN and MM1 conditionsMorteza Mahdilar
The document describes modifications made to a queue simulation program to measure the average time packets spend in the system. It is modified to track the arrival and departure time of each packet. When a packet arrives, its arrival time is recorded. When the packet departs, its processing time is calculated as the difference between departure and arrival time. These processing times are summed over all packets to calculate the average time spent in the system. The key code changes involve adding variables to track each packet's arrival and departure times, calculate processing time, and sum times across all packets.
fog or: How I Learned to Stop Worrying and Love the Cloud (OpenStack Edition)Wesley Beary
The document discusses how to use the Fog library to interact with cloud services. Fog allows interacting with multiple cloud providers like AWS, Rackspace, etc in a portable way. It provides models, collections, and methods to manage resources like servers, storage, DNS etc. in an abstracted way across providers. The document demonstrates how to boot a server, install SSH keys, run commands via SSH, and ping a target using the Fog and Ruby APIs in just a few lines of code.
Deconstructing the Functional Web with ClojureNorman Richards
Programming for the web in Clojure isn't hard, but with layers of abstraction you can easily lose track of what is going on. In this talk, we'll dig deep into Ring, the request/response library that most Clojure web programming is based on. We'll see exactly what a Ring handler is and look at the middleware abstraction in depth. We'll then take this knowledge and deconstruct the Compojure routing framework to understand precisely how your web application responds to request. At the end of the talk you should thoroughly understand everything that happens in the request/response stack and be able to customize your web application stack with confidence.
Updated for Houston Clojure Meetup 2/28/14
Analyzing the Performance Effects of Meltdown + Spectre on Apache Spark Workl...Databricks
Meltdown and Spectre are two security vulnerabilities disclosed in early 2018 that expose systems to cross-VM and cross-process attacks. They were the first of their kind and opened up a new class of exploits that allow one program to scan another program’s memory. The kernel and VM patches released to address these vulnerabilities have shown to degrade the performance of Apache Spark workloads in the cloud by 2-5%.
This talk will dive deep into the exploits and their patches in order to help explain the origin of this decline in performance.
The document discusses BackgroundRB, a Ruby gem that allows running background jobs and scheduling periodic tasks. It provides an overview of concepts like workers, jobs, caching, and configuration. Examples are given of creating a worker that increments a counter periodically via Ajax, scheduling jobs to run in the future, and periodically running methods via configuration. Advanced uses like connecting workers to distributed systems are also mentioned.
deathstar is a tool for seamlessly running R code across local and cloud computing nodes. It uses ZeroMQ to distribute data and code across nodes. The deathstar daemon runs on each node and listens for work. The R package exposes a zmq.cluster.lapply function similar to regular lapply to distribute jobs. Examples show using deathstar to estimate pi across nodes and calculate stock betas in a distributed manner. deathstar allows scaling R code to multiple cores locally or launching EC2 instances on AWS for more capacity.
Services allow grouping of related sessions in an Oracle RAC environment for high availability and workload management. Key aspects of services include configuring and managing services using tools like DBCA and Enterprise Manager, using services with applications and components like the Resource Manager and Scheduler, setting performance metrics and thresholds on services, and configuring service aggregation and tracing.
Similar to InfluxData Platform Future and Vision (20)
InfluxData is excited to announce InfluxDB Clustered, the self-managed version of InfluxDB 3.0 with unparalleled flexibility, speed, performance, and scale. The evolution of InfluxDB Enterprise, InfluxDB Clustered is delivered as a collection of Kubernetes-based containers and services, which enables you to run and operate InfluxDB 3.0 where you need it, whether that's on-premises or in a private cloud environment. With this new enterprise offering, we’re excited to provide our customers with real-time queries, low-cost object storage, unlimited cardinality, and SQL language support – all with improved data access, support, and security! The newest version of InfluxDB was built on Apache Arrow, and through the open source ecosystem and integrations, extends the value of your time-stamped data.
Join this webinar to learn more about InfluxDB Clustered, and how to manage your large mission-critical workloads in the highly available database service offering!
In this webinar, Balaji Palani and Gunnar Aasen will dive into:
Key features of the new InfluxDB Clustered solution
Use cases for using the newest version of the purpose-built time series database
Live demo
During this 1-hour technical webinar, you’ll also get a chance to ask your questions live.
Best Practices for Leveraging the Apache Arrow EcosystemInfluxData
Apache Arrow is an open source project intended to provide a standardized columnar memory format for flat and hierarchical data. It enables more efficient analytics workloads for modern CPU and GPU hardware, which makes working with large data sets easier and cheaper.
InfluxData and Dremio are both members of the Apache Software Foundation (ASF). Dremio is a data lakehouse management service known for its scalability and capacity for direct querying across diverse data sources. InfluxDB is the purpose-built time series database, and InfluxDB 3.0 has a new columnar storage engine and uses the Arrow format for representing data and moving data to and from Parquet. Discover how InfluxDB and Dremio have advanced their solutions by relying on the Apache Arrow framework.
Join this live panel as Alex Merced and Anais Dotis-Georgiou dive into:
Advantages to utilizing the Apache Arrow ecosystem
Tips and tricks for implementing the columnar data structure
How developers can best utilize the ASF to innovate and contribute to new industry standards
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...InfluxData
Bevi are the creators of smart water dispensers which empower people to choose their desired beverage — flat or sparkling, their desired flavor and temperature. Since 2014, Bevi users have saved more than 350 million bottles and cans. Their "smart" water coolers have prevented the extraction of 1.4 trillion oz of oil from Earth and have saved 21.7 billion grams of CO2 from the atmosphere.
Discover how Bevi uses a time series database to enable better predictive maintenance and alerting of their entire ecosystem — including the hardware and software. They are using InfluxDB to collect sensor data in real-time remotely from their internet-connected machines about their status and activity — i.e., flavor and CO2 levels, water temp, filter status, etc. They a7re using these metrics to improve their customer experience and continuously improve their sustainability practices. Gain tips and tricks on how to best utilize InfluxDB's schema-less design.
Join this webinar as Spencer Gagnon dives into:
Bevi's approach to reducing organizations' carbon footprint — they are saving 50K+ bottles and cans annually
Their entire system architecture — including InfluxDB Cloud, Grafana, Kafka, and DigitalOcean
The importance of using time-stamped data to extend the life of their machines
Power Your Predictive Analytics with InfluxDBInfluxData
If you're using InfluxDB to store and manage your time series data, you're already off to a great start. But why stop there? In our upcoming webinar, we'll show you how to take your data analysis to the next level by building predictive analytics using a variety of tools and techniques.
We will demonstrate how to use Quix to create custom dashboards and visualizations that allow you to monitor your data in real-time. We'll also introduce you to Hugging Face, a powerful tool for building models that can predict future trends and identify anomalies. With these tools at your disposal, you'll be able to extract valuable insights from your data and make more informed decisions about the future. Don't miss out on this opportunity to improve your data analysis skills and take your business to the next level!
What you will learn:
Use InfluxDB to store and manage time series data
Utilize Quix and Hugging Face to build models, visualize trends, and identify anomalies
Extract valuable insights from your data
Improve your data analysis skills to make informed decision
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base InfluxData
Are you considering replacing your legacy data historian and moving your OT data to the cloud? Join this technical webinar to learn how to adopt InfluxDB and IO Base - a digital platform used to improve operational efficiencies!
Teréga Solutions are the creators of digital solutions used to improve energy efficiencies and to address decarbonization challenges. Their network includes 5,000+ km of gas pipelines within France; they aim to help France attain carbon neutrality by 2050. With these impressive goals in mind, Teréga has created IO-Base — the digital platform to improve industrial performance, and increase profitability. Creating digital twins for their clients allows them to collect data from all production sites and view it in real time, from anywhere and at any time.
Discover how Teréga uses InfluxDB, Docker, and AWS to monitor its gas and hydrogen pipeline infrastructure. They chose to replace their legacy data historian with InfluxDB — the purpose built time series database. They are collecting more than 100K different metrics at various frequencies — some are collected every 5 seconds to only every 1-2 minutes. THey have reduced overall IT spend by 50% and collect 2x the amount of data at 20x frequency! By using various industrial protocols (Modbus, OPC-UA, etc.), Teréga improved output, reduced the TCO, and is now able to create added-value services: forecast, monitoring, predictive maintenance.
Join this webinar as Thomas Delquié dives into:
Teréga's approach to modernizing fossil fuel pipelines IT systems while improving yields and safety
Their centralized methodology to collecting sensor, hardware, and network metrics
The importance of time series data and why they chose InfluxDB
Build an Edge-to-Cloud Solution with the MING StackInfluxData
FlowForge enables organizations to reliably deliver Node-RED applications in a continuous, collaborative, and secure manner. Node-RED is the popular, low-code programming solution that makes it easy to connect different services using a visual programming environment. InfluxData is the creator of InfluxDB, the purpose-built time series database run by developers at scale and in any environment in the cloud, on-premises, or at the edge.
Jump-start monitoring your industrial IoT devices and discover how to build an edge-to-cloud solution with the MING stack. The MING stack includes Mosquitto/MQTT, InfluxDB, Node-RED, and Grafana. This solution can be used to improve fleet management, enable predictive maintenance of industrial machines and power generation equipment (i.e. turbines and generators) and increase safety practices (i.e. buildings, construction sites). Join this webinar to learn best practices from industrial IoT SME's.
In this webinar, Robert Marcer and Jay Clifford dive into:
Best practices for monitoring sensor data collected by everyone — from the edge to the factory
Tips and tricks for using Node-RED and InfluxDB together
Demo — see Node-RED and InfluxDB live
Meet the Founders: An Open Discussion About Rewriting Using RustInfluxData
The document is an agenda for a discussion between the CTO and founder of Ockam, Mrinal Wadhwa, and the CTO and founder of InfluxData, Paul Dix, about rewriting products using the Rust programming language. It includes an introduction of the founders, an overview of the discussion topics like why they decided to rewrite in Rust and the challenges they faced, how they got their engineers comfortable with Rust, tips they learned in the process, benefits gained from moving to Rust, and how their communities responded to the switch.
InfluxData is excited to announce the general availability of InfluxDB Cloud Dedicated! It is a fully managed time series database service running on cloud infrastructure resources that are dedicated to a single tenant. With this new offering, we’re excited to provide our customers with additional security options, and more custom configuration options to best suit customers’ workload requirements. Join this webinar to learn more about InfluxDB Cloud, and the new dedicated database service offering!
In this webinar, Balaji Palani and Gary Fowler will dive into:
Key features of the new InfluxDB Cloud Dedicated solution
Use cases for using the newest version of the purpose-built time series database
Live demo
During this 1-hour technical webinar, you’ll also get a chance to ask your questions live.
Gain Better Observability with OpenTelemetry and InfluxDB InfluxData
Many developers and DevOps engineers have become aware of using their observability data to gain greater insights into their infrastructure systems. InfluxDB is the purpose-built time series database used to collect metrics and gain observability into apps, servers, containers, and networks. Developers use InfluxDB to improve the quality and efficiency of their CI/CD pipelines. Start using InfluxDB to aggregate infrastructure and application performance monitoring metrics to enable better anomaly detection, root-cause analysis, and alerting.
This session will demonstrate how to record metrics, logs, and traces with one library — OpenTelemetry — and store them in one open source time series database — InfluxDB. Zoe will demonstrate how easy it is to set up the OpenTelemetry Operator for Kubernetes and to store and analyze your data in InfluxDB.
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...InfluxData
American Metal Processing Company ("AMP") is the US' largest commercial rotary heat treat facility with customers in the automotive, construction, military, and agriculture industries. They use their atmosphere-protected rotary retort furnaces to provide their clients with three primary hardening services: neutral hardening (quench and temper), carburizing, and carbonitriding.
This furnace style ensures consistent, uniform heat treatment process vs. traditional batch-or-belt-style furnaces; excels at processing high volumes of smaller parts with tight tolerances; and improves the strength and toughness of plain carbon steels. Discover why AMP’s use of Telegraf, InfluxDB, Node-RED, and Grafana allows them to gain 24/7 insights into their plant operations and metallurgical results. Learn how they use time-stamped data to gain accurate metrics about their consumables usage, furnace profiles, and machine status.
Join this webinar as Grant Pinkos dives into:
American Metal Processing's approach to heat treating in a digitized environment through connected systems
Their approach to collecting and measuring sensor data to enable predictive maintenance and improve product quality
Why they need a time series database for managing and analyzing vast amounts of time-stamped data
How Delft University's Engineering Students Make Their EV Formula-Style Race ...InfluxData
Delft University is the oldest and largest technical university in the Netherlands with 25,000+ students. Since 1999, they have had a team of students (undergraduate and graduate) designing, building, and racing cars, as part of the Formula Student worldwide competition. The competition has grown to include teams from 1K+ universities in 20+ countries. Students are responsible for all aspects of car manufacturing (research, construction, testing, developing, marketing, management, and fundraising). Delft University's team includes 90 students across disciplines.
Discover how Delft University's team uses Marple and InfluxDB to collect telemetry and sensor metrics while they develop, test, and race their electrics cars. They collect sensor data about their EV's control systems using a time series platform. During races, they are collecting IoT data about their batteries, accelerometer, gyroscope, tires, etc. The engineers are able to share important car stats during races which help the drivers tweak their driving decisions — all with the goal of winning. After races, the entire team are able to analyze data in Marple to understand what to do better next time. By using Marple + InfluxDB, their team are able to collect, share and analyze high frequency car data used to make their car faster at competitions.
Join this webinar as Robbin Baauw and Nero Vanbiervliet dive into:
Marple's approach to empowering engineers to organize, analyze, and visualize their data
Delft University's collaborative methodology to building and racing their Formula-style race car
How InfluxDB is crucial to their collaborative engineering and racing process
Introducing InfluxDB’s New Time Series Database Storage EngineInfluxData
InfluxData is excited to announce the general availability of InfluxDB Cloud's new storage engine! It is a cloud-native, real-time, columnar database optimized for time series data. InfluxDB's rebuilt core was coded in Rust and sits on top of Apache Arrow and DataFusion. InfluxData's team picked Apache Parquet as the persistent format. In this webinar, Paul Dix and Balaji Palani will demonstrate key product features including the removal of cardinality limits!
They will dive into:
The next phase of the InfluxDB platform
How using Apache Arrow's ecosystem has improved InfluxDB's performance and scalability
Key features of InfluxDB Cloud's new core — including SQL native support
Start Automating InfluxDB Deployments at the Edge with balena InfluxData
balena.io helps companies develop, deploy, update, and manage IoT devices. By using Linux containers and other cloud technologies, balena enables teams to quickly and easily build fleets of connected devices. Developers are able to use containers with the language of choice and pull IoT sensor data from 70+ different single board computers into balenaCloud. Discover how to use balena.io to automate your InfluxDB deployments at the edge!
During this one-hour session, experts from balena and InfluxData will demonstrate how to build and deploy your own air quality IoT solution. You will learn:
The fundamentals of IoT sensor deployment and management using balena.
How to use a time series platform to collect and visualize metrics from edge devices.
Tips and tricks to using balenaCloud to automate InfluxDB deployments and Telegraf configurations.
How to use InfluxDB's Edge Data Replication feature to collect sensor data and push it to InfluxDB Cloud for analysis.
No coding experience required, just a curiosity to start your own IoT adventure.
Understanding InfluxDB’s New Storage EngineInfluxData
Learn more about InfluxDB’s new storage engine! The team developed a cloud-native, real-time, columnar database optimized for time series data. We built it all in Rust and it sits on top of Apache Arrow and DataFusion. We chose Apache Parquet as the persistent format, which is an open source columnar data file format. This new storage engine provides InfluxDB Cloud users with new functionality, including the removal of cardinality limits, so developers can bring in massive amounts of time series data at scale.
In this webinar, Anais Dotis-Georgiou will dive into:
Requirements for rebuilding InfluxDB’s core
Key product features and timeline
How Apache Arrow’s ecosystem is used to meet those requirements
Stick around for a demo and live Q&A
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDBInfluxData
RudderStack — the creators of the leading open source Customer Data Platform (CDP) — needed a scalable way to collect and store metrics related to customer events and processing times (down to the nanosecond). They provide their clients with data pipelines that simplify data collection from applications, websites, and SaaS platforms. RudderStack's solution enables clients to stream customer data in real time — they quickly deploy flexible data pipelines that send the data to the customer's entire stack without engineering headaches. Customers are able to stream data from any tool using their 16+ SDK's, and they are able to transform the data in-transit using JavaScript or Python. How does RudderStack use a time series platform to provide their customers with real-time analytics?
Join this webinar as Ryan McCrary dives into:
RudderStack's approach to streamlining data pipelines with their 180+ out-of-the-box integrations
Their data architecture including Kapacitor for alerting and Grafana for customized dashboards
Why using InfluxDB was crucial for them for fast data collection and providing single-sources of truths for their customers
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...InfluxData
Customers using ThingWorx and the Manufacturing Solutions often need to store property data longer than the Solutions default to. These customers are recommended to use InfluxDB, and this presentation will cover the key considerations for moving to InfluxDB vs the standard ThingWorx value streams. Join this session as Ward highlights ThingWorx’s solution and its easy implementation process.
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022InfluxData
Two new features are coming to Flux that add flexibility
and functionality to your data workflow—polymorphic
labels and dynamic types. This session walks through
these new features and shows how they work.
This document outlines the schedule for Day 2 of InfluxDays 2022, an event hosted by InfluxData. The schedule includes sessions on building developer experience, how developers like to work, an overview of the InfluxDB developer console and API, demos of client libraries and the InfluxDB v2 API, tips for getting involved in the InfluxDB community and university, use cases for networking monitoring, crypto/fintech, monitoring/observability, and IIoT, and closing thoughts. Recordings of all sessions will be made available to registered attendees by November 7th. Upcoming events include advanced Flux training in London and resources through the community forums, Slack channel, and online university.
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...InfluxData
This document contains the agenda for Day 2 of InfluxDays 2022, which includes:
- Welcome and introductory remarks from Zoe Steinkamp and Jay Clifford of InfluxData.
- Fireside chats and presentations on building great developer experiences, how developers like to work, and use cases for InfluxDB from companies like Tesla, InfluxData, and others.
- Sessions on the InfluxDB developer console, APIs, client libraries, getting involved in the community, accelerating time to awesome with InfluxDB University, and tips for analyzing IoT data with InfluxDB.
- Closing thoughts from Zoe Steinkamp and Jay Clifford, as well as
The document summarizes the agenda and sessions for Day 1 of InfluxDays 2022. It includes sessions on InfluxDB data collection, scripting languages like Flux, the InfluxDB time series engine, tasks, storage, and a closing discussion. The agenda involves talks from InfluxData employees on building applications with real-time data, navigating the developer experience, solving problems, the InfluxDB platform, community, education, use cases in crypto/fintech and IIoT, and tips/tricks for analysis.
Meet up Milano 14 _ Axpo Italia_ Migration from Mule3 (On-prem) to.pdfFlorence Consulting
Quattordicesimo Meetup di Milano, tenutosi a Milano il 23 Maggio 2024 dalle ore 17:00 alle ore 18:30 in presenza e da remoto.
Abbiamo parlato di come Axpo Italia S.p.A. ha ridotto il technical debt migrando le proprie APIs da Mule 3.9 a Mule 4.4 passando anche da on-premises a CloudHub 1.0.
Gen Z and the marketplaces - let's translate their needsLaura Szabó
The product workshop focused on exploring the requirements of Generation Z in relation to marketplace dynamics. We delved into their specific needs, examined the specifics in their shopping preferences, and analyzed their preferred methods for accessing information and making purchases within a marketplace. Through the study of real-life cases , we tried to gain valuable insights into enhancing the marketplace experience for Generation Z.
The workshop was held on the DMA Conference in Vienna June 2024.
Instagram has become one of the most popular social media platforms, allowing people to share photos, videos, and stories with their followers. Sometimes, though, you might want to view someone's story without them knowing.
Ready to Unlock the Power of Blockchain!Toptal Tech
Imagine a world where data flows freely, yet remains secure. A world where trust is built into the fabric of every transaction. This is the promise of blockchain, a revolutionary technology poised to reshape our digital landscape.
Toptal Tech is at the forefront of this innovation, connecting you with the brightest minds in blockchain development. Together, we can unlock the potential of this transformative technology, building a future of transparency, security, and endless possibilities.
Understanding User Behavior with Google Analytics.pdfSEO Article Boost
Unlocking the full potential of Google Analytics is crucial for understanding and optimizing your website’s performance. This guide dives deep into the essential aspects of Google Analytics, from analyzing traffic sources to understanding user demographics and tracking user engagement.
Traffic Sources Analysis:
Discover where your website traffic originates. By examining the Acquisition section, you can identify whether visitors come from organic search, paid campaigns, direct visits, social media, or referral links. This knowledge helps in refining marketing strategies and optimizing resource allocation.
User Demographics Insights:
Gain a comprehensive view of your audience by exploring demographic data in the Audience section. Understand age, gender, and interests to tailor your marketing strategies effectively. Leverage this information to create personalized content and improve user engagement and conversion rates.
Tracking User Engagement:
Learn how to measure user interaction with your site through key metrics like bounce rate, average session duration, and pages per session. Enhance user experience by analyzing engagement metrics and implementing strategies to keep visitors engaged.
Conversion Rate Optimization:
Understand the importance of conversion rates and how to track them using Google Analytics. Set up Goals, analyze conversion funnels, segment your audience, and employ A/B testing to optimize your website for higher conversions. Utilize ecommerce tracking and multi-channel funnels for a detailed view of your sales performance and marketing channel contributions.
Custom Reports and Dashboards:
Create custom reports and dashboards to visualize and interpret data relevant to your business goals. Use advanced filters, segments, and visualization options to gain deeper insights. Incorporate custom dimensions and metrics for tailored data analysis. Integrate external data sources to enrich your analytics and make well-informed decisions.
This guide is designed to help you harness the power of Google Analytics for making data-driven decisions that enhance website performance and achieve your digital marketing objectives. Whether you are looking to improve SEO, refine your social media strategy, or boost conversion rates, understanding and utilizing Google Analytics is essential for your success.
48. // get the last value written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> last()
49. // get the last value written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> last()
Result: _result
Block: keys: [_field, _measurement, host, region] bounds: [1677-09-21T00:12:43.145224192Z, 2018-02-12T15:53:04.361902250Z)
_time _field _measurement host region _value
------------------------------ --------------- --------------- --------------- --------------- ----------------------
2018-02-12T15:53:00.000000000Z usage_system cpu server0 east 60.6284
Block: keys: [_field, _measurement, host, region] bounds: [1677-09-21T00:12:43.145224192Z, 2018-02-12T15:53:04.361902250Z)
_time _field _measurement host region _value
------------------------------ --------------- --------------- --------------- --------------- ----------------------
2018-02-12T15:53:00.000000000Z usage_user cpu server0 east 39.3716
50. // get the last minute of data from a specific
// measurement & field & host
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
51. // get the last minute of data from a specific
// measurement & field & host
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
Result: _result
Block: keys: [_field, _measurement, host, region] bounds: [2018-02-12T16:01:45.677502014Z, 2018-02-12T16:02:45.677502014Z)
_time _field _measurement host region _value
------------------------------ --------------- --------------- --------------- --------------- ----------------------
2018-02-12T16:01:50.000000000Z usage_user cpu server0 east 50.549
2018-02-12T16:02:00.000000000Z usage_user cpu server0 east 35.4458
2018-02-12T16:02:10.000000000Z usage_user cpu server0 east 30.0493
2018-02-12T16:02:20.000000000Z usage_user cpu server0 east 44.3378
2018-02-12T16:02:30.000000000Z usage_user cpu server0 east 11.1584
2018-02-12T16:02:40.000000000Z usage_user cpu server0 east 46.712
52. // get the mean in 10m intervals of last hour
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu")
|> range(start:-1h)
|> window(every:15m)
|> mean()
Result: _result
Block: keys: [_field, _measurement, host, region] bounds: [2018-02-12T15:05:06.708945484Z, 2018-02-12T16:05:06.708945484Z)
_time _field _measurement host region _value
------------------------------ --------------- --------------- --------------- --------------- ----------------------
2018-02-12T15:28:41.128654848Z usage_user cpu server0 east 50.72841444444444
2018-02-12T15:43:41.128654848Z usage_user cpu server0 east 51.19163333333333
2018-02-12T15:13:41.128654848Z usage_user cpu server0 east 45.5091088235294
2018-02-12T15:58:41.128654848Z usage_user cpu server0 east 49.65145555555555
2018-02-12T16:05:06.708945484Z usage_user cpu server0 east 46.41292368421052
Block: keys: [_field, _measurement, host, region] bounds: [2018-02-12T15:05:06.708945484Z, 2018-02-12T16:05:06.708945484Z)
_time _field _measurement host region _value
------------------------------ --------------- --------------- --------------- --------------- ----------------------
2018-02-12T15:28:41.128654848Z usage_system cpu server0 east 49.27158555555556
2018-02-12T15:58:41.128654848Z usage_system cpu server0 east 50.34854444444444
2018-02-12T16:05:06.708945484Z usage_system cpu server0 east 53.58707631578949
2018-02-12T15:13:41.128654848Z usage_system cpu server0 east 54.49089117647058
2018-02-12T15:43:41.128654848Z usage_system cpu server0 east 48.808366666666664
54. Functional
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
55. Functional
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
built in functions
56. Functional
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
anonymous functions
57. Functional
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
pipe forward operator
58. Named Parameters
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
named parameters only!
64. Inputs
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
no input
65. Outputs
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
output is entire db
66. Outputs
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
pipe that output to filter
67. Filter function input
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
anonymous filter function
input is a single record
{“_measurement”:”cpu”, ”_field”:”usage_user", “host":"server0", “region":"west", "_value":23.2}
68. Filter function input
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
A record looks like a flat object
or row in a table
{“_measurement”:”cpu”, ”_field”:”usage_user", “host":"server0", “region":"west", "_value":23.2}
69. Record Properties
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
tag key
{“_measurement”:”cpu”, ”_field”:”usage_user", “host":"server0", “region":"west", "_value":23.2}
70. Record Properties
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r.host == "server0")
|> range(start:-1m)
same as before
{“_measurement”:”cpu”, ”_field”:”usage_user", “host":"server0", “region":"west", "_value":23.2}
71. Special Properties
starts with _
reserved for system
attributes
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
{“_measurement”:”cpu”, ”_field”:”usage_user", “host":"server0", “region":"west", "_value":23.2}
72. Special Properties
works other way
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r._measurement == "cpu" and
r._field == "usage_user")
|> range(start:-1m)
|> max()
{“_measurement”:”cpu”, ”_field”:”usage_user", “host":"server0", “region":"west", "_value":23.2}
73. Special Properties
_measurement and _field
present for all InfluxDB data
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
{“_measurement”:”cpu”, ”_field”:”usage_user", “host":"server0", “region":"west", "_value":23.2}
74. Special Properties
_value exists in all series
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == “usage_user" and
r[“_value"] > 50.0)
|> range(start:-1m)
|> max()
{“_measurement”:”cpu”, ”_field”:”usage_user", “host":"server0", “region":"west", "_value":23.2}
75. Filter function output
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
filter function output
is a boolean to determine if record is in set
76. Filter Operators
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
!=
=~
!~
in
77. Filter Boolean Logic
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => (r[“host"] == “server0" or
r[“host"] == “server1") and
r[“_measurement”] == “cpu")
|> range(start:-1m)
parens for precedence
78. Function with explicit return
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => {return r[“host"] == “server0"})
|> range(start:-1m)
long hand function definition
79. Outputs
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
filter output
is set of data matching filter function
80. Outputs
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
piped to range
which further filters by a time range
81. Outputs
// get the last 1 hour written for anything from a given host
from(db:"mydb")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1m)
range output is the final query result
83. Does order matter?
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
84. Does order matter?
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
range and filter switched
85. Does order matter?
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
results the same
Result: _result
Block: keys: [_field, _measurement, host, region] bounds: [2018-02-12T17:52:02.322301856Z, 2018-02-12T17:53:02.322301856Z)
_time _field _measurement host region _value
------------------------------ --------------- --------------- --------------- --------------- ----------------------
2018-02-12T17:53:02.322301856Z usage_user cpu server0 east 97.3174
86. Does order matter?
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
is this the same as the top two?
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
|> range(start:-1m)
87. Does order matter?
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
moving max to here
changes semantics
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
|> range(start:-1m)
88. Does order matter?
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
here it operates on
only the last minute of data
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
|> range(start:-1m)
89. Does order matter?
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
here it operates on
data for all time
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
|> range(start:-1m)
90. Does order matter?
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
then that result
is filtered down to
the last minute
(which will likely be empty)
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
|> range(start:-1m)
93. Optimization
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
this is more efficient
94. Optimization
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == "usage_user")
|> max()
query DAG different
plan DAG same as one on left
95. Optimization
from(db:"mydb")
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == “usage_user”
r[“_value"] > 22.0)
|> range(start:-1m)
|> max()
from(db:"mydb")
|> range(start:-1m)
|> filter(fn: (r) =>
r["host"] == "server0" and
r["_measurement"] == "cpu" and
r["_field"] == “usage_user"
r[“_value"] > 22.0)
|> max()
this does a full table scan
97. Variables & Closures
db = "mydb"
measurement = "cpu"
from(db:db)
|> filter(fn: (r) => r._measurement == measurement and
r.host == "server0")
|> last()
anonymous filter function
closure over surrounding context
98. User Defined Functions
db = "mydb"
measurement = “cpu"
fn = (r) => r._measurement == measurement and
r.host == "server0"
from(db:db)
|> filter(fn: fn)
|> last()
assign function to variable fn
99. User Defined Functions
from(db:"mydb")
|> filter(fn: (r) =>
r["_measurement"] == "cpu" and
r["_field"] == "usage_user" and
r["host"] == "server0")
|> range(start:-1h)
100. User Defined Functions
from(db:"mydb")
|> filter(fn: (r) =>
r["_measurement"] == "cpu" and
r["_field"] == "usage_user" and
r["host"] == "server0")
|> range(start:-1h)
get rid of some common boilerplate?
101. User Defined Functions
select = (db, m, f) => {
return from(db:db)
|> filter(fn: (r) => r._measurement == m and r._field == f)
}
102. User Defined Functions
select = (db, m, f) => {
return from(db:db)
|> filter(fn: (r) => r._measurement == m and r._field == f)
}
select(db: "mydb", m: "cpu", f: "usage_user")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1h)
103. User Defined Functions
select = (db, m, f) => {
return from(db:db)
|> filter(fn: (r) => r._measurement == m and r._field == f)
}
select(m: "cpu", f: "usage_user")
|> filter(fn: (r) => r["host"] == "server0")
|> range(start:-1h)
throws error
error calling function "select": missing required keyword argument "db"
128. Having Query
from(db:"mydb")
|> filter(fn: (r) => r._measurement == "cpu")
|> range(start:-1h)
|> window(every:10m)
|> mean()
// this is the having part
|> filter(fn: (r) => r._value > 90)
129. Grouping
// group - average utilization across regions
from(db:"mydb")
|> filter(fn: (r) => r._measurement == "cpu" and
r._field == "usage_system")
|> range(start: -1h)
|> group(by: ["region"])
|> window(every:10m)
|> mean()
134. Functions Implemented as IFQL
// _sortLimit is a helper function, which sorts
// and limits a table.
_sortLimit = (n, desc, cols=["_value"], table=<-) =>
table
|> sort(cols:cols, desc:desc)
|> limit(n:n)
// top sorts a table by cols and keeps only the top n records.
top = (n, cols=["_value"], table=<-) =>
_sortLimit(table:table, n:n, cols:cols, desc:true)