Ryan Betts [InfluxData] | Influxdays Keynote: Engineering Update | InfluxDays NA 2021

•

0 likes•175 views

InfluxData

In this session, InfluxData Vice President of Engineering, Ryan Betts will provide an engineering update on InfluxDB.

Technology

Engineering Update
Ryan Betts
VP Engineering, InﬂuxData

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Agenda
● Introductions
● InﬂuxData engineering: why, who, what
● InﬂuxDB Cloud: under the covers
● InﬂuxDB Cloud: development & operations
● Software updates
● Wrap-up and Q&A

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Who: a culture-ﬁrst team
We...
● Value each other
● Get stuff done
● Believe humility drives learning
● Embrace failure
● Are committed to open source
Remote-ﬁrst
● 20-ish US States
● 10-ish Countries

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Why: we ♥ developer tools
TSDB where you need it
Build what the world needs
wherever you are:
● OSS
● On-premises
● Cloud
Built for Time to Awesome
Real time analytics are hard from
scratch. Start with:
● Ingest, query, alerting built-in
● Real time ops monitoring
● For teams of developers

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
What: functions within engineering
- Dev-Rel
- Ops (cloud, dev. IT)
- TICK stack
- Tier-3 support
Cloud
- Users, billing, marketplaces
- Storage (IOx, TSM)
- Query (Flux, InﬂuxQL)
- UI and developer tools
- Internal ergonomics (CI,
CD, Remocal)
In partnership with our product, support, documentation, design,
analytics, and sales engineering teams.

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Outside
Resource API
Client SDKs
UI (web)
VSCode
CLI

© 2021 InﬂuxData Inc. All Rights Reserved.
Browser-based Tooling IDE integration with VS Code

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
CLI
brew install influxdb-cli

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Client SDKs
https://portal.influxdata.com/downloads/

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Resource API
https://gitlab.com/rickspencer3/jigowatt
https://docs.influxdata.com/influxdb/cloud/reference/api/

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Inside: users, marketplaces, billing

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Inside: service provider regions

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Development and Operations
Building
Continuous delivery model based
around 2-week sprints roughly
organized into 45-day themes.
Operating
Owner-operator model with
teams responsible for their
services.

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
CI/CD Pipeline
Staging
Region 1
Region 2
Commit -> Bot fast-forwards -> CIs (using Circle) -> ArgoCD to staging (testing) -> ArgoCD sequences prod
deploys.

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
CI/CD observability
Created `pipescope` to collect CI/CD traces.
Created `glance` to see PR progress to production.

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Operations
● DevOps model
● Lots of dog-fooding
● Augmented by other tools
● Slack-based incident process

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Commitment to Open Source Software
● Dedicated Telegraf team
● Telegraf release cadence on
track, quarterly
Telegraf

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Commitment to Open Source Software
● Works with latest editions
of Cloud and OSS
● v2 Auth support
● View Flux scripts in
Kapacitor
● Dedicated Telegraf team
● Telegraf release cadence on
track, quarterly
● Flux and UI features
landing regularly
● Staffing increases
● Monthly releases planned
starting with 2.1: available
now!
Telegraf
InﬂuxDB
Chronograf

© 2021 InﬂuxData Inc. All Rights Reserved.
© 2021 InﬂuxData Inc. All Rights Reserved.
Commitment to Commercial Software
● Incremental improvements to InﬂuxQL
● Integrated Flux for those who need it now
● Improved operational metrics
● Improvements to eventual consistency repair
● TSI memory footprint reductions
● Backup-restore improvement. More planned
● Security improvements for FIPS compliance
starting with password hashing
● Expanding the v2 API compatibility
InﬂuxDB Enterprise

© 2021 InﬂuxData Inc. All Rights Reserved.
Questions?

© 2021 InﬂuxData Inc. All Rights Reserved.
Thank You

Hari Prasad was using InfluxDB for his personal weekend project as part of his hobby to map the water levels of lakes in his city, rainfall, evaporation index, etc. His Eureka moment: Software development or any human activity flows with time. He started mapping software development at Fujitsu with time series and created an IOT of Software Development. The POC was well-received and obtained funding to be taken to production as a strategic information system. Earlier, developing a similar system for business required huge budget and efforts, whereas with InfluxDB and its ecosystem, the Quality Cost & Delivery was unbelievable. The Flux addition with 2.0 helped Fujitsu with the power of computing, in addition to queries. They build mean, median, Mode, Quantiles with Flux queries and use other built-in functions. The talk will share the Templates, Flux queries, Scraper code to open source community, all of which will be in GitHub where anyone can reference them. The talk will show how quickly, reliably and cost-effectively you can do data-driven software development by writing custom code. It is so generic such that all software development teams from small to large can benefit with little or no maintenance.

Tim Hall [InfluxData] | InfluxDays Keynote: InfluxDB Roadmap | InfluxDays NA ...

InfluxData

Brian Gilmore [InfluxData] | InfluxDB in an IoT Application Architecture | In...

InfluxData

There are many challenges to building production IoT applications — whether deployed on the shop floor or in millions of homes. Data, specifically time series data, need not be one of them. In this session, Brian Gilmore, IoT Product Manager at InfluxData, outlines the key components of architecture for capturing and analyzing IoT data at ANY scale and showcases how he has implemented these recommendations in his own lab. You will leave this virtual talk with a blueprint for getting started yourself — this talk also covers integrations with Machine Learning and other advanced topics, so InfluxDB users of all experience are welcome!

Russ Savage [Ngrok] | InfluxDB QuickStart | InfluxDays NA 2021

InfluxData

Tanny Ng, Nadeem Syed [WP Engine] | How WP Engine Transformed Monitoring Into...

InfluxData

WP Engine powers more of the top 10M sites than anyone else in WordPress. We serve 5.2 billion web requests per day! Our solution gives our customers performance, intelligence and integrations they need to drive their business forward faster. We needed a monitoring solution without single failure points to support our scale. Discover how WP Engine transformed their monitoring solution into an Observability Platform with InfluxDB, gaining better visibility and reducing infrastructure downtime.

Ana-Maria Calin [InfluxData] | Migrating from OSS to InfluxDB Cloud | InfluxD...

InfluxData

Paul Dix [InfluxData] | InfluxDays Keynote: Future of InfluxDB | InfluxDays N...

InfluxData

Prescient Devices is the creator of a low-code DataOps solution for the edge. Prescient Devices’ flagship product, Prescient Designer, is a simple, graphical, and programmable SaaS workflow development platform which helps clients to work with time series data at the edge. It enables IT/OT teams, data engineers and system integrators to build and manage agile data pipelines with zero lines of code. Discover how InfluxDB enables Prescient Devices’ clients to aggregate their IIoT sensor data at the edge for easy reporting and dashboards. By using a time series database, their customers make distributed data operations, such as ETL and edge analytics, faster and more convenient for end users. Join this webinar as Pablo Acosta and Ashish Yadav dive into: Prescient Devices’ approach to building advanced edge-to-cloud data pipelines How to store, retrieve, and aggregate data from the edge for distributed data operations Their methodology to using a time series database for distributed ETL and edge analytics

Timothy Spann [StreamNative] | Using FLaNK with InfluxDB for EdgeAI IoT at Sc...

InfluxData

Using FLaNK with InfluxDB for EdgeAI IoT at Scale Timothy from StreamNative take you on a hands-on deep-dive on using Pulsar, Apache NiFi + Edge Flow Manager + MiniFi Agents with Apache MXNet, OpenVino, TensorFlow Lite, and other Deep Learning Libraries on the actual edge devices including Raspberry Pi with Movidius 2, Google Coral TPU and NVidia Jetson Nano. The team run deep learning models on the edge devices and send images, and capture real-time GPS and sensor data. Their low-coding IoT applications provide easy edge routing, transformation, data acquisition and alerting before they decide what data to stream real-time to their data space. These edge applications classify images and sensor readings real-time at the edge and then send Deep Learning results to Flink SQL and Apache NiFi for transformation, parsing, enrichment, querying, filtering and merging data to InfluxDB.

Tobias Braun [Herrenknecht AG] | Going Underground with InfluxDB | InfluxDays...

InfluxData

Tobias Braun [Herrenknecht AG] | Going Underground with InfluxDB | InfluxDays NA 2021 Going Underground with InfluxDB Herrenknecht AG is the world’s leading manufacturer of tunnel boring machines. This talk shows how the company is using InfluxDB for capturing IIoT data on hundreds of tunneling construction projects, how they’re transferring this data into the cloud over unreliable network connections and what kind of insights they’re getting out of it.

Michael Hall [InfluxData] | InfluxDB Community Update | InfluxDays NA 2021

InfluxData

How Cisco Provides World-Class Technology Conference Experiences Using Automa...

InfluxData

Cisco Systems’ Cisco Live conference is held annually in the winter in Europe and in the summer in the United States. A typical US event hosts 600+ breakout sessions, dozens of keynotes, Certification Testing and walk-in labs. The conference serves over 26,000 attendees and all their mobile devices. The internal network team is responsible for ensuring that 2,200 wireless access points and 800 switches are providing sufficient network connectivity, availability and bandwidth for all attendees, speakers and organizers across 2 million square feet of conference space. Over 9 days (4 days of setup and 5 conference days), event staff and attendees have pushed over 84 terabytes of data from the conference to the internet! Dual 100 Gigabit/second primary links and backup 10 Gigabit/second links handle anything the users can throw at it. The infrastructure required for this event also includes servers, VMs, and containerized workloads. With the growing need for hybrid events, Cisco’s team also ensures 100% video streaming uptime. Discover how Cisco uses InfluxDB to store key performance metrics across many IT domains alongside their commercial management solutions. The team continues to iterate and improve year-over-year to gain visibility into their network and devices to streamline troubleshooting and quickly respond to events before they become service impacting. In this webinar, Jason Davis dives into: Cisco’s approach to using automation, orchestration, Python scripts, SNMP, and streaming telemetry to collect network data Their methodology to troubleshooting, prioritizing, and scheduling fixes to ensure the best client experience How a time series platform is crucial to their real-time data analysis

Evan Kaplan [InfluxData] | InfluxDays Opening Remarks | InfluxDays NA 2021

InfluxData

Bhagvan Kommadi [Value Momentum] | TeleHealth Platform: DevOps-Based Progress...

InfluxData

TeleHealth Platform: DevOps-Based Progressive Delivery The talk covers a real-life experience related to building a DevOps Delivery-powered AI platform for doctors’ community and telehealth support for patients during COVID-19 lockdown. The doctors’ community interacts related to cases and triage for different patient cases. They can extend telehealth support using medical practice management solutions. Patients can order medicines online through integrated pharmacies on the platform. AI Platform has digital, voice, and knowledge assistants to provide information to the doctor. DevOps is enabled using Jenkins on AWS which helps in continuous integration and progressive delivery of features to Mobile and web apps (Apple & Google app stores). Historical data is used for predictive analytics by the machine learning platform. The platform helps healthcare enterprises: 1. Deploy voice tech to facilitate clinical documentation 2. Reduce physicians’ administrative burden 3. Increase patient volume and billable revenue 4. Eliminate transcription costs 5. Use voice to increase touchpoints and increase patient engagement.

InfluxDB + Kepware: Start Monitoring Industrial Data Quickly

InfluxData

Kepware (a PTC Technology) is the market leader in Industrial Connectivity platforms, and is helping companies bridge the gap between IT and OT. For the past 25 years, Kepware has developed industrial connectivity tools including over 150 industrial drivers, and supporting over 300 protocols – from traditional automation protocols such as OPC UA or OPC DA to more modern IIoT protocols like MQTT or REST. Together with InfluxDB, customers achieve quick time to value, gaining insights from their industrial time-stamped data. Developers are able to collect, process, store and analyze thousands of metrics faster! Join this webinar to learn multiple approaches to sending IIoT data to a time series database. In this webinar, Kyle Carreau and Jay Clifford dive into: How to use Telegraf to send OPC UA and MQTT metrics to InfluxDB The new Kepware IoT Gateway Advanced Template features to send data directly to InfluxDB Best practices for using InfluxDB + Kepware for industrial automation – stick around for a demo!

Virtual training intro to InfluxDB - June 2021

InfluxData

In this training webinar, we will walk you through the basics of InfluxDB – the purpose-built time series database. InfluxDB has everything you need from a time series platform in a single binary – a multi-tenanted time series database, UI and dashboarding tools, background processing and monitoring agent. This one-hour session will include the training and time for live Q&A. What you will learn Core concepts of time series databases An overview of the InfluxDB platform How to ingesting and query data in InfluxDB

Webinar Registration Getting Started with Building Your First IoT App

InfluxData

If you’re an IoT developer, engineer, project owner, or business, you won’t want to miss the free ($300 value!) virtual Getting Started with Building Your First IoT App workshop on April 8. This workshop showcases a fully functional sample application called IoT Center that is built on InfluxDB. IoT Center demonstrates the capabilities of the InfluxDB platform to develop a JavaScript-enabled time-series-based application. It collects, stores and displays a set of values that include temperature, humidity, pressure, CO2 concentration, air quality, as well as provides GPS coordinates from a set of IoT devices (sensors). With this data stored in InfluxDB, the application can query it for display as well as write data back into the database.

Building an IoT Monitoring App with InfluxDB and LoRa

InfluxData

The Internet of Things (IoT) is increasingly driven by sensor data, with devices taking measured actions based on everything from illumination intensity, temperature, and machinery performance to an ecosystem that creates networks, devices and solutions using LoRaWAN®. In this session we will build a fully functional sample IoT monitoring application built on InfluxDB and powered by the Things Network, collecting sensor metrics from a LoRa device. We will then showcase core InfluxDB tools to query, visualize and generate alert notifications using a custom user interface. In this workshop you will learn: How to install and configure the InfluxDB VS code plugin How to use Telegraf to receive data from the Things Network, and write that data to InfluxDB How to query InfluxDB so that your app can visualize your IoT data within a custom Flask application How to deploy a downsampling task in InfluxDB to “clean” and transform raw sensor data while also reducing overall storage costs How to use InfluxDB’s powerful “Checks and Notifications” system to help you provide custom alerting to your users

InfluxDB + Telegraf Operator: Easy Kubernetes Monitoring

InfluxData

Telegraf is an open-source server agent designed to collect metrics from stacks, sensors, and systems — with nearly 300 inputs and outputs. Telegraf Operator makes it easy to use Telegraf for monitoring your Kubernetes workloads. It enables developers to define a common output destination for all metrics, and configure Sidecar monitoring on your application pods using annotations. With the Telegraf sidecar container added, it will collect data and start pushing the metrics to a time series database, like InfluxDB. Discover how to use the Telegraf Operator as a control center for managing individual Telegraf instances which are deployed throughout Kubernetes clusters. Find out how to use the InfluxDB and Telegraf Operator to monitor and get metrics from your Kubernetes workloads. Join this webinar as InfluxData's Pat Gaughen and Wojciech Kocjan provide: InfluxDB & Telegraf overview Telegraf Operator deep-dive Live demos of sample deployments!

Upgrading Made Easy: Moving to InfluxDB 2.x or InfluxDB Cloud with Cribl LogS...

InfluxData

Many organizations agree that migrating workloads to the cloud or to a newer version of existing tooling can result in cost savings and flexibility. A well-designed observability pipeline is often the key to a quick and painless transition, leading to positive impacts on cost optimization, data visibility, and performance. Cribl’s LogStream product helps teams implement such an observability pipeline. In this hands-on technical discussion, the audience will learn how to leverage Cribl LogStream to successfully upgrade from InfluxDB 1.x to InfluxDB 2.x or move to InfluxDB Cloud. Join us as we walk through the pros and cons of workload migration, share architecture best practices, and give a live demo on how to combine Cribl LogStream with the latest version of InfluxDB.

Getting Started: Intro to Telegraf - July 2021

InfluxData

In this training webinar, Samantha Wang will walk you through the basics of Telegraf. Telegraf is the open source server agent which is used to collect metrics from your stacks, sensors and systems. It is InfluxDB’s native data collector that supports nearly 300 inputs and outputs. Learn how to send data from a variety of systems, apps, databases and services in the appropriate format to InfluxDB. Discover tips and tricks on how to write your own plugins. The know-how learned here can be applied to a multitude of use cases and sectors. This one-hour session will include the training and time for live Q&A. Join this training as Samantha Wang dives into: Types of Telegraf plugins (i.e. input, output, aggregator and processor) Specific plugins including Execd input plugins and the Starlark processor plugin How to install and start using Telegraf

David Henthorn [Rose-Hulman Institute of Technology] | Illuminating the Dark ...

InfluxData

Critical infrastructure, such as manufacturing facilities, power plants, dams, and chemical plants, has long experienced a phenomenon known as dark data – crucial information is locked away in disparate and proprietary systems. At Rose-Hulman Institute of Technology, we are teaching students to unlock this data with the help of tools such as Telegraf and InfluxDB to better make informed decisions. In addition, these tools allow our students to investigate best practices in data handling and critical infrastructure cybersecurity.

Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021

InfluxData

How to Streamline Incident Response with InfluxDB, PagerDuty and Rundeck

InfluxData

Mean Time to Resolution (MTTR) is a foundational KPI for most organizations. DevOps and SRE teams are under intense pressure to reduce MTTR when resolving incidents. Often parts of incident response processes are manual, bringing together alerts, runbooks, ad-hoc scripts, and people to form a response. In this webinar, we will show you how to improve resolution time by configuring InfluxDB notification endpoints to PagerDuty and triggering auto-remediations with Rundeck. Using Rundeck’s automated runbooks, customers have experienced up to 50% reduction in incident response time, greatly improving team productivity and reducing unnecessary outage time.

Bernard Paques & Kevin Polossat [AWS] | Combining the Power of InfluxDB and A...

InfluxData

Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...

InfluxData

Red Hat is the provider of enterprise open source solutions. Its portfolio of products includes hybrid cloud infrastructure, middleware, cloud-native apps and automation solutions. Its internal network supports all lines of business — including 60+ sites. Discover how Red Hat uses InfluxDB and Flux for better real-time monitoring of their networks to improve performance and to understand utilization better.

Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience Lond...

InfluxData

Tim Hall and Ryan Betts [InfluxData] | InfluxDB Roadmap and Engineering Updat...

InfluxData

KUDO - Kubernetes Operators, the easy way

Nick Jones

Kubernetes Operators are the next phase of the journey towards automating complex applications in containers. Many Operators that exist today handle initial deployment, but they don’t provide automation for tasks like binary upgrades, configuration updates, and failure recovery. Implementing a production-grade controller for a complex workload typically requires thousands of lines of code and many months of development. As a result, the quality of operators that are available today varies. The goal of KUDO is to help solve some of these problems by providing a comprehensive toolkit for building Kubernetes operators declaratively using YAML, with a focus on distributed stateful applications.

What's hot

How to Manage Your Time Series Data Pipeline at the Edge with InfluxDB

InfluxData

Timothy Spann [StreamNative] | Using FLaNK with InfluxDB for EdgeAI IoT at Sc...

InfluxData

Tobias Braun [Herrenknecht AG] | Going Underground with InfluxDB | InfluxDays...

InfluxData

Michael Hall [InfluxData] | InfluxDB Community Update | InfluxDays NA 2021

InfluxData

How Cisco Provides World-Class Technology Conference Experiences Using Automa...

InfluxData

Evan Kaplan [InfluxData] | InfluxDays Opening Remarks | InfluxDays NA 2021

InfluxData

Bhagvan Kommadi [Value Momentum] | TeleHealth Platform: DevOps-Based Progress...

InfluxData

InfluxDB + Kepware: Start Monitoring Industrial Data Quickly

InfluxData

Virtual training intro to InfluxDB - June 2021

InfluxData

Webinar Registration Getting Started with Building Your First IoT App

InfluxData

Building an IoT Monitoring App with InfluxDB and LoRa

InfluxData

InfluxDB + Telegraf Operator: Easy Kubernetes Monitoring

InfluxData

Upgrading Made Easy: Moving to InfluxDB 2.x or InfluxDB Cloud with Cribl LogS...

InfluxData

Getting Started: Intro to Telegraf - July 2021

InfluxData

David Henthorn [Rose-Hulman Institute of Technology] | Illuminating the Dark ...

InfluxData

Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021

InfluxData

How to Streamline Incident Response with InfluxDB, PagerDuty and Rundeck

InfluxData

Bernard Paques & Kevin Polossat [AWS] | Combining the Power of InfluxDB and A...

InfluxData

Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...

InfluxData

Tim Hall [InfluxData] | InfluxDB Roadmap | InfluxDays Virtual Experience Lond...

InfluxData

What's hot (20)