SlideShare a Scribd company logo
1 of 27
Download to read offline
Going Underground
with InfluxDB
Tobias Braun – Software Architect
Herrenknecht AG, Germany
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Overview
- Herrenknecht AG and the presenter
- Building an IIoT platform for Herrenknecht AG
- Focus on handling of sensor data
- InfluxDB as central time series storage
- Migration to InfluxDB Enterprise
- Current status of the product
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Herrenknecht AG
- Founded 1977 in Germany
- World market leader for tunnel
boring machines
- For tunnels from 0.1m to 19m
- Traffic, utility and mining
- Vertical and horizontal
- All geologies, even under water
- Major share of global market
- ~ 5,300 employees
- Revenue 2020: 1,007 mio. €
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Tobias Braun
Software Architect, Herrenknecht AG
Build an IIoT platform with insight into live and historic data for all tunnel
boring machines delivered by Herrenknecht AG.
Small agile team of 4 developers. Frontend development is supported by an
external partner. We have to compromise and gradually improve what we
build.
E-Mail: braun.tob@herrenknecht.com
Software Architect at Herrenknecht for 2 ½ years
- More than 20 years of experience in highly available, distributed systems
- Worked for:
1&1 IONOS – Web Hosting Core
Burda GmbH – E-Commerce
Trumpf Werkzeugmaschinen GmbH – IIoT platform for a machine tool manufacturer
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Building an IIoT platform for
Herrenknecht AG
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Tunnel diameter and amount of data correlate
1985
HERA
5.95 m
1996
Sydney
10.70 m
1997
Hamburg
14.20 m
2006
Madrid
15.20 m
2006
Shanghai
15.43 m
2010
Sparvo
15.62 m
2016
Santa Lucia
15.87 m
2013
Hongkong
17.6 m
Concept
St. Petersburg
19.25 m
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Current, correct data is vital for TBM operator
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Building an IIoT platform for Herrenknecht AG
• Has to support >2000 TBM, several hundred are working simultaneously
• Up to 5000 sensors on a single machine
• Some sensors with sample rates <= 100ms
• Limited bandwidth to the job-site
• Machines can be offline for days, weeks, even months
• Historic data from 40 years and in different formats (SQL, DBase, even CSV)
has to be made available
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Building an IIoT platform for Herrenknecht AG
• Can be developed and maintained by a small team
• Start with the basics: Make sure data is recorded consistently
• Development first for TBM, cloud as a second step
• Data storage for the next 10-15 years
• Robust, flexible architecture
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
InfluxDB as central time
series storage
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Why we chose InfluxDB
• Open Source with commercial „Enterprise Edition“
• Windows-Service available (was relevant „back-then“)
• Very space efficient storage
• Query language similar to SQL with powerful aggregations
• Stored in simple, flat files
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Where is sensor data stored?
• On-site
• Control room of the TBM is equipped with
ruggedized industrial PCs
• Windows operating system with limited
computing power and SSD
• Data cleansing and storage in InfluxDB OSS
• Cloud
• InfluxDB Enterprise running on Microsoft
Azure
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
How is data stored?
• Basically one big measurement
• Every sensor is a „float“ field
• Only few tags are used
• Low tag cardinality
• Only write values when there
was a significant change
-> 1-5 GB / month & machine
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Simple architecture for InfluxDB in the cloud
• When entering the cloud, we used a quick and pragmatic
approach based on Kubernetes
• One InfluxDB OSS container per TBM
• No clustering of InfluxDB itself
• Azure File Storage (equivalent to Windows share)
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Advantages of this approach
• Architecture almost identical to the environment on the TBM,
therefore minimal development effort
• Isolated impact zones
• Cheap pay-per-use storage
• File-based backup is included with Azure File Storage
• InfluxDB OSS – no licensing cost
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
How is data transferred to the cloud?
• Historic data is converted to InfluxDB format in batches
• Live Data:
• Data cleansing is already done on the machine
• So we simply mirror all the data from the machine to the cloud
• We built a custom synchronization, based on a REST API
• Result: Data is identical on the TBM and in the cloud
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Migration to InfluxDB
Enterprise
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
First issues with the pragmatic approach
• Scaled well to ~100 InfluxDB containers,
• But with >> 100 containers,
• the overall system became unreliable
• … and response times for even simple queries exploded
• … and the infrastructure became prohibitively expensive
• Azure File Storage „forgot“ to unlock files in rare cases
• with hundreds of containers, even rare cases tend to happen
regularly
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Decision to migrate to InfluxDB Enterprise
• 750 GB of data in 200 databases
• At most a couple of minutes of downtime for each TBM
• Small team with limited capacity, migration had to be automated
• Executed successfully in Q1/2020, took less than two weeks
• No read-downtime, write-downtime of less than 60 minutes per TBM
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Results after the migration
• Reduced TCO of Influx databases by 1/3
• Extensive monitoring and stable operation
• Very reliable and reproducable response times for all
queries
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Status of the product
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Successful rollout to first customers
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Successful rollout to first customers
© 2021 InfluxData Inc. All Rights Reserved.
© 2021 InfluxData Inc. All Rights Reserved.
Status of the product
• Very positive feedback from customers and internal users
• Team can concentrate on feature development, little
DevOps efforts
• InfluxDB OSS on every TBM
• InfluxDB Enterprise in the cloud just works, unnoticable in
the background
© 2021 InfluxData Inc. All Rights Reserved.
Questions?
braun.tob@herrenknecht.com
© 2021 InfluxData Inc. All Rights Reserved.
Thank You

More Related Content

What's hot

Three Ways InfluxDB Enables You to Use Time Series Data Across Your Entire En...
Three Ways InfluxDB Enables You to Use Time Series Data Across Your Entire En...Three Ways InfluxDB Enables You to Use Time Series Data Across Your Entire En...
Three Ways InfluxDB Enables You to Use Time Series Data Across Your Entire En...
InfluxData
 

What's hot (20)

Paul Dix [InfluxData] | InfluxDays Opening Keynote | InfluxDays EMEA 2021
Paul Dix [InfluxData] | InfluxDays Opening Keynote | InfluxDays EMEA 2021Paul Dix [InfluxData] | InfluxDays Opening Keynote | InfluxDays EMEA 2021
Paul Dix [InfluxData] | InfluxDays Opening Keynote | InfluxDays EMEA 2021
 
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
Martin Moucka [Red Hat] | How Red Hat Uses gNMI, Telegraf and InfluxDB to Gai...
 
InfluxDB Cloud Product Update
InfluxDB Cloud Product Update InfluxDB Cloud Product Update
InfluxDB Cloud Product Update
 
Hari-Prasad Sudharshan [Fujitsu Network Communications] | ML-Based Data-Drive...
Hari-Prasad Sudharshan [Fujitsu Network Communications] | ML-Based Data-Drive...Hari-Prasad Sudharshan [Fujitsu Network Communications] | ML-Based Data-Drive...
Hari-Prasad Sudharshan [Fujitsu Network Communications] | ML-Based Data-Drive...
 
Nilden Tutular, Volkan Balikci, Uygar Zubari [Eldor Corporation] | MQTT - Mac...
Nilden Tutular, Volkan Balikci, Uygar Zubari [Eldor Corporation] | MQTT - Mac...Nilden Tutular, Volkan Balikci, Uygar Zubari [Eldor Corporation] | MQTT - Mac...
Nilden Tutular, Volkan Balikci, Uygar Zubari [Eldor Corporation] | MQTT - Mac...
 
Three Ways InfluxDB Enables You to Use Time Series Data Across Your Entire En...
Three Ways InfluxDB Enables You to Use Time Series Data Across Your Entire En...Three Ways InfluxDB Enables You to Use Time Series Data Across Your Entire En...
Three Ways InfluxDB Enables You to Use Time Series Data Across Your Entire En...
 
Evan Kaplan [InfluxData] | InfluxDays Opening Remarks | InfluxDays EMEA 2021
Evan Kaplan [InfluxData] | InfluxDays Opening Remarks | InfluxDays EMEA 2021Evan Kaplan [InfluxData] | InfluxDays Opening Remarks | InfluxDays EMEA 2021
Evan Kaplan [InfluxData] | InfluxDays Opening Remarks | InfluxDays EMEA 2021
 
Tim Hall and Ryan Betts [InfluxData] | InfluxDB Roadmap and Engineering Updat...
Tim Hall and Ryan Betts [InfluxData] | InfluxDB Roadmap and Engineering Updat...Tim Hall and Ryan Betts [InfluxData] | InfluxDB Roadmap and Engineering Updat...
Tim Hall and Ryan Betts [InfluxData] | InfluxDB Roadmap and Engineering Updat...
 
Ryan Betts [InfluxData] | Influxdays Keynote: Engineering Update | InfluxDays...
Ryan Betts [InfluxData] | Influxdays Keynote: Engineering Update | InfluxDays...Ryan Betts [InfluxData] | Influxdays Keynote: Engineering Update | InfluxDays...
Ryan Betts [InfluxData] | Influxdays Keynote: Engineering Update | InfluxDays...
 
Russ Savage [Ngrok] | InfluxDB QuickStart | InfluxDays NA 2021
Russ Savage [Ngrok] | InfluxDB QuickStart | InfluxDays NA 2021Russ Savage [Ngrok] | InfluxDB QuickStart | InfluxDays NA 2021
Russ Savage [Ngrok] | InfluxDB QuickStart | InfluxDays NA 2021
 
Monitor Kubernetes in Rancher using InfluxData
Monitor Kubernetes in Rancher using InfluxDataMonitor Kubernetes in Rancher using InfluxData
Monitor Kubernetes in Rancher using InfluxData
 
Bhagvan Kommadi [Value Momentum] | TeleHealth Platform: DevOps-Based Progress...
Bhagvan Kommadi [Value Momentum] | TeleHealth Platform: DevOps-Based Progress...Bhagvan Kommadi [Value Momentum] | TeleHealth Platform: DevOps-Based Progress...
Bhagvan Kommadi [Value Momentum] | TeleHealth Platform: DevOps-Based Progress...
 
How to Create a Modern IIoT Monitoring Solution On iOS Using Swift, MQTT and ...
How to Create a Modern IIoT Monitoring Solution On iOS Using Swift, MQTT and ...How to Create a Modern IIoT Monitoring Solution On iOS Using Swift, MQTT and ...
How to Create a Modern IIoT Monitoring Solution On iOS Using Swift, MQTT and ...
 
Vasilis Papavasiliou [Mist.io] | Integrating Telegraf, InfluxDB and Mist to M...
Vasilis Papavasiliou [Mist.io] | Integrating Telegraf, InfluxDB and Mist to M...Vasilis Papavasiliou [Mist.io] | Integrating Telegraf, InfluxDB and Mist to M...
Vasilis Papavasiliou [Mist.io] | Integrating Telegraf, InfluxDB and Mist to M...
 
How to Manage Your Time Series Data Pipeline at the Edge with InfluxDB
How to Manage Your Time Series Data Pipeline at the Edge with InfluxDBHow to Manage Your Time Series Data Pipeline at the Edge with InfluxDB
How to Manage Your Time Series Data Pipeline at the Edge with InfluxDB
 
Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021
Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021
Alex Nauda [Nobl9] | How Not to Build an SLO Platform | InfluxDays NA 2021
 
Virtual training intro to InfluxDB - June 2021
Virtual training  intro to InfluxDB  - June 2021Virtual training  intro to InfluxDB  - June 2021
Virtual training intro to InfluxDB - June 2021
 
Taming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using TelegrafTaming the Tiger: Tips and Tricks for Using Telegraf
Taming the Tiger: Tips and Tricks for Using Telegraf
 
David Henthorn [Rose-Hulman Institute of Technology] | Illuminating the Dark ...
David Henthorn [Rose-Hulman Institute of Technology] | Illuminating the Dark ...David Henthorn [Rose-Hulman Institute of Technology] | Illuminating the Dark ...
David Henthorn [Rose-Hulman Institute of Technology] | Illuminating the Dark ...
 
How to Deliver a Critical and Actionable Customer-Facing Metrics Product with...
How to Deliver a Critical and Actionable Customer-Facing Metrics Product with...How to Deliver a Critical and Actionable Customer-Facing Metrics Product with...
How to Deliver a Critical and Actionable Customer-Facing Metrics Product with...
 

Similar to Tobias Braun [Herrenknecht AG] | Going Underground with InfluxDB | InfluxDays NA 2021

How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
InfluxData
 

Similar to Tobias Braun [Herrenknecht AG] | Going Underground with InfluxDB | InfluxDays NA 2021 (20)

Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...
Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...
Kafka & InfluxDB: BFFs for Enterprise Data Applications | Russ Savage, Influx...
 
Paul Dix [InfluxData] | InfluxDays Keynote: Future of InfluxDB | InfluxDays N...
Paul Dix [InfluxData] | InfluxDays Keynote: Future of InfluxDB | InfluxDays N...Paul Dix [InfluxData] | InfluxDays Keynote: Future of InfluxDB | InfluxDays N...
Paul Dix [InfluxData] | InfluxDays Keynote: Future of InfluxDB | InfluxDays N...
 
Cloud Computing and the Gaming Industry - ProfitBricks Talk
Cloud Computing and the Gaming Industry - ProfitBricks TalkCloud Computing and the Gaming Industry - ProfitBricks Talk
Cloud Computing and the Gaming Industry - ProfitBricks Talk
 
Serving your phone calls with microservices
Serving your phone calls with microservicesServing your phone calls with microservices
Serving your phone calls with microservices
 
Moving to software-based production workflows and containerisation of media a...
Moving to software-based production workflows and containerisation of media a...Moving to software-based production workflows and containerisation of media a...
Moving to software-based production workflows and containerisation of media a...
 
Aberdeen Oil & Gas Event - AWS Partner Eurotech
Aberdeen Oil & Gas Event - AWS Partner EurotechAberdeen Oil & Gas Event - AWS Partner Eurotech
Aberdeen Oil & Gas Event - AWS Partner Eurotech
 
3 Reasons to Select Time Series Platforms for Cloud Native Applications Monit...
3 Reasons to Select Time Series Platforms for Cloud Native Applications Monit...3 Reasons to Select Time Series Platforms for Cloud Native Applications Monit...
3 Reasons to Select Time Series Platforms for Cloud Native Applications Monit...
 
HiveMQ Cloud - The Cloud Native IoT Messaging Layer
HiveMQ Cloud - The Cloud Native IoT Messaging LayerHiveMQ Cloud - The Cloud Native IoT Messaging Layer
HiveMQ Cloud - The Cloud Native IoT Messaging Layer
 
Introducing HiveMQ Cloud
Introducing HiveMQ CloudIntroducing HiveMQ Cloud
Introducing HiveMQ Cloud
 
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageWebinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
 
Pushing it to the edge in IoT
Pushing it to the edge in IoTPushing it to the edge in IoT
Pushing it to the edge in IoT
 
HiveMQ + Kafka - The Ideal Solution for IoT MQTT Data Integration
HiveMQ + Kafka - The Ideal Solution for IoT MQTT Data IntegrationHiveMQ + Kafka - The Ideal Solution for IoT MQTT Data Integration
HiveMQ + Kafka - The Ideal Solution for IoT MQTT Data Integration
 
Cloud Bursting 101: What to do When Cloud Computing Demand Exceeds Capacity
Cloud Bursting 101: What to do When Cloud Computing Demand Exceeds CapacityCloud Bursting 101: What to do When Cloud Computing Demand Exceeds Capacity
Cloud Bursting 101: What to do When Cloud Computing Demand Exceeds Capacity
 
HiveMQ + Kafka: The ideal solution for IoT MQTT data integration
HiveMQ + Kafka: The ideal solution for IoT MQTT data integrationHiveMQ + Kafka: The ideal solution for IoT MQTT data integration
HiveMQ + Kafka: The ideal solution for IoT MQTT data integration
 
A TRUE STORY ABOUT DATABASE ORCHESTRATION
A TRUE STORY ABOUT DATABASE ORCHESTRATIONA TRUE STORY ABOUT DATABASE ORCHESTRATION
A TRUE STORY ABOUT DATABASE ORCHESTRATION
 
Migrating a build farm from on-prem to AWS
Migrating a build farm from on-prem to AWSMigrating a build farm from on-prem to AWS
Migrating a build farm from on-prem to AWS
 
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
How Teréga Replaces Legacy Data Historians with InfluxDB, AWS and IO-Base
 
Open Source Investments in Mainframe Through the Next Generation - Showcasing...
Open Source Investments in Mainframe Through the Next Generation - Showcasing...Open Source Investments in Mainframe Through the Next Generation - Showcasing...
Open Source Investments in Mainframe Through the Next Generation - Showcasing...
 
Kubernetes in a grown environment and integration into continuous delivery
Kubernetes in a grown environment and integration into continuous deliveryKubernetes in a grown environment and integration into continuous delivery
Kubernetes in a grown environment and integration into continuous delivery
 
The Kubernetes WebLogic revival (part 1)
The Kubernetes WebLogic revival (part 1)The Kubernetes WebLogic revival (part 1)
The Kubernetes WebLogic revival (part 1)
 

More from InfluxData

How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
InfluxData
 
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
InfluxData
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
InfluxData
 
Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022
Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022
Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022
InfluxData
 

More from InfluxData (20)

Announcing InfluxDB Clustered
Announcing InfluxDB ClusteredAnnouncing InfluxDB Clustered
Announcing InfluxDB Clustered
 
Best Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow EcosystemBest Practices for Leveraging the Apache Arrow Ecosystem
Best Practices for Leveraging the Apache Arrow Ecosystem
 
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
How Bevi Uses InfluxDB and Grafana to Improve Predictive Maintenance and Redu...
 
Power Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDBPower Your Predictive Analytics with InfluxDB
Power Your Predictive Analytics with InfluxDB
 
Build an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING StackBuild an Edge-to-Cloud Solution with the MING Stack
Build an Edge-to-Cloud Solution with the MING Stack
 
Meet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using RustMeet the Founders: An Open Discussion About Rewriting Using Rust
Meet the Founders: An Open Discussion About Rewriting Using Rust
 
Introducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud DedicatedIntroducing InfluxDB Cloud Dedicated
Introducing InfluxDB Cloud Dedicated
 
Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB Gain Better Observability with OpenTelemetry and InfluxDB
Gain Better Observability with OpenTelemetry and InfluxDB
 
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
How a Heat Treating Plant Ensures Tight Process Control and Exceptional Quali...
 
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...How Delft University's Engineering Students Make Their EV Formula-Style Race ...
How Delft University's Engineering Students Make Their EV Formula-Style Race ...
 
Introducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage EngineIntroducing InfluxDB’s New Time Series Database Storage Engine
Introducing InfluxDB’s New Time Series Database Storage Engine
 
Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena Start Automating InfluxDB Deployments at the Edge with balena
Start Automating InfluxDB Deployments at the Edge with balena
 
Understanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage EngineUnderstanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage Engine
 
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDBStreamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
Streamline and Scale Out Data Pipelines with Kubernetes, Telegraf, and InfluxDB
 
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
Ward Bowman [PTC] | ThingWorx Long-Term Data Storage with InfluxDB | InfluxDa...
 
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
Scott Anderson [InfluxData] | New & Upcoming Flux Features | InfluxDays 2022
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts | InfluxDays 2022
 
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
Steinkamp, Clifford [InfluxData] | Welcome to InfluxDays 2022 - Day 2 | Influ...
 
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
Steinkamp, Clifford [InfluxData] | Closing Thoughts Day 1 | InfluxDays 2022
 
Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022
Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022
Paul Dix [InfluxData] The Journey of InfluxDB | InfluxDays 2022
 

Recently uploaded

Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
FIDO Alliance
 

Recently uploaded (20)

Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdfFrisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
Frisco Automating Purchase Orders with MuleSoft IDP- May 10th, 2024.pptx.pdf
 
How to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in PakistanHow to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in Pakistan
 
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Generative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdfGenerative AI Use Cases and Applications.pdf
Generative AI Use Cases and Applications.pdf
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 

Tobias Braun [Herrenknecht AG] | Going Underground with InfluxDB | InfluxDays NA 2021

  • 1. Going Underground with InfluxDB Tobias Braun – Software Architect Herrenknecht AG, Germany
  • 2. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Overview - Herrenknecht AG and the presenter - Building an IIoT platform for Herrenknecht AG - Focus on handling of sensor data - InfluxDB as central time series storage - Migration to InfluxDB Enterprise - Current status of the product
  • 3. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Herrenknecht AG - Founded 1977 in Germany - World market leader for tunnel boring machines - For tunnels from 0.1m to 19m - Traffic, utility and mining - Vertical and horizontal - All geologies, even under water - Major share of global market - ~ 5,300 employees - Revenue 2020: 1,007 mio. €
  • 4. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved.
  • 5. © 2021 InfluxData Inc. All Rights Reserved. Tobias Braun Software Architect, Herrenknecht AG Build an IIoT platform with insight into live and historic data for all tunnel boring machines delivered by Herrenknecht AG. Small agile team of 4 developers. Frontend development is supported by an external partner. We have to compromise and gradually improve what we build. E-Mail: braun.tob@herrenknecht.com Software Architect at Herrenknecht for 2 ½ years - More than 20 years of experience in highly available, distributed systems - Worked for: 1&1 IONOS – Web Hosting Core Burda GmbH – E-Commerce Trumpf Werkzeugmaschinen GmbH – IIoT platform for a machine tool manufacturer
  • 6. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Building an IIoT platform for Herrenknecht AG
  • 7. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Tunnel diameter and amount of data correlate 1985 HERA 5.95 m 1996 Sydney 10.70 m 1997 Hamburg 14.20 m 2006 Madrid 15.20 m 2006 Shanghai 15.43 m 2010 Sparvo 15.62 m 2016 Santa Lucia 15.87 m 2013 Hongkong 17.6 m Concept St. Petersburg 19.25 m
  • 8. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Current, correct data is vital for TBM operator
  • 9. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Building an IIoT platform for Herrenknecht AG • Has to support >2000 TBM, several hundred are working simultaneously • Up to 5000 sensors on a single machine • Some sensors with sample rates <= 100ms • Limited bandwidth to the job-site • Machines can be offline for days, weeks, even months • Historic data from 40 years and in different formats (SQL, DBase, even CSV) has to be made available
  • 10. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Building an IIoT platform for Herrenknecht AG • Can be developed and maintained by a small team • Start with the basics: Make sure data is recorded consistently • Development first for TBM, cloud as a second step • Data storage for the next 10-15 years • Robust, flexible architecture
  • 11. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. InfluxDB as central time series storage
  • 12. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Why we chose InfluxDB • Open Source with commercial „Enterprise Edition“ • Windows-Service available (was relevant „back-then“) • Very space efficient storage • Query language similar to SQL with powerful aggregations • Stored in simple, flat files
  • 13. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Where is sensor data stored? • On-site • Control room of the TBM is equipped with ruggedized industrial PCs • Windows operating system with limited computing power and SSD • Data cleansing and storage in InfluxDB OSS • Cloud • InfluxDB Enterprise running on Microsoft Azure
  • 14. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. How is data stored? • Basically one big measurement • Every sensor is a „float“ field • Only few tags are used • Low tag cardinality • Only write values when there was a significant change -> 1-5 GB / month & machine
  • 15. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Simple architecture for InfluxDB in the cloud • When entering the cloud, we used a quick and pragmatic approach based on Kubernetes • One InfluxDB OSS container per TBM • No clustering of InfluxDB itself • Azure File Storage (equivalent to Windows share)
  • 16. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Advantages of this approach • Architecture almost identical to the environment on the TBM, therefore minimal development effort • Isolated impact zones • Cheap pay-per-use storage • File-based backup is included with Azure File Storage • InfluxDB OSS – no licensing cost
  • 17. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. How is data transferred to the cloud? • Historic data is converted to InfluxDB format in batches • Live Data: • Data cleansing is already done on the machine • So we simply mirror all the data from the machine to the cloud • We built a custom synchronization, based on a REST API • Result: Data is identical on the TBM and in the cloud
  • 18. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Migration to InfluxDB Enterprise
  • 19. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. First issues with the pragmatic approach • Scaled well to ~100 InfluxDB containers, • But with >> 100 containers, • the overall system became unreliable • … and response times for even simple queries exploded • … and the infrastructure became prohibitively expensive • Azure File Storage „forgot“ to unlock files in rare cases • with hundreds of containers, even rare cases tend to happen regularly
  • 20. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Decision to migrate to InfluxDB Enterprise • 750 GB of data in 200 databases • At most a couple of minutes of downtime for each TBM • Small team with limited capacity, migration had to be automated • Executed successfully in Q1/2020, took less than two weeks • No read-downtime, write-downtime of less than 60 minutes per TBM
  • 21. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Results after the migration • Reduced TCO of Influx databases by 1/3 • Extensive monitoring and stable operation • Very reliable and reproducable response times for all queries
  • 22. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Status of the product
  • 23. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Successful rollout to first customers
  • 24. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Successful rollout to first customers
  • 25. © 2021 InfluxData Inc. All Rights Reserved. © 2021 InfluxData Inc. All Rights Reserved. Status of the product • Very positive feedback from customers and internal users • Team can concentrate on feature development, little DevOps efforts • InfluxDB OSS on every TBM • InfluxDB Enterprise in the cloud just works, unnoticable in the background
  • 26. © 2021 InfluxData Inc. All Rights Reserved. Questions? braun.tob@herrenknecht.com
  • 27. © 2021 InfluxData Inc. All Rights Reserved. Thank You