Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Christoph Bussler
June 23rd, 2020
IoT Event Processing and
Analytics with InfluxDB in
Google Cloud
Agenda
Energy
Sector
Energy sector
use case
analysis
Life cycle
of an
IoT event
Google Cloud
overview
1 2 3 4 5 6
InfluxDB ...
● Production
○ Energy production systems monitoring and anomaly detection
■ Oil, gas, wind, hydro, solar and others
● Dist...
● Event collection
○ From production systems
○ From consumption locations
○ Absence check: requires static
equipment inven...
Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dyn...
Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dyn...
Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dyn...
Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dyn...
Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dyn...
Life Cycle of an IoT Event
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dyn...
Google Cloud: Regions and Zones
https://cloud.google.com/about/locations
Google Cloud: Global Network
https://cloud.google.com/about/locations#network
Google Cloud: Cloud Services
https://cloud.google.com/-select“Products”
Google Cloud: Cloud Services
https://github.com/gregsramblings/google-cloud-4-words/bl
ob/master/DarkPoster-lowres.png
Google Cloud: Select Services
BigQuery (https://cloud.google.com/bigquery/)
● Cloud native analytics database
● Columnar -...
InfluxDB - TICK Architecture
https://www.influxdata.com/time-series-platform/
Cloud Native IoT Event Processing Architecture
Gateway
Cloud Native IoT Event Processing Architecture
Gateway
Cloud IoT
Core
Cloud Native IoT Event Processing Architecture
Gateway
Cloud IoT
Core
TICK stack
Cloud Native IoT Event Processing Architecture
Gateway
Offline Analytics
Cloud IoT
Core
TICK stack
Cloud
Pub/Sub
Cloud
Datafl...
Cloud Native IoT Event Processing Architecture
Gateway
Offline Analytics
Application Systems
Cloud IoT
Core
TICK stack
Cloud...
Cloud Native IoT Event Processing Architecture
Gateway
Offline Analytics
Application Systems
Machine Learning
Cloud IoT
Core...
Cloud Native IoT Event Processing Architecture
Gateway
Offline Analytics
Application Systems
Machine Learning
Archiving
Clou...
Summary
Gateway
Ingest
Time Series
Database
Real-time
Analytics
Offline
Analytics
Static data
sources
Dynamic data
sources
A...
Thank You!
Upcoming SlideShare
Loading in …5
×

Christoph Bussler [Google Cloud] | IoT Event Processing and Analytics with InfluxDB in Google Cloud | InfluxDays Virtual Experience London 2020

70 views

Published on

The presentation introduces a Google Cloud native architecture for collecting, processing, analyzing and archiving events from IoT devices, vehicles as well as upstream software systems. InfluxDB and its connection to global native Google Cloud services like BigQuery or Cloud Machine Learning Engine as well as Kubernetes is at the center of the architecture. The architecture demonstrates how access to global scaling cloud services address use cases from the Energy Sector.

Published in: Technology
  • Be the first to comment

Christoph Bussler [Google Cloud] | IoT Event Processing and Analytics with InfluxDB in Google Cloud | InfluxDays Virtual Experience London 2020

  1. 1. Christoph Bussler June 23rd, 2020 IoT Event Processing and Analytics with InfluxDB in Google Cloud
  2. 2. Agenda Energy Sector Energy sector use case analysis Life cycle of an IoT event Google Cloud overview 1 2 3 4 5 6 InfluxDB TICK architecture Google Cloud native IoT event processing architecture
  3. 3. ● Production ○ Energy production systems monitoring and anomaly detection ■ Oil, gas, wind, hydro, solar and others ● Distribution ○ Smart grid: maintaining an equilibrium across energy supply and demand ■ Renewable and non-renewable energy production/demand forecasting ■ https://towardsdatascience.com/how-machine-learning-can-transform-the-ener gy-industry-caaa965e282a (devices: synchrophasers) ● Consumption ○ Fleet performance optimization (cars, trucks, planes, etc.) ○ Commercial manufacturing sites, office buildings ○ Public infrastructure ○ Private households Use Cases: Energy Sector
  4. 4. ● Event collection ○ From production systems ○ From consumption locations ○ Absence check: requires static equipment inventory data ● Event monitoring ○ Outages ○ Trends ○ Anomalies ■ Was there actually an outage? Use Cases: Analysis ● Forecasting / prediction ○ Combination of ■ Current events ■ Historic events ■ Non-event data like models, weather data, road conditions, etc. ● Off-line analysis ○ Combination with non-event data ● Archiving ○ Long duration analysis ○ Regulatory requirements
  5. 5. Life Cycle of an IoT Event Gateway Ingest Time Series Database Real-time Analytics Offline Analytics Static data sources Dynamic data sources Application Systems Short / Long term archive
  6. 6. Life Cycle of an IoT Event Gateway Ingest Time Series Database Real-time Analytics Offline Analytics Static data sources Dynamic data sources Application Systems Short / Long term archive
  7. 7. Life Cycle of an IoT Event Gateway Ingest Time Series Database Real-time Analytics Offline Analytics Static data sources Dynamic data sources Application Systems Short / Long term archive
  8. 8. Life Cycle of an IoT Event Gateway Ingest Time Series Database Real-time Analytics Offline Analytics Static data sources Dynamic data sources Application Systems Short / Long term archive
  9. 9. Life Cycle of an IoT Event Gateway Ingest Time Series Database Real-time Analytics Offline Analytics Static data sources Dynamic data sources Application Systems Short / Long term archive
  10. 10. Life Cycle of an IoT Event Gateway Ingest Time Series Database Real-time Analytics Offline Analytics Static data sources Dynamic data sources Application Systems Short / Long term archive
  11. 11. Google Cloud: Regions and Zones https://cloud.google.com/about/locations
  12. 12. Google Cloud: Global Network https://cloud.google.com/about/locations#network
  13. 13. Google Cloud: Cloud Services https://cloud.google.com/-select“Products”
  14. 14. Google Cloud: Cloud Services https://github.com/gregsramblings/google-cloud-4-words/bl ob/master/DarkPoster-lowres.png
  15. 15. Google Cloud: Select Services BigQuery (https://cloud.google.com/bigquery/) ● Cloud native analytics database ● Columnar - SQL - large scale (PB) - multi-regional service Cloud Spanner (https://cloud.google.com/spanner/) ● Cloud native global relational database ● Relational - SQL - consistent - linear scaling - multi-regional and intercontinental service Coldline Storage (https://cloud.google.com/storage/docs/storage-classes) ● Cloud native storage - long term storage Cloud Machine Learning Engine (https://cloud.google.com/ai-platform) ● Cloud native ML engine - online and batch predictions
  16. 16. InfluxDB - TICK Architecture https://www.influxdata.com/time-series-platform/
  17. 17. Cloud Native IoT Event Processing Architecture Gateway
  18. 18. Cloud Native IoT Event Processing Architecture Gateway Cloud IoT Core
  19. 19. Cloud Native IoT Event Processing Architecture Gateway Cloud IoT Core TICK stack
  20. 20. Cloud Native IoT Event Processing Architecture Gateway Offline Analytics Cloud IoT Core TICK stack Cloud Pub/Sub Cloud Dataflow Big Query Cloud SQL Cloud Dataflow
  21. 21. Cloud Native IoT Event Processing Architecture Gateway Offline Analytics Application Systems Cloud IoT Core TICK stack Cloud Pub/Sub Cloud Dataflow Big Query Cloud SQL Cloud Dataflow Cloud Pub/Sub Cloud Dataflow Cloud Spanner Kubernet es Engine
  22. 22. Cloud Native IoT Event Processing Architecture Gateway Offline Analytics Application Systems Machine Learning Cloud IoT Core TICK stack Cloud Pub/Sub Cloud Dataflow Big Query Cloud SQL Cloud Dataflow Cloud Pub/Sub Cloud Dataflow Cloud Spanner Kubernet es Engine Cloud Pub/Sub Cloud Run Cloud Storage AI Platform
  23. 23. Cloud Native IoT Event Processing Architecture Gateway Offline Analytics Application Systems Machine Learning Archiving Cloud IoT Core TICK stack Cloud Pub/Sub Cloud Dataflow Big Query Cloud SQL Cloud Dataflow Cloud Pub/Sub Cloud Dataflow Cloud Spanner Kubernet es Engine Cloud Pub/Sub Cloud Run Cloud Storage AI Platform Cloud Run Cloud Storage Cloud Schedul er
  24. 24. Summary Gateway Ingest Time Series Database Real-time Analytics Offline Analytics Static data sources Dynamic data sources Application Systems Short / Long term archive
  25. 25. Thank You!

×