Viele Autos, noch mehr Daten: IoT-Daten-Streaming mit MQTT & Kafka ...

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

1
Best Practices for Streaming IoT Data
with MQTT and Apache Kafka
Kai Waehner
Technology Evangelist
kai.waehner@confluent...

2
Agenda
• Use Case
• Architecture
• Live Demo
• Best Practices
• Next steps

3
Agenda
• Use Case
• Architecture
• Live Demo
• Best Practices
• Next steps

4
Global Automotive Company
Builds Connected Car Infrastructure
Use Cases:
• Connected Car Infrastructure (Cars, Traffic L...

55
Other Components
Kafka Ecosystem
(3) Read Data
TensorFlow I/O
(5) Deploy
Model
(2) Preprocess
Data
(8a)
Alert Car
Real ...

6
Agenda
• Use Case
• Architecture
• Live Demo
• Best Practices
• Next steps

7
Cloud Native
Infrastructure
Benefits
• Scalable
• Flexible
• Agile
• Elastic
• Automated
• Etc.

8
• IoT-specific features for bad network / connectivity
• Widely used (mostly IoT, but also web and mobile
apps via MQTT ...

9
MQTT Trade-Offs
Pros
• Lightweight
• All programming languages supported
• Built for poor connectivity / high latency sc...

10
A Streaming Platform is the Underpinning of an Event-driven Architecture
Ubiquitous
connectivity
Globally scalable plat...

11
Kafka Trade-Offs (from IoT perspective)
Pros
• Stream processing, not just pub/sub
• High throughput
• Large scale
• Hi...

12
(De facto) Standards for Processing IoT Data
A Match Made In Heaven
+ =

13
Agenda
• Use Case
• Architecture
• Live Demo
• Best Practices
• Next steps

14
Advanced Demo

15
MVP

16
Live Demo
End-to-End Integration and Data Processing for
100000 Connected Cars

17
https://github.com/kaiwaehner/hivemq-mqtt-tensorflow-kafka-realtime-iot-machine-learning-training-inference
or
http://b...

18
Agenda
• Use Case
• Architecture
• Live Demo
• Best Practices
• Next steps

19
Typical Journey
Value
Maturity (Investment &
time)
2
Enterprise
Streaming Pilot /
Early Production
Pub + Sub Store Proc...

20
Start Small, but prepare for Scalability from Beginning
1. Use cloud native and scalable
components
• Confluent Platfor...

21
Choose the right tool stack and infrastructure
Understand Trade-Offs and choose the right options for
deployments
• Edg...

22
Separation of concerns
1. Devices
2. Gateway
3. Integration
4. Data Streaming
5. Consumer Apps
Decouple tasks
• Source ...

23
Different data for different use cases
• Database, Data Lake
• Search
• Real time, Near Real Time, Batch
• Streaming, R...

24
Agenda
• Use Case
• Architecture
• Live Demo
• Best Practices
• Next steps

25
The HiveMQ Platform

26
The HiveMQ Platform – Open Source and Enterprise-grade

27
Confluent Platform
Operations and Security
Development & Stream Processing
Support,Services,Training&
Partners
Apache K...

28
Confluent Cloud
Cloud-Native Confluent Platform Fully-Managed Service
Available on the leading public clouds with missi...

29
Next steps…
Try out the demo in 30 minutes:
https://github.com/kaiwaehner/hivemq-mqtt-tensorflow-kafka-realtime-iot-mac...

30
Questions? Feedback?
Kai Waehner
Technology Evangelist
kai.waehner@confluent.io
LinkedIn
@KaiWaehner
www.confluent.io
w...

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

2024 Trend Updates: What Really Wor... by Search Engine Jou... 1036216 views
Storytelling For The Web: Integrate... by Chiara Aliotta 948112 views
Artificial Intelligence, Data and C... by OECD Directorate ... 870219 views
How to Leverage AI to Boost Employe... by SocialHRCamp 365506 views
2024 State of Marketing Report – by... by Marius Sescu 207694 views
Everything You Need To Know About C... by Expeed Software 227895 views

View on Slideshare

1 of 30 Ad

View on Slideshare

1 of 30 Ad