Mit Streaming die Brücken zum Erfolg bauen

Mit Streaming die Brücken zum
Erfolg bauen
Wie eine event-getriebene Architektur einen nahtlosen 360-
Datenfluss ermöglicht
Henrik Berner
Projektleiter
Mercedes-Benz AG
Kai Waehner
Field CTO
Confluent

The Rise of Data in Motion in the Automotive Industry
Use Cases, Architectures and Examples powered by Data Streaming
Kai Waehner
Field CTO
kai.waehner@confluent.io
confluent.io
kai-waehner.de
linkedin.com/in/kaiwaehner
@KaiWaehner

@KaiWaehner - www.kai-waehner.de – Data in Motion Automotive and Manufacturing
Real-time Data beats Slow Data.
Transportation
Predictive
maintenance
Driver-rider match
ETA updates
Banking
Instant payments
Fraud detection
Mobile applications /
customer experience
Retail
Real-time inventory
Real-time POS
reporting
Personalization
Entertainment
Real-time
recommendations
Personalized
news feed
In-car purchases

This is a fundamental paradigm shift...
4
Infrastructure
as code
Data in motion
as continuous
streams of events
Future of the
datacenter
Future of data
Cloud
Data
Streaming

What is Data Streaming?

‘Event’ is what happens in your business
Transportation
Tire-pressure sensor in Carol’s car detected low tire-pressure at 5:11am.
Kafka
Mobile Payment
Alice paid $250 for the auxiliary heating subscription on Friday at 7:34pm.
Kafka
Dealership
John booked the next maintenance service for his Mercedes via the mobile app.
Kafka

Data in Motion in the Automotive Industry
Your Business as Streams of Events, powered by Apache Kafka
Manufacturing
Event streams are stored for
reuse and with high availability.
Shipping
Events are processed in real-time
as soon as they happen.
Car
Configurator
Reporting
Add new use cases easily by
tapping into existing
streams.
Orders
Event-driven apps and services
communicate through streams.

Apache Kafka is the Platform for Data in Motion
MES
ERP
Sensors
Mobile
Customer 360
Real-time
Alerting System
Data
warehouse
Producers
Consumers
Streams and storage of real time events
Stream
processing
apps
Connectors
Connectors
Stream
processing
apps
Supplier
Alert
Forecast
Inventory Customer
Order
8

Kafka is a cloud-native streaming middleware!
à More than just data ingestion or message queue
DWH
APP
STREAM
PROCESSING
CONNECTORS
ksqlDB
KStreams
APP
Streaming ETL
Data Processing
Real-time Analytics
Stateless and Stateful
Business Applications
Fully-managed
Pipelines
Connectivity to
Data Infrastructure,
SaaS, AI/ML
Data Governance
Connectivity
Filtering and Routing
Change Data Capture
Built-in Scale and Fault Tolerance
Oracle
DB
ORACLE CDC
SOURCE
PREMIUM
CONNECTOR
Real-time Data Sharing
across Hybrid and Multi-Cloud
Storage
Backpressure Handling
Slow Consumers
Replayability

IoT and Digital Twin

BMW Group
Mission-critical workloads at the edge and in the cloud
• Why Kafka? Decoupling. Transparency. Innovation.
• Why Confluent? Stability is key in manufacturing
• Decoupling between logistics and production systems
• Integration from edge to serverless Confluent Cloud on Azure
• Use case
• Logistics and supply chain in global plants
• Right stock in place (physically and in ERP systems like SAP)
• Just in time, just in sequence
• Lot of critical applications
11
Jay Kreps, Confluent CEO
Felix Böhm, BMW Plant Digitalization and Cloud Transformation
Keynote at Kafka Summit Eurpoe 2021:
https://www.youtube.com/watch?v=3cG2ud7TRs4
(My Notes from the BMW Keynote at Kafka Summit EU 2021)

Mercury Systems – Digital Twin
Global technology company serving the aerospace and
defense industry
Digital Thread
• Bring together design and product information across
the product life cycle
• Mendix based portal with combined data from
PLM/ERP/MES
• Confluent for cloud-native and reliable real-time event
streaming across applications
• Apply AI and ML techniques
• All data in one location viewed with common tools
Benefits
• Faster time to market
• Ability to scale
• Improved innovation pipeline and reduced cost of failure
https://www.confluent.io/events/kafka-summit-americas-2021/driving-a-
digital-thread-program-in-manufacturing-with-apache-kafka/

Mobility Services

https://www.confluent.io/thank-you/uber-kafka-uber-worlds-realtime-transit-infrastructure/
https://www.confluent.io/thank-you/stream-processing-kafka-uber/
Trillions of messages and
multiple petabytes of data per day

FREE NOW
Stateful stream processing with Confluent Cloud, Kafka Connect, Kafka Streams, Schema Registry
Cloud-native application elasticity and scalability leveraging Kafka and Kubernetes capabilities
Use cases: Dynamic pricing, fraud detection, real-time analytics for marketing campaigns, etc.
Various information about the trip, location and business performance
15
https://www.confluent.io/events/kafka-summit-europe-2021/development-of-
dynamic-pricing-for-tours-using-real-time-data-feeds/

Fraud Detection @ Grab
GrabDefence SaaS service build with Confluent Cloud, Kafka Streams and ML for stateful stream processing
Billions of fraud and safety detections performed daily for millions of transactions (1.6% is lost in fraud in Southeast Asia)
Data science and engineering platform to search for anomalous and suspicious transactions and identifying
high-risk individuals
Example: An individual who pretends to be both the driver and passenger, and makes cashless payments to get
promotions

Cross-Company Stream Exchange for Data Sharing
Streaming Replication and API Management
Cluster Linking
Tier 1 Mobility
Service
Streaming integration
between companies
API Management
(REST et al) is not appropriate
for streaming data
OEM

Here Technologies
Captures location content such as road networks, buildings, parks and traffic patterns
Sells or licenses mapping content, along with map related navigation and location services to other businesses
https://developer.here.com/documentation/data-client-library/dev_guide/client/direct-kafka.html

Omnichannel and Aftersales

Porsche’s Streamzilla
A central platform strategy across data centers, clouds, and regions
20
https://www.confluent.io/events/kafka-summit-europe-2021/developing-a-custom-kafka-connector-make-it-shine/

‘My Porsche’
A digital service platform for customers, fans, and enthusiasts
21
https://medium.com/porschedev

Omnichannel Retail
Time
P
App1 App2
App3
Sales Talk on site in
Car Dealership
Right now
Location-based
Customer Action
Customer 360
(Website, Mobile App, On Site in Store, In-Car)
Car Configurator
10 and 8 days ago
Context-specific
Marketing Campaign
90 and 60 days ago

Global Data Streaming
Streaming Replication between Kafka Clusters
Bridge to Databases, Data Lakes, Apps, APIs, SaaS
Aggregate Small Footprint
Edge Deployments with
Replication (Aggregation)
Simplify Disaster Recovery
Operations with
Multi-Region Clusters
with RPO=0 and RTO~0
Stream Data Globally with
Replication and Cluster Linking
23

Ship-Shore Highway – Swimming Retail Stores
https://www.confluent.io/kafka-summit-lon19/seamless-guest-experience-with-kafka-streams/

Car Engine Car Self-driving Car
Confluent completes Apache Kafka. Cloud-native. Everywhere.

Kai Waehner
Field CTO
kai.waehner@confluent.io
confluent.io
linkedin.com/in/kaiwaehner
@KaiWaehner
kai-waehner.de
Questions? Feedback?
Let’s connect!

Henrik Berner, Mercedes-Benz AG
Building the streaming bridges to
success: How an event-driven
architecture is enabling a seamless
360 data flow

Internal
KAFKA Platform – Timeline
Q3/2018
Proove of Concept
Approval
Q1/2020
Production
GoLive
> 50 System on
Production
Cross-Division Usage
Q3/2021
First Production Usage
8 Systems
Q2/2019
Today
~ 70 System on
Production
Cross-Divsional Usage
Hybrid KAFKA
Cluster Replication
Mercedes-Benz: Building the streaming bridges to success | September 2022 31

Internal
Platform Features (Excerp)
• Admin UI
• KAFKA Core Components
• Confluent Features
Connectors
Streams
Schema Registry
Rest Proxy
• Customer Connector Logs
• Automated Self-Services
• Data Lake Connectivity
• Developer Documentation
Multi-Tenancy Concept
• Fast Onboarding
• Full Administration Access with Namespace Concept
• Role & Right Management on e.g. Topic-Level
KAFKA Platform – KAFKA out of the box
Full Managed Kafka Platform Ready to Use
32

Internal
Mercedes-Benz: Building the streaming bridges to success | September 2022
Mercedes-Benz Streaming Use
Cases
• Data Lake Streaming
• 360 Data & Data Replication
(Hybrid Scenario)
33

Internal
Seamless Data Streaming to MBC Data Lake
• Defined Onboarding Processes
• Prebuild Connector by Kafka Platfrom Team
• Re-Usage of Connector by Platfrom Customers
• Secured Connection to Data Lake
• Integrated PW Vault
• automated rolling Update of PW every 90d
• From Batch Jobs to Data Streaming
• Reduction of Different Formats
• Reduction of complex ETL Processes
• Data Streaming of 3 different Sources via KAFKA
Platform
Source: Mercedes-Benz
34

Internal
360 Data & Data Replication (Hybrid Scenario)
TOPIC 1
TOPIC 2
CRM System
Event Integration Layer
KAFKA Platform supporting 360 Customer View
• Automated & scripted Data Streaming e.g. connecting new Sources
• Data Streaming is supporting permanent completion of data
• Changes synchronization in Near Real Time
Seamless Data Exchange - Cluster Replication
• Data OnPrem also requested in Cloud Custer
• Request to Mirror
• Topics from OnPrem Cluster to Cloud Cluster
• Topics from Cloud Cluster to OnPrem Cluster
System 1 System 2
Seamless 360 Data – available in any Cluster - Everywhere
Application Layer

Internal
Mercedes-Benz: Building the streaming bridges to success | September 2022
Thank you for your attention
Questions?
36

Mit Streaming die Brücken zum Erfolg bauen

More Related Content

What's hot

Similar to Mit Streaming die Brücken zum Erfolg bauen

More from confluent

Recently uploaded

Mit Streaming die Brücken zum Erfolg bauen