Confluent meets Starburst feat: BearingPoint

1. @yourtwitterhandle | developer.confluent.io
What are the best practices to debug client applications
(producers/consumers in general but also Kafka Streams
applications)?
Starting soon…

2. @yourtwitterhandle | developer.confluent.io
What are the best practices to debug client applications
(producers/consumers in general but also Kafka Streams
applications)?

3. Our Partner Technical Sales Enablement offering
Scheduled sessions On-demand
Join us for these live sessions
where our experts will guide you
through sessions of different level
and will be available to answer your
questions. Some examples of
sessions are below:
● Confluent 101: for new starters
● Workshops
● Path to production series
Learn the basics with a guided
experience, at your own pace with our
learning paths on-demand. You will
also find an always growing repository
of more advanced presentations to
dig-deeper. Some examples are below:
● Confluent 10
● Confluent Use Cases
● Positioning Confluent Value
● Confluent Cloud Networking
● … and many more
AskTheExpert /
Workshops
For selected partners, we’ll offer
additional support to:
● Technical Sales workshop
● JIT coaching on spotlight
opportunity
● Build CoE inside partners by
getting people with similar
interest together
● Solution discovery
● Tech Talk
● Q&A

4. @yourtwitterhandle | developer.confluent.io
What are the best practices to debug client applications
(producers/consumers in general but also Kafka Streams
applications)?

5. @yourtwitterhandle | developer.confluent.io
What are the best practices to debug client applications
(producers/consumers in general but also Kafka Streams
applications)?

6. Goal
Partners Tech Talks are webinars where subject matter experts from a Partner talk about a
specific use case or project. The goal of Tech Talks is to provide best practices and
applications insights, along with inspiration, and help you stay up to date about innovations
in confluent ecosystem.

7. Confluent Perspective on
Data Mesh Concepts
7

8. What kind of Data Mesh do you want?
Physical vs. Virtual
8
Physical data movement
(data moves to query)
Fully decentralized→Better autonomy
Data Virtualization Layer
Orders Shipments
Customer
s
Virtualized data (query goes to data)
Centralized query→Easier to
implement low-diversity use cases
Data Engineer
App Developer
Data Scientist
Business Analyst

9. What kind of Data Mesh do you want?
Both?
9
Physical data movement
(data moves to query)
Fully decentralized→Better autonomy
Data Virtualization Layer
Orders Shipments
Customer
s
Virtualized data (query goes to data)
Centralized query→Easier to
implement low-diversity use cases
Data Engineer
App Developer
Data Scientist
Business Analyst

10. Easily build real-time data pipelines to your data
warehouse, database, and data lake
Customer’s cloud env
Data stores
(I.e. PostgreSQL, MongoDB Atlas,
MySQL, DB2, MSSQL, Oracle DB)
Application data
(i.e. Salesforce, ServiceNow,
Github, SAP, Zendesk)
Log data &
messaging systems
(i.e. MQTT, Azure Service Bus, Azure
Event Hubs, Tibco, Solace)
Amazon
Redshift
Source
connectors
Optional: SMT
ksqlDB
Sink
connectors
Optional: SMT
Confluent Cloud
Kafka topics
Data Warehouses
Snowflake Google
BigQuery
Azure Synapse
Analytics
MongoDB
Atlas
Amazon
DynamoDB
Azure
Cosmos DB
Google
BigTable
Databases
Databricks
Delta Lake
Amazon S3 Google Cloud
Storage
Azure Blob
Storage
Data Lakes
/ /
● 200+ connectors
● infinite storage
● unlimited replaying of events
● hybrid & multi-cloud
● real-time analysis using ksqlDB
● complete
● everywhere
● cloud-native

11. Data ownership by
domain
Data as a product Data governed
wherever it is
Data available
everywhere, self
serve
1 2 3 4
The Principles of a Data Mesh

12. Pillars of a Streaming Data Mesh (by Confluent)
Data Ownership
- Have a Confluent
cluster dedicated to a
domain
- Clusters could be
sized to appropriate
scale
- No monolithic Kafka
cluster
Data as a Product
- Cleanses, secures,
governs data with
ksqlDB
- Query data with
ksqlDB.
- Data optimized for
reading.
- ACLs & RBAC
- infinite/tiered storage
Self-serve data
platform
- Automated
deployment and
access process with
REST endpoints,
ksqlDB and Confluent
CLI.
- Cluster linking
Federated
computational
governance
- Confluent governance
tools
- Confluent security
RBAC/ACLs

13. @yourtwitterhandle | developer.confluent.io
What are the best practices to debug client applications
(producers/consumers in general but also Kafka Streams
applications)?

14. Speaker
Michael Stockhammer
Lead Data Mesh & Data
Platform
Gerald Tretter
Lead center of excellence real
time enterprise & event
streaming

15. BearingPoint – Who we are

16. 5,253
employees
50
countries in which
BearingPoint
carries out projects
41
offices in 23
countries
BearingPoint has a global reach.

17. We don't just offer you
consulting services,
we also work with you
to keep an eye on your future.
Capital
& JVs
Joint Ventures
• Joint Venture with IFS (Arcwide)
• Joint Venture with Six
Products
IP Products & Services
• HR People Development Cloud
• Compliance Services FOSS & SAM
• HyperCube / Nitro / Optix
• Emissions Calculator (LogEC)
• ETM.next / DemandSens
• Coding Platform / Application Services / Security Services
• Salesforce / SAP / Azure
Consulting
Market Segments
• Automotive, Industrial
Equipment and
Manufacturing
• Banking & Capital Markets
• Chemicals, Life Sciences
& Resources
• Communications, Media
&
Entertainment
• Consumer Goods & Retail
• Government & Public Sector
• Insurance
• Utilities, Postal &
Transportation
People &
Strategy
Custome
r
&
Growth
Financ
e
& Risk
Operation
s
Technolog
y
Capital
• M&A Advisory
• Investments & Ventures
• Standalone software portfolio

18. 18
Our Event Streaming ecosystem: Experts in all roles from Architects and Business Analysts to Developers and Operations
Center of Excellence – independent event streaming consulting
Overview of our Real-Time Event Streaming practice @ BearingPoint
Our Event Streaming competencies
50+ Event Streaming professionals
10+ Architects working with Event Streaming
15+ Business Analysts working with Event Streaming
6+ years experience in building Event Streaming applications
Technology Stack
Industry-specific
expertise in
• Transport and Logistics
• Postal
• Utilities
• Telecommunications
• Banking
• Insurance
Partnerships, awards and
contributions
• Consulting and SI Partner for Confluent since 2018
(Premier Partner since 2020)
• Contribution to Kafka Meetups in Vienna and Graz
• Digital Skipper Assistant was finalist at the Digital
Business Trends Award 2018
• Several contributions to conferences about event
streaming
• Contribution to Open Source tools in the event
streaming ecosystem
Flagship Projects
Total Recall
Total Recall is an event streaming system for log and metric data. Kafka is used to connect and process
the event streams from different sources and persist the results in elastic search.
Digital Skipper Assistant (DSA)
Digital assistant for skippers that includes waterway maps, routing functionality, ETA calculation,
display of current water levels as well as water level predictions.
Real-Time Transport Monitoring
Dynamic evaluation of progress of a transport along defined goals with the help of generated events
(e.g. ETA, geofencing) to signal the next process steps.
Transformation of Compliance Organization in a Bank
Real-time notification of breaches in compliance checks. Increasing efficiency by reduction of
organizational barriers, workload, data storage & applications. Developing a Far-sighted and future
proof IT-Architecture which improves time2market and responsiveness to new requirements.
Experience
Real-Time Transport Monitoring
Digital Skipper Assistant (DSA)
Registered Consulting and SI Partner for Confluent
Confluent Premium
Partner
Enable ERP host system decomposition (by data streams)
2020 2021
2016 2019
2017 2018 2022
Compliance Organization
Real-Time
Enterprise Service
Portfolio

19. 19
The transformation to a real-time enterprise can only be achieved through
a paradigm shift in our maturity dimensions
REACTIVE
Decisions are made on the basis of
outdated information
Past
PROACTIVE
Decisions can be made proactively at
the right time
Now
1. Technology becomes an 'enabler'
As the technical core, event streaming becomes
the central nervous system for an event-driven
organization
2. Processes go digital
Through automatic and proactive decisions,
processes are redesigned and established
3. Data becomes a product
Events are part of domain-oriented data products
and create added value in various areas of the
company
Events become new
insights in context
Events provide
information at the
right time
Data become a
trigger as an event
4. Competencies enable transformation
Comprehensive design leverages hidden potential
of corporate events and the associated
information gain
5. People & Culture
New roles, tasks and acceptance form the basis for
the transformation to a real-time enterprise
Strategy & Organization defines the direction of
the transformation
Scaling of event thinking in the organization by
anchoring the transformation in the strategy
Realtime
Enterprise
Data
People &
Culture
Strategy &
Organization
Competence
Processes
Technology

20. Pillars of Data Mesh (by Starburst and Confluent)
Data Ownership
- Role management
in self service
platform
- Data is assigned to
individual Domains
Data as a Product
- Data Product is
assigned to
individual domains
- Data is accessed
via SQL no matter
which source
Self-serve data platform
- Platform as enabler
for business to
access data easily
Federated computational
governance
- Work together
beyond silos

21. WHY KYC so slow?
Real-Time KYC & Compl.
checks
Compliance Data Products
Event triggered recalculation
• Know Your Customer processes are to slow
• Data is not available in the right way
What‘s the common
denominator for business?

22. KYC Case Architecture

23. Show case

24. Call to Connect
Contact us if > we love to collaborate
Technical and Organisational
Data Mesh
Architecture consulting Software development

25. Call to Connect
Contact us if > we love to collaborate

26. Q&A