SlideShare a Scribd company logo
1 of 14
Download to read offline
How to Discover, Visualize,
Catalog, Share and Reuse
your Kafka Event Streams
Jonathan Schabowsky
Senior Architect, CTO, Solace
Agenda
2
• Your growing Kafka event streams
• Flying blind
• Event portal: an API portal for event streams
• Demo: How to discover, visualize, catalog, share and reuse
your Kafka event streams
• Summary
© Solace
2
Does Your Organization Look Like this?
3
© Solace
Data Stream Flow
Legacy Application Infrastructure
….
?
?
?
JSON, XML, Plain Text…
IoT/Device Management
Producers
Consumers
Connectors
Stream
Processors
Avro
You’re Flying Blind…
Where do you discover
event streams for reuse?
Why does a given
event stream exist?
What topic do you use
to subscribe to it?
What changed?
How do you determine the
data structure of its payload?
Who should have access to it?
Who made the
latest revision?
Who can tell you more
about an event?
Is your change backward
compatible?
Who is impacted
by your change?
Does it comply with
security policy?
4
© Solace
Some Try Solutions Not Intended
for Event Stream Management
• It's a pain…
– No single place to discover
and share/reuse event streams
– No best practices guidance
• You’re breaking systems and
don’t know who changed
what???
– Suffer from changes with unintended
consequences
– Hard to audit changes when data is
stored in different tools
Brittle,
complex,
manual
solutions
Custom
Developed
Catalog
PowerPoint
API
Management
Spread
sheets
GitConfluence
5
© Solace
API Management
Solved this for REST
6
© Solace
They answer Who,
What, When,
Where, Why and
How For RESTful
APIs…
but what about
event streams?
Inspired by API Management Platforms
7
© Solace
API
Gateway
Documentation
• Discover
Registration
• Govern
Analysis
• Improve
• Monetize
Community
• Collaborate
Architects
and
Developers
(Internal, Partner
and Third-Party)
Productivity
• Generate Code
API
Portal
Sinks
“While most organizations have
basic event processing infrastructure
- such as notification services,
message buses, and event brokers -
many don’t have the high-level
productivity tooling that helps
developers design, develop, test and
manage event-centric applications.
Nor do they have tools for
governing, publishing and managing
event-based interfaces. The market
for these tools is sparse.”
Gartner “Top 3 Trends in Application Architecture That Enable
Digital Business” Anne Thomas, Yefim Natis, Mark O’Neill, 28 Oct 2019
8
© Solace
Event
Broker
Security
Persistence
Protocol
Translation
One-to-Many
Distribution
Source
Runtime
We Need an Event Portal
for Asynchronous Interactions
Design, Develop
and Manage
Event
Portal
Documentation
• Discover
Registration
• Govern
Analysis
• Improve
• Monetize
Community
• Collaborate
Productivity
• Generate Code (code gen)
(spec)
9
© Solace
Single place to design, create, catalog, visualize, discover, share,
secure and manage all events within your ecosystem
Introducing Event Portal
Event Portal
Mainframe/
SAP
Distributed
Apps IoTMicro
services
Server
Less
Event Brokers
Event Portal
How Users Interact with Event Portal
10
Event
Management
(Design Time)
© Solace
Your Event-Driven
Enterprise
B2B/B2C
Ecosystem
IoTMicro-
services
Server-
less
Generate
Code
REST
API
Existing Tools
• Schema registries
• Data governance
• CI/CD pipelines
Customers
& Partners
• Discover and
consume events
in real-time
Architects
• Discover, create, update, review
apps, events and payload
schemas
• Map payload schemas to events;
and events to apps
• Govern event consumption
across app domain boundaries
Developers
• Collaborate with
architects on design
• Export AsyncAPI app
contracts for each app
• Generate and implement
event-driven apps that
provide business value
Data Scientists
• Understand
event-driven data
• Discover new insights
by combining events
Demo
11
© Solace
Benefits of an Event Portal
12
Solace
Your business can…
• Understand who is consuming which
events, how much, for what
• Create value-added services and
gain insights by combining events
• Make it easy for users across app
teams, LoBs and partner ecosystem
to find and reuse events
• Monetize popular or particular
high-value event streams
Reuse/Sharing
Architects & developers can:
• Collaborate on events
and event-driven apps
• Visually see event-driven
interactions to understand
impact of changes
• Low Code: Generate app code by
exporting AsyncAPI definitions
• Benefit from best practices &
consistently apply conventions
Productivity
CDO/data governance can:
• Understand where data is
coming from (lineage) and going
• Track changes and audit for
deviations while promoting apps
and dependencies though Dev,
Test, QA, and Prod
• Use application/event
relationships to create security
policies and validate compliance
with schema
Control
So, if you…
Have a lot of event-
driven applications
13
© Solace
You need an event portal!
Have (or need
to define) a lot of
event streams
Want to
manage events
as easily as you
manage your APIs
Event Portal = API Portal for Events/Streams
One tool to design,
create, catalog,
visualize, discover,
share, secure and
manage all of your
events and streams
(including Kafka and any
other event broker)
14
© Solace
Sign up for free!
• About data
and apps, NOT
managing Kafka
infrastructure
• High-level
productivity
tooling to help
you fully
embrace EDA
• Governance
of apps, event
and data

More Related Content

What's hot

EDA Governance Model: a multicloud approach based on GitOps | Alejandro Alija...
EDA Governance Model: a multicloud approach based on GitOps | Alejandro Alija...EDA Governance Model: a multicloud approach based on GitOps | Alejandro Alija...
EDA Governance Model: a multicloud approach based on GitOps | Alejandro Alija...HostedbyConfluent
 
Accelerating Innovation with Apache Kafka, Heikki Nousiainen | Heikki Nousiai...
Accelerating Innovation with Apache Kafka, Heikki Nousiainen | Heikki Nousiai...Accelerating Innovation with Apache Kafka, Heikki Nousiainen | Heikki Nousiai...
Accelerating Innovation with Apache Kafka, Heikki Nousiainen | Heikki Nousiai...HostedbyConfluent
 
How a distributed graph analytics platform uses Apache Kafka for data ingesti...
How a distributed graph analytics platform uses Apache Kafka for data ingesti...How a distributed graph analytics platform uses Apache Kafka for data ingesti...
How a distributed graph analytics platform uses Apache Kafka for data ingesti...HostedbyConfluent
 
Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...
Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...
Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...HostedbyConfluent
 
Server Sent Events using Reactive Kafka and Spring Web flux | Gagan Solur Ven...
Server Sent Events using Reactive Kafka and Spring Web flux | Gagan Solur Ven...Server Sent Events using Reactive Kafka and Spring Web flux | Gagan Solur Ven...
Server Sent Events using Reactive Kafka and Spring Web flux | Gagan Solur Ven...HostedbyConfluent
 
Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...
Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...
Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...HostedbyConfluent
 
Low-latency real-time data processing at giga-scale with Kafka | John DesJard...
Low-latency real-time data processing at giga-scale with Kafka | John DesJard...Low-latency real-time data processing at giga-scale with Kafka | John DesJard...
Low-latency real-time data processing at giga-scale with Kafka | John DesJard...HostedbyConfluent
 
Mainframe Integration, Offloading and Replacement with Apache Kafka | Kai Wae...
Mainframe Integration, Offloading and Replacement with Apache Kafka | Kai Wae...Mainframe Integration, Offloading and Replacement with Apache Kafka | Kai Wae...
Mainframe Integration, Offloading and Replacement with Apache Kafka | Kai Wae...HostedbyConfluent
 
Streaming data in the cloud with Confluent and MongoDB Atlas | Robert Waters,...
Streaming data in the cloud with Confluent and MongoDB Atlas | Robert Waters,...Streaming data in the cloud with Confluent and MongoDB Atlas | Robert Waters,...
Streaming data in the cloud with Confluent and MongoDB Atlas | Robert Waters,...HostedbyConfluent
 
5 lessons learned for successful migration to Confluent cloud | Natan Silinit...
5 lessons learned for successful migration to Confluent cloud | Natan Silinit...5 lessons learned for successful migration to Confluent cloud | Natan Silinit...
5 lessons learned for successful migration to Confluent cloud | Natan Silinit...HostedbyConfluent
 
Moving 150 TB of data resiliently on Kafka With Quorum Controller on Kubernet...
Moving 150 TB of data resiliently on Kafka With Quorum Controller on Kubernet...Moving 150 TB of data resiliently on Kafka With Quorum Controller on Kubernet...
Moving 150 TB of data resiliently on Kafka With Quorum Controller on Kubernet...HostedbyConfluent
 
Building a Modern, Scalable Cyber Intelligence Platform with Apache Kafka | J...
Building a Modern, Scalable Cyber Intelligence Platform with Apache Kafka | J...Building a Modern, Scalable Cyber Intelligence Platform with Apache Kafka | J...
Building a Modern, Scalable Cyber Intelligence Platform with Apache Kafka | J...HostedbyConfluent
 
Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...
Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...
Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...HostedbyConfluent
 
Streaming Data Analytics with ksqlDB and Superset | Robert Stolz, Preset
Streaming Data Analytics with ksqlDB and Superset | Robert Stolz, PresetStreaming Data Analytics with ksqlDB and Superset | Robert Stolz, Preset
Streaming Data Analytics with ksqlDB and Superset | Robert Stolz, PresetHostedbyConfluent
 
Kubernetes connectivity to Cloud Native Kafka | Evan Shortiss and Hugo Guerre...
Kubernetes connectivity to Cloud Native Kafka | Evan Shortiss and Hugo Guerre...Kubernetes connectivity to Cloud Native Kafka | Evan Shortiss and Hugo Guerre...
Kubernetes connectivity to Cloud Native Kafka | Evan Shortiss and Hugo Guerre...HostedbyConfluent
 
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...HostedbyConfluent
 
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...HostedbyConfluent
 
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...HostedbyConfluent
 
Developing custom transformation in the Kafka connect to minimize data redund...
Developing custom transformation in the Kafka connect to minimize data redund...Developing custom transformation in the Kafka connect to minimize data redund...
Developing custom transformation in the Kafka connect to minimize data redund...HostedbyConfluent
 
Kafka Summit NYC 2017 - Achieving Predictability and Compliance with BNY Mell...
Kafka Summit NYC 2017 - Achieving Predictability and Compliance with BNY Mell...Kafka Summit NYC 2017 - Achieving Predictability and Compliance with BNY Mell...
Kafka Summit NYC 2017 - Achieving Predictability and Compliance with BNY Mell...confluent
 

What's hot (20)

EDA Governance Model: a multicloud approach based on GitOps | Alejandro Alija...
EDA Governance Model: a multicloud approach based on GitOps | Alejandro Alija...EDA Governance Model: a multicloud approach based on GitOps | Alejandro Alija...
EDA Governance Model: a multicloud approach based on GitOps | Alejandro Alija...
 
Accelerating Innovation with Apache Kafka, Heikki Nousiainen | Heikki Nousiai...
Accelerating Innovation with Apache Kafka, Heikki Nousiainen | Heikki Nousiai...Accelerating Innovation with Apache Kafka, Heikki Nousiainen | Heikki Nousiai...
Accelerating Innovation with Apache Kafka, Heikki Nousiainen | Heikki Nousiai...
 
How a distributed graph analytics platform uses Apache Kafka for data ingesti...
How a distributed graph analytics platform uses Apache Kafka for data ingesti...How a distributed graph analytics platform uses Apache Kafka for data ingesti...
How a distributed graph analytics platform uses Apache Kafka for data ingesti...
 
Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...
Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...
Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 20...
 
Server Sent Events using Reactive Kafka and Spring Web flux | Gagan Solur Ven...
Server Sent Events using Reactive Kafka and Spring Web flux | Gagan Solur Ven...Server Sent Events using Reactive Kafka and Spring Web flux | Gagan Solur Ven...
Server Sent Events using Reactive Kafka and Spring Web flux | Gagan Solur Ven...
 
Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...
Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...
Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...
 
Low-latency real-time data processing at giga-scale with Kafka | John DesJard...
Low-latency real-time data processing at giga-scale with Kafka | John DesJard...Low-latency real-time data processing at giga-scale with Kafka | John DesJard...
Low-latency real-time data processing at giga-scale with Kafka | John DesJard...
 
Mainframe Integration, Offloading and Replacement with Apache Kafka | Kai Wae...
Mainframe Integration, Offloading and Replacement with Apache Kafka | Kai Wae...Mainframe Integration, Offloading and Replacement with Apache Kafka | Kai Wae...
Mainframe Integration, Offloading and Replacement with Apache Kafka | Kai Wae...
 
Streaming data in the cloud with Confluent and MongoDB Atlas | Robert Waters,...
Streaming data in the cloud with Confluent and MongoDB Atlas | Robert Waters,...Streaming data in the cloud with Confluent and MongoDB Atlas | Robert Waters,...
Streaming data in the cloud with Confluent and MongoDB Atlas | Robert Waters,...
 
5 lessons learned for successful migration to Confluent cloud | Natan Silinit...
5 lessons learned for successful migration to Confluent cloud | Natan Silinit...5 lessons learned for successful migration to Confluent cloud | Natan Silinit...
5 lessons learned for successful migration to Confluent cloud | Natan Silinit...
 
Moving 150 TB of data resiliently on Kafka With Quorum Controller on Kubernet...
Moving 150 TB of data resiliently on Kafka With Quorum Controller on Kubernet...Moving 150 TB of data resiliently on Kafka With Quorum Controller on Kubernet...
Moving 150 TB of data resiliently on Kafka With Quorum Controller on Kubernet...
 
Building a Modern, Scalable Cyber Intelligence Platform with Apache Kafka | J...
Building a Modern, Scalable Cyber Intelligence Platform with Apache Kafka | J...Building a Modern, Scalable Cyber Intelligence Platform with Apache Kafka | J...
Building a Modern, Scalable Cyber Intelligence Platform with Apache Kafka | J...
 
Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...
Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...
Exposing and Controlling Kafka Event Streaming with Kong Konnect Enterprise |...
 
Streaming Data Analytics with ksqlDB and Superset | Robert Stolz, Preset
Streaming Data Analytics with ksqlDB and Superset | Robert Stolz, PresetStreaming Data Analytics with ksqlDB and Superset | Robert Stolz, Preset
Streaming Data Analytics with ksqlDB and Superset | Robert Stolz, Preset
 
Kubernetes connectivity to Cloud Native Kafka | Evan Shortiss and Hugo Guerre...
Kubernetes connectivity to Cloud Native Kafka | Evan Shortiss and Hugo Guerre...Kubernetes connectivity to Cloud Native Kafka | Evan Shortiss and Hugo Guerre...
Kubernetes connectivity to Cloud Native Kafka | Evan Shortiss and Hugo Guerre...
 
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
 
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...
 
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
 
Developing custom transformation in the Kafka connect to minimize data redund...
Developing custom transformation in the Kafka connect to minimize data redund...Developing custom transformation in the Kafka connect to minimize data redund...
Developing custom transformation in the Kafka connect to minimize data redund...
 
Kafka Summit NYC 2017 - Achieving Predictability and Compliance with BNY Mell...
Kafka Summit NYC 2017 - Achieving Predictability and Compliance with BNY Mell...Kafka Summit NYC 2017 - Achieving Predictability and Compliance with BNY Mell...
Kafka Summit NYC 2017 - Achieving Predictability and Compliance with BNY Mell...
 

Similar to How to Discover, Visualize, Catalog, Share and Reuse your Kafka Streams (Jonathan Schabowsky, Solace) Kafka Summit 2020

How Curing “Kafkatosis” Can Improve Your Stream Management and Governance
How Curing “Kafkatosis” Can Improve Your Stream Management and GovernanceHow Curing “Kafkatosis” Can Improve Your Stream Management and Governance
How Curing “Kafkatosis” Can Improve Your Stream Management and GovernanceHostedbyConfluent
 
Mesh-ing around with Streams across the Enterprise | Phil Scanlon, Solace
Mesh-ing around with Streams across the Enterprise | Phil Scanlon, SolaceMesh-ing around with Streams across the Enterprise | Phil Scanlon, Solace
Mesh-ing around with Streams across the Enterprise | Phil Scanlon, SolaceHostedbyConfluent
 
Solace Connect NYC 2019 - Enabling the Event-Driven Enterprise
Solace Connect NYC 2019 - Enabling the Event-Driven EnterpriseSolace Connect NYC 2019 - Enabling the Event-Driven Enterprise
Solace Connect NYC 2019 - Enabling the Event-Driven EnterpriseSolace
 
Creating an Event Backbone for the Hybrid Cloud
Creating an Event Backbone for the Hybrid CloudCreating an Event Backbone for the Hybrid Cloud
Creating an Event Backbone for the Hybrid CloudSolace
 
Enabling Event Driven Architecture with PubSub+
Enabling Event Driven Architecture with PubSub+Enabling Event Driven Architecture with PubSub+
Enabling Event Driven Architecture with PubSub+Himanshu Gupta
 
How to Define and Share your Event APIs using AsyncAPI and Event API Products...
How to Define and Share your Event APIs using AsyncAPI and Event API Products...How to Define and Share your Event APIs using AsyncAPI and Event API Products...
How to Define and Share your Event APIs using AsyncAPI and Event API Products...HostedbyConfluent
 
ActiveEon - Automate, Accelerate, Scale
ActiveEon - Automate, Accelerate, ScaleActiveEon - Automate, Accelerate, Scale
ActiveEon - Automate, Accelerate, ScaleActiveeon
 
Compliance and Governance Webinar
Compliance and Governance WebinarCompliance and Governance Webinar
Compliance and Governance WebinarCollabNet
 
Sukumar Nayak-Agile-DevOps-Cloud Management
Sukumar Nayak-Agile-DevOps-Cloud ManagementSukumar Nayak-Agile-DevOps-Cloud Management
Sukumar Nayak-Agile-DevOps-Cloud ManagementSukumar Nayak
 
2022 apidays LIVE Helsinki & North_Event API Products – Maximizing the Value ...
2022 apidays LIVE Helsinki & North_Event API Products – Maximizing the Value ...2022 apidays LIVE Helsinki & North_Event API Products – Maximizing the Value ...
2022 apidays LIVE Helsinki & North_Event API Products – Maximizing the Value ...apidays
 
Enabling a Real-Time, Agile, Event-Driven Enterprise
Enabling a Real-Time, Agile, Event-Driven EnterpriseEnabling a Real-Time, Agile, Event-Driven Enterprise
Enabling a Real-Time, Agile, Event-Driven EnterpriseSolace
 
Kochi mulesoft meetup 02
Kochi mulesoft meetup 02Kochi mulesoft meetup 02
Kochi mulesoft meetup 02sumitahuja94
 
Data Distribution for the Event-Driven Business
Data Distribution for the Event-Driven BusinessData Distribution for the Event-Driven Business
Data Distribution for the Event-Driven BusinessSolace
 
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHow to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceWSO2
 
Voxxed Athens 2018 - Eventing, Serverless, and the Extensible Enterprise
Voxxed Athens 2018 - Eventing, Serverless, and the Extensible EnterpriseVoxxed Athens 2018 - Eventing, Serverless, and the Extensible Enterprise
Voxxed Athens 2018 - Eventing, Serverless, and the Extensible EnterpriseVoxxed Athens
 
The Significant role of event driven apps in software development
The Significant role of event driven apps in software development					The Significant role of event driven apps in software development
The Significant role of event driven apps in software development Shelly Megan
 
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHow to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHARMAN Services
 
Challenges In Building Enterprise Mashups - Rick B
Challenges In Building Enterprise Mashups - Rick BChallenges In Building Enterprise Mashups - Rick B
Challenges In Building Enterprise Mashups - Rick BRoopa Nadkarni
 
5 challenges in_building_enterprise_mashups-rick_b
5 challenges in_building_enterprise_mashups-rick_b5 challenges in_building_enterprise_mashups-rick_b
5 challenges in_building_enterprise_mashups-rick_bIBM
 
Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...Sri Ambati
 

Similar to How to Discover, Visualize, Catalog, Share and Reuse your Kafka Streams (Jonathan Schabowsky, Solace) Kafka Summit 2020 (20)

How Curing “Kafkatosis” Can Improve Your Stream Management and Governance
How Curing “Kafkatosis” Can Improve Your Stream Management and GovernanceHow Curing “Kafkatosis” Can Improve Your Stream Management and Governance
How Curing “Kafkatosis” Can Improve Your Stream Management and Governance
 
Mesh-ing around with Streams across the Enterprise | Phil Scanlon, Solace
Mesh-ing around with Streams across the Enterprise | Phil Scanlon, SolaceMesh-ing around with Streams across the Enterprise | Phil Scanlon, Solace
Mesh-ing around with Streams across the Enterprise | Phil Scanlon, Solace
 
Solace Connect NYC 2019 - Enabling the Event-Driven Enterprise
Solace Connect NYC 2019 - Enabling the Event-Driven EnterpriseSolace Connect NYC 2019 - Enabling the Event-Driven Enterprise
Solace Connect NYC 2019 - Enabling the Event-Driven Enterprise
 
Creating an Event Backbone for the Hybrid Cloud
Creating an Event Backbone for the Hybrid CloudCreating an Event Backbone for the Hybrid Cloud
Creating an Event Backbone for the Hybrid Cloud
 
Enabling Event Driven Architecture with PubSub+
Enabling Event Driven Architecture with PubSub+Enabling Event Driven Architecture with PubSub+
Enabling Event Driven Architecture with PubSub+
 
How to Define and Share your Event APIs using AsyncAPI and Event API Products...
How to Define and Share your Event APIs using AsyncAPI and Event API Products...How to Define and Share your Event APIs using AsyncAPI and Event API Products...
How to Define and Share your Event APIs using AsyncAPI and Event API Products...
 
ActiveEon - Automate, Accelerate, Scale
ActiveEon - Automate, Accelerate, ScaleActiveEon - Automate, Accelerate, Scale
ActiveEon - Automate, Accelerate, Scale
 
Compliance and Governance Webinar
Compliance and Governance WebinarCompliance and Governance Webinar
Compliance and Governance Webinar
 
Sukumar Nayak-Agile-DevOps-Cloud Management
Sukumar Nayak-Agile-DevOps-Cloud ManagementSukumar Nayak-Agile-DevOps-Cloud Management
Sukumar Nayak-Agile-DevOps-Cloud Management
 
2022 apidays LIVE Helsinki & North_Event API Products – Maximizing the Value ...
2022 apidays LIVE Helsinki & North_Event API Products – Maximizing the Value ...2022 apidays LIVE Helsinki & North_Event API Products – Maximizing the Value ...
2022 apidays LIVE Helsinki & North_Event API Products – Maximizing the Value ...
 
Enabling a Real-Time, Agile, Event-Driven Enterprise
Enabling a Real-Time, Agile, Event-Driven EnterpriseEnabling a Real-Time, Agile, Event-Driven Enterprise
Enabling a Real-Time, Agile, Event-Driven Enterprise
 
Kochi mulesoft meetup 02
Kochi mulesoft meetup 02Kochi mulesoft meetup 02
Kochi mulesoft meetup 02
 
Data Distribution for the Event-Driven Business
Data Distribution for the Event-Driven BusinessData Distribution for the Event-Driven Business
Data Distribution for the Event-Driven Business
 
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHow to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
 
Voxxed Athens 2018 - Eventing, Serverless, and the Extensible Enterprise
Voxxed Athens 2018 - Eventing, Serverless, and the Extensible EnterpriseVoxxed Athens 2018 - Eventing, Serverless, and the Extensible Enterprise
Voxxed Athens 2018 - Eventing, Serverless, and the Extensible Enterprise
 
The Significant role of event driven apps in software development
The Significant role of event driven apps in software development					The Significant role of event driven apps in software development
The Significant role of event driven apps in software development
 
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and GovernanceHow to Manage APIs in your Enterprise for Maximum Reusability and Governance
How to Manage APIs in your Enterprise for Maximum Reusability and Governance
 
Challenges In Building Enterprise Mashups - Rick B
Challenges In Building Enterprise Mashups - Rick BChallenges In Building Enterprise Mashups - Rick B
Challenges In Building Enterprise Mashups - Rick B
 
5 challenges in_building_enterprise_mashups-rick_b
5 challenges in_building_enterprise_mashups-rick_b5 challenges in_building_enterprise_mashups-rick_b
5 challenges in_building_enterprise_mashups-rick_b
 
Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...
 

More from HostedbyConfluent

Renaming a Kafka Topic | Kafka Summit London
Renaming a Kafka Topic | Kafka Summit LondonRenaming a Kafka Topic | Kafka Summit London
Renaming a Kafka Topic | Kafka Summit LondonHostedbyConfluent
 
Evolution of NRT Data Ingestion Pipeline at Trendyol
Evolution of NRT Data Ingestion Pipeline at TrendyolEvolution of NRT Data Ingestion Pipeline at Trendyol
Evolution of NRT Data Ingestion Pipeline at TrendyolHostedbyConfluent
 
Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques
Ensuring Kafka Service Resilience: A Dive into Health-Checking TechniquesEnsuring Kafka Service Resilience: A Dive into Health-Checking Techniques
Ensuring Kafka Service Resilience: A Dive into Health-Checking TechniquesHostedbyConfluent
 
Exactly-once Stream Processing with Arroyo and Kafka
Exactly-once Stream Processing with Arroyo and KafkaExactly-once Stream Processing with Arroyo and Kafka
Exactly-once Stream Processing with Arroyo and KafkaHostedbyConfluent
 
Fish Plays Pokemon | Kafka Summit London
Fish Plays Pokemon | Kafka Summit LondonFish Plays Pokemon | Kafka Summit London
Fish Plays Pokemon | Kafka Summit LondonHostedbyConfluent
 
Tiered Storage 101 | Kafla Summit London
Tiered Storage 101 | Kafla Summit LondonTiered Storage 101 | Kafla Summit London
Tiered Storage 101 | Kafla Summit LondonHostedbyConfluent
 
Building a Self-Service Stream Processing Portal: How And Why
Building a Self-Service Stream Processing Portal: How And WhyBuilding a Self-Service Stream Processing Portal: How And Why
Building a Self-Service Stream Processing Portal: How And WhyHostedbyConfluent
 
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...HostedbyConfluent
 
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...HostedbyConfluent
 
Navigating Private Network Connectivity Options for Kafka Clusters
Navigating Private Network Connectivity Options for Kafka ClustersNavigating Private Network Connectivity Options for Kafka Clusters
Navigating Private Network Connectivity Options for Kafka ClustersHostedbyConfluent
 
Apache Flink: Building a Company-wide Self-service Streaming Data Platform
Apache Flink: Building a Company-wide Self-service Streaming Data PlatformApache Flink: Building a Company-wide Self-service Streaming Data Platform
Apache Flink: Building a Company-wide Self-service Streaming Data PlatformHostedbyConfluent
 
Explaining How Real-Time GenAI Works in a Noisy Pub
Explaining How Real-Time GenAI Works in a Noisy PubExplaining How Real-Time GenAI Works in a Noisy Pub
Explaining How Real-Time GenAI Works in a Noisy PubHostedbyConfluent
 
TL;DR Kafka Metrics | Kafka Summit London
TL;DR Kafka Metrics | Kafka Summit LondonTL;DR Kafka Metrics | Kafka Summit London
TL;DR Kafka Metrics | Kafka Summit LondonHostedbyConfluent
 
A Window Into Your Kafka Streams Tasks | KSL
A Window Into Your Kafka Streams Tasks | KSLA Window Into Your Kafka Streams Tasks | KSL
A Window Into Your Kafka Streams Tasks | KSLHostedbyConfluent
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
Mastering Kafka Producer Configs: A Guide to Optimizing Performance
Mastering Kafka Producer Configs: A Guide to Optimizing PerformanceMastering Kafka Producer Configs: A Guide to Optimizing Performance
Mastering Kafka Producer Configs: A Guide to Optimizing PerformanceHostedbyConfluent
 
Data Contracts Management: Schema Registry and Beyond
Data Contracts Management: Schema Registry and BeyondData Contracts Management: Schema Registry and Beyond
Data Contracts Management: Schema Registry and BeyondHostedbyConfluent
 
Code-First Approach: Crafting Efficient Flink Apps
Code-First Approach: Crafting Efficient Flink AppsCode-First Approach: Crafting Efficient Flink Apps
Code-First Approach: Crafting Efficient Flink AppsHostedbyConfluent
 
Debezium vs. the World: An Overview of the CDC Ecosystem
Debezium vs. the World: An Overview of the CDC EcosystemDebezium vs. the World: An Overview of the CDC Ecosystem
Debezium vs. the World: An Overview of the CDC EcosystemHostedbyConfluent
 
Beyond Tiered Storage: Serverless Kafka with No Local Disks
Beyond Tiered Storage: Serverless Kafka with No Local DisksBeyond Tiered Storage: Serverless Kafka with No Local Disks
Beyond Tiered Storage: Serverless Kafka with No Local DisksHostedbyConfluent
 

More from HostedbyConfluent (20)

Renaming a Kafka Topic | Kafka Summit London
Renaming a Kafka Topic | Kafka Summit LondonRenaming a Kafka Topic | Kafka Summit London
Renaming a Kafka Topic | Kafka Summit London
 
Evolution of NRT Data Ingestion Pipeline at Trendyol
Evolution of NRT Data Ingestion Pipeline at TrendyolEvolution of NRT Data Ingestion Pipeline at Trendyol
Evolution of NRT Data Ingestion Pipeline at Trendyol
 
Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques
Ensuring Kafka Service Resilience: A Dive into Health-Checking TechniquesEnsuring Kafka Service Resilience: A Dive into Health-Checking Techniques
Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques
 
Exactly-once Stream Processing with Arroyo and Kafka
Exactly-once Stream Processing with Arroyo and KafkaExactly-once Stream Processing with Arroyo and Kafka
Exactly-once Stream Processing with Arroyo and Kafka
 
Fish Plays Pokemon | Kafka Summit London
Fish Plays Pokemon | Kafka Summit LondonFish Plays Pokemon | Kafka Summit London
Fish Plays Pokemon | Kafka Summit London
 
Tiered Storage 101 | Kafla Summit London
Tiered Storage 101 | Kafla Summit LondonTiered Storage 101 | Kafla Summit London
Tiered Storage 101 | Kafla Summit London
 
Building a Self-Service Stream Processing Portal: How And Why
Building a Self-Service Stream Processing Portal: How And WhyBuilding a Self-Service Stream Processing Portal: How And Why
Building a Self-Service Stream Processing Portal: How And Why
 
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
 
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
 
Navigating Private Network Connectivity Options for Kafka Clusters
Navigating Private Network Connectivity Options for Kafka ClustersNavigating Private Network Connectivity Options for Kafka Clusters
Navigating Private Network Connectivity Options for Kafka Clusters
 
Apache Flink: Building a Company-wide Self-service Streaming Data Platform
Apache Flink: Building a Company-wide Self-service Streaming Data PlatformApache Flink: Building a Company-wide Self-service Streaming Data Platform
Apache Flink: Building a Company-wide Self-service Streaming Data Platform
 
Explaining How Real-Time GenAI Works in a Noisy Pub
Explaining How Real-Time GenAI Works in a Noisy PubExplaining How Real-Time GenAI Works in a Noisy Pub
Explaining How Real-Time GenAI Works in a Noisy Pub
 
TL;DR Kafka Metrics | Kafka Summit London
TL;DR Kafka Metrics | Kafka Summit LondonTL;DR Kafka Metrics | Kafka Summit London
TL;DR Kafka Metrics | Kafka Summit London
 
A Window Into Your Kafka Streams Tasks | KSL
A Window Into Your Kafka Streams Tasks | KSLA Window Into Your Kafka Streams Tasks | KSL
A Window Into Your Kafka Streams Tasks | KSL
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
Mastering Kafka Producer Configs: A Guide to Optimizing Performance
Mastering Kafka Producer Configs: A Guide to Optimizing PerformanceMastering Kafka Producer Configs: A Guide to Optimizing Performance
Mastering Kafka Producer Configs: A Guide to Optimizing Performance
 
Data Contracts Management: Schema Registry and Beyond
Data Contracts Management: Schema Registry and BeyondData Contracts Management: Schema Registry and Beyond
Data Contracts Management: Schema Registry and Beyond
 
Code-First Approach: Crafting Efficient Flink Apps
Code-First Approach: Crafting Efficient Flink AppsCode-First Approach: Crafting Efficient Flink Apps
Code-First Approach: Crafting Efficient Flink Apps
 
Debezium vs. the World: An Overview of the CDC Ecosystem
Debezium vs. the World: An Overview of the CDC EcosystemDebezium vs. the World: An Overview of the CDC Ecosystem
Debezium vs. the World: An Overview of the CDC Ecosystem
 
Beyond Tiered Storage: Serverless Kafka with No Local Disks
Beyond Tiered Storage: Serverless Kafka with No Local DisksBeyond Tiered Storage: Serverless Kafka with No Local Disks
Beyond Tiered Storage: Serverless Kafka with No Local Disks
 

Recently uploaded

Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessWSO2
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
Automation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementAutomation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementDianaGray10
 
Deliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceDeliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceOpsTree solutions
 
Why Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile EvolutionWhy Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile EvolutionDEEPRAJ PATHAK
 
THE STATE OF STARTUP ECOSYSTEM - INDIA x JAPAN 2023
THE STATE OF STARTUP ECOSYSTEM - INDIA x JAPAN 2023THE STATE OF STARTUP ECOSYSTEM - INDIA x JAPAN 2023
THE STATE OF STARTUP ECOSYSTEM - INDIA x JAPAN 2023Joshua Flannery
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?Rustici Software
 
The work to make the piecework work: An ethnographic study of food delivery w...
The work to make the piecework work: An ethnographic study of food delivery w...The work to make the piecework work: An ethnographic study of food delivery w...
The work to make the piecework work: An ethnographic study of food delivery w...stockholm university
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationBuild Intuit
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationDianaGray10
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 

Recently uploaded (20)

Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with Platformless
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
Automation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions managementAutomation Ops Series: Session 3 - Solutions management
Automation Ops Series: Session 3 - Solutions management
 
Deliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceDeliver Latency Free Customer Experience
Deliver Latency Free Customer Experience
 
Why Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile EvolutionWhy Agile? - A handbook behind Agile Evolution
Why Agile? - A handbook behind Agile Evolution
 
THE STATE OF STARTUP ECOSYSTEM - INDIA x JAPAN 2023
THE STATE OF STARTUP ECOSYSTEM - INDIA x JAPAN 2023THE STATE OF STARTUP ECOSYSTEM - INDIA x JAPAN 2023
THE STATE OF STARTUP ECOSYSTEM - INDIA x JAPAN 2023
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?
 
The work to make the piecework work: An ethnographic study of food delivery w...
The work to make the piecework work: An ethnographic study of food delivery w...The work to make the piecework work: An ethnographic study of food delivery w...
The work to make the piecework work: An ethnographic study of food delivery w...
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientation
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automation
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 

How to Discover, Visualize, Catalog, Share and Reuse your Kafka Streams (Jonathan Schabowsky, Solace) Kafka Summit 2020

  • 1. How to Discover, Visualize, Catalog, Share and Reuse your Kafka Event Streams Jonathan Schabowsky Senior Architect, CTO, Solace
  • 2. Agenda 2 • Your growing Kafka event streams • Flying blind • Event portal: an API portal for event streams • Demo: How to discover, visualize, catalog, share and reuse your Kafka event streams • Summary © Solace 2
  • 3. Does Your Organization Look Like this? 3 © Solace Data Stream Flow Legacy Application Infrastructure …. ? ? ? JSON, XML, Plain Text… IoT/Device Management Producers Consumers Connectors Stream Processors Avro
  • 4. You’re Flying Blind… Where do you discover event streams for reuse? Why does a given event stream exist? What topic do you use to subscribe to it? What changed? How do you determine the data structure of its payload? Who should have access to it? Who made the latest revision? Who can tell you more about an event? Is your change backward compatible? Who is impacted by your change? Does it comply with security policy? 4 © Solace
  • 5. Some Try Solutions Not Intended for Event Stream Management • It's a pain… – No single place to discover and share/reuse event streams – No best practices guidance • You’re breaking systems and don’t know who changed what??? – Suffer from changes with unintended consequences – Hard to audit changes when data is stored in different tools Brittle, complex, manual solutions Custom Developed Catalog PowerPoint API Management Spread sheets GitConfluence 5 © Solace
  • 6. API Management Solved this for REST 6 © Solace
  • 7. They answer Who, What, When, Where, Why and How For RESTful APIs… but what about event streams? Inspired by API Management Platforms 7 © Solace API Gateway Documentation • Discover Registration • Govern Analysis • Improve • Monetize Community • Collaborate Architects and Developers (Internal, Partner and Third-Party) Productivity • Generate Code API Portal
  • 8. Sinks “While most organizations have basic event processing infrastructure - such as notification services, message buses, and event brokers - many don’t have the high-level productivity tooling that helps developers design, develop, test and manage event-centric applications. Nor do they have tools for governing, publishing and managing event-based interfaces. The market for these tools is sparse.” Gartner “Top 3 Trends in Application Architecture That Enable Digital Business” Anne Thomas, Yefim Natis, Mark O’Neill, 28 Oct 2019 8 © Solace Event Broker Security Persistence Protocol Translation One-to-Many Distribution Source Runtime We Need an Event Portal for Asynchronous Interactions Design, Develop and Manage Event Portal Documentation • Discover Registration • Govern Analysis • Improve • Monetize Community • Collaborate Productivity • Generate Code (code gen) (spec)
  • 9. 9 © Solace Single place to design, create, catalog, visualize, discover, share, secure and manage all events within your ecosystem Introducing Event Portal Event Portal Mainframe/ SAP Distributed Apps IoTMicro services Server Less Event Brokers
  • 10. Event Portal How Users Interact with Event Portal 10 Event Management (Design Time) © Solace Your Event-Driven Enterprise B2B/B2C Ecosystem IoTMicro- services Server- less Generate Code REST API Existing Tools • Schema registries • Data governance • CI/CD pipelines Customers & Partners • Discover and consume events in real-time Architects • Discover, create, update, review apps, events and payload schemas • Map payload schemas to events; and events to apps • Govern event consumption across app domain boundaries Developers • Collaborate with architects on design • Export AsyncAPI app contracts for each app • Generate and implement event-driven apps that provide business value Data Scientists • Understand event-driven data • Discover new insights by combining events
  • 12. Benefits of an Event Portal 12 Solace Your business can… • Understand who is consuming which events, how much, for what • Create value-added services and gain insights by combining events • Make it easy for users across app teams, LoBs and partner ecosystem to find and reuse events • Monetize popular or particular high-value event streams Reuse/Sharing Architects & developers can: • Collaborate on events and event-driven apps • Visually see event-driven interactions to understand impact of changes • Low Code: Generate app code by exporting AsyncAPI definitions • Benefit from best practices & consistently apply conventions Productivity CDO/data governance can: • Understand where data is coming from (lineage) and going • Track changes and audit for deviations while promoting apps and dependencies though Dev, Test, QA, and Prod • Use application/event relationships to create security policies and validate compliance with schema Control
  • 13. So, if you… Have a lot of event- driven applications 13 © Solace You need an event portal! Have (or need to define) a lot of event streams Want to manage events as easily as you manage your APIs
  • 14. Event Portal = API Portal for Events/Streams One tool to design, create, catalog, visualize, discover, share, secure and manage all of your events and streams (including Kafka and any other event broker) 14 © Solace Sign up for free! • About data and apps, NOT managing Kafka infrastructure • High-level productivity tooling to help you fully embrace EDA • Governance of apps, event and data