Download as PDF, PPTX
Uploaded byRed Hat Developers
Serverless stream processing of Debezium data change events with Knative | DevNation Tech Talk
The document discusses the implementation of serverless stream processing for Debezium data change events using Knative, focusing on consuming temperature measurements, enriching the data stream, routing, and visualizing it. Key components include Apache Kafka, Kafka Connect, Debezium for change data capture, Strimzi for managing Kafka on Kubernetes, and Knative for serverless management. It provides an overview of the solution architecture and links for further exploration and demos.
More Related Content
![[Lakmal] Automate Microservice to API](https://cdn.slidesharecdn.com/ss_thumbnails/lakmalautomatemicroservicetoapi-210512215049-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Serverless stream processing of Debezium data change events with Knative | DevNation Tech Talk
Serverless stream processing of Debezium data change events with Knative | DevNation Tech Talk
- 1. Serverless Stream Processing ofDebezium Data Change Events with Knative Gunnar Morling @gunnarmorling Principal Software Engineer, Red Hat Matthias Wessendorf @mwessendorf Principal Software Engineer, Red Hat
- 2. Gunnar Morling 2 Gunnar Morling ●Open source software engineer at Red Hat ● Debezium ● Hibernate ● Spec Lead for Bean Validation 2.0 ● Java Champion ● @gunnarmorling
- 3. Matthias Wessendorf Matthias Wessendorf ●Open source software engineer at Red Hat ● Knative Eventing ● @mwessendorf
- 4. Today’sObjective 4 Today’s Objective Consume astream of temperature measurements Enrich the stream with master data Route to different different consumers based on metadata Visualize contents of the stream on a map
- 5.
- 6. 6 Solution Overview | || | | | | | | | | | | | | | | | | | | | | Apache Kafka Kafka Connect DBZ PG Sensors Solution Overview Kafka Streams Knative Eventing Broker Kafka Event Source
- 7. Solution Overview 7 Sensor Data || | | | | | Apache Kafka Sensors
- 8. Solution Overview 8 Database andDebezium | | | | | | | | | | | | | | | Apache Kafka Kafka Connect DBZ PG Sensors
- 9. Solution Overview 9 Apache Kafkaand Strimzi | | | | | | | | | | | | | | | Apache Kafka Kafka Connect DBZ PG Sensors
- 10. Solution Overview 10 Kafka Streamsand Quarkus | | | | | | | | | | | | | | | Apache Kafka Kafka Connect DBZ PG Sensors Kafka Streams
- 11. Solution Overview 11 Kafka Streamsand Quarkus | | | | | | | | | | | | | | | Apache Kafka Kafka Connect DBZ PG Sensors | | | | | | | | Kafka Streams
- 12. Solution Overview 12 Knative Eventing || | | | | | | | | | | | | | Apache Kafka Kafka Connect DBZ PG Sensors | | | | | | | | Knative Eventing Broker Kafka Streams Kafka Event Source
- 13. 13 Maps Application | || | | | | | | | | | | | | | | | | | | | | Apache Kafka Kafka Connect DBZ PG Sensors Solution Overview Kafka Streams Knative Eventing Broker Kafka Event Source
- 14. 14 Kubernetes | | || | | | | | | | | | | | | | | | | | | | Apache Kafka Kafka Connect DBZ PG Sensors Solution Overview Kafka Streams Knative Eventing Broker Kafka Event Source
- 15. SolutionElements 15 Solution Elements Debezium forChange Data Capture Strimzi for running Apache Kafka on Kubernetes Quarkus and Kafka Streams for stream processing Knative for serverless application management
- 16. Debezium Open-Source Change DataCapture Platform ■ Log-based CDC for a variety of databases ■ MySQL, Postgres, SQL Server, MongoDB etc. ■ Transparent to writing apps ■ Snapshotting, Filtering etc. ■ Fully open-source, very active community ■ Based on Apache Kafka (Connect) ■ Use Cases: replication, streaming queries, microservices data exchange etc. 16
- 17. Strimzi Managing your ApacheKafka Cluster on Kubernetes 17 Topic & User operators Zookeeper cluster Kafka cluster Cluster operator Kafka Custom Resource
- 18. Supersonic Subatomic Java CloudNative Microservices Serverless 18
- 19. Supersonic Subatomic Java QuarkusBenefits Developer Joy Supersonic Subatomic Java Unifies imperative and reactive Best of breed libraries and standards 19
- 20.
- 21. Knative Kubernetes-based Serverless Platform 21 Serving Arequest-driven model that serves the container with your application and can "scale to zero". Eventing Common infrastructure for consuming and producing events that will stimulate applications.
- 22. 22 Knative Eventing Source →Service: Broker & Trigger
- 23.
- 24.
- 25. 25 Kubernetes | | || | | | | | | | | | | | | | | | | | | | Apache Kafka Kafka Connect DBZ PG Sensors Solution Overview Kafka Streams Knative Eventing Broker Kafka Event Source
- 26. Call to Action Resources 26 ●Learn more about Debezium ○ https://debezium.io/ ● Explore the demo ○ https://github.com/gunnarmorling/kafka-knative-demo/ ● Debezium GA in Red Hat Integration: ○ https://access.redhat.com/documentation/en-us/red_hat _integration/2020-04/html/ ● Install from resource descriptors on Kubernetes Cluster ○ https://knative.dev/docs/install/ ● Red Hat OpenShift Serverless - Technical Preview ○ https://www.openshift.com/learn/topics/serverless
- 27.