SlideShare a Scribd company logo

What is Apache Kafka? Why is it so popular? Should I use it?

•
•
Guido Schmutz
Guido Schmutz

Slides on the usage of Kafka which I used for the Speed Session at DOAG2019 at our booth.

Data & Analytics
1 of 25
Download to read offline
BASEL | BERN | BRUGG | BUCHAREST | COPENHAGEN | DÜSSELDORF | FRANKFURT A.M. | FREIBURG I.BR. GENEVA | HAMBURG | LAUSANNE | MANNHEIM | MUNICH | STUTTGART | VIENNA | ZURICH http://guidoschmutz@wordpress.com@gschmutz What is Apache Kafka? Why is it so popular? Should I use it? Guido Schmutz Trivadis Speed Session 2019
BASEL | BERN | BRUGG | BUKAREST | DÜSSELDORF | FRANKFURT A.M. | FREIBURG I.BR. | GENF HAMBURG | KOPENHAGEN | LAUSANNE | MANNHEIM | MÜNCHEN | STUTTGART | WIEN | ZÜRICH Guido Working at Trivadis for more than 22 years Consultant, Trainer, Platform Architect for Java, Oracle, SOA and Big Data / Fast Data Oracle Groundbreaker Ambassador & Oracle ACE Director @gschmutz guidoschmutz.wordpress.com 174th edition
Event Hub Kafka Message Broker – Key properties • Publish / Subscribe Messaging – message can be consumed by 0 – n consumers • horizontally scalable – throughput increases with more nodes • highly available – no SPOF • durable – messages are not lost • Schema-less – Kafka broker has no knowledge on message content and format
Event Hub Vehicle Weather Streaming Data Sources
Event Hub Stream Data Integration Stream Data Integration Vehicle Weather Streaming Data Sources Streaming Data Ingestion
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Streaming Data Sources Stream Analytics • Stream-to-Stream Joins • Stream-to-Table Joins • Time Windowed State Management • Event Pattern Detection • Machine Learning Model Execution (Inference)
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Streaming Data Sources Data Lake / DWH Batch Data Integration Batch Visualize Data Lake Ingestion • Machine Learning • Graph Algorithms • Natural Language Processing
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Batch Data Sources Streaming Data Sources Data Lake / DWH Batch Data Integration Batch Visualize
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Batch Data Sources Streaming Data Sources Data Lake / DWH Batch Data Integration Batch Visualize
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Data Lake / DWH Batch Data Integration Batch Visualize (Right-Time) Legacy Integration
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Data Lake / DWH Batch Data Integration Batch Visualize (Right-Time) Legacy Integration
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Data Lake / DWH Batch Data Integration Batch Visualize (Right-Time) Legacy Integration
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Data Lake / DWH Batch Data Integration Batch Visualize (Right-Time) Legacy Integration
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Batch Visualize Streaming Visualization
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Result Store Integration
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Micro service Highly Decoupled Modern Apps
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Micro service Micro service Highly Decoupled Modern Apps
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Micro service Micro service Gateway Data Source talks to Kafka through MQTT
Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Micro service Micro service Gateway Kafka becomes central nervous system for data
Apache Kafka Kafka Cluster Consumer 1 Consume 2r Broker 1 Broker 2 Broker 3 Zookeeper Ensemble ZK 1 ZK 2ZK 3 Schema Registry Service 1 Management Control Center Kafka Manager KAdmin Producer 1 Producer 2 kafkacat Data Retention: • Never • Time (TTL) or Size-based • Log-Compacted based Producer3Producer3 ConsumerConsumer 3
• No SPoF, highly available • Consumer polls for new messages Apache Kafka • horizontally scalable, guaranteed order
Apache Kafka
Apache Kafka – Streaming Platform Source Connector Kafka Broker Sink Connector Stream Processing Schema Registry Kafka Kafka MQTT Proxy REST Proxy
Meine Präsentationen @ DOAG 2019 Location Intelligence: Real-Time Geofencing mit Kafka Mittwoch, 20. Nov., 14:00 - 14:45 Visualisierung von Streaming-Daten Donnerstag, 21. Nov., 12:00 - 12:45 Bi-direktionale Integration (Oracle) RDBMS und Apache Kafka Donnerstag, 21. Nov., 14:00 - 14:45
What is Apache Kafka? Why is it so popular? Should I use it?

Recommended

Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Data Ingestion in Big Data and IoT platforms
Data Ingestion in Big Data and IoT platformsData Ingestion in Big Data and IoT platforms
Data Ingestion in Big Data and IoT platformsGuido Schmutz
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Streaming Visualisation
Streaming VisualisationStreaming Visualisation
Streaming VisualisationGuido Schmutz
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQL
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQLIngesting and Processing IoT Data - using MQTT, Kafka Connect and KSQL
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQLGuido Schmutz
 

Recommended

Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Data Ingestion in Big Data and IoT platforms
Data Ingestion in Big Data and IoT platformsData Ingestion in Big Data and IoT platforms
Data Ingestion in Big Data and IoT platformsGuido Schmutz
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Streaming Visualisation
Streaming VisualisationStreaming Visualisation
Streaming VisualisationGuido Schmutz
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQL
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQLIngesting and Processing IoT Data - using MQTT, Kafka Connect and KSQL
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQLGuido Schmutz
 
Building Event-Driven (Micro) Services with Apache Kafka
Building Event-Driven (Micro) Services with Apache KafkaBuilding Event-Driven (Micro) Services with Apache Kafka
Building Event-Driven (Micro) Services with Apache KafkaGuido Schmutz
 
Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Guido Schmutz
 
Event Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureEvent Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureGuido Schmutz
 
Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka Guido Schmutz
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
 
Event Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data ArchitectureEvent Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data ArchitectureGuido Schmutz
 
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaSolutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaGuido Schmutz
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaGuido Schmutz
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaGuido Schmutz
 
Building Event-Driven (Micro)Services with Apache Kafka
Building Event-Driven (Micro)Services with Apache KafkaBuilding Event-Driven (Micro)Services with Apache Kafka
Building Event-Driven (Micro)Services with Apache KafkaGuido Schmutz
 
Building event-driven (Micro)Services with Apache Kafka Ecosystem
Building event-driven (Micro)Services with Apache Kafka EcosystemBuilding event-driven (Micro)Services with Apache Kafka Ecosystem
Building event-driven (Micro)Services with Apache Kafka EcosystemGuido Schmutz
 
Fundamentals Big Data and AI Architecture
Fundamentals Big Data and AI ArchitectureFundamentals Big Data and AI Architecture
Fundamentals Big Data and AI ArchitectureGuido Schmutz
 
Event Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) ArchitectureEvent Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) ArchitectureGuido Schmutz
 
Microservices with Kafka Ecosystem
Microservices with Kafka EcosystemMicroservices with Kafka Ecosystem
Microservices with Kafka EcosystemGuido Schmutz
 
Spark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka StreamsSpark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka StreamsGuido Schmutz
 
Ingesting streaming data into Graph Database
Ingesting streaming data into Graph DatabaseIngesting streaming data into Graph Database
Ingesting streaming data into Graph DatabaseGuido Schmutz
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Guido Schmutz
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
Internet of Things and Big Data
Internet of Things and Big DataInternet of Things and Big Data
Internet of Things and Big DataSwiss Data Forum Swiss Data Forum
 

More Related Content

What's hot

Building Event-Driven (Micro) Services with Apache Kafka
Building Event-Driven (Micro) Services with Apache KafkaBuilding Event-Driven (Micro) Services with Apache Kafka
Building Event-Driven (Micro) Services with Apache KafkaGuido Schmutz
 
Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Guido Schmutz
 
Event Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureEvent Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureGuido Schmutz
 
Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka Guido Schmutz
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
 
Event Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data ArchitectureEvent Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data ArchitectureGuido Schmutz
 
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaSolutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaGuido Schmutz
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaGuido Schmutz
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaGuido Schmutz
 
Building Event-Driven (Micro)Services with Apache Kafka
Building Event-Driven (Micro)Services with Apache KafkaBuilding Event-Driven (Micro)Services with Apache Kafka
Building Event-Driven (Micro)Services with Apache KafkaGuido Schmutz
 
Building event-driven (Micro)Services with Apache Kafka Ecosystem
Building event-driven (Micro)Services with Apache Kafka EcosystemBuilding event-driven (Micro)Services with Apache Kafka Ecosystem
Building event-driven (Micro)Services with Apache Kafka EcosystemGuido Schmutz
 
Fundamentals Big Data and AI Architecture
Fundamentals Big Data and AI ArchitectureFundamentals Big Data and AI Architecture
Fundamentals Big Data and AI ArchitectureGuido Schmutz
 
Event Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) ArchitectureEvent Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) ArchitectureGuido Schmutz
 
Microservices with Kafka Ecosystem
Microservices with Kafka EcosystemMicroservices with Kafka Ecosystem
Microservices with Kafka EcosystemGuido Schmutz
 
Spark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka StreamsSpark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka StreamsGuido Schmutz
 
Ingesting streaming data into Graph Database
Ingesting streaming data into Graph DatabaseIngesting streaming data into Graph Database
Ingesting streaming data into Graph DatabaseGuido Schmutz
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaGuido Schmutz
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Guido Schmutz
 

What's hot (20)

Building Event-Driven (Micro) Services with Apache Kafka
Building Event-Driven (Micro) Services with Apache KafkaBuilding Event-Driven (Micro) Services with Apache Kafka
Building Event-Driven (Micro) Services with Apache Kafka
 
Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?Kafka as an event store - is it good enough?
Kafka as an event store - is it good enough?
 
Event Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data ArchitectureEvent Hub (i.e. Kafka) in Modern Data Architecture
Event Hub (i.e. Kafka) in Modern Data Architecture
 
Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming Visualization
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
 
Event Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data ArchitectureEvent Broker (Kafka) in a Modern Data Architecture
Event Broker (Kafka) in a Modern Data Architecture
 
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaSolutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache Kafka
 
Building Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache KafkaBuilding Event Driven (Micro)services with Apache Kafka
Building Event Driven (Micro)services with Apache Kafka
 
Building Event-Driven (Micro)Services with Apache Kafka
Building Event-Driven (Micro)Services with Apache KafkaBuilding Event-Driven (Micro)Services with Apache Kafka
Building Event-Driven (Micro)Services with Apache Kafka
 
Building event-driven (Micro)Services with Apache Kafka Ecosystem
Building event-driven (Micro)Services with Apache Kafka EcosystemBuilding event-driven (Micro)Services with Apache Kafka Ecosystem
Building event-driven (Micro)Services with Apache Kafka Ecosystem
 
Fundamentals Big Data and AI Architecture
Fundamentals Big Data and AI ArchitectureFundamentals Big Data and AI Architecture
Fundamentals Big Data and AI Architecture
 
Event Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) ArchitectureEvent Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
Event Hub (i.e. Kafka) in Modern Data (Analytics) Architecture
 
Microservices with Kafka Ecosystem
Microservices with Kafka EcosystemMicroservices with Kafka Ecosystem
Microservices with Kafka Ecosystem
 
Spark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka StreamsSpark (Structured) Streaming vs. Kafka Streams
Spark (Structured) Streaming vs. Kafka Streams
 
Ingesting streaming data into Graph Database
Ingesting streaming data into Graph DatabaseIngesting streaming data into Graph Database
Ingesting streaming data into Graph Database
 
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache KafkaSolutions for bi-directional integration between Oracle RDBMS & Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS & Apache Kafka
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming Visualization
 
Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka Location Analytics - Real-Time Geofencing using Kafka
Location Analytics - Real-Time Geofencing using Kafka
 

Similar to What is Apache Kafka? Why is it so popular? Should I use it?

Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...Guido Schmutz
 
Architecture of Big Data Solutions
Architecture of Big Data SolutionsArchitecture of Big Data Solutions
Architecture of Big Data SolutionsGuido Schmutz
 
Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache KafkaBuilding event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache KafkaGuido Schmutz
 
Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?Guido Schmutz
 
DevDay 2018: Ulrich Deiters - Offline First - kein Netz, kein Fehler, zufried...
DevDay 2018: Ulrich Deiters - Offline First - kein Netz, kein Fehler, zufried...DevDay 2018: Ulrich Deiters - Offline First - kein Netz, kein Fehler, zufried...
DevDay 2018: Ulrich Deiters - Offline First - kein Netz, kein Fehler, zufried...DevDay Dresden
 
Kafka as your Data Lake - is it Feasible? (Guido Schmutz, Trivadis) Kafka Sum...
Kafka as your Data Lake - is it Feasible? (Guido Schmutz, Trivadis) Kafka Sum...Kafka as your Data Lake - is it Feasible? (Guido Schmutz, Trivadis) Kafka Sum...
Kafka as your Data Lake - is it Feasible? (Guido Schmutz, Trivadis) Kafka Sum...HostedbyConfluent
 
Big Data Architecture
Big Data ArchitectureBig Data Architecture
Big Data ArchitectureGuido Schmutz
 
Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...
Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...
Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...Trivadis
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming VisualizationGuido Schmutz
 
Events on the outside, on the inside and at the core (jaxlondon)
Events on the outside, on the inside and at the core (jaxlondon)Events on the outside, on the inside and at the core (jaxlondon)
Events on the outside, on the inside and at the core (jaxlondon)Chris Richardson
 
Events on the outside, on the inside and at the core - Chris Richardson
Events on the outside, on the inside and at the core - Chris RichardsonEvents on the outside, on the inside and at the core - Chris Richardson
Events on the outside, on the inside and at the core - Chris RichardsonJAXLondon_Conference
 
Ingesting and Processing IoT Data Using MQTT, Kafka Connect and Kafka Streams...
Ingesting and Processing IoT Data Using MQTT, Kafka Connect and Kafka Streams...Ingesting and Processing IoT Data Using MQTT, Kafka Connect and Kafka Streams...
Ingesting and Processing IoT Data Using MQTT, Kafka Connect and Kafka Streams...confluent
 
Open Data Hub - Patrick Ohnewein - Update about the Open Data Hub Project
Open Data Hub - Patrick Ohnewein - Update about the Open Data Hub ProjectOpen Data Hub - Patrick Ohnewein - Update about the Open Data Hub Project
Open Data Hub - Patrick Ohnewein - Update about the Open Data Hub ProjectSouth Tyrol Free Software Conference
 
The Three Pillars of Agile Integration: Connector, Container & API
The Three Pillars of Agile Integration: Connector, Container & APIThe Three Pillars of Agile Integration: Connector, Container & API
The Three Pillars of Agile Integration: Connector, Container & APIJudy Breedlove
 
Pivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan
Pivoting Spring XD to Spring Cloud Data Flow with Sabby AnandanPivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan
Pivoting Spring XD to Spring Cloud Data Flow with Sabby AnandanPivotalOpenSourceHub
 
Spring on PAS - Fabio Marinelli
Spring on PAS - Fabio MarinelliSpring on PAS - Fabio Marinelli
Spring on PAS - Fabio MarinelliVMware Tanzu
 
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...Timothy Spann
 

Similar to What is Apache Kafka? Why is it so popular? Should I use it? (20)

Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream Processing
 
Internet of Things and Big Data
Internet of Things and Big DataInternet of Things and Big Data
Internet of Things and Big Data
 
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
IoT Architecture - Are Traditional Architectures Good Enough or do we Need Ne...
 
Building High-scalable Enterprise Solutions,
Building High-scalable Enterprise Solutions, Building High-scalable Enterprise Solutions,
Building High-scalable Enterprise Solutions,
 
Architecture of Big Data Solutions
Architecture of Big Data SolutionsArchitecture of Big Data Solutions
Architecture of Big Data Solutions
 
Building event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache KafkaBuilding event-driven (Micro)Services with Apache Kafka
Building event-driven (Micro)Services with Apache Kafka
 
Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?Kafka as your Data Lake - is it Feasible?
Kafka as your Data Lake - is it Feasible?
 
DevDay 2018: Ulrich Deiters - Offline First - kein Netz, kein Fehler, zufried...
DevDay 2018: Ulrich Deiters - Offline First - kein Netz, kein Fehler, zufried...DevDay 2018: Ulrich Deiters - Offline First - kein Netz, kein Fehler, zufried...
DevDay 2018: Ulrich Deiters - Offline First - kein Netz, kein Fehler, zufried...
 
Kafka as your Data Lake - is it Feasible? (Guido Schmutz, Trivadis) Kafka Sum...
Kafka as your Data Lake - is it Feasible? (Guido Schmutz, Trivadis) Kafka Sum...Kafka as your Data Lake - is it Feasible? (Guido Schmutz, Trivadis) Kafka Sum...
Kafka as your Data Lake - is it Feasible? (Guido Schmutz, Trivadis) Kafka Sum...
 
Big Data Architecture
Big Data ArchitectureBig Data Architecture
Big Data Architecture
 
Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...
Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...
Azure Days 2019: Wie bringt man eine Data Analytics Plattform in die Cloud? (...
 
Streaming Visualization
Streaming VisualizationStreaming Visualization
Streaming Visualization
 
Events on the outside, on the inside and at the core (jaxlondon)
Events on the outside, on the inside and at the core (jaxlondon)Events on the outside, on the inside and at the core (jaxlondon)
Events on the outside, on the inside and at the core (jaxlondon)
 
Events on the outside, on the inside and at the core - Chris Richardson
Events on the outside, on the inside and at the core - Chris RichardsonEvents on the outside, on the inside and at the core - Chris Richardson
Events on the outside, on the inside and at the core - Chris Richardson
 
Ingesting and Processing IoT Data Using MQTT, Kafka Connect and Kafka Streams...
Ingesting and Processing IoT Data Using MQTT, Kafka Connect and Kafka Streams...Ingesting and Processing IoT Data Using MQTT, Kafka Connect and Kafka Streams...
Ingesting and Processing IoT Data Using MQTT, Kafka Connect and Kafka Streams...
 
Open Data Hub - Patrick Ohnewein - Update about the Open Data Hub Project
Open Data Hub - Patrick Ohnewein - Update about the Open Data Hub ProjectOpen Data Hub - Patrick Ohnewein - Update about the Open Data Hub Project
Open Data Hub - Patrick Ohnewein - Update about the Open Data Hub Project
 
The Three Pillars of Agile Integration: Connector, Container & API
The Three Pillars of Agile Integration: Connector, Container & APIThe Three Pillars of Agile Integration: Connector, Container & API
The Three Pillars of Agile Integration: Connector, Container & API
 
Pivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan
Pivoting Spring XD to Spring Cloud Data Flow with Sabby AnandanPivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan
Pivoting Spring XD to Spring Cloud Data Flow with Sabby Anandan
 
Spring on PAS - Fabio Marinelli
Spring on PAS - Fabio MarinelliSpring on PAS - Fabio Marinelli
Spring on PAS - Fabio Marinelli
 
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
2024 February 28 - NYC - Meetup Unlocking Financial Data with Real-Time Pipel...
 

More from Guido Schmutz

30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as Code30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as CodeGuido Schmutz
 
Big Data, Data Lake, Fast Data - Dataserialiation-Formats
Big Data, Data Lake, Fast Data - Dataserialiation-FormatsBig Data, Data Lake, Fast Data - Dataserialiation-Formats
Big Data, Data Lake, Fast Data - Dataserialiation-FormatsGuido Schmutz
 
ksqlDB - Stream Processing simplified!
ksqlDB - Stream Processing simplified!ksqlDB - Stream Processing simplified!
ksqlDB - Stream Processing simplified!Guido Schmutz
 
Location Analytics - Real-Time Geofencing using Apache Kafka
Location Analytics - Real-Time Geofencing using Apache KafkaLocation Analytics - Real-Time Geofencing using Apache Kafka
Location Analytics - Real-Time Geofencing using Apache KafkaGuido Schmutz
 
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache KafkaSolutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache KafkaGuido Schmutz
 
Location Analytics Real-Time Geofencing using Kafka
Location Analytics Real-Time Geofencing using KafkaLocation Analytics Real-Time Geofencing using Kafka
Location Analytics Real-Time Geofencing using KafkaGuido Schmutz
 
Location Analytics - Real Time Geofencing using Apache Kafka
Location Analytics - Real Time Geofencing using Apache KafkaLocation Analytics - Real Time Geofencing using Apache Kafka
Location Analytics - Real Time Geofencing using Apache KafkaGuido Schmutz
 
Stream Processing – Concepts and Frameworks
Stream Processing – Concepts and FrameworksStream Processing – Concepts and Frameworks
Stream Processing – Concepts and FrameworksGuido Schmutz
 
Kafka as an Event Store - is it Good Enough?
Kafka as an Event Store - is it Good Enough?Kafka as an Event Store - is it Good Enough?
Kafka as an Event Store - is it Good Enough?Guido Schmutz
 
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaSolutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaGuido Schmutz
 

More from Guido Schmutz (10)

30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as Code30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as Code
 
Big Data, Data Lake, Fast Data - Dataserialiation-Formats
Big Data, Data Lake, Fast Data - Dataserialiation-FormatsBig Data, Data Lake, Fast Data - Dataserialiation-Formats
Big Data, Data Lake, Fast Data - Dataserialiation-Formats
 
ksqlDB - Stream Processing simplified!
ksqlDB - Stream Processing simplified!ksqlDB - Stream Processing simplified!
ksqlDB - Stream Processing simplified!
 
Location Analytics - Real-Time Geofencing using Apache Kafka
Location Analytics - Real-Time Geofencing using Apache KafkaLocation Analytics - Real-Time Geofencing using Apache Kafka
Location Analytics - Real-Time Geofencing using Apache Kafka
 
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache KafkaSolutions for bi-directional integration between Oracle RDBMS and Apache Kafka
Solutions for bi-directional integration between Oracle RDBMS and Apache Kafka
 
Location Analytics Real-Time Geofencing using Kafka
Location Analytics Real-Time Geofencing using KafkaLocation Analytics Real-Time Geofencing using Kafka
Location Analytics Real-Time Geofencing using Kafka
 
Location Analytics - Real Time Geofencing using Apache Kafka
Location Analytics - Real Time Geofencing using Apache KafkaLocation Analytics - Real Time Geofencing using Apache Kafka
Location Analytics - Real Time Geofencing using Apache Kafka
 
Stream Processing – Concepts and Frameworks
Stream Processing – Concepts and FrameworksStream Processing – Concepts and Frameworks
Stream Processing – Concepts and Frameworks
 
Kafka as an Event Store - is it Good Enough?
Kafka as an Event Store - is it Good Enough?Kafka as an Event Store - is it Good Enough?
Kafka as an Event Store - is it Good Enough?
 
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache KafkaSolutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
Solutions for bi-directional Integration between Oracle RDMBS & Apache Kafka
 

Recently uploaded

科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理e4aez8ss
 
Predictive Analysis - Using Insight-informed Data to Determine Factors Drivin...
Predictive Analysis - Using Insight-informed Data to Determine Factors Drivin...Predictive Analysis - Using Insight-informed Data to Determine Factors Drivin...
Predictive Analysis - Using Insight-informed Data to Determine Factors Drivin...ThinkInnovation
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...limedy534
 
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一fhwihughh
 
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样vhwb25kk
 
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝soniya singh
 
Customer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxCustomer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxEmmanuel Dauda
 
9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home ServiceSapana Sha
 
Industrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdfIndustrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdfLars Albertsson
 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptxthyngster
 
GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]📊 Markus Baersch
 
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degreeyuu sss
 
DBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfDBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfJohn Sterrett
 
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Callshivangimorya083
 
04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationshipsccctableauusergroup
 
RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998YohFuh
 
Brighton SEO | April 2024 | Data Storytelling
Brighton SEO | April 2024 | Data StorytellingBrighton SEO | April 2024 | Data Storytelling
Brighton SEO | April 2024 | Data StorytellingNeil Barnes
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024thyngster
 
INTERNSHIP ON PURBASHA COMPOSITE TEX LTD
INTERNSHIP ON PURBASHA COMPOSITE TEX LTDINTERNSHIP ON PURBASHA COMPOSITE TEX LTD
INTERNSHIP ON PURBASHA COMPOSITE TEX LTDRafezzaman
 

Recently uploaded (20)

科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
科罗拉多大学波尔得分校毕业证学位证成绩单-可办理
 
Predictive Analysis - Using Insight-informed Data to Determine Factors Drivin...
Predictive Analysis - Using Insight-informed Data to Determine Factors Drivin...Predictive Analysis - Using Insight-informed Data to Determine Factors Drivin...
Predictive Analysis - Using Insight-informed Data to Determine Factors Drivin...
 
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
Effects of Smartphone Addiction on the Academic Performances of Grades 9 to 1...
 
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
办理学位证纽约大学毕业证(NYU毕业证书）原版一比一
 
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证）昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
 
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Defence Colony Delhi 💯Call Us 🔝8264348440🔝
 
Customer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptxCustomer Service Analytics - Make Sense of All Your Data.pptx
Customer Service Analytics - Make Sense of All Your Data.pptx
 
9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service9654467111 Call Girls In Munirka Hotel And Home Service
9654467111 Call Girls In Munirka Hotel And Home Service
 
Industrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdfIndustrialised data - the key to AI success.pdf
Industrialised data - the key to AI success.pdf
 
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptxEMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
EMERCE - 2024 - AMSTERDAM - CROSS-PLATFORM TRACKING WITH GOOGLE ANALYTICS.pptx
 
GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]
 
E-Commerce Order PredictionShraddha Kamble.pptx
E-Commerce Order PredictionShraddha Kamble.pptxE-Commerce Order PredictionShraddha Kamble.pptx
E-Commerce Order PredictionShraddha Kamble.pptx
 
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
 
DBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfDBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdf
 
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
꧁❤ Greater Noida Call Girls Delhi ❤꧂ 9711199171 ☎️ Hard And Sexy Vip Call
 
04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships04242024_CCC TUG_Joins and Relationships
04242024_CCC TUG_Joins and Relationships
 
RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998RA-11058_IRR-COMPRESS Do 198 series of 1998
RA-11058_IRR-COMPRESS Do 198 series of 1998
 
Brighton SEO | April 2024 | Data Storytelling
Brighton SEO | April 2024 | Data StorytellingBrighton SEO | April 2024 | Data Storytelling
Brighton SEO | April 2024 | Data Storytelling
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
 
INTERNSHIP ON PURBASHA COMPOSITE TEX LTD
INTERNSHIP ON PURBASHA COMPOSITE TEX LTDINTERNSHIP ON PURBASHA COMPOSITE TEX LTD
INTERNSHIP ON PURBASHA COMPOSITE TEX LTD
 

What is Apache Kafka? Why is it so popular? Should I use it?

  • 1. BASEL | BERN | BRUGG | BUCHAREST | COPENHAGEN | DÜSSELDORF | FRANKFURT A.M. | FREIBURG I.BR. GENEVA | HAMBURG | LAUSANNE | MANNHEIM | MUNICH | STUTTGART | VIENNA | ZURICH http://guidoschmutz@wordpress.com@gschmutz What is Apache Kafka? Why is it so popular? Should I use it? Guido Schmutz Trivadis Speed Session 2019
  • 2. BASEL | BERN | BRUGG | BUKAREST | DÜSSELDORF | FRANKFURT A.M. | FREIBURG I.BR. | GENF HAMBURG | KOPENHAGEN | LAUSANNE | MANNHEIM | MÜNCHEN | STUTTGART | WIEN | ZÜRICH Guido Working at Trivadis for more than 22 years Consultant, Trainer, Platform Architect for Java, Oracle, SOA and Big Data / Fast Data Oracle Groundbreaker Ambassador & Oracle ACE Director @gschmutz guidoschmutz.wordpress.com 174th edition
  • 3. Event Hub Kafka Message Broker – Key properties • Publish / Subscribe Messaging – message can be consumed by 0 – n consumers • horizontally scalable – throughput increases with more nodes • highly available – no SPOF • durable – messages are not lost • Schema-less – Kafka broker has no knowledge on message content and format
  • 5. Event Hub Stream Data Integration Stream Data Integration Vehicle Weather Streaming Data Sources Streaming Data Ingestion
  • 6. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Streaming Data Sources Stream Analytics • Stream-to-Stream Joins • Stream-to-Table Joins • Time Windowed State Management • Event Pattern Detection • Machine Learning Model Execution (Inference)
  • 7. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Streaming Data Sources Data Lake / DWH Batch Data Integration Batch Visualize Data Lake Ingestion • Machine Learning • Graph Algorithms • Natural Language Processing
  • 8. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Batch Data Sources Streaming Data Sources Data Lake / DWH Batch Data Integration Batch Visualize
  • 9. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Batch Data Sources Streaming Data Sources Data Lake / DWH Batch Data Integration Batch Visualize
  • 10. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Data Lake / DWH Batch Data Integration Batch Visualize (Right-Time) Legacy Integration
  • 11. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Data Lake / DWH Batch Data Integration Batch Visualize (Right-Time) Legacy Integration
  • 12. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Data Lake / DWH Batch Data Integration Batch Visualize (Right-Time) Legacy Integration
  • 13. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Data Lake / DWH Batch Data Integration Batch Visualize (Right-Time) Legacy Integration
  • 14. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Batch Visualize Streaming Visualization
  • 15. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Result Store Integration
  • 16. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Micro service Highly Decoupled Modern Apps
  • 17. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Micro service Micro service Highly Decoupled Modern Apps
  • 18. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Micro service Micro service Gateway Data Source talks to Kafka through MQTT
  • 19. Event Hub Stream Data Integration Stream Data Integration Stream Analytics Vehicle Weather Legacy App Machine IIoT Stream Data Integration Batch Data Sources Streaming Data Sources CDC Stream Data Integration CDC Streaming Visualize Data Lake / DWH Batch Data Integration Stream Data Integration Stream Data Integration NOSQL NewSQL Batch Visualize Micro service Micro service Gateway Kafka becomes central nervous system for data
  • 20. Apache Kafka Kafka Cluster Consumer 1 Consume 2r Broker 1 Broker 2 Broker 3 Zookeeper Ensemble ZK 1 ZK 2ZK 3 Schema Registry Service 1 Management Control Center Kafka Manager KAdmin Producer 1 Producer 2 kafkacat Data Retention: • Never • Time (TTL) or Size-based • Log-Compacted based Producer3Producer3 ConsumerConsumer 3
  • 21. • No SPoF, highly available • Consumer polls for new messages Apache Kafka • horizontally scalable, guaranteed order
  • 23. Apache Kafka – Streaming Platform Source Connector Kafka Broker Sink Connector Stream Processing Schema Registry Kafka Kafka MQTT Proxy REST Proxy
  • 24. Meine Präsentationen @ DOAG 2019 Location Intelligence: Real-Time Geofencing mit Kafka Mittwoch, 20. Nov., 14:00 - 14:45 Visualisierung von Streaming-Daten Donnerstag, 21. Nov., 12:00 - 12:45 Bi-direktionale Integration (Oracle) RDBMS und Apache Kafka Donnerstag, 21. Nov., 14:00 - 14:45