A Kafka cluster stores streams of records (messages) in categories called topics. It is the architectural backbone for integrating streaming data with a Data Lake, Microservices and Stream Processing. Today’s enterprises have their core systems often implemented on top of relational databases, such as the Oracle RDBMS. Implementing a new solution supporting the digital strategy using Kafka and the ecosystem can not always be done completely separate from the traditional legacy solutions. Often streaming data has to be enriched with state data which is held in an RDBMS of a legacy application. It’s important to cache this data in the stream processing solution, so that It can be efficiently joined to the data stream. But how do we make sure that the cache is kept up-to-date, if the source data changes? We can either poll for changes from Kafka using Kafka Connect or let the RDBMS push the data changes to Kafka. But what about writing data back to the legacy application, i.e. an anomaly is detected inside the stream processing solution which should trigger an action inside the legacy application. Using Kafka Connect we can write to a database table or view, which could trigger the action. But this not always the best option. If you have an Oracle RDBMS, there are many other ways to integrate the database with Kafka, such as Advanced Queueing (message broker in the database), CDC through Golden Gate or Debezium, Oracle REST Database Service (ORDS) and more. In this session, we present various blueprints for integrating an Oracle RDBMS with Apache Kafka in both directions and discuss how these blueprints can be implemented using the products mentioned before.

1. BASEL | BERN | BRUGG | BUCHAREST | COPENHAGEN | DÜSSELDORF | FRANKFURT A.M. | FREIBURG I.BR.
GENEVA | HAMBURG | LAUSANNE | MANNHEIM | MUNICH | STUTTGART | VIENNA | ZURICH
http://guidoschmutz@wordpress.com@gschmutz
Solutions for bi-directional integration
between Oracle RDBMS & Apache Kafka
Guido Schmutz (guido.schmutz@trivadis.com)

2. Agenda
1. Introduction
2. Blueprints Oracle RDBMS => Apache Kafka
3. Blueprints Apache Kafka => Oracle RDBMS
4. Summary

3. BASEL | BERN | BRUGG | BUKAREST | DÜSSELDORF | FRANKFURT A.M. | FREIBURG I.BR. | GENF
HAMBURG | KOPENHAGEN | LAUSANNE | MANNHEIM | MÜNCHEN | STUTTGART | WIEN | ZÜRICH
Guido
Working at Trivadis for more than 22 years
Consultant, Trainer, Platform Architect for Java,
Oracle, SOA and Big Data / Fast Data
Oracle Groundbreaker Ambassador & Oracle ACE
Director
@gschmutz guidoschmutz.wordpress.com
170th
edition

4. Introduction

5. Microservices / Modern Applications
• Highly decoupled
• Independently deployable
• Bounded Context/Aggregate (DDD)
• Responsible for their data
• Favour asynchronous, event-driven
interaction over synchronous
• Smart Endpoints and Dump Pipes
• Use Anti-Corruption Layer (ACL) if no
fit! M3M2
ACL
Event
Hub
M1

6. Microservices / Modern Applications
Integrate with Traditional System
M3M2
ACL
Event
Hub
M1
ACL
• Highly decoupled
• Independently deployable
• Bounded Context/Aggregate (DDD)
• Responsible for their data
• Favour asynchronous, event-driven
interaction over synchronous
• Smart Endpoints and Dump Pipes
• Use Anti-Corruption Layer (ACL) if no
fit!
Traditional
App

7. Use Case
Customer Microservice
{ }
Customer API CustomerCustomer Logic
Order Processing System
{ }
Order API OrderOrder Logic
REST
REST
Event Hub
Customer
Mat View
Order
Customer
(compacted)
Notification Microservice
Notification Logic
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense
“Modern Apps”Traditional Apps (Legacy)

8. MessageMessageMessageMessage
MessageMessage
Properties - Message
Message Message
A1 A2 A3
Message
B1 B2 B3 B4
A1 A2 A3 B
B1 B2 B3 B4
Table A
A1
A2
A3
Table B
B1
B2
B3
B4
FlatDB Model Aggregate
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

9. Properties - Latency
Traditional System Event
Hub
Data
Flow
RDBMS
latency
latency
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

10. Properties – Anti-Corruption Layer (ACL)
Traditional System Event
Hub
Data
Flow
RDBMS
Traditional System Event
Hub
Data
Flow
RDBMS
ACL
ACL
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense
Examples:
• View Layer
• Storage Procedure
• JSON Support in DB
• …
Examples:
• StreamSets
• Kafka Connect
• Kafka Streams / KSQL
• …
Database Dataflow

11. Blueprints Oracle RDBMS =>
Apache Kafka

12. Blueprints Oracle RDBMS => Apache Kafka (DB-K)
Customer Microservice
{ }
Customer API CustomerCustomer Logic
Order Processing System
{ }
Order API OrderOrder Logic
REST
REST
Event Hub
Customer
Mat View
Order
(compacted)
Customer
(compacted)
Notification Microservice
Notification Logic
Schema
Registry
DB-K_1: Polling of RDBMS table/view
DB-K_2: Change Data Capture (CDC) on RDBMS
DB-K_3: Polling of RDBMS API
DB-K_4: Produce to Event Hub from RDBMS
DB-K_5: RDBMS Que with bridge to Event Hub
DB_K-1
DB_K-2
DB_K-3
DB_K-4
DB_K-5

13. DB-K_1: Polling of RDBMS table/view
Event
Hub
Stream Data
Integration
API
Applications / Data Sources
Data FlowRDBMS
Application
Logic
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

14. Event
Hub
Stream Data
Integration
API
Applications / Data Sources
Data FlowRDBMS
Application
Logic
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense
DB-K_1: Polling of RDBMS table/view
Kafka Connect with JDBC Source Connector

15. Kafka Connect & JDBC Connector
• Many connectors available
• Single Message Transforms (SMT)
• declarative style, simple data flows
• framework is part of Apache Kafka

16. DB-K_1 – Configure JDBC Connector
#!/bin/bash
curl -X "POST" "http://192.168.69.138:8083/connectors"
-H "Content-Type: application/json"
-d $'{
"name": "jdbc-driver-source",
"config": {
"connector.class": "JdbcSourceConnector",
"connection.url":"jdbc:oracle:thin//oracle-db:1521/XEPDB1",
"mode": "timestamp",
"timestamp.column.name":”modified_at",
"table.whitelist":”order",
"validate.non.null":"false",
"topic.prefix":”orderprocessing_",
"key.converter":"org.apache.kafka.connect.json.JsonConverter",
"key.converter.schemas.enable": "false",
"value.converter":"org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "false",
"name": "jdbc-driver-source",
"transforms":"createKey,extractInt",
"transforms.createKey.type":"org.apache.kafka.connect.transforms.ValueToKey",
"transforms.createKey.fields":"id",
"transforms.extractInt.type":"org.apache.kafka.connect.transforms.ExtractField$Key",
"transforms.extractInt.field":"id"
}
}'

17. DB-K_2: Change Data Capture (CDC) on RDBMS
Stream Data
Integration & Analytics
Stream
Analytics
Event
Hub
Stream Data
Integration
API
Data Flow
Application / Data Sources
Data Flow
Application
Logic
RDBMS
Redo Log
REST to
Event Hub
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

18. Stream Data
Integration & Analytics
Stream
Analytics
Event
Hub
Stream Data
Integration
API
Data Flow
Application / Data Sources
Data Flow
Application
Logic
RDBMS
Redo Log
REST to
Event Hub
Rest Proxy
DB-K_2: Change Data Capture (CDC) on RDBMS
Using Oracle GoldenGate
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense
Alternatives:
StreamSets Data Collector
Attunity
Debezium
…

19. DB-K_3: Polling of RDBMS API
Event
Hub
Stream Data
Integration
API
Applications / Data Sources
Data Flow
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

20. Event
Hub
Stream Data
Integration
API
Applications / Data Sources
Data Flow
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
DB-K_3: Polling of RDBMS API
StreamSets invokes Oracle Rest Data Service
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

21. Oracle REST Data Services (ORDS)
• makes it easy to develop modern REST interfaces for relational data in the
Oracle Database and the Oracle Database 18c JSON Document Store
• ORDS maps HTTP(S) verbs (GET, POST, PUT, DELETE, etc.) to database
transactions and returns any results formatted using JSON
• Java middle tier application on WebLogic, Tomcat, Docker, Standalone (for
development)

22. DB-K_3 – Setup ORDS (I)
ORDS.ENABLE_SCHEMA(
p_enabled => TRUE,
p_schema => 'ORDER_PROCESSING',
p_url_mapping_type => 'BASE_PATH',
p_url_mapping_pattern => 'order_processing',
p_auto_rest_auth => FALSE);
ORDS.DEFINE_MODULE(
p_module_name => 'order_processing',
p_base_path => '/orders/',
p_items_per_page => 25,
p_status => 'PUBLISHED',
p_comments => NULL);
ORDS.DEFINE_TEMPLATE(
p_module_name => 'order_processing',
p_pattern => 'changes/:offset',
p_priority => 0,
p_etag_type => 'HASH',
p_etag_query => NULL,
p_comments => NULL);

23. DB-K_3 – Setup ORDS (II)
ORDS.DEFINE_HANDLER(
p_module_name => 'order_processing',
p_pattern => 'changes/:offset',
p_method => 'GET',
p_source_type => 'resource/lob',
p_items_per_page => 25,
p_source =>
'SELECT ''application/json'', json_object(''orderId'' VALUE po.id,
''orderDate'' VALUE po.order_date,
''orderMode'' VALUE po.order_mode,
''customer'' VALUE
json_object(''firstName'' VALUE cu.first_name,
''lastName'' VALUE cu.last_name
''emailAddress'' VALUE cu.email),
''lineItems'' VALUE (SELECT json_arrayagg(
json_object(''ItemNumber'' VALUE li.id,
''Product'' VALUE
json_object(''id'' VALUE li.product_id,
''name'' VALUE li.product_name,
''unitPrice'' VALUE li.unit_price),
''quantity'' VALUE li.quantity))
FROM order_item_t li WHERE po.id = li.order_id),
''offset'' VALUE TO_CHAR(po.modified_at, ''YYYYMMDDHH24MISS''))
FROM order_t po LEFT JOIN customer_t cu ON (po.customer_id = cu.id)
WHERE po.modified_at > TO_DATE(:offset, ''YYYYMMDDHH24MISS'')'

24. StreamSets Data Collector
• GUI-based, drag-and
drop Data Flow Pipelines
• Both stream and batch
processing
• custom sources, sinks,
processors
• Monitoring and Error
Detection

25. DB-K_4: Produce to Event Hub from RDBMS
Event
Hub
Stream Data
Integration
API
Applications / Data Sources
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
REST to
Event Hub
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

26. Event
Hub
Stream Data
Integration
API
Applications / Data Sources
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
REST to
Event Hub
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense
DB-K_4: Produce to Event Hub from RDBMS
Native Kafka Producer using Java in DB
Does not feel right!

27. Event
Hub
Stream Data
Integration
API
Applications / Data Sources
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
REST to
Event Hub
Rest Proxy
?
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense
DB-K_4: Produce to Event Hub from RDBMS
Invoke REST Proxy from PL/SQL
Invoking a REST Service
from DB not well-supported

28. Event
Hub
Stream Data
Integration
API
Applications / Data Sources
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
REST to
Event Hub
Oracle Big Data SQL
Coming soon …
DB-K_4: Produce to Event Hub from RDBMS
Oracle Big Data SQL integrates with Kafka
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

29. DB-K_5: RDBMS Queue with bridge to Event Hub
Stream Data
Integration & Analytics
Stream
Analytics
Event
Hub
Stream Data
Integration
API
Data Flow
Application / Data Sources
Data Flow
Application
Logic
RDBMS
Queue
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

30. DB-K_5: RDBMS Queue with bridge to Event Hub
Oracle Advanced Queuing & Kafka Connect JMS
Stream Data
Integration & Analytics
Stream
Analytics
Event
Hub
Stream Data
Integration
API
Data Flow
Application / Data Sources
Data Flow
Application
Logic
RDBMS
QueueAQ
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

31. BEGIN
dbms_aqadm.create_queue_table (
queue_table => 'ORDER_QT',
queue_payload_type => 'SYS.AQ$_JMS_TEXT_MESSAGE',
sort_list => 'PRIORITY,ENQ_TIME',
multiple_consumers => FALSE,
message_grouping => dbms_aqadm.none
);
END;
/
DB-K_5 – Setup AQ and Kafka Connect (I)
BEGIN
dbms_aqadm.create_queue (
queue_name => 'ORDER_AQ',
queue_table => 'ORDER_QT',
max_retries => 1,
retry_delay => 2, -- seconds
retention_time => 60*60*24*7 -- 1w
);
END;
/
BEGIN
dbms_aqadm.start_queue(
queue_name => 'ORDER_AQ',
enqueue => TRUE,
dequeue => TRUE
);
END;
/

32. curl -X "POST" "$DOCKER_HOST_IP:8083/connectors"
-H "Content-Type: application/json"
--data '{
"name": "jms-source",
"config": {
"name": "jms-source",
"connector.class": "com.datamountaineer...JMSSourceConnector",
"connect.jms.initial.context.factory":
"oracle.jms.AQjmsInitialContextFactory",
"connect.jms.initial.context.extra.params":
"db_url=jdbc:oracle:thin:@//192.168.73.86:1521/XEPDB1,java.naming.security.princ
ipal=order_processing,java.naming.security.credentials=order_processing",
"tasks.max": "1",
"connect.jms.connection.factory": "ConnectionFactory",
"connect.jms.url": "jdbc:oracle:thin:@//192.168.73.86:1521/XEPDB1",
"connect.jms.kcql": "INSERT INTO order SELECT * FROM order_aq WITHTYPE QUEUE
WITHCONVERTER=`com.datamountaineer.streamreactor.connect.converters.source.JsonS
impleConverter`"
}
}'
DB-K_5 – Setup AQ and Kafka Connect (I)

33. DB-K_5: RDBMS Queue with bridge to Event Hub
Oracle AQ with Kafka API & MirrorMaker
Stream Data
Integration & Analytics
Stream
Analytics
Event
Hub
Stream Data
Integration
API
Data Flow
Application / Data Sources
Data Flow
Application
Logic
RDBMS
Queue
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense
AQ (Kafka API)
Oracle works on a Kafka API
for Advanced Queuing

34. Blueprints Apache Kafka =>
Oracle RDBMS

35. Blueprints Apache Kafka => Oracle RDBMS (K-DB)
Customer Microservice
{ }
Customer API CustomerCustomer Logic
Order Processing System
{ }
Order API OrderOrder Logic
REST
REST
Event Hub
Customer
Mat View
Order
(compacted)
Customer
(compacted)
Notification Microservice
Notification Logic
Schema
Registry
K-DB_1: Write to RDBMS table/view
K-DB_2: Write over RDBMS API
K-DB_3: Consume from Event Hub
K-DB_4: Event Hub with bridge to RDBMS Queue
K_DB-1
K_DB-2
K_DB-3
K_DB-4

36. K-DB_1: Write to RDBMS table/view
Event
Hub
Stream Data
Integration
API
Applications / Data Sources
Data FlowRDBMS
Application
Logic
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

37. Event
Hub
Stream Data
Integration
API
Applications / Data Sources
Data FlowRDBMS
Application
Logic
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense
K-DB_1: Write to RDBMS table/view
Kafka Connect and JDBC Sink Connector

38. K-DB_2: Write over RDBMS API
Event
Hub
Stream Data
Integration
API
Applications / Data Sources
Data Flow
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

39. Event
Hub
Stream Data
Integration
API
Applications / Data Sources
Data Flow
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense
K-DB_2: Write over RDBMS API
Kafka Connect invokes Oracle Rest Data Service

40. DB-K_2 – Setup ORDS (I)
ORDS.DEFINE_HANDLER(
p_module_name => 'customer',
p_pattern => 'customer',
p_method => 'POST',
p_source_type => 'plsql/block',
p_items_per_page => 0,
p_source =>
'DECLARE
L_CU CLOB := :body_text;
BEGIN
INSERT INTO customer_t (id, first_name, last_name, title, notification_on, email, slack_handle,
twitter_handle)
SELECT * FROM json_table(L_CU, ''$''
COLUMNS (
id NUMBER PATH ''$.id'',
first_name VARCHAR2 PATH ''$.firstName'',
last_name VARCHAR2 PATH ''$.lastName'',
title VARCHAR2 PATH ''$.title'',
notification_on VARCHAR2 PATH ''$.notificationOn'',
email VARCHAR2 PATH ''$.email'',
slack_handle VARCHAR2 PATH ''$.slackHandle'',
twitter_handle VARCHAR2 PATH ''$.twitterHandle''
));
INSERT INTO address_t (customer_id, id, street, nr, city, postcode, country)
SELECT * FROM json_table( ... )

41. K-DB_3: Consume from Event Hub
Event
Hub
Stream Data
Integration
API
Applications / Data Sources
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
REST to
Event Hub
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

42. Event
Hub
Stream Data
Integration
API
Applications / Data Sources
RDBMS
Application
Logic
API
Stream Data
Integration & Analytics
Stream
Analytics
Data Flow
REST to
Event Hub
Oracle Big Data SQL
Coming soon…
K-DB_3: Consume from Event Hub
Oracle Big Data SQL exposes topic as table
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

43. K-DB_4: Event Hub with bridge to RDBMS queue
Stream Data
Integration & Analytics
Stream
Analytics
Event
Hub
Stream Data
Integration
API
Data Flow
Application / Data Sources
Data Flow
Application
Logic
RDBMS
Queue
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

44. Stream Data
Integration & Analytics
Stream
Analytics
Event
Hub
Stream Data
Integration
API
Data Flow
Application / Data Sources
Data Flow
Application
Logic
RDBMS
QueueAQ
K-DB_4: Event Hub with bridge to RDBMS queue
Oracle Advanced Queuing & Kafka Connect JMS
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

45. Stream Data
Integration & Analytics
Stream
Analytics
Event
Hub
Stream Data
Integration
API
Data Flow
Application / Data Sources
Data Flow
Application
Logic
RDBMS
QueueAQ (Kafka API)
Oracle works on a Kafka API
for Advanced Queuing
K-DB_4: Event Hub with bridge to RDBMS queue
Oracle AQ with Kafka API & MirrorMaker
Flat Aggregate
Low Latency High Latency
DB Dataflow
Message
Latency
ACL
Open Source CommercialLicense

46. Summary

47. Summary
Customer Microservice
{ }
Customer API CustomerCustomer Logic
Order Processing System
{ }
Order API OrderOrder Logic
REST
REST
Event Hub
Customer
Mat View
Order
(compacted)
Customer
(compacted)
Notification Microservice
Notification Logic
Schema
Registry
K-DB_1: Write to RDBMS table/view
K-DB_2: Write over RDBMS API
K-DB_3: Consume from Event Hub
K-DB_4: Event Hub with bridge to RDBMS Queue
K_DB-1
K_DB-2
K_DB-3
K_DB-4
DB_K-1
DB_K-2
DB_K-3
DB_K-4
DB_K-5
https://github.com/gschmutz/various-demos/tree/master/bidirectional-integration-oracle-kafka
DB-K_1: Polling of RDBMS table/view
DB-K_2: Change Data Capture (CDC) on RDBMS
DB-K_3: Polling of RDBMS API
DB-K_4: Produce to Event Hub from RDBMS
DB-K_5: RDBMS Queue with bridge to Event Hub

48. Bulk Source
Ref Architecture
Data Platform
Service
Event
Stream
Bulk
Data
Flow
Event Source
Location
DB
Extract
File
Weather
DB
IoT
Data
Mobile
Apps
Social
File Import / SQL Import
Consumer
BI Apps
Data Science
Workbench
Enterprise
App
Enterprise Data
Warehouse
SQL / Search
SQL
“Native” Raw
RDBMS
“SQL” / Search
Service
Event
Hub
Hadoop ClusterdHadoop ClusterBig Data Platform
SQL
Export
Storage
Storage
Raw
Refined/
UsageOpt
Microservice Cluster
Stream Processing Cluster
Stream
Processor
Model /
State
Edge Node
Rules
Event Hub
Storage
Governance
Data Catalog
Rules
Engine
Parallel
Processing
Query
Engine
Microservice Data
{ }
API
Event
Stream
Event Stream
Modern Data Platform
Event Stream

49. Bulk Source
Ref Architecture
Data Platform
Service
Event
Stream
Bulk
Data
Flow
Event Source
Location
DB
Extract
File
Weather
DB
IoT
Data
Mobile
Apps
Social
File Import / SQL Import
Consumer
BI Apps
Data Science
Workbench
Enterprise
App
Enterprise Data
Warehouse
SQL / Search
SQL
“Native” Raw
RDBMS
“SQL” / Search
Service
sEvent
Hub
Hadoop ClusterdHadoop ClusterBig Data Platform
SQL
Export
Storage
Storage
Raw
Refined/
UsageOpt
Microservice Cluster
Stream Processing Cluster
Stream
Processor
Model /
State
Edge Node
Rules
Event Hub
Storage
Governance
Data Catalog
Rules
Engine
Parallel
Processing
Query
Engine
Microservice Data
{ }
API
Event
Stream
Event Stream
Modern Data Platform
Event Stream