SlideShare a Scribd company logo
1 of 68
Download to read offline
1
Building Event Driven Services
with Kafka Streams
Ben Stopford
Technologist, Office of the CTO
@benstopford
2
There is a book!
http://bit.ly/designing-event-driven-systems
3
Storage
Messaging
Connect
KSQL
Messaging
Database
4
Build
Features
Build for
the Future
5
Evolution!
6
EcosystemsApp
Increasingly we build ecosystems
7
But these aren’t static, they grow
7
8
Span data centers
US East 1 US East 2
9
Span regions or clouds
10
Handle Disconnectedness
11
Span teams and departments
12
How do you do this?
13
Microservices
HTTP
HTTP
HTTP
HTTP
HTTP HTTP
HTTP
HTTP
HTTP
HTTP
HTTP
HTTP
14
Returns
Service
Email
Service
T2’’
Tax
Service
Order
Service
Or we can use event streams
15
One Problem is DATA
16
Most services share the same core facts.
Catalog
Most services live
in here
17
A Global API
A Global Source of Truth
Events
18
Events have two hats
Notification Data
replication
19
Spectrum of use cases
Finer Grained,
Collaborative,
Connected
Courser Grained,
Non-collaborative,
Disconnected
Notification Data Replication
20
Buying an iPad (with REST/RPC)
Submit
Order
shipOrder() getCustomer()
Orders
Service
Shipping
Service
Customer
Service
Webserver
21
Events for Notification Only
Message Broker (Kafka)
Submit
Order
Order
Created
getCustomer()
REST
Notification
Orders
Service
Shipping
Service
Customer
Service
Webserver
KAFKA
22
Pluggability
Submit
Order
Order
Created
Orders
Service
Shipping
Service
Customer
Service
Webserver
KAFKA
Repricing
getCustomer()
REST
Notification
23
Events for Data Locality
Customer
Updated
Submit
Order
Order
Created
Data is
replicated
Orders
Service
Shipping
Service
Customer
Service
Webserver
KAFKA
24
Stateless / Stateful Stream Processing
Relates to these hats
25
Message Broker (Kafka)
Submit
Order
Order
Created
getCustomer()
REST
Orders
Service
Customer
Service
Webserver
KAFKA
KStreams
Shipping Service
Stateless Stream Processing
Notification
26
Message Broker (Kafka)
Submit
Order
Order
Created
Orders
Service
Customer
Service
Webserver
KAFKA
KStreams
Shipping Service
KTable
Customer
Updated
Stateful Stream Processing
Data
replication
27
Events have two hats
Notification
Data
Replication
Stateless Stream
Processing
Stateful Stream
Processing
28
Kafka Streams is an API library
2
29
Features
• Buffering
• Filters
• Join (Stream-Stream, Stream-Table, Table-
Table)
• Aggregations (Sum, Count...)
• Any arbitrary code
• HA (dynamic scaling, standby replicas,
checkpoints, etc.)
30
Building a simple KStreams Email Service
31
KAFKA
Buffer 5 mins
Send Email
Stateless Streaming
(stream-stream join)
32
KAFKA
Whole customer
dataset cached in
Email Service
Stateful Streaming
(Stream-Stream-Table join)
33
KAFKA
Whole customer
dataset cached in
Email Service
Customer information in 3 places
34
POST
GET
Load
Balancer
ORDERSORDERS
OVTOPIC
Order
Validations
KAFKA
INVENTORY
Orders
Inventory
Fraud
Service
Order
Details
Service
Inventory
Service
Order
Created
Order
Validated
Orders
Service
CQRS
Materialized View
35
POST
GET
KAFKA
Validate Order
Validate
Inventory
Validate FraudView
Proxy
CQRS
Validate Order
36
Validate Order
kStreams.stream(“Orders”)
.mapValues((key, order) ->
isValid(order) ? pass() : fail())
.to(“OrderValidations”);
37
POST
GET
KAFKA
Validate Order
Validate
Inventory
Validate FraudView
Proxy
Validate Warehouse Inventory
38
Ensure there is enough Stock
Order Event
Get # IPads in
Stock
- Validate Quantity
- Reserve Item
Atomic Compare and Set
39
Table
Stream
State Store
“Reserved Items” Topic
Orders
Topic
Warehouse Stock Topic
Orders Topic
1. Join
40
Table
Stream
State Store
“Reserved Items” Topic
Orders
Topic
Order Validations Topic
2. Reserve Stock
State store behaves like a
little database
Warehouse Stock Topic
41
KStreams Code for Inventory Service
//Create a State Store
kStreams.addStateStore(Stores.keyValueStoreBuilder(
Stores.persistentKeyValueStore(STOCK_STORE_NAME));
//Process
kStreams.stream(“Orders”)
.join(inventory, KeyValue::new)
.transform(InventoryValidator::new, STOCK_STORE_NAME)
.to(“OrderValidations”);
42
KStreams Code for Inventory Service
//Create a State Store
kStreams.addStateStore(Stores.keyValueStoreBuilder(
Stores.persistentKeyValueStore(STOCK_STORE_NAME));
//Process
kStreams.stream(“Orders”)
.join(inventory, KeyValue::new)
.transform(InventoryValidator::new, STOCK_STORE_NAME)
.to(“OrderValidations”);
43
KStreams Code for Inventory Service
//Create a State Store
kStreams.addStateStore(Stores.keyValueStoreBuilder(
Stores.persistentKeyValueStore(STOCK_STORE_NAME));
//Process
kStreams.stream(“Orders”)
.join(inventory, KeyValue::new)
.transform(InventoryValidator::new, STOCK_STORE_NAME)
.to(“OrderValidations”);
44
KStreams Code for Inventory Service
//Create a State Store
kStreams.addStateStore(Stores.keyValueStoreBuilder(
Stores.persistentKeyValueStore(STOCK_STORE_NAME));
//Process
kStreams.stream(“Orders”)
.join(inventory, KeyValue::new)
.transform(InventoryValidator::new, STOCK_STORE_NAME)
.to(“OrderValidations”);
45
Validate and update State Store
public KeyValue transform(…) {…
Long reserved = reservedStore.get(order.getProduct());
if (inWarehouse - reserved - order.quantity() >= 0) {
int amount = reserved + order.getQuantity();
reservedStore.put(order.product(), amount);
pass();
} else {
fail()
}
…
}
46
Validate and update State Store
public KeyValue transform(…) {…
Long reserved = reservedStore.get(order.getProduct());
if (inWarehouse - reserved - order.quantity() >= 0) {
int amount = reserved + order.getQuantity();
reservedStore.put(order.product(), amount);
pass();
} else {
fail()
}
…
}
47
Validate and update State Store
public KeyValue transform(…) {…
Long reserved = reservedStore.get(order.getProduct());
if (inWarehouse - reserved - order.quantity() >= 0) {
int amount = reserved + order.getQuantity();
reservedStore.put(order.product(), amount);
pass();
} else {
fail()
}
…
}
48
Validate and update State Store
public KeyValue transform(…) {…
Long reserved = reservedStore.get(order.getProduct());
if (inWarehouse - reserved - order.quantity() >= 0) {
int amount = reserved + order.getQuantity();
reservedStore.put(order.product(), amount);
pass();
} else {
fail()
}
…
}
49
Table
Stream
State Store
“Reserved Items” Topic
Orders
Topic
Inventory topic
Order Validations Topic
Transactional
Wrapper
State store behaves like a
little database
50
We need one more thing!
51
Rekey by Product
builder.stream(“Orders”)
.selectKey((id, order) -> order.getProduct())
.join(inventory, KeyValue::new)
.transform(InventoryValidator::new, STOCK_STORE_NAME)
.to(“OrderValidations”);
52
Rekey does two things in this case
- Collocate data that must join
- Serially execute critical section
against single data.
53
Instance 1 Instance 2
Partitioned by ProductId
Everything “iPad” Everything “iWatch”
Rekey: Collocates data that must join
JOIN JOIN
54
Instance 1 Instance 2
Partitioned by ProductId
Everything “iPad” Everything “iWatch”
- Rekey: Serially execute critical section against
single data.
CaS CaS
55
POST
GET KAFKA
Validate Order
Validate
Inventory
Validate FraudView
Proxy
CQRS
Validate Fraud
56
Fraud Service
“Fail” an order if the total value of orders
for that user eclipses a threshold.
57
Fraud Service
CREATE STREAM OrderValidations AS
SELECT userId, sum(amount)
FROM Orders
WINDOW TUMBLING(SIZE 60 MINUTE)
GROUP BY userId
HAVING sum(amount) > 10000;
58
POST
GET
KAFKA
Validate Order
Validate
Inventory
Validate FraudView
Proxy
Mapping Synchronous and
Asynchronous
59
Create a materialized view
kStreams.stream(“Orders”)
.groupByKey()
.reduce((agg, v) -> v, STORE_NAME));
60
POST
GET
Load
Balancer
ORDERSORDERS
OVTOPIC
Order
Validations
KAFKA
INVENTORY
Orders
Inventory
Fraud
Service
Order
Details
Service
Inventory
Service
Order
Created
Order
Validated
Orders
Service
CQRS
Materialized View
Fine grained Ecosystem
61
Gotchas/Guidelines
- Use Schemas (Schema registry)
- Global KTables & KTables are subtly different on replay.
- Stream-table joins only join one way
- The defaults aren’t enough. See resiliency settings in the
documentation.
- Releasing
- Change application id trick:
- loses committed offsets
- Doesn’t delete topics, leaves them hanging
- Name state store changelog topics
- Start simple (Stateless)
62
So…
63
Good architectures have little to do
with this:
64
It’s about how systems evolves over time
65
Events provide the key to evolutionary
architectures
Notification Data
replication
66
Spectrum of use cases
Finer Grained,
Collaborative,
Connected
Courser Grained,
Non-collaborative,
Disconnected
Notification Data Replication
67
Streaming is the toolset
for dealing with events
as they move
68
Thank You
@benstopford
Book: http://bit.ly/designing-event-driven-systems
Twitter: @benstopford
Slides: http://www.benstopford.com/

More Related Content

What's hot

[Spring Camp 2018] 11번가 Spring Cloud 기반 MSA로의 전환 : 지난 1년간의 이야기
[Spring Camp 2018] 11번가 Spring Cloud 기반 MSA로의 전환 : 지난 1년간의 이야기[Spring Camp 2018] 11번가 Spring Cloud 기반 MSA로의 전환 : 지난 1년간의 이야기
[Spring Camp 2018] 11번가 Spring Cloud 기반 MSA로의 전환 : 지난 1년간의 이야기YongSung Yoon
 
Microservices in the Apache Kafka Ecosystem
Microservices in the Apache Kafka EcosystemMicroservices in the Apache Kafka Ecosystem
Microservices in the Apache Kafka Ecosystemconfluent
 
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Kai Wähner
 
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...confluent
 
Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout
Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron SchildkroutKafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout
Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkroutconfluent
 
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)Kai Wähner
 
Apache Kafka 0.8 basic training - Verisign
Apache Kafka 0.8 basic training - VerisignApache Kafka 0.8 basic training - Verisign
Apache Kafka 0.8 basic training - VerisignMichael Noll
 
APACHE KAFKA / Kafka Connect / Kafka Streams
APACHE KAFKA / Kafka Connect / Kafka StreamsAPACHE KAFKA / Kafka Connect / Kafka Streams
APACHE KAFKA / Kafka Connect / Kafka StreamsKetan Gote
 
Real Time Integration with Salesforce Platform Events
Real Time Integration with Salesforce Platform EventsReal Time Integration with Salesforce Platform Events
Real Time Integration with Salesforce Platform EventsSalesforce Developers
 
컨테이너 (PaaS) 환경으로의 애플리케이션 전환 방법과 고려사항
컨테이너 (PaaS) 환경으로의 애플리케이션 전환 방법과 고려사항컨테이너 (PaaS) 환경으로의 애플리케이션 전환 방법과 고려사항
컨테이너 (PaaS) 환경으로의 애플리케이션 전환 방법과 고려사항Opennaru, inc.
 
MicroCPH - Managing data consistency in a microservice architecture using Sagas
MicroCPH - Managing data consistency in a microservice architecture using SagasMicroCPH - Managing data consistency in a microservice architecture using Sagas
MicroCPH - Managing data consistency in a microservice architecture using SagasChris Richardson
 
Communication in a Microservice Architecture
Communication in a Microservice ArchitectureCommunication in a Microservice Architecture
Communication in a Microservice ArchitecturePer Bernhardt
 
Reactive Microservices with Quarkus
Reactive Microservices with QuarkusReactive Microservices with Quarkus
Reactive Microservices with QuarkusNiklas Heidloff
 
Data Pipelines with Apache Kafka
Data Pipelines with Apache KafkaData Pipelines with Apache Kafka
Data Pipelines with Apache KafkaBen Stopford
 
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...Yevgeniy Brikman
 
Developing functional domain models with event sourcing (sbtb, sbtb2015)
Developing functional domain models with event sourcing (sbtb, sbtb2015)Developing functional domain models with event sourcing (sbtb, sbtb2015)
Developing functional domain models with event sourcing (sbtb, sbtb2015)Chris Richardson
 
Stream Processing with Apache Kafka and .NET
Stream Processing with Apache Kafka and .NETStream Processing with Apache Kafka and .NET
Stream Processing with Apache Kafka and .NETconfluent
 
Toi uu hoa he thong 30 trieu nguoi dung
Toi uu hoa he thong 30 trieu nguoi dungToi uu hoa he thong 30 trieu nguoi dung
Toi uu hoa he thong 30 trieu nguoi dungIT Expert Club
 
Introduction to Apache Camel
Introduction to Apache CamelIntroduction to Apache Camel
Introduction to Apache CamelClaus Ibsen
 
Apache Kafka for Automotive Industry, Mobility Services & Smart City
Apache Kafka for Automotive Industry, Mobility Services & Smart CityApache Kafka for Automotive Industry, Mobility Services & Smart City
Apache Kafka for Automotive Industry, Mobility Services & Smart CityKai Wähner
 

What's hot (20)

[Spring Camp 2018] 11번가 Spring Cloud 기반 MSA로의 전환 : 지난 1년간의 이야기
[Spring Camp 2018] 11번가 Spring Cloud 기반 MSA로의 전환 : 지난 1년간의 이야기[Spring Camp 2018] 11번가 Spring Cloud 기반 MSA로의 전환 : 지난 1년간의 이야기
[Spring Camp 2018] 11번가 Spring Cloud 기반 MSA로의 전환 : 지난 1년간의 이야기
 
Microservices in the Apache Kafka Ecosystem
Microservices in the Apache Kafka EcosystemMicroservices in the Apache Kafka Ecosystem
Microservices in the Apache Kafka Ecosystem
 
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
Architecture patterns for distributed, hybrid, edge and global Apache Kafka d...
 
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...
Apache Kafka vs. Integration Middleware (MQ, ETL, ESB) - Friends, Enemies or ...
 
Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout
Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron SchildkroutKafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout
Kafka + Uber- The World’s Realtime Transit Infrastructure, Aaron Schildkrout
 
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
Confluent REST Proxy and Schema Registry (Concepts, Architecture, Features)
 
Apache Kafka 0.8 basic training - Verisign
Apache Kafka 0.8 basic training - VerisignApache Kafka 0.8 basic training - Verisign
Apache Kafka 0.8 basic training - Verisign
 
APACHE KAFKA / Kafka Connect / Kafka Streams
APACHE KAFKA / Kafka Connect / Kafka StreamsAPACHE KAFKA / Kafka Connect / Kafka Streams
APACHE KAFKA / Kafka Connect / Kafka Streams
 
Real Time Integration with Salesforce Platform Events
Real Time Integration with Salesforce Platform EventsReal Time Integration with Salesforce Platform Events
Real Time Integration with Salesforce Platform Events
 
컨테이너 (PaaS) 환경으로의 애플리케이션 전환 방법과 고려사항
컨테이너 (PaaS) 환경으로의 애플리케이션 전환 방법과 고려사항컨테이너 (PaaS) 환경으로의 애플리케이션 전환 방법과 고려사항
컨테이너 (PaaS) 환경으로의 애플리케이션 전환 방법과 고려사항
 
MicroCPH - Managing data consistency in a microservice architecture using Sagas
MicroCPH - Managing data consistency in a microservice architecture using SagasMicroCPH - Managing data consistency in a microservice architecture using Sagas
MicroCPH - Managing data consistency in a microservice architecture using Sagas
 
Communication in a Microservice Architecture
Communication in a Microservice ArchitectureCommunication in a Microservice Architecture
Communication in a Microservice Architecture
 
Reactive Microservices with Quarkus
Reactive Microservices with QuarkusReactive Microservices with Quarkus
Reactive Microservices with Quarkus
 
Data Pipelines with Apache Kafka
Data Pipelines with Apache KafkaData Pipelines with Apache Kafka
Data Pipelines with Apache Kafka
 
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...How to test infrastructure code: automated testing for Terraform, Kubernetes,...
How to test infrastructure code: automated testing for Terraform, Kubernetes,...
 
Developing functional domain models with event sourcing (sbtb, sbtb2015)
Developing functional domain models with event sourcing (sbtb, sbtb2015)Developing functional domain models with event sourcing (sbtb, sbtb2015)
Developing functional domain models with event sourcing (sbtb, sbtb2015)
 
Stream Processing with Apache Kafka and .NET
Stream Processing with Apache Kafka and .NETStream Processing with Apache Kafka and .NET
Stream Processing with Apache Kafka and .NET
 
Toi uu hoa he thong 30 trieu nguoi dung
Toi uu hoa he thong 30 trieu nguoi dungToi uu hoa he thong 30 trieu nguoi dung
Toi uu hoa he thong 30 trieu nguoi dung
 
Introduction to Apache Camel
Introduction to Apache CamelIntroduction to Apache Camel
Introduction to Apache Camel
 
Apache Kafka for Automotive Industry, Mobility Services & Smart City
Apache Kafka for Automotive Industry, Mobility Services & Smart CityApache Kafka for Automotive Industry, Mobility Services & Smart City
Apache Kafka for Automotive Industry, Mobility Services & Smart City
 

Similar to Building Event Driven Services with Kafka Streams

Bringing critical applications to the cloud - Luc Bos
Bringing critical applications to the cloud - Luc BosBringing critical applications to the cloud - Luc Bos
Bringing critical applications to the cloud - Luc BosInspireX
 
APAC ksqlDB Workshop
APAC ksqlDB WorkshopAPAC ksqlDB Workshop
APAC ksqlDB Workshopconfluent
 
Embrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with RippleEmbrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with RippleSean Cribbs
 
Azure SQL Database - Connectivity Best Practices
Azure SQL Database - Connectivity Best PracticesAzure SQL Database - Connectivity Best Practices
Azure SQL Database - Connectivity Best PracticesJose Manuel Jurado Diaz
 
Building a Streaming Platform with Kafka
Building a Streaming Platform with KafkaBuilding a Streaming Platform with Kafka
Building a Streaming Platform with Kafkaconfluent
 
Kick your database_to_the_curb_reston_08_27_19
Kick your database_to_the_curb_reston_08_27_19Kick your database_to_the_curb_reston_08_27_19
Kick your database_to_the_curb_reston_08_27_19confluent
 
Microsoft Windows Server AppFabric
Microsoft Windows Server AppFabricMicrosoft Windows Server AppFabric
Microsoft Windows Server AppFabricMark Ginnebaugh
 
Scale Your Data Tier with Windows Server AppFabric
Scale Your Data Tier with Windows Server AppFabricScale Your Data Tier with Windows Server AppFabric
Scale Your Data Tier with Windows Server AppFabricWim Van den Broeck
 
10 essentials steps for kafka streaming services
10 essentials steps for kafka streaming services10 essentials steps for kafka streaming services
10 essentials steps for kafka streaming servicesinovia
 
Building Event Driven Services with Apache Kafka and Kafka Streams - Devoxx B...
Building Event Driven Services with Apache Kafka and Kafka Streams - Devoxx B...Building Event Driven Services with Apache Kafka and Kafka Streams - Devoxx B...
Building Event Driven Services with Apache Kafka and Kafka Streams - Devoxx B...Ben Stopford
 
Avoiding the Pit of Despair - Event Sourcing with Akka and Cassandra
Avoiding the Pit of Despair - Event Sourcing with Akka and CassandraAvoiding the Pit of Despair - Event Sourcing with Akka and Cassandra
Avoiding the Pit of Despair - Event Sourcing with Akka and CassandraLuke Tillman
 
NDC London 2017 - The Data Dichotomy- Rethinking Data and Services with Streams
NDC London 2017  - The Data Dichotomy- Rethinking Data and Services with StreamsNDC London 2017  - The Data Dichotomy- Rethinking Data and Services with Streams
NDC London 2017 - The Data Dichotomy- Rethinking Data and Services with StreamsBen Stopford
 
Oracle Drivers configuration for High Availability
Oracle Drivers configuration for High AvailabilityOracle Drivers configuration for High Availability
Oracle Drivers configuration for High AvailabilityLudovico Caldara
 
Stephane Lapointe, Frank Boucher & Alexandre Brisebois: Les micro-services et...
Stephane Lapointe, Frank Boucher & Alexandre Brisebois: Les micro-services et...Stephane Lapointe, Frank Boucher & Alexandre Brisebois: Les micro-services et...
Stephane Lapointe, Frank Boucher & Alexandre Brisebois: Les micro-services et...MSDEVMTL
 
Query service in vCloud Director
Query service in vCloud DirectorQuery service in vCloud Director
Query service in vCloud DirectorMayank Goyal
 
Composable and streamable Play apps
Composable and streamable Play appsComposable and streamable Play apps
Composable and streamable Play appsYevgeniy Brikman
 
Ingesting streaming data for analysis in apache ignite (stream sets theme)
Ingesting streaming data for analysis in apache ignite (stream sets theme)Ingesting streaming data for analysis in apache ignite (stream sets theme)
Ingesting streaming data for analysis in apache ignite (stream sets theme)Tom Diederich
 
[WSO2Con EU 2017] Streaming Analytics Patterns for Your Digital Enterprise
[WSO2Con EU 2017] Streaming Analytics Patterns for Your Digital Enterprise[WSO2Con EU 2017] Streaming Analytics Patterns for Your Digital Enterprise
[WSO2Con EU 2017] Streaming Analytics Patterns for Your Digital EnterpriseWSO2
 
The art of the event streaming application: streams, stream processors and sc...
The art of the event streaming application: streams, stream processors and sc...The art of the event streaming application: streams, stream processors and sc...
The art of the event streaming application: streams, stream processors and sc...confluent
 

Similar to Building Event Driven Services with Kafka Streams (20)

Bringing critical applications to the cloud - Luc Bos
Bringing critical applications to the cloud - Luc BosBringing critical applications to the cloud - Luc Bos
Bringing critical applications to the cloud - Luc Bos
 
APAC ksqlDB Workshop
APAC ksqlDB WorkshopAPAC ksqlDB Workshop
APAC ksqlDB Workshop
 
Embrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with RippleEmbrace NoSQL and Eventual Consistency with Ripple
Embrace NoSQL and Eventual Consistency with Ripple
 
Azure SQL Database - Connectivity Best Practices
Azure SQL Database - Connectivity Best PracticesAzure SQL Database - Connectivity Best Practices
Azure SQL Database - Connectivity Best Practices
 
Building a Streaming Platform with Kafka
Building a Streaming Platform with KafkaBuilding a Streaming Platform with Kafka
Building a Streaming Platform with Kafka
 
Kick your database_to_the_curb_reston_08_27_19
Kick your database_to_the_curb_reston_08_27_19Kick your database_to_the_curb_reston_08_27_19
Kick your database_to_the_curb_reston_08_27_19
 
Viadeo - The Kasper way
Viadeo - The Kasper wayViadeo - The Kasper way
Viadeo - The Kasper way
 
Microsoft Windows Server AppFabric
Microsoft Windows Server AppFabricMicrosoft Windows Server AppFabric
Microsoft Windows Server AppFabric
 
Scale Your Data Tier with Windows Server AppFabric
Scale Your Data Tier with Windows Server AppFabricScale Your Data Tier with Windows Server AppFabric
Scale Your Data Tier with Windows Server AppFabric
 
10 essentials steps for kafka streaming services
10 essentials steps for kafka streaming services10 essentials steps for kafka streaming services
10 essentials steps for kafka streaming services
 
Building Event Driven Services with Apache Kafka and Kafka Streams - Devoxx B...
Building Event Driven Services with Apache Kafka and Kafka Streams - Devoxx B...Building Event Driven Services with Apache Kafka and Kafka Streams - Devoxx B...
Building Event Driven Services with Apache Kafka and Kafka Streams - Devoxx B...
 
Avoiding the Pit of Despair - Event Sourcing with Akka and Cassandra
Avoiding the Pit of Despair - Event Sourcing with Akka and CassandraAvoiding the Pit of Despair - Event Sourcing with Akka and Cassandra
Avoiding the Pit of Despair - Event Sourcing with Akka and Cassandra
 
NDC London 2017 - The Data Dichotomy- Rethinking Data and Services with Streams
NDC London 2017  - The Data Dichotomy- Rethinking Data and Services with StreamsNDC London 2017  - The Data Dichotomy- Rethinking Data and Services with Streams
NDC London 2017 - The Data Dichotomy- Rethinking Data and Services with Streams
 
Oracle Drivers configuration for High Availability
Oracle Drivers configuration for High AvailabilityOracle Drivers configuration for High Availability
Oracle Drivers configuration for High Availability
 
Stephane Lapointe, Frank Boucher & Alexandre Brisebois: Les micro-services et...
Stephane Lapointe, Frank Boucher & Alexandre Brisebois: Les micro-services et...Stephane Lapointe, Frank Boucher & Alexandre Brisebois: Les micro-services et...
Stephane Lapointe, Frank Boucher & Alexandre Brisebois: Les micro-services et...
 
Query service in vCloud Director
Query service in vCloud DirectorQuery service in vCloud Director
Query service in vCloud Director
 
Composable and streamable Play apps
Composable and streamable Play appsComposable and streamable Play apps
Composable and streamable Play apps
 
Ingesting streaming data for analysis in apache ignite (stream sets theme)
Ingesting streaming data for analysis in apache ignite (stream sets theme)Ingesting streaming data for analysis in apache ignite (stream sets theme)
Ingesting streaming data for analysis in apache ignite (stream sets theme)
 
[WSO2Con EU 2017] Streaming Analytics Patterns for Your Digital Enterprise
[WSO2Con EU 2017] Streaming Analytics Patterns for Your Digital Enterprise[WSO2Con EU 2017] Streaming Analytics Patterns for Your Digital Enterprise
[WSO2Con EU 2017] Streaming Analytics Patterns for Your Digital Enterprise
 
The art of the event streaming application: streams, stream processors and sc...
The art of the event streaming application: streams, stream processors and sc...The art of the event streaming application: streams, stream processors and sc...
The art of the event streaming application: streams, stream processors and sc...
 

More from Ben Stopford

10 Principals for Effective Event-Driven Microservices with Apache Kafka
10 Principals for Effective Event-Driven Microservices with Apache Kafka10 Principals for Effective Event-Driven Microservices with Apache Kafka
10 Principals for Effective Event-Driven Microservices with Apache KafkaBen Stopford
 
10 Principals for Effective Event Driven Microservices
10 Principals for Effective Event Driven Microservices10 Principals for Effective Event Driven Microservices
10 Principals for Effective Event Driven MicroservicesBen Stopford
 
The Future of Streaming: Global Apps, Event Stores and Serverless
The Future of Streaming: Global Apps, Event Stores and ServerlessThe Future of Streaming: Global Apps, Event Stores and Serverless
The Future of Streaming: Global Apps, Event Stores and ServerlessBen Stopford
 
A Global Source of Truth for the Microservices Generation
A Global Source of Truth for the Microservices GenerationA Global Source of Truth for the Microservices Generation
A Global Source of Truth for the Microservices GenerationBen Stopford
 
Building Event Driven Services with Stateful Streams
Building Event Driven Services with Stateful StreamsBuilding Event Driven Services with Stateful Streams
Building Event Driven Services with Stateful StreamsBen Stopford
 
Devoxx London 2017 - Rethinking Services With Stateful Streams
Devoxx London 2017 - Rethinking Services With Stateful StreamsDevoxx London 2017 - Rethinking Services With Stateful Streams
Devoxx London 2017 - Rethinking Services With Stateful StreamsBen Stopford
 
Event Driven Services Part 2: Building Event-Driven Services with Apache Kafka
Event Driven Services Part 2:  Building Event-Driven Services with Apache KafkaEvent Driven Services Part 2:  Building Event-Driven Services with Apache Kafka
Event Driven Services Part 2: Building Event-Driven Services with Apache KafkaBen Stopford
 
Event Driven Services Part 1: The Data Dichotomy
Event Driven Services Part 1: The Data Dichotomy Event Driven Services Part 1: The Data Dichotomy
Event Driven Services Part 1: The Data Dichotomy Ben Stopford
 
Event Driven Services Part 3: Putting the Micro into Microservices with State...
Event Driven Services Part 3: Putting the Micro into Microservices with State...Event Driven Services Part 3: Putting the Micro into Microservices with State...
Event Driven Services Part 3: Putting the Micro into Microservices with State...Ben Stopford
 
Strata Software Architecture NY: The Data Dichotomy
Strata Software Architecture NY: The Data DichotomyStrata Software Architecture NY: The Data Dichotomy
Strata Software Architecture NY: The Data DichotomyBen Stopford
 
The Power of the Log
The Power of the LogThe Power of the Log
The Power of the LogBen Stopford
 
Streaming, Database & Distributed Systems Bridging the Divide
Streaming, Database & Distributed Systems Bridging the DivideStreaming, Database & Distributed Systems Bridging the Divide
Streaming, Database & Distributed Systems Bridging the DivideBen Stopford
 
Microservices for a Streaming World
Microservices for a Streaming WorldMicroservices for a Streaming World
Microservices for a Streaming WorldBen Stopford
 
A little bit of clojure
A little bit of clojureA little bit of clojure
A little bit of clojureBen Stopford
 
Big iron 2 (published)
Big iron 2 (published)Big iron 2 (published)
Big iron 2 (published)Ben Stopford
 
The return of big iron?
The return of big iron?The return of big iron?
The return of big iron?Ben Stopford
 
Big Data & the Enterprise
Big Data & the EnterpriseBig Data & the Enterprise
Big Data & the EnterpriseBen Stopford
 
Where Does Big Data Meet Big Database - QCon 2012
Where Does Big Data Meet Big Database - QCon 2012Where Does Big Data Meet Big Database - QCon 2012
Where Does Big Data Meet Big Database - QCon 2012Ben Stopford
 
Advanced databases ben stopford
Advanced databases   ben stopfordAdvanced databases   ben stopford
Advanced databases ben stopfordBen Stopford
 

More from Ben Stopford (20)

10 Principals for Effective Event-Driven Microservices with Apache Kafka
10 Principals for Effective Event-Driven Microservices with Apache Kafka10 Principals for Effective Event-Driven Microservices with Apache Kafka
10 Principals for Effective Event-Driven Microservices with Apache Kafka
 
10 Principals for Effective Event Driven Microservices
10 Principals for Effective Event Driven Microservices10 Principals for Effective Event Driven Microservices
10 Principals for Effective Event Driven Microservices
 
The Future of Streaming: Global Apps, Event Stores and Serverless
The Future of Streaming: Global Apps, Event Stores and ServerlessThe Future of Streaming: Global Apps, Event Stores and Serverless
The Future of Streaming: Global Apps, Event Stores and Serverless
 
A Global Source of Truth for the Microservices Generation
A Global Source of Truth for the Microservices GenerationA Global Source of Truth for the Microservices Generation
A Global Source of Truth for the Microservices Generation
 
Building Event Driven Services with Stateful Streams
Building Event Driven Services with Stateful StreamsBuilding Event Driven Services with Stateful Streams
Building Event Driven Services with Stateful Streams
 
Devoxx London 2017 - Rethinking Services With Stateful Streams
Devoxx London 2017 - Rethinking Services With Stateful StreamsDevoxx London 2017 - Rethinking Services With Stateful Streams
Devoxx London 2017 - Rethinking Services With Stateful Streams
 
Event Driven Services Part 2: Building Event-Driven Services with Apache Kafka
Event Driven Services Part 2:  Building Event-Driven Services with Apache KafkaEvent Driven Services Part 2:  Building Event-Driven Services with Apache Kafka
Event Driven Services Part 2: Building Event-Driven Services with Apache Kafka
 
Event Driven Services Part 1: The Data Dichotomy
Event Driven Services Part 1: The Data Dichotomy Event Driven Services Part 1: The Data Dichotomy
Event Driven Services Part 1: The Data Dichotomy
 
Event Driven Services Part 3: Putting the Micro into Microservices with State...
Event Driven Services Part 3: Putting the Micro into Microservices with State...Event Driven Services Part 3: Putting the Micro into Microservices with State...
Event Driven Services Part 3: Putting the Micro into Microservices with State...
 
Strata Software Architecture NY: The Data Dichotomy
Strata Software Architecture NY: The Data DichotomyStrata Software Architecture NY: The Data Dichotomy
Strata Software Architecture NY: The Data Dichotomy
 
The Power of the Log
The Power of the LogThe Power of the Log
The Power of the Log
 
Streaming, Database & Distributed Systems Bridging the Divide
Streaming, Database & Distributed Systems Bridging the DivideStreaming, Database & Distributed Systems Bridging the Divide
Streaming, Database & Distributed Systems Bridging the Divide
 
JAX London Slides
JAX London SlidesJAX London Slides
JAX London Slides
 
Microservices for a Streaming World
Microservices for a Streaming WorldMicroservices for a Streaming World
Microservices for a Streaming World
 
A little bit of clojure
A little bit of clojureA little bit of clojure
A little bit of clojure
 
Big iron 2 (published)
Big iron 2 (published)Big iron 2 (published)
Big iron 2 (published)
 
The return of big iron?
The return of big iron?The return of big iron?
The return of big iron?
 
Big Data & the Enterprise
Big Data & the EnterpriseBig Data & the Enterprise
Big Data & the Enterprise
 
Where Does Big Data Meet Big Database - QCon 2012
Where Does Big Data Meet Big Database - QCon 2012Where Does Big Data Meet Big Database - QCon 2012
Where Does Big Data Meet Big Database - QCon 2012
 
Advanced databases ben stopford
Advanced databases   ben stopfordAdvanced databases   ben stopford
Advanced databases ben stopford
 

Recently uploaded

A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfMarharyta Nedzelska
 
Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Anthony Dahanne
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López
 
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsSensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsChristian Birchler
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceBrainSell Technologies
 
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingOpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingShane Coughlan
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanyChristoph Pohl
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecturerahul_net
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Developmentvyaparkranti
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesKrzysztofKkol1
 
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Cizo Technology Services
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringHironori Washizaki
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLionel Briand
 
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full RecordingOpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full RecordingShane Coughlan
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Matt Ray
 
What is Advanced Excel and what are some best practices for designing and cre...
What is Advanced Excel and what are some best practices for designing and cre...What is Advanced Excel and what are some best practices for designing and cre...
What is Advanced Excel and what are some best practices for designing and cre...Technogeeks
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfDrew Moseley
 
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...Akihiro Suda
 
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Natan Silnitsky
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxAndreas Kunz
 

Recently uploaded (20)

A healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdfA healthy diet for your Java application Devoxx France.pdf
A healthy diet for your Java application Devoxx France.pdf
 
Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
 
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsSensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. Salesforce
 
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full RecordingOpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
OpenChain AI Study Group - Europe and Asia Recap - 2024-04-11 - Full Recording
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecture
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Development
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
 
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their Engineering
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and Repair
 
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full RecordingOpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
OpenChain Education Work Group Monthly Meeting - 2024-04-10 - Full Recording
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
 
What is Advanced Excel and what are some best practices for designing and cre...
What is Advanced Excel and what are some best practices for designing and cre...What is Advanced Excel and what are some best practices for designing and cre...
What is Advanced Excel and what are some best practices for designing and cre...
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdf
 
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
 
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
 

Building Event Driven Services with Kafka Streams