Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1Confidential
Building Event-Driven Microservices
with Stateful Streams
Kai Waehner
Technology Evangelist
kontakt@kai-waeh...
Microservices
Orders
Service
Basket
Service
Payment
Service
Fulfillment
Service
Stock
Service
Autonomy
Orders
Service
Independently Deployable
Orders
Service
Basket
Service
Payment
Service
Fulfillment
Service
Stock
Service
Allows us to scale
Scale in people terms
Scale in infrastructure terms
Service
Service
Service
Service A instance 1
Service A instance 2
Service B instance 1
Servi...
Scalable Streaming Microservices
How do we get there?
• Loose Coupling
• Data Enabled
• Event Driven
• Streaming Platform
à It is NOT a Queue! J
• All messaging systems are storage systems
• Slow consumers
• Reprocessing
KSQL – A Streaming SQL Engine for Apache Kafka
Apache Kafka + Kafka Connect + Kafka Streams
Apache Kafka: The Rise of a Streaming Platform
The Log ConnectorsConnectors
Producer Consumer
Streaming Engine
54
Highly Scalable Microservices with Apache Kafka +
Mesos
Kai Waehner
Technology Evangelist
kontakt@kai-waehner.de
@KaiWa...
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL
Upcoming SlideShare
Loading in …5
×

Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL

1,213 views

Published on

Building Event-Driven Microservices with Stateful Streams - Apache Kafka, Kafka Streams, KSQL, and more…

Event Driven Services come in many shapes and sizes from tiny event-driven functions that dip into an event stream, right through to heavy, stateful services which can facilitate request response. This talk makes the case for building this style of system using Stream Processing tools, defining a microservices architecture and leveraging Apache Kafka ecosystem including Kafka Streams and KSQL. We also walk through a number of patterns for how we actually put these things together to enable independent teams and autonomous development of microservices.

Kudos to my colleagues Ben and Jay who created many of the slides.

Published in: Technology

Event-Driven Microservices with Apache Kafka, Kafka Streams and KSQL

  1. 1. 1Confidential Building Event-Driven Microservices with Stateful Streams Kai Waehner Technology Evangelist kontakt@kai-waehner.de LinkedIn @KaiWaehner www.confluent.io www.kai-waehner.de Apache Kafka, Kafka Streams, KSQL, and more… (Kudos to my colleagues Ben and Jay)
  2. 2. Microservices Orders Service Basket Service Payment Service Fulfillment Service Stock Service
  3. 3. Autonomy Orders Service
  4. 4. Independently Deployable Orders Service Basket Service Payment Service Fulfillment Service Stock Service
  5. 5. Allows us to scale
  6. 6. Scale in people terms
  7. 7. Scale in infrastructure terms Service Service Service Service A instance 1 Service A instance 2 Service B instance 1 Service B instance 2 Cluster (many machines)
  8. 8. Scalable Streaming Microservices How do we get there? • Loose Coupling • Data Enabled • Event Driven • Streaming Platform
  9. 9. à It is NOT a Queue! J
  10. 10. • All messaging systems are storage systems • Slow consumers • Reprocessing
  11. 11. KSQL – A Streaming SQL Engine for Apache Kafka
  12. 12. Apache Kafka + Kafka Connect + Kafka Streams
  13. 13. Apache Kafka: The Rise of a Streaming Platform The Log ConnectorsConnectors Producer Consumer Streaming Engine
  14. 14. 54 Highly Scalable Microservices with Apache Kafka + Mesos Kai Waehner Technology Evangelist kontakt@kai-waehner.de @KaiWaehner www.confluent.io www.kai-waehner.de LinkedIn Questions? Feedback? Please contact me! Download Apache Kafka & Confluent Platform confluent.io/download

×