The saying goes that there are only two hard things in Computer Science: cache invalidation, and naming things. Well, turns out the first one is solved actually ;)
Join us for this session to learn how to keep read views of your data in distributed caches close to your users, always kept in sync with your primary data stores change data capture. You will learn how to
- Implement a low-latency data pipeline for cache updates based on Debezium, Apache Kafka, and Infinispan
- Create denormalized views of your data using Kafka Streams and make them accessible via plain key look-ups from a cache cluster close by
- Propagate updates between cache clusters using cross-site replication
We'll also touch on some advanced concepts, such as detecting and rejecting writes to the system of record which are derived from outdated cached state, and show in a demo how all the pieces come together, of course connected via Apache Kafka.
5. #Debezium @gunnarmorling
The Challenge
● Deployments in different geographies, single DB
● 90% read requests
● Complex queries
Multi-site Application With Shared Database
6. #Debezium @gunnarmorling
… Multi-site application with shared system-of-record database
… With local, denormalized read views (CQRS)
... Automatically kept in sync after writes
Today’s Mission
🤔
Explore How to Build a…
7. #Debezium @gunnarmorling
● Software engineer at Red Hat
○ Debezium
○ Quarkus
● kcctl 🧸, JfrUnit, ModiTect, MapStruct
● Spec Lead for Bean Validation 2.0
● Java Champion
Gunnar Morling
13. #Debezium @gunnarmorling
App 1
Data
App 2
Data
1, Maria
2, Jenny
2, Jenny
3, Juan
App 3
Data
1, Maria
3, Juan
Put 4 Will Put 4 Will
Infinispan Deployment
Distributed Cache
14. #Debezium @gunnarmorling
App 1
Data
App 2
Data
1, Maria
2, Jenny
4, Will
2, Jenny
3, Juan
4, Will
App 3
Data
1, Maria
3, Juan
Infinispan Deployment
Distributed Cache
17. #Debezium @gunnarmorling
Infinispan Cross-Site Replication
AWS (LON)
GCP (NYC)
Load Balancer
APP
APP
Service
APP
APP
Service
Shared
State
Shared
State
Shared
State
Shared
State
Data
Data
Data NYC
Data LON
RELAY2
21. #Debezium @gunnarmorling
Debezium in a Nutshell
Open-Source Change Data Capture
● A CDC Platform
○ Based on transaction logs
○ Snapshotting, filtering, etc.
○ Outbox support
○ Web-based UI
● Fully open-source, very active
community
● Large production deployments
27. #Debezium @gunnarmorling
● Fast start-up, low memory consumption
● Developer joy
● Imperative and Reactive
● Best-of-breed libraries
● Run via HotSpot and GraalVM native binaries
Quarkus - Supersonic Subatomic Java
A Stack for Building Cloud-native Apps