Pere Urbon-Bayes, Solutions Architect, Confluent
Wax on Wax off: The Learnings of Karate Kid applied to Apache Kafka® https://www.meetup.com/Berlin-Apache-Kafka-Meetup-by-Confluent/events/266567425/
1
Wax on Wax off:
The Learnings of Karate kid
applied to Apache Kafka
Pere Urbon-Bayes (@purbon)
Solutions Architect
2
@purbon
Topics for today
● Common questions when building an Streaming Platform with Apache Kafka?
○ Topic Management
○ Schemas Management
● Building self sufficient teams with the Kafka Topology Builder
● Database migrations for Apache Kafka, the Kafka Migration tool
17
@purbon
But how do we organise this
How do you replicate state between environments:
● Upgrade changes across environments.
○ Merge changes across descriptors.
○ Apply migrations across environments.
● Graduate state from lower environments.
○ Useful for schema registry using the version/latest Resource
○ Very hard for Apache Kafka (pulling latest state is very hard)
● State replication needs CI/CD and automation.
18
@purbon
As teams using Apache Kafka
As a user of an Apache Kafka based system, I want to:
● Be self sufficient
● Care about my area of work / project
● Get test and validation for all the changes requested
19
@purbon
Approaches for making self sufficient teams
There are different approaches to make self sufficient teams:
● Via CI/CD and github.
● Using a web interface (ala Control Center, Kafkawize or KafkaTool)
● Using service brokers / operators (ala Containers)