Watch replay here: https://videos.confluent.io/watch/kZiXwKJfAdJso84PjNW9Uk? Speaker: Brett Randall

4. Let’s Start with WHY?

5. Lost in Translation
(multiple Serialization formats)

12. kafka-topics --create --bootstrap-server localhost:9092 --replication-
factor 1
--partitions 1 --topic test-validation-true
--config confluent.value.schema.validation=true
https://www.confluent.io/blog/data-governance-with-schema-validation/

14. WHEN is this observed?

15. App 1
App 2
App 3
Many sources without a policy
causes mayhem in a centralized
data pipeline
Ensuring downstream systems
can use the data is key to an
operational stream pipeline
Even within a single application,
different formats can be
presented
Incompatibly formatted
message
The Challenge of Data Compatibility at Scale

16. so HOW do we solve this?

17. {"namespace": "example.avro",
"type": "record",
"name": "user",
"fields": [
{"name": "name", "type": "string"},
{"name": "favorite_number", "type": "int"}
]
}

18. and WHERE does it fit?

19. ●
○
○
●
○
●
○
●
●
●
●

20. ●
●
●
●
Serializer
App 1
Serializer
App 2
!
! Schema Registry
Kafka
topic

21. and exactly WHAT does it do?

24. WHO should be using this?

27. Let’s dive deeper!

28. What is a topic versus a schema versus a subject?

29. <Insert whiteboard drawing>

30. BACKWARD
●
●
BACKWARD_TRANSITIVE
●
●
FORWARD
●
●
FORWARD_TRANSITIVE
●
●
FULL
●
●
FULL_TRANSITIVE
●
●
NONE ●

33. We have a 100 developers and many topics and
schemas, do we need to set this up multiple times?

36. and what if I want to move to the cloud?

42. 42