27. Format Structure Adaptability
CSV
Positional,
no type definition
Possible if appending
new columns
XML
flexible,
strong typed
Append, remove only via
version (no standard)
supports defaults
JSON
flexible,
untyped
Append and remove are
handled by parser
no support for defaults
Avro
flexible,
strong typed
Append, Removal
supports defaults
Version is built in
38. Avro Converter
• Scans the classpath for *.avsc
files and register them
• During writes, infer the schema
from payload (SpecificDatum,
GenericDatum, Reflection)
• During reads uses message
headers to discover the schema
being used
42. Avro Converter
• Each component still needs the
avsc file
• Avro versioning only works if
both writer and reader schemas
are available
• Transmitting the schema with the
message is an overhead
43. Schema registry
• Centralized store for schemas
• Idempotent registration (same
schema payload always return the
same id)
• Compatibility test
• Schema utilization
45. Schema registry
• Allows developers to check if new
schemas can break existing ones in
the registry
• BACKWARD: new schema can read old
versions
• FORWARD: Old schema can read new
version
• FULL: BACKWARD + FORWARD