2. Using Kafka as Your Primary Data Store
• Compacted Topics Guarantees
• Compacted Topics and Log Cleaner
• Use Cases
• MongoDB?
• KafkaSubscriptionBackingStore
• Multiple Node Support
3. Compacted Topics Guarantees
1. Stay up with head, see every message.
2. Message Order Always Maintained.
3. Offset never changes.
4. Consumer will see at least the final state.
5. Use Cases
1. Database change subscription
2. Event sourcing
3. Journaling for high-availability
6. Why not just use MongoDB?
• MongoDB doesn’t have triggers, notifications or listeners.
• Would have to develop a custom solution in order to see changes.
• One less point of failure
1. Messages will have sequential offsets
2. Compaction doesn't reorder, just removes duplicates
3. Offsets are permanent identifiers.
4. ...in the order they were written.
1. In oracle, this is known as Database Change Notification. Use this if you have a cache and want to update that cache when the database changes.
2. Design that co-locates query processing with application design and uses a log of changes as the primary store for the application.
3. Save local state so another process can reload changes and carry on when the original process fails.
