Download as PDF, PPTX
Uploaded byNikolay Stoitsev
Using Apache Kafka from Go
The document discusses using Apache Kafka with Go, detailing message handling, topic partitioning, and consumer groups for efficient data processing. It highlights the differences between at-most-once and at-least-once delivery semantics and provides references to logging frameworks and Kafka libraries in Go. The author, Nikolay Stoitsev, emphasizes performance aspects of message consumption and publishing in a Kafka environment.
More Related Content
Similar to Using Apache Kafka from Go
Using Apache Kafka from Go
- 1. Using Apache Kafka fromGo Nikolay Stoitsev, Software Engineer
- 2. Trillion+ Messages/Day PBs Data Volume Tens ofThousands Topics
- 3. Message Message MessageMessage Message Message Topic
- 4. Partition - ordered,immutable sequence Message Message Message Message Message Message Message Message Message Message Message Message Message Message Message Partition 0 Partition 1 Partition 2
- 5. Offset Message Message MessageMessage MessagePartition 0 Offset: 1
- 6. Offset Message Message MessageMessage MessagePartition 0 Offset: 2
- 7. Offset Message Message MessageMessage MessagePartition 0 Offset: 2 Offset: 3 Consumer: 1Consumer: 2
- 8.
- 9.
- 10.
- 11.
- 12.
- 14.
- 15. At-most-once cluster forlogging
- 16. At-least-one cluster formessage bus
- 17. Publishing to lossykafka for logging
- 18. log.Printf("failed to fetchURL %s", url)
- 19.
- 21.
- 22.
- 23.
- 24.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35.
- 36.
- 39.
- 42.
- 43.
- 44.
- 45.