This document summarizes a presentation about Apache Kafka. It introduces Apache Kafka as a modern, distributed platform for data streams made up of distributed, immutable, append-only commit logs. It describes Kafka's scalability similar to a filesystem and guarantees similar to a database, with the ability to rewind and replay data. The document discusses Kafka topics and partitions, partition leadership and replication, and provides resources for further information.

1. 1
Mid-Ohio Data Users Group
Meetup
Organizer: Derek Kane
Host: Navigator Management Partners

2. 2
Apache Kafka®
Fundamentals for
Architects, Admins and
Developers

3. 3
About Me
● Patrick Druley
● Senior Systems Engineer @ Confluent (1yr)
● Formerly at Oracle and Teradata
● Ohioan, currently in Medina Ohio
● Github: https://github.com/pdruley
● Twitter: @PatrickLovesAK

4. 4

5. 5
A Modern, Distributed Platform for
Data Streams

6. 6
Apache Kafka® is made up of
distributed, immutable, append-only
commit logs

7. 7
Intelligent Clients and Naive Pipes
Writers
Kafka
cluster
Readers

8. 88
Scalability of a filesystem
• hundreds of MB/s throughput
• many TB per server
• commodity hardware

9. 99
Guarantees of a Database
• Strict ordering
• Persistence

10. 1010
Rewind & Replay
Reset to any point in the shared narrative

11. 1111
Distributed by design
• Replication
• Fault Tolerance
• Partitioning
• Elastic Scaling

12. 1212
Kafka Topics
my-topic
my-topic-partition-0
my-topic-partition-1
my-topic-partition-2
broker-1
broker-2
broker-3

13. 1313
Partition Leadership and Replication
Broker 1
Topic1
partition1
Broker 2 Broker 3 Broker 4
Topic1
partition1
Topic1
partition1
Leader Follower
Topic1
partition2
Topic1
partition2
Topic1
partition2
Topic1
partition3
Topic1
partition4
Topic1
partition3
Topic1
partition3
Topic1
partition4
Topic1
partition4

14. 1414
Partition Leadership and Replication - node failure
Broker 1
Topic1
partition1
Broker 2 Broker 3 Broker 4
Topic1
partition1
Topic1
partition1
Leader Follower
Topic1
partition2
Topic1
partition2
Topic1
partition2
Topic1
partition3
Topic1
partition4
Topic1
partition3
Topic1
partition3
Topic1
partition4
Topic1
partition4

15. 1515
Demo Time!
https://github.com/pdruley/public_notebooks/blob/master/Glorious_Kafka_Command_Line_Utilites.ipynb

16. 1616
Resources
Free E-Books from Confluent!
I Heart Logs:
https://www.confluent.io/ebook/i-heart-logs-event-data-stream-processing-and-data-integration/
Kafka: The Definitive Guide: https://www.confluent.io/resources/kafka-the-definitive-guide/
Confluent Blog: https://www.confluent.io/blog
Confluent Podcast:
https://podcasts.apple.com/us/podcast/streaming-audio-a-confluent-podcast-about-apache-kafka/id1
401509765?mt=2
Thank You!
patrick@confluent.io
@PatrickLovesAK

17. 1717
Thank You!