Apache Kafka sits at the center of a technology ecosystem that can be a bit overwhelming to someone just getting started. Fortunately, Apache Kafka is also at the heart of an amazing community that is able and eager to help! So, if you are new, or relatively new, to Apache Kafka, welcome! I’d like to introduce you to the Kafka ecosystem, and present you with a plan for how to learn and be productive with it. I’d also like to introduce you to one of the most helpful and welcoming software communities I’ve ever encountered. I’ll take you through the basics of Kafka—the brokers, the partitions, the topics—and then on and up into the different APIs and tools that are available to work with it. Consider it a Kafka 101, if you will. We’ll stay at a high level, but we’ll cover a lot of ground, with an emphasis on where and how you can dig in deeper. I am still learning myself, so I will share with you what and who have helped me in my journey, and then I’ll invite you to continue that journey with me. It’s going to be a great adventure!

1. Dave Klein
@daveklein
Welcome to Kafka
We’re Glad You’re Here

2. The Plan
•Overview of the Apache Kafka Ecosystem
•Strategies to go about learning it
•Resources for every learning style
•Friends to help you on your journey

6. 1 2 3 4 5 6 7 8 9 10 11

7. Key
Value

8. "key": 20200824,
"value": {
"eventType": “add-to-cart”,
"title": “Kafka Streams in Action",
"author": “Bill Bejeck",
"price": 44.99
}

9. 1 2 3 4 5 6 7 8 9 10 11
Producer

10. 1 2 3 4 5 6 7 8 9 10 11
Producer

11. 1 2 3 4 5 6 7 8 9 10 11
Producer
Consumer

12. 1 2 3 4 5 6 7 8 9 10 11
Producer
Consumer

13. 1 2 3 4 5 6 7 8 9 10 11
Producer
Consumer

14. 1 2 3 4 5 6 7 8 9 10 11
Producer
Consumer
Committed Offset: 3

15. Consumer
1 2 3 4 5 6 7 8 9 10 11
Producer
Committed Offset: 3

16. 1 2 3 4 5 6 7 8 9 10 11
Producer
Consumer
Committed Offset: 3

17. Schema Registry
Producer
Consumer
1 2 3 4 5 6 7 8 9 10

18. Kafka Connect
Producer
JDBC

19. Kafka Connect
Consumer
JDBC

20. Producer
Consumer
stream(input topic)
.filter()
.map()
.flatMap()
.branch().
.groupByKey()
.peak()
.to(output topic)
Kafka Streams

21. ksqlDB
ksql> CREATE STREAM books (
title VARCHAR, author VARCHAR, year_published
INT
) WITH (
KAFKA_TOPIC=‘booklist’, VALUE_FORMAT=‘AVRO’
);
ksql> SELECT * FROM books WHERE title LIKE ‘%kafka%’ EMIT CHANGES;
+--------------------------------+-----------------------------+------------+
|TITLE |AUTHOR |YEAR |
+--------------------------------+-----------------------------+------------+
|Kafka: The Definitive Guide |Shapira, Narkhede, Palino |2017 |
|Kafka Streams in Action |Bill Bejeck |2018 |
|Kafka in Action |Dylan |2020 |
|Apache Kafka Cookbook |Raul Estrada |2017 |

22. curl -X POST -H "Content-Type: application/vnd.kafka.json.v2+json"
-H "Accept: application/vnd.kafka.v2+json"
--data '{"records":[{"value":{"foo":"bar"}}]}'
"http://localhost:8082/topics/jsontest"

24. Command Line Tools
Kafka Shell Scripts
- Start and stop brokers
- Create / manage topics
- Produce / consume events
Confluent CLI
- Start / stop components of Confluent Platform
- Manage access control and secrets
Kafkacat
- Produce / consume events in style
Confluent Cloud CLI
- Create / manage topics on Confluent Cloud
- Produce / consume events on Confluent Cloud

25. -Fully managed, just create your topics
-Includes Confluent Schema Registry
-Managed ksqlDB
-Kafka Connect (several connectors, more coming)
-Pay for what you use, scale up and down easily
-Available on Big 3 Cloud Providers’ Marketplaces

27. Install / Configure
Build Applications
Create Topics
Produce / Consume
> _
> _ {REST}
Monitor

33. 18 The Future of Serverless and Streaming - Neil Avery
24 It’s Time for Streaming to Have a Maturity Model - Nick Dearden
27 Ben Stopford on Microservices and Event Streaming
37 Deploying Confluent Platform, from Zero to Hero - Mitch Henderson
53 Jay Kreps on the Last 10 Years of Apache Kafka and Event Streaming
59 Data Modeling for Apache Kafka – Streams, Topics & More - Dani Traphagen
65 The Pro’s Guide to Fully Managed Apache Kafka Services - Ricardo Ferreira
64 Kafka Screams: The Scariest JIRAs and How To Survive Them - Anna McDonald
70 Real-Time Payments with Clojure and Apache Kafka - Bobby Calderwood
83 Paving a Data Highway with Kafka Connect - Liz Bennett
86 Distributed Systems Engineering with Apache Kafka - Gwen Shapira
90 Explore, Expand, and Extract with 3X Thinking - Kent Beck
98 Apache Kafka Fundamentals: The Concept of Streams and Tables - Michael Noll
105 From Monolith to Microservices - Sam Newman
108 Benchmarking Apache Kafka Latency at the 99th Percentile - Anna Povzner

40. Formatted Messages Are Easier To Read
For a single line of code, select and click
or wrap it in a pair of single back-ticks: `foo`
or wrap it all in triple back-ticks
```for (n : lotsOn) {
System.out.println(“foo”);
}```
For code blocks (or logs) select all and click
https://slack.com/help/articles/202288908-Format-your-messages

41. events.confluent.io/meetups

42. Anna McDonald
@jbfletch_
William Bejeck
@bbejeck
Boyang Chen Christoph Schubert
@ChristophSchub5
Gnanaguru Sattanathan
@avoguru
Daniel Hinojosa
@dhinojosa
David Simons Dominik Fries
@sarvo001
Erik Tank
@erik_tank
Gwen Shapira
@gwenshap
Hans-Peter Grahsl
@hpgrahsl
Jason Gustafson
Jay Smith
@thejaysmith
Jennifer Snipes
@snipermomma
John Roesler
@VVCepheiB
Kat Grigg
@katgrigg
Mario Molina
@mmolimar_
Mitch Henderson
@Mr_mitchellh
Naveen Nandan Neil Buesing
@nbuesing
Pere Urbón-Bayes
@purbon
Perry Krol
@perkrol
Philipp Schirmer
@philipp94831
Qais Babie Ricardo Ferreira
@riferrei
Robert Fu Robin Moffatt
@rmoff
Sam Harley Simon Aubury
@simonaubury
Tim Berglund
@tlberglund
Viktor Gamov
@gAmUssA
Amazing
Meetup
Presenters

43. Confluent Community Catalysts 2019/2020

46. Confluent Community and Dev Rel Teams
Ale Murray Joseph Shore Shay Meyer
Tim Berglund Robin Moffatt Ricardo Ferreira Viktor Gamov

47. Thank you!
Links:
- Confluent Developer Portal: https://developer.confluent.io
- Meetup Hub: https://events.confluent.io/meetups
- KIPS: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
- Confluent Blog: https://www.confluent.io/blog
- Apache Kafka Blog: https://blogs.apache.org/kafka
- Confluent O’Reilly Book Bundle: https://www.confluent.io/apache-kafka-stream-processing-book-bundle
- Confluent Community Slack: https://launchpass.com/confluentcommunity
- Dave’s email: daveklein@usa.net
- Dave’s Twitter: https://twitter.com/daveklein
- Dave’s Blog: http://dave-klein.blogspot.com