Demonstrates :- Setting up confluent kafka on Docker [docker setup is not part of this article] Starting up ZooKeeper Creating topics and consumer Checking zookeeper, consumer, topic and offsets (Reference taken from confluent official page)

1. Confluent Kafka on Docker
Get confluent kafka using docker pull command:
$docker pull confluentinc/cp-kafka
Zookeeper configuration:
$docker run -d --net=host --name=zookeeper -e ZOOKEEPER_CLIENT_PORT=32181 -e
ZOOKEEPER_TICK_TIME=2000 -e ZOOKEEPER_SYNC_LIMIT=2 confluentinc/cp-zookeeper:5.2.2
Kafka configuration:
$docker run -d --net=host --name=kafka -e KAFKA_ZOOKEEPER_CONNECT=localhost:32181 -e
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:29092 -e KAFKA_BROKER_ID=2 -e
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 confluentinc/cp-kafka:5.2.2
Create a docker network issuing the below command :
$docker network create confluent
Note :- building a single node docker environment for this demonstration.
Start Zookeeper:
$docker run -d --net=confluent --name=zookeeper -e ZOOKEEPER_CLIENT_PORT=2181
confluentinc/cp-zookeeper:5.0.0
Check zookeeper Logs :
$docker logs zookeeper
Run Kafka:
$ docker run -d --net=confluent --name=kafka -e
KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 -e
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 -e
KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092 -e
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 confluentinc/cp-kafka:5.0.0

2. If want to access kafka outside of docker, then do the port mapping-
$docker run --net=confluent -p 9092:9092 --expose=9093 --name=kafka -e
KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 -e
KAFKA_ADVERTISED_LISTENERS=INSIDE://kafka:9093,OUTSIDE://localhost:9092 -e
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT -e
KAFKA_LISTENERS=INSIDE://0.0.0.0:9093,OUTSIDE://0.0.0.0:9092 -e
KAFKA_INTER_BROKER_LISTENER_NAME=INSIDE -e
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
confluentinc/cp-kafka:5.0.0
Check kafka logs -
$ docker logs kafka
Create Topic :
$docker run --net=confluent --rm confluentinc/cp-kafka:5.0.0 kafka-topics --create --topic foo
--partitions 1 --replication-factor 1 --if-not-exists --zookeeper zookeeper:2181
Create a consumer and read messages from the topic :
$docker run --net=confluent --rm confluentinc/cp-kafka:5.0.0 kafka-console-consumer
--bootstrap-server kafka:9093 --topic foo --from-beginning --max-messages 42
Describe Zookeeper :
$docker run --net=confluent --rm confluentinc/cp-kafka:5.0.0 kafka-topics --zookeeper zookeeper:2181
--list
Describe the topic we have created in the above step :
$ docker run --net=confluent --rm confluentinc/cp-kafka:5.0.0 kafka-topics --describe --topic foo
--zookeeper zookeeper:2181
Describe the Consumer :
$ docker run --net=confluent --rm confluentinc/cp-kafka:5.0.0 kafka-consumer-groups
--bootstrap-server kafka:9093 -list
Response will be something like below-
Console-consumer-20383
Console-consumer-49069
Describe the particular consumer group :
$docker run --net=confluent confluentinc/cp-kafka:5.0.0 kafka-consumer-groups --bootstrap-server
kafka:9093 --group test-consumer-group --describe

3. Check the earliest offset of the topic :
$ docker run --net=confluent --rm confluentinc/cp-kafka:5.0.0 kafka-run-class
kafka.tools.GetOffsetShell --broker-list kafka:9093 --topic foo --time -2
Check the latest offset in the topic: [Do it by describing the particular topic]
$ docker run --net=confluent confluentinc/cp-kafka:5.0.0 kafka-consumer-groups --bootstrap-server
kafka:9093 --group test-consumer-group --describe