Kafka Connect/Connector is a framework that allows data to be continuously streamed between Apache Kafka and other external systems such as databases. It provides a scalable and reliable way to import and export large amounts of data to and from Kafka. The Kafka Connect/Connector architecture includes a separate cluster of workers that run connector tasks to move data in a distributed and load-balanced way. Common use cases are processing data from old systems into new systems in a continuous streaming manner.

1. Presented By: Shivam Roy &
Harsh Vardhan
Introduction to Kafka Connect
/ Connector
& Connector

2. Lack of etiquette and manners is a huge turn off.
KnolX Etiquettes
Punctuality
Respect Knolx session timings, you
are requested not to join sessions
after a 5 minutes threshold post
the session start time.
Feedback
Make sure to submit a constructive
feedback for all sessions as it is
very helpful for the presenter.
Silent Mode
Keep your mobile devices in silent
mode, feel free to move out of
session in case you need to attend
an urgent call.
Avoid Disturbance
Avoid unwanted chit chat during
the session.

3. 01 Kafka Connect/Connector Introduction
Agenda
02 Why Kafka Connect/Connector?
03 Kafka Connect/Connector Architecture
04 Use cases of Kafka Connect/Connector
05 Kafka Connect/Connector features
06 Installation and setup

4. What is Kafka Connect/Connector?
● Kafka connect/connector is a framework for
connecting kafka with external systems
including databases.
● The Kafka connect/connector is designed to
run in a Kafka Connect cluster to read data
from Kafka topics and write the data into
Snowflake tables
● The Kafka Connect/Connector cluster
supports running and scaling out connectors.

5. Why Kafka Connect/Connector ?
● Kafka connect/connector is a tool that provides
scalably and reliably streaming data between
Apache Kafka® and other data systems.
● With the help of kafka connector we can import
data from external systems into Kafka topics.
● Also, export data from Kafka topics into external
systems.
● There are connectors that help to move huge
data sets into and out of the Kafka system.

6. Kafka Connect/Connector Architecture
● Kafka Connect is a separate Cluster.
● Each worker contains one or many connector
task.
● A cluster can have multiple workers and worker
runs on the cluster only.
● Task are automatically load-balanced if there is
any failure.
● Task in Kafka Connect act as a Producer or
Consumer depending on the type of Connector.
● Kafka Connect cluster has a multiple loaded
connectors.

7. Use Cases of Kafka Connect/Connector?
● kafka connect/connectors is generally use
for synced operation.
● Kafka connect/connectors is designed for
connecting Kafka with other data systems.
● With the help of Kafka connect we can
Process a data from outdated application
to the new systems.

8. Kafka Connect/Connector Features
● A common framework for
connecting external system with
kafka
● Distributed and standalone
modes
● REST interface
● Automatic offset management
● Distributed and scalable by
default

9. Installation and setup

10. ● Linux System
● Java installation
● download confluent bundle
curl -O http://packages.confluent.io/archive/7.1/confluent-7.1.1.zip
● unzip the file
unzip confluent-7.1.1.zip
● export CONFLUENT_HOME=~/confluent-7.1.1
● export PATH=$CONFLUENT_HOME/bin:$PATH
● confluent local services start
Steps for installing and setup