This session is about the overview of kafka connect.

1. Presented By: Shreya Saxena & Chiranjeev Kumar
Overview of kafka Connectors
& Connector

2. Lack of etiquette and manners is a huge turn off.
KnolX Etiquettes
Punctuality
Respect Knolx session timings,
you are requested not to join
sessions after a 5 minutes
threshold post the session start
time.
Feedback
Make sure to submit a
constructive feedback for all
sessions as it is very helpful for
the presenter.
Silent Mode
Keep your mobile devices in
silent mode, feel free to move
out of session in case you need
to attend an urgent call.
Avoid Disturbance
Avoid unwanted chit chat during
the session.

3. Agenda
02 How Kafka Connect/Connector works?
Kafka Connect/Connector Introduction
05 Use cases of Kafka Connect/Connector
06 Kafka Connect/Connector features
Kafka Connect/Connector Architecture
07 DEMO
002
2
02
Why Kafka Connect/Connector?
03
04
01

4. What is Kafka Connect/Connector?
● Kafka connect/connector is a framework for
connecting kafka with external systems
including databases.
● The Kafka connect/connector is designed to
run in a Kafka Connect cluster to read data
from Kafka topics and write the data into
Snowflake tables
● The Kafka Connect/Connector cluster
supports running and scaling out connectors.

5. How is Kafka Connect/Connector works?
● Kafka Connect act as a standalone process
that runs jobs on a single machine (for
example, log collection), or as a distributed,
scalable, fault-tolerant service supporting an
entire organization
● Two types
1 Source connector
2 Sink connector

6. Why Kafka Connect/Connector ?
● Kafka connect/connector is a tool that provides
scalably and reliably streaming data between
Apache Kafka® and other data systems.
● With the help of kafka connector we can import
data from external systems into Kafka topics.
● Also, export data from Kafka topics into external
systems.
● There are connectors that help to move huge
data sets into and out of the Kafka system.

7. Kafka Connect/Connector Architecture
● Kafka Connect is a separate Cluster.
● Each worker contains one or many connector
task.
● A cluster can have multiple workers and worker
runs on the cluster only.
● Task are automatically load-balanced if there is
any failure.
● Task in Kafka Connect act as a Producer or
Consumer depending on the type of Connector.
● Kafka Connect cluster has a multiple loaded
connectors.

8. Use Cases of Kafka Connect/Connector?
● kafka connect/connectors is generally use
for synced operation.
● Kafka connect/connectors is designed for
connecting Kafka with other data systems.
● With the help of Kafka connect we can
Process a data from outdated application
to the new systems.

9. Kafka Connect/Connector Features
● A common framework for
connecting external system with
kafka
● Distributed and standalone
modes
● REST interface
● Automatic offset management
● Distributed and scalable by
default

10. DEMO
(Installation of Confluent
Control center)

11. Steps for installing and setup
● Linux System
● Java installation
● download confluent bundle
curl -O http://packages.confluent.io/archive/7.1/confluent-7.1.1.zip
● unzip the file
unzip confluent-7.1.1.zip
● export CONFLUENT_HOME=~/confluent-7.1.1
● export PATH=$CONFLUENT_HOME/bin:$PATH
● confluent local services start