Apache Kafka Meetup
From Postgres to Event-Driven: using
docker-compose to build CDC pipelines
into Kafka
Mark Teehan
Singapore
16-Mar-2021
developer.confluent.io
Learn Kafka.
Start building with Apache
Kafka at Confluent Developer.
Discover The World Of Event Streaming
Kafka Summit is the premier event for developers, architects, data engineers, devops professionals, and
anyone else who wants to learn about streaming data. It brings the Apache Kafka community together to
share best practices, learn how to build next-generation systems, and discuss the future of streaming
technologies.
Kafka Summit APAC 2021 will be hosted virtually, and is free to charge to attend.
● Register now free of charge: https://www.kafka-summit.org/
● Submit an abstract via the Call for Papers (closes April 5th):
Watch short video from Tim Berglund on how to write a great CFP
https://www.youtube.com/watch?v=N0g3QoCuqH4
cnfl.io/ask-the-community
Ask the
Community!
Ask questions, share
knowledge and chat
with your fellow
community members!
Free eBooks Designing Event-Driven Systems
Ben Stopford
Kafka: The Definitive Guide
Neha Narkhede, Gwen Shapira, Todd
Palino
Making Sense of Stream Processing
Martin Kleppmann
I ❤ Logs
Jay Kreps
http://cnfl.io/book-bundle
confluent.io/blog
confluent.io/blog
Apache Kafka Meetup
From Postgres to Event-Driven: using
docker-compose to build CDC pipelines
into Kafka
Mark Teehan
Singapore
16-Mar-2021
Mark Teehan
Principal
Solution
Engineer
Confluent
(Singapore)
https://github.com/markteehan/
CDC_Postgres_Kafka
From Postgres
to Event-Driven:
using
docker-compose
to build CDC
pipelines into
Kafka
Use the Debezium CDC connector to
capture database changes from a
Postgres database (or MySQL or
Oracle); streaming into Kafka topics
and onwards to an external data store.
.
I’m Live
But I’ve
recorded some
videos
(They are short)
Ask me questions in the chat at
any time
D1 - lets get started
D2
D3
D4
……
Discover The World Of Event Streaming
Kafka Summit is the premier event for developers, architects, data engineers, devops professionals, and
anyone else who wants to learn about streaming data. It brings the Apache Kafka community together to
share best practices, learn how to build next-generation systems, and discuss the future of streaming
technologies.
Kafka Summit APAC 2021 will be hosted virtually, and is free to charge to attend.
● Register now free of charge: https://www.kafka-summit.org/
● Submit an abstract via the Call for Papers (closes April 5th):
https://sessionize.com/kafka-summit-apac-2021/
● Watch short video from Tim Berglund on how to write a great CFP
https://www.youtube.com/watch?v=N0g3QoCuqH4
developer.confluent.io
Learn Kafka.
Start building with Apache
Kafka at Confluent Developer.
developer.confluent.io
Apache Kafka Meetup
Thank You!
https://github.com/markteehan/CDC_Postgres_Kafka
Mark Teehan
Singapore
16-Mar-2021
……

From Postgres to Event-Driven: using docker-compose to build CDC pipelines into Apache Kafka®

  • 1.
    Apache Kafka Meetup FromPostgres to Event-Driven: using docker-compose to build CDC pipelines into Kafka Mark Teehan Singapore 16-Mar-2021
  • 2.
    developer.confluent.io Learn Kafka. Start buildingwith Apache Kafka at Confluent Developer.
  • 3.
    Discover The WorldOf Event Streaming Kafka Summit is the premier event for developers, architects, data engineers, devops professionals, and anyone else who wants to learn about streaming data. It brings the Apache Kafka community together to share best practices, learn how to build next-generation systems, and discuss the future of streaming technologies. Kafka Summit APAC 2021 will be hosted virtually, and is free to charge to attend. ● Register now free of charge: https://www.kafka-summit.org/ ● Submit an abstract via the Call for Papers (closes April 5th): Watch short video from Tim Berglund on how to write a great CFP https://www.youtube.com/watch?v=N0g3QoCuqH4
  • 4.
    cnfl.io/ask-the-community Ask the Community! Ask questions,share knowledge and chat with your fellow community members!
  • 5.
    Free eBooks DesigningEvent-Driven Systems Ben Stopford Kafka: The Definitive Guide Neha Narkhede, Gwen Shapira, Todd Palino Making Sense of Stream Processing Martin Kleppmann I ❤ Logs Jay Kreps http://cnfl.io/book-bundle
  • 6.
  • 7.
  • 8.
    Apache Kafka Meetup FromPostgres to Event-Driven: using docker-compose to build CDC pipelines into Kafka Mark Teehan Singapore 16-Mar-2021
  • 9.
  • 10.
    From Postgres to Event-Driven: using docker-compose tobuild CDC pipelines into Kafka Use the Debezium CDC connector to capture database changes from a Postgres database (or MySQL or Oracle); streaming into Kafka topics and onwards to an external data store. .
  • 11.
    I’m Live But I’ve recordedsome videos (They are short) Ask me questions in the chat at any time
  • 12.
    D1 - letsget started D2 D3 D4
  • 13.
  • 14.
    Discover The WorldOf Event Streaming Kafka Summit is the premier event for developers, architects, data engineers, devops professionals, and anyone else who wants to learn about streaming data. It brings the Apache Kafka community together to share best practices, learn how to build next-generation systems, and discuss the future of streaming technologies. Kafka Summit APAC 2021 will be hosted virtually, and is free to charge to attend. ● Register now free of charge: https://www.kafka-summit.org/ ● Submit an abstract via the Call for Papers (closes April 5th): https://sessionize.com/kafka-summit-apac-2021/ ● Watch short video from Tim Berglund on how to write a great CFP https://www.youtube.com/watch?v=N0g3QoCuqH4
  • 15.
    developer.confluent.io Learn Kafka. Start buildingwith Apache Kafka at Confluent Developer.
  • 16.
  • 17.
    Apache Kafka Meetup ThankYou! https://github.com/markteehan/CDC_Postgres_Kafka Mark Teehan Singapore 16-Mar-2021
  • 18.