This document provides an overview of replicating data from MySQL to MongoDB and Kafka using Tungsten Replicator. It reviews the replicator workflow, prerequisites for MongoDB and Kafka targets, and sample configurations. The presenter demonstrates Tungsten Replicator's ability to extract data from MySQL, convert rows to messages with metadata, and apply the messages to MongoDB collections or Kafka topics. Requirements like user accounts, ports, and Zookeeper configuration are discussed.

1. The MySQL Availability Company
Tungsten Replicator Master Class
Advanced: Working with MongoDB and Kafka
Chris Parker, Customer Success Director, EMEA & APAC

2. Topics
In this short course, we will
• Review replicator flow
• Explore MongoDB and Kafka specific pre-requisites
• Review configurations
• Demo

3. Replicator Flow

4. How MongoDB Replication Works
THL THL
Source Replicator:
Extractor
Target Replicator:
Applier
THL = Events + Metadata
DBMS
logs
Download transactions via
network
Mongo
Applier
Lib
Extractor
Options
MySQL Binary Logging
2 1

5. How Kafka Replication Works
THL THL
Source Replicator:
Extractor
Target Replicator:
Applier
THL = Events + Metadata
DBMS
logs
Download transactions
via network
Kafka Applier
(Native)Extractor
Options
MySQL Binary Logging
2 1
Zookeeper

6. What Tungsten Replicator Does to Apply into Kafka
• Takes an incoming row and converts it to a message
• Message consists of metadata:
• Schema name, table name
• Sequence number
• Commit timestamp
• Operation Type
• Embedded Message Content

7. Object Mapping
• Mongo
• MySQL Database à Database
• Table à Collection
• Row à Document
• Kafka
• MySQL Schema & Table à Topic (schemaname_tablename)
• Row à Message

8. Prerequisites
• Review online documentation
• https://docs.continuent.com
• Download the Prerequisite Checklist
• Extractor/Applier Hosts
• OS User
• /etc/hosts
• sudoers
• Ruby
• Java
• Network
• Review Port Requirements
• MySQL
• my.cnf settings
• User accounts
• MongoDB
• User Account
• If using Atlas, export certificates
• Add replicator to Atlas Whitelist
• Kafka
• Configure zookeeper

9. Extractor Config
[defaults]
user=tungsten
install-directory=/opt/continuent
mysql-allow-intensive-checks=true
profile-script=~/.bash_profile
disable-security-controls=true
[alpha]
master=tr-ext-2
members=tr-ext-2
replication-user=tungsten
replication-password=secret
replication-port=3306
enable-heterogeneous-service=true

10. Applier Configs
[defaults]
user=tungsten
install-directory=/opt/continuent
profile-script=~/.bash_profile
disable-security-controls=false
rmi-ssl=false
thl-ssl=false
rmi-authentication=false
[alpha]
master=tr-ext-2
members=tr-app-1
datasource-type=mongodb
replication-user=dbadmin
replication-password=Secret123
replication-port=27017
svc-applier-filters=dropstatementdata
replication-host=atlasendpoint.mongodb.net
property=replicator.applier.dbms.connectString=mongodb+srv://$
{replicator.global.db.user}:${replicator.global.db.password}@$
{replicator.global.db.host}/?retryWrites=true&w=majority
[defaults]
user=tungsten
install-directory=/opt/continuent
profile-script=~/.bash_profile
disable-security-controls=true
[alpha]
master=tr-ext-2
members=tr-app-1
datasource-type=mongodb
replication-user=dbadmin
replication-password=Secret123
replication-port=27017
svc-applier-filters=dropstatementdata
MongoDB AtlasMongoDB

11. Applier Configs
[defaults]
user=tungsten
install-directory=/opt/continuent
profile-script=~/.bash_profile
disable-security-controls=true
[alpha]
master=tr-ext-2
members=tr-app-1
datasource-type=kafka
replication-user=root
replication-password=null
replication-port=9092
property=replicator.applier.dbms.zookeeperString=localhost:2181
property=replicator.applier.dbms.requireacks=1
Kafka

12. Demonstration

13. Summary
What we have learnt today
• Reviewed replicator flow
• Explored Kafka and Mongo specific pre-requisites
• Reviewed configurations

14. Congratulations!
• Thank you for following the Tungsten Replicator Master Class!
• Make sure you complete all the tests to receive your digital certificate!
• Re-watch all the training videos as many times as you want!
• How did we do?
• We’d love to hear your feedback (The good and the bad!)
• Email: customer.success@continuent.com
• Any topics we didn’t cover? Let us know!

15. THANK YOU FOR LISTENING
continuent.com
The MySQL Availability Company
Chris Parker, Customer Success Director, EMEA & APAC