Do you have the background necessary to take full advantage of Tungsten Replicator in your environments? Tungsten offers enterprise-quality replication features in an open source package hosted on Google Code. This virtual course teaches you how to set up innovative topologies that solve complex replication problems. We start with single MySQL servers running MySQL replication and show a simple path migration path to Tungsten. Course Topics - Checking host and MySQL prerequisites - Downloading code from http://code.google.com/p/tungsten-replicator/ - Installation using the tungsten-installer utility - Transaction filtering using standard filters as well as customized filters you write yourself - Enabling and managing parallel replication - Configuring multi-master and fan-in using multiple replication services - Backup and restore integration - Troubleshooting replication problems - Logging bugs and participating in the Tungsten Replicator community Replication is a powerful technology that takes knowledge and planning to use effectively. We give you the background that makes replication easier to set up, and allows you to take full advantage of the Tungsten Replicator benefits. Learn how to configure and use it more effectively for your projects in the cloud as well as on-premises hardware.

1. Tungsten University:
Set Up and Operate
Tungsten Replication
Robert Hodges, CEO
Giuseppe Maxia, Director of QA
©Continuent 2013

2. About Continuent
• The leading provider of clustering and
replication for open source DBMS
• Tungsten Clustering - Commercial-grade HA,
performance scaling and data management
for MySQL
• Tungsten Replication - Real-time, "exible,
high-performance replication
©Continuent 2013 2

3. Introducing Tungsten Replicator
©Continuent 2013 3

4. Tungsten Replicator Overview
Master
Replicator THL
Download
transactions
via network (Transactions + Metadata)
DBMS
Logs
Slave Replicator THL
Apply using JDBC (Transactions + Metadata)
©Continuent 2013 4

5. Tungsten Replication Service
Pipeline
Stage Stage Stage
Extract Filter Apply Extract Filter Apply Extract Filter Apply
Master Transaction In-Memory Slave
DBMS History Log Queue DBMS
©Continuent 2013 5

6. Overview of Installation Process
1. Set up hosts
2. Prepare MySQL replicas
3. Download software
4. Install using tungsten-installer
Amazon Setup:
https://docs.continuent.com/wiki/display/TEDOC/
Preparing+EC2+Servers
©Continuent 2013 6

7. How tungsten-installer Works
Staging copy db1
of files
db2
check prereqs
copy code db3
configure
©Continuent 2013 7

8. What Is Tungsten Doing?
©Continuent 2013 8

9. Commands to Check Replication
• trepctl services -- All services
• trepctl status -- Overall pipe status
• trepctl status -name stages -- Stage summary
• trepctl status -name stores -- Store summary
• trepctl status -name tasks -- Task detail
• trepctl [-service name] heartbeat -- Is it Alive?
©Continuent 2013 9

10. Changing Replicator Con!guration
©Continuent 2013 10

11. Commands to Change Con!guration
• con#gure-service - Alter con#guration
• update - Upgrade the replicator to a new
version
Hint: Don’t change replicator
configuration files directly...
©Continuent 2013 11

12. Recon!guring a Replicator
• No Restart
trepctl offline
tools/configure-service ...
trepctl configure
trepctl online
• With Restart
tools/configure-service ...
replicator restart
©Continuent 2013 12

13. Enabling Parallel Replication
©Continuent 2013 13

14. Parallel Replication of Sharded Data
Pipeline
Stage
Stage Stage Extract Filter Apply
Extract Filter Apply Extract Filter Apply Extract Filter Apply
Extract Filter Apply
Master Transaction Parallel Slave
DBMS History Log Queue DBMS
©Continuent 2013 14

15. Fixing Problems
©Continuent 2013 15

16. The Scene of the Crime
trepctl -host db2 status
Processing status command...
NAME VALUE
---- -----
...
pendingError : Event application failed:
seqno=1957469 fragno=0 message=java.sql.SQLException:
Statement failed on slave but succeeded on master
pendingErrorCode : NONE
pendingErrorEventId : mysql-bin.000008:0000000461736666;0
pendingErrorSeqno : 1957469
pendingExceptionMessage: java.sql.SQLException: Statement
failed on slave but succeeded on master
create table trouble(id int primary
key) /* ___SERVICE___ = [university] */
...
Finished status command...
©Continuent 2013 16

17. Correcting problems
1. Look at the transaction (thl list -seqno NNN)
2. Look at the replicator message log
3. Fix the problem and go online
OR
4. Skip over the problem and go online
OR
5. Reprovision the slave
©Continuent 2013 17

18. Tungsten Logs
©Continuent 2013 18

19. Simple Facts about Logs
• Replicator log = THL (Transaction History Log)
• One THL per replication service
• 2x the size of MySQL binlog
• Default retention of 7 days
• Take service o$ine to move or delete logs
safely
• The thl tool shows you log contents
©Continuent 2013 19

20. Resetting Replication
©Continuent 2013 20

21. Resetting and Reinstalling
Reset
1. Stop replicator
2. Remove THL #les
3. Remove tungsten_<svc> databases
Reinstall
1. All of the above, plus delete /opt/continuent/*
2. Rerun installation
©Continuent 2013 21

22. Backup and Restore
©Continuent 2013 22

23. What Backup Tools Does MySQL Have?
MySQL
mysqldump
Server
Process (dataset < 10Gb)
xtrabackup
DBMS Files (10Gb < dataset < 1Tb
tar/rsync
(dataset < 1Tb)
file system snapshot
File System
(1Tb < dataset)
©Continuent 2013 23

24. Advanced Installation
using Cookbooks
©Continuent 2013 24

25. How Do I Install Fan-In Replication?
db1
Replicator
Service db1
Replicator
Service db1
Service db2
db2
Replicator
Service db2 db3
©Continuent 2013 25

26. How Do I install Multi-Master?
db1 Replicator
Service db1
Service db2
db3
Service db3
Service db1
Service db2
db2
Service db3
Service db1
Service db2
Service db3
Replicator
©Continuent 2013 26

27. Tungsten Cookbook Helps!
• Installs complex topologies automatically
• Directions in cookbook/README
• Parameters in cookbook/USER_VALUES.sh
• Node names in COMMON_NODES.sh
• Installation script for each supported
topology
©Continuent 2013 27

28. Joining the Community
©Continuent 2013 28

29. Tungsten Replicator is Open Source
• Project home:
http://code.google.com/p/tungsten-replicator/
• Log bugs, #nd builds, post in replicator discussion
group
• Documentation:
https://docs.continuent.com/wiki/display/TEDOC/
Tungsten+Documentation+Home
©Continuent 2013 29

30. Wrapping Up
©Continuent 2013 30

31. Tungsten University Sessions
• Unleashing the Power of Tungsten Connectors
(February 21 & 26)
• Managing Multi-Master and Fan-in Topologies
(March)
• Zero-downtime DBMS Migration and
Maintenance (March)
Send any feedback to: tu@continuent.com
©Continuent 2013 31

32. 560 S. Winchester Blvd., Suite 500 Our Blogs:
San Jose, CA 95128 http://scale-out-blog.blogspot.com
Tel +1 (866) 998-3642 http://datacharmer.org/blog
Fax +1 (408) 668-1009 http://www.continuent.com/news/blogs
e-mail: sales@continuent.com
Continuent Web Page:
http://www.continuent.com
Tungsten Replicator 2.0:
http://code.google.com/p/tungsten-replicator
©Continuent 2012.