Download as PDF, PPTX
![©Continuent 2014
Tpm installation overview
8
/opt/continuent/etc/tungsten/tungsten.ini
check prereqs
[copy code]
configure topology
[defaults]!
user=tungsten!
home-directory=/opt/continuent!
datasource-user=tungsten!
datasource-password=secret!
application-port=3306!
application-user=app_user!
application-password=secret!
skip-validation-check=MySQLPermissionsCheck!
start-and-report=true!
!
[east]!
master=db1!
slaves=db2,db3!
topology=master-slave!
tpm update
(Installation)](https://image.slidesharecdn.com/tungstenuniversity05-15-2014-140515114427-phpapp02/85/Setup-Operate-Tungsten-Replicator-8-320.jpg)
![©Continuent 2014
Commands to Check Replication
• trepctl services -- All services
• trepctl status -- Overall pipeline status
• trepctl status -name stages -- Stage summary
• trepctl status -name stores -- Store summary
• trepctl status -name tasks -- Task detail
• trepctl [-service name] heartbeat -- Is it Alive?
10](https://image.slidesharecdn.com/tungstenuniversity05-15-2014-140515114427-phpapp02/85/Setup-Operate-Tungsten-Replicator-10-320.jpg)
![©Continuent 2014
The Scene of the Crime
16
trepctl -host db2 status
Processing status command...
NAME VALUE
---- -----
...
pendingError : Event application failed:
seqno=1957469 fragno=0 message=java.sql.SQLException:
Statement failed on slave but succeeded on master
pendingErrorCode : NONE
pendingErrorEventId : mysql-bin.000008:0000000461736666;0
pendingErrorSeqno : 1957469
pendingExceptionMessage: java.sql.SQLException: Statement
failed on slave but succeeded on master
create table trouble(id int primary
key) /* ___SERVICE___ = [university] */
...
Finished status command...](https://image.slidesharecdn.com/tungstenuniversity05-15-2014-140515114427-phpapp02/85/Setup-Operate-Tungsten-Replicator-16-320.jpg)
Uploaded byContinuent
Setup & Operate Tungsten Replicator
This document provides a comprehensive overview of the Tungsten replicator, a solution for clustering and replication in open source databases, primarily MySQL. It covers installation procedures, configuration, commands for checking and troubleshooting replication, as well as advanced installation patterns. Additionally, it includes resources for community involvement and further learning about Tungsten replicator.
More Related Content
Similar to Setup & Operate Tungsten Replicator
Setup & Operate Tungsten Replicator
- 1. ©Continuent 2014 Tungsten University: SetUp and Operate Tungsten Replication Robert Hodges, CEO
- 2. ©Continuent 2014 Introducing Continuent 2 •The leading provider of clustering and replication for open source DBMS • Our Product: Continuent Tungsten • Clustering - Commercial-grade HA, performance scaling and data management for MySQL • Replication - Flexible, high-performance data movement
- 3. ©Continuent 2014 Introducing TungstenReplicator 3
- 4. ©Continuent 2014 Tungsten ReplicatorOverview 4 Master (Transactions + Metadata) Slave THL DBMS Logs Replicator (Transactions + Metadata) THLReplicator Download transactions via network Apply using JDBC
- 5. ©Continuent 2014 Master ReplicationService 5 Extract Filter Apply Stage Extract Filter Apply Stage Pipeline MySQL Master Transaction History Log In-Memory Queue Slave ReplicatorsBinlog tcp/ip
- 6. ©Continuent 2014 Slave ReplicationService 6 Extract Filter Apply Stage Extract Filter Apply Stage Extract Filter Apply Stage Pipeline Transaction History Log In-Memory Queue Slave DBMS Master Replicator tcp/ip
- 7. ©Continuent 2014 EC2 InstallationUsing Github project 1.Set up Github vagrant project git clone https://github.com/continuent/ continuent-vagrant 2.Pick EC2 Vagrantfile and fill in keys 3.Copy examples/MasterSlave/default.pp to manifests/ 4.Run launch.sh 7 (Follow directions in README.md)
- 8. ©Continuent 2014 Tpm installationoverview 8 /opt/continuent/etc/tungsten/tungsten.ini check prereqs [copy code] configure topology [defaults]! user=tungsten! home-directory=/opt/continuent! datasource-user=tungsten! datasource-password=secret! application-port=3306! application-user=app_user! application-password=secret! skip-validation-check=MySQLPermissionsCheck! start-and-report=true! ! [east]! master=db1! slaves=db2,db3! topology=master-slave! tpm update (Installation)
- 9. ©Continuent 2014 What IsTungsten Doing? 9
- 10. ©Continuent 2014 Commands toCheck Replication • trepctl services -- All services • trepctl status -- Overall pipeline status • trepctl status -name stages -- Stage summary • trepctl status -name stores -- Store summary • trepctl status -name tasks -- Task detail • trepctl [-service name] heartbeat -- Is it Alive? 10
- 11. ©Continuent 2014 Changing ReplicatorConfiguration 11
- 12. ©Continuent 2014 Reconfiguring aReplicator • Edit /etc/tungsten/tungsten.ini • No restart on local host trepctl offline! tpm update --no-restart! trepctl configure! trepctl online! ! • With automatic restart tpm update 12 Don’t change replicator configuration files directly!
- 13. ©Continuent 2014 Enabling ParallelReplication 13
- 14. ©Continuent 2014 Parallel Replicationof Sharded Data 14 Extract Filter Apply Stage Extract Filter Apply Stage Stage Pipeline Transaction History Log Parallel Queue Slave DBMS Extract Filter Apply Extract Filter Apply Extract Filter Apply Master Replicator tcp/ip
- 15.
- 16. ©Continuent 2014 The Sceneof the Crime 16 trepctl -host db2 status Processing status command... NAME VALUE ---- ----- ... pendingError : Event application failed: seqno=1957469 fragno=0 message=java.sql.SQLException: Statement failed on slave but succeeded on master pendingErrorCode : NONE pendingErrorEventId : mysql-bin.000008:0000000461736666;0 pendingErrorSeqno : 1957469 pendingExceptionMessage: java.sql.SQLException: Statement failed on slave but succeeded on master create table trouble(id int primary key) /* ___SERVICE___ = [university] */ ... Finished status command...
- 17. ©Continuent 2014 Correcting problems 1.Lookat the transaction (thl list -seqno NNN) 2.Look at the replicator message log 3.Fix the problem and go online OR 4.Skip over the problem and go online OR 5.Reprovision the slave 17
- 18.
- 19. ©Continuent 2014 Simple Factsabout Logs • Replicator log = THL (Transaction History Log) • One THL per replication service • 2x the size of MySQL binlog • Default retention of 7 days • Take service offline to move or delete logs safely • The thl tool shows you log contents 19
- 20.
- 21. ©Continuent 2014 Resetting/Reinstalling Automatically Reset 1.tpmreset Reinstall 1.tpm uninstall --i-am-sure 2.Rerun installation 21
- 22. ©Continuent 2014 Resetting/Reinstalling Manually Reset 1.Stopreplicator 2.Remove THL files 3.Remove tungsten_<svc> databases Reinstall 1.All of the above, plus delete /opt/continuent/* 2.Rerun installation 22
- 23. ©Continuent 2014 Backup andRestore 23
- 24. ©Continuent 2014 What BackupTools Does MySQL Have? 24 MySQL Server Process File System DBMS Files mysqldump (dataset < 10Gb) xtrabackup (10Gb < dataset < 1Tb tar/rsync (dataset < 1Tb) file system snapshot (1Tb << dataset)
- 25.
- 26. ©Continuent 2014 How DoI Install Fan-In Replication? 26 db1 Replicator db3 Service db1 db2 Replicator Service db2 Replicator Service db1 Service db2
- 27. ©Continuent 2014 How DoI install Multi-Master? 27 db1 Replicator Service db1 Service db2 Service db3 db3 Service db1 Service db2 Service db3 db2 Replicator Service db1 Service db2 Service db3
- 28. ©Continuent 2014 Tpm UnderstandsTopologies! • Use --topology argument! 28 Fan-In StarAll-Masters --topology=fan-in --topology=all-masters --topology=star
- 29. ©Continuent 2014 Tpm installfrom staging directory 29 db1 db2 db3 Staging copy of files check prereqs copy code configure topology tpm configure ... tpm update
- 30. ©Continuent 2014 Commands toPerform Staging Install 30 $ wget http://downloads.tungsten-replicator.org/download.php? file=tungsten-replicator-2.2.1-403.tar.gz ! $ tar -xvzf tungsten-replicator-2.2.1-403.tar.gz ! $ ln -s tungsten-replicator-2.2.1-403 tungsten ! $ tungsten/tools/tpm configure defaults --reset --user=tungsten --home-directory=/opt/continuent --datasource-user=tungsten --datasource-password=secret --skip-validation-check=MySQLPermissionsCheck --start-and-report=true ! $ tungsten/tools/tpm configure fanin --members=db1,db2,db3 --masters=db1,db2 --master-services=db1,db2 --topology=fan-in ! $ tungsten/tools/tpm update
- 31. ©Continuent 2014 Joining theCommunity 31
- 32. ©Continuent 2014 Tungsten Replicatoris Open Source 32 • Project home: http://code.google.com/p/tungsten-replicator/ • Log bugs, find builds, post in replicator discussion group • Documentation: https://docs.continuent.com
- 33. ©Continuent 2014 Heterogeneous OracleMySQL OracleMySQL MySQL star master-slave fan-in slave all-masters MySQLOracle Oracle
- 34.
- 35. ©Continuent 2014 More TungstenUniversity Webinars! 35 • Set Up and Operate Tungsten Clusters (22 May 2014 10am PDT) • Set Up and Operate Real-Time Data Loading into Hadoop (29 May 2014, 10am PDT) • Set Up and Operate Open Source Oracle Replication (5 June 2014, 10am PDT) Send feedback to: tu@continuent.com
- 36. ©Continuent 2014 Continuent WebPage: http://www.continuent.com ! Tungsten Replicator 2.0: http://code.google.com/p/tungsten-replicator Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.org/blog http://www.continuent.com/news/blogs http://flyingclusters.blogspot.com/ 560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: sales@continuent.com