Download as PDF, PPTX
![©Continuent 2013
Tungsten Replicator 2.1.2
• Releases as GA
• Main features:
• installation with tpm
• security layer
• full support for MySQL 5.6
• improved management tools
• trepctl properties
• trepctl services [-full] [-json]
• trepctl clients
• trepctl status [-json]
• thl list [-headers] [-json]
9
9Sunday, September 22, 13](https://image.slidesharecdn.com/juggleyourdatawithtungstenreplicator-130922155738-phpapp02/85/Juggle-your-data-with-Tungsten-Replicator-9-320.jpg)
Uploaded byGiuseppe Maxia
Juggle your data with Tungsten Replicator
The document provides an overview of Continuent's Tungsten Replicator, highlighting its clustering and replication capabilities for open-source DBMS, particularly MySQL. It details installation processes, including different replication topologies such as master/slave, bi-directional, and multi-master installations, as well as the added security features for data encryption. Key installation commands and configurations are presented for various setups and use cases within the Tungsten ecosystem.
More Related Content
Similar to Juggle your data with Tungsten Replicator
Juggle your data with Tungsten Replicator
- 1. ©Continuent 2013 Juggle yourdata with Tungsten Replicator Giuseppe Maxia, Director of QA Continuent, Inc 1Sunday, September 22, 13
- 2. ©Continuent 2013 Introducing Continuent 2 •The leading provider of clustering and replication for open source DBMS • Our Product: Continuent Tungsten • Clustering - Commercial-grade HA, performance scaling and data management for MySQL • Replication - Flexible, high-performance data movement 2Sunday, September 22, 13
- 3. ©Continuent 2013 A Reviewof Tungsten Replicator 3 3Sunday, September 22, 13
- 4. ©Continuent 2013 Tungsten ReplicatorOverview 4 Master (Transactions + Metadata) Slave THL DBMS Logs Replicator (Transactions + Metadata) THLReplicator Download transactions via network Apply using JDBC 4Sunday, September 22, 13
- 5. ©Continuent 2013 Master ReplicationService 5 Extract Filter Apply Stage Extract Filter Apply Stage Pipeline MySQL Master Transaction History Log In-Memory Queue Slave ReplicatorsBinlog tcp/ip 5Sunday, September 22, 13
- 6. ©Continuent 2013 Slave ReplicationService 6 Extract Filter Apply Stage Extract Filter Apply Stage Extract Filter Apply Stage Pipeline Transaction History Log In-Memory Queue Slave DBMS Master Replicator tcp/ip 6Sunday, September 22, 13
- 7. ©Continuent 2013 star master-slave Heterogeneous fan-inslave all-masters MySQL Oracle Oracle MySQL Oracle Oracle MySQL MySQL 7Sunday, September 22, 13
- 8. ©Continuent 2013 Tungsten Replicator2.1.1 8 8Sunday, September 22, 13
- 9. ©Continuent 2013 Tungsten Replicator2.1.2 • Releases as GA • Main features: • installation with tpm • security layer • full support for MySQL 5.6 • improved management tools • trepctl properties • trepctl services [-full] [-json] • trepctl clients • trepctl status [-json] • thl list [-headers] [-json] 9 9Sunday, September 22, 13
- 10. ©Continuent 2013 Working withTungsten 10 10Sunday, September 22, 13
- 11. ©Continuent 2013 Overview ofInstallation Process 1. Set up hosts 2. Prepare MySQL replicas 3. Download software 4. Install using tungsten-installer & con!gure- service tpm 11 Amazon Setup: https://docs.continuent.com/wiki/display/TEDOC/ Preparing+EC2+Servers 11Sunday, September 22, 13
- 12. ©Continuent 2013 How tungsten-installerWorks for Basic Master/Slave Deployment 12 db1 db2 db3 Staging copy of files check prereqs copy code configure 12Sunday, September 22, 13
- 13. ©Continuent 2013 Tungsten master/slavereplication 13 host1 host2 host3 alpha alpha alpha THL THL THL installer Old installation and New installation 13Sunday, September 22, 13
- 14. ©Continuent 2013 Bi-directional replication 14 host1 host2 alpha alpha bravo bravo Installmaster on host1 installer install master on host2 installer install slave service on host1 installer install slave service on host2 installer OLD INSTALLATION 14Sunday, September 22, 13
- 15. ©Continuent 2013 Bi-directional replication 15 host1 host2 alpha alpha bravo bravo NEWINSTALLATION Install all master and slave services on all hosts at once installer 15Sunday, September 22, 13
- 16. ©Continuent 2013 16 ./tools/tungsten-installer --master-slave --master-host=$MASTER2 --datasource-user=tungsten --datasource-password=secret --service-name=bravo --home-directory=$TUNGSTEN_HOME --cluster-hosts=$MASTER2 --start $TUNGSTEN_HOME/tungsten/tools/configure-service-C --quiet --host=$MASTER1 --datasource=$MASTER1 --local-service-name=alpha --role=slave --service-type=remote --release-directory=$TUNGSTEN_HOME/tungsten --skip-validation-check=THLStorageCheck --master-thl-host=$MASTER2 --master-thl-port=2112 --svc-start bravo ./tools/tungsten-installer --master-slave --master-host=$MASTER1 --datasource-user=tungsten --datasource-password=secret --service-name=alpha --home-directory=$TUNGSTEN_HOME --cluster-hosts=$MASTER1 --start $TUNGSTEN_HOME/tungsten/tools/configure-service -C --quiet --host=$MASTER2 --datasource=$MASTER2 --local-service-name=bravo --role=slave --service-type=remote --release-directory=$TUNGSTEN_HOME/tungsten --skip-validation-check=THLStorageCheck --master-thl-host=$MASTER1 --master-thl-port=2112 --svc-start alpha 1 2 3 4 Multi-master installation (2 nodes) until TR 2.1.0 ~ 3 minutes using tungsten-installer 16Sunday, September 22, 13
- 17. ©Continuent 2013 4 nodesall-masters 17 host1 host4 host2 host3 alpha alpha bravo bravo charlie charlie delta delta bravobravo alpha delta delta charlie charlie alpha 17Sunday, September 22, 13
- 18. ©Continuent 2013 18 Multi-masterinstallation (4 nodes) until TR 2.1.0 ~ 8 minutes tungsten-installer host1 - alpha 1 tungsten-installer host2 - bravo 2 tungsten-installer host3 - charlie 3 tungsten-installer host4 - delta 4 con!gure-service host1 - bravo 5 con!gure-service host1 - charlie 6 con!gure-service host1 - delta 7 con!gure-service host2 - alpha 8 con!gure-service host2 - charlie 9 con!gure-service host2 - delta 10 con!gure-service host3 - alpha 11 con!gure-service host3 - bravo 12 con!gure-service host3 - delta 13 con!gure-service host4 - alpha 14 con!gure-service host4 - bravo 15 con!gure-service host4 - charlie 16 using tungsten-installer 18Sunday, September 22, 13
- 19. ©Continuent 2013 19 ./tools/tpminstall my_cluster --topology=all-masters --home-directory=$TUNGSTEN_HOME --replication-user=tungsten --replication-password=secret --masters=host1,host2 --master-services=alpha,bravo --start Multi-master installation (2 nodes) with TR 2.1.1 ~ 40 seconds using tpm 19Sunday, September 22, 13
- 20. ©Continuent 2013 20 ./tools/tpminstall my_cluster --topology=all-masters --home-directory=$TUNGSTEN_HOME --replication-user=tungsten --replication-password=secret --masters=host1,host2,host3,host4 --master-services=alpha,bravo,charlie,delta --start Multi-master installation (4 nodes) with TR 2.1.1 ~ 45 seconds using tpm 20Sunday, September 22, 13
- 21. ©Continuent 2013 One morereason why tpm is better • Security: • all data in transit (transaction history logs, or THL) is encrypted using SSL • all administrative tra"c is encrypted with SSL • Transparent to the user • Independent of the database server (works also for heterogeneous replication) 21 21Sunday, September 22, 13
- 22. ©Continuent 2013 Tungsten replicatorwithout security 22 replicator services host1 host4 host2 host3 alpha alpha alpha alpha master slave THL THL THLTHL plain text 22Sunday, September 22, 13
- 23. ©Continuent 2013 Tungsten Replicatorwith security 23 replicator services host1 host4 host2 host3 alpha alpha alpha alpha master slave THL THL THLTHL SSL SSL SSL SSL 23Sunday, September 22, 13
- 24. ©Continuent 2013 Installing withsecurity 1: create keys 24 ./cookbook/create-security-keys HarryPotter ... ... # Options for tpm (stored in ./cookbook/security.options) --thl-ssl=true --rmi-ssl=true --rmi-authentication=true --rmi-user=cookbook --java-keystore-password=HarryPotter --java-truststore-password=HarryPotter --java-truststore-path=$PWD/cookbook/truststore.ts --java-keystore-path=$PWD/cookbook/keystore.jks --java-jmxremote-access-path=$PWD/cookbook/jmxremote.access --java-passwordstore-path=$PWD/cookbook/passwords.store 24Sunday, September 22, 13
- 25. ©Continuent 2013 Installing withsecurity - 2: run tpm 25 ./tools/tpm install alpha --topology=master-slave --home-directory=/opt/continuent/replicator --replication-user=tungsten --replication-password=secret --master=host1 --slaves=host2,host3,host4 $(cat ./cookbook/security.options) --start ./tools/tpm install my_cluster --topology=all-masters --home-directory=$TUNGSTEN_HOME --replication-user=tungsten --replication-password=secret --masters=host1,host2,host3,host4 $(cat ./cookbook/security.options) --master-services=alpha,bravo,charlie,delta --start 25Sunday, September 22, 13
- 26. ©Continuent 2013 Demo :security 26 26Sunday, September 22, 13
- 27. ©Continuent 2013 Advanced Installation usingCookbook 27 27Sunday, September 22, 13
- 28. ©Continuent 2013 Installing Master/SlaveReplication ... 28 ./cookbook/install_master_slave host1 host2 host3 alpha alpha alpha THL THL THL 28Sunday, September 22, 13
- 29. ©Continuent 2013 master/slave usingtpm 29 ./tools/tpm install alpha --topology=master-slave --home-directory=/opt/continuent/replicator --replication-user=tungsten --replication-password=secret --master=host1 --slaves=host2,host3,host4 --start 29Sunday, September 22, 13
- 30. ©Continuent 2013 Installing Fan-InReplication 30 ./cookbook/install_fan_in host1 host4 host2 host3 alpha bravo charlie bravo charlie alpha 30Sunday, September 22, 13
- 31. ©Continuent 2013 fan-in usingtpm 31 ./tools/tpm install many_towns --replication-user=tungsten --replication-password=secret --home-directory=/opt/continuent/replication --masters=host1,host2,host3 --slaves=host4 --master-services=alpha,bravo,charlie --topology=fan-in --start 31Sunday, September 22, 13
- 32. ©Continuent 2013 Install Multi-Masterreplication 32 ./cookbook/install_all_masters host1 host2 alpha alpha bravo bravo 32Sunday, September 22, 13
- 33. ©Continuent 2013 multi-master usingtpm 33 ../tools/tpm install musketeers --reset --topology=all-masters --home-directory=/opt/continuent/replicator --replication-user=tungsten --replication-password=secret --masters=host1,host2 --master-services=alpha,bravo --start 33Sunday, September 22, 13
- 34.
- 35. ©Continuent 2013 extending multi-master (1:install) 35 ../tools/tpminstall musketeers --reset --topology=all-masters --home-directory=/opt/continuent/replicator --replication-user=tungsten --replication-password=secret --masters=host1,host2 --master-services=alpha,bravo --start 35Sunday, September 22, 13
- 36. ©Continuent 2013 extending multi-master (2:add a node) 36 ./tools/tpm configure musketeers --topology=all-masters --home-directory=/opt/continuent/replicator --replication-user=tungsten --replication-password=secret --masters=host1,host2,host3 --master-services=alpha,bravo,charlie --start ./tools/tpm update 36Sunday, September 22, 13
- 37. ©Continuent 2013 Extending further... 37 host1 host4 host2 host3 alpha alpha bravo bravo charlie charlie delta delta bravobravo alpha delta delta charlie charlie alpha host1 host2 host3 alpha alpha bravo bravo charlie charlie bravo alpha charlie 37Sunday, September 22, 13
- 38. ©Continuent 2013 extending multi-master (3:add one more node) 38 ./tools/tpm configure musketeers --topology=all-masters --home-directory=/opt/continuent/replicator --replication-user=tungsten --replication-password=secret --masters=host1,host2,host3,host4 --master-services=alpha,bravo,charlie,delta --start ./tools/tpm update 38Sunday, September 22, 13
- 39. ©Continuent 2013 Install aStar Topology 39 ./cookbook/install_star host1 host4 host2 host3 alpha bravo charlie charlie delta bravo alpha delta charlie charlie hub 39Sunday, September 22, 13
- 40. ©Continuent 2013 star usingtpm 40 ./tools/tpm install constellation --replication-user=tungsten --replication-password=secret --home-directory=/opt/continuent/replication --masters=host1,host2,host4 --hub=host3 --hub-service=charlie --master-services=alpha,bravo,delta --topology=star --start 40Sunday, September 22, 13
- 41. ©Continuent 2012. More complextopologies 41 41Sunday, September 22, 13
- 42. ©Continuent 2013 all-masters withattached slaves 42 host1 host4 host2 host3 alpha alpha bravo bravo bravo alpha data from goes to host1 host2,host3 host2 host1,host4 42Sunday, September 22, 13
- 43. ©Continuent 2013 all-masters withattached slaves 43 host1 host4 host2 host3 alpha alpha bravo bravo charlie charlie delta delta bravobravo alpha delta delta charlie charlie alpha host6host5 deltacharlie data from goes to host1 host2,host3,host4 host2 host1,host3,host4 host3 host1,host2,host4,host5 host4 host1,host2,host3,host6 43Sunday, September 22, 13
- 44. ©Continuent 2013 four all-masterswith attached slaves 1: con!gure defaults 44 #1 ./tools/tpm configure defaults --reset --replication-user=tungsten --replication-password=secret --home-directory=/home/tungsten/installs/cookbook --start 44Sunday, September 22, 13
- 45. ©Continuent 2013 four all-masterswith attached slaves 2: con!gure main all-masters 45 #2 ./tools/tpm configure four_musketeers --masters=host1,host2,host3,host4 --master-services=alpha,bravo,charlie,delta --topology=all-masters 45Sunday, September 22, 13
- 46. ©Continuent 2013 four all-masterswith attached slaves 3: con!gure !rst slave 46 #3 ./tools/tpm configure charlie --hosts=host3,host5 --slaves=host5 --master=host3 46Sunday, September 22, 13
- 47. ©Continuent 2013 four all-masterswith attached slaves 4: con!gure second slave 47 #4 ./tools/tpm configure delta --hosts=host4,host6 --slaves=host6 --master=host4 #5 ./tools/tpm install 47Sunday, September 22, 13
- 48. ©Continuent 2013 four all-masterswith attached slaves 5: !nal step: install 48 #5 ./tools/tpm install 48Sunday, September 22, 13
- 49. ©Continuent 2013 all-masters withattached mongoDB slave 49 host1 host4 host2 host3 alpha alpha bravo bravo charlie charlie delta delta bravobravo alpha delta delta charlie charlie alpha host5 alpha 49Sunday, September 22, 13
- 50. ©Continuent 2013 all-masters withmongodb slave 1: install masters with heterogeneous options 50 ./tools/tpm configure all_masters --reset --log=timestamp --replication-port=3306 --replication-user=tungsten --replication-password=secret --home-directory=/home/tungsten/installs/cookbook --repl-disable-relay-logs=true -a --masters=ubuntu1,ubuntu2,ubuntu3,ubuntu4 --master-services=alpha,bravo,charlie,delta --topology=all-masters --java-file-encoding=UTF8 --mysql-use-bytes-for-string=false --mysql-enable-enumtostring=true --mysql-enable-settostring=true --svc-extractor-filters=colnames,pkey --start-and-report 50Sunday, September 22, 13
- 51. ©Continuent 2013 all-masters withmongodb slave 2: add mongodb slave and install 51 ./tools/tpm configure alpha --datasource-type=mongodb --hosts=host1 --master=ubuntu1 --replication-port=6700 --home-directory=~/deploy/mongodb --slaves=host1 --role=slave --master-thl-port=2112 --thl-port=2113 --rmi-port=10100 --start-and-report ./tools/tpm install --verbose 51Sunday, September 22, 13
- 52. ©Continuent 2013 DEMO: multi masterinstallation 52 52Sunday, September 22, 13
- 53. ©Continuent 2012. Overview ofTungsten Cookbook 53 53Sunday, September 22, 13
- 54.
- 55.
- 56. ©Continuent 2013 Tungsten Cookbook $cat COMMON_NODES.sh export NODE1=host1 export NODE2=host2 export NODE3=host3 export NODE4=host4 56 56Sunday, September 22, 13
- 57. ©Continuent 2013 Tungsten Cookbook $cat USER_VALUES.sh # User defined values for the cluster to be installed. export TUNGSTEN_BASE=$HOME/installs/cookbook export DATABASE_USER=tungsten export BINLOG_DIRECTORY=/var/lib/mysql export MY_CNF=/etc/my.cnf export DATABASE_PASSWORD=secret export DATABASE_PORT=3306 export TUNGSTEN_SERVICE=cookbook export RMI_PORT=10000 export THL_PORT=2112 export START_OPTION=start 57 57Sunday, September 22, 13
- 58. ©Continuent 2013 Sample all-mastersInstallation • edit cookbook/COMMON_NODES.sh • edit cookbook/USER_VALUES.sh • run cookbook/install_all_masters • and then: • run cookbook/show_cluster • run cookbook/test_cluster 58 58Sunday, September 22, 13
- 59. ©Continuent 2013 DEMO: cookbook maintools * show status * show logs * edit con!guration * show services 59 59Sunday, September 22, 13
- 60. ©Continuent 2013 Joining theCommunity 60 60Sunday, September 22, 13
- 61. ©Continuent 2013 Tungsten Replicatoris Open Source 61 • Project home: http://code.google.com/p/tungsten-replicator/ • Log bugs, !nd builds, post in replicator discussion group • Documentation: https://docs.continuent.com/wiki/display/TEDOC/ Tungsten+Documentation+Home 61Sunday, September 22, 13
- 62. ©Continuent 2013 Wrapping Up 62 62Sunday,September 22, 13
- 63. ©Continuent 2012. Continuent WebPage: http://www.continuent.com Tungsten Replicator 2.1: http://code.google.com/p/tungsten-replicator Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.blogspot.com http://www.continuent.com/news/blogs 560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: sales@continuent.com 63Sunday, September 22, 13