Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tungsten University: Replicate Between MySQL And Oracle


Published on

Oracle is the most powerful DBMS in the world. However, Oracle's expensive and complex replication makes it difficult to build highly available applications or move data in real-time to data warehouses and popular databases like MySQL.

In this webinar you learn how Continuent Tungsten solves problems with Oracle replication at a fraction of the cost of other solutions and with less management overhead too – picture Oracle GoldenGate without the price tag!

We demo constructing a highly available site using Oracle-to-Oracle replication. We then show you how to replicate data in real time from Oracle to MySQL as well as load a data warehouse.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Tungsten University: Replicate Between MySQL And Oracle

  1. 1. ©Continuent 2013.Tungsten UniversityReplicate betweenMySQL and OracleLinas Virbalas, Senior Software EngineerRobert Hodges, CEO
  2. 2. ©Continuent 2013Introducing Continuent2• The leading provider of clustering andreplication for open source DBMS• Our Product: Continuent Tungsten• Clustering - Commercial-grade HA, performancescaling and data management for MySQL• Replication - Flexible, high-performance datamovement
  3. 3. ©Continuent 2013.Tungsten Replicator3Tungsten Replicator is a fast,open source, databasereplication engineDesigned for speed and flexibilityGPLV2 license100% open sourceAnnual support subscription available from Continuent“Golden Gate without the Price Tag”®
  4. 4. ©Continuent 2013.MasterSlaveTHL(Transactions + Metadata)Login asreplicationclientApply using JDBCReplicatorMySQLBinlogTHLReplicatorStoreforwardStoreReplicator Overview(Transactions + Metadata)
  5. 5. ©Continuent 2012 5Extract Filter ApplyStageExtract Filter ApplyStageExtract Filter ApplyStagePipelineMasterDBMSTransactionHistory LogQueueSlaveDBMSTungsten Replication Service
  6. 6. ©Continuent 2013.Multiple Services per ReplicatorServicefrommysqlServicefromoraclefrommysqlfromoracleReplicatorReplicatorReplicatorAggregated
  7. 7. ©Continuent 2013starmaster-slave Heterogeneousfan-in slave all-mastersMySQLOracleOracleMySQL OracleOracleMySQL MySQL
  8. 8. ©Continuent 2012.EnablingHeterogeneousReplication
  9. 9. ©Continuent 2013.Steps to Heterogeneous Replication1. Prepare (translate) schema forthe slave DBMS2. Set up replication3. Provision initial data4. Continue real-time replicationddlscantungsten-installerParallel Apply
  10. 10. ©Continuent 2012.Replicating fromMySQL to Oracle
  11. 11. ©Continuent 2013.Use Case: Real-Time Sales/OrderFul!llmentReal-Time ReplicationWeb-based Sales Purchase Ordering3MOrderItems~1MXacts/Day
  12. 12. ©Continuent 2012.1.Translating schemafor the slave
  13. 13. ©Continuent 2012Introducing ddlscan13• Helps to prepare schema for heterogeneousreplication• Single command-line call• Part of Tungsten Replicator 2.1, GPL v2
  14. 14. ©Continuent 2012Translating Schema14• Goal - set up heterogeneous replicationReplication
  15. 15. ©Continuent 2012Translating Schema15• Beginning - how to convert tables?emptyTables
  16. 16. ©Continuent 2012Translating Schema16• Beginning - how to convert tables?emptyTables•Data types?•Column lengths?•Naming conventions?•Reserved words?
  17. 17. ©Continuent 2012Translating Schema17• ddlscan automates these conversionsemptyTablesddlscan
  18. 18. ©Continuent 2012Features18• Translates schema with replication in mind• Provides errors and warnings proactively• Can rename schema/tables/columns• Easy• Straightforward to extend (write newtemplates)
  19. 19. ©Continuent 2012Usage (MySQL to Oracle Example)19$ cd tungsten-replicator/bin$ ./ddlscan -db test -template ddl-mysql-oracle.vm -user tungsten -pass secret
  20. 20. ©Continuent 2012Translating Schema20• ddlscan looks into source schemaemptyTables
  21. 21. ©Continuent 2012Translating Schema21• ddlscan translates and renders DDL commandsemptyTablesoracle-ddl.sql
  22. 22. ©Continuent 2012Result of ddlscan22CREATE TABLE 3colors/* ERROR: table starts with a number, use rename option and afilter */(id NUMBER(10, 0),color VARCHAR2(1) /* ENUM(R,G,B) */,enabled NUMBER(1) /* BIT(1) - no constraints for 0 or 1 */,acolumnthatdoesntfittooracleat NUMBER(10, 0)/* WARN: truncated column name exceeding 30 characters(acolumnthatdoesntfittooracleatall) *//* ERROR: table must have a primary key! */);CREATE TABLE talks...
  23. 23. ©Continuent 2012Translating Schema23• You run resulting SQL file on OracleemptyTablesoracle-ddl.sql
  24. 24. ©Continuent 2012Translating Schema24• Tables are ready!Tables Translatedtables (empty)
  25. 25. ©Continuent 2013.Architecture25Tables$ ./bin/ddlscanJDBCmetadataLibrariesTungstenReplicatorTemplateengineApacheVelocitySchemaas JAVAobjectsMySQL,Oracle,PostgreSQL*.vmtemplateof choiceRenderedtext output
  26. 26. ©Continuent 2012.2. Set Up Replication
  27. 27. ©Continuent 2012Master Installation27tools/tungsten-installer -a --master-slave --service-name=frommysql --cluster-hosts=db1 --master-host=db1 --datasource-port=3306 --datasource-user=tungsten --datasource-password=secret --home-directory=/opt/continuent --mysql-use-bytes-for-string=false --mysql-enable-enumtostring=true --mysql-enable-settostring=true --start-and-report
  28. 28. ©Continuent 2012Slave Installation28tools/tungsten-installer -a --master-slave --service-name=frommysql --cluster-hosts=db2 --master-host=db1 --datasource-type=oracle --datasource-oracle-service=ORCL --datasource-user=tungsten_frommysql --datasource-password=secret --home-directory=/opt/continuent --svc-applier-filters=dropstatementdata --start-and-report
  29. 29. ©Continuent 2012.3. ProvisioningInitial Data
  30. 30. ©Continuent 2012Provisioning (Single Threaded)30• Restore a backup and let it replicateReplicationProvisioningSchemaTranslatedTablesProductionSchema
  31. 31. ©Continuent 2013.Parallel Apply on ReplicasExtractFilter(Shard)ApplyExtract/Filter/ApplyStage Stage StagePipelineMasterDBMSSlaveDBMSTransactionHistory LogParallelQueueExtract/Filter/ApplyExtract/Filter/ApplyExtract/Filter/ApplyProvisioningMulti-tenantsource
  32. 32. ©Continuent 2012Provisioning (Parallel Apply)32• Restore a backup and let it replicateProductionSchemaTranslatedtablesReplicationReplicationReplicationReplicationReplicationReplicationReplicationProvisioningSchema
  33. 33. ©Continuent 2012.4. Continue Real-TimeReplication
  34. 34. ©Continuent 2012Continue Replicating Real-Time Data34• Discard provisioning schema and leave singlechannelProductionSchemaProvisionedTablesReplicationReplicationReplicationReplicationReplicationReplicationReplicationProvisioningSchema
  35. 35. ©Continuent 2013.How Does Tungsten Do That?35MySQLBinlogbinlog_format=rowServicefrommysqlTungsten MasterReplicatorMySQLExtractorSpecial Filters•TransformENUM to string•Transform SETto stringServicefrommysqlTungsten SlaveReplicatorSpecial Filters•Drop DDL•Map names to upper case•Rename too long or reservedobjects•Optimize updates to removeunchanged columns•Ignore extra tablesOracleApplierMySQLExtractor
  36. 36. ©Continuent 2012.Replicating fromOracle to MySQL* newly open-sourced!
  37. 37. ©Continuent 2013.Use Case: Web Content PublishingReal-Time PublicationBackend Office Web-Based Catalog
  38. 38. ©Continuent 2013.How Does Tungsten Do That?38ServicefromoracleTungsten MasterReplicatorOracleCDCExtractorNo Special FiltersServicefromoracleTungsten SlaveReplicatorSpecial Filters•Map names to lower case•Ignore extra tables•Heartbeat table renamingdemoschemademo_pubschemaOracle CDC(Synchronous orAsync Hotlog)MySQLApplierOracleCDCExtractor
  39. 39. ©Continuent 2012.Replicating fromOracle to Oracle* newly open-sourced!
  40. 40. ©Continuent 2013.How Does Tungsten Do That?40ServicefromoracleTungsten MasterReplicatorOracleCDCExtractorNo Special FiltersServicefromoracleTungsten SlaveReplicatorSpecial FiltersNo Special Filtersdemoschemademo_pubschemaOracle CDC(Synchronous orAsync Hotlog)Tungsten SlaveReplicatorOracleApplierOracleCDCExtractor
  41. 41. ©Continuent 2013.More ReplicationPossibilities!
  42. 42. ©Continuent 2013.Oracle RAC and Continuent TungstenContinuent TungstenCluster for MySQLOracle 11G RACConnect your favorite clusters!
  43. 43. ©Continuent 2013.Oracle-Based Analytics and ReportingOracle Data WarehouseSharded MySQLOperational DataMySQL Fan-InAggregation Server
  44. 44. ©Continuent 2013.Oracle-Based Analytics and ReportingOracle Data WarehouseSharded MySQLOperational Data (HA)MySQL Fan-InAggregation Server
  45. 45. ©Continuent 2013.Oracle-Based Analytics and ReportingWeb-facing MySQL DBMSwith HA and scalingBackend Oracleapp BBackend Oracleapp AAggregate data
  46. 46. ©Continuent 2013.46More Replication CombinationsPostgreSQLMongoDBVerticaMySQL
  47. 47. ©Continuent 2013Support for Tungsten Replicator47• Community support is available on projectmailing list• Continuent o!ers implementation services andannual 24/7 support subscriptions• We do not o!er hourly or per-incident support(sorry!)• For high availability or disaster recovery needs,check out Continuent Tungsten clustering• Meanwhile, have fun and try it out!
  48. 48. ©Continuent 2013Tungsten University Sessions48• Introduction to Continuent Tungsten 2.0 (May16 and May 21)• Load a Vertica Data Warehouse with MySQLData (May 30 and June 4)Send any feedback to:
  49. 49. ©Continuent 2013. 49Continuent Web Page:http://www.continuent.comTungsten Replicator: Blogs:http://"yingclusters.blogspot.comhttp://scale-out-blog.blogspot.comhttp://datacharmer.blogspot.comhttp://continuent-tungsten.blogspot.com560 S. Winchester Blvd., Suite 500San Jose, CA 95128Tel +1 (866) 998-3642Fax +1 (408) 668-1009e-mail: