Tungsten University: Replicate Between MySQL And Oracle

Tungsten University: Replicate Between MySQL And Oracle



Oracle is the most powerful DBMS in the world. However, Oracle's expensive and complex replication makes it difficult to build highly available applications or move data in real-time to data ...

Oracle is the most powerful DBMS in the world. However, Oracle's expensive and complex replication makes it difficult to build highly available applications or move data in real-time to data warehouses and popular databases like MySQL.

In this webinar you learn how Continuent Tungsten solves problems with Oracle replication at a fraction of the cost of other solutions and with less management overhead too – picture Oracle GoldenGate without the price tag!

We demo constructing a highly available site using Oracle-to-Oracle replication. We then show you how to replicate data in real time from Oracle to MySQL as well as load a data warehouse.



Total Views
Slideshare-icon Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Tungsten University: Replicate Between MySQL And Oracle Tungsten University: Replicate Between MySQL And Oracle Presentation Transcript

    • ©Continuent 2013.Tungsten UniversityReplicate betweenMySQL and OracleLinas Virbalas, Senior Software EngineerRobert Hodges, CEO
    • ©Continuent 2013Introducing Continuent2• The leading provider of clustering andreplication for open source DBMS• Our Product: Continuent Tungsten• Clustering - Commercial-grade HA, performancescaling and data management for MySQL• Replication - Flexible, high-performance datamovement
    • ©Continuent 2013.Tungsten Replicator3Tungsten Replicator is a fast,open source, databasereplication engineDesigned for speed and flexibilityGPLV2 license100% open sourceAnnual support subscription available from Continuent“Golden Gate without the Price Tag”®
    • ©Continuent 2013.MasterSlaveTHL(Transactions + Metadata)Login asreplicationclientApply using JDBCReplicatorMySQLBinlogTHLReplicatorStoreforwardStoreReplicator Overview(Transactions + Metadata)
    • ©Continuent 2012 5Extract Filter ApplyStageExtract Filter ApplyStageExtract Filter ApplyStagePipelineMasterDBMSTransactionHistory LogQueueSlaveDBMSTungsten Replication Service
    • ©Continuent 2013.Multiple Services per ReplicatorServicefrommysqlServicefromoraclefrommysqlfromoracleReplicatorReplicatorReplicatorAggregated
    • ©Continuent 2013starmaster-slave Heterogeneousfan-in slave all-mastersMySQLOracleOracleMySQL OracleOracleMySQL MySQL
    • ©Continuent 2012.EnablingHeterogeneousReplication
    • ©Continuent 2013.Steps to Heterogeneous Replication1. Prepare (translate) schema forthe slave DBMS2. Set up replication3. Provision initial data4. Continue real-time replicationddlscantungsten-installerParallel Apply
    • ©Continuent 2012.Replicating fromMySQL to Oracle
    • ©Continuent 2013.Use Case: Real-Time Sales/OrderFul!llmentReal-Time ReplicationWeb-based Sales Purchase Ordering3MOrderItems~1MXacts/Day
    • ©Continuent 2012.1.Translating schemafor the slave
    • ©Continuent 2012Introducing ddlscan13• Helps to prepare schema for heterogeneousreplication• Single command-line call• Part of Tungsten Replicator 2.1, GPL v2
    • ©Continuent 2012Translating Schema14• Goal - set up heterogeneous replicationReplication
    • ©Continuent 2012Translating Schema15• Beginning - how to convert tables?emptyTables
    • ©Continuent 2012Translating Schema16• Beginning - how to convert tables?emptyTables•Data types?•Column lengths?•Naming conventions?•Reserved words?
    • ©Continuent 2012Translating Schema17• ddlscan automates these conversionsemptyTablesddlscan
    • ©Continuent 2012Features18• Translates schema with replication in mind• Provides errors and warnings proactively• Can rename schema/tables/columns• Easy• Straightforward to extend (write newtemplates)
    • ©Continuent 2012Usage (MySQL to Oracle Example)19$ cd tungsten-replicator/bin$ ./ddlscan -db test -template ddl-mysql-oracle.vm -user tungsten -pass secret
    • ©Continuent 2012Translating Schema20• ddlscan looks into source schemaemptyTables
    • ©Continuent 2012Translating Schema21• ddlscan translates and renders DDL commandsemptyTablesoracle-ddl.sql
    • ©Continuent 2012Result of ddlscan22CREATE TABLE 3colors/* ERROR: table starts with a number, use rename option and afilter */(id NUMBER(10, 0),color VARCHAR2(1) /* ENUM(R,G,B) */,enabled NUMBER(1) /* BIT(1) - no constraints for 0 or 1 */,acolumnthatdoesntfittooracleat NUMBER(10, 0)/* WARN: truncated column name exceeding 30 characters(acolumnthatdoesntfittooracleatall) *//* ERROR: table must have a primary key! */);CREATE TABLE talks...
    • ©Continuent 2012Translating Schema23• You run resulting SQL file on OracleemptyTablesoracle-ddl.sql
    • ©Continuent 2012Translating Schema24• Tables are ready!Tables Translatedtables (empty)
    • ©Continuent 2013.Architecture25Tables$ ./bin/ddlscanJDBCmetadataLibrariesTungstenReplicatorTemplateengineApacheVelocitySchemaas JAVAobjectsMySQL,Oracle,PostgreSQL*.vmtemplateof choiceRenderedtext output
    • ©Continuent 2012.2. Set Up Replication
    • ©Continuent 2012Master Installation27tools/tungsten-installer -a --master-slave --service-name=frommysql --cluster-hosts=db1 --master-host=db1 --datasource-port=3306 --datasource-user=tungsten --datasource-password=secret --home-directory=/opt/continuent --mysql-use-bytes-for-string=false --mysql-enable-enumtostring=true --mysql-enable-settostring=true --start-and-report
    • ©Continuent 2012Slave Installation28tools/tungsten-installer -a --master-slave --service-name=frommysql --cluster-hosts=db2 --master-host=db1 --datasource-type=oracle --datasource-oracle-service=ORCL --datasource-user=tungsten_frommysql --datasource-password=secret --home-directory=/opt/continuent --svc-applier-filters=dropstatementdata --start-and-report
    • ©Continuent 2012.3. ProvisioningInitial Data
    • ©Continuent 2012Provisioning (Single Threaded)30• Restore a backup and let it replicateReplicationProvisioningSchemaTranslatedTablesProductionSchema
    • ©Continuent 2013.Parallel Apply on ReplicasExtractFilter(Shard)ApplyExtract/Filter/ApplyStage Stage StagePipelineMasterDBMSSlaveDBMSTransactionHistory LogParallelQueueExtract/Filter/ApplyExtract/Filter/ApplyExtract/Filter/ApplyProvisioningMulti-tenantsource
    • ©Continuent 2012Provisioning (Parallel Apply)32• Restore a backup and let it replicateProductionSchemaTranslatedtablesReplicationReplicationReplicationReplicationReplicationReplicationReplicationProvisioningSchema
    • ©Continuent 2012.4. Continue Real-TimeReplication
    • ©Continuent 2012Continue Replicating Real-Time Data34• Discard provisioning schema and leave singlechannelProductionSchemaProvisionedTablesReplicationReplicationReplicationReplicationReplicationReplicationReplicationProvisioningSchema
    • ©Continuent 2013.How Does Tungsten Do That?35MySQLBinlogbinlog_format=rowServicefrommysqlTungsten MasterReplicatorMySQLExtractorSpecial Filters•TransformENUM to string•Transform SETto stringServicefrommysqlTungsten SlaveReplicatorSpecial Filters•Drop DDL•Map names to upper case•Rename too long or reservedobjects•Optimize updates to removeunchanged columns•Ignore extra tablesOracleApplierMySQLExtractor
    • ©Continuent 2012.Replicating fromOracle to MySQL* newly open-sourced!
    • ©Continuent 2013.Use Case: Web Content PublishingReal-Time PublicationBackend Office Web-Based Catalog
    • ©Continuent 2013.How Does Tungsten Do That?38ServicefromoracleTungsten MasterReplicatorOracleCDCExtractorNo Special FiltersServicefromoracleTungsten SlaveReplicatorSpecial Filters•Map names to lower case•Ignore extra tables•Heartbeat table renamingdemoschemademo_pubschemaOracle CDC(Synchronous orAsync Hotlog)MySQLApplierOracleCDCExtractor
    • ©Continuent 2012.Replicating fromOracle to Oracle* newly open-sourced!
    • ©Continuent 2013.How Does Tungsten Do That?40ServicefromoracleTungsten MasterReplicatorOracleCDCExtractorNo Special FiltersServicefromoracleTungsten SlaveReplicatorSpecial FiltersNo Special Filtersdemoschemademo_pubschemaOracle CDC(Synchronous orAsync Hotlog)Tungsten SlaveReplicatorOracleApplierOracleCDCExtractor
    • ©Continuent 2013.More ReplicationPossibilities!
    • ©Continuent 2013.Oracle RAC and Continuent TungstenContinuent TungstenCluster for MySQLOracle 11G RACConnect your favorite clusters!
    • ©Continuent 2013.Oracle-Based Analytics and ReportingOracle Data WarehouseSharded MySQLOperational DataMySQL Fan-InAggregation Server
    • ©Continuent 2013.Oracle-Based Analytics and ReportingOracle Data WarehouseSharded MySQLOperational Data (HA)MySQL Fan-InAggregation Server
    • ©Continuent 2013.Oracle-Based Analytics and ReportingWeb-facing MySQL DBMSwith HA and scalingBackend Oracleapp BBackend Oracleapp AAggregate data
    • ©Continuent 2013.46More Replication CombinationsPostgreSQLMongoDBVerticaMySQL
    • ©Continuent 2013Support for Tungsten Replicator47• Community support is available on projectmailing list• Continuent o!ers implementation services andannual 24/7 support subscriptions• We do not o!er hourly or per-incident support(sorry!)• For high availability or disaster recovery needs,check out Continuent Tungsten clustering• Meanwhile, have fun and try it out!
    • ©Continuent 2013Tungsten University Sessions48• Introduction to Continuent Tungsten 2.0 (May16 and May 21)• Load a Vertica Data Warehouse with MySQLData (May 30 and June 4)Send any feedback to: tu@continuent.com
    • ©Continuent 2013. 49Continuent Web Page:http://www.continuent.comTungsten Replicator:http://code.google.com/p/tungsten-replicatorOur Blogs:http://"yingclusters.blogspot.comhttp://scale-out-blog.blogspot.comhttp://datacharmer.blogspot.comhttp://continuent-tungsten.blogspot.com560 S. Winchester Blvd., Suite 500San Jose, CA 95128Tel +1 (866) 998-3642Fax +1 (408) 668-1009e-mail: sales@continuent.com