SlideShare a Scribd company logo
1 of 30
Download to read offline
Robert Hodges and Linas Virbalas
                    Continuent, Inc.



© Continuent 2010
/  Introductions
  /  What is Tungsten?
  /  A Word About MySQL Replication
  /  Tungsten’s MySQL to PostgreSQL Replication
  /  Demo!
  /  Questions and Answers




© Continuent 2010
© Continuent 2010
/  Our Value: Make open source as robust as
     commercial databases
           •  Enterprise capability
           •  Cloud flexibility
           •  Open source cost
  /  Our Solution: Tungsten Database Clustering
  /  Our Company:
           •  Founded in Finland, HQ in USA
           •  US/European presence
           •  Deep experience in replication, distributed systems, and RDBMS




© Continuent 2010
What is Tungsten and How
                    Does It Work?



© Continuent 2010
What Is Tungsten?

  /  Tungsten creates data services using off-
     the-shelf open source databases to:

           1.  Ensure reliable, fast access to data
           2.  Simplify common management tasks
           3.  Enable disaster recovery

           …Without changing applications




© Continuent 2010
Apache/Mod_PHP                             Apache/Mod_PHP



                                 Connector                 Connector




                    Replicator               Replicator            Replicator
                    Manager                  Manager                   Manager



© Continuent 2010
Technology: Replication Pipelines




© Continuent 2010
So Why Are We Here?




© Continuent 2010
Why Would You Want to Do That?

  /  Feed MySQL web transactions to PostgreSQL
     applications
  /  Feed PG-based data warehouse and reporting
  /  Migrate MySQL data to PG with minimum downtime
  /  Scott McNealy says we should share (a PG-West joke)




© Continuent 2010
© Continuent 2010
Slave readable    Slave
                                                                 writeable
                     MySQL Statement Based            +             +
                          MySQL Row Based             +             +
                               MySQL Mixed            +             +
                    PostgreSQL WAL Shipping           -              -
                       PostgreSQL Streaming           +              -
                    Replication (Hot Standby)
/  Tungsten for MySQL
       •  You can both read and write to the DB operating in slave mode
       •  This, potentially, allows you to break consistency
       •  In some scenarios that is a plus
/  Tungsten for PostgreSQL
       •  In Streaming Replication you can read from slaves
       •  Still, you cannot write to them
       •  In effect, you cannot have different data/structure on a slave

© Continuent 2010
/  Statement Based Replication




  /  Row Based Replication




© Continuent 2010
Logical   Physical
                           MySQL Statement Based           x
                                 MySQL Row Based           x
                                      MySQL Mixed          x
                         PostgreSQL WAL Shipping                     x
                PostgreSQL Streaming Replication                     x
               Filters (data transformation) possible     +          -
                    Different data/structure on slave     +          -
                                             possible




© Continuent 2010
© Continuent 2010
© Continuent 2010
Tungsten     Tungsten
                    Replicator   Replicator




© Continuent 2010
Master         Slave
                    Replicator    Replicator

                    Transaction   Transaction
                    History Log   History Log
                      Filters       Filters
                     MySQL        PostgreSQL
                     Extractor      Applier




© Continuent 2010
/  Provisioning
  /  Data Type Differences
  /  Database vs. Schema
  /  Default (Implicitly Defined) Schema Selection
  /  SQL Dialect Differences
           •  Statement Replication vs. Row Replication
  /  Character Sets and Binary Data
  /  Old Versions of MySQL




© Continuent 2010
Provisioning

 /  Harder way: Dump data explicitly




 /  Easier way: Replicate a mysqldump backup



                          Replicator




© Continuent 2010
/  Note the type differences between MySQL and PG

                        MySQL                PostgreSQL
                    !   TINYINT              SMALLINT
                        SMALLINT             SMALLINT
                        INTEGER              INTEGER
                        BIGINT               BIGINT
                    !   CHAR(1)              CHAR(5) = {‘true’, ‘false’}
                        CHAR(x)              CHAR(x)
                        VARCHAR(x)           VARCHAR(x)
                        DATE                 DATE
                        TIMESTAMP            TIMESTAMP
                    !   TEXT (diff. sizes)   TEXT
                    !   BLOB                 BYTEA
                        …
© Continuent 2010
Database vs. Schema

  /  In MySQL these are the same:
    ! !CREATE DATABASE foo!
    ! !CREATE SCHEMA foo!
  /  In PostgreSQL these are very different:
                    CREATE DATABASE foo!
    ! !CREATE SCHEMA foo!
  /  Tungsten uses filters to rectify MySQL databases to
     PostgreSQL schemas




© Continuent 2010
/  MySQL: Trivial to use `USE`
  /  MySQL: Going without `USE` generates different
     events
                        MySQL Implicit            MySQL Explicit
                        CREATE SCHEMA s;          CREATE SCHEMA s;
                        USE s;
                    !   CREATE TABLE t (i int);   CREATE TABLE s.t (i int);
                    !   INSERT INTO t (1);        INSERT INTO s.t (1);


  /  PG: Extract the default schema from the event
  /  PG: Set it before applying
                    MySQL            PostgreSQL
                    USE s;       >   SET search_path TO s, "$user”;
© Continuent 2010
/  Differences between DDL and DML statement SQL
     dialects
  /  Row Replication resolves issues rising from
     differences in DML, but still leaves DDL to handle
  /  Tungsten Replicator Filters come to the rescue!
           •  Simple to develop Java or JavaScript extensions
           •  Event structure IN -> Filter -> Event structure OUT

          MySQL                          PostgreSQL
          CREATE TABLE complex (id       CREATE TABLE complex (id
          INTEGER AUTO_INCREMENT         SERIAL PRIMARY KEY, i INT);
          PRIMARY KEY, i INT);

          CREATE TABLE dt (i TINYINT);    CREATE TABLE dt (i SMALLINT);
          …


© Continuent 2010
/  Statement replication: MySQL syntax is “permissive”
      /  Embedded binary / alternate charsets
      /  Different charsets for different clients
  /  Row replication: database/table/column charsets
     may differ
  /  Answer: Stick with one character set throughout; use
     row replication to move binary data
          MySQL                           PostgreSQL
          INSERT INTO embedded_blob ARGH!!! (SQL statement fails)
          (key, data) VALUES (1, ‘?0^Es
          0^0’’)
          create table xlate(id int, d1   ARGH!!! (no way to translate to
          varchar(25) character set       common charset)
          latin1, d2 varchar(25)
          character set utf8);
© Continuent 2010
MySQL Versions

  /  Problem: Data stored on hard-to-replicate MySQL
     versions or configurations
           •  Row replication not enabled (5.1)
           •  No row replication support (5.0, 4.1)
           •  Tungsten cannot read binlog (4.1)
  /  Answer: MySQL blackhole replication
           •  (Blackhole = no store, just a binlog)
           •  Caveat: Check MySQL docs carefully


                                                      Replicator




© Continuent 2010
What’s Left?

  /  We have covered the basics but there is more…
  /  Data transformation (more Javascript filtering)
  /  Making it go *really* fast (think parallel replication)




© Continuent 2010
© Continuent 2010
© Continuent 2010
HQ and Americas                      EMEA and APAC
   560 S. Winchester Blvd., Suite 500   Lars Sonckin kaari 16
   San Jose, CA 95128                   02600 Espoo, Finland
   Tel (866) 998-3642                   Tel +358 50 517 9059
   Fax (408) 668-1009                   Fax +358 9 863 0060

   e-mail: sales@continuent.com


                        Continuent Web Site:
                     http://www.continuent.com



© Continuent 2010

More Related Content

What's hot

Mongo DBを半年運用してみた
Mongo DBを半年運用してみたMongo DBを半年運用してみた
Mongo DBを半年運用してみたMasakazu Matsushita
 
Cassandra 1.1
Cassandra 1.1Cassandra 1.1
Cassandra 1.1jbellis
 
DPC2007 MySQL Stored Routines for PHP Developers (Roland Bouman)
DPC2007 MySQL Stored Routines for PHP Developers (Roland Bouman)DPC2007 MySQL Stored Routines for PHP Developers (Roland Bouman)
DPC2007 MySQL Stored Routines for PHP Developers (Roland Bouman)dpc
 
DPC2007 PDO (Lukas Kahwe Smith)
DPC2007 PDO (Lukas Kahwe Smith)DPC2007 PDO (Lukas Kahwe Smith)
DPC2007 PDO (Lukas Kahwe Smith)dpc
 
DPC2007 PHP And Oracle (Kuassi Mensah)
DPC2007 PHP And Oracle (Kuassi Mensah)DPC2007 PHP And Oracle (Kuassi Mensah)
DPC2007 PHP And Oracle (Kuassi Mensah)dpc
 
ConFoo MySQL Replication Evolution : From Simple to Group Replication
ConFoo  MySQL Replication Evolution : From Simple to Group ReplicationConFoo  MySQL Replication Evolution : From Simple to Group Replication
ConFoo MySQL Replication Evolution : From Simple to Group ReplicationDave Stokes
 
Instalación y Configuración : Bases de datos fuera del asistente
Instalación y Configuración : Bases de datos fuera del asistente Instalación y Configuración : Bases de datos fuera del asistente
Instalación y Configuración : Bases de datos fuera del asistente SolidQ
 
The Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication TutorialThe Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication TutorialJean-François Gagné
 
Fosdem2014 MariaDB CONNECT Storage Engine
Fosdem2014 MariaDB CONNECT Storage EngineFosdem2014 MariaDB CONNECT Storage Engine
Fosdem2014 MariaDB CONNECT Storage EngineSerge Frezefond
 
SQL Server knowledge-session (SQL Server vs Oracle, and performance)
SQL Server knowledge-session (SQL Server vs Oracle, and performance)SQL Server knowledge-session (SQL Server vs Oracle, and performance)
SQL Server knowledge-session (SQL Server vs Oracle, and performance)Pierre van der Ven
 
Upgrading_and_Migrating_to_Exchange_Server_2007_and_Windows_2008
Upgrading_and_Migrating_to_Exchange_Server_2007_and_Windows_2008Upgrading_and_Migrating_to_Exchange_Server_2007_and_Windows_2008
Upgrading_and_Migrating_to_Exchange_Server_2007_and_Windows_2008Balaji Ravi
 
PNWPHP -- What are Databases so &#%-ing Difficult
PNWPHP -- What are Databases so &#%-ing DifficultPNWPHP -- What are Databases so &#%-ing Difficult
PNWPHP -- What are Databases so &#%-ing DifficultDave Stokes
 

What's hot (13)

Mongo DBを半年運用してみた
Mongo DBを半年運用してみたMongo DBを半年運用してみた
Mongo DBを半年運用してみた
 
Cassandra 1.1
Cassandra 1.1Cassandra 1.1
Cassandra 1.1
 
DPC2007 MySQL Stored Routines for PHP Developers (Roland Bouman)
DPC2007 MySQL Stored Routines for PHP Developers (Roland Bouman)DPC2007 MySQL Stored Routines for PHP Developers (Roland Bouman)
DPC2007 MySQL Stored Routines for PHP Developers (Roland Bouman)
 
DPC2007 PDO (Lukas Kahwe Smith)
DPC2007 PDO (Lukas Kahwe Smith)DPC2007 PDO (Lukas Kahwe Smith)
DPC2007 PDO (Lukas Kahwe Smith)
 
DPC2007 PHP And Oracle (Kuassi Mensah)
DPC2007 PHP And Oracle (Kuassi Mensah)DPC2007 PHP And Oracle (Kuassi Mensah)
DPC2007 PHP And Oracle (Kuassi Mensah)
 
ConFoo MySQL Replication Evolution : From Simple to Group Replication
ConFoo  MySQL Replication Evolution : From Simple to Group ReplicationConFoo  MySQL Replication Evolution : From Simple to Group Replication
ConFoo MySQL Replication Evolution : From Simple to Group Replication
 
Instalación y Configuración : Bases de datos fuera del asistente
Instalación y Configuración : Bases de datos fuera del asistente Instalación y Configuración : Bases de datos fuera del asistente
Instalación y Configuración : Bases de datos fuera del asistente
 
The Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication TutorialThe Full MySQL and MariaDB Parallel Replication Tutorial
The Full MySQL and MariaDB Parallel Replication Tutorial
 
Postgresql
PostgresqlPostgresql
Postgresql
 
Fosdem2014 MariaDB CONNECT Storage Engine
Fosdem2014 MariaDB CONNECT Storage EngineFosdem2014 MariaDB CONNECT Storage Engine
Fosdem2014 MariaDB CONNECT Storage Engine
 
SQL Server knowledge-session (SQL Server vs Oracle, and performance)
SQL Server knowledge-session (SQL Server vs Oracle, and performance)SQL Server knowledge-session (SQL Server vs Oracle, and performance)
SQL Server knowledge-session (SQL Server vs Oracle, and performance)
 
Upgrading_and_Migrating_to_Exchange_Server_2007_and_Windows_2008
Upgrading_and_Migrating_to_Exchange_Server_2007_and_Windows_2008Upgrading_and_Migrating_to_Exchange_Server_2007_and_Windows_2008
Upgrading_and_Migrating_to_Exchange_Server_2007_and_Windows_2008
 
PNWPHP -- What are Databases so &#%-ing Difficult
PNWPHP -- What are Databases so &#%-ing DifficultPNWPHP -- What are Databases so &#%-ing Difficult
PNWPHP -- What are Databases so &#%-ing Difficult
 

Viewers also liked

Deployment topologies for high availability (ha)
Deployment topologies for high availability (ha)Deployment topologies for high availability (ha)
Deployment topologies for high availability (ha)Deepak Mane
 
MySQL real-time replication configuration
MySQL real-time replication configurationMySQL real-time replication configuration
MySQL real-time replication configurationkelechi Anyanwu
 
Modern MySQL Monitoring and Dashboards.
Modern MySQL Monitoring and Dashboards.Modern MySQL Monitoring and Dashboards.
Modern MySQL Monitoring and Dashboards.Mydbops
 
Supriya Shailaja Latest Gallery
 Supriya Shailaja Latest Gallery Supriya Shailaja Latest Gallery
Supriya Shailaja Latest Gallerytelugustop.com
 
MySQL High Availability Sprint: Launch the Pacemaker
MySQL High Availability Sprint: Launch the PacemakerMySQL High Availability Sprint: Launch the Pacemaker
MySQL High Availability Sprint: Launch the Pacemakerhastexo
 
MySQL Database Replication - A Guide by RapidValue Solutions
MySQL Database Replication - A Guide by RapidValue SolutionsMySQL Database Replication - A Guide by RapidValue Solutions
MySQL Database Replication - A Guide by RapidValue SolutionsRapidValue
 
Open stack ha design & deployment kilo
Open stack ha design & deployment   kiloOpen stack ha design & deployment   kilo
Open stack ha design & deployment kiloSteven Li
 
Galera Cluster - Node Recovery - Webinar slides
Galera Cluster - Node Recovery - Webinar slidesGalera Cluster - Node Recovery - Webinar slides
Galera Cluster - Node Recovery - Webinar slidesSeveralnines
 
MySQL HA with PaceMaker
MySQL HA with  PaceMakerMySQL HA with  PaceMaker
MySQL HA with PaceMakerKris Buytaert
 
MySQL with DRBD/Pacemaker/Corosync on Linux
 MySQL with DRBD/Pacemaker/Corosync on Linux MySQL with DRBD/Pacemaker/Corosync on Linux
MySQL with DRBD/Pacemaker/Corosync on LinuxPawan Kumar
 
Pacemaker: OpenStack's Pid 1
Pacemaker: OpenStack's Pid 1Pacemaker: OpenStack's Pid 1
Pacemaker: OpenStack's Pid 1David Vossel
 
High Availability (HA) Explained
High Availability (HA) ExplainedHigh Availability (HA) Explained
High Availability (HA) ExplainedMaciej Lasyk
 
MySQL Replication Evolution -- Confoo Montreal 2017
MySQL Replication Evolution -- Confoo Montreal 2017MySQL Replication Evolution -- Confoo Montreal 2017
MySQL Replication Evolution -- Confoo Montreal 2017Dave Stokes
 
AWS re:Invent 2016: Get the Most from AWS KMS: Architecting Applications for ...
AWS re:Invent 2016: Get the Most from AWS KMS: Architecting Applications for ...AWS re:Invent 2016: Get the Most from AWS KMS: Architecting Applications for ...
AWS re:Invent 2016: Get the Most from AWS KMS: Architecting Applications for ...Amazon Web Services
 
High Availability for OpenStack
High Availability for OpenStackHigh Availability for OpenStack
High Availability for OpenStackKamesh Pemmaraju
 
Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison
Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison
Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison Severalnines
 
Galera cluster for MySQL - Introduction Slides
Galera cluster for MySQL - Introduction SlidesGalera cluster for MySQL - Introduction Slides
Galera cluster for MySQL - Introduction SlidesSeveralnines
 
High Availability in 37 Easy Steps
High Availability in 37 Easy StepsHigh Availability in 37 Easy Steps
High Availability in 37 Easy StepsTim Serong
 

Viewers also liked (20)

Project: OpenStack, #OSATH
Project: OpenStack, #OSATH Project: OpenStack, #OSATH
Project: OpenStack, #OSATH
 
Deployment topologies for high availability (ha)
Deployment topologies for high availability (ha)Deployment topologies for high availability (ha)
Deployment topologies for high availability (ha)
 
MySQL real-time replication configuration
MySQL real-time replication configurationMySQL real-time replication configuration
MySQL real-time replication configuration
 
Modern MySQL Monitoring and Dashboards.
Modern MySQL Monitoring and Dashboards.Modern MySQL Monitoring and Dashboards.
Modern MySQL Monitoring and Dashboards.
 
Supriya Shailaja Latest Gallery
 Supriya Shailaja Latest Gallery Supriya Shailaja Latest Gallery
Supriya Shailaja Latest Gallery
 
MySQL High Availability Sprint: Launch the Pacemaker
MySQL High Availability Sprint: Launch the PacemakerMySQL High Availability Sprint: Launch the Pacemaker
MySQL High Availability Sprint: Launch the Pacemaker
 
MySQL Database Replication - A Guide by RapidValue Solutions
MySQL Database Replication - A Guide by RapidValue SolutionsMySQL Database Replication - A Guide by RapidValue Solutions
MySQL Database Replication - A Guide by RapidValue Solutions
 
Open stack ha design & deployment kilo
Open stack ha design & deployment   kiloOpen stack ha design & deployment   kilo
Open stack ha design & deployment kilo
 
Galera Cluster - Node Recovery - Webinar slides
Galera Cluster - Node Recovery - Webinar slidesGalera Cluster - Node Recovery - Webinar slides
Galera Cluster - Node Recovery - Webinar slides
 
MySQL HA with PaceMaker
MySQL HA with  PaceMakerMySQL HA with  PaceMaker
MySQL HA with PaceMaker
 
MySQL with DRBD/Pacemaker/Corosync on Linux
 MySQL with DRBD/Pacemaker/Corosync on Linux MySQL with DRBD/Pacemaker/Corosync on Linux
MySQL with DRBD/Pacemaker/Corosync on Linux
 
Pacemaker: OpenStack's Pid 1
Pacemaker: OpenStack's Pid 1Pacemaker: OpenStack's Pid 1
Pacemaker: OpenStack's Pid 1
 
High Availability (HA) Explained
High Availability (HA) ExplainedHigh Availability (HA) Explained
High Availability (HA) Explained
 
OpenStack HA
OpenStack HAOpenStack HA
OpenStack HA
 
MySQL Replication Evolution -- Confoo Montreal 2017
MySQL Replication Evolution -- Confoo Montreal 2017MySQL Replication Evolution -- Confoo Montreal 2017
MySQL Replication Evolution -- Confoo Montreal 2017
 
AWS re:Invent 2016: Get the Most from AWS KMS: Architecting Applications for ...
AWS re:Invent 2016: Get the Most from AWS KMS: Architecting Applications for ...AWS re:Invent 2016: Get the Most from AWS KMS: Architecting Applications for ...
AWS re:Invent 2016: Get the Most from AWS KMS: Architecting Applications for ...
 
High Availability for OpenStack
High Availability for OpenStackHigh Availability for OpenStack
High Availability for OpenStack
 
Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison
Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison
Galera Cluster for MySQL vs MySQL (NDB) Cluster: A High Level Comparison
 
Galera cluster for MySQL - Introduction Slides
Galera cluster for MySQL - Introduction SlidesGalera cluster for MySQL - Introduction Slides
Galera cluster for MySQL - Introduction Slides
 
High Availability in 37 Easy Steps
High Availability in 37 Easy StepsHigh Availability in 37 Easy Steps
High Availability in 37 Easy Steps
 

Similar to Liberating Your Data From MySQL: Cross-Database Replication to the Rescue!

Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...Continuent
 
Implementing the Future of PostgreSQL Clustering with Tungsten
Implementing the Future of PostgreSQL Clustering with TungstenImplementing the Future of PostgreSQL Clustering with Tungsten
Implementing the Future of PostgreSQL Clustering with TungstenCommand Prompt., Inc
 
Database Performance With Proxy Architectures
Database  Performance With  Proxy  ArchitecturesDatabase  Performance With  Proxy  Architectures
Database Performance With Proxy ArchitecturesPerconaPerformance
 
Built-in Replication in PostgreSQL
Built-in Replication in PostgreSQLBuilt-in Replication in PostgreSQL
Built-in Replication in PostgreSQLMasao Fujii
 
Intro to Azure SQL database
Intro to Azure SQL databaseIntro to Azure SQL database
Intro to Azure SQL databaseSteve Knutson
 
Replicate Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
Replicate Oracle to Oracle, Oracle to MySQL, and Oracle to AnalyticsReplicate Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
Replicate Oracle to Oracle, Oracle to MySQL, and Oracle to AnalyticsLinas Virbalas
 
Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0
Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0
Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0Continuent
 
Replicate from Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
Replicate from Oracle to Oracle, Oracle to MySQL, and Oracle to AnalyticsReplicate from Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
Replicate from Oracle to Oracle, Oracle to MySQL, and Oracle to AnalyticsContinuent
 
MySQL's NoSQL -- Texas Linuxfest August 22nd 2015
MySQL's NoSQL  -- Texas Linuxfest August 22nd 2015MySQL's NoSQL  -- Texas Linuxfest August 22nd 2015
MySQL's NoSQL -- Texas Linuxfest August 22nd 2015Dave Stokes
 
001 hbase introduction
001 hbase introduction001 hbase introduction
001 hbase introductionScott Miao
 
Vote NO for MySQL
Vote NO for MySQLVote NO for MySQL
Vote NO for MySQLUlf Wendel
 
MySQL overview
MySQL overviewMySQL overview
MySQL overviewMarco Tusa
 
Five Lessons in Distributed Databases
Five Lessons  in Distributed DatabasesFive Lessons  in Distributed Databases
Five Lessons in Distributed Databasesjbellis
 
IBM Spark Meetup - RDD & Spark Basics
IBM Spark Meetup - RDD & Spark BasicsIBM Spark Meetup - RDD & Spark Basics
IBM Spark Meetup - RDD & Spark BasicsSatya Narayan
 
"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007
"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007
"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007eLiberatica
 
Drizzle Keynote from O'Reilly's MySQL's Conference
Drizzle Keynote from O'Reilly's MySQL's ConferenceDrizzle Keynote from O'Reilly's MySQL's Conference
Drizzle Keynote from O'Reilly's MySQL's ConferenceBrian Aker
 
Ohio Linux Fest -- MySQL's NoSQL
Ohio Linux Fest -- MySQL's NoSQLOhio Linux Fest -- MySQL's NoSQL
Ohio Linux Fest -- MySQL's NoSQLDave Stokes
 

Similar to Liberating Your Data From MySQL: Cross-Database Replication to the Rescue! (20)

Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
Tungsten Use Case: How Gittigidiyor (a subsidiary of eBay) Replicates Data In...
 
Implementing the Future of PostgreSQL Clustering with Tungsten
Implementing the Future of PostgreSQL Clustering with TungstenImplementing the Future of PostgreSQL Clustering with Tungsten
Implementing the Future of PostgreSQL Clustering with Tungsten
 
Database Performance With Proxy Architectures
Database  Performance With  Proxy  ArchitecturesDatabase  Performance With  Proxy  Architectures
Database Performance With Proxy Architectures
 
Built-in Replication in PostgreSQL
Built-in Replication in PostgreSQLBuilt-in Replication in PostgreSQL
Built-in Replication in PostgreSQL
 
MySQL8.0 in COSCUP2017
MySQL8.0 in COSCUP2017MySQL8.0 in COSCUP2017
MySQL8.0 in COSCUP2017
 
Intro to Azure SQL database
Intro to Azure SQL databaseIntro to Azure SQL database
Intro to Azure SQL database
 
PostgreSQL replication
PostgreSQL replicationPostgreSQL replication
PostgreSQL replication
 
Replicate Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
Replicate Oracle to Oracle, Oracle to MySQL, and Oracle to AnalyticsReplicate Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
Replicate Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
 
Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0
Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0
Real-Time Data Loading from MySQL to Hadoop with New Tungsten Replicator 3.0
 
Replicate from Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
Replicate from Oracle to Oracle, Oracle to MySQL, and Oracle to AnalyticsReplicate from Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
Replicate from Oracle to Oracle, Oracle to MySQL, and Oracle to Analytics
 
MySQL's NoSQL -- Texas Linuxfest August 22nd 2015
MySQL's NoSQL  -- Texas Linuxfest August 22nd 2015MySQL's NoSQL  -- Texas Linuxfest August 22nd 2015
MySQL's NoSQL -- Texas Linuxfest August 22nd 2015
 
001 hbase introduction
001 hbase introduction001 hbase introduction
001 hbase introduction
 
Vote NO for MySQL
Vote NO for MySQLVote NO for MySQL
Vote NO for MySQL
 
MySQL overview
MySQL overviewMySQL overview
MySQL overview
 
Five Lessons in Distributed Databases
Five Lessons  in Distributed DatabasesFive Lessons  in Distributed Databases
Five Lessons in Distributed Databases
 
IBM Spark Meetup - RDD & Spark Basics
IBM Spark Meetup - RDD & Spark BasicsIBM Spark Meetup - RDD & Spark Basics
IBM Spark Meetup - RDD & Spark Basics
 
"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007
"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007
"Advanced MySQL 5 Tuning" by Michael Monty Widenius @ eLiberatica 2007
 
Drizzle Keynote from O'Reilly's MySQL's Conference
Drizzle Keynote from O'Reilly's MySQL's ConferenceDrizzle Keynote from O'Reilly's MySQL's Conference
Drizzle Keynote from O'Reilly's MySQL's Conference
 
Ohio Linux Fest -- MySQL's NoSQL
Ohio Linux Fest -- MySQL's NoSQLOhio Linux Fest -- MySQL's NoSQL
Ohio Linux Fest -- MySQL's NoSQL
 
NoSQL
NoSQLNoSQL
NoSQL
 

Recently uploaded

2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessWSO2
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Karmanjay Verma
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialJoão Esperancinha
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Nikki Chapple
 

Recently uploaded (20)

2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with Platformless
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#
 
Kuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorialKuma Meshes Part I - The basics - A tutorial
Kuma Meshes Part I - The basics - A tutorial
 
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
Microsoft 365 Copilot: How to boost your productivity with AI – Part two: Dat...
 

Liberating Your Data From MySQL: Cross-Database Replication to the Rescue!

  • 1. Robert Hodges and Linas Virbalas Continuent, Inc. © Continuent 2010
  • 2. /  Introductions /  What is Tungsten? /  A Word About MySQL Replication /  Tungsten’s MySQL to PostgreSQL Replication /  Demo! /  Questions and Answers © Continuent 2010
  • 4. /  Our Value: Make open source as robust as commercial databases •  Enterprise capability •  Cloud flexibility •  Open source cost /  Our Solution: Tungsten Database Clustering /  Our Company: •  Founded in Finland, HQ in USA •  US/European presence •  Deep experience in replication, distributed systems, and RDBMS © Continuent 2010
  • 5. What is Tungsten and How Does It Work? © Continuent 2010
  • 6. What Is Tungsten? /  Tungsten creates data services using off- the-shelf open source databases to: 1.  Ensure reliable, fast access to data 2.  Simplify common management tasks 3.  Enable disaster recovery …Without changing applications © Continuent 2010
  • 7. Apache/Mod_PHP Apache/Mod_PHP Connector Connector Replicator Replicator Replicator Manager Manager Manager © Continuent 2010
  • 9. So Why Are We Here? © Continuent 2010
  • 10. Why Would You Want to Do That? /  Feed MySQL web transactions to PostgreSQL applications /  Feed PG-based data warehouse and reporting /  Migrate MySQL data to PG with minimum downtime /  Scott McNealy says we should share (a PG-West joke) © Continuent 2010
  • 12. Slave readable Slave writeable MySQL Statement Based + + MySQL Row Based + + MySQL Mixed + + PostgreSQL WAL Shipping - - PostgreSQL Streaming + - Replication (Hot Standby) /  Tungsten for MySQL •  You can both read and write to the DB operating in slave mode •  This, potentially, allows you to break consistency •  In some scenarios that is a plus /  Tungsten for PostgreSQL •  In Streaming Replication you can read from slaves •  Still, you cannot write to them •  In effect, you cannot have different data/structure on a slave © Continuent 2010
  • 13. /  Statement Based Replication /  Row Based Replication © Continuent 2010
  • 14. Logical Physical MySQL Statement Based x MySQL Row Based x MySQL Mixed x PostgreSQL WAL Shipping x PostgreSQL Streaming Replication x Filters (data transformation) possible + - Different data/structure on slave + - possible © Continuent 2010
  • 17. Tungsten Tungsten Replicator Replicator © Continuent 2010
  • 18. Master Slave Replicator Replicator Transaction Transaction History Log History Log Filters Filters MySQL PostgreSQL Extractor Applier © Continuent 2010
  • 19. /  Provisioning /  Data Type Differences /  Database vs. Schema /  Default (Implicitly Defined) Schema Selection /  SQL Dialect Differences •  Statement Replication vs. Row Replication /  Character Sets and Binary Data /  Old Versions of MySQL © Continuent 2010
  • 20. Provisioning /  Harder way: Dump data explicitly /  Easier way: Replicate a mysqldump backup Replicator © Continuent 2010
  • 21. /  Note the type differences between MySQL and PG MySQL PostgreSQL ! TINYINT SMALLINT SMALLINT SMALLINT INTEGER INTEGER BIGINT BIGINT ! CHAR(1) CHAR(5) = {‘true’, ‘false’} CHAR(x) CHAR(x) VARCHAR(x) VARCHAR(x) DATE DATE TIMESTAMP TIMESTAMP ! TEXT (diff. sizes) TEXT ! BLOB BYTEA … © Continuent 2010
  • 22. Database vs. Schema /  In MySQL these are the same: ! !CREATE DATABASE foo! ! !CREATE SCHEMA foo! /  In PostgreSQL these are very different: CREATE DATABASE foo! ! !CREATE SCHEMA foo! /  Tungsten uses filters to rectify MySQL databases to PostgreSQL schemas © Continuent 2010
  • 23. /  MySQL: Trivial to use `USE` /  MySQL: Going without `USE` generates different events MySQL Implicit MySQL Explicit CREATE SCHEMA s; CREATE SCHEMA s; USE s; ! CREATE TABLE t (i int); CREATE TABLE s.t (i int); ! INSERT INTO t (1); INSERT INTO s.t (1); /  PG: Extract the default schema from the event /  PG: Set it before applying MySQL PostgreSQL USE s; > SET search_path TO s, "$user”; © Continuent 2010
  • 24. /  Differences between DDL and DML statement SQL dialects /  Row Replication resolves issues rising from differences in DML, but still leaves DDL to handle /  Tungsten Replicator Filters come to the rescue! •  Simple to develop Java or JavaScript extensions •  Event structure IN -> Filter -> Event structure OUT MySQL PostgreSQL CREATE TABLE complex (id CREATE TABLE complex (id INTEGER AUTO_INCREMENT SERIAL PRIMARY KEY, i INT); PRIMARY KEY, i INT); CREATE TABLE dt (i TINYINT); CREATE TABLE dt (i SMALLINT); … © Continuent 2010
  • 25. /  Statement replication: MySQL syntax is “permissive” /  Embedded binary / alternate charsets /  Different charsets for different clients /  Row replication: database/table/column charsets may differ /  Answer: Stick with one character set throughout; use row replication to move binary data MySQL PostgreSQL INSERT INTO embedded_blob ARGH!!! (SQL statement fails) (key, data) VALUES (1, ‘?0^Es 0^0’’) create table xlate(id int, d1 ARGH!!! (no way to translate to varchar(25) character set common charset) latin1, d2 varchar(25) character set utf8); © Continuent 2010
  • 26. MySQL Versions /  Problem: Data stored on hard-to-replicate MySQL versions or configurations •  Row replication not enabled (5.1) •  No row replication support (5.0, 4.1) •  Tungsten cannot read binlog (4.1) /  Answer: MySQL blackhole replication •  (Blackhole = no store, just a binlog) •  Caveat: Check MySQL docs carefully Replicator © Continuent 2010
  • 27. What’s Left? /  We have covered the basics but there is more… /  Data transformation (more Javascript filtering) /  Making it go *really* fast (think parallel replication) © Continuent 2010
  • 30. HQ and Americas EMEA and APAC 560 S. Winchester Blvd., Suite 500 Lars Sonckin kaari 16 San Jose, CA 95128 02600 Espoo, Finland Tel (866) 998-3642 Tel +358 50 517 9059 Fax (408) 668-1009 Fax +358 9 863 0060 e-mail: sales@continuent.com Continuent Web Site: http://www.continuent.com © Continuent 2010