A MySQL Odyssey - A Blackhole Crossover
Upcoming SlideShare
Loading in...5
×
 

A MySQL Odyssey - A Blackhole Crossover

on

  • 524 views

The OpenSUSE presentation I shared in Thessaloniki on using MySQL Replication and the Blackhole storage engine.

The OpenSUSE presentation I shared in Thessaloniki on using MySQL Replication and the Blackhole storage engine.

Statistics

Views

Total Views
524
Views on SlideShare
524
Embed Views
0

Actions

Likes
0
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

A MySQL Odyssey - A Blackhole Crossover A MySQL Odyssey - A Blackhole Crossover Presentation Transcript

  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.1
  • A MySQL Odyssey - a Blackhole crossover Keith Hollman MySQL Principal Sales Consultant EMEA
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remain at the sole discretion of Oracle. Safe Harbor Statement
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4 Agenda MySQL Replication: the crossover. From Dev… …thru’ the BLACK HOLE… Knowledge is key Non- implosion non- destructive …into Pre / Production Windows ¿Why? Oracle Linux The Future Nebulae
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5 ¿Why?  Sharing how to automate a cross-platform data transfer with MySQL, applying data filters at the same time.  Small, quick and easy to setup, minimizing performance & resource overheads.  Maneuvering the High Availability door to other possibilities, such as Security control, Auditing, horizontal & vertical scalability, read-write segregations. What’s this all about:
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6 ¿Why?  A Developer using MySQL on Win (Testing).  Need to export & import data / schema definition into DBA’s / Sys.Admin’s Linux environment (Pre-Prod, UAT, Production).  Certain changes are required and it has to be easy, and unobtrusive.  And data sensitive columns? Wouldn’t it be nice if I could have it all transferred directly, automatically and not have to worry about platform differences? Just “a” Scenario
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7 ¿Why? The Setup “Dev” pc Win.7 Pro SP1 Intel i5 2.67GHz 8Gb RAM 64bit CREATE TABLE `crossover` …… ENGINE=InnoDB; “BLACKHOLE” env Oracle Linux 6.3 UEK 2.6.39 VM 1xCPU 512Mb RAM 32bit 5Gb disk ALTER TABLE `crossover` ENGINE=BLACKHOLE; “Prod” env SLES 11 SP2 3.0.74 VM 1xCPU 1Gb RAM 10Gb disk ALTER TABLE `crossover` ENGINE=InnoDB; MySQL 5.6.11 EE Replication Setup using GTID’s: Master (Win) -> Slave / Master (OL) -> Slave (SLES)
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8 ¿Why?  BLACKHOLE storage engine: “.. accepts data but throws it away and does not store it. Retrievals always return an empty result” – Row Based Replication best (vs Statement, autoinc & PK’s) – UPDATE’s & DELETE’s triggers aren’t activated: will break replication. – All operations on BLACKHOLE will be replicated to SLES slave.1  Others: – Replicating with lower_case_table_names=1. – replicate-ignore-db, replicate-do-table, replicate-wild-*. – Double check FW (iptables –F for easiness). MySQL BLACKHOLE & Other Awareness 1 Should we be tempted to temporarily ‘fix’ replication by changing SE and back.
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9 The BLACKHOLE Crossover
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10 From Dev… thru’ the BLACKHOLE…  Logical Dump of the master database “sakila” from Win to OL: [mysql@ol63uek01 ~]$ mysqldump -uroot -poracle -h 192.168.56.101 -P3307 -B sakila --master-data=2 | mysql -uroot ol63 10:40 sakila> show databases;  Set Replication up…  … and make sure we only replicate “sakila”: [mysql@ol63uek01 ~]$ vi /etc/my.cnf : [mysqld] replicate-do-db =sakila  Check it’s working: ol63 11:00 sakila> show slave statusG Win 11:00 sakila> call BlackholeInsert (1000); ..& look for the “*–relay.bin*” logs. [ From Win to Oracle Linux ]
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11 Win 11:01 sakila> select count(*) from crossover;  Stepping into the BLACKHOLE: ol63 11:02 sakila> select count(*) from crossover; ol63 11:02 sakila> show create table crossover; ol63 11:02 sakila> alter table crossover engine=blackhole; Query OK, 1001 rows affected (0.16 sec) Records: 1001 Duplicates: 0 Warnings: 0  Confirming: ol63 11:02 sakila> show create table crossover; ol63 11:03 sakila> select count(*) from crossover; From Dev… thru’ the BLACKHOLE… Continued… +----------+ | count(*) | +----------+ | 1001 | +----------+ 1 row in set (0.00 sec) +----------+ | count(*) | +----------+ | 1001 | +----------+ 1 row in set (0.00 sec) | crossover | CREATE TABLE `crossover` ( `ID` int(7) NOT NULL AUTO_INCREMENT, `Name` char(20) NOT NULL DEFAULT '', PRIMARY KEY (`ID`) ) ENGINE=InnoDB AUTO_INCREMENT=1002 DEFAULT CHARSET=utf8 | 1 row in set (0.00 sec) | crossover | CREATE TABLE `crossover` ( `ID` int(7) NOT NULL AUTO_INCREMENT, `Name` char(20) NOT NULL DEFAULT '', PRIMARY KEY (`ID`) ) ENGINE=BLACKHOLE DEFAULT CHARSET=utf8 | 1 row in set (0.00 sec) +----------+ | count(*) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec)
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12 … thru’ the BLACKHOLE… into Prod.  Logical dump of ol63uek BLACKHOLE table to SLES Production env: [mysql@suse01 ~]$ mysqldump -uroot -p -h 192.168.56.102 -B sakila --master-data=2 | mysql -uroot suse01 11:11 sakila> show databases;  Set Replication up… again (ol63 is our Master now) & check it works: suse01 11:12 sakila> show slave statusG  Coming out of the BLACKHOLE, into InnoDB@SLES: suse01 11:12 sakila> alter table crossover engine=InnoDB;  Pull data from Win into SLES (non-blackhole tables): [mysql@suse01 ~]$ mysqldump -uroot –p –h 192.168.56.101 -P3307 --no-create-info --no-create-db –B sakila | mysql -uroot
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13 Non-Implosion and Non- Destructive. http://ourwikiworld.blogspot.com.es/2011/04/amazing-physics-stunning-building.html
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14 Non-implosion & non-destructive  Truncate table crossover;  Bulk INSERT’s.  DDL modifications.  Finer Replication capabilities.  Schema manoeuvrability.  Performance analysis.  Located on same Master server.  Isolated env’s & modular. ! UPDATE & DELETE triggers. ! Added Operational commitments. ! Data models, PK & Autoinc cols. ! IO monitoring / impact evaluation removed. ! Infrastructure becomes inter- dependent (replication).
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15 Non-implosion & non-destructive cont. Win ol63uek SLES OBJECT_NAME crossover Diff vs orig. crossover diff crossover diff COUNT_STAR 10,002 -1 10,001 0 40,007 -20,004 SUM_TIMER_WAIT 231,905,415,096 -26,335,040 7,504,322,436 0 3,783,883,427,646 -9,161,505,807 MIN_TIMER_WAIT 11,728,944 0 22,464 0 24,384 0 AVG_TIMER_WAIT 23,185,664 -376 750,087 0 94,580,202 94,127,574 MAX_TIMER_WAIT 50,530,335,064 0 426,302,487 0 428,082,829,761 0 COUNT_READ 1 -1 0 0 30,006 -20,004 SUM_TIMER_READ 26,335,040 -26,335,040 0 0 15,641,005,167 -9,161,505,807 MIN_TIMER_READ 26,335,040 -26,335,040 0 0 24,384 0 AVG_TIMER_READ 26,335,040 -26,335,040 0 0 521,208 126,492 MAX_TIMER_READ 26,335,040 -26,335,040 0 0 374,708,547 0 COUNT_WRITE 10,001 0 10,001 0 10,001 0 SUM_TIMER_WRITE 231,879,080,056 0 7,504,322,436 0 3,768,242,422,479 0 MIN_TIMER_WRITE 11,728,944 0 22,464 0 24,384 0 AVG_TIMER_WRITE 23,185,288 0 750,087 0 376,786,521 0 MAX_TIMER_WRITE 50,530,335,064 0 426,302,487 0 428,082,829,761 0 COUNT_FETCH 1 -1 0 0 30,006 -20,004 SUM_TIMER_FETCH 26,335,040 -26,335,040 0 0 15,641,005,167 -9,161,505,807 MIN_TIMER_FETCH 26,335,040 -26,335,040 0 0 24,384 0 AVG_TIMER_FETCH 26,335,040 -26,335,040 0 0 521,208 126,492 MAX_TIMER_FETCH 26,335,040 -26,335,040 0 0 374,708,547 0 COUNT_INSERT 10,001 0 10,001 0 10,001 0 SUM_TIMER_INSERT 231,879,080,056 0 7,504,322,436 0 3,768,242,422,479 0 MIN_TIMER_INSERT 11,728,944 0 22,464 0 24,384 0 AVG_TIMER_INSERT 23,185,288 0 750,087 0 376,786,521 0 MAX_TIMER_INSERT 50,530,335,064 0 426,302,487 0 428,082,829,761 0 COUNT_UPDATE 0 0 0 0 0 0 We could use common O.S. activity measuring tools. Using MySQL’s own Performance_Schema: After a: truncate table crossover; Obtained via a: select * from table_io_waits_summary_by _table where object_schema ='sakila' and object_name = 'crossover‘ G
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16 Knowledge & The Future http://ourwikiworld.blogspot.com.es/2011/04/amazing-physics-stunning-building.html
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17  mysql CLI & plenty of commands.  MySQL Workbench Utilities. – mysqldbcopy, mysqlfailover, mysqlreplicate, mysqlrpladmin, mysqlrplcheck, etc.  MySQL Performance_Schema.  MySQL Enterprise Monitor. – Replication Advisor – Query Analyzer Knowledge is key: Monitoring & Utilities. MySQL Solutions
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18 The Future Nebulae  Existing / inherited Applications. – eg.Using CSV type tables, from Win to Linux or vice-versa.(Address books, User provisioning, etc.)  Cloud / Virtualization platforms. – HA Potential: Replication, 1x Blackhole for 10+ slaves) & MySQL Cluster. – Column filtering / Replication filtering / Triggers from Blackhole.  Eliminate sensitive or un-required columns.  Create a number of slaves from BLACKHOLE each with it’s own dataset.  Big Data / BI / custom solutions. – Repeat bulk loads for troubleshooting / performance comparisons. – Analyze binlogs on BLACKHOLE / backup strategies, etc. Who can benefit?
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19 Reference material  The BLACKHOLE Storage Engine (ref.man.)  MySQL Replication Tutorial (TWP)  The CSV Storage Engine (ref.man.)  MySQL Workbench download. (documentation)  Download Utilities without Workbench: http://dev.mysql.com/downloads/tools/utilities/1.3.html  MySQL Enterprise Monitor: – Trial 30 day evaluation http://edelivery.oracle.com  Performance Schema documentation. (table_io_waits_summary_by_table descr.)  Oracle Linux & Unbreakable Enterprise Kernel (Getting Started).  SUSE Linux Enterprise Server 11 SP2 for x86 included packages.  Advanced Linux Command Mastery
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20 keith.hollman@oracle.com
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21 Efxaristo OSC13, SUSE, Thessaloniki, everyone. (Thank you)
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22 Graphic Section Divider  Keynotes  Conferences Sessions  Hands-on Labs  Birds-of-a-feather sessions  Tutorials  Receptions  Demo Pods  Exhibition Hall San Francisco, September 21-23 Additional Day of Tutorials Oracle.com/mysqlconnect Early Bird Discount: Register by July 19 and Save US$500! Email to partner-engagement_ww@oracle.com for exhibition and sponsorship opportunities
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23
  • Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24