A MySQL Odyssey - A Blackhole Crossover

A MySQL Odyssey - a
Blackhole crossover
Keith Hollman
MySQL Principal Sales Consultant EMEA

The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated
into any contract.
It is not a commitment to deliver any material, code, or functionality,
and should not be relied upon in making purchasing decisions.
The development, release, and timing of any features or functionality
described for Oracle’s products remain at the sole discretion of Oracle.
Safe Harbor Statement

Agenda
MySQL Replication: the crossover.
From
Dev…
…thru’ the
BLACK
HOLE…
Knowledge
is key
Non-
implosion
non-
destructive
…into Pre /
Production
Windows
¿Why?
Oracle Linux
The
Future
Nebulae

¿Why?
 Sharing how to automate a cross-platform data transfer with
MySQL, applying data filters at the same time.
 Small, quick and easy to setup, minimizing performance & resource
overheads.
 Maneuvering the High Availability door to other possibilities, such as
Security control, Auditing, horizontal & vertical scalability, read-write
segregations.
What’s this all about:

¿Why?
 A Developer using MySQL on Win (Testing).
 Need to export & import data / schema definition into DBA’s /
Sys.Admin’s Linux environment (Pre-Prod, UAT, Production).
 Certain changes are required and it has to be easy, and unobtrusive.
 And data sensitive columns?
Wouldn’t it be nice if I could have it all transferred directly, automatically
and not have to worry about platform differences?
Just “a” Scenario

¿Why?
The Setup
“Dev” pc
Win.7 Pro SP1 Intel i5 2.67GHz 8Gb RAM 64bit
CREATE TABLE `crossover` …… ENGINE=InnoDB;
“BLACKHOLE” env
Oracle Linux 6.3 UEK 2.6.39 VM 1xCPU 512Mb RAM 32bit 5Gb disk
ALTER TABLE `crossover` ENGINE=BLACKHOLE;
“Prod” env
SLES 11 SP2 3.0.74 VM 1xCPU 1Gb RAM 10Gb disk
ALTER TABLE `crossover` ENGINE=InnoDB;
MySQL 5.6.11 EE
Replication Setup
using GTID’s:
Master (Win)
-> Slave / Master (OL)
-> Slave (SLES)

¿Why?
 BLACKHOLE storage engine:
“.. accepts data but throws it away and does not store it. Retrievals always return an empty
result”
– Row Based Replication best (vs Statement, autoinc & PK’s)
– UPDATE’s & DELETE’s triggers aren’t activated: will break replication.
– All operations on BLACKHOLE will be replicated to SLES slave.1
 Others:
– Replicating with lower_case_table_names=1.
– replicate-ignore-db, replicate-do-table, replicate-wild-*.
– Double check FW (iptables –F for easiness).
MySQL BLACKHOLE & Other Awareness
1 Should we be tempted to temporarily ‘fix’
replication by changing SE and back.

The BLACKHOLE
Crossover

From Dev… thru’ the BLACKHOLE…
 Logical Dump of the master database “sakila” from Win to OL:
[mysql@ol63uek01 ~]$ mysqldump -uroot -poracle -h 192.168.56.101 -P3307 -B sakila
--master-data=2 | mysql -uroot
ol63 10:40 sakila> show databases;
 Set Replication up…
 … and make sure we only replicate “sakila”:
[mysql@ol63uek01 ~]$ vi /etc/my.cnf : [mysqld] replicate-do-db =sakila
 Check it’s working:
ol63 11:00 sakila> show slave statusG
Win 11:00 sakila> call BlackholeInsert (1000); ..& look for the “*–relay.bin*” logs.
[ From Win to Oracle Linux ]

Win 11:01 sakila> select count(*) from crossover;
 Stepping into the BLACKHOLE:
ol63 11:02 sakila> select count(*) from crossover;
ol63 11:02 sakila> show create table crossover;
ol63 11:02 sakila> alter table crossover engine=blackhole;
Query OK, 1001 rows affected (0.16 sec)
Records: 1001 Duplicates: 0 Warnings: 0
 Confirming:
ol63 11:02 sakila> show create table crossover;
ol63 11:03 sakila> select count(*) from crossover;
From Dev… thru’ the BLACKHOLE…
Continued…
+----------+
| count(*) |
+----------+
| 1001 |
+----------+
1 row in set (0.00 sec)
+----------+
| count(*) |
+----------+
| 1001 |
+----------+
| crossover | CREATE TABLE
`crossover` (
ÌD` int(7) NOT NULL
AUTO_INCREMENT,
`Name` char(20) NOT NULL
DEFAULT '',
PRIMARY KEY (ÌD`)
) ENGINE=InnoDB
AUTO_INCREMENT=1002 DEFAULT
CHARSET=utf8 |
| crossover | CREATE TABLE
`crossover` (
ÌD` int(7) NOT NULL
AUTO_INCREMENT,
`Name` char(20) NOT NULL
DEFAULT '',
PRIMARY KEY (ÌD`)
) ENGINE=BLACKHOLE DEFAULT
CHARSET=utf8 |
+----------+
| count(*) |
+----------+
| 0 |
+----------+

… thru’ the BLACKHOLE… into Prod.
 Logical dump of ol63uek BLACKHOLE table to SLES Production env:
[mysql@suse01 ~]$ mysqldump -uroot -p -h 192.168.56.102 -B sakila --master-data=2
| mysql -uroot
suse01 11:11 sakila> show databases;
 Set Replication up… again (ol63 is our Master now) & check it works:
suse01 11:12 sakila> show slave statusG
 Coming out of the BLACKHOLE, into InnoDB@SLES:
suse01 11:12 sakila> alter table crossover engine=InnoDB;
 Pull data from Win into SLES (non-blackhole tables):
[mysql@suse01 ~]$ mysqldump -uroot –p –h 192.168.56.101 -P3307 --no-create-info
--no-create-db –B sakila | mysql -uroot

Non-Implosion and Non-
Destructive.
http://ourwikiworld.blogspot.com.es/2011/04/amazing-physics-stunning-building.html

Non-implosion & non-destructive
 Truncate table crossover;
 Bulk INSERT’s.
 DDL modifications.
 Finer Replication capabilities.
 Schema manoeuvrability.
 Performance analysis.
 Located on same Master server.
 Isolated env’s & modular.
! UPDATE & DELETE triggers.
! Added Operational commitments.
! Data models, PK & Autoinc cols.
! IO monitoring / impact evaluation
removed.
! Infrastructure becomes inter-
dependent (replication).

Non-implosion & non-destructive cont.
Win ol63uek SLES
OBJECT_NAME crossover Diff vs orig. crossover diff crossover diff
COUNT_STAR 10,002 -1 10,001 0 40,007 -20,004
SUM_TIMER_WAIT 231,905,415,096 -26,335,040 7,504,322,436 0 3,783,883,427,646 -9,161,505,807
MIN_TIMER_WAIT 11,728,944 0 22,464 0 24,384 0
AVG_TIMER_WAIT 23,185,664 -376 750,087 0 94,580,202 94,127,574
MAX_TIMER_WAIT 50,530,335,064 0 426,302,487 0 428,082,829,761 0
COUNT_READ 1 -1 0 0 30,006 -20,004
SUM_TIMER_READ 26,335,040 -26,335,040 0 0 15,641,005,167 -9,161,505,807
MIN_TIMER_READ 26,335,040 -26,335,040 0 0 24,384 0
AVG_TIMER_READ 26,335,040 -26,335,040 0 0 521,208 126,492
MAX_TIMER_READ 26,335,040 -26,335,040 0 0 374,708,547 0
COUNT_WRITE 10,001 0 10,001 0 10,001 0
SUM_TIMER_WRITE 231,879,080,056 0 7,504,322,436 0 3,768,242,422,479 0
MIN_TIMER_WRITE 11,728,944 0 22,464 0 24,384 0
AVG_TIMER_WRITE 23,185,288 0 750,087 0 376,786,521 0
MAX_TIMER_WRITE 50,530,335,064 0 426,302,487 0 428,082,829,761 0
COUNT_FETCH 1 -1 0 0 30,006 -20,004
SUM_TIMER_FETCH 26,335,040 -26,335,040 0 0 15,641,005,167 -9,161,505,807
MIN_TIMER_FETCH 26,335,040 -26,335,040 0 0 24,384 0
AVG_TIMER_FETCH 26,335,040 -26,335,040 0 0 521,208 126,492
MAX_TIMER_FETCH 26,335,040 -26,335,040 0 0 374,708,547 0
COUNT_INSERT 10,001 0 10,001 0 10,001 0
SUM_TIMER_INSERT 231,879,080,056 0 7,504,322,436 0 3,768,242,422,479 0
MIN_TIMER_INSERT 11,728,944 0 22,464 0 24,384 0
AVG_TIMER_INSERT 23,185,288 0 750,087 0 376,786,521 0
MAX_TIMER_INSERT 50,530,335,064 0 426,302,487 0 428,082,829,761 0
COUNT_UPDATE 0 0 0 0 0 0
We could use
common O.S. activity
measuring tools.
Using MySQL’s own
Performance_Schema:
After a:
truncate table crossover;
Obtained via a:
select *
from
table_io_waits_summary_by
_table
where
object_schema ='sakila'
and
object_name = 'crossover‘
G

Knowledge & The Future
http://ourwikiworld.blogspot.com.es/2011/04/amazing-physics-stunning-building.html

 mysql CLI & plenty of commands.
 MySQL Workbench Utilities.
– mysqldbcopy, mysqlfailover,
mysqlreplicate, mysqlrpladmin,
mysqlrplcheck, etc.
 MySQL Performance_Schema.
 MySQL Enterprise Monitor.
– Replication Advisor
– Query Analyzer
Knowledge is key: Monitoring & Utilities.
MySQL Solutions

The Future Nebulae
 Existing / inherited Applications.
– eg.Using CSV type tables, from Win to Linux or vice-versa.(Address books, User
provisioning, etc.)
 Cloud / Virtualization platforms.
– HA Potential: Replication, 1x Blackhole for 10+ slaves) & MySQL Cluster.
– Column filtering / Replication filtering / Triggers from Blackhole.
 Eliminate sensitive or un-required columns.
 Create a number of slaves from BLACKHOLE each with it’s own dataset.
 Big Data / BI / custom solutions.
– Repeat bulk loads for troubleshooting / performance comparisons.
– Analyze binlogs on BLACKHOLE / backup strategies, etc.
Who can benefit?

Reference material
 The BLACKHOLE Storage Engine (ref.man.)
 MySQL Replication Tutorial (TWP)
 The CSV Storage Engine (ref.man.)
 MySQL Workbench download. (documentation)
 Download Utilities without Workbench: http://dev.mysql.com/downloads/tools/utilities/1.3.html
 MySQL Enterprise Monitor:
– Trial 30 day evaluation http://edelivery.oracle.com
 Performance Schema documentation. (table_io_waits_summary_by_table descr.)
 Oracle Linux & Unbreakable Enterprise Kernel (Getting Started).
 SUSE Linux Enterprise Server 11 SP2 for x86 included packages.
 Advanced Linux Command Mastery

keith.hollman@oracle.com

Efxaristo OSC13, SUSE, Thessaloniki, everyone.
(Thank you)

Graphic Section Divider
 Keynotes
 Conferences Sessions
 Hands-on Labs
 Birds-of-a-feather sessions
 Tutorials
 Receptions
 Demo Pods
 Exhibition Hall
San Francisco, September 21-23
Additional Day of Tutorials
Oracle.com/mysqlconnect
Early Bird
Discount: Register
by July 19 and Save
US$500!
Email to partner-engagement_ww@oracle.com for
exhibition and sponsorship opportunities

A MySQL Odyssey - A Blackhole Crossover

More Related Content

What's hot

Viewers also liked

Similar to A MySQL Odyssey - A Blackhole Crossover

More from Keith Hollman

Recently uploaded

A MySQL Odyssey - A Blackhole Crossover