MySQL for Oracle DBAs
Upcoming SlideShare
Loading in...5
×
 

MySQL for Oracle DBAs

on

  • 6,036 views

Presentation for MySQL UC 2006 on MySQL for the Oracle DBA

Presentation for MySQL UC 2006 on MySQL for the Oracle DBA

Statistics

Views

Total Views
6,036
Views on SlideShare
4,864
Embed Views
1,172

Actions

Likes
8
Downloads
411
Comments
1

6 Embeds 1,172

http://www.markleith.co.uk 1153
http://s386513550.websitehome.co.uk 14
http://sqldump.cool-tools.co.uk 2
http://westone.moodlesites.com 1
http://www.slideshare.net 1
http://www.docshut.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

MySQL for Oracle DBAs MySQL for Oracle DBAs Presentation Transcript

  • MySQL for Oracle DBAs Mark Leith Support Engineer, MySQL AB mleith@mysql.com 1
  • Introduction 9 years database experience, starting with Oracle 5 years MySQL experience Extensive experience with database tools Working within the MySQL Support Group, for the Americas region 2
  • Agenda MySQL Architecture Backup and Recovery Managing space and tables Managing Memory Tracing Sessions / Auditing Monitoring Tools and Tips Connecting MySQL to Oracle Questions 3
  • Architecture Product Architecture Internal Memory Structures Filesystem Layout Binaries Database / Table Datafiles Log Files Permissions / Ownership 4
  • Product Architecture 5
  • Internal Memory Structures 6
  • Filesystem Layout - Binaries Windows default: C:Program FilesMySQLMySQL Server x.xbin Tarball default: /usr/local/mysql/bin RPM default: /usr/lib/mysql, /usr/bin & /usr/sbin (depending on RPM) MySQL Server executables mysqld, mysqld-debug, mysqld-max MySQL Client Utilities mysql, mysqldump, mysqlcheck, mysqlbinlog .. Upgrade and Security scripts mysql_upgrade, mysql_fix_privilege_tables, mysql_secure_installation .. 7
  • Filesystem Layout - Datafiles Windows default: C:Program FilesMySQLMySQL Server x.xdata Tarball default: /usr/local/mysql/data RPM default: /var/lib/mysql Databases use “datadir” Default “mysql” database (SYSTEM Tablespace) One directory per database Table frm, data and index files within respective database directories InnoDB uses “innodb_data_home_dir” Single tablespace file (ibdata1) located in “datadir” by default Set multiple datafiles with “innodb_datafile_path” frm files still created within each database directory under datadir Respect datadir conventions when using “innodb_file_per_table” 8
  • Filesystem Layout - Log files Windows Default: C:Program FilesMySQLMySQL Server x.xdata Tarball default: /usr/local/mysql/data RPM default: /var/log & /var/lib/mysql Error Log (Alert Log) Set path with “log_error”, defaults to [hostname].err in “datadir” Binary Logs (Archive Redo Logs) Set with “log_bin” Record of all INSERT/UPDATE/DELETE activity <=5.0 Statement Based >=5.1 Statement Based and Row Based InnoDB Transaction logs (Online Redo Logs) ib_logfile[x] set with “innodb_log_files_in_group” (default 2) Provides “Durability” to ACID compliance Control “checkpoint” times by altering the size of “innodb_log_file_size” 9
  • Permissions / Ownership No concept of “ownership” Permissions stored in the “mysql” database “schema” == “database” rather than “users objects” mysql.user Username, host, password hash, global privileges, grant option mysql.db Database level privileges mysql.tables_priv Table level privileges mysql.columns_priv Column level privileges mysql.procs_priv Procedure / Function privileges 10
  • Backup and Recovery Cold Backup Filesystem Snapshot (cp / rsync) Warm Backup mysqldump Hot Backup mysqldump, mysqlhotcopy Using Slaves Recovery tips 11
  • Backup - Cold Backup Full cold backup All database directories and contents (including “mysql”) my.cnf/ini file If using InnoDB All InnoDB datafiles All InnoDB Transaction Redo Log files Incremental Cold Backup All of the above as a starting point Binary logging enabled (”log-bin”) Only requires the latest binary logs since the last snapshot Beware of recovery times - applying the binary logs is a lot slower than file copying 12
  • Backup - Warm Backup mysqldump Uses FLUSH TABLES WITH READ LOCK Locks all other sessions /* Backup db1, db2 and db3 to the same file, mixed MyISAM / InnoDB / other engines */ mysqldump -u root --opt --databases db1, db2, db3 > /backup/file.sql /* Backup all databases to seperate files per database schema */ #!/bin/bash DATABASES=`mysql -u root -N -e 'SHOW DATABASES'` OPTIONS1='--extended-insert --no-autocommit --quick --set-charset' OPTIONS2='--disable-keys --skip-add-locks --skip-comments' DUMPOPTIONS=quot;$OPTIONS1 $OPTIONS2quot; for D in $DATABASES ; do echo dumping $D ; echo quot;use $D;quot; > $D-data.sql echo quot;set autocommit=0;quot; >> $D-data.sql mysqldump -u root $DUMPOPTIONS --databases $D >> $D-data.sql ; done 13
  • Backup - Hot Backup mysqldump InnoDB only Uses transactions and multi-versioning /* Backup InnoDB databases to seperate files per database schema */ #!/bin/bash DATABASES=`mysql -u root -N -e 'SHOW DATABASES'` OPTIONS1='--extended-insert --no-autocommit --quick --set-charset --disable-keys' OPTIONS2=' --skip-add-locks --skip-comments --single-transaction --master-data' DUMPOPTIONS=quot;$OPTIONS1 $OPTIONS2quot; for D in $DATABASES ; do echo dumping $D ; echo quot;use $D;quot; > $D-data.sql echo quot;set autocommit=0;quot; >> $D-data.sql mysqldump -u root $DUMPOPTIONS --databases $D >> $D-data.sql ; done 14
  • Backup - Hot Backup mysqlhotcopy MyISAM and ARCHIVE tables only LOCK TABLES ..; FLUSH TABLES; [cp | scp]; UNLOCK TABLES; Requires Perl DBI /* Hot Backup MyISAM / ARCHIVE table files to seperate directories per database schema */ #!/bin/bash DATABASES=`mysql -u root -N -e 'SHOW DATABASES'` DUMPOPTIONS='-u root --method=cp --noindices --flushlog' for D in $DATABASES ; do echo copying $D ; DIRNAME=$D-`date '+%d%m%y'` mkdir ~/backup/$DIRNAME mysqlhotcopy $DUMPOPTIONS $D ~/backup/$DIRNAME done 15
  • Backup - Slave Hot Backup Requires no downtime on the master system Allows full cold backups easily, giving faster recovery times Adds a server that can also be used to spread load Master Slave cp scp rsync server-id 1 server-id 2 LVM Application Users mysqldump mysqlhotcopy (Dataguard) Reporting Users 16
  • Recovery Tips mysqlbinlog Perform point in time recovery directly to MySQL mysqlbinlog bin.000001 bin.000002 bin.000003 --stop-datetime=’2006-03-26 12:00:00’ | mysql -u root -p Output statements for single databases directly to MySQL mysqlbinlog bin.000001 bin.000002 bin.000003 --stop-datetime=’2006-03-26 12:00:00’ --database sakila | mysql -u root -p Output statements to plain text SQL files mysqlbinlog bin.000001 bin.000002 bin.000003 --stop-datetime=’2006-03-26 12:00:00’ --database sakila > /tmp/sakila-binlog-dump.sql 17
  • Recovery Tips Always use binary logging if possible On recovery, make an initial copy of all current database files Work on original set, leaving copy as secondary backup Take regular full backups, to minimize recovery time InnoDB Log file size controls full checkpoint activity If using mysqlhotcopy, with --noindices, to rebuild indexes: mysqlcheck --all-databases --auto-repair --use-frm 18
  • Managing Space and Tables MyISAM Storage Engine Gathering stats, managing space InnoDB Storage Engine Gathering stats, managing space Partitioning Partition Management 19
  • Managing MyISAM Space All table files stored within specific database directory Row data held within [tablename].MYD Index data held within [tablename].MYI Table structure held within [tablename].frm medusa:/usr/local/mysql/data/sakila root# ls -l film_text* -rw-rw---- 1 mysql wheel 119616 Apr 4 20:51 film_text.MYD -rw-rw---- 1 mysql wheel 205824 Apr 4 20:51 film_text.MYI -rw-rw---- 1 mysql wheel 8642 Apr 4 20:51 film_text.frm Deleted row space retained for future INSERT/UPDATE Supports FULLTEXT Indexes (Oracle Text) Table level locking (SELECT = Shared, INSERT, UPDATE, DELETE = Write) Data warehousing storage engine 20
  • Managing MyISAM Space SHOW TABLE STATUS mysql> SHOW TABLE STATUS LIKE 'film_text'G *************************** 1. row *************************** Name: film_text Variable Length rows (VARCHAR etc.) Engine: MyISAM “Fixed” = CHAR etc. static sized Version: 10 Row_format: Dynamic Data size in bytes (~117KB) Rows: 950 Avg_row_length: 119 Data_length: 119616 Max size for the internal row pointer Max_data_length: 281474976710655 Index_length: 205824 Index size in bytes (~201KB) Data_free: 6136 Auto_increment: NULL Space reserved but unused Create_time: 2006-04-03 14:16:02 (~6KB) Update_time: 2006-04-04 20:51:02 Check_time: 2006-04-04 20:51:17 Collation: utf8_general_ci Checksum: NULL Create_options: Comment: 1 row in set (0.00 sec) 21
  • Managing MyISAM Space INFORMATION_SCHEMA TABLES (DBA_ / USER_ tables) SELECT s.schema_name, CONCAT(IFNULL(ROUND((SUM(t.data_length)+SUM(t.index_length)) /1024/1024,2),0.00),quot;Mbquot;) total_size, CONCAT(IFNULL(ROUND(((SUM(t.data_length)+SUM(t.index_length))-SUM(t.data_free))/ 1024/1024,2),0.00),quot;Mbquot;) data_used, CONCAT(IFNULL(ROUND(SUM(data_free)/1024/1024,2),0.00),quot;Mbquot;) data_free, IFNULL(ROUND((((SUM(t.data_length)+SUM(t.index_length))-SUM(t.data_free)) /((SUM (t.data_length)+SUM(t.index_length)))*100),2),0) pct_used, COUNT(table_name) total_tables FROM INFORMATION_SCHEMA.SCHEMATA s LEFT JOIN INFORMATION_SCHEMA.TABLES t ON s.schema_name = t.table_schema WHERE s.schema_name = quot;sakilaquot; GROUP BY s.schema_name ORDER BY pct_used DESCG *************************** 1. row *************************** schema_name: sakila total_size: 6.62Mb data_used: 6.62Mb data_free: 0.01Mb pct_used: 99.91 total_tables: 22 1 row in set (0.08 sec) 22
  • Managing MyISAM Space INFORMATION_SCHEMA TABLES (DBA_ / USER_ tables) SELECT table_name, ROUND(((t.data_length+t.index_length)-t.data_free)/(t.data_length+t.index_length)*100) pct_used FROM information_schema.tables t WHERE table_schema = 'sakila' HAVING pct_used < 99 ORDER BY table_nameG *************************** 1. row *************************** table_name: film_text pct_used: 98 1 row in set (0.05 sec) To reclaim space to the filesystem run “OPTIMIZE TABLE” Creates a copy of the table, drops the original, and renames the copy Requires a table lock If frequent activity on the table run “ANALYZE TABLE” Requires a table lock 23
  • Managing MyISAM Space myisamchk -dvv Should not be used whilst table is being used LOCK TABLE .. WRITE; or shutdown medusa:/Users/markleith/mysql/mysql-5.1.7/data/sakila root# ../../bin/myisamchk -dvv film_text MyISAM file: film_text Record format: Packed Character set: latin1_swedish_ci (8) File-version: 1 Creation time: 2006-04-18 15:57:45 Status: changed Data records: 1000 Deleted blocks: 0 Datafile parts: 1000 Deleted data: 0 Datafile pointer (bytes): 6 Keyfile pointer (bytes): 6 Datafile length: 119616 Keyfile length: 205824 Max datafile length: 281474976710654 Max keyfile length: 288230376151710719 Recordlength: 782 table description: Key Start Len Index Type Rec/key Root Blocksize 1 2 2 unique short 1 29696 1024 2 5 254 fulltext ? packed 0 49152 1024 1 4 float 0 Field Start Length Nullpos Nullbit Type 1 1 1 2 2 2 no zeros 3 4 767 varchar 4 771 10 1 1 blob 24
  • Managing InnoDB Space Row level locking Supports all transaction isolation levels Tables, Indexes and rollback created in a single tablespace Multiple datafiles, with last datafile allowed to autoextend medusa:/usr/local/mysql/data root# ls -l ib* -rw-rw---- 1 mysql wheel 5242880 Apr 8 21:25 ib_logfile0 } Redo Logs -rw-rw---- 1 mysql wheel 5242880 Apr 8 21:25 ib_logfile1 -rw-rw---- 1 mysql wheel 18874368 Apr 8 21:25 ibdata1 } Data Files -rw-rw---- 1 mysql wheel 10485760 Apr 8 21:25 ibdata2 Clustered Indexes (Index Organized Tables) Pages and extents managed like a filesystem Filesystem InnoDB Oracle Disk Partition Tablespace Tablespace(s) File Segment (Table/Index) Segment (Table/Index) Inode Inode Filesystem Block Extent (64 pages, 1Mb) Extent Disk Block Page (UNIV_PAGE_SIZE 16Kb) DB_BLOCK_SIZE 25
  • Managing InnoDB Space SHOW TABLE STATUS mysql> SHOW TABLE STATUS LIKE 'rental'G *************************** 1. row *************************** Name: rental New 5.0 InnoDB compact row format Engine: InnoDB Version: 10 Row_format: Compact Data size in bytes (~1.5Mb) Rows: 16305 Avg_row_length: 97 Data_length: 1589248 No Maximum (64Tb for tablespace) Max_data_length: 0 Index_length: 1261568 Index size in bytes (~1.2Mb) Data_free: 0 Auto_increment: 16050 Unused space not reported by table Create_time: 2006-04-03 14:16:02 Update_time: NULL Check_time: NULL Freespace reported by tablespace Collation: utf8_general_ci (4Mb) Checksum: NULL Create_options: Comment: InnoDB free: 4096 kB; (`customer_id`) REFER `sakila/customer`(`customer_id`) ON 1 row in set (0.10 sec) 26
  • Managing InnoDB Space InnoDB Table and Tablespace Monitors Dumped to the error log, or console (stdout) if no log-error Dump the InnoDB internal Data Dictionary information Switched on with special InnoDB tables InnoDB Table Monitor create table innodb_table_monitor (a int) engine = innodb; InnoDB Tablespace Monitor create table innodb_tablespace_monitor (a int) engine = innodb; 27
  • Managing InnoDB Space InnoDB Tablespace Monitor ================================================ 060403 14:20:31 INNODB TABLESPACE MONITOR OUTPUT ================================================ FILE SPACE INFO: id 0 size 1152, free limit 832, free extents 3 not full frag extents 4: used pages 202, full frag extents 2 first seg id not used 0 114 SEGMENT id 0 86 space 0; page 209; res 28 used 28; full ext 0 fragm pages 28; free extents 0; not full extents 0: pages 0 SEGMENT id 0 87 space 0; page 209; res 1 used 1; full ext 0 fragm pages 1; free extents 0; not full extents 0: pages 0 SEGMENT id 0 88 space 0; page 209; res 16 used 16; full ext 0 fragm pages 16; free extents 0; not full extents 0: pages 0 SEGMENT id 0 89 space 0; page 209; res 1 used 1; full ext 0 fragm pages 1; free extents 0; not full extents 0: pages 0 SEGMENT id 0 90 space 0; page 209; res 16 used 16; full ext 0 fragm pages 16; free extents 0; not full extents 0: pages 0 SEGMENT id 0 91 space 0; page 209; res 1 used 1; full ext 0 fragm pages 1; free extents 0; not full extents 0: pages 0 SEGMENT id 0 92 space 0; page 209; res 13 used 13; full ext 0 fragm pages 13; free extents 0; not full extents 0: pages 0 ...... 28
  • Managing InnoDB Space InnoDB Tablespace Monitor ...... SEGMENT id 0 84 space 0; page 2; res 96 used 53; full ext 0 fragm pages 32; free extents 0; not full extents 1: pages 21 SEGMENT id 0 84 space 0; page 2; The Internal data dictionary id for the tablespace segment “space 0” is the system tablespace (ibdata files) “page 2” is where the inode of the segment is res 96 used 53; “res” is total number of reserved pages “used” is the number of pages with data allocated full ext 0 The number of extents that are completely used fragm pages 32; The first 32 pages are fully allocated free extents 0; The number of extents that have no pages at all used not full extents 1: pages 21 The number of extents that have pages used but are not full “pages” is how many pages are used within these extents 29
  • Managing InnoDB Space InnoDB Table Monitor TABLE: name sakila/rental, id 0 26, columns 11, indexes 5, appr.rows 16305 COLUMNS: rental_id: DATA_INT len 4 prec 0; rental_date: DATA_INT len 8 prec 0; inventory_id: DATA_INT len 3 prec 0; customer_id: DATA_INT len 2 prec 0; return_date: DATA_INT len 8 prec 0; staff_id: DATA_INT len 1 prec 0; last_update: DATA_INT len 4 prec 0; DB_ROW_ID: DATA_SYS prtype 8 len 6 prec 0; DB_TRX_ID: DATA_SYS prtype 8 len 6 prec 0; DB_ROLL_PTR: DATA_SYS prtype 8 len 7 prec 0; INDEX: name PRIMARY, id 0 44, fields 1/9, type 3 root page 207, appr.key vals 16305, leaf pages 53, size pages 97 FIELDS: rental_id DB_TRX_ID DB_ROLL_PTR rental_date inventory_id customer_id return_date staff_id last_update INDEX: name rental_date, id 0 45, fields 3/4, type 2 root page 208, appr.key vals 17655, leaf pages 28, size pages 29 FIELDS: rental_date inventory_id customer_id rental_id INDEX: name idx_fk_inventory_id, id 0 46, fields 1/2, type 0 root page 210, appr.key vals 4467, leaf pages 16, size pages 17 FIELDS: inventory_id rental_id INDEX: name idx_fk_customer_id, id 0 47, fields 1/2, type 0 root page 211, appr.key vals 589, leaf pages 16, size pages 17 FIELDS: customer_id rental_id INDEX: name idx_fk_staff_id, id 0 48, fields 1/2, type 0 root page 212, appr.key vals 1, leaf pages 13, size pages 14 FIELDS: staff_id rental_id FOREIGN KEY CONSTRAINT sakila/fk_rental_staff: sakila/rental ( staff_id ) REFERENCES sakila/staff ( staff_id ) FOREIGN KEY CONSTRAINT sakila/fk_rental_inventory: sakila/rental ( inventory_id ) REFERENCES sakila/inventory ( inventory_id ) FOREIGN KEY CONSTRAINT sakila/fk_rental_customer: sakila/rental ( customer_id ) REFERENCES sakila/customer ( customer_id ) FOREIGN KEY CONSTRAINT sakila/fk_payment_rental: sakila/payment ( rental_id ) REFERENCES sakila/rental ( rental_id ) 30
  • Managing InnoDB Space InnoDB Table Monitor TABLE: name sakila/rental, id 0 26, columns 11, indexes 5, appr.rows 16305 COLUMNS: rental_id: DATA_INT len 4 prec 0; rental_date: DATA_INT len 8 prec 0; inventory_id: DATA_INT len 3 prec 0; customer_id: DATA_INT len 2 prec 0; return_date: DATA_INT len 8 prec 0; staff_id: DATA_INT len 1 prec 0; last_update: DATA_INT len 4 prec 0; DB_ROW_ID: DATA_SYS prtype 8 len 6 prec 0; DB_TRX_ID: DATA_SYS prtype 8 len 6 prec 0; DB_ROLL_PTR: DATA_SYS prtype 8 len 7 prec 0; “len” is column length in bytes, “prec” is the columns precision (DECIMAL etc.) “DB_ROW_ID”, “DB_TRX_ID” and “DB_ROLL_PTR” internal columns for transactions / multi-versioning INDEX: name PRIMARY, id 0 44, fields 1/9, type 3 root page 207, appr.key vals 16305, leaf pages 53, size pages 97 FIELDS: rental_id DB_TRX_ID DB_ROLL_PTR rental_date inventory_id customer_id return_date staff_id last_update “fields 1/9” is fields specifically specified in the index (1) and total fields in the index (9) “type” is index type: 3 = PRIMARY, 2 = UNIQUE, 0 = KEY, 1 = AUTO GENERATED PK “leaf pages 53” is the number of pages with data allocated (”used” in tablespace monitor) “size pages 97” is the total number of allocated pages (”res” in tablespace monitor) Rows: 16305 Avg_row_length: 97 16,384 * 97 = 1,589,248 (1.5Mb) Data_length: 1589248 Max_data_length: 0 16,384 * (29+17+17+14) = 1,261,568 (1.2mb) Index_length: 1261568 16,384 * (97-53) = 720,896 (~700Kb) Data_free: 0 31
  • Managing InnoDB Space Allocates the first 32 pages individually After 32 pages allocated, allocates 1Mb extents To reclaim space to the tablespace run “OPTIMIZE TABLE” Reorganization process similar to MyISAM Requires a table lock “ANALYZE TABLE” rarely needed, as InnoDB updates stats Not Uniform extent allocation, fragmentation highly possible To reorganize the tablespace: mysqldump all tables Stop MySQL mv the InnoDB tablespace datafiles and log files to a backup location Start MySQL Load the mysqldump 32
  • Partitioning (5.1) No extra costs All storage engines support partitioning “Local” indexes only Indexes only created locally to each partition “Global” indexes scheduled for 5.2 Indexes created across the entire partitioned table Partition pruning already available at the optimizer level Allow far greater table maintenance flexibility, with partition level locks 33
  • Partitioning (5.1) RANGE partitioning: (As in Oracle 8i RANGE partitions) Assigns rows to partitions based on column values falling within a given range. LIST partitioning: (As in Oracle 9i LIST partitions) Similar to partitioning by range, except that the partition is selected based on columns matching one of a set of discrete values. HASH partitioning: (Not possible in Oracle) A partition is selected based on the value returned by a user-defined expression that operates on column values in rows to be inserted into the table. The function may consist of any expression valid in MySQL that yields a non-negative integer value. KEY partitioning: (As in Oracle 8i HASH partitions) Similar to partitioning by hash, except that only one or more columns to be evaluated are supplied, and the MySQL server provides its own hashing function. These columns can contain other than integer values, since the hashing function supplied by MySQL guarantees an integer result regardless of the column data type. Subpartitioning: (As in Oracle 9i composite partitions) Subpartition tables that are partitioned by RANGE or LIST. Subpartitions may use either HASH or KEY partitioning. 34
  • Partitioning (5.1) Moving to Partitioned tables PARTITION key must be a part of the PRIMARY KEY, if any, if not, any column PARTITION key must be on an INT column, or expression returning an INT Do not support FOREIGN KEY constraints CREATE TABLE `RENTAL` ( `rental_id` int(11) NOT NULL auto_increment, `rental_date` datetime NOT NULL, `inventory_id` mediumint(8) unsigned NOT NULL, `customer_id` smallint(5) unsigned NOT NULL, `return_date` datetime default NULL, `staff_id` tinyint(3) unsigned NOT NULL, `last_update` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP, PRIMARY KEY (`rental_id`), UNIQUE KEY `rental_date` (`rental_date`,`inventory_id`,`customer_id`), KEY `idx_fk_inventory_id` (`inventory_id`), KEY `idx_fk_customer_id` (`customer_id`), KEY `idx_fk_staff_id` (`staff_id`), CONSTRAINT `fk_rental_staff` FOREIGN KEY (`staff_id`) REFERENCES `staff` (`staff_id`) ON UPDATE CASCADE, CONSTRAINT `fk_rental_inventory` FOREIGN KEY (`inventory_id`) REFERENCES `inventory` (`inventory_id`) ON UPDATE CASCADE, CONSTRAINT `fk_rental_customer` FOREIGN KEY (`customer_id`) REFERENCES `customer` (`customer_id`) ON UPDATE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8 35
  • Partitioning (5.1) Moving to Partitioned tables DROP TABLE rental_parts; CREATE TABLE `rental_parts` ( `rental_id` int(11) NOT NULL auto_increment, `rental_date` datetime NOT NULL, `inventory_id` mediumint(8) unsigned NOT NULL, `customer_id` smallint(5) unsigned NOT NULL, `return_date` datetime default NULL, `staff_id` tinyint(3) unsigned NOT NULL, `last_update` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP, PRIMARY KEY (`rental_id`, `rental_date`), UNIQUE KEY `rental_date` (`rental_date`,`inventory_id`,`customer_id`), KEY `idx_inventory_id` (`inventory_id`), KEY `idx_customer_id` (`customer_id`), KEY `idx_staff_id` (`staff_id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 PARTITION BY RANGE (YEAR(`rental_date`))( PARTITION rental_2002 VALUES LESS THAN (2003), PARTITION rental_2003 VALUES LESS THAN (2004), PARTITION rental_2004 VALUES LESS THAN (2005), PARTITION rental_2005 VALUES LESS THAN (2006), PARTITION rental_2006 VALUES LESS THAN MAXVALUE ); INSERT INTO rental_parts (SELECT * FROM rental); DROP TABLE rental; RENAME TABLE rental_parts TO rental; 36
  • Partitioning (5.1) INFORMATION_SCHEMA.PARTITIONS SELECT table_schema, table_name, partition_name, table_rows, data_length, index_length FROM information_schema.partitions WHERE table_name = 'rental’ AND table_schema = ‘sakila’ ************************** 1. row *************************** *************************** 4. row *************************** table_schema: sakila table_schema: sakila table_name: rental table_name: rental partition_name: rental_2002 partition_name: rental_2005 table_rows: 0 table_rows: 16272 data_length: 16384 data_length: 1589248 index_length: 65536 index_length: 1818624 *************************** 2. row *************************** *************************** 5. row *************************** table_schema: sakila table_schema: sakila table_name: rental table_name: rental partition_name: rental_2003 partition_name: rental_2006 table_rows: 0 table_rows: 182 data_length: 16384 data_length: 16384 index_length: 65536 index_length: 65536 *************************** 3. row *************************** 5 rows in set (0.02 sec) table_schema: sakila table_name: rental partition_name: rental_2004 table_rows: 0 data_length: 16384 index_length: 65536 37
  • Partitioning (5.1) Partition Management - ALTER TABLE REBUILD PARTITION (No Oracle equivalent) Used to rebuild/reorganize a partition ALTER TABLE rental REBUILD PARTITION rental_2005; OPTIMIZE PARTITION (No Oracle equivalent) Used like OPTIMIZE TABLE for each storage engine, at the partition level ALTER TABLE rental OPTIMIZE PARTITION rental_2004; ANALYZE PARTITION (analyze table emp partition(p1) compute statistics;) Reads and stores the key distributions for partitions ALTER TABLE rental ANALYZE PARTITION rental_2005; CHECK PARTITION (No Oracle equivalent) Checks specific partitions for errors (like CHECK TABLE) ALTER TABLE rental CHECK PARTITION rental_2003; REPAIR PARTITION (No Oracle equivalent) Used for MyISAM partitions, if they become corrupt (like REPAIR TABLE) ALTER TABLE rental REPAIR PARTITION rental_2003; 38
  • Partitioning (5.1) Partition Management - ALTER TABLE ADD PARTITION (Same as Oracle) Add a partition to the table (to the end of the ALTER TABLE rental ADD PARTITION (PARTITION rental_2007 VALUES LESS THAN (2008); DROP PARTITION (Same as Oracle) Remove a partition from the table ALTER TABLE rental DROP PARTITION rental_2002; REORGANIZE PARTITION (ALTER TABLE .. SPLIT / MERGE PARTITION) Split a partition in to multiple partitions, or merge several partitions in to one ALTER TABLE rental REORGANIZE PARTITION rental_2002 INTO ( PARTITION rental_pre_2002 VALUES LESS THAN (2002), PARTITION rental_2002 VALUES LESS THAN (2003)); ALTER TABLE rental REORGANIZE PARTITION rental_pre_2002, rental_2002 INTO (PARTITION rental_pre_2003 VALUES LESS THAN (2003)); 39
  • Managing Memory SHOW [GLOBAL | SESSION] STATUS MyISAM Key Cache InnoDB Buffer Pool Query Cache Thread Buffers 40
  • Managing Memory SHOW [GLOBAL | SESSION] STATUS [LIKE ‘string’] Provides session and global level server statistics Gives information about various aspects of the server such as locks, memory usage, disk usage, network activity etc. Akin to various of the V$ views in parts Reset statistics with “FLUSH STATUS” mysql> SHOW STATUS; +-------------------------------------+-----------------+ | Variable_name | Value | +-------------------------------------+-----------------+ | Aborted_clients |0 | | Aborted_connects |0 | | Bytes_received | 155372598 | | Bytes_sent | 1176560426 | | Connections | 30023 | ... | Created_tmp_disk_tables | 0 | | Created_tmp_tables | 8340 | | Created_tmp_files | 60 | 41
  • Managing Memory MyISAM Key Cache Caches only MyISAM index data Key status indicators: Key_reads - Index blocks read from disk Key_read_requests - Index blocks read from cache Key_writes - Index blocks written on disk Key_write_requests - Index blocks written within cache Monitor overall usage with a hit ratio indicator SHOW GLOBAL STATUS LIKE ‘key_read%’; 100-((key_reads/key_read_requests)*100) Set with “key_buffer_size” in my.cnf, or interactively: SET GLOBAL key_buffer_size = 50*1024*1024; Current maximum 4Gb 42
  • Managing Memory InnoDB Buffer Pool Caches InnoDB table and index data Key status indicators (5.0 or greater): Innodb_buffer_pool_reads - Pages read from disk Innodb_buffer_pool_read_requests - Pages read from the cache Innodb_buffer_pool_wait_free - Number of waits to create or flush page Innodb_buffer_pool_read_ahead_seq - Incremented for full table scans Monitor overall usage with a hit ratio indicator SHOW GLOBAL STATUS LIKE ‘innodb_buffer_pool_read%’; 100-((Innodb_buffer_pool_reads/Innodb_buffer_pool_read_requests)*100) Set with “innodb_buffer_pool_size” in my.cnf Not a dynamic variable (requires a restart) Typically 50-80% of dedicated server’s available RAM 43
  • Managing Memory Query Cache Caches query result sets as pre-formed network packets Key status indicators: Qcache_hits - Queries satisfied from the cache QCache_inserts - Query result sets added to the cache Qcache_lowmem_prunes - Query results flushed due to lack of memory Qcache_free_memory - Bytes free Monitor overall usage with a hit ratio indicator 100-((Qcache_inserts/Qcache_hits)*100) Set with “query_cache_size” and “query_cache_type” SET GLOBAL query_cache_size = 32*1024*1024; Control block allocation size with “query_alloc_block_size” Control size of result sets cached with “query_cache_limit” 44
  • Managing Memory Per Thread Buffers Possible to set dynamically at the session level with “SET SESSION variable = ..” sort_buffer_size Allocated per thread for each sort operation (ORDER BY / GROUP BY) Values larger than 2-4Mb rarely benefit performance Monitor for high “Sort_merge_passes” in SHOW GLOBAL STATUS read_rnd_buffer_size Used when reading sorted rows after an ORDER BY on an indexed column Larger result sets get better performance from larger buffer size Leave GLOBAL default and use SET SESSION when needed for large queries read_buffer_size Allocated per thread for each full MyISAM table scan Values larger than 2-4Mb rarely benefit performance Monitor for high “Handler_read_rnd_next” / “Select_scan” join_buffer_size Allocated per thread for each JOIN causing a full table scan Possible many allocated per thread with complex/multiple joins Values larger than 2-4Mb rarely benefit performance 45
  • Managing Memory Per Thread Buffers tmp_table_size Allocated per thread for each complex GROUP BY or subquries etc. Monitor with “Created_tmp_tables” and “Created_tmp_disk_tables” 32-64Mb depending on memory available binlog_cache_size Also allocated per thread Caches Binary Log entries for multiple statement transactions Monitor with “Binlog_cache_use” and “Binlog_disk_cache_use” satus variables Limit with “max_binlog_cache_size” system variable thread_cache_size GLOBAL level variable Caches threads/sessions with an already allocated stack for faster connections When sessions disconnect they have their buffers flushed and are re-added Monitor with ratio of “Threads_created” to “Connections” 46
  • Tracing Sessions / Auditing SHOW FULL PROCESSLIST (V$SESSION) PROCESS privilege required to view the full session list Use FULL keyword to include currently executing SQL mysql> SHOW FULL PROCESSLIST; +----+------+-----------------------+---------+------------------+--------------+-------------+----------------------------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+------+-----------------------+---------+------------------+---------------+-------------+---------------------------------------+ | 1 | root | localhost:32820 | merlin | Sleep | 1| | NULL | | 3 | root | localhost:32822 | merlin | Sleep | 507 | | NULL | | 4 | root | localhost:32823 | merlin | Sleep | 507 | | NULL | | 5 | root | localhost:32824 | merlin | Sleep | 506 | | NULL | | 6 | root | localhost:32826 | merlin | Sleep | 21207 | | NULL | | 7 | root | localhost:32827 | merlin | Query | 0 | Sending data | SELECT data_id,value | FROM `dc_INTEGER` WHERE `begin_time` =(SELECT MAX(`begin_time`) FROM `dc_INTEGER` | WHERE `collect_id`=90) AND `collect_id`=90 FOR UPDATE | | 16 | root | localhost:33447 | NULL | Sleep | 3| | NULL | | 19 | root | localhost:48885 | merlin | Sleep | 7| | NULL | | 27 | root | localhost | NULL | Query | 0 | NULL | SHOW FULL PROCESSLIST | +----+------+-----------------------+---------+------------------+--------------+-------------+----------------------------------------+ 9 rows in set (0.00 sec) 47
  • Tracing Sessions / Auditing PROCESSLIST states Sleeping - The thread is idle Opening Tables - Trying to open a new table Copying to tmp table - Creating in memory temporary table Copying to tmp table on disk - Creating disk based temporary table Sorting for group - Performing a GROUP BY operation Sorting for order - Performing an ORDER BY operation Sending Data - Processing and returning SELECT results Updating - Currently updating row values Writing to net - Server is writing packets to the client Locked - The session is currently blocked by another sessions lock(s) Killed - Kill flag set, waiting for next opportunity to release thread 39
  • Tracing Sessions / Auditing General Query Log Records all connections and statements, successful or failed Set with the “log” server variable log=/var/log/mysql-query.log Does incur a slight performance overhead [markleith@medusa:~] $ sudo tail -n 15 mysql/mysql-5.1.7/data/general.log Password: ./mysql-5.1.7/bin/mysqld-debug, Version: 5.1.7-beta-debug-log. started with: Tcp port: 5017 Unix socket: /Users/markleith/mysql/mysql-5.1.7.sock Time Id Command Argument 060419 11:58:49 1 Connect root@localhost on 060419 11:58:57 1 Query SELECT DATABASE() 1 Init DB sakila 060419 11:59:00 1 Query show tables 060419 11:59:19 1 Query desc staff 060419 12:00:04 1 Query select staff_id, first_name, last_name, email, username, password from staff 49
  • Tracing Sessions / Auditing New in 5.1 (True V$SESSION, V$SQLTEXT, sys.aud$) INFORMATION_SCHEMA.PROCESSLIST mysql.general_log - Set with “--log-output=TABLE,FILE” (or either) mysql.slow_log - Set with “--log-output=TABLE,FILE” (or either) mysql> SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST; +----+---------+--------------+---------+----------------+---------+--------------+------------------------------------------------------------------------------------+ | ID | USER | HOST | DB | COMMAND | TIME | STATE | INFO | +----+---------+--------------+---------+----------------+---------+--------------+------------------------------------------------------------------------------------+ | 2 | root | localhost | | Query |0 | preparing | SELECT * FROM INFORMATION_SCHEMA.PROCESSLIST | | 1 | root | localhost | sakila | Sleep | 2434 | | | +----+---------+--------------+---------+----------------+---------+--------------+------------------------------------------------------------------------------------+ 2 rows in set (0.00 sec) mysql> SELECT p.id, g.event_time, p.user, p.host, g.command_type, replace(g.argument,'n', '') statement -> FROM information_schema.processlist p -> JOIN mysql.general_log g ON p.id = g.thread_id -> WHERE p.id = 1 AND g.event_time > NOW() - INTERVAL 3 HOUR ORDER BY event_time; +----+----------------------------+------+--------------+----------------+------------------------------------------------------------------------------------------------------------+ | id | event_time | user | host | command_type | statement | +----+----------------------------+------+--------------+----------------+------------------------------------------------------------------------------------------------------------+ | 1 | 2006-04-19 11:58:49 | root | localhost | Connect | root@localhost on | | 1 | 2006-04-19 11:58:57 | root | localhost | Query | SELECT DATABASE() | | 1 | 2006-04-19 11:58:57 | root | localhost | Init DB | sakila | | 1 | 2006-04-19 11:59:00 | root | localhost | Query | show tables | | 1 | 2006-04-19 11:59:19 | root | localhost | Query | desc staff | | 1 | 2006-04-19 12:00:04 | root | localhost | Query | select staff_id, first_name, last_name, email, username, passwordfrom staff | +----+----------------------------+------+--------------+----------------+------------------------------------------------------------------------------------------------------------+ 51
  • Tracing Sessions / Auditing Triggers DML triggers availble [BEFORE | AFTER] [INSERT | UPDATE | DELETE] System triggers such as “LOGON” not yet available Only one trigger of each type allowed (i.e. BEFORE UPDATE) Can perform multiple actions between BEGIN ... END blocks CREATE TABLE audit_payment ( audit_id INT AUTO_INCREMENT PRIMARY KEY, payment_id INT, new_staff_id TINYINT, old_staff_id TINYINT, new_amount DECIMAL(5,2), old_amount DECIMAL(5,2), updated_by VARCHAR(64), update_time DATETIME ); CREATE TRIGGER tr_audit_payment AFTER UPDATE ON payment FOR EACH ROW INSERT INTO audit_payment VALUES (NULL,OLD.payment_id,NEW.staff_id,OLD.staff_id,NEW.amount,OLD.amount,CURRENT_USER(),NOW()); 52
  • Tracing Sessions / Auditing Triggers mysql> select * from payment where payment_id = 16049; +-----------------+-------------------+-----------+-------------+------------+----------------------------+-----------------------------+ | payment_id | customer_id | staff_id | rental_id | amount | payment_date | last_update | +-----------------+-------------------+-----------+-------------+------------+----------------------------+-----------------------------+ | 16049 | 599 |2 | 15725 | 2.99 | 2005-08-23 11:25:00 | 2006-02-15 22:24:13 | +-----------------+-------------------+-----------+-------------+------------+----------------------------+-----------------------------+ 1 row in set (0.00 sec) mysql> UPDATE payment SET amount = 1.99 WHERE payment_id = 16049; Query OK, 1 row affected (0.03 sec) Rows matched: 1 Changed: 1 Warnings: 0 mysql> select * from audit_paymentG *************************** 1. row *************************** audit_id: 1 payment_id: 16049 new_staff_id: 2 old_staff_id: 2 new_amount: 1.99 old_amount: 2.99 updated_by: root@localhost update_time: 2006-04-19 13:33:48 1 row in set (0.00 sec) 53
  • Monitoring Tools MySQL Network 2.0 Server Overview 54
  • Monitoring Tools Alert overview 55
  • Monitoring Tools Alert History Graphing/Trending 56
  • Connecting Oracle to MySQL Heterogeneous Services Connects Oracle to MySQL via MyODBC Can create database links from Oracle Oracle -> MySQL only Install and configure MyODBC Create and edit $ORACLE_HOME/hs/admin and add: /* Replace quot;MySQLquot; below with your DSN specified within your MyODBC setup */ HS_FDS_CONNECT_INFO = MySQL HS_FDS_TRACE_LEVEL = OFF Edit $ORACLE_HOME/network/admin/listener.ora and add: (SID_DESC = (PROGRAM = hsodbc) (ORACLE_HOME = oracle/product/92) /* Your $ORACLE_HOME */ (SID_NAME = MySQL) /* Your DSN */ ) 57
  • Connecting Oracle to MySQL Heterogeneous Services Add the following to the tnsnames.ora file: MYSQL = (DESCRIPTION= (ADDRESS=(PROTOCOL=tcp)(HOST=localhost)(PORT=1521)) (CONNECT_DATA=(SID=MYSQL)) (HS=OK) ) Reload your Oracle listener (lsnrctl reload) and then log onto to the Oracle database. To set up the database link: CREATE DATABASE LINK mysql CONNECT TO quot;my_userquot; IDENTIFIED BY quot;my_passwordquot; USING 'mysql'; 58
  • Connecting Oracle to MySQL Heterogeneous Services SQL> COUNT(*) FROM film@mysql; COUNT(*) ---------- 1000 SQL> DESC film@mysql5; Name Null? Type ----------------------------------------- -------- ---------------------------- film_id NUMBER(10) category_id NOT NULL NUMBER(10) title NOT NULL VARCHAR2(27) description LONG rental_duration NOT NULL NUMBER(3) length NUMBER(10) rating CHAR(5) SQL>INSERT INTO film@mysql5 VALUES (1000000,1,'test','test',1,1,'PG'); 1 row created. ---change prompts--- mysql> USE sakila mysql> SELECT max(film_id) FROM film; +------------------+ | max(film_id) | +------------------+ | 1000000 | +------------------+ 59
  • Questions? 60