• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
MySQL 5.5

MySQL 5.5



A look at the features coming up with MySQL 5.5

A look at the features coming up with MySQL 5.5



Total Views
Views on SlideShare
Embed Views



9 Embeds 219

http://www.mysqlpub.com 157
http://www.slideshare.net 31
http://phpprotip.com 20
http://mysqlpub.com 4
http://www.planet-php.net 3
http://www.lmodules.com 1
http://www.php-talks.com 1
http://www.linkedin.com 1
http://51mariadb.com 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • In linked in and twitter
  • bullet 2 - fully synchronous replication means that the master will commit a transaction and *all* the slaves will have to also commit the transaction for the master to return to the session as completed. Think about that - fully synchronous means that the commit must be done - everywhere... that can take a while if a slave is on a slow network. We are stuck waiting for the slowest server for *every* commit. implemented using plugins. only for Linux. Other platforms are is not yet supported.
  • Note: InnoDB has recently added a wealth of information on it’s internal state to the Performance Schema. bullet 6: While the server behavior does not change, for InnoDB it does introduce some performance overhead so be sure to test. Examples of information available: o Mutexes in the MUTEX_INSTANCES table. o RW-locks in the RWLOCK_INSTANCES table. o File I/O operations in the FILE_INSTANCES, FILE_SUMMARY_BY_EVENT_NAME, and FILE_SUMMARY_BY_INSTANCE tables. o Threads in the PROCESSLIST table.
  • if pval = 0 SQLSTATE values that begin with '01 == warning'. warning does not terminate the procedure, and can be seen with SHOW WARNINGS if pval = 1, an error ('45000', which means “unhandled user-defined exception.” ) , sets the MESSAGE_TEXT condition information item. error * terminates* the procedure, and the text is returned with the error information. if pval = 2 SQLSTATE value is specified using a named condition in this case. (DECLARE) A gotcha - SIGNAL/RESIGNAL can only use a CONDITION defined with SQLSTATE - MySQL error number (4 digits) does not count. anything else first signals a warning (SQLSTATE begins with “01”) and sets the message text and error number condition information items. warning does not terminate the procedure, so execution continues then signals an error. error does * terminate* the procedure. message text and error number set by the warning are replaced by the values set by the error, which are returned with the error information.
  • Note: Catch it - do something to it, then rethrow it
  • Easier just to think it rewrites the error
  • Expands on Range, List, Column, Hash and Key Partitioning bullet 2 - supposedly partition pruning works better with these enhancements. bullet 3 - in 5.1 partitioning could only work with integer values. so dates had to be converted with functions Background: RANGE partitioning - assigns rows to partitions based on column values falling within a given range. LIST partitioning - Similar to RANGE , but is selected based on columns matching one of a set of discrete values. HASH partitioning - selected based on the value returned by a user-defined expression that operates on column values in rows to be inserted into the table. The function may consist of any expression valid in MySQL that yields a nonnegative integer value. KEY partitioning - similar to HASH , except that only one or more columns to be evaluated are supplied, and the MySQL server provides its own hashing function. These columns can contain other than integer values, since the hashing function supplied by MySQL guarantees an integer result regardless of the column data type.
  • - similar to RANGE partitioning, but allows you to define partitions using ranges based on multiple column values bullet 2 sub-bullet 2: partitions are based on comparisons between tuples (lists of column values) rather than comparisons between scalar values. Placement of rows in RANGE COLUMNS partitions is also based on comparisons between tuples. bullet 2 sub-bullet 3: string, DATE and DATE TIME col umns can also be used as partitioning columns.
  • - Yes it is kinda Counter intuitive... - The big thing is that you have to think in tuples using the whole value not the individual parts! So (5, 10) is less then (5, 12), and (5, 11) is less then (5, 12). But (5, 12) is not less then (5, 12) - Mathematically: (a < 5) OR ((a = 5) AND ( b < 12))
  • Suppose that you have a business that has customers in 12 cities which, for sales and marketing purposes, you organize into 4 regions of 3 cities each With LIST COLUMNS partitioning, you can create a table for customer data that assigns a row to any of 4 partitions corresponding to these regions based on the name of the city where a customer resides
  • NOTE: InnoDB will be the default storage engine at this point
  • MUTEX Definition: I nformal abbreviation for “mutex variable”. (Mutex itself is short for “mutual exclusion”.) Used to enforce exclusive-access locks to internal in-memory data structures. Once the lock is acquired, any other process, thread, and so on is prevented from acquiring the same lock. Contrast with rw-locks, which allow shared access. Mutexes and rw-locks are known collectively as latches.
  • bullet 2 - leading high-performance memory allocators include Hoard, libumem, mtmalloc, ptmalloc, tbbmalloc, and TCMalloc bullet 3 - especially those where memory is frequently allocated and released (such as multi-table joins) bullet 4 - you control whether InnoDB uses its own memory allocator or an allocator of the operating system - 1.0.3
  • bullet 1: example: applications with a high volume of DML operations such as bulk inserts. sub-bullet 1: when the relevant page is *not* in the buffer pool. This saves on IO since it will not read in the row that is to be changed from disk bullet 2, sub-bullet 2: more data will be held that needs to be purged. This can be mitigated with a more aggressive innodb_max_purge_lag setting and innodb_adaptive_flushing. Also through the purge thread we will discuss a little later. bullet 3: 1.0.3 for inserts, 1.1 for deletes and purges sub-bullet 1: all - The default value: buffer inserts, delete-marking operations, and purges. none - Do not buffer any operations. inserts - Buffer insert operations. deletes - Buffer delete-marking operations. changes - Buffer both inserts and delete-marking. purges - Buffer the physical deletion operations that happen in the background.
  • bullet 1: in anticipation of being needed soon bullet 2: Random was removed since it often resulted in performance degradation. bullet 3: If InnoDB reads at least innodb_read_ahead_threshold pages sequentially from an extent (64 pages), it initiates an asynchronous read for the entire following extent. bullet 3 sub-bullet 1: The default value is 56. bullet 4: smaller ‘evicted’ is to ‘ahead’ the better - Innodb_buffer_pool_read_ahead_evicted - the number of such pages evicted without ever being accessed - Innodb_buffer_pool_read_ahead - the number of pages read in as part of read ahead
  • bullet 1: can become a bottleneck. serialize events and doesn’t take full advantage of multi-core bullet 2: default value for these parameters is 4 and the permissible values range from 1-64. - won't do *any* work on it's own. When other threads submit requests, it helps complete those requests. bullet 3: Each background thread can handle up to 256 pending I/O requests. A major source of background I/O is the read ahead requests. - allows for more parallelism to handle the background IO
  • bullet 1 sub-bullet 2: purge can be moved to its own thread now on the new version. Get into that in a little bit.. bullet 2: not directly related to the background IO threads we discussed earlier
  • Def purge: a type of garbage collection - removes obsolete values from indexes; physically removing rows that were marked for deletion by previous DELETE statements. bullet 1: So we have one thread handling/scheduling multiple operations. Can become overwhelmed since the master thread is also handing flushing the dirty pages
  • Def dirty pages: those pages that have been changed but are not yet written to the database files Def hard check point: flushing the appropriate dirty pages to make space available in the log file bullet 1 sub-bullet 3: problem can occur in a workload that is write intensive and it generates a lot of redo information (writes to the log file). Log file can fill before the dirty_pages_pct is reached. This will cause a hard checkpoint === bad === temporary reduction in throughput while it flushing everything. bullet 2 sub-bullet 2: Internal benchmarking has also shown that this algorithm not only maintains throughput over time, but can also improve overall throughput significantly bullet 2 sub-bullet 3: With it On, it can significantly affect the I/O pattern of a workload. Default is for it to be On.
  • Def spin loop: while waiting for a lock, InnoDB executes a tight loop of instructions repeatedly to avoid having the InnoDB process and threads be rescheduled by the operating system until hopefully the lock becomes available.
  • bullet 1 sub-bullet 1: due to improved algorithms for memory management
  • LRU: Least Recently Used Def innodb_old_blocks_time: Dynamic setting; specifies the time window (in milliseconds) after the first access to a page during which it can be accessed without being moved to the front (most-recently used end) of the LRU list. Default value is 0 corresponding to the original behavior Def innodb_old_blocks_pct: Dynamic setting; controls the percentage of “old” blocks in the LRU list. The default value of innodb_old_blocks_pct is 37, corresponding to the original fixed ratio of 3/8. bullet 1: Without going into details it is possible to push all of the frequently used data out of the buffer pool during large scans (mysqldump, full table scans or large range scans for reports, etc.). This can cause the other/normal/online queries to pushed out of the buffer pool. bullet 2: without going into detail for why it happens (see me later if you really want to know why) there are some new setting to help prevent this from happening * History:Historically, InnoDB has inserted newly read blocks into the middle of the list representing the buffer cache, to avoid pollution of the cache due to excessive read-ahead. The idea is that the read-ahead algorithm should not pollute the buffer cache by forcing the frequently accessed (“hot”) pages out of the LRU list. To achieve this, InnoDB internally maintains a pointer at 3/8 from the tail of the LRU list, and all newly read pages are inserted at this location in the LRU list. The pages are moved to the front of the list (the most-recently used end) when they are accessed from the buffer cache for the *first time*. Thus pages that are never accessed never make it to the front 5/8 of the LRU list.
  • bullet 2: Ex: buffer pool mutex, LRUs bullet 3: uses a hashing function bullet 5: number of buffer pools. 1 (the default) to 64 (the maximum). Takes effect when you set the innodb_buffer_pool_size to a size of 1 gigabyte or more bullet 6: recommend specifying a combination of innodb_buffer_pool_instances and innodb_buffer_pool_size so that each buffer pool instance is a least 1 gigabyte.
  • Note: If you run 5.5.4 on a database, then roll back to some older release, the change still takes effect. bullet 2: - The single rollback segment is divided into 131,072 (128K) segments, - each of which can support up to 1023 transactions that perform writes. - Each transaction is assigned to one of the rollback segments using a hashing function, and remains tied to that rollback segment for the duration . bullet 4 : InnoDB makes the required changes inside the system tablespace automatically, the first time you restart after performing a slow shutdown.
  • Def LSN: log sequence number - an ever-increasing value representing a point in time corresponding to operations recorded in the redo log. It is used internally by InnoDB during crash recovery and for managing the buffer pool. bullet 1: so writing to a log could block access to the pages in the buffer pool that are changed with a mini-transaction commit. These 2 actions have to be scheduled serially. Allows more parallelism. bullet 2: the new log_buf mutex controls writes to buffer pool pages due to mini-transactions.

MySQL 5.5 MySQL 5.5 Presentation Transcript

  • MySQL 5.5 and Scalability
    • Ligaya Turmelle
    • Senior Technical Support Engineer - MySQL
    • [email_address]
    • http://joind.in/1584
    <Insert Picture Here>
  • The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  • Areas to Cover
    • Semi-Synchronous Replication
    • Performance Schema
    • More Partitioning Options
    • InnoDB - LOTS of InnoDB!
      • Performance Improvements
      • Scalability Improvements
    <Insert Picture Here>
  • Semi-Synchronous Replication http://dev.mysql.com/doc/refman/5.5/en/replication-semisync.html <Insert Picture Here>
  • History
    • By default is asynchronous
      • Master writes to binary log
      • Slave connects and “pulls” contents of the binary log
    • Master crashes
      • No guarantee that a slave has all committed transactions
      • Potential lost transactions
    • Failover to slave
      • May be missing transactions from master
      • Loss of data integrity
  • What is Semi-Synchronous Replication?
    • Alternative to asynchronous replication
    • Midway point between asynchronous and fully synchronous
    • Doesn’t have to wait for all the slaves to *receive* the event
      • The master blocks, waiting for a single slave to acknowledge the commit or timeout
    • Doesn’t have to wait for all slaves to *commit* the transaction
      • The slave acknowledges after it receives all the events for the transaction, write it to its relay logs, and flush to disk
    • Provides improved data integrity at the cost of performance
  • How does it works
    • Best Case
      • COMMIT
      • Master blocks waiting for acknowledgment
      • On acknowledgement the master returns to the session
    • Worst Case
      • COMMIT
      • Master blocks
      • Timeout
      • Master reverts to asynchronous mode until at least one semi-synchronous slave catches up
      • Revert back to semi-sychronous mode
  • Performance Schema http://dev.mysql.com/doc/refman/5.5/en/performance-schema.html <Insert Picture Here>
  • Basic Info
    • Monitoring MySQL Server execution at a *very* low level.
      • Used to inspect internal execution of the server at run time
    • Focuses on performance data
    • Uses the Performance Schema storage engine and the Performance_Schema database
    • Tables in Performance_Schema are views or temporary tables.
    • Available on all platforms but some limitations might apply
    • Activation causes no change in server behavior
    • Average user has no need to mess with it - designed for advanced users
  • SIGNAL/RESIGNAL http://dev.mysql.com/doc/refman/5.5/en/signal-resignal.html <Insert Picture Here>
    • Way to “return” an error.
    • AKA exception-handling logic for stored procedures, stored functions, triggers, events, and database applications - think throw
    • Allows you to control error number, SQLSTATE value, message
    • Can indicate errors, warnings, or “not found.”
    • Requires no special permissions
  • CREATE PROCEDURE p (pval INT) BEGIN DECLARE specialty CONDITION FOR SQLSTATE '45000'; IF pval = 0 THEN SIGNAL SQLSTATE '01000'; ELSEIF pval = 1 THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT = 'An error occurred'; ELSEIF pval = 2 THEN SIGNAL specialty SET MESSAGE_TEXT = 'An error occurred'; ELSE SIGNAL SQLSTATE '01000' SET MESSAGE_TEXT = 'A warning occurred', MYSQL_ERRNO = 1000; SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT = 'An error occurred', MYSQL_ERRNO = 1001; END IF; END;
    • passes on error condition information available during execution of a handler.
    • Requires an active handler to execute
    • Possible to both handle an error and return the error information. Think catch/throw block.
    • May change some or all information before passing it on.
    • No special permissions needed to use
    • Multiple possible uses
      • Alone
      • With new signal information
      • With a condition value and possibly new signal information
    • Otherwise follows the rules of SIGNAL
  • RESIGNAL alone
    • Pass on error with no change
    • Allows for the performance of additional actions in the active handler, and then passing on the error message without changing the original condition information
  • RESIGNAL with New Signal Information
    • Pass on error with changes
    • Allows for the performance of additional actions in the active handler, and can change any or all of the signal information items
  • RESIGNAL with a Condition Value and Possibly New Signal Information
    • Push a new “condition” into the current error stack
    • Similar effects of previous example, except instead of rewriting the signal information it adds a new signal to the top of the error stack
  • Column Partitioning Options http://dev.mysql.com/doc/refman/5.5/en/partitioning-columns.html <Insert Picture Here>
  • Column Partitioning
    • Variants on RANGE and LIST partitioning
    • Allows the use of multiple columns in partitioning keys
      • All columns taken into account
        • for placing rows in partitions
        • for use in partition pruning.
    • Support the use of non-integer columns
      • All integer types
      • DATE and DATETIME
    • Great article at http://dev.mysql.com/tech-resources/articles/mysql_55_partitioning.html
  • Range Column
    • Allows for multiple column values
    • Major differences from just RANGE
      • Does not accept expressions, only names of columns
      • Accepts a list of one or more columns
      • Not restricted to integer columns
    • Go to the example
  • CREATE TABLE rc1 (a INT, b INT) PARTITION BY RANGE COLUMNS(a, b) ( PARTITION p0 VALUES LESS THAN (5, 12), PARTITION p3 VALUES LESS THAN (MAXVALUE, MAXVALUE) ); mysql> INSERT INTO rc1 VALUES (5,10), (5,11), (5,12); Query OK, 3 rows affected (0.00 sec) Records: 3 Duplicates: 0 Warnings: 0 mysql> SELECT PARTITION_NAME,TABLE_ROWS -> FROM INFORMATION_SCHEMA.PARTITIONS -> WHERE TABLE_NAME = 'rc1'; +--------------+----------------+------------+ | TABLE_SCHEMA | PARTITION_NAME | TABLE_ROWS | +--------------+----------------+------------+ | p | p0 | 2 | | p | p1 | 1 | +--------------+----------------+------------+
  • List Column
    • Again allows for multiple column values
    • Allows you to use
      • string types
      • DATE ,
      • DATETIME
    • Do not need to convert values to integers to work with
    • Much easier to read
    • Go to the example
  • CREATE TABLE customers_1 ( first_name VARCHAR(25), last_name VARCHAR(25), street_1 VARCHAR(30), city VARCHAR(15), renewal DATE ) PARTITION BY LIST COLUMNS(city) ( PARTITION NorthEast VALUES IN('Boston', 'New York', 'Providence'), PARTITION South VALUES IN('Miami', 'Atlanta', 'New Orleans'), PARTITION MidWest VALUES IN('Chicago', 'Houston', 'Denver'), PARTITION West VALUES IN('Los Angeles', 'San Francisco', 'Seattle') );
  • InnoDB http://dev.mysql.com/doc/innodb-plugin/1.1/en/innodb-performance.html <Insert Picture Here>
  • System Mutex Library
    • Originally used its own implementation of mutexes and read/write locks with Pthreads
    • From 1.0.3, changed to the GNU Compiler Collection (GCC) for atomic memory access instead of using the Pthreads approach previously used.
    • Faster more efficient locking
    • Greater scalability on multi-core platforms.
    • Enabled out of the box on platforms where supported
    • Freebie
  • Memory Allocator
    • History:
      • Originally built its own and protected by a single mutex
      • Also included a wrapper around the system allocator (malloc and free)
        • also protected with a single mutex
      • All bad for multi-core computers
    • Great improvements have occurred since then
      • Perform better and are more scalable than they were in the past
    • Most workloads benefit from using the new schedulers.
    • innodb_use_sys_malloc
  • Change Buffer
    • Most relevant to IO bound systems
      • Will cache changes to secondary index entries
      • Can buffer inserts, delete-marking operations, and purges
    • But...
      • Resides in buffer pool - reducing memory to cache data
      • Increases risk of a purge hold
    • Can control whether InnoDB performs this buffering using innodb_change_buffering.
      • Multiple values available to buffer only what you want
  • Read Ahead
    • Prefetch multiple pages in the buffer cache asynchronously
    • Only the Linear algorithm now
      • Based on access pattern of pages, not just their number
    • Can now control when InnoDB performs a read-ahead operation with innodb_read_ahead_threshold
      • Higher the value, the more strict the access pattern
    • Efficiency can be found with Innodb_buffer_pool_read_ahead_evicted < Innodb_buffer_pool_read_ahead
  • Background IO Threads
    • History
      • 1 read, 1 write, 1 log, 1 insert buffer (non-Windows)
      • Windows used innodb_file_io_threads (deprecated)
    • Now configurable
      • Number of background *helper* threads tasked with servicing any read and write I/O on the data pages
    • Makes InnoDB more scalable on high end systems
    • Controls the IO capacity of the InnoDB Master thread.
      • It controls how many dirty pages are flushed and the size of the ibuf contraction
      • Also handles the purge but...
    • Should be set to approximately the number of I/O operations that the system can perform per second
  • Improved Purge Scheduling
    • History
      • One of the operations handled by the Master Thread
    • Now moved to its own thread
      • separates main database operations from maintenance work happening in the background.
    • Improves scalability
    • May not see improvement on speed since it may hit new types of contention
      • lays groundwork for further tuning and potentially more then one purge thread.
    • Enable with innodb_purge_threads=1
  • Adaptive Dirty Page Flushing
    • History
      • Performed in background
      • Performed by master thread
      • Currently will aggressively flush buffer pool pages if innodb_max_dirty_pages_pct percentage exceeded
    • New algorithm
      • Intent is to smooth overall performance, eliminating steep dips in throughput
      • Function of the number of dirty pages in the buffer cache and the rate at which redo is being generated
        • self adapting to sudden changes in workload
        • better performance
      • Turn on and off with innodb_adaptive_flushing
  • PAUSE in spin loops
    • Most people won’t care about this
    • But... if you are CPU bound - you will
      • Increases overall performance with CPU-bound workloads
      • Added benefit of minimizing power consumption during the execution of the spin loops.
    • Ugly details:
      • If the spin loops are executed too quickly, system resources are wasted, imposing a relatively severe penalty on transaction throughput
      • Uses a PAUSE instruction in its spin loops on all platforms where such an instruction is available.
    • Nothing needs to be done to take advantage of this
  • Faster Recovery
    • A number of optimizations speed up certain steps of the recovery that happens on the next startup after a crash.
      • Particularly scanning the redo log and applying the redo log are faster.
    • If you kept the size of your logfiles artificially low because recovery took a long time (1 hour to 1 day), you can consider increasing the logfile size.
    • No action needed to get this
  • Buffer Cache - Scan Resistant
    • Problem: with large scans it is possible to move all your “hot” data out of the buffer pool.
      • More IO loading the hot data back into the buffer pool
      • Slow response times
    • How to prevent it:
      • Mixed OLTP type with periodic batch reporting queries which result in large scans
        • set innodb_old_blocks_time for the duration of the batch run
      • Scanning large tables that cannot fit entirely in the buffer pool
        • set innodb_old_blocks_pct to a small value
    • Always benchmark to verify the effectiveness
  • Multiple Buffer Pools
    • Primarily useful for people with a large buffer pool size
    • Reduce the chances of encountering bottlenecks trying to access the same buffer pool
    • Pages are stored in or read from random buffer pools
    • Each buffer pool manages its own internal structures and is protected by its own buffer pool mutex
    • innodb_buffer_pool_instances
    • innodb_buffer_pool_size is divided up among all the buffer pools
  • Multiple Rollback Segments
    • History:
      • A bottleneck on high-capacity systems - currently it can handle a maximum of 1023 concurrent transactions that change any data
    • Limit on concurrent transactions is greatly expanded
    • Improves
      • Scalability - higher number of concurrent transactions
      • Performance - less contention when different threads access the rollback segments
    • To enable
      • Do a slow shutdown (innodb_fast_shutdown=0) before upgrading
  • Log Sys Mutex
    • Historically did double duty:
      • Controlled access to internal data structures related to log records and the LSN
      • Controlled access to pages in the buffer pool that are changed when a mini-transaction is committed.
    • Now broken out into separate mutexs
    • Improves performance
    • Nothing to enable - it is free
  • Questions?