Inno db datafiles backup and retore


Published on

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Inno db datafiles backup and retore

  1. 1. Index:A. The INNODB Storage EngineB. Backing Up and Recovering an InnoDBDatabaseC. How Checkpoint Processing WorksD. Moving or Copying InnoDB Tables to Another MachineE. MySQL 5.5 supported storage engines
  2. 2. (A)The InnoDB Storage Engine:InnoDB is a high-reliability and high-performance storage engine for MySQL.Starting with MySQL 5.5, it is the default MySQL storage engine. Key advantagesof InnoDB include:Its design follows the ACID model, with transactions featuring commit,rollback, and crash-recovery capabilities to protect user data.Row-level locking and Oracle-style consistent reads increase multi-userconcurrency and performance.InnoDB tables arrange your data on disk to optimize common queries basedon primary keys. Each InnoDB table has a primary key index called theclustered index that organizes the data to minimize I/O for primary keylookups.To maintain data integrity, InnoDB also supports FOREIGN KEY referential-integrity constraints.You can freely mix InnoDB tables with tables from other MySQL storageengines, even within the same statement. For example, you can use a joinoperation to combine data from InnoDB and MEMORY tables in a singlequery.To determine whether your server supports InnoDB use the SHOW ENGINESstatement.
  3. 3. (B)Backing Up and Recovering an InnoDBDatabase :(1) The InnoDB Recovery Process(2) Forcing InnoDB Recovery(3) InnoDB CheckpointsThe key to safe database management is making regular backups. Depending onyour data volume, number of MySQL servers, and database workload, you can usethese techniques, alone or in combination: Hot backupwith MySQL Enterprise Backup; Cold backup by copying files while the MySQL server is shut down; physical backup for fast operation (especially for restore); logical backup with mysqldumpfor smaller data volumes or to record thestructure of schema objects.Hot Backups :The mysqlbackup command, part of the MySQL Enterprise Backup component,lets you back up a running MySQL instance, including InnoDB and MyISAM tables,with minimal disruption to operations while producing a consistent snapshot ofthe database. When mysqlbackup is copying InnoDB tables, reads and writes toboth InnoDB and MyISAM tables can continue. During the copying of MyISAMtables, reads (but not writes) to those tables are permitted. MySQL EnterpriseBackup can also create compressed backup files, and back up subsets of tablesand databases. In conjunction with MySQL’s binary log, users can perform point-in-time recovery. MySQL Enterprise Backup is part of the MySQL Enterprisesubscription. For more details, (see “MySQL Enterprise Backup”)Cold Backups :If you can shut down your MySQL server, you can make a binary backup thatconsists of all files used by InnoDBto manage its tables. Use the followingprocedure: Do a slow shutdown of the MySQL server and make sure that it stopswithout errors. Copy all InnoDB data files (ibdata files and .ibd files) into a safe place. Copy all the .frm files for InnoDBtables to a safe place. Copy all InnoDB log files (ib_logfile files) to a safe place.
  4. 4.  Copy your my.cnf configuration file or files to a safe place.Alternative Backup Types :In addition to making binary backups as just described, regularly make dumps ofyour tables with mysqldump. A binary file might be corrupted without younoticing it. Dumped tables are stored into text files that are human-readable, sospotting table corruption becomes easier. Also, because the format is simpler, thechance for serious data corruption is smaller. mysqldump also has a --single-transaction option for making a consistent snapshot without locking out otherclients. (See Section 7.3.1, “Establishing a Backup Policy”)Replication works with InnoDB tables, so you can use MySQL replicationcapabilities to keep a copy of your database at database sites requiring highavailability.Performing Recovery :To recover your InnoDB database to the present from the time at which thebinary backup was made, you must run your MySQL server with binary loggingturned on, even before taking the backup. To achieve point-in-time recovery afterrestoring a backup, you can apply changes from the binary log that occurred afterthe backup was made. See (Section 7.5, “Point-in-Time (Incremental) RecoveryUsing the Binary Log”)To recover from a crash of your MySQL server, the onlyrequirement is to restart it. InnoDB automatically checks the logs and performs aroll-forward of the database to the present. InnoDB automatically rolls backuncommitted transactions that were present at the time of the crash. Duringrecovery, mysqld displays output something like this:InnoDB: Database was not shut down normally.InnoDB: Starting recovery from log files...InnoDB: Starting log scan based on checkpoint atInnoDB: log sequence number 0 13674004InnoDB: Doing recovery: scanned up to log sequence number 0 13739520InnoDB: Doing recovery: scanned up to log sequence number 0 13805056InnoDB: Doing recovery: scanned up to log sequence number 0 13870592InnoDB: Doing recovery: scanned up to log sequence number 0 13936128InnoDB: Doing recovery: scanned up to log sequence number 0 20555264InnoDB: Doing recovery: scanned up to log sequence number 0 20620800InnoDB: Doing recovery: scanned up to log sequence number 0 20664692
  5. 5. InnoDB: 1 uncommitted transaction(s) which must be rolled backInnoDB: Starting rollback of uncommitted transactionsInnoDB: Rolling back trx no 16745InnoDB: Rolling back of trx no 16745 completedInnoDB: Rollback of uncommitted transactions completedInnoDB: Starting an apply batch of log records to the database...InnoDB: Apply batch completedInnoDB: Startedmysqld: ready for connectionsIf your database becomes corrupted or disk failure occurs, you must perform therecovery using a backup. In the case of corruption, first find a backup that is notcorrupted. After restoring the base backup, do a point-in-time recovery from thebinary log files using mysqlbinlog and mysql to restore the changes that occurredafter the backup was made.In some cases of database corruption it is enough just to dump, drop, and re-create one or a few corrupt tables. You can use the CHECK TABLESQL statementto check whether a table is corrupt, although CHECK TABLE naturally cannotdetect every possible kind of corruption. You can use the Table space Monitor tocheck the integrity of the file space management inside the table space files. Insome cases, apparent database page corruption is actually due to the operatingsystem corrupting its own file cache, and the data on disk may be okay. It is bestfirst to try restarting your computer. Doing so may eliminate errors that appearedto be database page corruption.(1) The InnoDB Recovery ProcessInnoDB crash recovery consists of several steps. The first step, redo logapplication, is performed during the initialization, before accepting anyconnections. If all changes were flushed from the buffer pool to the tablespaces(ibdata* and *.ibd files) at the time of the shutdown or crash, the redo logapplication can be skipped. If the redo log files are missing at startup, InnoDBskips the redo log application. The remaining steps after redo log application donot depend on the redo log (other than for logging the writes) and are performedin parallel with normal processing.These include:Rolling back incomplete transactions: Any transactions that were active atthe time of crash or fast shutdown.
  6. 6. Insert buffer merge: Applying changes from the insert buffer tree (from theshared tablespace) to leaf pages of secondary indexes as the index pagesare read to the buffer pool.Purge: Deleting delete-marked records that are no longer visible for anyactive transaction.Of these, only rollback of incomplete transactions is special to crash recovery. Theinsert buffer merge and the purge are performed during normal processing.(2) Forcing InnoDBRecovery :If there is database page corruption, you may want to dump your tables from thedatabase with SELECT ... INTO OUTFILE. Usually, most of the data obtained in thisway is intact. However, it is possible that the corruption might cause SELECT *FROM tbl_name statements or InnoDB background operations to crash or assert,or even cause InnoDB roll-forward recovery to crash. In such cases, you can usethe innodb_force_recovery option to force the InnoDB storage engine to start upwhile preventing background operations from running, so that you can dump yourtables. For example, you can add the following line to the [mysqld] section of youroption file before restarting the server:[mysqld]innodb_force_recovery = 4innodb_force_recovery is 0 by default (normal startup without forced recovery).The permissible nonzero values for innodb_force_recovery follow. A largernumber includes all precautions of smaller numbers. If you can dump your tableswith an option value of at most 4, then you are relatively safe that only some dataon corrupt individual pages is lost.A value of 6 is more drastic because database pages are left in an obsolete state,which in turn may introduce more corruption into B-trees and other databasestructures.1 (SRV_FORCE_IGNORE_CORRUPT)Let the server run even if it detects a corrupt page. Try to make SELECT *FROM tbl_name jump over corrupt index records and pages, which helps indumping tables.2 (SRV_FORCE_NO_BACKGROUND)Prevent the master threadfrom running. If a crash would occur during thepurgeoperation, this recovery value prevents it.
  7. 7. 3 (SRV_FORCE_NO_TRX_UNDO)Do not run transaction rollbacks aftercrash recovery.4 (SRV_FORCE_NO_IBUF_MERGE)Prevent insert buffer merge operations. If they would cause a crash, do notdo them. Do not calculate table statistics.5 (SRV_FORCE_NO_UNDO_LOG_SCAN)Do not look at undo logs when starting the database: InnoDB treats evenincomplete transactions as committed.6 (SRV_FORCE_NO_LOG_REDO)Do not do the redo logroll-forward in connection with recovery. With thisvalue, you might not be able to do queries other than a basic SELECT *FROM t, with no WHERE, ORDER BY, or other clauses. More complexqueries could encounter corrupted data structures and fail.If corruption within the table data prevents you from dumping the entiretable contents, a query with an ORDER BY primary_key DESC clause mightbe able to dump the portion of the table after the corrupted part.The database must not otherwise be used with any nonzero value ofinnodb_force_recoveryAs a safety measure, InnoDB prevents users fromperforming INSERT, UPDATE, or DELETE operations when innodb_force_recoveryis greater than 0.You can SELECT from tables to dump them, or DROP or CREATE tables even ifforced recovery is used. If you know that a given table is causing a crash onrollback, you can drop it. You can also use this to stop a runaway rollback causedby a failing mass import or ALTER TABLE. You can kill the mysqld process and setinnodb_force_recovery to 3 to bring the database up without the rollback,thenDROP the table that is causing the runaway rollback.(3) InnoDBCheckpoints :Making your log files very large may reduce disk I/O during checkpointing. It oftenmakes sense to set the total size of the log files as large as the buffer pool or evenlarger. Although in the past large log files could make crash recovery takeexcessive time, starting with MySQL 5.5, performance enhancements to crashrecovery make it possible to use large log files with fast startup after a crash.(Strictly speaking, this performance improvement is available for MySQL 5.1 withthe InnoDB Plugin 1.0.7 and higher. It is with MySQL 5.5 and InnoDB 1.1 that thisimprovement is available in the default InnoDB storage engine.)
  8. 8. (C)How Checkpoint Processing WorksInnoDB implements a checkpoint mechanism known as “fuzzy”checkpointing.InnoDB flushes modified database pages from the buffer pool in small batches.There is no need to flush the buffer pool in one single batch, which would inpractice stop processing of user SQL statements during the checkpointing process.During crash recovery, InnoDB looks for a checkpoint label written to the log files.It knows that all modifications to the database before the label are present in thedisk image of the database. Then InnoDB scans the log files forward from thecheckpoint, applying the logged modifications to the database.InnoDB writes to its log files on a rotating basis. It also writes checkpointinformation to the first log file at each checkpoint. All committed modificationsthat make the database pages in the buffer pool different from the images on diskmust be available in the log files in case InnoDB has to do a recovery. This meansthat when InnoDB starts to reuse a log file, it has to make sure that the databasepage images on disk contain the modifications logged in the log file that InnoDB isgoing to reuse. In other words, InnoDB must create a checkpoint and this ofteninvolves flushing of modified database pages to disk.(D)Moving or Copying InnoDB Tables to Another Machine :The various techniques for moving or copying some or all InnoDB tables to adifferent server.For example,You might move an entire MySQL instance to a larger, faster server; you mightclone an entire MySQL instance to a new replication slave server; you might copyindividual tables to another server to development and test an application, or to adata warehouse server to produce reports.On Windows, InnoDB always stores database and table names internally inlowercase. To move databases in a binary format from Unix to Windows or fromWindows to Unix, create all databases and tables using lowercase names. Aconvenient way to accomplish this is to add the following line to the [mysqld]section of your my.cnf or my.ini file before creating any databases or tables:[mysqld]lower_case_table_names=1
  9. 9. Like MyISAM data files, InnoDB data and log files are binary-compatible on allplatforms having the same floating-point number format. You can move anInnoDB database simply by copying all the relevant files listed in (See“Backing Upand Recovering an InnoDB Database” ).If the floating-point formats differ but you have not used FLOATorDOUBLE datatypes in your tables, then the procedure is the same: simply copy the relevantfiles. If you use mysqldump to dump your tables on one machine and then importthe dump files on the other machine, it does not matter whether the formatsdiffer or your tables contain floating-point data.One way to increase performance is to switch off autocommit mode whenimporting data, assuming that the tablespace has enough space for the bigrollback segment that the import transactions generate. Do the commit only afterimporting a whole table or a segment of a table.InnoDB installation path:
  10. 10. mysql>help show binary logsName: SHOW BINARY LOGSDescription:Syntax:SHOW BINARY LOGSSHOW MASTER LOGSLists the binary log files on the server. This statement is used as part of theprocedure described in [purge-binary-logs], that shows howto determine which logs can be purged.mysql> SHOW BINARY LOGS;+---------------+-----------+| Log_name | File_size |+---------------+-----------+| binlog.000015 | 724935 || binlog.000016 | 733481 |+---------------+-----------+
  11. 11. SET sql_log_binSyntax :SET sql_log_bin = {0|1}The sql_log_bin variable controls whether logging to the binary log is done. Thedefault value is 1 (do logging). To change logging for the current session, changethe session value of this variable. The session user must have the SUPER privilegeto set this variable.(E)MySQL 5.5 supported storage engines:InnoDB: A transaction-safe (ACID compliant) storage engine for MySQL thathas commit, rollback, and crash-recovery capabilities to protect user data.InnoDB row-level locking (without escalation to coarser granularity locks)and Oracle-style consistent nonlocking reads increase multi-userconcurrency and performance. InnoDB stores user data in clustered indexesto reduce I/O for common queries based on primary keys. To maintain dataintegrity, InnoDB also supports FOREIGN KEY referential-integrityconstraints. InnoDB is the default storage engine as of MySQL 5.5.5.MyISAM: The MySQL storage engine that is used the most in Web, datawarehousing, and other application environments. MyISAM is supported inall MySQL configurations, and is the default storage engine prior to MySQL5.5.5.Memory: Stores all data in RAM for extremely fast access in environmentsthat require quick lookups of reference and other like data. This engine wasformerly known as the HEAP engine.
  12. 12. Merge: Enables a MySQL DBA or developer to logically group a series ofidentical MyISAM tables and reference them as one object. Good for VLDBenvironments such as data warehousing.Archive: Provides the perfect solution for storing and retrieving largeamounts of seldom-referenced historical, archived, or security auditinformation.Federated: Offers the ability to link separate MySQL servers to create onelogical database from many physical servers. Very good for distributed ordata mart environments.NDB (also known as NDBCLUSTER)—This clustered database engine isparticularly suited for applications that require the highest possible degreeof uptime and availability.