SlideShare a Scribd company logo
1 of 42
Recovery of lost or corrupted InnoDB tables MySQL User Conference 2010, Santa Clara [email_address] Percona Inc. http://MySQLPerformanceBlog.com
Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object],- - Three things are certain: Death, taxes and lost data. Guess which has occurred?
1. InnoDB format overview
How MySQL stores data in InnoDB ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
 
How MySQL stores data in InnoDB ,[object Object],     TABLE: name test/site_folders, id 0 119, columns 9, indexes 1, appr.rows 1       COLUMNS: id: DATA_INT len 4 prec 0; name: type 12 len 765 prec 0; sites_count: DATA_INT len 4 prec 0;                            created_at: DATA_INT len 8 prec 0; updated_at: DATA_INT len 8 prec 0;                     DB_ROW_ID: DATA_SYS prtype 256 len 6 prec 0; DB_TRX_ID: DATA_SYS prtype 257 len 6 prec 0;                     DB_ROLL_PTR: DATA_SYS prtype 258 len 7 prec 0;            INDEX: name PRIMARY, id  0 254 , fields 1/7, type 3            root page 271, appr.key vals 1, leaf pages 1, size pages 1            FIELDS:  id DB_TRX_ID DB_ROLL_PTR name sites_count created_at updated_at  mysql> CREATE TABLE innodb_table_monitor(x int) engine=innodb Error log:
InnoDB page format Fil Trailer  Page Directory FREE SPACE USER RECORDS   INFINNUM+SUPREMUM RECORDS PAGE_HEADER FIL HEADER
InnoDB page format Fil Header   the latest archived log file number at the time that  FIL_PAGE_FILE_FLUSH_LSN  was written (in the log)  4  FIL_PAGE_ARCH_LOG_NO   "the file has been flushed to disk at least up to this lsn" (log serial number), valid only on the first page of the file  8  FIL_PAGE_FILE_FLUSH_LSN   current defined types are:  FIL_PAGE_INDEX ,  FIL_PAGE_UNDO_LOG ,  FIL_PAGE_INODE ,  FIL_PAGE_IBUF_FREE_LIST   2  FIL_PAGE_TYPE   log serial number of page's latest log record  8  FIL_PAGE_LSN   offset of next page in key order  4  FIL_PAGE_NEXT   offset of previous page in key order  4  FIL_PAGE_PREV   ordinal page number from start of space  4  FIL_PAGE_OFFSET   4 ID of the space the page is in  4  FIL_PAGE_SPACE   Remarks   Size   Name   Data are stored in  FIL_PAGE_INODE  == 0x03
InnoDB page format Page  Header  "file segment header for the non-leaf pages in a B-tree" (this is irrelevant here)  10  PAGE_BTR_SEG_TOP   "file segment header for the leaf pages in a B-tree" (this is irrelevant here)  10  PAGE_BTR_SEG_LEAF   identifier of the index the page belongs to  8  PAGE_INDEX_ID   level within the index (0 for a leaf page)  2  PAGE_LEVEL   the highest ID of a transaction which might have changed a record on the page (only set for secondary indexes)  8  PAGE_MAX_TRX_ID   number of user records  2  PAGE_N_RECS   number of consecutive inserts in the same direction, e.g. "last 5 were all to the left"  2  PAGE_N_DIRECTION   either  PAGE_LEFT ,  PAGE_RIGHT , or  PAGE_NO_DIRECTION   2  PAGE_DIRECTION   record pointer to the last inserted record  2  PAGE_LAST_INSERT   "number of bytes in deleted records"  2  PAGE_GARBAGE   record pointer to first free record  2  PAGE_FREE   number of heap records; initial value = 2  2  PAGE_N_HEAP   record pointer to first record in heap  2  PAGE_HEAP_TOP   number of directory slots in the Page Directory part; initial value = 2  2  PAGE_N_DIR_SLOTS   Remarks   Size   Name   index_id Highest bit is row format(1 -COMPACT, 0 - REDUNDANT )
InnoDB page format (REDUNDANT) Extra bytes   pointer to next record in page  16 bits  next 16 bits   1 if each Field Start Offsets is 1 byte long (this item is also called the "short" flag)  1 bit  1byte_offs_flag  number of fields in this record, 1 to 1023  10 bits  n_fields  record's order number in heap of index page  13 bits  heap_no  number of records owned by this record  4 bits  n_owned  1 if record is predefined minimum record  1 bit  min_rec_flag  1 if record is deleted  1 bit  deleted_flag  _ORDINAR Y,  _NODE_PTR ,  _INFIMUM ,  _SUPREMUM 2  bit s   record_status Description   Size   Name
InnoDB page format (COMPACT) Extra bytes   a relative pointer to the next record in the page 16 next 16 bits   000=conventional, 001=node pointer (inside B-tree),  010=infimum, 011=supremum, 1xx=reserved 3 record type the order number of this record in the heap of the index page 1 3 heap_no  the number of records owned by this record (this term is explained in page0page.h)  4   n_owned  4 bits used to delete mark a record, and mark a predefined minimum record in alphabetical order 4   Description   Size , bits Name
How to check row format? ,[object Object],[object Object],[object Object]
Rows in an InnoDB page ,[object Object],[object Object],[object Object],[object Object],infimum next supremum 0 100 data... next 101 data... next 102 data... next 103 data... next
Records are saved in insert order ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Row format EXAMPLE: CREATE TABLE ` t1 ` ( ` ID ` int( 11 ) unsigned NOT NULL, ` NAME ` varchar(120), ` N_FIELDS ` int(10), PRIMARY KEY  (`ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 depends on content  Field Contents  6  bytes (5  bytes  if COMPACT format) Extra Bytes  (F*1) or (F*2) bytes  Field Start Offsets  Size   Name
REDUNDANT A row:  (10 , ‘abcdef’, 20 ) 4 6 7 Actualy stored as:  (10 , TRX_ID, PTR_ID, ‘abcdef’, 20 ) 6 4 Field Offsets … . next Extra 6 bytes: 0x00 00 00 0A record_status deleted_flag  min_rec_flag  n_owned  heap_no  n_fields  1byte_offs_flag   Fields ... ... abcdef 0x80 00 00 14
COMPACT A row:  (10 , ‘abcdef’, 20 ) 6 NULLS Actualy stored as:  (10 , TRX_ID, PTR_ID, ‘abcdef’, 20 ) Field Offsets … . next Extra 5 bytes: 0x00 00 00 0A Fields ... ... abcdef 0x80 00 00 14 A bit per NULL-able field
Data types ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
BLOB and other long fields ,[object Object],[object Object],[object Object],[object Object]
2 . Internal system tables SYS_INDEXES and SYS_TABLES
Why are SYS_* tables needed? ,[object Object],[object Object]
How MySQL stores data in InnoDB ,[object Object],[object Object],CREATE TABLE `SYS_INDEXES` ( ` TABLE_ID ` bigint(20) unsigned NOT NULL default '0', ` ID ` bigint(20) unsigned NOT NULL default '0', ` NAME ` varchar(120) default NULL, ` N_FIELDS ` int(10) unsigned default NULL, ` TYPE ` int(10) unsigned default NULL, ` SPACE ` int(10) unsigned default NULL, ` PAGE_NO ` int(10) unsigned default NULL, PRIMARY KEY  (`TABLE_ID`,`ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 CREATE TABLE `SYS_TABLES` ( ` NAME ` varchar(255) NOT NULL default '', ` ID ` bigint(20) unsigned NOT NULL default '0', ` N_COLS ` int(10) unsigned default NULL, ` TYPE ` int(10) unsigned default NULL, ` MIX_ID ` bigint(20) unsigned default NULL, ` MIX_LEN ` int(10) unsigned default NULL, ` CLUSTER_NAME ` varchar(255) default NULL, ` SPACE ` int(10) unsigned default NULL, PRIMARY KEY  (`NAME`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 index_id = 0-3 index_id = 0-1 Name:  PRIMARY GEN_CLUSTER_ID or unique index name
How MySQL stores data in InnoDB NAME   ID  …   SYS_TABLES "archive19/9299_msg_store"  40694  8 1 0 0 NULL 0 SYS_TABLES "archive19/9299_msg_store"  40694  8 1 0 0 NULL 0 SYS_TABLES "archive19/9299_msg_store"  40694  8 1 0 0 NULL 0 TABLE_ID   ID   NAME   … SYS_INDEXES  40694   196389  "PRIMARY" 2 3 0 21031026 SYS_INDEXES  40694   196390 "msg_hash" 1 0 0 21031028 SYS_TABLES SYS_INDEXES Example:
3. InnoDB Primary and Secondary keys
Primary key ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Secondary key ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
4. Typical failure scenarios
Deleted records ,[object Object],[object Object],[object Object],[object Object]
How delete is performed? ,[object Object],[object Object],[object Object]
Dropped table/database ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Corrupted InnoDB tablespace ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Wrong UPDATE statement ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
5. InnoDB recovery tool
Recovery prerequisites ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
How to get CREATE info from .frm files ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
InnoDB recovery tool ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
InnoDB recovery tool server #  ./page_parser -4 -f /var/lib/mysql/ibdata1 Opening file: /var/lib/mysql/ibdata1 Read data from fn=3... Read page #0.. saving it to pages-1259793800/0-18219008/0-00000000.page Read page #1.. saving it to pages-1259793800/0-0/1-00000001.page Read page #2.. saving it to pages-1259793800/4294967295-65535/2-00000002.page Read page #3.. saving it to pages-1259793800/0-0/3-00000003.page page_parser
Page signature check ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
InnoDB recovery tool server #  ./constraints_parser -4 -f pages-1259793800/0-16/51-00000051.page constraints_parser Table structure is defined in  "include/table_defs.h" See HOWTO for details http://code.google.com/p/innodb-tools/wiki/InnodbRecoveryHowto Filters inside table_defs.h are very important
Check InnoDB page before reading recs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Import result ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Questions ? ,[object Object],[object Object],[object Object],[object Object],- - Applause :-)

More Related Content

What's hot

PE102 - a Windows executable format overview (booklet V1)
PE102 - a Windows executable format overview (booklet V1)PE102 - a Windows executable format overview (booklet V1)
PE102 - a Windows executable format overview (booklet V1)Ange Albertini
 
Data Warehouse and Business Intelligence - Recipe 4 - Staging area - how to v...
Data Warehouse and Business Intelligence - Recipe 4 - Staging area - how to v...Data Warehouse and Business Intelligence - Recipe 4 - Staging area - how to v...
Data Warehouse and Business Intelligence - Recipe 4 - Staging area - how to v...Massimo Cenci
 
Pe Format
Pe FormatPe Format
Pe FormatHexxx
 
Online Vegetable Selling project Presentation
Online Vegetable Selling project PresentationOnline Vegetable Selling project Presentation
Online Vegetable Selling project Presentationmayur patel
 
OER UNIT 4 PARTITION
OER UNIT 4 PARTITIONOER UNIT 4 PARTITION
OER UNIT 4 PARTITIONGirija Muscut
 
Using MySQL in a web-scale environment
Using MySQL in a web-scale environmentUsing MySQL in a web-scale environment
Using MySQL in a web-scale environmentDavid Landgren
 

What's hot (8)

PE102 - a Windows executable format overview (booklet V1)
PE102 - a Windows executable format overview (booklet V1)PE102 - a Windows executable format overview (booklet V1)
PE102 - a Windows executable format overview (booklet V1)
 
Data Warehouse and Business Intelligence - Recipe 4 - Staging area - how to v...
Data Warehouse and Business Intelligence - Recipe 4 - Staging area - how to v...Data Warehouse and Business Intelligence - Recipe 4 - Staging area - how to v...
Data Warehouse and Business Intelligence - Recipe 4 - Staging area - how to v...
 
Sql notes
Sql notesSql notes
Sql notes
 
Pe Format
Pe FormatPe Format
Pe Format
 
Online Vegetable Selling project Presentation
Online Vegetable Selling project PresentationOnline Vegetable Selling project Presentation
Online Vegetable Selling project Presentation
 
OER UNIT 4 PARTITION
OER UNIT 4 PARTITIONOER UNIT 4 PARTITION
OER UNIT 4 PARTITION
 
Using MySQL in a web-scale environment
Using MySQL in a web-scale environmentUsing MySQL in a web-scale environment
Using MySQL in a web-scale environment
 
How sqlite works
How sqlite worksHow sqlite works
How sqlite works
 

Viewers also liked

Le E Marketing vu par les E Marketers
Le E Marketing vu par les E MarketersLe E Marketing vu par les E Marketers
Le E Marketing vu par les E MarketersGenaro Bardy
 
Add edit delete in Codeigniter in PHP
Add edit delete in Codeigniter in PHPAdd edit delete in Codeigniter in PHP
Add edit delete in Codeigniter in PHPVineet Kumar Saini
 
WEB MARKETING : Ou comment améliorer sa visibilité grâce à Internet
WEB MARKETING : Ou comment améliorer  sa visibilité grâce à InternetWEB MARKETING : Ou comment améliorer  sa visibilité grâce à Internet
WEB MARKETING : Ou comment améliorer sa visibilité grâce à InternetLilian FOURCADIER
 
E-marketing pour les débutants
E-marketing pour les débutantsE-marketing pour les débutants
E-marketing pour les débutantsStrasWeb
 
PHP Data Objects
PHP Data ObjectsPHP Data Objects
PHP Data ObjectsWez Furlong
 
Emailing comme outil marketing en 2014
Emailing comme outil marketing en 2014Emailing comme outil marketing en 2014
Emailing comme outil marketing en 2014Pierre-François Danse
 
CodeIgniter 2.0.X
CodeIgniter 2.0.XCodeIgniter 2.0.X
CodeIgniter 2.0.XBo-Yi Wu
 
Object Calisthenics Applied to PHP
Object Calisthenics Applied to PHPObject Calisthenics Applied to PHP
Object Calisthenics Applied to PHPGuilherme Blanco
 
Mieux Développer en PHP avec Symfony
Mieux Développer en PHP avec SymfonyMieux Développer en PHP avec Symfony
Mieux Développer en PHP avec SymfonyHugo Hamon
 
The MD5 hashing algorithm
The MD5 hashing algorithmThe MD5 hashing algorithm
The MD5 hashing algorithmBob Landstrom
 
Hierarchical data models in Relational Databases
Hierarchical data models in Relational DatabasesHierarchical data models in Relational Databases
Hierarchical data models in Relational Databasesnavicorevn
 
232 md5-considered-harmful-slides
232 md5-considered-harmful-slides232 md5-considered-harmful-slides
232 md5-considered-harmful-slidesDan Kaminsky
 
Hash Functions, the MD5 Algorithm and the Future (SHA-3)
Hash Functions, the MD5 Algorithm and the Future (SHA-3)Hash Functions, the MD5 Algorithm and the Future (SHA-3)
Hash Functions, the MD5 Algorithm and the Future (SHA-3)Dylan Field
 
Le marketing à la performance en BtoB : quelle stratégie adopter ?
Le marketing à la performance en BtoB : quelle stratégie adopter ?Le marketing à la performance en BtoB : quelle stratégie adopter ?
Le marketing à la performance en BtoB : quelle stratégie adopter ?agence b2b
 
[Support de cours] WebMarketing et communication web - IPAC 2014
[Support de cours] WebMarketing et communication web - IPAC 2014[Support de cours] WebMarketing et communication web - IPAC 2014
[Support de cours] WebMarketing et communication web - IPAC 2014QWEB.ECO
 
Singletons in PHP - Why they are bad and how you can eliminate them from your...
Singletons in PHP - Why they are bad and how you can eliminate them from your...Singletons in PHP - Why they are bad and how you can eliminate them from your...
Singletons in PHP - Why they are bad and how you can eliminate them from your...go_oh
 

Viewers also liked (20)

e-marketing p1c1
e-marketing p1c1e-marketing p1c1
e-marketing p1c1
 
Le E Marketing vu par les E Marketers
Le E Marketing vu par les E MarketersLe E Marketing vu par les E Marketers
Le E Marketing vu par les E Marketers
 
Add edit delete in Codeigniter in PHP
Add edit delete in Codeigniter in PHPAdd edit delete in Codeigniter in PHP
Add edit delete in Codeigniter in PHP
 
WEB MARKETING : Ou comment améliorer sa visibilité grâce à Internet
WEB MARKETING : Ou comment améliorer  sa visibilité grâce à InternetWEB MARKETING : Ou comment améliorer  sa visibilité grâce à Internet
WEB MARKETING : Ou comment améliorer sa visibilité grâce à Internet
 
E-marketing pour les débutants
E-marketing pour les débutantsE-marketing pour les débutants
E-marketing pour les débutants
 
PHP Data Objects
PHP Data ObjectsPHP Data Objects
PHP Data Objects
 
Emailing comme outil marketing en 2014
Emailing comme outil marketing en 2014Emailing comme outil marketing en 2014
Emailing comme outil marketing en 2014
 
CodeIgniter 2.0.X
CodeIgniter 2.0.XCodeIgniter 2.0.X
CodeIgniter 2.0.X
 
Object Calisthenics Applied to PHP
Object Calisthenics Applied to PHPObject Calisthenics Applied to PHP
Object Calisthenics Applied to PHP
 
Mieux Développer en PHP avec Symfony
Mieux Développer en PHP avec SymfonyMieux Développer en PHP avec Symfony
Mieux Développer en PHP avec Symfony
 
PHP 7
PHP 7PHP 7
PHP 7
 
The MD5 hashing algorithm
The MD5 hashing algorithmThe MD5 hashing algorithm
The MD5 hashing algorithm
 
Hierarchical data models in Relational Databases
Hierarchical data models in Relational DatabasesHierarchical data models in Relational Databases
Hierarchical data models in Relational Databases
 
232 md5-considered-harmful-slides
232 md5-considered-harmful-slides232 md5-considered-harmful-slides
232 md5-considered-harmful-slides
 
Hash Functions, the MD5 Algorithm and the Future (SHA-3)
Hash Functions, the MD5 Algorithm and the Future (SHA-3)Hash Functions, the MD5 Algorithm and the Future (SHA-3)
Hash Functions, the MD5 Algorithm and the Future (SHA-3)
 
Le marketing à la performance en BtoB : quelle stratégie adopter ?
Le marketing à la performance en BtoB : quelle stratégie adopter ?Le marketing à la performance en BtoB : quelle stratégie adopter ?
Le marketing à la performance en BtoB : quelle stratégie adopter ?
 
[Support de cours] WebMarketing et communication web - IPAC 2014
[Support de cours] WebMarketing et communication web - IPAC 2014[Support de cours] WebMarketing et communication web - IPAC 2014
[Support de cours] WebMarketing et communication web - IPAC 2014
 
Dbms project.ppt
Dbms project.pptDbms project.ppt
Dbms project.ppt
 
Cours e-marketing
Cours e-marketingCours e-marketing
Cours e-marketing
 
Singletons in PHP - Why they are bad and how you can eliminate them from your...
Singletons in PHP - Why they are bad and how you can eliminate them from your...Singletons in PHP - Why they are bad and how you can eliminate them from your...
Singletons in PHP - Why they are bad and how you can eliminate them from your...
 

Similar to Recovery of lost or corrupted inno db tables(mysql uc 2010)

cPanelCon 2014: InnoDB Anatomy
cPanelCon 2014: InnoDB AnatomycPanelCon 2014: InnoDB Anatomy
cPanelCon 2014: InnoDB AnatomyRyan Robson
 
Data Warehouse and Business Intelligence - Recipe 2
Data Warehouse and Business Intelligence - Recipe 2Data Warehouse and Business Intelligence - Recipe 2
Data Warehouse and Business Intelligence - Recipe 2Massimo Cenci
 
InnoDB: архитектура транзакционного хранилища (Константин Осипов)
InnoDB: архитектура транзакционного хранилища (Константин Осипов)InnoDB: архитектура транзакционного хранилища (Константин Осипов)
InnoDB: архитектура транзакционного хранилища (Константин Осипов)Ontico
 
Page Cache in Linux 2.6.pdf
Page Cache in Linux 2.6.pdfPage Cache in Linux 2.6.pdf
Page Cache in Linux 2.6.pdfycelgemici1
 
OakTable World 2015 - Using XMLType content with the Oracle In-Memory Column...
OakTable World 2015  - Using XMLType content with the Oracle In-Memory Column...OakTable World 2015  - Using XMLType content with the Oracle In-Memory Column...
OakTable World 2015 - Using XMLType content with the Oracle In-Memory Column...Marco Gralike
 
[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...
[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...
[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...Insight Technology, Inc.
 
0104 abap dictionary
0104 abap dictionary0104 abap dictionary
0104 abap dictionaryvkyecc1
 
15 Ways to Kill Your Mysql Application Performance
15 Ways to Kill Your Mysql Application Performance15 Ways to Kill Your Mysql Application Performance
15 Ways to Kill Your Mysql Application Performanceguest9912e5
 
cPanelCon 2015: InnoDB Alchemy
cPanelCon 2015: InnoDB AlchemycPanelCon 2015: InnoDB Alchemy
cPanelCon 2015: InnoDB AlchemyRyan Robson
 
UKOUG Tech14 - Using Database In-Memory Column Store with Complex Datatypes
UKOUG Tech14 - Using Database In-Memory Column Store with Complex DatatypesUKOUG Tech14 - Using Database In-Memory Column Store with Complex Datatypes
UKOUG Tech14 - Using Database In-Memory Column Store with Complex DatatypesMarco Gralike
 
Data Warehouse and Business Intelligence - Recipe 3
Data Warehouse and Business Intelligence - Recipe 3Data Warehouse and Business Intelligence - Recipe 3
Data Warehouse and Business Intelligence - Recipe 3Massimo Cenci
 
Write a task that will perform some of the functions performed by a s.docx
 Write a task that will perform some of the functions performed by a s.docx Write a task that will perform some of the functions performed by a s.docx
Write a task that will perform some of the functions performed by a s.docxajoy21
 
Btree. Explore the heart of PostgreSQL.
Btree. Explore the heart of PostgreSQL. Btree. Explore the heart of PostgreSQL.
Btree. Explore the heart of PostgreSQL. Anastasia Lubennikova
 
How to use Parquet as a basis for ETL and analytics
How to use Parquet as a basis for ETL and analyticsHow to use Parquet as a basis for ETL and analytics
How to use Parquet as a basis for ETL and analyticsJulien Le Dem
 
MySQL innoDB split and merge pages
MySQL innoDB split and merge pagesMySQL innoDB split and merge pages
MySQL innoDB split and merge pagesMarco Tusa
 

Similar to Recovery of lost or corrupted inno db tables(mysql uc 2010) (20)

cPanelCon 2014: InnoDB Anatomy
cPanelCon 2014: InnoDB AnatomycPanelCon 2014: InnoDB Anatomy
cPanelCon 2014: InnoDB Anatomy
 
Data Warehouse and Business Intelligence - Recipe 2
Data Warehouse and Business Intelligence - Recipe 2Data Warehouse and Business Intelligence - Recipe 2
Data Warehouse and Business Intelligence - Recipe 2
 
Optimizando MySQL
Optimizando MySQLOptimizando MySQL
Optimizando MySQL
 
InnoDB: архитектура транзакционного хранилища (Константин Осипов)
InnoDB: архитектура транзакционного хранилища (Константин Осипов)InnoDB: архитектура транзакционного хранилища (Константин Осипов)
InnoDB: архитектура транзакционного хранилища (Константин Осипов)
 
Page Cache in Linux 2.6.pdf
Page Cache in Linux 2.6.pdfPage Cache in Linux 2.6.pdf
Page Cache in Linux 2.6.pdf
 
OakTable World 2015 - Using XMLType content with the Oracle In-Memory Column...
OakTable World 2015  - Using XMLType content with the Oracle In-Memory Column...OakTable World 2015  - Using XMLType content with the Oracle In-Memory Column...
OakTable World 2015 - Using XMLType content with the Oracle In-Memory Column...
 
Mips
MipsMips
Mips
 
[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...
[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...
[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...
 
0104 abap dictionary
0104 abap dictionary0104 abap dictionary
0104 abap dictionary
 
15 Ways to Kill Your Mysql Application Performance
15 Ways to Kill Your Mysql Application Performance15 Ways to Kill Your Mysql Application Performance
15 Ways to Kill Your Mysql Application Performance
 
Explain that explain
Explain that explainExplain that explain
Explain that explain
 
Less08 Schema
Less08 SchemaLess08 Schema
Less08 Schema
 
cPanelCon 2015: InnoDB Alchemy
cPanelCon 2015: InnoDB AlchemycPanelCon 2015: InnoDB Alchemy
cPanelCon 2015: InnoDB Alchemy
 
UKOUG Tech14 - Using Database In-Memory Column Store with Complex Datatypes
UKOUG Tech14 - Using Database In-Memory Column Store with Complex DatatypesUKOUG Tech14 - Using Database In-Memory Column Store with Complex Datatypes
UKOUG Tech14 - Using Database In-Memory Column Store with Complex Datatypes
 
Data Warehouse and Business Intelligence - Recipe 3
Data Warehouse and Business Intelligence - Recipe 3Data Warehouse and Business Intelligence - Recipe 3
Data Warehouse and Business Intelligence - Recipe 3
 
Write a task that will perform some of the functions performed by a s.docx
 Write a task that will perform some of the functions performed by a s.docx Write a task that will perform some of the functions performed by a s.docx
Write a task that will perform some of the functions performed by a s.docx
 
Btree. Explore the heart of PostgreSQL.
Btree. Explore the heart of PostgreSQL. Btree. Explore the heart of PostgreSQL.
Btree. Explore the heart of PostgreSQL.
 
How to use Parquet as a basis for ETL and analytics
How to use Parquet as a basis for ETL and analyticsHow to use Parquet as a basis for ETL and analytics
How to use Parquet as a basis for ETL and analytics
 
MySQL innoDB split and merge pages
MySQL innoDB split and merge pagesMySQL innoDB split and merge pages
MySQL innoDB split and merge pages
 
Implementation
ImplementationImplementation
Implementation
 

Recently uploaded

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdflior mazor
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...apidays
 
A Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusA Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusZilliz
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Zilliz
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...apidays
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesrafiqahmad00786416
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024The Digital Insurer
 

Recently uploaded (20)

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
A Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusA Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source Milvus
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 

Recovery of lost or corrupted inno db tables(mysql uc 2010)

  • 1. Recovery of lost or corrupted InnoDB tables MySQL User Conference 2010, Santa Clara [email_address] Percona Inc. http://MySQLPerformanceBlog.com
  • 2.
  • 3. 1. InnoDB format overview
  • 4.
  • 5.  
  • 6.
  • 7. InnoDB page format Fil Trailer Page Directory FREE SPACE USER RECORDS INFINNUM+SUPREMUM RECORDS PAGE_HEADER FIL HEADER
  • 8. InnoDB page format Fil Header the latest archived log file number at the time that FIL_PAGE_FILE_FLUSH_LSN was written (in the log) 4 FIL_PAGE_ARCH_LOG_NO "the file has been flushed to disk at least up to this lsn" (log serial number), valid only on the first page of the file 8 FIL_PAGE_FILE_FLUSH_LSN current defined types are: FIL_PAGE_INDEX , FIL_PAGE_UNDO_LOG , FIL_PAGE_INODE , FIL_PAGE_IBUF_FREE_LIST 2 FIL_PAGE_TYPE log serial number of page's latest log record 8 FIL_PAGE_LSN offset of next page in key order 4 FIL_PAGE_NEXT offset of previous page in key order 4 FIL_PAGE_PREV ordinal page number from start of space 4 FIL_PAGE_OFFSET 4 ID of the space the page is in 4 FIL_PAGE_SPACE Remarks Size Name Data are stored in FIL_PAGE_INODE == 0x03
  • 9. InnoDB page format Page Header "file segment header for the non-leaf pages in a B-tree" (this is irrelevant here) 10 PAGE_BTR_SEG_TOP "file segment header for the leaf pages in a B-tree" (this is irrelevant here) 10 PAGE_BTR_SEG_LEAF identifier of the index the page belongs to 8 PAGE_INDEX_ID level within the index (0 for a leaf page) 2 PAGE_LEVEL the highest ID of a transaction which might have changed a record on the page (only set for secondary indexes) 8 PAGE_MAX_TRX_ID number of user records 2 PAGE_N_RECS number of consecutive inserts in the same direction, e.g. "last 5 were all to the left" 2 PAGE_N_DIRECTION either PAGE_LEFT , PAGE_RIGHT , or PAGE_NO_DIRECTION 2 PAGE_DIRECTION record pointer to the last inserted record 2 PAGE_LAST_INSERT "number of bytes in deleted records" 2 PAGE_GARBAGE record pointer to first free record 2 PAGE_FREE number of heap records; initial value = 2 2 PAGE_N_HEAP record pointer to first record in heap 2 PAGE_HEAP_TOP number of directory slots in the Page Directory part; initial value = 2 2 PAGE_N_DIR_SLOTS Remarks Size Name index_id Highest bit is row format(1 -COMPACT, 0 - REDUNDANT )
  • 10. InnoDB page format (REDUNDANT) Extra bytes pointer to next record in page 16 bits next 16 bits 1 if each Field Start Offsets is 1 byte long (this item is also called the "short" flag) 1 bit 1byte_offs_flag number of fields in this record, 1 to 1023 10 bits n_fields record's order number in heap of index page 13 bits heap_no number of records owned by this record 4 bits n_owned 1 if record is predefined minimum record 1 bit min_rec_flag 1 if record is deleted 1 bit deleted_flag _ORDINAR Y, _NODE_PTR , _INFIMUM , _SUPREMUM 2 bit s record_status Description Size Name
  • 11. InnoDB page format (COMPACT) Extra bytes a relative pointer to the next record in the page 16 next 16 bits 000=conventional, 001=node pointer (inside B-tree), 010=infimum, 011=supremum, 1xx=reserved 3 record type the order number of this record in the heap of the index page 1 3 heap_no the number of records owned by this record (this term is explained in page0page.h) 4 n_owned 4 bits used to delete mark a record, and mark a predefined minimum record in alphabetical order 4 Description Size , bits Name
  • 12.
  • 13.
  • 14.
  • 15. Row format EXAMPLE: CREATE TABLE ` t1 ` ( ` ID ` int( 11 ) unsigned NOT NULL, ` NAME ` varchar(120), ` N_FIELDS ` int(10), PRIMARY KEY (`ID`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 depends on content Field Contents 6 bytes (5 bytes if COMPACT format) Extra Bytes (F*1) or (F*2) bytes Field Start Offsets Size Name
  • 16. REDUNDANT A row: (10 , ‘abcdef’, 20 ) 4 6 7 Actualy stored as: (10 , TRX_ID, PTR_ID, ‘abcdef’, 20 ) 6 4 Field Offsets … . next Extra 6 bytes: 0x00 00 00 0A record_status deleted_flag min_rec_flag n_owned heap_no n_fields 1byte_offs_flag Fields ... ... abcdef 0x80 00 00 14
  • 17. COMPACT A row: (10 , ‘abcdef’, 20 ) 6 NULLS Actualy stored as: (10 , TRX_ID, PTR_ID, ‘abcdef’, 20 ) Field Offsets … . next Extra 5 bytes: 0x00 00 00 0A Fields ... ... abcdef 0x80 00 00 14 A bit per NULL-able field
  • 18.
  • 19.
  • 20. 2 . Internal system tables SYS_INDEXES and SYS_TABLES
  • 21.
  • 22.
  • 23. How MySQL stores data in InnoDB NAME ID … SYS_TABLES "archive19/9299_msg_store" 40694 8 1 0 0 NULL 0 SYS_TABLES "archive19/9299_msg_store" 40694 8 1 0 0 NULL 0 SYS_TABLES "archive19/9299_msg_store" 40694 8 1 0 0 NULL 0 TABLE_ID ID NAME … SYS_INDEXES 40694 196389 "PRIMARY" 2 3 0 21031026 SYS_INDEXES 40694 196390 "msg_hash" 1 0 0 21031028 SYS_TABLES SYS_INDEXES Example:
  • 24. 3. InnoDB Primary and Secondary keys
  • 25.
  • 26.
  • 27. 4. Typical failure scenarios
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 34.
  • 35.
  • 36.
  • 37. InnoDB recovery tool server # ./page_parser -4 -f /var/lib/mysql/ibdata1 Opening file: /var/lib/mysql/ibdata1 Read data from fn=3... Read page #0.. saving it to pages-1259793800/0-18219008/0-00000000.page Read page #1.. saving it to pages-1259793800/0-0/1-00000001.page Read page #2.. saving it to pages-1259793800/4294967295-65535/2-00000002.page Read page #3.. saving it to pages-1259793800/0-0/3-00000003.page page_parser
  • 38.
  • 39. InnoDB recovery tool server # ./constraints_parser -4 -f pages-1259793800/0-16/51-00000051.page constraints_parser Table structure is defined in "include/table_defs.h" See HOWTO for details http://code.google.com/p/innodb-tools/wiki/InnodbRecoveryHowto Filters inside table_defs.h are very important
  • 40.
  • 41.
  • 42.