还原Oracle中真实的cache recovery

还原 Oracle 中真实的 Cache Recovery by Maclean.liu liu.maclean@gmail.com www.oracledatabase12g.com

About Mel Email:liu.maclean@gmail.coml Blog:www.oracledatabase12g.coml Oracle Certified Database Administrator Master 10gand 11gl Over 6 years experience with Oracle DBA technologyl Over 7 years experience with Linux technologyl Member Independent Oracle Users Groupl Member All China Users Groupl Presents for advanced Oracle topics: RAC,DataGuard, Performance Tuning and Oracle Internal.

我们在学习 Oracle 基础知识的时候会了解到实例恢复(Instance Recovery)或者说崩溃恢复(Crash recovery)的概念，有时候甚至于这 2 个名词在我们日常的语言中表达同样的意思。实际上 Instance Recovery 与 Crash Recovery 是存在区别的：针对单实例(single instance)或者RAC 中所有节点全部崩溃后的恢复，我们称之为 Crash Recovery。而对于 RAC 中的某一个节点失败，存活节点(surviving instance)试图对失败节点线程上 redo 做应用的情况，我们称之为 Instance Recovery。不管是 Instance Recovery 还是 Crash Recovery，都由 2 个部分组成：cache recovery 继以transaction recovery。根据官方文档的介绍，Cache Recovery 也叫 Rolling Forward，就是我们常说的前滚；而Transaction Recovery 也叫 Rolling Back，就是我们常说的回滚。前滚和回滚贯穿 Oracle 恢复的基本概念，是我们入门必要学习的知识，在次不多介绍。有文事者，必济之以武略。理论学得再好，不实践也无用。所幸 Crash Recovery 是很容易做成的实验，我们不妨来看一看:SQL> shutdown abort;ORACLE instance shut down.SQL> startup mount;ORACLE instance started.Total System Global Area 1065353216 bytesFixed Size 2089336 bytesVariable Size 486542984 bytesDatabase Buffers 570425344 bytesRedo Buffers 6295552 bytesDatabase mounted.SQL> alter database open;Crash Recovery 将从 alter database open 开始，我们来观察其日志==================alert.log====================alter database openTue Jun 14 18:19:53 2011Beginning crash recovery of 1 threads parallel recovery started with 2 processesTue Jun 14 18:19:53 2011Started redo scanTue Jun 14 18:19:53 2011Completed redo scan 0 redo blocks read, 0 data blocks need recoveryTue Jun 14 18:19:53 2011Started redo application at Thread 1: logseq 1004, block 1124, scn 17136185

Tue Jun 14 18:19:53 2011Recovery of Online Redo Log: Thread 1 Group 2 Seq 1004 Reading mem 0 Mem# 0: /flashcard/oradata/G10R2/onlinelog/o1_mf_2_6v34jokt_.log Mem# 1: /s01/flash_recovery_area/G10R2/onlinelog/o1_mf_2_6v34jotq_.logTue Jun 14 18:19:53 2011Completed redo applicationTue Jun 14 18:19:53 2011Completed crash recovery at Thread 1: logseq 1004, block 1124, scn 17156186 0 data blocks read, 0 data blocks written, 0 redo blocks readTue Jun 14 18:19:53 2011LGWR: STARTING ARCH PROCESSESARC0: Archival startedLGWR: STARTING ARCH PROCESSES COMPLETEARC0 started with pid=16, OS id=7829Tue Jun 14 18:19:53 2011Thread 1 advanced to log sequence 1005 (thread open)Thread 1 opened at log sequence 1005 Current log# 3 seq# 1005 mem# 0:/flashcard/oradata/G10R2/onlinelog/o1_mf_3_6v34jpmp_.log Current log# 3 seq# 1005 mem# 1:/s01/flash_recovery_area/G10R2/onlinelog/o1_mf_3_6v34jpyn_.logSuccessful open of redo thread 1Tue Jun 14 18:19:53 2011ARC0: Becoming the no FAL ARCHARC0: Becoming the no SRL ARCHARC0: Becoming the heartbeat ARCHTue Jun 14 18:19:53 2011SMON: enabling cache recoveryTue Jun 14 18:19:53 2011db_recovery_file_dest_size of 204800 MB is 6.81% used. This is auser-specified limit on the amount of space that will be used by thisdatabase for recovery-related files, and does not reflect the amount ofspace available in the underlying filesystem or ASM diskgroup.Tue Jun 14 18:19:54 2011Successfully onlined Undo Tablespace 1.Tue Jun 14 18:19:54 2011SMON: enabling tx recoveryDatabase Characterset is UTF8Opening with internal Resource Manager planwhere NUMA PG = 1, CPUs = 2replication_dependency_tracking turned off (no async multimaster replicationfound)Starting background process QMNCQMNC started with pid=17, OS id=7831Tue Jun 14 18:19:55 2011Completed: alter database open

注意上述单实例 Crash Recovery 到数据库打开的整个过程: • alter database open • Beginning crash recovery of 1 threads • Started redo scan • Completed redo scan • Started redo application at • Completed redo application • Completed crash recovery at • SMON: enabling cache recovery • Successfully onlined Undo Tablespace 1 • SMON: enabling tx recovery • Completed: alter database open从上述步骤中我们可以看到三种恢复名词，即： • crash recovery • cache recovery • tx recovery这和官方文档所描述的 Crash Recovery 概念是不一致的，我们现在来理清这几种 recovery。crash recovery 包含对 redo 的 scan 和 application，显然其完成的是 Rolling Forward 前滚的工作，告警日志中出现的 crash recovery 等同于官方文档中介绍的”cache recovery”，我们可以将” Completed crash recovery”看做前滚完成的标志。而 tx recovery 从字面就可以看出实际上是 Transaction Recovery，tx recovery 发生在 Undo Tablespace online 之后(回滚事务的前提是Undo 可用)，数据完成打开操作之前(“Completed: alter database open”)。注意 tx recovery 并不要求数据库打开前完成，仅仅是在数据库打开之前由 smon 启动(“SMON: enabling txrecovery”)。剩下的唯一的问题是，这里的 cache recovery 是什么？显然它不是官方文档中所描述的”cacherecovery”，几乎没有任何文档介绍存在这样一个 recovery 操作，这也是本文重点要介绍的。

我们来看另一个演示，这个演示用以说明 cache recovery 还存在于最普通的不包含 CrashRecovery 的数据库打开过程中:SQL> shutdown immediate;Database closed.Database dismounted.ORACLE instance shut down.SQL> startup mount;ORACLE instance started.Total System Global Area 1065353216 bytesFixed Size 2089336 bytesVariable Size 486542984 bytesDatabase Buffers 570425344 bytesRedo Buffers 6295552 bytesDatabase mounted.SQL> alter database open;Database altered.SQL> select * from v$version;BANNER----------------------------------------------------------------Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64biPL/SQL Release 10.2.0.4.0 - ProductionCORE 10.2.0.4.0 ProductionTNS for Linux: Version 10.2.0.4.0 - ProductionNLSRTL Version 10.2.0.4.0 - ProductionSQL> select * from global_name;GLOBAL_NAME--------------------------------------------------------------------------------www.oracledatabase12g.com==================alert.log====================alter database openTue Jun 14 18:43:52 2011LGWR: STARTING ARCH PROCESSESARC0: Archival startedLGWR: STARTING ARCH PROCESSES COMPLETEARC0 started with pid=14, OS id=8133Tue Jun 14 18:43:52 2011Thread 1 opened at log sequence 1005Current log# 3 seq# 1005 mem# 0:/flashcard/oradata/G10R2/onlinelog/o1_mf_3_6v34jpmp_.logCurrent log# 3 seq# 1005 mem# 1:/s01/flash_recovery_area/G10R2/onlinelog/o1_mf_3_6v34jpyn_.logSuccessful open of redo thread 1Tue Jun 14 18:43:52 2011ARC0: Becoming the no FAL ARCHARC0: Becoming the no SRL ARCHARC0: Becoming the heartbeat ARCHTue Jun 14 18:43:52 2011SMON: enabling cache recoveryTue Jun 14 18:43:53 2011Successfully onlined Undo Tablespace 1.Tue Jun 14 18:43:53 2011SMON: enabling tx recovery

Database Characterset is UTF8Opening with internal Resource Manager planwhere NUMA PG = 1, CPUs = 2replication_dependency_tracking turned off (no async multimaster replicationfound)Tue Jun 14 18:43:53 2011Incremental checkpoint up to RBA [0x3ed.624.0], current log tail at RBA[0x3ed.944.0]Tue Jun 14 18:43:53 2011Starting background process QMNCQMNC started with pid=15, OS id=8135Tue Jun 14 18:43:53 2011Completed: alter database open因为是 clean shutdown，所以这里不存在 crash recovery。但这里同样出现了”SMON: enablingcache recovery”，可见 cache recovery 是每次实例启动 instance startup 必要执行的一种恢复操作。但问题是，这个恢复操作到底针对何种对象？实际上 cache recovery 所要恢复的是 rowcache，也就是我们常说的字典缓存(dictionarycache)。关于这个结论，肯定有很多人要问我这样说的依据是什么，对应于这个”cacherecovery”的问题，我们很难从 google 中得到一些启示，因为它和官方文档所描述的”cacherecovery-rolling forward”存在重名的关系。为了证明 cache recovery 所恢复的是 rowcache，我们需要一个实证，从正式的系统中得到验证。要做到这一点是比较困难的，我们需要 Oracle 愿意把整个 database open 的过程变成慢动作来供我们参考，验证要用到一些调试工具，例如 gdb 或者 dbx。我们首先将实例启动到 mount 状态，并对执行 startup 的 LOCAL 进程做 gdb 的 breakpoint 断点调试:SQL> shutdown abort;ORACLE instance shut down.SQL> startup mount;ORACLE instance started.Total System Global Area 1065353216 bytesFixed Size 2089336 bytesVariable Size 486542984 bytesDatabase Buffers 570425344 bytesRedo Buffers 6295552 bytesDatabase mounted.找出 LOCAL 进程的系统进程号 SPIDSQL> select spid from v$process2 where addr in (3 select paddr from v$session4 where sid=(select distinct sid from v$mystat))5 /SPID

------------8326在实例 startup nomount/mount 后共享池的 library cache 就是可用的SQL> select namespace from v$librarycache where gets!=0;NAMESPACE---------------SQL AREATABLE/PROCEDURE而 rowcache 则尚未被填充，因为字典缓存来源于自举对象(bootstrap$)和字典基表SQL> select parameter,count,gets from v$rowcache where count!=0;no rows selected另开一个 terminal 窗口，并执行对 LOCAL 进程 8326 的 gdb breakpoint 调试[oracle@rh2 ~]$ gdb $ORACLE_HOME/bin/oracle 8326GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-23.el5)Copyright (C) 2009 Free Software Foundation, Inc.License GPLv3+: GNU GPL version 3 or laterThis is free software: you are free to change and redistribute it.There is NO WARRANTY, to the extent permitted by law. Type "show copying"and "show warranty" for details.This GDB was configured as "x86_64-redhat-linux-gnu".For bug reporting instructions, please see:...Reading symbols from /s01/db_1/bin/oracle...(no debugging symbols found)...done.Attaching to program: /s01/db_1/bin/oracle, process 8326Reading symbols from /s01/db_1/lib/libskgxp10.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libskgxp10.soReading symbols from /s01/db_1/lib/libhasgen10.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libhasgen10.soReading symbols from /s01/db_1/lib/libskgxn2.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libskgxn2.soReading symbols from /s01/db_1/lib/libocr10.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libocr10.soReading symbols from /s01/db_1/lib/libocrb10.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libocrb10.soReading symbols from /s01/db_1/lib/libocrutl10.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libocrutl10.soReading symbols from /s01/db_1/lib/libjox10.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libjox10.soReading symbols from /s01/db_1/lib/libclsra10.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libclsra10.soReading symbols from /s01/db_1/lib/libdbcfg10.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libdbcfg10.soReading symbols from /s01/db_1/lib/libnnz10.so...(no debugging symbolsfound)...done.Loaded symbols for /s01/db_1/lib/libnnz10.soReading symbols from /usr/lib64/libaio.so.1...(no debugging symbols

found)...done.Loaded symbols for /usr/lib64/libaio.so.1Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.Loaded symbols for /lib64/libdl.so.2Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.Loaded symbols for /lib64/libm.so.6Reading symbols from /lib64/libpthread.so.0...(no debugging symbolsfound)...done.[Thread debugging using libthread_db enabled]Loaded symbols for /lib64/libpthread.so.0Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found)...done.Loaded symbols for /lib64/libnsl.so.1Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.Loaded symbols for /lib64/libc.so.6Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbolsfound)...done.Loaded symbols for /lib64/ld-linux-x86-64.so.2Reading symbols from /lib64/libnss_files.so.2...(no debugging symbolsfound)...done.Loaded symbols for /lib64/libnss_files.so.20x0000003181a0d8e0 in __read_nocancel () from /lib64/libpthread.so.0输入断点 kcrf_commit_force 和 kqlobjlod(gdb) break kcrf_commit_forceBreakpoint 1 at 0x2724b6c(gdb) break kqlobjlodBreakpoint 2 at 0x1ac5e8c在之前的 terminal 中执行数据库打开操作，因为 breakpoint 的关系这个 open 操作会hang 住，这时我们通过观察告警日志来了解恢复进度SQL> alter database open; --这里会 hang 住在 gdb 下输入 continue，(gdb) cContinuing.Breakpoint 1, 0x0000000002724b6c in kcrf_commit_force ()观察告警日志可以发现 redo application 已经完成，但还未进入 cache recovery 阶段alter database openTue Jun 14 19:14:33 2011Beginning crash recovery of 1 threadsparallel recovery started with 2 processesTue Jun 14 19:14:33 2011Started redo scanTue Jun 14 19:14:33 2011Completed redo scan39 redo blocks read, 74 data blocks need recoveryTue Jun 14 19:14:33 2011Started redo application atThread 1: logseq 1006, block 1155Tue Jun 14 19:14:33 2011Recovery of Online Redo Log: Thread 1 Group 1 Seq 1006 Reading mem 0Mem# 0: /flashcard/oradata/G10R2/onlinelog/o1_mf_1_6v34jnkn_.log

Mem# 1: /s01/flash_recovery_area/G10R2/onlinelog/o1_mf_1_6v34jnst_.logTue Jun 14 19:14:33 2011Completed redo applicationTue Jun 14 19:14:33 2011Completed crash recovery atThread 1: logseq 1006, block 1194, scn 1720019374 data blocks read, 74 data blocks written, 39 redo blocks readTue Jun 14 19:14:33 2011LGWR: STARTING ARCH PROCESSESARC0: Archival startedLGWR: STARTING ARCH PROCESSES COMPLETEARC0 started with pid=17, OS id=8656Tue Jun 14 19:14:33 2011Thread 1 advanced to log sequence 1007 (thread open)Thread 1 opened at log sequence 1007Current log# 2 seq# 1007 mem# 0:/flashcard/oradata/G10R2/onlinelog/o1_mf_2_6v34jokt_.logCurrent log# 2 seq# 1007 mem# 1:/s01/flash_recovery_area/G10R2/onlinelog/o1_mf_2_6v34jotq_.logSuccessful open of redo thread 1Tue Jun 14 19:14:33 2011ARC0: Becoming the no FAL ARCHARC0: Becoming the no SRL ARCHARC0: Becoming the heartbeat ARCHdb_recovery_file_dest_size of 204800 MB is 6.81% used. This is auser-specified limit on the amount of space that will be used by thisdatabase for recovery-related files, and does not reflect the amount ofspace available in the underlying filesystem or ASM diskgroup.Tue Jun 14 19:14:37 2011Incremental checkpoint up to RBA [0x3ef.3.0], current log tail at RBA[0x3ef.3.0]且此时 rowcache 仍未被填充SQL> select parameter,count,gets from v$rowcache where count!=0;no rows selected在 gdb 界面下再次执行 continue 2 次(gdb) cContinuing.Breakpoint 1, 0x0000000002724b6c in kcrf_commit_force ()(gdb) cContinuing.Breakpoint 2, 0x0000000001ac5e8c in kqlobjlod ()观察告警日志可以发现已开始 cache recovery，但也卡陷在 cache recovery 上，这保证我们的演示不受骚扰Tue Jun 14 19:16:44 2011SMON: enabling cache recovery此时 rowcache 中出现唯一的一个 dc_objects 对象select parameter,count,gets from v$rowcache where count!=0;PARAMETER COUNT GETS-------------------------------- ---------- ----------dc_objects 1 1

这个对象是什么呢？也许你已经猜到了，我们做一个 rowcache dump 来看一下:SQL> ALTER SESSION SET EVENTS immediate trace name row_cache level 10;================row_cache trace===================BUCKET 43170:row cache parent object: address=0x92326060 cid=8(dc_objects)hash=f3d1a8a1 typ=11 transaction=(nil) flags=00000001own=0x92326130[0x9230f628,0x9230f628] wat=0x92326140[0x92326140,0x92326140]mode=Sstatus=EMPTY/-/-/-/-/-/-/-/-set=0, complete=FALSEdata=00000000 4f42000a 5453544f 24504152 00000000 00000000 00000000 0000000000000000 00000001 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000BUCKET 43170 total object count=1可以看到上述 dc_objects 尚未完成加载(status=EMPTY & complete=FALSE )，那么这是一个什么 object 呢？4f42000a 5453544f 24504152 => 转换为文本即：OB TSTO$PAR 也就是BOOTSTRAP$换而言之在 cache recovery 时第一个恢复的字典缓存对象是 BOOTSTRAP$，这并不出乎我们的意料。启动实例的 LOCAL 进程的等待事件为 instance state change，这是常规情况下我们观察不到得SQL> select event,p1text,p1 from v$session where wait_class!=Idle;EVENT P1TEXTP1-------------------------------------------------------------------------------- ----------instance state change layer2在 gdb 界面下再次 continue，将载入更多的 rowcache 对象(gdb) cContinuing.Breakpoint 2, 0x0000000001ac5e8c in kqlobjlod ()BUCKET 37:row cache parent object: address=0x916cd980 cid=3(dc_rollback_segments)hash=5fed2a24 typ=9 transaction=(nil) flags=000000a6own=0x916cda50[0x916cda50,0x916cda50] wat=0x916cda60[0x916cda60,0x916cda60]

mode=Nstatus=VALID/INSERT/-/FIXED/-/-/-/-/-data=00000000 00000000 00000001 00000009 59530006 4d455453 00000000 0000000000000000 00000000 00000000 00000000 00000003 00000000 00000000 0000000000000000 00000000 00000000 00000000BUCKET 37 total object count=1595300064d455453 -> SYSTEM 属于 dc_rollback_segments 也就是著名的system 回滚段BUCKET 55556:row cache parent object: address=0x916d8cd0 cid=8(dc_objects)hash=ce89d903 typ=11 transaction=(nil) flags=00000001own=0x916d8da0[0x9230f628,0x9230f628] wat=0x916d8db0[0x916d8db0,0x916d8db0]mode=Sstatus=EMPTY/-/-/-/-/-/-/-/-set=0, complete=FALSEdata=00000000 5f430006 234a424f 00000000 00000000 00000000 00000000 0000000000000000 00000005 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000 00000000 00000000 00000000 00000000 00000000 0000000000000000 00000000BUCKET 55556 total object count=15f430006 234a424f -> C_OBJ# 是著名的 bootstrap 对象之一，可以从$ORACLE_HOME/rdbms/admin/sql.bsq 中找到它create rollback segment SYSTEM tablespace SYSTEMstorage (initial 50K next 50K)/create cluster c_obj# (obj# number)pctfree 5 size 800 /* dont waste too much space *//* A table of 32 cols, 2 index, 2 col per index requires about 2K.* A table of 10 cols, 2 index, 2 col per index requires about 750.*/storage (initial 130K next 200k maxextents unlimited pctincrease 0)/* avoid space management during IOR I *//我们还可以通过 v$rowcache_parent 视图来了解 dictionary cache 的情况SQL> col cache_name for a20SQL> col keystr for a31SQL> set linesize 200SQL> select address,cache_name,existent,lock_mode,saddr,substr(key,1,30) keystrfrom v$rowcache_parent;ADDRESS CACHE_NAME E LOCK_MODE SADDR KEYSTR---------------- -------------------- - ---------- -----------------------------------------------00000000916CCE20 dc_tablespaces N 0 0000000000000000000000000000000000000000916CD980 dc_rollback_segments Y 0 00000000000000000000000000000000

0000000092326060 dc_objects Y 0 00000000000A00424F4F54535452415000000000916D8CD0 dc_objects N 3 000000009BD91328000000000600435F4F424A2300000000000000916DA830 dc_object_ids Y 0 00380000000000000000000000000000可以看到持有 row cache lock 的会话是000000009BD91328，且该 dc_objects 对象还处于 non-existent 状态，换而言之真正装载 rowcache 的是启动实例的 LOCAL 服务进程SQL> select sid,program,event,p1,p2,p3 from v$session wheresaddr=000000009BD91328;SID PROGRAM EVENTP1 P2 P3----- ---------------------------------------------------------------------------------------- -- ---- --3294 sqlplus@rh2.oracle.com (TNS V1-V3) db file scattered read1 378 3该进程正在等待 db file scattered read，fileid->1,block-378，这些块属于BOOTSTRAP$表BOOTSTRAP$对象已从 rowcache 被载入到 library cache 中SQL> select kglhdadr,kglnaobj from x$kglob where kglobtyp=2 and kglnaobj notlike X$%;KGLHDADR KGLNAOBJ-------------------- --------------------0000000092326990 BOOTSTRAP$SQL> select owner||.||Name from v$db_object_cache where type=TABLE and namenot like X$%;OWNER||.||NAME--------------------------------------------------------------------------------SYS.BOOTSTRAP$初步总结： 1. 在数据库正式 open 前需要恢复字典缓存，这个步骤被称为 cache recovery，其实是 row cache recovery。与官方文档中描述的”cache recovery”不同，row cache recovery 应当是 Oracle Internal 的叫法。 2. 实际执行 row cache recovery 的不是 SMON 进程，而是启动实例的服务进程

还原Oracle中真实的cache recovery

by Maclean Liu on Sep 07, 2011

Accessibility

Categories

Upload Details

Usage Rights

Statistics

还原Oracle中真实的cache recovery Document Transcript