Your SlideShare is downloading. ×
0
IO Waits           Kyle HaileyKyle.hailey@embarcadero.com   http://oraclemonitor.com
Waits Covered in this Section  1.    db file sequential read  2.    db file scattered read                     Buffer Cach...
IO Waits                  SGA        Log      Library            Buffer        Buffer   Cache              CacheUser1     ...
Standard Table/Index IO   1. db file sequential read         Single block read   1. db file scattered read         Multi...
db file sequential readBuffer Cache    The top wait                Single Block Read                  Data via Index an...
db file scattered read                     Multi Block ReadBuffer Cache            FullTable Scan                       ...
db file parallel readBuffer Cache    Process issues multiple single block reads in parallel                   Documentat...
Read by other SessionBuffer Cache                    Multiple sessions reading the                    same data that requi...
IO – Further Investigationselect parameter1, parameter2, parameter3 from v$event_nameNAME                        P1       ...
sequential reads – blocks readselect       event,       ash.p3,       o.object_name objn,       o.object_type otype,      ...
scattered reads – multi blocks read  select         event,         ash.p3,         o.object_name objn,         o.object_ty...
IO by Fileselect io.cnt,           round(io.cnt/(&v_minutes*60),2) aas,           io.event,           io.p1 p1,           ...
IO by Objectselect     count(*) cnt,     CURRENT_OBJ#|| ||o.object_name objn,     o.object_type otypefrom v$active_session...
IO by SQLselect       round(sum(cnt) over ( partition by io.sql_id order by sql_id ) / (&v_minutes*60),2) aas,       io.sq...
Missing Object IDs dba_extents – get object name         select segment_name,          select segment_name,              ...
Copy of dba_extentsEnter value for file: 3Enter value for block: 6619OWNER          SEGMENT_NAME              SEGMENT_TYPE...
IO Solutions 1. Check average read times per file       Should be between 5-20 ms       Data in Statspack under “File IO...
Step 1a: Ave Read TimeFile IO Stats DB/Inst:labsf03 Snaps: 1-2 File IO Stats DB/Inst:labsf03 Snaps: 1-2Tablespace Tablespa...
Step 1b: Ave Read Timeselect select    to_char(begin_interval_time,yyyy-mm-dd hh24:mi) snap_time,     to_char(begin_interv...
Step 1c: Ave Read Timeselect select     to_char(begin_time,yyyy-mm-dd hh24:mi) begin_time,      to_char(begin_time,yyyy-mm...
Step1 & OEM ADDM advisory expects I/Os to be under 10ms Can change the behavior with exec DBMS_ADVISOR.SET_DEFAULT_TASK_...
Step 2 : Buffer Pool AdvisoryBuffer Pool Advisory Buffer Pool Advisory Buffers forSize for Size for       Size            ...
Step 3: Find Top I/O SQLselect count(*),sql_id, event select count(*),sql_id, eventfrom v$active_session_history from v$ac...
Step 3: OEM find top SQL            Copyright 2006 Kyle Hailey
IO Configuration Issues Unix Buffer Cache Disk Spindles Machine Memory Raid 5              Unix File                   ...
IO ThroughputOne Disk 74G – Few IOPS                 92 IO/sec37 Disks 2G – Many IOPS                   1887 IO/sec       ...
Disk 2Gb vs 70GB                                Seagate                  Seagate                                Barracuda ...
Raid 5http://www.miracleas.com/BAARF/BAARF2.html              P1             P2                                           ...
Raid Levels  Raid Level              none           0             0+1             5                                       ...
Direct I/O WAITSDirect I/O :This mechanism lets the client bypass the buffer cache forI/O intensive operations.The disk bl...
Direct IO        Shadow        Process        PGA       Sort Area                             Buffer Cache      Direct Pat...
direct path read Buffer Cache                 Parallel Query                    See     v$px_session     X              ...
Further Investigation ASHselect select      session_id sid,       session_id sid,      QC_SESSION_ID qsid,       QC_SESSIO...
PQO - ASHselect    ash.SQL_ID,    QC_SESSION_ID qsid,    count(*) cnt,    count (distinct session_id) deg,    nvl(o.object...
direct path write                           Occurs when:                              insert /*+ APPEND */              ...
Direct Path Read TempDirect Path Write Temp                                     select parameter1, parameter2, parameter3 ...
Direct Path Read TempDirect Path Write Temp                   This event happens when sorting overflows the memory buffers...
PGA Aggregate Target  Manual override broken 10.2.0.3 fixed in 11.1.0.7        alter session set workarea_size_policy = ma...
Data file init write  When autoextend is set and many extensions are   happening  Can be the root cause of other waits  ...
Local Write Wait  Truncating a table  Wait for Data Cache to be cleared of all blocks of   truncated table  Wait by sha...
Further Investigationselect * from v$event_namewhere name = ‘local write waitNAME NAME                   P1               ...
Temporary File #’sSQL> select file# from v$datafile;  FILE#    1                     Wait Temporary File#’s =    2        ...
Local Write Wait:extent allocation CREATE TEMPORARY TABLESPACE "NEWTEMP"  TEMPFILE /d3/temp01.dbf SIZE 100M  AUTOEXTEND ON...
Local Write Waits              Copyright 2006 Kyle Hailey
Local Write Wait: Solutions  Truncating a table    Use  GTT    ( reduce size of buffer cache – unlikely)  Temporary Ta...
Summary I/O Buffer Cache IO   db file sequential read   db file scattered read   db file parallel read Tune   I/O’s ...
Summary Direct I/O and Other  direct path read       PQO  direct path write     direct    path operations  direct pat...
Upcoming SlideShare
Loading in...5
×

Oracle 10g Performance: chapter 07 io

892

Published on

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
892
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
100
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • col block_type for a18 col objn for a25 col otype for a15 col event for a25 select event, ash.p3, CURRENT_OBJ#||' '||o.object_name objn, o.object_type otype, CURRENT_FILE# filen, CURRENT_BLOCK# blockn, ash.SQL_ID from v$active_session_history ash, all_objects o where event like 'db file s%' and o.object_id (+)= ash.CURRENT_OBJ# Order by sample_time /
  • col block_type for a18 col objn for a25 col otype for a15 col event for a25 select event, ash.p3, CURRENT_OBJ#||' '||o.object_name objn, o.object_type otype, CURRENT_FILE# filen, CURRENT_BLOCK# blockn, ash.SQL_ID from v$active_session_history ash, all_objects o where event like 'db file s%' and o.object_id (+)= ash.CURRENT_OBJ# Order by sample_time /
  • col block_type for a18 col objn for a25 col otype for a15 col event for a25 col blockn for 999999 col p1 for 9999 col aas for 999.99 col f_minutes new_value v_minutes select &minutes f_minutes from dual; select io.cnt, round(io.cnt/(&v_minutes*60),2) aas, io.event, io.p1 p1, f.tablespace_name from ( select count(*) cnt, substr(event,0,25) event, ash.p1 p1 from v$active_session_history ash where ( event like 'db file s%' or event like 'direct%' ) and sample_time > sysdate - &v_minutes/(60*24) group by event , ash.p1 ) io, dba_data_files f where f.file_id = io.p1 Order by io.cnt /
  • col block_type for a18 col obj for a20 col objn for 999999 col otype for a15 col event for a15 col blockn for 999999 col p1 for 9999 col tablespace_name for a15 col f_minutes new_value v_minutes select &minutes f_minutes from dual; break on sql_id on tcnt select sum(cnt) over ( partition by io.sql_id order by sql_id ) tcnt, io.sql_id, io.cnt cnt, io.aas aas, --io.event event, io.objn objn, io.obj obj, io.p1 p1, f.tablespace_name tablespace_name from ( select sql_id, count(*) cnt, round(count(*)/(&v_minutes*60),2) aas, CURRENT_OBJ# objn, nvl(o.object_name,decode(CURRENT_OBJ#,-1,0,CURRENT_OBJ#)) obj, o.object_type otype, ash.p1 from v$active_session_history ash ,all_objects o where ( event like 'db file s%' or event like 'direct%' ) and o.object_id (+)= ash.CURRENT_OBJ# and sample_time > sysdate - &v_minutes/(60*24) group by CURRENT_OBJ#, o.object_name , o.object_type , ash.p1, sql_id ) io, dba_data_files f where f.file_id = io.p1 Order by tcnt, io.sql_id, io.cnt /
  • col avgrd_ms format 9,999.9 col avgwr_ms format 9,999.9 col fid for 999 col pr for 9,999 col pw for 9,999 select to_char(begin_time,'yyyy-mm-dd hh24:mi') begin_time, file_id fid, average_read_time *10 avgrd_ms, average_write_time *10 avgwr_ms, physical_reads pr, physical_writes pw from V$FILEMETRIC_HISTORY f order by begin_time /
  • There is a parameter - DBIO_EXPECTED - that tells ADDM what is an acceptable average single block read in micro seconds. The parameter defaults to 10000 micro seconds and can be changed with:   exec DBMS_ADVISOR.SET_DEFAULT_TASK_PARAMETER('ADDM','DBIO_EXPECTED', <new value>); If your CT (as expected) hasn't changed from default then the 20 mS per single block read is the reason for the ADDM finding.
  • select --+ parallel(a,4) count(*) from toto a; o col block_type for a18 col objn for a25 col otype for a15 col event for a25 col p3 for 999 col fn for 999 col sid for 9999 col qsid for 9999 select session_id sid, QC_SESSION_ID qsid, --event, --ash.p1, --ash.p2, ash.p3, CURRENT_OBJ#||' '||o.object_name objn, o.object_type otype, CURRENT_FILE# fn, CURRENT_BLOCK# blockn, ash.SQL_ID from v$active_session_history ash, all_objects o where event like 'direct path read' and o.object_id (+)= ash.CURRENT_OBJ# Order by sample_time /
  • col qsid for 9999 col cnt for 9999 col deg for 999 col obj for a15 col otype for a10 select ash.SQL_ID, QC_SESSION_ID qsid, count(*) cnt, count (distinct session_id) deg, --ash.p3, nvl(o.object_name,to_char(CURRENT_OBJ#)) obj, o.object_type otype, --CURRENT_FILE# fn, --CURRENT_BLOCK# blockn, decode(session_state, 'WAITING',event,'CPU') event from v$active_session_history ash, all_objects o where -- event like 'direct path read' o.object_id (+)= ash.CURRENT_OBJ# and qc_session_id is not null group by qc_session_id, sql_id, o.object_name, o.object_type, CURRENT_OBJ#, event, session_state Order by qc_session_id, sql_id / ================================ From Greg Rahn select s.sql_id,       px.INST_ID "Inst",       px.SERVER_GROUP "Group",       px.SERVER_SET "Set",       px.DEGREE "Degree",       px.REQ_DEGREE "Req Degree",       w.event "Wait Event" from GV$SESSION s, GV$PX_SESSION px, GV$PROCESS p, GV$SESSION_WAIT w where s.sid (+) = px.sid and      s.inst_id (+) = px.inst_id and      s.sid = w.sid (+) and      s.inst_id = w.inst_id (+) and      s.paddr = p.addr (+) and      s.inst_id = p.inst_id (+) ORDER BY decode(px.QCINST_ID,  NULL, px.INST_ID,  px.QCINST_ID),         px.QCSID,         decode(px.SERVER_GROUP, NULL, 0, px.SERVER_GROUP),         px.SERVER_SET,         px.INST_ID;
  • Transcript of "Oracle 10g Performance: chapter 07 io"

    1. 1. IO Waits Kyle HaileyKyle.hailey@embarcadero.com http://oraclemonitor.com
    2. 2. Waits Covered in this Section 1. db file sequential read 2. db file scattered read Buffer Cache I/O 3. db file parallel read 4. read by other session 5. direct path read 6. direct path write 7. direct path read temp 8. direct path write temp PGA I/O 9. direct path write (lob) 10. Local write wait 11. data file init write Special Cases Copyright 2006 Kyle Hailey
    3. 3. IO Waits SGA Log Library Buffer Buffer Cache CacheUser1 DBWR Data Files Copyright 2006 Kyle Hailey
    4. 4. Standard Table/Index IO 1. db file sequential read  Single block read 1. db file scattered read  Multi block read 1. db file parallel read  Non-contiguous multi block read 1. read by other session  Wait for another session to do the io Copyright 2006 Kyle Hailey
    5. 5. db file sequential readBuffer Cache  The top wait  Single Block Read  Data via Index and/or Rowid  Rollback (if expecting FTS => problem) Select **from emp where empno = 9999; Select from emp where empno = 9999; Shadow Index on emp.empno Process 1) Search Buffer Cache for the block by rowid 2) Fails Note: “sequential” means 3) Reads of disk A sequence like a • File rowid • Block Copyright 2006 Kyle Hailey
    6. 6. db file scattered read  Multi Block ReadBuffer Cache  FullTable Scan  Index Fast Full Scans SQL> select * from emp; SQL> select * from emp; Shadow Process 1) Search Buffer Cache for the blocks Note: Scattered Means 2) Fails Blocks are read and 3) Reads off disk • File scattered throughout • Block buffer cache • Multi Block Read Count db_file_multiblock_read_count Copyright 2006 Kyle Hailey
    7. 7. db file parallel readBuffer Cache  Process issues multiple single block reads in parallel  Documentation says only for recovery  But seems to happen for normal read ops  Async Call – wait for all reads to complete  Examples  Not contiguous multi block read  Index full or range scan Shadow  Where values in Select **from emp where Process Select from emp where Empno in (1,2,3,4,5,6); Empno in (1,2,3,4,5,6); 1) Search Buffer Cache for the blocks 2) Fails 3) Reads that block off Disk Copyright 2006 Kyle Hailey
    8. 8. Read by other SessionBuffer Cache Multiple sessions reading the same data that requires IO Ex) two users doing a full table scan at the same time S1 S2 1. Block not found in cache 2. Read block from disk 3. Found other session already reading block from disk 4. Wait for the other session to finish IO Copyright 2006 Kyle Hailey
    9. 9. IO – Further Investigationselect parameter1, parameter2, parameter3 from v$event_nameNAME P1 P2 P3read by other session file# block# class#db file sequential read file# block# blocksdb file scattered read file# block# blocksdb file parallel read files blocks requests P1 and P2 are file# , block# We can use P1, P2, P3 from ASH to get more info Exception: db file parallel read – p1,p2,p3 not useful Copyright 2006 Kyle Hailey
    10. 10. sequential reads – blocks readselect event, ash.p3, o.object_name objn, o.object_type otype, CURRENT_FILE# filen, CURRENT_BLOCK# blockn, ash.SQL_IDfrom v$active_session_history ash, all_objects owhere event like db file sequential read and o.object_id (+)= ash.CURRENT_OBJ#order by sample_time; P3= # of blocks EVENT P3 OBJN OTYPE FILEN BLOCKN SQL_ID db file sequential read 1 49890 MGMT_METRICS_1HOUR_ INDEX 3 41737 db file sequential read 1 50908 MGMT_DB_SGA_ECM TABLE 3 28489 35at43xqj7bs0 db file sequential read 1 55303 TOTO TABLE 1 60218 7xftj55rvjw9s Copyright 2006 Kyle Hailey
    11. 11. scattered reads – multi blocks read select event, ash.p3, o.object_name objn, o.object_type otype, CURRENT_FILE# filen, CURRENT_BLOCK# blockn, ash.SQL_ID from v$active_session_history ash, all_objects o where event like db file scattered read and o.object_id (+)= ash.CURRENT_OBJ# P3= # of order by sample_time; blocks EVENT P3 OBJN OTYPE FILEN BLOCKN SQL_ID db file scattered read 8 9078 WRH$_SYSMETRIC_HISTO TABLE 3 52363 db file scattered read 5 8781 WRI$_ALERT_HISTORY TABLE 3 2676 db file scattered read 8 57 OBJAUTH$ TABLE 1 33993 3t2mk1prj24hu Copyright 2006 Kyle Hailey
    12. 12. IO by Fileselect io.cnt, round(io.cnt/(&v_minutes*60),2) aas, io.event, io.p1 p1, f.tablespace_namefrom ( select count(*) cnt, substr(event,0,25) event, ash.p1 p1 from v$active_session_history ash where ( event like db file s% or event like direct% ) and sample_time > sysdate - &v_minutes/(60*24) group by event , ash.p1 ) io, dba_data_files f CNT AAS EVENT P1 TABLESPACEwhere 1 .00 db file sequential read 1 SYSTEM f.file_id = io.p1 2 .00 db file sequential read 3 SYSAUXOrder by io.cnt ; 38 .06 db file sequential read 6 SOE 179 .30 db file sequential read 7 SOEINDEX
    13. 13. IO by Objectselect count(*) cnt, CURRENT_OBJ#|| ||o.object_name objn, o.object_type otypefrom v$active_session_history ash, all_objects owhere ( event like db file s% or event like direct% ) and o.object_id (+)= ash.CURRENT_OBJ# and sample_time > sysdate - &1/(60*24) and session_state=WAITINGgroup by CURRENT_OBJ#, o.object_name , o.object_typeOrder by count(*) CNT AAS OBJN OTYPE PARTITION.00 79 52949 ORDER_ITEMS TABLE 97 .00 -1 130 .00 53117 ORD_STATUS_IX INDEX 498 .01 53120 CUST_EMAIL_IX INDEX 512 .01 0 1632 .03 53112 ITEM_ORDER_IX INDEX
    14. 14. IO by SQLselect round(sum(cnt) over ( partition by io.sql_id order by sql_id ) / (&v_minutes*60),2) aas, io.sql_id, io.cnt cnt, 100*cnt/sum(cnt) over ( partition by io.sql_id order by sql_id ) pct, o.object_name obj, o.subobject_name sub_obj, o.object_type otype, substr(io.event,8,10) event, AAS SQL_ID % OBJ TABLESPACE io.p1 file#, ----- ------------- --- --------------- ---------- f.tablespace_name tablespace_name, tbs.contents .18 0yas01u2p9ch4 6 ITEM_PRODUCT_IX SOEINDEXfrom( 6 ORDER_ITEMS_UK SOEINDEX select 88 ITEM_ORDER_IX SOEINDEX sql_id, event, .32 6v6gm0fd1rgrz 6 WAIT_OBJECTS SYSTEM count(*) cnt, 94 UNDO UNDOTBS1 count(*) / (&v_minutes*60) aas, CURRENT_OBJ# , ash.p1 from v$active_session_history ash where ( event like db file s% or event like direct% ) and sample_time > sysdate - &v_minutes/(60*24) group by CURRENT_OBJ#, event, ash.p1, sql_id) io, dba_data_files f ,all_objects o , dba_tablespaces tbswhere f.file_id = io.p1 and o.object_id (+)= io.CURRENT_OBJ# and tbs.tablespace_name= f.tablespace_nameOrder by tcnt, sql_id, cnt/
    15. 15. Missing Object IDs dba_extents – get object name select segment_name, select segment_name, segment_type segment_type from dba_extents from dba_extents where file_id = P1 where file_id = P1 and P2 between and P2 between block_id and block_id + blocks – 1; block_id and block_id + blocks – 1; Dba_extents is notoriously slow Options  Create a table as select – use table  recreate as object extents change  Catch blocks in x$bh when waits occur Copyright 2006 Kyle Hailey
    16. 16. Copy of dba_extentsEnter value for file: 3Enter value for block: 6619OWNER SEGMENT_NAME SEGMENT_TYPEWMSYS LOG_TAB_PK INDEXElapsed: 00:00:41.25create table myextents as select * from dba_extentsTable created.Elapsed: 00:01:25.73CNT OWN SEGMENT_NAME SEGMENT_TYPE 11 SYS SMON_SCN_TO_TIME CLUSTER 993 SYS _SYSSMU7$ TYPE2 UNDO150 rows selected.Elapsed: 00:00:01.03
    17. 17. IO Solutions 1. Check average read times per file  Should be between 5-20 ms  Data in Statspack under “File IO Stats” 1. Check Cache buffer Hit ratio  Check db_cache_advice 9i and higher  Data in Statspack under “Buffer Pool Advisory”  Want to optimize data caching 1. Tune High IO SQL  Check reads off of UNDO Copyright 2006 Kyle Hailey
    18. 18. Step 1a: Ave Read TimeFile IO Stats DB/Inst:labsf03 Snaps: 1-2 File IO Stats DB/Inst:labsf03 Snaps: 1-2Tablespace Tablespace Filename Filename------------------------ ---------------------------------------------------- ------------------------ ---------------------------------------------------- Av Av MxMxAv Av Av Av Rd Rd RdRd Av Av Av Av Buffer BufWt Buffer BufWt Reads Reads/s (ms) Bkt Blks/Rd Reads Reads/s (ms) Bkt Blks/Rd Writes Writes/s Writes Writes/s Waits (ms) Waits (ms)---------- ---- ------- ----- --- ------- ------------ -------- ---------- -- ---------- ---- ------- ----- --- ------- ------------ -------- ---------- --SYSTEM SYSTEM /u01/app/oracle/oradata/labsf03/system01.dbf /u01/app/oracle/oradata/labsf03/system01.dbf 445 445 15 15 0.4 16 0.4 16 1.0 1.0 1,157 1,157 39 39 2,744 2,744 93.3 93.3USERS USERS /u01/app/oracle/oradata/labsf03/users01.dbf /u01/app/oracle/oradata/labsf03/users01.dbf 223 223 77 0.5 ### 0.5 ### 1.0 1.0 9,725 9,725 324 324 44 100.0 100.0 IO should be under 20ms Fastest possible is around 7ms Faster speeds are due to read caching Copyright 2006 Kyle Hailey
    19. 19. Step 1b: Ave Read Timeselect select to_char(begin_interval_time,yyyy-mm-dd hh24:mi) snap_time, to_char(begin_interval_time,yyyy-mm-dd hh24:mi) snap_time, file#, readtim/nullif(phyrds,0) avg_read_ms, phyrds file#, readtim/nullif(phyrds,0) avg_read_ms, phyrdsfrom from SNAP_TIME FILE# AVG_READ_MS PHYRDS SNAP_TIME FILE# AVG_READ_MS PHYRDS DBA_HIST_FILESTATXS f, 2008-01-05 12:00 DBA_HIST_FILESTATXS f, 2008-01-05 12:00 99 36.67 39 36.67 39 dba_hist_snapshot ss dba_hist_snapshot 2008-01-05 12:00 2008-01-05 12:00 10 10 32.31 32.31 39 39 2008-01-05 13:00 2008-01-05 13:00 11 11.63 11.63 178224 178224where f.snap_id=s.snap_id ; ; 2008-01-05 13:00where f.snap_id=s.snap_id 2008-01-05 13:00 22 56.37 56.37 2014 2014 2008-01-05 13:00 2008-01-05 13:00 33 17.38 17.38 50668 50668 2008-01-05 13:00 2008-01-05 13:00 44 9.39 9.39 565321 565321 2008-01-05 13:00 2008-01-05 13:00 55 38.78 38.78 41 41 2008-01-05 13:00 2008-01-05 13:00 66 28.29 28.29 41 41 2008-01-05 13:00 2008-01-05 13:00 77 27.44 27.44 39 39 2008-01-05 13:00 2008-01-05 13:00 88 42.56 42.56 39 39 2008-01-05 13:00 2008-01-05 13:00 99 36.67 36.67 39 39 2008-01-05 13:00 2008-01-05 13:00 10 10 32.31 32.31 39 39IO should be under 20msFastest possible is around 7msFaster speeds are due to read caching Copyright 2006 Kyle Hailey
    20. 20. Step 1c: Ave Read Timeselect select to_char(begin_time,yyyy-mm-dd hh24:mi) begin_time, to_char(begin_time,yyyy-mm-dd hh24:mi) begin_time, file_id fid, file_id fid, average_read_time *10 avgrd_ms, average_read_time *10 avgrd_ms, average_write_time *10 avgwr_ms, average_write_time *10 avgwr_ms, physical_reads pr, physical_reads pr, physical_writes pw physical_writes pw BEGIN_TIME FID AVGRD_MS AVGWR_MS PR PWfrom from BEGIN_TIME FID AVGRD_MS AVGWR_MS PR PW ---------------- ---- -------- -------- ---------- ---------- ---------------- ---- -------- -------- ---------- ---------- V$FILEMETRIC_HISTORY ff 2008-01-30 12:25 11 V$FILEMETRIC_HISTORY 2008-01-30 12:25 5.2 5.2 2.5 2.5 100 100 24 24 2008-01-30 12:25 5 80.0 20.0 2 2order by begin_time; order by begin_time; 2008-01-30 12:25 2008-01-30 12:25 2 5 80.0 36.7 20.0 2.0 3 2 218 2 2008-01-30 12:25 2 36.7 2.0 3 218 2008-01-30 12:25 4 22.8 1.8 89 2754 2008-01-30 12:25 4 22.8 1.8 89 2754 2008-01-30 12:25 3 22.9 2.6 14 47 2008-01-30 12:25 3 22.9 2.6 14 47IO should be under 20msFastest possible is around 7msFaster speeds are due to read caching Copyright 2006 Kyle Hailey
    21. 21. Step1 & OEM ADDM advisory expects I/Os to be under 10ms Can change the behavior with exec DBMS_ADVISOR.SET_DEFAULT_TASK_PARAMETER( ADDM, DBIO_EXPECTED, <new value>);
    22. 22. Step 2 : Buffer Pool AdvisoryBuffer Pool Advisory Buffer Pool Advisory Buffers forSize for Size for Size Size Buffers for Read Read Estimated EstimatedPP Est (M) Factor Est (M) Factor Estimate Estimate Factor Factor Physical Reads Physical Reads--- -------- ------ ------------ --- -------- ------ ------------ ------ ------ -------------- --------------DD 56 56 .1 .1 6,986 6,986 2.3 2.3 58,928 58,928DD 112 112 .2 .2 13,972 13,972 1.6 1.6 42,043 42,043DD 224 224 .4 .4 27,944 27,944 1.0 1.0 25,772 25,772DD 336 336 .6 .6 41,916 41,916 1.0 1.0 25,715 25,715DD 448 448 .8 .8 55,888 55,888 1.0 1.0 25,715 25,715DD 596 596 1.0 1.0 74,351 74,351 1.0 1.0 25,715 25,715DD 728 728 1.2 1.2 90,818 90,818 1.0 1.0 25,715 25,715DD 840 840 1.4 1.4 104,790 104,790 1.0 1.0 25,715 25,715DD 952 952 1.6 1.6 118,762 118,762 1.0 1.0 25,715 25,715DD 1,064 1,064 1.8 1.8 132,734 132,734 1.0 1.0 25,715 25,715 Copyright 2006 Kyle Hailey
    23. 23. Step 3: Find Top I/O SQLselect count(*),sql_id, event select count(*),sql_id, eventfrom v$active_session_history from v$active_session_historywhere event in (db file sequential read, where event in (db file sequential read, db file scattered read, db file scattered read, db file parallel read) db file parallel read)group by sql_id, event group by sql_id, eventorder by count(*); order by count(*); COUNT(*) SQL_ID COUNT(*) SQL_ID EVENT EVENT 10 8hk7xvhua40va db file sequential read 10 8hk7xvhua40va db file sequential read 335 3hatpjzrqvfn7 db file sequential read 335 3hatpjzrqvfn7 db file sequential read 343 0uuqgjq7k12nf db file sequential read 343 0uuqgjq7k12nf db file sequential read 549 75621g9y3xmvd db file sequential read 549 75621g9y3xmvd db file sequential read 1311 0bzhqhhj9mpaa db file sequential read 1311 0bzhqhhj9mpaa db file sequential read 1523 41zu158rqf4kf db file sequential read 1523 41zu158rqf4kf db file sequential read 1757 0yas01u2p9ch4 db file sequential read 1757 0yas01u2p9ch4 db file sequential read Copyright 2006 Kyle Hailey
    24. 24. Step 3: OEM find top SQL Copyright 2006 Kyle Hailey
    25. 25. IO Configuration Issues Unix Buffer Cache Disk Spindles Machine Memory Raid 5 Unix File Not Unix File Physical IO SGA Cache Physical IO If average IO read times are under 7ms them probably coming from Unix File Cache Copyright 2006 Kyle Hailey
    26. 26. IO ThroughputOne Disk 74G – Few IOPS 92 IO/sec37 Disks 2G – Many IOPS 1887 IO/sec Copyright 2006 Kyle Hailey
    27. 27. Disk 2Gb vs 70GB Seagate Seagate Barracuda 4LP Cheetah 73LP Capacity 2.16GB 73.4GB Rotation Speed 7200rpm 10000rpm Rotational Delay(avg) 4.1ms 3ms Time to read 32Kb 6ms 3ms Seek Time (avg) 9.4ms 4.9 Total time for Single I/O 19.5ms 10.9ms I/O per second 51 92 (conservative) IOPs/sec per 100GB 2550 126 James Morle http://scaleability.com Copyright 2006 Kyle Hailey
    28. 28. Raid 5http://www.miracleas.com/BAARF/BAARF2.html P1 P2 Readers on differentB – Battle disks run without conflictA – AgainstA – Any P2R – Raid Writes require two readsF - Five plus a write – read data, read parity then write Reads can conflict on P1 smaller stripe sizes 32K or 64K Copyright 2006 Kyle Hailey
    29. 29. Raid Levels Raid Level none 0 0+1 5 (stripe) (striped then mirrored) Control file 2 1 1 3 Redo log 4 1 1 3 System tablespace 2 1 1 3 Sort segment 4 1 1 3 Rollback segment 2 1 1 5 Index read only 2 1 1 1 Sequential read only 4 1 1 3 DBWR intensive files 1 1 1 5 Direct load intensive 4 1 1 3 Data protection 4 5 1 2 Operating costs 1 1 5 3http://www.miracleas.com/BAARF/1.Millsap2000.01.03-RAID5.pdf Copyright 2006 Kyle Hailey
    30. 30. Direct I/O WAITSDirect I/O :This mechanism lets the client bypass the buffer cache forI/O intensive operations.The disk blocks are written into and read from process private memory. direct path read :  Parallel Query direct path write  sqlldr  loading/reading LOBs  parallel DMLs  create table as select  create index direct path read temp , direct path write temp  Sorting Copyright 2006 Kyle Hailey
    31. 31. Direct IO Shadow Process PGA Sort Area Buffer Cache Direct Path PQO Reads X Copyright 2006 Kyle Hailey
    32. 32. direct path read Buffer Cache  Parallel Query  See v$px_session X select parameter1, parameter2, parameter3 from v$event_name where name=direct path read; Shadow PARAMETER1 PARAMETER2 PARAMETER3 Process file number first dba block cnt PGA PQO _serial_direct_read = true Copyright 2006 Kyle Hailey
    33. 33. Further Investigation ASHselect select session_id sid, session_id sid, QC_SESSION_ID qsid, QC_SESSION_ID qsid, ash.p3, ash.p3, CURRENT_OBJ#|| ||o.object_name objn, CURRENT_OBJ#|| ||o.object_name objn, o.object_type otype, o.object_type otype, CURRENT_FILE# fn, CURRENT_FILE# fn, CURRENT_BLOCK# blockn, CURRENT_BLOCK# blockn, ash.SQL_ID ash.SQL_IDfrom v$active_session_history ash, from v$active_session_history ash, all_objects o all_objects owhere event like direct path read where event like direct path read and o.object_id (+)= ash.CURRENT_OBJ# and o.object_id (+)= ash.CURRENT_OBJ#Order by sample_time; Order by sample_time; SID QSID P3 OBJN OTYPE FN BLOCKN SQL_ID SID QSID P3 OBJN OTYPE FN BLOCKN SQL_ID --- ---- -- ----- ------ -- ------ ------------- --- ---- -- ----- ------ -- ------ ------------- 149 152 8 TOTO TABLE 149 152 8 TOTO TABLE 1 194072 4gp8tg0b2s722 1 194072 4gp8tg0b2s722 144 152 8 TOTO TABLE 144 152 8 TOTO TABLE 1 192304 4gp8tg0b2s722 1 192304 4gp8tg0b2s722 60 152 8 TOTO TABLE 60 152 8 TOTO TABLE 1 190592 4gp8tg0b2s722 1 190592 4gp8tg0b2s722 54 152 6 TOTO TABLE 54 152 6 TOTO TABLE 1 201274 4gp8tg0b2s722 1 201274 4gp8tg0b2s722 149 152 8 TOTO TABLE 149 152 8 TOTO TABLE 1 198552 4gp8tg0b2s722 1 198552 4gp8tg0b2s722 Copyright 2006 Kyle Hailey
    34. 34. PQO - ASHselect ash.SQL_ID, QC_SESSION_ID qsid, count(*) cnt, count (distinct session_id) deg, nvl(o.object_name,to_char(CURRENT_OBJ#)) obj, o.object_type otype, decode(session_state, WAITING,event,CPU) eventfrom v$active_session_history ash, all_objects owhere o.object_id (+)= ash.CURRENT_OBJ# and qc_session_id is not null SQL_ID QSID CNT DEG OBJ OTYPE EVENTgroup by qc_session_id, sql_id, o.object_name, 7p3jt75phub2d 144 386 4 WAIT_OBJECTS TABLE PX Deq Credit: send blkd o.object_type, CURRENT_OBJ#, event, session_stateTABLE 144 4 3 WAIT_OBJECTS PX qref latchOrder by qc_session_id, sql_id144 37 1 WAIT_OBJECTS TABLE db file sequential read 144 3 2 WAIT_OBJECTS TABLE direct path read 144 70 1 WAIT_OBJECTS TABLE CPU 144 21 4 0 PX Deq Credit: send blkd 144 12 4 0 db file sequential read
    35. 35. direct path write  Occurs when:  insert /*+ APPEND */  sql*loader direct=y  Create table as select  Loading LOBS  Parallel insert Buffer Cache  Create index Shadow select parameter1, parameter2, parameter3 Process from v$event_name where name=direct path write; DBWR PGA PARAMETER1 PARAMETER2 PARAMETER3 file number first dba block cnt Seems to be a bug on 10g where direct path write waits are DATA incorrectly recorded as CPU Copyright 2006 Kyle Hailey
    36. 36. Direct Path Read TempDirect Path Write Temp select parameter1, parameter2, parameter3  Sorting from v$event_name where name in (direct path write temp‘,  Write to Temp ‘direct path read temp’); PARAMETER1 PARAMETER2 PARAMETER3  Read from Temp file number first dba block cnt file number first dba block cnt Shadow Process PGA TEMP DATA Buffer Cache Copyright 2006 Kyle Hailey
    37. 37. Direct Path Read TempDirect Path Write Temp This event happens when sorting overflows the memory buffers and has to be written to disk. If its a problem consider increasing parameter pga_aggregate_target The explain plan can give estimated sort size requirements and extended row source stats in the explain plan can give the actual sort size usage of the SQL statement Buffer Cache Shadow Process select s.sid, s. process, s.sql_id, SID SQL_ID SEGTYPE MB TABLESPACE tmp.segtype, --- ------------- ------- -- ---------- ((tmp.blocks*8)/1024) MB, 90 gm5s0vc0vzppc SORT 2 TEMP03 PGA from tmp.tablespace 105 gm5s0vc0vzppc SORT 2 TEMP03 v$tempseg_usage tmp, 80 gm5s0vc0vzppc SORT 2 TEMP03 v$session s 105 gm5s0vc0vzppc HASH 2 TEMP03 where tmp.session_num=s.serial# 102 gm5s0vc0vzppc SORT 2 TEMP03 and segtype in (HASH,SORT) 90 gm5s0vc0vzppc HASH 2 TEMP03 order by blocks desc TEMP DATA Copyright 2006 Kyle Hailey
    38. 38. PGA Aggregate Target Manual override broken 10.2.0.3 fixed in 11.1.0.7 alter session set workarea_size_policy = manual; alter session set sort_area_size = 104857600; Workaround, run command twice alter session set sort_area_size = 104857600; alter session set sort_area_size = 104857600; But doesn’t work for parallel connections that still fall back on default which is 64K unless otherwise specified HASH_AREA_SIZE seems unaffected by problem 9i, a single sessions maximum sort area using PGA_AGGREGATE_TARGET was fixed to 100M unless setting _pga_max_size: Maximum PGA size for a single process Defaults to 200M _smm_max_size: Maximum workarea size for one process , defined in KILO BYTES Defaults Min(0.05*pga_aggregate_target,0.5*_pga_max_size,100M) In 10.2 max sort size is 10% of PGA_AGGREGATE_TARGET (100M applies as long as pga_aggregate_target is smaller than 1GB) example : pga_aggregate_target set to 5GB _smm_max_size of 500MB (was 100MB ) _pga_max_size of 1000MB (was 200MB). Copyright 2006 Kyle Hailey
    39. 39. Data file init write  When autoextend is set and many extensions are happening  Can be the root cause of other waits  Enq: TX - contention Copyright 2006 Kyle Hailey
    40. 40. Local Write Wait  Truncating a table  Wait for Data Cache to be cleared of all blocks of truncated table  Wait by shadow for DBWR  If a problem try  GTT  (reduce db cache size , not usually an option) Copyright 2006 Kyle Hailey
    41. 41. Further Investigationselect * from v$event_namewhere name = ‘local write waitNAME NAME P1 P1 P2 P2 P3 P3local write wait file# block# local write wait file# block#select ash.p1,ash.p2, ash.SQL_ID, count(*)from v$ash ashwhere event=local write waitgroup by P1 P2 SQL_ID COUNTash.p1,ash.p2, ash.SQL_ID; 201 2 0q8k9xfmz0k2k 194 201 5 0q8k9xfmz0k2k 34 201 6 0q8k9xfmz0k2k 9 201 4 0q8k9xfmz0k2k 10 Copyright 2006 Kyle Hailey
    42. 42. Temporary File #’sSQL> select file# from v$datafile; FILE# 1 Wait Temporary File#’s = 2 Db_files + file# 3 4 SQL> show parameters db_files 5 NAME VALUE 6 7 db_files 200 8SQL> select file# from v$tempfile; FILE# 2 File# 201 = v$tempfile 1 1 Copyright 2006 Kyle Hailey
    43. 43. Local Write Wait:extent allocation CREATE TEMPORARY TABLESPACE "NEWTEMP" TEMPFILE /d3/temp01.dbf SIZE 100M AUTOEXTEND ON EXTENT MANAGEMENT LOCAL UNIFORM SIZE 64K; ALTER DATABASE DEFAULT TEMPORARY TABLESPACE newtemp; select a.*, b.* from dba_objects a, dba_objects b order by a.owner Copyright 2006 Kyle Hailey
    44. 44. Local Write Waits Copyright 2006 Kyle Hailey
    45. 45. Local Write Wait: Solutions  Truncating a table  Use GTT  ( reduce size of buffer cache – unlikely)  Temporary Table Space Allocations  Increase extent size Copyright 2006 Kyle Hailey
    46. 46. Summary I/O Buffer Cache IO  db file sequential read  db file scattered read  db file parallel read Tune  I/O’s should be < 10 ms  Check Buffer Cache Advisory  Tune SQL Copyright 2006 Kyle Hailey
    47. 47. Summary Direct I/O and Other  direct path read  PQO  direct path write  direct path operations  direct path read temp / direct path write temp  sorts  direct path write (lob)  stored NOCACHE  local write wait –  Temp extensions  Truncates  data file init write – data file extension Copyright 2006 Kyle Hailey
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×