Shared Pool Waits

             Kyle Hailey
http://perfvision.com/ftp/emea2010
  Kyle.hailey@embarcadero.com
Shared Pool Waits
1. Latches
    Latch: Library Cache
    Latch: Shared Pool Latch
1. Mutexes
2. Library Cache locks and pins
3. Row Cache Lock



                  Copyright 2006 Kyle Hailey
Library Cache



        Lib
        Cache




                Copyright 2006 Kyle Hailey
Shared Pool Structure
  Hash
  Table
                                              SQL statements are hashed
          handle
                                              On their text. The resulting
          handle   handle                     Hash is used to find the appropriate
                                              bucket, which is searched for the
                                              Compiled SQL. If it’s not there,
                                              then we parse it.


          handle   handle

          handle

          handle   handle            handle




          handle   handle            handle


          handle


                            Copyright 2006 Kyle Hailey
Shared Pool Latch
 Cause: Hard Parses by Concurrent Sessions
   Contention can arise when too many sessions are hard
    parsing and looking for space in the shared pool.
 Protects Space Allocation in the Shared Pool
   Protects   the structure containing memory chunks
 Ensures two users don’t get same chunk of
  memory




                         Copyright 2006 Kyle Hailey
Shared Pool Latch
Shared Pool Latch covers changes in the lists
of free memory chunks




               Shared Pool Free Space


                      Copyright 2006 Kyle Hailey
Shared Pool Latch 8.1.6

     Bucket   sizes
     0        < 80 bytes
     1        < 144
     2        < 272
     3        < 528                 Shared Pool pre 8.1.6
     4        < 1040                Memory Chunk Buckets
     5         < 2064
     6        < 4112
     7        < 8208
     8        < 16400
     9        < 32784
     10        bigger

                Copyright 2006 Kyle Hailey
Shared Pool Latch


 Shared Pool Latch




            Shared Pool Free Space 8.1.6+
            Before 8.1.6, oversizing the shared pool could be
            a problem, after 8.1.6 should be fine

                         Copyright 2006 Kyle Hailey
Shared Pool Latch
 Shared Pool
 Memory Chunk Buckets 8.1.6 and up
    0          16 bytes
    1          20 bytes
    …          (0-198 only have one chunk size in bucket)
    198        808 bytes
    199        812 to 872
    ….          (199-248 only have 16 possible chunk sizes per bucket)
    248        3948 - 4008
    249        4012 - 4104
    250        4108 - 8204
    251        8204 - 16392
    252        16396 - 32776
    253        32780 - 65544
    254        bigger


                          Copyright 2006 Kyle Hailey
Library Cache Latches
 Protects changes in Library Cache
 Library Locks are not atomic
      Thus need library cache latch
 Broken out into
    library cache pin allocation
    library cache lock allocation
    library cache lock
    library cache
    library cache pin
    library cache load lock




                              Copyright 2006 Kyle Hailey
Library Cache
   Hash Table




                pin      lock

                pin      lock

                handle          handle




                                                   Find and Lock

                                                   Pin (and Load)

                                         Copyright 2006 Kyle Hailey
Library Cache Structures
   Hash Table         waiters

                  pin      lock

                  pin      lock
                                                                 Library Cache Latches

                pin     lock
                                  holders
                pin     lock

                Handle
                handle                handle            handle
                Cursor(0)
                flags
                Cursor (0)
                                                                          Heap 1
                                       pin     lock
                                                       Heap 0
                                                                          Heap 6
                                       pin     lock

                                    Child cursor 1

                                    Child cursor
                                    2

                                   Child cursor 3

                                             Copyright 2006 Kyle Hailey
Library Cache Latch Contention
Library Cache Latch Contention because of these
  possibilities:

 Excessive Hard Parsing
   Not Sharing SQL – use of Literal Values
   Shared Pool too small
   Too many invalidations

 Excessive Soft Parsing




                       Copyright 2006 Kyle Hailey
Sharing SQL & Literals

     select
      select
                plan_hash_value,
                 plan_hash_value,
               count(plan_hash_value)
                count(plan_hash_value)
     from
      from
            v$sql
             v$sql
     group by plan_hash_value
     group by plan_hash_value
     order by count(plan_hash_value)
     order by count(plan_hash_value)

     SQL> @dups
      SQL> @dups
     PLAN_HASH_VALUE
      PLAN_HASH_VALUE        CNT
                              CNT
     --------------- ----------
      --------------- ----------
           272002086
            272002086        520
                              520
                        Copyright 2006 Kyle Hailey
Sharing SQL & Literals
                                  SQL> @dups
select sql_text
 select sql_text                   SQL> @dups
  from v$sql
   from v$sql                     PLAN_HASH_VALUE CNT
                                   PLAN_HASH_VALUE CNT
where
 where
                                  --------------- ----
                                   --------------- ----
                                         272002086 520
     plan_hash_value = 272002086
      plan_hash_value = 272002086        272002086 520

   and rownum < 10;
   and rownum < 10;
    SQL_TEXT
     SQL_TEXT
    -----------------------------------------------
     -----------------------------------------------
    SELECT * FROM dual WHERE dummy=-634891633
     SELECT * FROM dual WHERE dummy=-634891633
    SELECT * FROM dual WHERE dummy=1987751014
     SELECT * FROM dual WHERE dummy=1987751014
    SELECT * FROM dual WHERE dummy=25965276
     SELECT * FROM dual WHERE dummy=25965276
    SELECT * FROM dual WHERE dummy=32449789
     SELECT * FROM dual WHERE dummy=32449789
    SELECT * FROM dual WHERE dummy=-364632215
     SELECT * FROM dual WHERE dummy=-364632215
    SELECT * FROM dual WHERE dummy=-34273351
     SELECT * FROM dual WHERE dummy=-34273351
    SELECT * FROM dual WHERE dummy=-699712683
     SELECT * FROM dual WHERE dummy=-699712683
    SELECT * FROM dual WHERE dummy=1752437199
     SELECT * FROM dual WHERE dummy=1752437199
    SELECT * FROM dual WHEREKyledummy=-1081512404
     SELECT * FROM dual WHERE dummy=-1081512404
                     Copyright 2006 Hailey
Cursor Sharing



   Bind Variables
     Select * from dual where dummy = :var;
     Select sum(money) from orders where
      odate > :odate;
   Cursor_Sharing
     Cursor_sharing     = Force
      Oracle replaces variables with bind variables
     Defaults   to Exact
                     Copyright 2006 Kyle Hailey
Shared Pool too Small


    SQL> select namespace, reloads
    SQL> select namespace, reloads
         from v$librarycache;
         from v$librarycache;
    NAMESPACE
    NAMESPACE          RELOADS
                       RELOADS
    --------------- ----------
    --------------- ----------
    SQL AREA
    SQL AREA               367
                           367
    TABLE/PROCEDURE
    TABLE/PROCEDURE        592
                           592

   Reloads means Cursor heaps were kicked
    out implying shared_pool too small

                  Copyright 2006 Kyle Hailey
Invalidations
       SQL> select namespace,
        SQL> select namespace,
               invalidations
                invalidations
               from v$librarycache;
                from v$librarycache;
       NAMESPACE
        NAMESPACE        INVALIDATIONS
                          INVALIDATIONS
       --------------- -------------
        --------------- -------------
       SQL AREA
        SQL AREA                   6065
                                    6065
 Changes in dependent objects invalidate
  cursor
  FOR i ININ 1..3000 LOOP
   FOR i 1..3000 LOOP
     l_cursor:=dbms_sql.open_cursor;
        l_cursor:=dbms_sql.open_cursor;
     dbms_sql.parse(l_cursor,
        dbms_sql.parse(l_cursor,
      'SELECT * FROM toto',dbms_sql.native);
         'SELECT * FROM toto',dbms_sql.native);
     execute immediate 'analyze table toto compute statistics';
        execute immediate 'analyze table toto compute statistics';
     dbms_sql.close_cursor(l_cursor);
        dbms_sql.close_cursor(l_cursor);
  END LOOP;
   END LOOP;
                          Copyright 2006 Kyle Hailey
Soft Parsing
                            Cursor Memory
  = Latch
             lock          lock          lock      lock           lock


      pin            pin                 pin              pin             pin




     Execute 1      Execute 2          Execute 3      Execute 4          Execute 5

            Re-Executing a Cursor
            1.   Libray Cache latch
            2.   Locks
            3.   Pins
                    Copyright 2006 Kyle Hailey
Session Cached Cursors

= Latch                Cursor Memory

                                   lock


    pin         pin                pin                   pin         pin




   Execute 1   Execute 2         Execute 3              Execute 4   Execute 5


Session_cached_cursor:
      If Opening/Closing keeps locked in Memory
                           Copyright 2006 Kyle Hailey
Session Cached Cursors
 FOR i iIN 1..30000 LOOP
  FOR IN 1..30000 LOOP
    l_cursor:=dbms_sql.open_cursor;
     l_cursor:=dbms_sql.open_cursor;
    dbms_sql.parse(l_cursor,'SELECT **FROM dual’,dbms_sql.native);
     dbms_sql.parse(l_cursor,'SELECT FROM dual’,dbms_sql.native);
    dbms_sql.close_cursor(l_cursor);
     dbms_sql.close_cursor(l_cursor);
 END LOOP;
  END LOOP;

Session_cached_cursors=0
Latch                                                 Gets
-----                                                 ----
library     cache lock                             120,028
library     cache                                  180,074
library     cache pin                               60,048
Session_cached_cursors=20
library cache lock                                        4
library cache                                        60,061
library cache pin 2006 Kyle Hailey
               Copyright
                                                     60,048
Cursor Space for Time

  = Latch                  Cursor Memory

Open
                                         lock
                                                                                      Close
Cursor                                    pin
                                                                                      Cursor




         Execute 1   Execute 2         Execute 3              Execute 4   Execute 5

 Cursor_space_for_time=true :
          if open and re-executing – keeps cursor pinned
          (Cursor already locked because cursor is kept open)
                                 Copyright 2006 Kyle Hailey
Cursor Space For Time
FOR i iIN 1..30000 LOOP
FOR IN 1..30000 LOOP
        rc:=dbms_sql.execute(l_cursor);
         rc:=dbms_sql.execute(l_cursor);
        IF DBMS_SQL.FETCH_ROWS (l_cursor) < 0 THEN
         IF DBMS_SQL.FETCH_ROWS (l_cursor) < 0 THEN
            DBMS_SQL.COLUMN_VALUE (l_cursor, 1, cnt);
             DBMS_SQL.COLUMN_VALUE (l_cursor, 1, cnt);
      end if;
        end if;
End loop;
End loop;
Cursor_space_for_time=false
Latch                                                Gets
-----                                                ----
library    cache lock                                  35
library    cache                                   60,096
library    cache pin                               60,044
Cursor_space_for_time=true
library    cache lock                                 30
library    cache                                      85
library    cache pin                                  42
                      Copyright 2006 Kyle Hailey
Cursor Sharing


   pin   lock

   pin   lock

   Handle
   handle
    handle              handle   handle
   Cursor(0)
   flags
   Cursor (0)
                                          select * from (
                                          select sql_id, count(*) cnt
               Child cursor 0
                                          from V$SQL_SHARED_CURSOR
                                          group by sql_id )
                                          where cnt > 5
               Child cursor 1             order by cnt;

                Child cursor 2



                                    Copyright 2006 Kyle Hailey
V$SQL_SHARED_CURSOR

  10gR2, 53 reasons why cursors aren’t shared
  If using “cursor_sharing=similar” might not
   work – bugs
  Examples
      OPTIMIZER_MODE_MISMATCH , see V$SQL_OPTIMIZER_ENV
      STATS_ROW_MISMATCH, could be sql trace
    AUTH_CHECK_MISMATCH
     TRANSLATION_MISMATCH – different object in SQL stmt
    BIND_MISMATCH – bind variable different sizes
    LANGUAGE_MISMATCH – NLS Language


  http://www.juliandyke.com/Presentations/Presentations.html#LibraryCacheInternals

                             Copyright 2006 Kyle Hailey
V$SQL_SHARED_CURSOR
 UNBOUND_CURSOR                        USER_BIND_PEEK_MISMATCH
 SQL_TYPE_MISMATCH
 OPTIMIZER_MISMATCH                    TYPCHK_DEP_MISMATCH
 OUTLINE_MISMATCH                      NO_TRIGGER_MISMATCH
 STATS_ROW_MISMATCH                    FLASHBACK_CURSOR
 LITERAL_MISMATCH                      ANYDATA_TRANSFORMATION
 SEC_DEPTH_MISMATCH                    INCOMPLETE_CURSOR
 EXPLAIN_PLAN_CURSOR                   TOP_LEVEL_RPI_CURSOR
 BUFFERED_DML_MISMATCH                 DIFFERENT_LONG_LENGTH
 PDML_ENV_MISMATCH                     LOGICAL_STANDBY_APPLY
 INST_DRTLD_MISMATCH                   DIFF_CALL_DURN
 SLAVE_QC_MISMATCH                     BIND_UACS_DIFF
 TYPECHECK_MISMATCH                    PLSQL_CMP_SWITCHS_DIFF
 AUTH_CHECK_MISMATCH                   CURSOR_PARTS_MISMATCH
 BIND_MISMATCH                         STB_OBJECT_MISMATCH
 DESCRIBE_MISMATCH                     ROW_SHIP_MISMATCH
 LANGUAGE_MISMATCH                     PQ_SLAVE_MISMATCH
 TRANSLATION_MISMATCH                  TOP_LEVEL_DDL_MISMATCH
 ROW_LEVEL_SEC_MISMATCH                MULTI_PX_MISMATCH
 INSUFF_PRIVS                          BIND_PEEKED_PQ_MISMATCH
 INSUFF_PRIVS_REM                      MV_REWRITE_MISMATCH
 REMOTE_TRANS_MISMATCH                 ROLL_INVALID_MISMATCH
 LOGMINER_SESSION_MISMATCH             OPTIMIZER_MODE_MISMATCH
 INCOMP_LTRL_MISMATCH                  PX_MISMATCH
 OVERLAP_TIME_MISMATCH                 MV_STALEOBJ_MISMATCH
 SQL_REDIRECT_MISMATCH                 FLASHBACK_TABLE_MISMATCH
 MV_QUERY_GEN_MISMATCH                 LITREP_COMP_MISMATCH

                       Copyright 2006 Kyle Hailey
10g : Mutex

  A new semaphore method which can replace latches

   Mutex
        Mutual exclusion object
   Similar to a latch, prevents
      Deallocation while someone is using it
      Read/write while someone else is modifying

   Different from latch
      Every object can have it’s own mutex
      A mutex can cover multiple objects
      Usually dynamically allocated along with structure they
       protect
      Can be stored in the structure, thus destroying structure
       deletes the mutex

                             Copyright 2006 Kyle Hailey
Mutex Views and Stats
                                                                      Views
                                                                        V$mutex_sleep
    Cursor:pin S                                                       V$mutex_sleep_history
          Pin cursor for execute, and cursor is currently being
           examined by another Session                                      Instead of “library cache pin”
          Bug 6968152                                                      cursor_space_for_time
    Cursor:pin S wait on X                                                 not needed
          Pinning a cursor for execute
          Bug on 10.2.0.3 typically with DBMS_STATS
          Metalink Note:401435.1, Note:5907779.8, bug 5907779



Instead of latching for execute pin we use a shared mutex
If can’t get the mutex spin                                           Fixed 10.2.0.4 ?
                                                                      Turn off with
                                                                      _kks_use_mutex_pin=FALSE"




                                        Copyright 2006 Kyle Hailey
Debugging Pin Waits
 NAME                        P1      P2         P3
 -------------------------   -----   -------   ---------------
 cursor: mutex X             idn     value     where|sleeps
 cursor: mutex S             idn     value     where|sleeps
 cursor: pin S wait on X     idn     value     where|sleeps
 cursor: pin X               idn     value     where|sleeps
 cursor: pin S               idn     value     where|sleeps

  select p1, p2raw, count(*)
   from v$session
   where event = ‘cursor: pin S wait on X’
     and wait_time = 0
  Group by p1, p2;                                                 Images from Marcin Przepiorowski

          P1   P2RAW            COUNT(*)
  ----------   ---------------- --------
  2700259466   0000139700000000        9
  <Mutex Id>   <   SId><RefCnt>

 select to_number(1397,'XXXX') from dual;
   TO_NUMBER(1397,'XXXX')
   ----------------------
                     5015
 Select serial# from v$session where sid=5015;
   SERIAL#
   --------
        82

 Alter system kill session ‘5015,82’;
                                      Copyright 2006 Kyle Hailey
lock and pins
 1. Library Cache Pin
 2. Library Cache Lock
 3. Library Cache Load Lock

  Contention when Sessions try to
    load/compile same SQL
    Compile package others are running

  Locks and Pins are usually in share
   mode unless modifications are being
   made
                    Copyright 2006 Kyle Hailey
Lib Cache Lock : blockers and waiters

 select
  select
     waiter.sid waiter,
      waiter.sid waiter,
     waiter.event wevent,
      waiter.event wevent,
     to_char(blocker_event.sid)||','||to_char(blocker_session.serial#) blocker,
      to_char(blocker_event.sid)||','||to_char(blocker_session.serial#) blocker,
     substr(decode(blocker_event.wait_time,
      substr(decode(blocker_event.wait_time,
               0, blocker_event.event,
                0, blocker_event.event,
              'ON CPU'),1,30) bevent
               'ON CPU'),1,30) bevent
 from
  from                                              WAITER WLOCKP1                       WEVENT                   BLOCKER BEVENT
     x$kglpn p,                                       WAITER WLOCKP1                       WEVENT                   BLOCKER BEVENT
      x$kglpn p,                                    ------- ---------------- ----------------- --------- -----------------
                                                      ------- ---------------- ----------------- --------- -----------------
     gv$session
      gv$session blocker_session,
                       blocker_session,             129 00000003B76AB620 library cache pin 135,15534 PL/SQL lock
     gv$session_wait waiter,                        timer 00000003B76AB620 library cache pin 135,15534 PL/SQL lock
                                                      129
      gv$session_wait waiter,                         timer
     gv$session_wait blocker_event
      gv$session_wait blocker_event
 where
  where
        p.kglpnuse=blocker_session.saddr
         p.kglpnuse=blocker_session.saddr
   and p.kglpnhdl=waiter.p1raw
    and p.kglpnhdl=waiter.p1raw
   and waiter.event in ( ('library cache pin' , ,
    and waiter.event in 'library cache pin'
                             'library cache lock' , ,
                               'library cache lock'
                             'library cache load lock')
                               'library cache load lock')
   and blocker_event.sid=blocker_session.sid
    and blocker_event.sid=blocker_session.sid
   and waiter.sid != blocker_event.sid
    and waiter.sid != blocker_event.sid
 order by
  order by
     waiter.p1raw,waiter.sid;
      waiter.p1raw,waiter.sid;
                                                     Copyright 2006 Kyle Hailey
Solutions
1. Library Cache Pin
2. Library Cache Lock
3. Library Cache Load Lock

 Have only one Session compile the same cursor
  at a time
 Avoid compiling while executing
 Waits – find “competing” Sessions




                   Copyright 2006 Kyle Hailey
row cache lock : args
 P1 = cache#
 P2 = Lock Mode Held
 P3 = Lock Mode Requested




               select parameter as “name”
                select parameter as “name”
               from v$rowcache
                from v$rowcache
               where cache# = P1;
                where cache# = P1;



                   Copyright 2006 Kyle Hailey
Row Cache Lock - ASH
select
    ash.session_id sid,
    ash.blocking_session bsid,
    nvl(o.object_name,to_char(CURRENT_OBJ#)) obj,
    o.object_type otype,
    CURRENT_FILE# filen,
    CURRENT_BLOCK# blockn,
    ash.SQL_ID,
    nvl(rc.name,to_char(ash.p3)) row_cache
from v$active_session_history ash,
   ( select cache#, parameter name from v$rowcache ) rc,
   all_objects o
where event='row cache lock'
 and rc.cache#(+)=ash.p1
 and o.object_id (+)= ash.CURRENT_OBJ#
 and ash.session_state='WAITING'
 and ash.sample_time > sysdate - &minutes/(60*24)
Order by sample_time
Row Cache Lock - ASH
select
    ash.session_id sid,
    ash.blocking_session bsid,
    nvl(o.object_name,to_char(CURRENT_OBJ#)) obj,
    o.object_type otype,
    CURRENT_FILE# filen,
    CURRENT_BLOCK# blockn,
    ash.SQL_ID,
    nvl(rc.name,to_char(ash.p3)) row_cache
from v$active_session_history ash,
   ( select cache#, parameter name from v$rowcache ) rc,
   all_objects o BSID OBJ
          SID                    OTYPE FILEN BLOCKN          SQL_ID        ROW_CACHE
where event='row cache lock'
          143 131         -1                0              0 41y8w0sfqb61m dc_sequences
 and rc.cache#(+)=ash.p1
          134 131         -1                0              0               dc_sequences
 and o.object_id (+)= ash.CURRENT_OBJ#
          151             -1                0              0               dc_sequences
 and ash.session_state='WAITING'
          134 151         -1                0              0               dc_sequences
 and ash.sample_time > sysdate - &minutes/(60*24)
          131 151         -1                0              0               dc_sequences
Order by sample_time
          151             -1                0              0               dc_sequences
Row Cache Lock


    Select seq.next_val
    Sequence cache set to 1
    Default sequence cache is 20

   SQL> @sqltext
   Enter value for 1: 41y8w0sfqb61m
   SQL_FULLTEXT

   SELECT TOTO_SEQ.NEXTVAL FROM DUAL


                   Copyright 2006 Kyle Hailey
Library Cache Latch Solutions

 Share Cursors
   Use bind variables
   User cursor_sharing=force

 Avoid invalidations and reloads
   Size shared_pool large enough
   Avoid changing dependent objects

 Soft Parsing
   Session_cached_cursors    =20 : keep across open/close
   Cursor_space_for_time=true : keep pinned across executes
   hold_cursor=true : used in precompilers


                      Copyright 2006 Kyle Hailey
Summary


 Shared Pool Latch
   Shared    pool too small or too much hard parsing
 Loading Same Cursor
   Library Cache Pin
   Library Cache Lock
   Library Cache Load Lock

 Row Cache Lock
   Depends    on the cache


                      Copyright 2006 Kyle Hailey

Oracle 10g Performance: chapter 10 libc

  • 1.
    Shared Pool Waits Kyle Hailey http://perfvision.com/ftp/emea2010 Kyle.hailey@embarcadero.com
  • 2.
    Shared Pool Waits 1.Latches  Latch: Library Cache  Latch: Shared Pool Latch 1. Mutexes 2. Library Cache locks and pins 3. Row Cache Lock Copyright 2006 Kyle Hailey
  • 3.
    Library Cache Lib Cache Copyright 2006 Kyle Hailey
  • 4.
    Shared Pool Structure Hash Table SQL statements are hashed handle On their text. The resulting handle handle Hash is used to find the appropriate bucket, which is searched for the Compiled SQL. If it’s not there, then we parse it. handle handle handle handle handle handle handle handle handle handle Copyright 2006 Kyle Hailey
  • 5.
    Shared Pool Latch Cause: Hard Parses by Concurrent Sessions  Contention can arise when too many sessions are hard parsing and looking for space in the shared pool.  Protects Space Allocation in the Shared Pool  Protects the structure containing memory chunks  Ensures two users don’t get same chunk of memory Copyright 2006 Kyle Hailey
  • 6.
    Shared Pool Latch SharedPool Latch covers changes in the lists of free memory chunks Shared Pool Free Space Copyright 2006 Kyle Hailey
  • 7.
    Shared Pool Latch8.1.6 Bucket sizes 0 < 80 bytes 1 < 144 2 < 272 3 < 528 Shared Pool pre 8.1.6 4 < 1040 Memory Chunk Buckets 5 < 2064 6 < 4112 7 < 8208 8 < 16400 9 < 32784 10 bigger Copyright 2006 Kyle Hailey
  • 8.
    Shared Pool Latch Shared Pool Latch Shared Pool Free Space 8.1.6+ Before 8.1.6, oversizing the shared pool could be a problem, after 8.1.6 should be fine Copyright 2006 Kyle Hailey
  • 9.
    Shared Pool Latch Shared Pool Memory Chunk Buckets 8.1.6 and up 0 16 bytes 1 20 bytes … (0-198 only have one chunk size in bucket) 198 808 bytes 199 812 to 872 …. (199-248 only have 16 possible chunk sizes per bucket) 248 3948 - 4008 249 4012 - 4104 250 4108 - 8204 251 8204 - 16392 252 16396 - 32776 253 32780 - 65544 254 bigger Copyright 2006 Kyle Hailey
  • 10.
    Library Cache Latches Protects changes in Library Cache  Library Locks are not atomic  Thus need library cache latch  Broken out into  library cache pin allocation  library cache lock allocation  library cache lock  library cache  library cache pin  library cache load lock Copyright 2006 Kyle Hailey
  • 11.
    Library Cache Hash Table pin lock pin lock handle handle Find and Lock Pin (and Load) Copyright 2006 Kyle Hailey
  • 12.
    Library Cache Structures Hash Table waiters pin lock pin lock Library Cache Latches pin lock holders pin lock Handle handle handle handle Cursor(0) flags Cursor (0) Heap 1 pin lock Heap 0 Heap 6 pin lock Child cursor 1 Child cursor 2 Child cursor 3 Copyright 2006 Kyle Hailey
  • 13.
    Library Cache LatchContention Library Cache Latch Contention because of these possibilities:  Excessive Hard Parsing  Not Sharing SQL – use of Literal Values  Shared Pool too small  Too many invalidations  Excessive Soft Parsing Copyright 2006 Kyle Hailey
  • 14.
    Sharing SQL &Literals select select plan_hash_value, plan_hash_value, count(plan_hash_value) count(plan_hash_value) from from v$sql v$sql group by plan_hash_value group by plan_hash_value order by count(plan_hash_value) order by count(plan_hash_value) SQL> @dups SQL> @dups PLAN_HASH_VALUE PLAN_HASH_VALUE CNT CNT --------------- ---------- --------------- ---------- 272002086 272002086 520 520 Copyright 2006 Kyle Hailey
  • 15.
    Sharing SQL &Literals SQL> @dups select sql_text select sql_text SQL> @dups from v$sql from v$sql PLAN_HASH_VALUE CNT PLAN_HASH_VALUE CNT where where --------------- ---- --------------- ---- 272002086 520 plan_hash_value = 272002086 plan_hash_value = 272002086 272002086 520 and rownum < 10; and rownum < 10; SQL_TEXT SQL_TEXT ----------------------------------------------- ----------------------------------------------- SELECT * FROM dual WHERE dummy=-634891633 SELECT * FROM dual WHERE dummy=-634891633 SELECT * FROM dual WHERE dummy=1987751014 SELECT * FROM dual WHERE dummy=1987751014 SELECT * FROM dual WHERE dummy=25965276 SELECT * FROM dual WHERE dummy=25965276 SELECT * FROM dual WHERE dummy=32449789 SELECT * FROM dual WHERE dummy=32449789 SELECT * FROM dual WHERE dummy=-364632215 SELECT * FROM dual WHERE dummy=-364632215 SELECT * FROM dual WHERE dummy=-34273351 SELECT * FROM dual WHERE dummy=-34273351 SELECT * FROM dual WHERE dummy=-699712683 SELECT * FROM dual WHERE dummy=-699712683 SELECT * FROM dual WHERE dummy=1752437199 SELECT * FROM dual WHERE dummy=1752437199 SELECT * FROM dual WHEREKyledummy=-1081512404 SELECT * FROM dual WHERE dummy=-1081512404 Copyright 2006 Hailey
  • 16.
    Cursor Sharing  Bind Variables  Select * from dual where dummy = :var;  Select sum(money) from orders where odate > :odate;  Cursor_Sharing  Cursor_sharing = Force Oracle replaces variables with bind variables  Defaults to Exact Copyright 2006 Kyle Hailey
  • 17.
    Shared Pool tooSmall SQL> select namespace, reloads SQL> select namespace, reloads from v$librarycache; from v$librarycache; NAMESPACE NAMESPACE RELOADS RELOADS --------------- ---------- --------------- ---------- SQL AREA SQL AREA 367 367 TABLE/PROCEDURE TABLE/PROCEDURE 592 592  Reloads means Cursor heaps were kicked out implying shared_pool too small Copyright 2006 Kyle Hailey
  • 18.
    Invalidations SQL> select namespace, SQL> select namespace, invalidations invalidations from v$librarycache; from v$librarycache; NAMESPACE NAMESPACE INVALIDATIONS INVALIDATIONS --------------- ------------- --------------- ------------- SQL AREA SQL AREA 6065 6065  Changes in dependent objects invalidate cursor FOR i ININ 1..3000 LOOP FOR i 1..3000 LOOP l_cursor:=dbms_sql.open_cursor; l_cursor:=dbms_sql.open_cursor; dbms_sql.parse(l_cursor, dbms_sql.parse(l_cursor, 'SELECT * FROM toto',dbms_sql.native); 'SELECT * FROM toto',dbms_sql.native); execute immediate 'analyze table toto compute statistics'; execute immediate 'analyze table toto compute statistics'; dbms_sql.close_cursor(l_cursor); dbms_sql.close_cursor(l_cursor); END LOOP; END LOOP; Copyright 2006 Kyle Hailey
  • 19.
    Soft Parsing Cursor Memory = Latch lock lock lock lock lock pin pin pin pin pin Execute 1 Execute 2 Execute 3 Execute 4 Execute 5 Re-Executing a Cursor 1. Libray Cache latch 2. Locks 3. Pins Copyright 2006 Kyle Hailey
  • 20.
    Session Cached Cursors =Latch Cursor Memory lock pin pin pin pin pin Execute 1 Execute 2 Execute 3 Execute 4 Execute 5 Session_cached_cursor: If Opening/Closing keeps locked in Memory Copyright 2006 Kyle Hailey
  • 21.
    Session Cached Cursors FOR i iIN 1..30000 LOOP FOR IN 1..30000 LOOP l_cursor:=dbms_sql.open_cursor; l_cursor:=dbms_sql.open_cursor; dbms_sql.parse(l_cursor,'SELECT **FROM dual’,dbms_sql.native); dbms_sql.parse(l_cursor,'SELECT FROM dual’,dbms_sql.native); dbms_sql.close_cursor(l_cursor); dbms_sql.close_cursor(l_cursor); END LOOP; END LOOP; Session_cached_cursors=0 Latch Gets ----- ---- library cache lock 120,028 library cache 180,074 library cache pin 60,048 Session_cached_cursors=20 library cache lock 4 library cache 60,061 library cache pin 2006 Kyle Hailey Copyright 60,048
  • 22.
    Cursor Space forTime = Latch Cursor Memory Open lock Close Cursor pin Cursor Execute 1 Execute 2 Execute 3 Execute 4 Execute 5 Cursor_space_for_time=true : if open and re-executing – keeps cursor pinned (Cursor already locked because cursor is kept open) Copyright 2006 Kyle Hailey
  • 23.
    Cursor Space ForTime FOR i iIN 1..30000 LOOP FOR IN 1..30000 LOOP rc:=dbms_sql.execute(l_cursor); rc:=dbms_sql.execute(l_cursor); IF DBMS_SQL.FETCH_ROWS (l_cursor) < 0 THEN IF DBMS_SQL.FETCH_ROWS (l_cursor) < 0 THEN DBMS_SQL.COLUMN_VALUE (l_cursor, 1, cnt); DBMS_SQL.COLUMN_VALUE (l_cursor, 1, cnt); end if; end if; End loop; End loop; Cursor_space_for_time=false Latch Gets ----- ---- library cache lock 35 library cache 60,096 library cache pin 60,044 Cursor_space_for_time=true library cache lock 30 library cache 85 library cache pin 42 Copyright 2006 Kyle Hailey
  • 24.
    Cursor Sharing pin lock pin lock Handle handle handle handle handle Cursor(0) flags Cursor (0) select * from ( select sql_id, count(*) cnt Child cursor 0 from V$SQL_SHARED_CURSOR group by sql_id ) where cnt > 5 Child cursor 1 order by cnt; Child cursor 2 Copyright 2006 Kyle Hailey
  • 25.
    V$SQL_SHARED_CURSOR  10gR2,53 reasons why cursors aren’t shared  If using “cursor_sharing=similar” might not work – bugs  Examples  OPTIMIZER_MODE_MISMATCH , see V$SQL_OPTIMIZER_ENV  STATS_ROW_MISMATCH, could be sql trace  AUTH_CHECK_MISMATCH TRANSLATION_MISMATCH – different object in SQL stmt  BIND_MISMATCH – bind variable different sizes  LANGUAGE_MISMATCH – NLS Language http://www.juliandyke.com/Presentations/Presentations.html#LibraryCacheInternals Copyright 2006 Kyle Hailey
  • 26.
    V$SQL_SHARED_CURSOR UNBOUND_CURSOR USER_BIND_PEEK_MISMATCH SQL_TYPE_MISMATCH OPTIMIZER_MISMATCH TYPCHK_DEP_MISMATCH OUTLINE_MISMATCH NO_TRIGGER_MISMATCH STATS_ROW_MISMATCH FLASHBACK_CURSOR LITERAL_MISMATCH ANYDATA_TRANSFORMATION SEC_DEPTH_MISMATCH INCOMPLETE_CURSOR EXPLAIN_PLAN_CURSOR TOP_LEVEL_RPI_CURSOR BUFFERED_DML_MISMATCH DIFFERENT_LONG_LENGTH PDML_ENV_MISMATCH LOGICAL_STANDBY_APPLY INST_DRTLD_MISMATCH DIFF_CALL_DURN SLAVE_QC_MISMATCH BIND_UACS_DIFF TYPECHECK_MISMATCH PLSQL_CMP_SWITCHS_DIFF AUTH_CHECK_MISMATCH CURSOR_PARTS_MISMATCH BIND_MISMATCH STB_OBJECT_MISMATCH DESCRIBE_MISMATCH ROW_SHIP_MISMATCH LANGUAGE_MISMATCH PQ_SLAVE_MISMATCH TRANSLATION_MISMATCH TOP_LEVEL_DDL_MISMATCH ROW_LEVEL_SEC_MISMATCH MULTI_PX_MISMATCH INSUFF_PRIVS BIND_PEEKED_PQ_MISMATCH INSUFF_PRIVS_REM MV_REWRITE_MISMATCH REMOTE_TRANS_MISMATCH ROLL_INVALID_MISMATCH LOGMINER_SESSION_MISMATCH OPTIMIZER_MODE_MISMATCH INCOMP_LTRL_MISMATCH PX_MISMATCH OVERLAP_TIME_MISMATCH MV_STALEOBJ_MISMATCH SQL_REDIRECT_MISMATCH FLASHBACK_TABLE_MISMATCH MV_QUERY_GEN_MISMATCH LITREP_COMP_MISMATCH Copyright 2006 Kyle Hailey
  • 27.
    10g : Mutex A new semaphore method which can replace latches  Mutex  Mutual exclusion object  Similar to a latch, prevents  Deallocation while someone is using it  Read/write while someone else is modifying  Different from latch  Every object can have it’s own mutex  A mutex can cover multiple objects  Usually dynamically allocated along with structure they protect  Can be stored in the structure, thus destroying structure deletes the mutex Copyright 2006 Kyle Hailey
  • 28.
    Mutex Views andStats  Views  V$mutex_sleep  Cursor:pin S  V$mutex_sleep_history  Pin cursor for execute, and cursor is currently being examined by another Session Instead of “library cache pin”  Bug 6968152 cursor_space_for_time  Cursor:pin S wait on X not needed  Pinning a cursor for execute  Bug on 10.2.0.3 typically with DBMS_STATS  Metalink Note:401435.1, Note:5907779.8, bug 5907779 Instead of latching for execute pin we use a shared mutex If can’t get the mutex spin Fixed 10.2.0.4 ? Turn off with _kks_use_mutex_pin=FALSE" Copyright 2006 Kyle Hailey
  • 29.
    Debugging Pin Waits NAME P1 P2 P3 ------------------------- ----- ------- --------------- cursor: mutex X idn value where|sleeps cursor: mutex S idn value where|sleeps cursor: pin S wait on X idn value where|sleeps cursor: pin X idn value where|sleeps cursor: pin S idn value where|sleeps select p1, p2raw, count(*) from v$session where event = ‘cursor: pin S wait on X’ and wait_time = 0 Group by p1, p2; Images from Marcin Przepiorowski P1 P2RAW COUNT(*) ---------- ---------------- -------- 2700259466 0000139700000000 9 <Mutex Id> < SId><RefCnt> select to_number(1397,'XXXX') from dual; TO_NUMBER(1397,'XXXX') ---------------------- 5015 Select serial# from v$session where sid=5015; SERIAL# -------- 82 Alter system kill session ‘5015,82’; Copyright 2006 Kyle Hailey
  • 30.
    lock and pins 1. Library Cache Pin 2. Library Cache Lock 3. Library Cache Load Lock  Contention when Sessions try to  load/compile same SQL  Compile package others are running  Locks and Pins are usually in share mode unless modifications are being made Copyright 2006 Kyle Hailey
  • 31.
    Lib Cache Lock: blockers and waiters select select waiter.sid waiter, waiter.sid waiter, waiter.event wevent, waiter.event wevent, to_char(blocker_event.sid)||','||to_char(blocker_session.serial#) blocker, to_char(blocker_event.sid)||','||to_char(blocker_session.serial#) blocker, substr(decode(blocker_event.wait_time, substr(decode(blocker_event.wait_time, 0, blocker_event.event, 0, blocker_event.event, 'ON CPU'),1,30) bevent 'ON CPU'),1,30) bevent from from WAITER WLOCKP1 WEVENT BLOCKER BEVENT x$kglpn p, WAITER WLOCKP1 WEVENT BLOCKER BEVENT x$kglpn p, ------- ---------------- ----------------- --------- ----------------- ------- ---------------- ----------------- --------- ----------------- gv$session gv$session blocker_session, blocker_session, 129 00000003B76AB620 library cache pin 135,15534 PL/SQL lock gv$session_wait waiter, timer 00000003B76AB620 library cache pin 135,15534 PL/SQL lock 129 gv$session_wait waiter, timer gv$session_wait blocker_event gv$session_wait blocker_event where where p.kglpnuse=blocker_session.saddr p.kglpnuse=blocker_session.saddr and p.kglpnhdl=waiter.p1raw and p.kglpnhdl=waiter.p1raw and waiter.event in ( ('library cache pin' , , and waiter.event in 'library cache pin' 'library cache lock' , , 'library cache lock' 'library cache load lock') 'library cache load lock') and blocker_event.sid=blocker_session.sid and blocker_event.sid=blocker_session.sid and waiter.sid != blocker_event.sid and waiter.sid != blocker_event.sid order by order by waiter.p1raw,waiter.sid; waiter.p1raw,waiter.sid; Copyright 2006 Kyle Hailey
  • 32.
    Solutions 1. Library CachePin 2. Library Cache Lock 3. Library Cache Load Lock  Have only one Session compile the same cursor at a time  Avoid compiling while executing  Waits – find “competing” Sessions Copyright 2006 Kyle Hailey
  • 33.
    row cache lock: args  P1 = cache#  P2 = Lock Mode Held  P3 = Lock Mode Requested select parameter as “name” select parameter as “name” from v$rowcache from v$rowcache where cache# = P1; where cache# = P1; Copyright 2006 Kyle Hailey
  • 34.
    Row Cache Lock- ASH select ash.session_id sid, ash.blocking_session bsid, nvl(o.object_name,to_char(CURRENT_OBJ#)) obj, o.object_type otype, CURRENT_FILE# filen, CURRENT_BLOCK# blockn, ash.SQL_ID, nvl(rc.name,to_char(ash.p3)) row_cache from v$active_session_history ash, ( select cache#, parameter name from v$rowcache ) rc, all_objects o where event='row cache lock' and rc.cache#(+)=ash.p1 and o.object_id (+)= ash.CURRENT_OBJ# and ash.session_state='WAITING' and ash.sample_time > sysdate - &minutes/(60*24) Order by sample_time
  • 35.
    Row Cache Lock- ASH select ash.session_id sid, ash.blocking_session bsid, nvl(o.object_name,to_char(CURRENT_OBJ#)) obj, o.object_type otype, CURRENT_FILE# filen, CURRENT_BLOCK# blockn, ash.SQL_ID, nvl(rc.name,to_char(ash.p3)) row_cache from v$active_session_history ash, ( select cache#, parameter name from v$rowcache ) rc, all_objects o BSID OBJ SID OTYPE FILEN BLOCKN SQL_ID ROW_CACHE where event='row cache lock' 143 131 -1 0 0 41y8w0sfqb61m dc_sequences and rc.cache#(+)=ash.p1 134 131 -1 0 0 dc_sequences and o.object_id (+)= ash.CURRENT_OBJ# 151 -1 0 0 dc_sequences and ash.session_state='WAITING' 134 151 -1 0 0 dc_sequences and ash.sample_time > sysdate - &minutes/(60*24) 131 151 -1 0 0 dc_sequences Order by sample_time 151 -1 0 0 dc_sequences
  • 36.
    Row Cache Lock  Select seq.next_val  Sequence cache set to 1  Default sequence cache is 20 SQL> @sqltext Enter value for 1: 41y8w0sfqb61m SQL_FULLTEXT SELECT TOTO_SEQ.NEXTVAL FROM DUAL Copyright 2006 Kyle Hailey
  • 37.
    Library Cache LatchSolutions  Share Cursors  Use bind variables  User cursor_sharing=force  Avoid invalidations and reloads  Size shared_pool large enough  Avoid changing dependent objects  Soft Parsing  Session_cached_cursors =20 : keep across open/close  Cursor_space_for_time=true : keep pinned across executes  hold_cursor=true : used in precompilers Copyright 2006 Kyle Hailey
  • 38.
    Summary  Shared PoolLatch  Shared pool too small or too much hard parsing  Loading Same Cursor  Library Cache Pin  Library Cache Lock  Library Cache Load Lock  Row Cache Lock  Depends on the cache Copyright 2006 Kyle Hailey

Editor's Notes

  • #15 select * from ( select PLAN_HASH_VALUE, count(PLAN_HASH_VALUE) cnt from v$sql group by PLAN_HASH_VALUE order by count(PLAN_HASH_VALUE) ) where cnt &gt; 5 and plan_hash_value &gt; 0 /
  • #16 select * from ( select PLAN_HASH_VALUE, count(PLAN_HASH_VALUE) cnt from v$sql group by PLAN_HASH_VALUE order by count(PLAN_HASH_VALUE) ) where cnt &gt; 5 and plan_hash_value &gt; 0 /
  • #19 CREATE OR REPLACE Procedure parse_same IS l_cursor integer default 0; rc integer default 0; stmt varchar2(1000); BEGIN FOR i IN 1..3000 LOOP l_cursor:=dbms_sql.open_cursor; dbms_sql.parse(l_cursor,&apos;SELECT * FROM toto&apos;,dbms_sql.native); execute immediate &apos;analyze table toto compute statistics&apos;; dbms_sql.close_cursor(l_cursor); END LOOP; END; / show errors execute parse_same;
  • #22 CREATE OR REPLACE Procedure parse_same IS l_cursor integer default 0; BEGIN FOR i IN 1..30000 LOOP l_cursor:=dbms_sql.open_cursor; dbms_sql.parse(l_cursor,&apos;SELECT * FROM dual&apos;,dbms_sql.native); dbms_sql.close_cursor(l_cursor); END LOOP; END; / show errors set serveroutput on execute dbms_output.enable(1000000); execute slatch.b; execute parse_same; execute slatch.e;
  • #24 REATE OR REPLACE Procedure parse_same IS l_cursor integer default 0; rc integer default 0; BEGIN l_cursor:=dbms_sql.open_cursor; dbms_sql.parse(l_cursor,&apos;select count(*) from dual&apos;,dbms_sql.native); FOR i IN 1..30000 LOOP rc:=dbms_sql.execute(l_cursor); rc:=DBMS_SQL.FETCH_ROWS (l_cursor); END LOOP; dbms_sql.close_cursor(l_cursor); END; / show errors set serveroutput on execute dbms_output.enable(1000000); execute slatch.b; execute parse_same; execute slatch.e; Cursor_space_for_time=false Latch Gets ----- ---- library cache lock 35 library cache 60,096 library cache pin 60,044 Cursor_space_for_time=true library cache lock 30 library cache 85 library cache pin 42
  • #25 select sid, KGLNAOBJ from x$kglob x, v$session_wait w where KGLHDADR=P1RAW and event like &apos;%library%&apos;; find the waiters and who blocks them column wevent format a20 column bevent format a20 select waiter.sid waiter, waiter.p1raw wlockp1, waiter.event wevent, blocker_event.sid blocker, blocker_event.event bevent from x$kglpn p, gv$session blocker_session, gv$session_wait waiter, gv$session_wait blocker_event where p.kglpnuse=blocker_session.saddr and p.kglpnhdl=waiter.p1raw and (waiter.event like &apos;library cache lock&apos; ) and blocker_event.sid=blocker_session.sid order by waiter.p1raw,waiter.sid / find how many sessions a blocker blocks column event format A20 select s.sid, blocked.p1raw, holder.event, count(s.sid) users_blocked from v$session s, x$kglpn p, v$session_wait blocked, v$session_wait holder where p.kglpnhdl=blocked.p1raw and s.saddr=p.kglpnuse and blocked.event like &apos;library cache lock&apos; and holder.sid=s.sid group by s.sid, blocked.p1raw,holder.event ;
  • #26 V$SYS_OPTIMIZER_ENV - Instance level V$SES_OPTIMIZER_ENV - Session level V$SQL_OPTIMIZER_ENV - Statement level Supported Optimizer parameters active_instance_count bitmap_merge_area_size cpu_count cursor_sharing hash_area_size optimizer_dynamic_sampling optimizer_features_enable optimizer_index_caching optimizer_index_cost_adj optimizer_mode optimizer_secure_view_merging parallel_ddl_mode parallel_dml_mod parallel_execution_enabled parallel_query_mode parallel_threads_per_cpu pga_aggregate_target query_rewrite_enabled query_rewrite_integrity skip_unusable_indexes sort_area_retained_size sort_area_size star_transformation_enabled statistics_level workarea_size_policy Unsupported parameters /* thanks to Julian Dyke */ SELECT pname_qkscesyrow FROM x$qkscesys WHERE SUBSTR (pname_qkscesyrow,1,1) = &apos;_&apos; ORDER BY 1;
  • #29 aSQL&gt; desc v$MUTEX_SLEEP_HISTORY Name Null? Type ----------------------------------------- -------- ------------------ SLEEP_TIMESTAMP TIMESTAMP(6) MUTEX_TYPE VARCHAR2(32) GETS NUMBER SLEEPS NUMBER REQUESTING_SESSION NUMBER BLOCKING_SESSION NUMBER LOCATION VARCHAR2(40) MUTEX_VALUE RAW(4) P1 NUMBER P1RAW RAW(4) P2 NUMBER P3 NUMBER P4 NUMBER P5 VARCHAR2(64) QL&gt; desc V$mutex_sleep Name Null? Type ----------------------------------------- -------- --------------- MUTEX_TYPE VARCHAR2(32) LOCATION VARCHAR2(40) SLEEPS NUMBER WAIT_TIME NUMBER cursor: mutex S A session waits on this event when it is requesting a mutex in shared mode, when another session is currently holding a this mutex in exclusive mode on the same cursor object. ParameterDescription P1 Hash value of cursor P2 Mutex value (top 2 bytes contain SID holding mutex in exclusive mode, and bottom two bytes usually hold the value 0) P3 Mutex where (an internal code locator) OR&apos;d with Mutex Sleeps cursor: mutex X The session requests the mutex for a cursor object in exclusive mode, and it must wait because the resource is busy. The mutex is busy because either the mutex is being held in exclusive mode by another session or the mutex is being held shared by one or more sessions. The existing mutex holder(s) must release the mutex before the mutex can be granted exclusively. ParameterDescription P1 Hash value of cursor P2 Mutex value (top 2 bytes contain SID holding mutex in exclusive mode, and bottom two bytes usually hold the value 0) P3 Mutex where (an internal code locator) OR&apos;d with Mutex Sleeps cursor: pin S A session waits on this event when it wants to update a shared mutex pin and another session is currently in the process of updating a shared mutex pin for the same cursor object.  This wait event should rarely be seen because a shared mutex pin update is very fast. Wait Time: Microseconds ParameterDescription P1 Hash value of cursor P2 Mutex value (top 2 bytes contains SID holding mutex in exclusive mode, and bottom two bytes usually hold the value 0) P3 Mutex where (an internal code locator) OR&apos;d with Mutex Sleeps cursor: pin S wait on X A session waits for this event when it is requesting a shared mutex pin and another session is holding an exclusive mutex pin on the same cursor object. Wait Time: Microseconds ParameterDescription P1 Hash value of cursor P2 Mutex value (top 2 bytes contains SID holding mutex in exclusive mode, and bottom two bytes usually hold the value 0) P3 Mutex where (an internal code locator) OR&apos;d with Mutex Sleeps cursor: pin X A session waits on this event when it is requesting an exclusive mutex pin for a cursor object and it must wait because the resource is busy. The mutex pin for a cursor object can be busy either because a session is already holding it exclusive, or there are one or more sessions which are holding shared mutex pin(s). The exclusive waiter must wait until all holders of the pin for that cursor object have released it, before it can be granted. Wait Time: Microseconds ParameterDescription P1 Hash value of cursor P2 Mutex value (top 2 bytes contains SID holding mutex in exclusive mode, and bottom two bytes usually hold the value 0) P3 Mutex where (an internal code locator) OR&apos;d with Mutex Sleeps An example of the ‘cursor: pin S wait on X’ event. • If a session is waiting on the wait event ‘cursor: pin S wait on X’, the session is most likely trying to execute a cursor (pin S), and must wait as another session (who is most likely parsing the cursor) has it pinned X (wait on X) • v$session.p1 can be used to compare with v$mutex_sleep_history.mutex_identifier • Example v$session data (64 bit platform), from an instance where the mutex holder was hung due to a bug not related to mutexes, causing requestors to back up behind the holder: select p1, p2raw, count(*) from v$session where event = ‘cursor: pin S wait on X’ and wait_time = 0 group by p1, p2; P1 P2RAW COUNT(*) ---------- ---------------- -------- 2700259466 0000139700000000 9 &lt;Mutex Id&gt; &lt; SId&gt;&lt;RefCnt&gt; As you can see, 9 sessions were waiting for Session Id 0x1397, which was holding exclusive the Mutex with the Id 2700259466. The Ref Count is zero, as we would expect. The 9 sessions are waiting to execute the cursor protected by mutex Id 2700259466. If latches had been in use instead of mutexes, it is likely this bug would have had a greater impact, as there would have been many more requestors backed up on a latch which protects many objects, than a mutex that in this case protects just one cursor. • To find the blocking session, use the top bytes of v$session.p2raw e.g. the top bytes of p2raw is the blocker 0x00001397 which when converted to decimal, is session Id 5015. Note that v$session.blocking_session is not populated for mutex related waits in 10.2
  • #32 -- Output -- WAITER WLOCKP1 WEVENT BLOCKER BEVENT -- ------- ---------------- ----------------- --------- ----------------- -- 129 00000003B76AB620 library cache pin 135,15534 PL/SQL lock timer column wevent format a20 column bevent format a20 column blocker format a10 column waiter format 99999 select waiter.sid waiter, waiter.event wevent, to_char(blocker_event.sid)||&apos;,&apos;||to_char(blocker_session.serial#) blocker, substr(decode(blocker_event.wait_time, 0, blocker_event.event, &apos;ON CPU&apos;),1,30) bevent, --blocker_event.event bevent, blocker_session.SQL_HASH_VALUE sql_hash, sql.sql_text from x$kglpn p, gv$session blocker_session, gv$session_wait waiter, gv$session_wait blocker_event, gv$sqltext sql where blocker_session.SQL_HASH_VALUE =sql.HASH_VALUE (+) and (sql.PIECE=0 or sql.piece is null) and p.kglpnuse=blocker_session.saddr and p.kglpnhdl=waiter.p1raw and (waiter.event = &apos;library cache pin&apos; or waiter.event = &apos;library cache lock&apos; or waiter.event = &apos;library cache load lock&apos;) and blocker_event.sid=blocker_session.sid and waiter.sid != blocker_event.sid order by waiter.p1raw,waiter.sid /
  • #35 set linesize 120 col block_type for a20 col objn for a25 col obj for a20 col otype for a15 col filen for 9999 col blockn for 9999999 col sid for 9999 col bsid for 9999 select ash.session_id sid, ash.blocking_session bsid, --ash.p1, --ash.p2, --ash.p3, nvl(o.object_name,to_char(CURRENT_OBJ#)) obj, o.object_type otype, CURRENT_FILE# filen, CURRENT_BLOCK# blockn, ash.SQL_ID, nvl(rc.name,to_char(ash.p3)) row_cache from v$active_session_history ash, ( select cache#, parameter name from v$rowcache ) rc, all_objects o where event=&apos;row cache lock&apos; and rc.cache#(+)=ash.p1 and o.object_id (+)= ash.CURRENT_OBJ# and ash.session_state=&apos;WAITING&apos; and ash.sample_time &gt; sysdate - &amp;minutes/(60*24) --and w.class# &gt; 18 Order by sample_time /
  • #36 set linesize 120 col block_type for a20 col objn for a25 col obj for a20 col otype for a15 col filen for 9999 col blockn for 9999999 col sid for 9999 col bsid for 9999 select ash.session_id sid, ash.blocking_session bsid, --ash.p1, --ash.p2, --ash.p3, nvl(o.object_name,to_char(CURRENT_OBJ#)) obj, o.object_type otype, CURRENT_FILE# filen, CURRENT_BLOCK# blockn, ash.SQL_ID, nvl(rc.name,to_char(ash.p3)) row_cache from v$active_session_history ash, ( select cache#, parameter name from v$rowcache ) rc, all_objects o where event=&apos;row cache lock&apos; and rc.cache#(+)=ash.p1 and o.object_id (+)= ash.CURRENT_OBJ# and ash.session_state=&apos;WAITING&apos; and ash.sample_time &gt; sysdate - &amp;minutes/(60*24) --and w.class# &gt; 18 Order by sample_time /