The document discusses the Oracle shared global area (SGA) and how it is used to share memory between Oracle processes and threads. It describes the different components of the SGA including the buffer cache, shared pool, large pool, java pool, redo buffers, and fixed area. It also discusses X$Tables, which provide a low-level view into the internal data structures of the SGA, and how they can be used to view additional details not available through V$Views.

1. 19.11.2009 DOAG 2009 Page DOAG 2009 X$Tables and SGA-Scanner Dipl.-Inform. Frank Beutelschiess

2. 19.11.2009 DOAG 2009 Page Contents Chapter 1 – Introduction Chapter 2 – SGA Chapter 3 – X$Tables Chapter 4 – V$Views Chapter 5 – SGA-Scanner

3. 19.11.2009 DOAG 2009 Page Chapter 1 Chapter 1 – Introduction Chapter 2 – SGA Chapter 3 – X$Tables Chapter 4 – V$Views Chapter 5 – SGA-Scanner

4. 19.11.2009 DOAG 2009 Page Who am I? Graduate of computer science out of a passion for the field Active freelancer for 17 years 10 years in the area of Siebel and OracleDB BzYxS.com for the past 2 years

5. 19.11.2009 DOAG 2009 Page What does BzYxS.com do? Training & Consulting Among other things, for Oracle University Siebel division Configuration reviews Component tuning Oracle Databank environment Tuning, preferred under use of tracefile analysis with QueryAdvisor

6. 19.11.2009 DOAG 2009 Page Chapter 2 Chapter 1 – Introduction Chapter 2 – SGA Chapter 3 – X$Tables Chapter 4 – V$Views Chapter 5 – SGA-Scanner

7. 19.11.2009 DOAG 2009 Page SGA – What is it? abbreviation for „Shared Global Area“ shared memory between all processes (Unix) or threads (Windows) allocated during the start of an instance in state NoMount allocated partially (Unix) or completely (Windows) according to release and OS Instance = Processes (Threads) + SGA

8. SGA 19.11.2009 DOAG 2009 Page SGA - Architecture Redo Buffer Large Pool Dedicated Server Process Shared Server Process CKPT Data RedoLog Archive Control ARCn LGWR SMON PMON BufferCache Streams Pool Java Pool Shared Pool PGA PGA Dispatcher User User User User Dedicated Server Shared Server DBWn Instance Database Fixed Area

9. 19.11.2009 DOAG 2009 Page SGA - Areas Buffer Pool Default/Keep/Recycle, n-Size (n = 2,4,8,16,32KB) Shared Pool Library Cache Shared SQL-areas & Private SQL-areas (only on shared servers) PL/SQL procedures Library Cache Handles Dictionary Cache Control structures (Latches & Enques) Redo Buffer Large Pool Java & Streams Pool

10. 19.11.2009 DOAG 2009 Page SGA – Fixed Area Part of the SGA contains general information about the state of the database and the instance, which the background processes need to access. No user data is stored here. http://download.oracle.com/docs/cd/B19306_01/server.102/b14220/memory.htm Oracle also maintains a "fixed" area in the SGA that contains a number of atomic variables, pointers, and other miscellaneous structures that reference areas of the SGA. http://www.dba-oracle.com/oracle_tips_sga_regions.htm

11. 19.11.2009 DOAG 2009 Page SGA - Dedicated/Shared Server Dedicated/Shared UGA (User Global Area) PGA/SGA Request/Response Queues PGA/SGA Runtime area Select PGA Runtime area DMLs/DDLs PGA

12. 19.11.2009 DOAG 2009 Page SGA – Fixed Structures Fixed during the start of an instance (V$SGA named = ‘Fixed Size‘)) Session Array Process Array Memory requirements are calculated at the start of the instance and immediately allocated completely. Changes are only available after a restart.

13. 19.11.2009 DOAG 2009 Page SGA – Dynamic Structures Dynamic, created at runtime as a heap. (V$SGA_DYNAMIC_COMPONENTS) Buffer Cache Shared Pool (Library Cache, Dictionary Cache) Large Pool

14. 19.11.2009 DOAG 2009 Page SGA - Parameters The type of parameter (fixed or dynamic), for example, is shown in the Enterprise Manager.

15. 19.11.2009 DOAG 2009 Page SGA – Allocation of Memory SGA_MAX_SIZE defines the maximum size of the SGA. The maximum possible size depends upon the release and OS. Allocation occurs immediately or in part in the form of individual blocks, which are called granules. The size of a granule depends upon the size of the SGA, release and OS, with 4 or 8, or 16MB, accordingly.

16. 19.11.2009 DOAG 2009 Page SGA – Granule Order Fixed Area Unused Buffer Cache Large Pool Java Pool Shared Pool Shared Pool Redo Buffer Increasing memory address Shared Pool Shared Pool Java Pool Streams Pool Shared Pool Shared Pool Large Pool Buffer Cache Buffer Cache Buffer Cache Buffer Cache Buffer Cache Buffer Cache Buffer Cache Buffer Cache Unused

17. 19.11.2009 DOAG 2009 Page Chapter 3 Chapter 1 – Introduction Chapter 2 – SGA Chapter 3 – X$Tables Chapter 4 – V$Views Chapter 5 – SGA-Scanner

18. 19.11.2009 DOAG 2009 Page X$Tables - Information (1/2) All examples are based upon Oracle 9.2 .0.5.0 All examples are based upon a 32 Bit Windows XP installation. Why this historical configuration? Basic concepts of the structure of a SGA, such as, for example, granules were first introduced in 9i.

19. 19.11.2009 DOAG 2009 Page X$Tables - Information (2/2) X$Tables are not officially documented from Oracle. X$Tables can change at any time, from release to release, both in their existence and their structure. Everything that is presented here in the following pages is based only upon well-founded presumptions. Therefore, please use it with sense and care!

20. 19.11.2009 DOAG 2009 Page X$KSMGE (1/3) Kernel/Service/Memory/Granule GranType 4 = ‘JavaPool‘ GranType 5 = ‘BufferPool‘

21. 19.11.2009 DOAG 2009 Page X$KSMGE (2/3) select 'X$KSMGE', decode(grantype,0,'Free',2,'SharedPool',3,'LargePool',4,'JavaPool', 5,'BufferPool',grantype) as Component,count(*),count(*)*4 from X$KSMGE group by GRANTYPE union all select 'X$KSMSD',KSMSDNAM,0,KSMSDVAL/(1024*1024) from X$KSMSD order by 1,2 Information: X$KSMSD (Kernel/Service/Memory/SizeDefinition) ~ V$SGA

22. 19.11.2009 DOAG 2009 Page X$KSMGE (3/3) Database Buffers 24.00 MB Variable Size 104.00 MB Shared Pool 64.00 MB Java Pool 32.00 MB Large Pool 8.00 MB Fixed Size 0.43 MB Redo Buffers 0.64 MB ------------------------------------------------ Total 129.07 MB

23. 19.11.2009 DOAG 2009 Page X$Tables – What are they? The window to the SGA. Corresponds to the mentioned Fixed Area. X$Tables correspond to data structures implemented in C. These structures are realized in the form of simple, atomic variables, pointers, arrays, linked lists,…

24. 19.11.2009 DOAG 2009 Page X$KSMMEM (1/4) Each byte of the SGA is mapped external

25. 19.11.2009 DOAG 2009 Page X$KSMMEM (2/4) Dump of the first bytes in a 32 Bit SGA Each KSMMMVAL contains 4 Bytes Please note: with Unix, it can exist without a limitation in the Where-part.!

26. 19.11.2009 DOAG 2009 Page X$KSMMEM (3/4) Dump of the first bytes in a 32 Bit SGA Each KSMMMVAL contains 4 Bytes

27. 19.11.2009 DOAG 2009 Page X$KSMMEM (4/4) Dump of the first bytes in a 32 Bit SGA Each KSMMMVAL contains 4 Bytes

28. 19.11.2009 DOAG 2009 Page X$-Tables - Why? Just for fun. Access to data which is otherwise not accessible, only a part of the information is accessible using the V$Views.

29. 19.11.2009 DOAG 2009 Page X$KSPPI & X$KSPPSV Access to all Parameters (incl. hidden) X$KSPPI Kernel/Service/Parameter/ParameterInfo X$KSPPSV Kernel/Service/Parameter/SystemValues select * from X$KSPPI P,X$KSPPSV V where P.INDX = V.INDX and P.INST_ID = V.INST_ID

30. 19.11.2009 DOAG 2009 Page X$KSPPI & X$KSPPSV select x.inst_id,y.ksppinm, decode(y.ksppity,1,'Boolean',2,'String',3,'Integer', 4,'ParameterFile',5,'Reserved',6,'BigInteger',y.ksppity), x.ksppstvl,x.ksppstdf, decode(bitand(y.ksppiflg/256,1),1,'TRUE','FALSE'), decode(bitand(y.ksppiflg/65536,3), 1,'IMMEDIATE',2,'DEFERRED',3,'IMMEDIATE','FALSE'), decode(bitand(x.ksppstvf,7),1,'MODIFIED',4,'SYSTEM_MOD','FALSE'), decode(bitand(x.ksppstvf,2),2,'TRUE','FALSE'), y.ksppdesc from X$KSPPI y, X$KSPPSV x where x.indx = y.indx and x.inst_id = y.inst_id order by 1,2

31. 19.11.2009 DOAG 2009 Page X$Tables - Access Rights to the X$Tables cannot simply be granted. Write-access would compromise the stability of the instance!

32. 19.11.2009 DOAG 2009 Page X$Tables - Terms Metalink Doc ID: Note:22241.1 Subject: List of X$ Tables and how the names are derived

33. 19.11.2009 DOAG 2009 Page X$Tables – Start of Analysis X$KSMMEM - memory access X$KQFTA - all X$Tables X$KQFDT - derivative tables X$KQFCO - column definitions

34. 19.11.2009 DOAG 2009 Page X$Tables – Offset Calculation select c.kqfconam field_name, c.kqfcooff offset, c.kqfcosiz sz from x$kqfco c, x$kqfta t where t.indx = c.kqfcotab and t.kqftanam='X$KSUSECST' order by offset X$KSUSECST = Kernel/Service/User/SessionStatusForEvents For example, use in V$SESSION

35. 19.11.2009 DOAG 2009 Page X$Tables - Structure Each X$Table consists of physically available columns virtual columns Virtual columns always available are ADDR = physical data address INDX = Index inside the table Often joins use the INDX column The order of the columns is not identical with the order inside the data structures!

36. 19.11.2009 DOAG 2009 Page X$Tables – Contention (1/2) X$KGLOB (KernelGenericLibraryObject) FullTableScans indicate that all latches have been traversed in the LibraryCache.

37. 19.11.2009 DOAG 2009 Page X$Tables – Contention (2/2) X$Tables are protected from latches as low level data structures. FullTableScans will cause latch contention! The X$KGLOB is especially compromised (see Metalink). Access to indices should be forced, no statistics should be generated according to the volatility of some X$Tables!

38. 19.11.2009 DOAG 2009 Page Chapter 4 Chapter 1 – Introduction Chapter 2 – SGA Chapter 3 – X$Tables Chapter 4 – V$Views Chapter 5 – SGA-Scanner

39. 19.11.2009 DOAG 2009 Page V$ Views In general, in accordance with their function, based on X$Tables. Are documented and also largely stable across all versions. V$Views are mapped from GV$Views limited by the instance. Access limitations to X$Tables are eliminated through a two-step mapping of the GV_$Views onto the underlying GV$Views.

40. 19.11.2009 DOAG 2009 Page V$Views - Definition X$KQFVI - name of V$ & GV$ views X$KQFVT - corresponding definitions V$FIXED_VIEW_DEFINITION Script for mapping in CATALOG.SQL create or replace view gv_$bh as select * from gv$bh; create or replace public synonym gv$bh for gv_$bh; grant select on gv_$bh to public;

41. 19.11.2009 DOAG 2009 Page Chapter 5 Chapter 1 – Introduction Chapter 2 – SGA Chapter 3 – X$Tables Chapter 4 – V$Views Chapter 5 – SGA-Scanner

42. 19.11.2009 DOAG 2009 Page SGA-Scanner – What? Direct readout of the SGA on memory level by bypassing the database instance. Self-contained program on the respective database server. SGA addresses etc. are determined using queries at the frontend of the database.

43. 19.11.2009 DOAG 2009 Page SGA-Scanner - Configuration

44. 19.11.2009 DOAG 2009 Page SGA-Scanner - Challenge 32 Bit / 64 Bit according to 4 / 8 Byte arithmatic / pointers Address alignment (e.g., a word always begins at an even address) Big (PowerPC) & Little Endian (X86) Register: =0A0B0C0D=> BigEndian: =0A0B0C0D=> LittleEndian: =0D0C0B0A=> Granules & nonlinear memory area

45. 19.11.2009 DOAG 2009 Page SGA-Scanner - Windows 1 process, many tasks 1 database

46. 19.11.2009 DOAG 2009 Page SGA-Scanner - Implementation <Psapi>.EnumProcesses(*pProcessIds,…) <Kernel32>.OpenProcess(0x410,…,dwProcessId) <Psapi>.GetModuleBaseName(hProcess,…,lpBaseName,…) <Kernel32>.CloseHandle(hObject) <Kernel32>.ReadProcessMemory(hProcess, lpBaseAddress,lpBuffer,…) Oracle.exe? Liste fertig? <Kernel32>.CloseHandle(hObject) Yes Yes No No Start End

47. 19.11.2009 DOAG 2009 Page SGA-Scanner - Why? Access to a no longer accessible/hard to reach database. No burden on the instance through contentions by shared memory (pure CPU load). High sampling speed (from 1000x/second as opposed to a maximum of 50x of V$Views).

48. 19.11.2009 DOAG 2009 Page SGA-Scanner – V$SESSION (1/9) - List of all sessions (same as V$SESSION)

49. 19.11.2009 DOAG 2009 Page SGA-Scanner – V$SESSION (2/9) - Session 10 (SID 10) is newly added.

50. 19.11.2009 DOAG 2009 Page SGA-Scanner – V$SESSION (3/9) - Session 11 (SID 12) is newly added.

51. 19.11.2009 DOAG 2009 Page Preparation of both sessions, autocommit is deactivated! SGA-Scanner – V$SESSION (4/9)

52. 19.11.2009 DOAG 2009 Page SGA-Scanner – V$SESSION (5/9) - Access to the table locked by the other session.

53. 19.11.2009 DOAG 2009 Page SGA-Scanner – V$SESSION (6/9) - Session 11 (SID 12) waits due to an enqueue.

54. 19.11.2009 DOAG 2009 Page SGA-Scanner – V$SESSION (7/9) - Lock is removed through a manual commit.

55. 19.11.2009 DOAG 2009 Page SGA-Scanner – V$SESSION (8/9) - Enqueue is broken.

56. 19.11.2009 DOAG 2009 Page SGA-Scanner – V$SESSION (9/9)

57. 19.11.2009 DOAG 2009 Page Thank you for your attention!