Your SlideShare is downloading. ×
0
Monitoring and Tuning Oracle for z/OS and Oracle for z/Linux
Thomas Niewel Oracle Deutschland GmbH [email_address]
Agenda <ul><li>Tuning Why ? </li></ul><ul><li>Reasons for bad Response Time </li></ul><ul><li>Statspack </li></ul><ul><li>...
Why do we need to tune ? <ul><li>Users report „bad“ response times because of </li></ul><ul><ul><li>CPU Time + Wait Time <...
What can be the reasons for “bad” Response Time <ul><li>High CPU Usage </li></ul><ul><li>High I/O Usage </li></ul><ul><li>...
Diagnose from the Oracle point of view  Statspack A short overview
Statspack – a short overview <ul><li>spcreate.sql - installs Statspack (run only once) </li></ul><ul><li>statspack.snap - ...
Capturing data <ul><li>Prerequisite:  timed_statistics=true </li></ul><ul><li>Use stored procedure statspack.snap </li></u...
Capturing data <ul><li>Get a baseline for future comparisons </li></ul><ul><li>Capture snapshots </li></ul><ul><ul><li>acr...
Reporting with Statspack <ul><li>All data is held in an Oracle database </li></ul><ul><li>Report between two or more snaps...
Reporting with Statspack <ul><li>SQL>  @spreport </li></ul><ul><li>DB Id DB Name  Instance# Instance </li></ul><ul><li>---...
Analyzing a Statspack report <ul><li>Top down analysis </li></ul><ul><li>Summary page  </li></ul><ul><ul><li>Enviroment  <...
Environment section STATSPACK report for DB Name  DB Id  Instance  Inst Num Release  Cluster Host ------------ -----------...
Load profile <ul><li>Contains a number of common ratios  </li></ul><ul><li>Allows characterisation of the application </li...
Load profile <ul><li>Useful if you have a comparable baseline </li></ul><ul><li>What has changed? </li></ul><ul><ul><li>tx...
Load profile <ul><li>Load Profile </li></ul><ul><li>~~~~~~~~~~~~  Per Second  Per Transaction </li></ul><ul><li>----------...
Instance Efficiency <ul><li>Gives an overview of how the instance is performing </li></ul><ul><li>Can also be used with a ...
Instance Efficiency Instance Efficiency Percentages (Target 100%) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Buffer Now...
Top 5 Timed Events <ul><li>CPU time – real work </li></ul><ul><li>Shows where Oracle sessions are waiting  </li></ul><ul><...
Top 5 Timed Events <ul><li>Sample drilldowns  </li></ul><ul><ul><li>CPU Time  „on CPU“ </li></ul></ul><ul><ul><li>enqueue ...
Top SQL   <ul><li>Helps to find problem statements </li></ul><ul><ul><li>SQL ordered by Gets  </li></ul></ul><ul><ul><li>S...
Top SQL   CPU  Elapsd Buffer Gets  Executions  Gets per Exec  %Total Time (s)  Time (s) Hash Value --------------- -------...
I/O Statistics   <ul><li>Help to find I/O Problems </li></ul><ul><ul><li>Tablespace IO Stats  </li></ul></ul><ul><ul><li>F...
I/O Statistics   Tablespace ------------------------------ Av  Av  Av  Av  Buffer Av Buf Reads Reads/s Rd(ms) Blks/Rd  Wri...
Diagnosing high CPU usage <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization </li></ul><ul><li>Memory Usage </l...
Diagnosing high CPU usage -Operating System-  <ul><li>Linux/390 </li></ul><ul><ul><li>sar -u 3 3333 </li></ul></ul><ul><ul...
Diagnosing high CPU usage <ul><li>What can be the reason for „high CPU“ Usage ? </li></ul><ul><ul><li>Shared_Pool / SQL-Ca...
Diagnosing high CPU usage CPU  Elapsd  Buffer Gets  Executions  Gets per Exec  %Total Time (s)  Time (s) Hash Value ------...
Diagnosing high CPU usage <ul><ul><li>spool cpu_users.lst select buffer_gets,disk_reads,executions, ratio_to_report(buffer...
Diagnosing high CPU usage <ul><ul><li>BUFFER_GETS DISK_READS EXECUTIONS BUFFER_RATIO DISK_RATIO </li></ul></ul><ul><ul><li...
SQL Tuning <ul><li>Check Object Statsitics </li></ul><ul><ul><li>Use DBMS_STATS  </li></ul></ul><ul><li>Analyze Execution ...
Diagnose <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization  </li></ul><ul><li>Memory Usage </li></ul><ul><li>N...
High I/O utilization <ul><li>Linux/390 </li></ul><ul><ul><li>sar -d 3 33333 </li></ul></ul><ul><ul><li>iostat -x 3 </li></...
High I/O utilization <ul><li>Disk I/O </li></ul><ul><ul><li>Disk access is slower than memory access (Factor 5000 to 10000...
High I/O utilization <ul><li>Reasons for High I/O utilization </li></ul><ul><ul><li>Database Cache too small (DB_CACHE_SIZ...
High I/O utilization <ul><li>Increase Cache Size </li></ul><ul><ul><li>Reduces physical I/O Operations </li></ul></ul><ul>...
High I/O utilization <ul><li>An Oracle server instance has a single SGA regardless    of the number of address spaces  or ...
High I/O utilization <ul><li>Linux/390 </li></ul><ul><ul><li>The default maximum SGA size on Linux/390 is 750 MB without c...
% Total Event  Waits  Time (s)  Wt Time -------------------------------------------- ------------ ------------ ------- db ...
Tablespace Av  Av  Av  Av  Buffer Av Buf Reads Reads/s Rd(ms) Blks/Rd  Writes Writes/s  Waits Wt(ms) -------------- ------...
D I R E C T  A C C E S S  D E V I C E  A C T I V I T Y DEVICE  AVG  AVG  AVG  AVG  AVG  AVG  AVG  AVG  %  %  %  AVG  %  % ...
<ul><li>RMF Report – Explanations </li></ul><ul><ul><li>IOSQ TIME = UCB Queueing time  </li></ul></ul><ul><ul><li>Avg Pend...
SQL Tuning <ul><li>Check Object Statsitics </li></ul><ul><ul><li>Use DBMS_STATS  </li></ul></ul><ul><li>Analyze Execution ...
Diagnose <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization  </li></ul><ul><li>Memory Usage </li></ul><ul><li>N...
Memory Problems <ul><li>How to determine Paging/Swapping </li></ul><ul><ul><li>Linux/390  </li></ul></ul><ul><ul><ul><li>V...
Diagnosing high CPU usage -Operating System-  <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization </li></ul><ul>...
Diagnosing Network problems <ul><li>Latency </li></ul><ul><ul><li>LAN: < 1ms </li></ul></ul><ul><ul><li>WAN: < 10ms - 500m...
Diagnosing high CPU usage -Operating System-  <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization </li></ul><ul>...
Idle System  <ul><li>One CPU is  100% used – All other CPU´s are idle </li></ul><ul><ul><li>Reason  </li></ul></ul><ul><ul...
Idle System  <ul><li>Latch Contentions  </li></ul><ul><ul><li>Use Statspack to diagnose </li></ul></ul><ul><li>Enqueue Wai...
Idle System  Top 5 Timed Events ~~~~~~~~~~~~~~~~~~  % Total Event  Waits  Time (s) Ela Time ------------------------------...
Idle System  Enqueue activity for DB: RECONPRD  Instance: RECONPRD  Snaps: 2 -31 -> Enqueue stats gathered prior to 9i sho...
SQL-Tuning <ul><li>Prerequisites </li></ul><ul><ul><li>Use Cost based optimizer </li></ul></ul><ul><ul><li>DBMS_STATS  (im...
SQL-Tuning <ul><li>SQL> explain plan for select a.* from scott.emp a, scott.dept b where a.deptno=b.deptno; </li></ul><ul>...
SQL-Tuning <ul><li>Optimizer features which help to improve execution plans </li></ul><ul><ul><li>Function based indexes (...
SQL-Tuning <ul><li>SQLTRACE </li></ul><ul><ul><li>Prerequisite: timed_statistics=true </li></ul></ul><ul><ul><li>Activate ...
z/OS WLM <ul><li>Everything works fine without peaks (e.g.CPU 30%) </li></ul><ul><li>Common Problems we had with WLM(durin...
Oracle for Linux /390 <ul><li>We had tuning work </li></ul><ul><ul><li>Linux on an LPAR </li></ul></ul><ul><ul><li>Linux u...
Oracle for z/OS <ul><li>The reasons for performance bottlenecks were </li></ul><ul><ul><li>WLM configuration </li></ul></u...
?
Upcoming SlideShare
Loading in...5
×

Thomas+Niewel+ +Oracletuning

1,224

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,224
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
102
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Thomas+Niewel+ +Oracletuning"

  1. 1. Monitoring and Tuning Oracle for z/OS and Oracle for z/Linux
  2. 2. Thomas Niewel Oracle Deutschland GmbH [email_address]
  3. 3. Agenda <ul><li>Tuning Why ? </li></ul><ul><li>Reasons for bad Response Time </li></ul><ul><li>Statspack </li></ul><ul><li>Diagnosing reasons for bad response Times </li></ul><ul><li>SQL Tuning </li></ul><ul><ul><ul><li>TKPROF </li></ul></ul></ul><ul><ul><ul><li>Explain Plan </li></ul></ul></ul><ul><li>WLM </li></ul>
  4. 4. Why do we need to tune ? <ul><li>Users report „bad“ response times because of </li></ul><ul><ul><li>CPU Time + Wait Time </li></ul></ul><ul><ul><li>Poor performing queries </li></ul></ul><ul><ul><ul><li>SQL-Tuning </li></ul></ul></ul><ul><ul><li>„ bad“ Database parameters </li></ul></ul><ul><ul><li>Bottlenecks in „System“ (Operating System, WLM, IO/Subsystem etc.) </li></ul></ul>
  5. 5. What can be the reasons for “bad” Response Time <ul><li>High CPU Usage </li></ul><ul><li>High I/O Usage </li></ul><ul><li>Memory Usage </li></ul><ul><li>Network problems </li></ul><ul><li>„ idle“ System </li></ul><ul><li>Operating System (WLM, VM) </li></ul>
  6. 6. Diagnose from the Oracle point of view Statspack A short overview
  7. 7. Statspack – a short overview <ul><li>spcreate.sql - installs Statspack (run only once) </li></ul><ul><li>statspack.snap - data capture (procedure) </li></ul><ul><li>spreport.sql - reporting </li></ul><ul><li>spdoc.txt - user documentation </li></ul><ul><li>sppurge.sql - delete Statspack data </li></ul><ul><li>spdrop.sql - drop Statspack </li></ul>
  8. 8. Capturing data <ul><li>Prerequisite: timed_statistics=true </li></ul><ul><li>Use stored procedure statspack.snap </li></ul><ul><li>SQL> execute statspack.snap; </li></ul>
  9. 9. Capturing data <ul><li>Get a baseline for future comparisons </li></ul><ul><li>Capture snapshots </li></ul><ul><ul><li>across peak load </li></ul></ul><ul><ul><li>across batch window </li></ul></ul><ul><ul><li>The time between snapshots should be <= 30 minutes </li></ul></ul><ul><li>Capture can be automated </li></ul><ul><ul><li>Use OS utility e.g. cron </li></ul></ul><ul><ul><li>Use dbms_job </li></ul></ul><ul><ul><ul><li>spauto.sql shipped as example </li></ul></ul></ul>
  10. 10. Reporting with Statspack <ul><li>All data is held in an Oracle database </li></ul><ul><li>Report between two or more snapshots </li></ul><ul><ul><li>cannot report across instance startup </li></ul></ul><ul><li>Spreport.sql creates a report </li></ul>
  11. 11. Reporting with Statspack <ul><li>SQL> @spreport </li></ul><ul><li>DB Id DB Name Instance# Instance </li></ul><ul><li>----------- ---------- ---------- ---------- </li></ul><ul><li>1361567071 DB21 1 MAIL </li></ul><ul><li>Completed Snapshots </li></ul><ul><li>Instance DB Name SnapId Snap Started Snap Level </li></ul><ul><li>---------- ---------- ------ ---------------------- ---------- </li></ul><ul><li>DB21 DB21 1 17 Aug 2003 10:00:16 5 </li></ul><ul><li>2 17 Aug 2003 10:30:28 5 </li></ul><ul><li>Enter beginning Snap Id: 1 </li></ul><ul><li>Enter ending Snap Id: 2 </li></ul><ul><li>Enter name of output file [sp_1_2] : <enter name or return> </li></ul>
  12. 12. Analyzing a Statspack report <ul><li>Top down analysis </li></ul><ul><li>Summary page </li></ul><ul><ul><li>Enviroment </li></ul></ul><ul><ul><li>Load profile </li></ul></ul><ul><ul><li>Instance efficiency </li></ul></ul><ul><ul><li>Shared pool usage </li></ul></ul><ul><ul><li>Top 5 Timed Events </li></ul></ul><ul><li>Top SQL </li></ul>
  13. 13. Environment section STATSPACK report for DB Name DB Id Instance Inst Num Release Cluster Host ------------ ----------- ------------ -------- ----------- ------- ------------ RECONPRD 1403107896 RECONPRD 1 9.2.0.2.0 NO lin390t1 Snap Id Snap Time Sessions Curs/Sess Comment ------- ------------------ -------- --------- ------------------- Begin Snap: 2 03-Mar-03 11:28:01 10 5.1 End Snap: 31 04-Mar-03 11:58:04 17 5.5 Elapsed: 30.05 (mins) Cache Sizes (end) ~~~~~~~~~~~~~~~~~ Buffer Cache: 256M Std Block Size: 16K Shared Pool Size: 48M Log Buffer: 128K
  14. 14. Load profile <ul><li>Contains a number of common ratios </li></ul><ul><li>Allows characterisation of the application </li></ul><ul><li>Can point to problems </li></ul><ul><ul><li>high hard parse rate </li></ul></ul><ul><ul><li>high IO rate </li></ul></ul><ul><ul><li>high login rate </li></ul></ul>
  15. 15. Load profile <ul><li>Useful if you have a comparable baseline </li></ul><ul><li>What has changed? </li></ul><ul><ul><li>txn/sec change implies changed workload </li></ul></ul><ul><ul><li>redo size/txn implies changed transaction mix </li></ul></ul><ul><ul><li>physical reads/txn implies changed SQL or plan </li></ul></ul>
  16. 16. Load profile <ul><li>Load Profile </li></ul><ul><li>~~~~~~~~~~~~ Per Second Per Transaction </li></ul><ul><li>--------------- --------------- </li></ul><ul><li>Redo size: 19,057.68 20,937.67 </li></ul><ul><li>Logical reads: 2,408.15 2,645.70 </li></ul><ul><li>Block changes: 98.64 108.37 </li></ul><ul><li>Physical reads: 990.47 1,088.18 </li></ul><ul><li>Physical writes: 6.92 7.61 </li></ul><ul><li>User calls: 76.40 83.93 </li></ul><ul><li>Parses: 7.08 7.78 </li></ul><ul><li>Hard parses: 0.02 0.02 </li></ul><ul><li>Sorts: 29.22 32.10 </li></ul><ul><li>Logons: 24.73 27.17 </li></ul><ul><li>Executes: 63.79 70.08 </li></ul><ul><li>Transactions: 0.91 </li></ul><ul><li>% Blocks changed per Read: 4.10 Recursive Call %: 72.76 </li></ul><ul><li>Rollback per transaction %: 36.52 Rows per Sort: 153.46 </li></ul>
  17. 17. Instance Efficiency <ul><li>Gives an overview of how the instance is performing </li></ul><ul><li>Can also be used with a comparable baseline </li></ul><ul><li>Shared pool Statistics allow quick identification of cursor sharing problems </li></ul>
  18. 18. Instance Efficiency Instance Efficiency Percentages (Target 100%) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Buffer Nowait %: 99.99 Redo NoWait %: 99.97 Buffer Hit %: 59.00 In-memory Sort %: 99.99 Library Hit %: 99.94 Soft Parse %: 99.69 Execute to Parse %: 88.89 Latch Hit %: 99.98 Parse CPU to Parse Elapsd %: 56.55 % Non-Parse CPU: 99.93 Shared Pool Statistics Begin End ------ ------ Memory Usage %: 38.86 66.81 % SQL with executions>1: 43.41 87.22 % Memory for SQL w/exec>1: 39.28 80.21
  19. 19. Top 5 Timed Events <ul><li>CPU time – real work </li></ul><ul><li>Shows where Oracle sessions are waiting </li></ul><ul><li>Compare Wait Time to elapsed time </li></ul><ul><li>% Total Wait Time shows potential benefits </li></ul><ul><li>Use as basis for directed drilldown </li></ul>% Total Event Waits Time (s) Ela Time ------------------------------- ------------ ----------- -------- CPU time 78,588 50.24 enqueue 1,560,523 59,961 38.33 db file sequential read 1,635,253 6,324 4.04 db file scattered read 14,620,725 5,907 3.78 control file parallel write 32,816 1,396 .89
  20. 20. Top 5 Timed Events <ul><li>Sample drilldowns </li></ul><ul><ul><li>CPU Time „on CPU“ </li></ul></ul><ul><ul><li>enqueue e.g TX Enqueue </li></ul></ul><ul><ul><li>db file sequential read </li></ul></ul><ul><ul><li>Index Access </li></ul></ul><ul><ul><li>db file scattered read Scan Operations control file parallel write </li></ul></ul>
  21. 21. Top SQL <ul><li>Helps to find problem statements </li></ul><ul><ul><li>SQL ordered by Gets </li></ul></ul><ul><ul><li>SQL ordered by Reads </li></ul></ul><ul><ul><li>SQL ordered by Executions </li></ul></ul><ul><ul><li>SQL ordered by Parse Calls </li></ul></ul>
  22. 22. Top SQL CPU Elapsd Buffer Gets Executions Gets per Exec %Total Time (s) Time (s) Hash Value --------------- ------------ -------------- ------ -------- --------- ---------- 79,562,398 8,114 9,805.6 34.6 27182.71 28127.71 1525844323 Module: SQL*Plus SELECT MAX(STMT_BKG_DATE_CLOSE) FROM GAH_T_STATEMENTS WHERE S TMT_ACCT_ID = :b1 AND ((:b2 = :b3 AND STMT_CARRIER != :b4 AND STMT_MSG_TYPE != :b5 AND (:b6 IS NULL OR :b6 = STMT_CARRIER ) AND ((:b8 IS NULL AND STMT_MSG_TYPE != :b9 ) OR (:b8 IS NOT NU LL AND :b8 = STMT_MSG_TYPE ))) OR (:b2 = :b13 AND STMT_CARRIER
  23. 23. I/O Statistics <ul><li>Help to find I/O Problems </li></ul><ul><ul><li>Tablespace IO Stats </li></ul></ul><ul><ul><li>File IO Stats </li></ul></ul>
  24. 24. I/O Statistics Tablespace ------------------------------ Av Av Av Av Buffer Av Buf Reads Reads/s Rd(ms) Blks/Rd Writes Writes/s Waits Wt(ms) -------------- ------- ------ ------- ------------ -------- ---------- ------ GAH_TS00_DT_MEDIUM 15,242,896 160 0.4 6.1 41,066 0 22,468 18.4 GAH_TS00_IX_ITEM 210,346 2 11.2 1.0 130,299 1 9 15.6 GAH_TS00_IX_MEDIUM 207,433 2 6.9 1.0 86,699 1 39 43.8 RECONPRD_TS00_TEMP 185,865 2 1.7 1.6 101,560 1 0 0.0 GAH_TS00_IX_ITEM_REF 155,027 2 8.4 1.0 34,867 0 1 0.0
  25. 25. Diagnosing high CPU usage <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization </li></ul><ul><li>Memory Usage </li></ul><ul><li>Network problems </li></ul><ul><li>„ idle“ System </li></ul><ul><li>Operating System (WLM, VM) </li></ul>
  26. 26. Diagnosing high CPU usage -Operating System- <ul><li>Linux/390 </li></ul><ul><ul><li>sar -u 3 3333 </li></ul></ul><ul><ul><li>iostat -x 3 </li></ul></ul><ul><ul><li>vmstat 3 </li></ul></ul><ul><ul><li>top </li></ul></ul><ul><ul><li>Etc. </li></ul></ul><ul><li>Z/OS </li></ul><ul><ul><li>SDSF </li></ul></ul><ul><ul><li>RMF </li></ul></ul><ul><ul><li>Omegamon </li></ul></ul><ul><ul><li>etc. </li></ul></ul>
  27. 27. Diagnosing high CPU usage <ul><li>What can be the reason for „high CPU“ Usage ? </li></ul><ul><ul><li>Shared_Pool / SQL-Cache </li></ul></ul><ul><ul><li>db_file_multiblock_read_count </li></ul></ul><ul><ul><li>Buffer_Cache/ Buffer_Pool </li></ul></ul><ul><ul><li>How can Statements with a great # of buffergets be seperated ? </li></ul></ul><ul><ul><ul><ul><li>Statspack </li></ul></ul></ul></ul><ul><ul><ul><ul><li>SQL Script </li></ul></ul></ul></ul>
  28. 28. Diagnosing high CPU usage CPU Elapsd Buffer Gets Executions Gets per Exec %Total Time (s) Time (s) Hash Value --------------- ------------ -------------- ------ -------- --------- ---------- 4,494,662 155 28,997.8 2.0 1049.63 2414.11 3961361411 SELECT * FROM GAH_T_STATEMENTS WHERE STMT_ACCT_ID = :b1 AND ((:b2 = :b3 AND STMT_CARRIER != :b4 AND STMT_MSG_TYPE != :b5 AND (:b6 IS NULL OR :b6 = STMT_CARRIER ) AND ((:b8 IS NULL AND STMT_MSG_TYPE != :b9 ) OR (:b8 IS NOT NULL AND :b8 = STMT_MSG_ TYPE ))) OR (:b2 = :b13 AND STMT_CARRIER = :b14 AND STMT_MSG_T Module: SQL*Plus
  29. 29. Diagnosing high CPU usage <ul><ul><li>spool cpu_users.lst select buffer_gets,disk_reads,executions, ratio_to_report(buffer_gets) over () * 100 buffer_ratio, ratio_to_report(disk_reads) over () * 100 disk_ratio, sql_text from v$sqlarea order by buffer_ratio desc; spool off </li></ul></ul>
  30. 30. Diagnosing high CPU usage <ul><ul><li>BUFFER_GETS DISK_READS EXECUTIONS BUFFER_RATIO DISK_RATIO </li></ul></ul><ul><ul><li>----------- ---------- ---------- ------------ ---------- </li></ul></ul><ul><ul><li>SQL_TEXT </li></ul></ul><ul><ul><li>---------------------------------------------------------------------------------------- </li></ul></ul><ul><ul><li>19564429 154 46908 65.9945773 5.40350877 </li></ul></ul><ul><ul><li>select t.schema, t.name, t.flags, q.name from system.aq$_queue_tables t, ys.aq$_queue_table_affinities aft, system.aq$_queues q where aft.table_objno = t.objno and aft.owner_instance = :1 and q.table_objno = t.objno and q.usage = 0 and bitand(t.flags, 4+16+32+64+128+256) = 0 for update of t.name, aft.table_objno skip locked </li></ul></ul>
  31. 31. SQL Tuning <ul><li>Check Object Statsitics </li></ul><ul><ul><li>Use DBMS_STATS </li></ul></ul><ul><li>Analyze Execution Plan </li></ul><ul><ul><li>Explain Query / V$SQL_PLAN </li></ul></ul><ul><ul><li>Optimize Query </li></ul></ul><ul><ul><li>Optimize Indexes </li></ul></ul><ul><ul><ul><li>Index Only Access, Function Based Indexes </li></ul></ul></ul>
  32. 32. Diagnose <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization </li></ul><ul><li>Memory Usage </li></ul><ul><li>Network problems </li></ul><ul><li>„ idle“ System </li></ul><ul><li>Operating System (WLM, VM) </li></ul>
  33. 33. High I/O utilization <ul><li>Linux/390 </li></ul><ul><ul><li>sar -d 3 33333 </li></ul></ul><ul><ul><li>iostat -x 3 </li></ul></ul><ul><ul><li>vmstat 3 </li></ul></ul><ul><li>Z/OS </li></ul><ul><ul><li>RMF </li></ul></ul><ul><ul><li>Omegamon etc </li></ul></ul>
  34. 34. High I/O utilization <ul><li>Disk I/O </li></ul><ul><ul><li>Disk access is slower than memory access (Factor 5000 to 100000) </li></ul></ul><ul><ul><li>One physical disk is able to perform 100-150 I/O´s per Second </li></ul></ul><ul><ul><li>Disk Reponse Times (Read operations) </li></ul></ul><ul><ul><ul><li>2ms (Read from disk cache) </li></ul></ul></ul><ul><ul><ul><li>10ms – 15ms (Physical Reads) </li></ul></ul></ul>
  35. 35. High I/O utilization <ul><li>Reasons for High I/O utilization </li></ul><ul><ul><li>Database Cache too small (DB_CACHE_SIZE) </li></ul></ul><ul><ul><li>Sortarea too small (sort_area_size) </li></ul></ul><ul><ul><li>Hasharea too small (hash_area_size) </li></ul></ul><ul><ul><li>Too many Checkpoints </li></ul></ul><ul><ul><li>Ineffective Execution Plans (e.g. Full-Table-Scans which are not necessary) </li></ul></ul>
  36. 36. High I/O utilization <ul><li>Increase Cache Size </li></ul><ul><ul><li>Reduces physical I/O Operations </li></ul></ul><ul><ul><li>Z/OS </li></ul></ul><ul><ul><ul><li>Limited by 31 Bit Arcitecture </li></ul></ul></ul><ul><ul><ul><li>Multiple Adress Spaces help to improve the Memory management </li></ul></ul></ul>
  37. 37. High I/O utilization <ul><li>An Oracle server instance has a single SGA regardless of the number of address spaces or regions configured. </li></ul><ul><li>The user context is distributed across all AS </li></ul>Single Shared SGA Across Address Spaces AS1 AS2 AS3 ASn
  38. 38. High I/O utilization <ul><li>Linux/390 </li></ul><ul><ul><li>The default maximum SGA size on Linux/390 is 750 MB without changing the base adress </li></ul></ul><ul><ul><li>the maximum SGA size to 1 GB by changing the SGA base address </li></ul></ul>
  39. 39. % Total Event Waits Time (s) Wt Time -------------------------------------------- ------------ ------------ ------- db file sequential read 89,086,819 11,009 93.13 db file scattered read 9,875,076 776 6.56 file open 505,227 23 .19 log file sync 440,409 8 .07 latch free 11,042,510 3 .03 High I/O utilization Top 5 Timed Events
  40. 40. Tablespace Av Av Av Av Buffer Av Buf Reads Reads/s Rd(ms) Blks/Rd Writes Writes/s Waits Wt(ms) -------------- ------- ------ ------- ------------ -------- ---------- ------ RECEIVABLE_T_01 18,398,460 213 12.0 1.6 59,325 1 4,892,686 0.0 SO_T_03 6,827,475 79 13.2 1.6 27,462 0 4,506 0.0 SO_I_01 5,356,393 62 9.0 1.3 18,388 0 35,935 0.0 PO_I_01 4,641,732 1400 21.7 1.8 72,563 1 217,799 0.0 Tablespace I/O Stats: High I/O utilization
  41. 41. D I R E C T A C C E S S D E V I C E A C T I V I T Y DEVICE AVG AVG AVG AVG AVG AVG AVG AVG % % % AVG % % STORAGE DEV DEVICE VOLUME LCU ACTIVITY RESP IOSQ DPB CUB DB PEND DISC CONN DEV DEV DEV NUMBER ANY MT GROUP NUM TYPE SERIAL RATE TIME TIME DLY DLY DLY TIME TIME TIME CONN UTIL RESV ALLOC ALLOC PEND DBORACLE 7651 33903 LEOR00 008F 0.817 4 0 0.0 0.0 0.0 0.2 2.6 0.8 0.06 0.28 0.0 1.0 100.0 0.0 DBORACLE 7652 33903 LEOR01 008F 0.878 9 0 0.0 0.0 0.0 0.2 0.3 8.7 0.76 0.79 0.0 3.0 100.0 0.0 DBORACLE 7653 33903 LEOR02 008F 0.502 2 0 0.0 0.0 0.0 0.2 0.0 1.5 0.08 0.08 0.0 6.0 100.0 0.0 DBORACLE 7654 33903 LEOR03 008F 108.968 56 52 0.0 0.0 0.0 0.2 2.4 0.8 0.08 0.32 0.0 1.0 100.0 0.0 DBORACLE 7655 33903 LEOR04 008F 0.828 3 0 0.0 0.0 0.0 0.2 2.3 0.8 0.06 0.25 0.0 1.0 100.0 0.0 DBORACLE 7656 33903 LEOR05 008F 98.779 50 48 0.0 0.0 0.0 0.2 1.7 0.8 0.13 0.42 0.0 1.0 100.0 0.0 DBORACLE 7657 33903 LEOR06 008F 2.768 2 0 0.0 0.0 0.0 0.3 1.3 0.7 0.20 0.56 0.0 1.0 100.0 0.0 DBORACLE 7658 33903 LEOR07 008F 0.943 3 0 0.0 0.0 0.0 0.2 2.3 0.7 0.07 0.28 0.0 1.0 100.0 0.0 DBORACLE 7659 33903 LEOR08 008F 1.003 4 0 0.0 0.0 0.0 0.2 3.5 0.8 0.08 0.43 0.0 1.0 100.0 0.0 DBORACLE 765A 33903 LEOR09 008F 0.945 3 0 0.0 0.0 0.0 0.2 2.2 0.8 0.07 0.28 0.0 1.0 100.0 0.0 DBORACLE 765B 33903 LEOR0A 008F 0.217 3 0 0.0 0.0 0.0 0.2 2.2 0.8 0.02 0.06 0.0 1.0 100.0 0.0 DBORACLE 765C 33903 LEOR0B 008F 0.833 4 0 0.0 0.0 0.0 0.2 2.5 0.8 0.06 0.28 0.0 2.0 100.0 0.0 DBORACLE 765D 33903 LEOR0C 008F 0.963 4 0 0.0 0.0 0.0 0.2 2.7 0.9 0.09 0.35 0.0 1.0 100.0 0.0 DBORACLE 765E 33903 LEOR0D 008F 0.013 3 0 0.0 0.0 0.0 0.2 2.6 0.5 0.00 0.00 0.0 1.0 100.0 0.0 DBORACLE 765F 33903 LEOR0E 008F 0.935 4 0 0.0 0.0 0.0 0.2 3.0 0.8 0.07 0.35 0.0 1.0 100.0 0.0 RMF Report ( Monitor 1; RMF Postprocessor) High I/O utilization
  42. 42. <ul><li>RMF Report – Explanations </li></ul><ul><ul><li>IOSQ TIME = UCB Queueing time </li></ul></ul><ul><ul><li>Avg Pend Time = ms, all Path´s to logical volume are busy </li></ul></ul><ul><ul><li>AVG Resp Time = Connect Time + Dicsonnect Time + Pending Time + IOSQ </li></ul></ul>High I/O utilization
  43. 43. SQL Tuning <ul><li>Check Object Statsitics </li></ul><ul><ul><li>Use DBMS_STATS </li></ul></ul><ul><li>Analyze Execution Plan </li></ul><ul><ul><li>Explain Query </li></ul></ul><ul><ul><li>Optimize Query </li></ul></ul><ul><ul><li>Optimize Indexes </li></ul></ul><ul><ul><ul><li>Index Only Access, Function Based Indexes </li></ul></ul></ul>
  44. 44. Diagnose <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization </li></ul><ul><li>Memory Usage </li></ul><ul><li>Network problems </li></ul><ul><li>„ idle“ System </li></ul><ul><li>Operating System (WLM, VM) </li></ul>
  45. 45. Memory Problems <ul><li>How to determine Paging/Swapping </li></ul><ul><ul><li>Linux/390 </li></ul></ul><ul><ul><ul><li>VMSTAT </li></ul></ul></ul><ul><ul><li>Z/OS </li></ul></ul><ul><ul><ul><li>RMF </li></ul></ul></ul><ul><ul><ul><li>OMEGAMON </li></ul></ul></ul><ul><li>Reasons for Paging/Swapping </li></ul><ul><ul><li>Too many processes/users </li></ul></ul><ul><ul><li>Database Parameters which are too generously </li></ul></ul><ul><ul><ul><li>DB_CACHE_SIZE </li></ul></ul></ul><ul><ul><ul><li>HASH_SIZE </li></ul></ul></ul><ul><ul><ul><li>SQL_CACHE </li></ul></ul></ul>
  46. 46. Diagnosing high CPU usage -Operating System- <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization </li></ul><ul><li>Memory Usage </li></ul><ul><li>Network problems </li></ul><ul><li>„ idle“ System </li></ul><ul><li>Operating System (WLM, VM) </li></ul>
  47. 47. Diagnosing Network problems <ul><li>Latency </li></ul><ul><ul><li>LAN: < 1ms </li></ul></ul><ul><ul><li>WAN: < 10ms - 500ms </li></ul></ul><ul><ul><ul><li>ISDN: < 50ms </li></ul></ul></ul><ul><ul><ul><li>VPN: 100-500 ms </li></ul></ul></ul><ul><li>Badwidth </li></ul><ul><ul><li>11-18 Mbit (Copper) </li></ul></ul><ul><ul><li>100 Mbit (Copper, fibre) </li></ul></ul><ul><ul><li>1 Gbit (fibre) </li></ul></ul><ul><li>Great number of small packets </li></ul><ul><ul><li>tcp_nodelay </li></ul></ul><ul><ul><li>SDU, TDU-Parameters (not available on z/os) </li></ul></ul>
  48. 48. Diagnosing high CPU usage -Operating System- <ul><li>High CPU Usage </li></ul><ul><li>High I/O utilization </li></ul><ul><li>Memory Usage </li></ul><ul><li>Network problems </li></ul><ul><li>„ idle“ System </li></ul><ul><li>Operating System (WLM, VM) </li></ul>
  49. 49. Idle System <ul><li>One CPU is 100% used – All other CPU´s are idle </li></ul><ul><ul><li>Reason </li></ul></ul><ul><ul><ul><li>dedicated Server </li></ul></ul></ul><ul><ul><ul><li>Only one process is running </li></ul></ul></ul><ul><ul><li>Solution </li></ul></ul><ul><ul><ul><li>Parallel Query </li></ul></ul></ul><ul><ul><ul><ul><li>Not useful for OLTP Aplications </li></ul></ul></ul></ul><ul><ul><ul><li>Split work - run more Processes </li></ul></ul></ul>
  50. 50. Idle System <ul><li>Latch Contentions </li></ul><ul><ul><li>Use Statspack to diagnose </li></ul></ul><ul><li>Enqueue Waits </li></ul><ul><ul><li>Use Statspack to diagnose </li></ul></ul><ul><ul><li>Often Block Contentions because of too small initrans, Freelist, Freelist goup settings </li></ul></ul><ul><li>Parsing because the use of Literals </li></ul><ul><ul><li>Use Statspack to diagnose </li></ul></ul><ul><ul><ul><li>Use CURSOR SHARING </li></ul></ul></ul><ul><ul><ul><li>Use Bind Variables </li></ul></ul></ul>
  51. 51. Idle System Top 5 Timed Events ~~~~~~~~~~~~~~~~~~ % Total Event Waits Time (s) Ela Time -------------------------------------------- ------------ ----------- -------- enqueue 1,560,523 78,588 50.24 CPU time 59,961 38.33 db file sequential read 1,635,253 6,324 4.04 db file scattered read 14,620,725 5,907 3.78 control file parallel write 32,816 1,396 .89 -------------------------------------------------------------
  52. 52. Idle System Enqueue activity for DB: RECONPRD Instance: RECONPRD Snaps: 2 -31 -> Enqueue stats gathered prior to 9i should not be compared with 9i data -> ordered by Wait Time desc, Waits desc Avg Wt Wait Eq Requests Succ Gets Failed Gets Waits Time (ms) Time (s) -- ------------ ------------ ----------- ----------- ------------- ------------ TX 438,961 438,941 20 114 512,902.49 58,471 TC 34,530 34,530 0 6,904 369.61 2,552 PS 9,526,323 9,386,524 139,799 1,517,315 .25 381 CF 42,761 42,751 10 23 897.57 21 CI 55,594 55,594 0 12 6.17 0 HW 11,356 11,356 0 8 .13 0 -------------------------------------------------------------
  53. 53. SQL-Tuning <ul><li>Prerequisites </li></ul><ul><ul><li>Use Cost based optimizer </li></ul></ul><ul><ul><li>DBMS_STATS (important) </li></ul></ul><ul><li>Explain Query </li></ul><ul><ul><li>Create Plan Table: UTLXPLAN </li></ul></ul><ul><li>Visualize Execution Plan </li></ul><ul><ul><li>UTLXPLS </li></ul></ul><ul><ul><li>UTLXPLP </li></ul></ul><ul><ul><li>Note: Scripts are located in xxxxxxxx.yyyyyyyy.SQL library (z/OS) </li></ul></ul><ul><ul><li>$ORACLE_HOME/rdbms/admin (Linux/Unix) </li></ul></ul>
  54. 54. SQL-Tuning <ul><li>SQL> explain plan for select a.* from scott.emp a, scott.dept b where a.deptno=b.deptno; </li></ul><ul><li>Explained. </li></ul><ul><li>SQL> save explain </li></ul><ul><li>Created file explain.sql </li></ul><ul><li>SQL> @?/rdbms/admin/utlxpls </li></ul><ul><li>PLAN_TABLE_OUTPUT </li></ul><ul><li>-------------------------------------------------------------------------------- </li></ul><ul><li>-------------------------------------------------------------------- </li></ul><ul><li>| Id | Operation | Name | Rows | Bytes | Cost | </li></ul><ul><li>-------------------------------------------------------------------- </li></ul><ul><li>| 0 | SELECT STATEMENT | | 14 | 560 | 2 | </li></ul><ul><li>| 1 | NESTED LOOPS | | 14 | 560 | 2 | </li></ul><ul><li>| 2 | TABLE ACCESS FULL | EMP | 14 | 518 | 2 | </li></ul><ul><li>|* 3 | INDEX UNIQUE SCAN | PK_DEPT | 1 | 3 | | </li></ul><ul><li>-------------------------------------------------------------------- </li></ul><ul><li>Predicate Information (identified by operation id): </li></ul><ul><li>PLAN_TABLE_OUTPUT </li></ul><ul><li>-------------------------------------------------------------------------------- </li></ul><ul><li>3 - access(&quot;A&quot;.&quot;DEPTNO&quot;=&quot;B&quot;.&quot;DEPTNO&quot;) </li></ul>
  55. 55. SQL-Tuning <ul><li>Optimizer features which help to improve execution plans </li></ul><ul><ul><li>Function based indexes (very important) </li></ul></ul><ul><ul><ul><li>SELECT * From emp where upper(ename) = ´SMITH´ </li></ul></ul></ul><ul><ul><li>Bitmap indexes (Useful in case of Read Only) </li></ul></ul><ul><ul><ul><li>Useful for Low Cardinality columns </li></ul></ul></ul><ul><ul><li>Parameter: Optimizer_index_cost_adj </li></ul></ul><ul><ul><ul><li>Optimizer access path selection can be adjusted to be more index friendly </li></ul></ul></ul>
  56. 56. SQL-Tuning <ul><li>SQLTRACE </li></ul><ul><ul><li>Prerequisite: timed_statistics=true </li></ul></ul><ul><ul><li>Activate </li></ul></ul><ul><ul><ul><li>Alter Session set SQL_trace=true </li></ul></ul></ul><ul><ul><ul><li>dbms_system.set_sql_trace_in_session </li></ul></ul></ul><ul><ul><li>Use TKPROF to show execution statistics </li></ul></ul><ul><ul><ul><li>sys=no,explain=uid/pw </li></ul></ul></ul>
  57. 57. z/OS WLM <ul><li>Everything works fine without peaks (e.g.CPU 30%) </li></ul><ul><li>Common Problems we had with WLM(during peak periods) </li></ul><ul><ul><li>The „Everything is important syndrom“ </li></ul></ul><ul><ul><ul><li>User didn´t classify any discretionary goals </li></ul></ul></ul><ul><ul><ul><li>Everything had the same importance </li></ul></ul></ul><ul><ul><li>Enclave(Sess) with response time goals </li></ul></ul><ul><ul><ul><li>Enclave goes to last period (which was discretionary) shortly after Logon </li></ul></ul></ul><ul><ul><li>No default service class for OSDI </li></ul></ul><ul><ul><ul><li>Mistake in classification rules will result in SYSOTHER being used – discretionary goal </li></ul></ul></ul>
  58. 58. Oracle for Linux /390 <ul><li>We had tuning work </li></ul><ul><ul><li>Linux on an LPAR </li></ul></ul><ul><ul><li>Linux under VM </li></ul></ul><ul><li>We did not have any VM related problems </li></ul><ul><li>The reasons for performance bottlenecks were </li></ul><ul><ul><li>Execution plan of a few SQL Queries </li></ul></ul><ul><ul><li>I/O Subsystem </li></ul></ul>
  59. 59. Oracle for z/OS <ul><li>The reasons for performance bottlenecks were </li></ul><ul><ul><li>WLM configuration </li></ul></ul><ul><ul><li>Execution plan of a few SQL Queries </li></ul></ul><ul><ul><li>I/O Subsystem </li></ul></ul><ul><ul><ul><li>Variances in disc response time </li></ul></ul></ul>
  60. 60. ?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×