About the course:
This Oracle performance tuning online course is designed for the audience who want to learn basics and core concepts of Oracle PT. You will be learning about Introduction, basic tuning diagnostics, how to use automatic workload repository, defining of problems, how to create AWR baselines, monitoring of applications Etc. All Oracle performance tuning classes will be live and interactive.
Course Target:
Oracle performance tuning online training is designed to teach you fundamentals of PT.
Understand basic tuning diagnostics.
Learn how to use Automatic workload repository.
Obtain knowledge of using metrics and alerts.
Clear understanding of how to monitor applications.
Need to identify problem SQL statements
Learn how to influence the optimizer.
Understand SQL performance management.
Tuning the shared pool, I/0, Buffer cache, PGA and temporary space.
Course Targeted Audience:
Any candidate can join our Oracle performance tuning online course.
People who are from professional background can join.
Researches can also participate in this course.
Prerequisites:
Candidates with basic knowledge of computer.
Basics of database are recommended.
Training Format:
Kernel Training provides Oracle performance tuning online course led by real time expert.
Registered Candidates can interact with instructor in live interactive sessions.
Candidates will have life time access to learning material.
Companies Using Oracle PT:
Major international IT companies perform Oracle performance tuning for their operations.
2. www.kerneltraining.com
In this Document
Purpose Questions and Answers ProActive Problem Avoidance and
Diagnostic Collection Performance Information Centres Troubleshooting
Guides Common Problem Topics Commonly searched performance related
Events, Warnings and Errors Slow Database Performance Concurrency
Issues Hangs Database Hangs/Spins Session Hangs/Spins Single SQL
Statement Hangs/Spins
Locking Issues Deadlock PreUpgrade Planning PostUpgrade
Performance Issue Resolution Debugging Waits for Various Events Library
Cache/Mutex Contention/Cursor Type Events: Other Types: Common
Causes of Performance issues Cursor Sharing/High Version Counts for
Cursors High CPU usage Issues With waits for 'log file sync' WAITED TOO
LONG FOR A ROW CACHE ENQUEUE LOCK! Buffer Busy/Cache Buffers
Chains Latch waits SYSAUX Issues
3. www.kerneltraining.com
In this Document
Performance Diagnostics References Performance Service Request
Diagnostic Collection (SRDC) documents General Diagnostics Overview
AWR/Statspack 10046 Trace Systemstates/Hanganalyze Errorstacks for
Performance Issues PStack PL/SQL Profiler OS Watcher LTOM 7/29/2015
Document 402983.1
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-
state=xpn86qnwb_201&id=402983.1 2/12 Trace/Result Interpretation
Performance (and other) Webcasts Performance and Scalability White
Papers and Documentation Interacting With Performance Support
Community: Database Tuning Community Discussions References
4. www.kerneltraining.com
APPLIES TO
Oracle Database Enterprise Edition Version 6.0.0.0 and later Oracle
Database Personal Edition Version 7.1.4.0 and later Enterprise Manager for
Oracle Database Version 8.1.7.4 and later Oracle Database Standard Edition
Version 7.0.16.0 and later Information in this document applies to any
platform.
5. www.kerneltraining.com
PURPOSE
This Document provides an overview reference to content that pertains
to the resolution of Performance issues affecting the whole database. For the
Database Performance Frequently Asked Questions FAQ,information
regarding Performance Related Features or issues involving individual SQL
statements, see: Document 1360119.1 FAQ: Database Performance
Frequently Asked Questions Document 1361401.1 Where to Find
Information About Performance Related Features Document 199083.1
Master Note: SQL Query Performance Overview Document 1516494.2
Information Center: SQL Query Performance Overview Document 398838.1
FAQ: SQL Query Performance Frequently Asked Questions
6. www.kerneltraining.com
QUESTIONS AND ANSWERS
ProActive Problem Avoidance and Diagnostic Collection Although some
problems may be unforeseen, in many cases problems may be avoidable if signs are
detected early enough. Additionally, if an issue does occur, it is no use collecting
information about that issue after the event. For information on suggested
preparations to avoid issues and in case diagnostics are required, see: Document
1482811.1 Best Practices: Proactively Avoiding Database and Query Performance
Issues Document 1477599.1 Best Practices Around Data Collection For Performance
Issues Performance Information Centres Performance Information Centres provide
access to various aspects of performance issues: Document 1512292.2 Information
Center: Database Performance Overview Document 1516494.2 Information Center:
SQL Query Performance Overview 7/29/2015 Document 402983.1
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-
state=xpn86qnwb_201&id=402983.1 3/12 Troubleshooting Guides There are a
number of troubleshooting guides that can help resolve various issues: Document
1543445.2 Troubleshooting Assistant: Troubleshooting Performance Issues
Document 1542678.2 Troubleshooting Assistant: SQL Performance Issues Document
1528847.1 Troubleshooting: Avoiding and Resolving Database Performance Related
Issues After Upgrade Document 60.1 Troubleshooting Assistant: Oracle Database
ORA00060 Errors on Single Instance (NonRAC) Diagnosing Using Deadlock Graphs
in ORA00060 Trace Files
7. www.kerneltraining.com
COMMON PROBLEM TOPICS
Commonly searched performance related Events, Warnings and Errors The
following lists commonly searched Events, Warnings and Errors that are related to
performance and points to key troubleshooting documents: Document 2008188.1
Commonly Searched for Database Performance Related Events, Warnings and
Errors Slow Database Performance If the database is performing slowly see the
following: Document 1362329.1 How To Investigate Slow or Hanging Database
Performance Issues Concurrency Issues If you have problems with the concurrency
of transactions, initially treat this as a "Slow Database Performance" issue and refer
to the following: Document 1362329.1 How To Investigate Slow Database
Performance Issues Hangs Database Hangs/Spins If the database encounters what
looks like a hang or a spin conditions refer to the following: Document 452358.1
How to Collect Diagnostics for Database Hanging Issues Document 1362329.1 How
To Investigate Slow or Hanging Database Performance Issues Document 68738.1 No
Response from the Server, Does it Hang or Spin? Document 392037.1 Database
Hangs. Sessions wait for 'resmgr:cpu quantum'
8. www.kerneltraining.com
COMMON PROBLEM TOPICS
Session Hangs/Spins If an individual session appears to hang or spin refer to the
relevant sections in the following articles: Document 1362329.1 How To Investigate
Slow or Hanging Database Performance Issues Document 352648.1 How to
Diagnose high CPU usage problems Single SQL Statement Hangs/Spins If an
individual SQL statement appears to hang or spin refer to the relevant sections in
the following articles: Document 1362329.1 How To Investigate Slow or Hanging
Database Performance Issues Document 1916479.1 How to Investigate Hanging SQL
Statements
9. www.kerneltraining.com
COMMON PROBLEM TOPICS
Locking Issues If sessions encounter locking conflicts refer to the following:
Document 1392319.1 Master Note: Locks, Enqueues and Deadlocks Document
15476.1 FAQ about Detecting and Resolving Locking Conflicts Document 62354.1 TX
Transaction locks Example wait scenarios Document 33453.1 REFERENTIAL
INTEGRITY AND LOCKING Deadlock ORA60 is an application error which
usually occurs because a consistent locking strategy has not been followed
throughout an application. Please refer to the following articles for more
information: Document 1509919.1 Master Note for Database Error ORA00060
Document 18251.1 OERR: ORA 60 "deadlock detected while waiting for resource"
Document 62365.1 What to do with "ORA60 Deadlock Detected" Errors Please
ensure that you are encountering none of the conditions in these articles. Pre-
Upgrade Planning For upgrades to 11g, there is a useful webcast entitled "11G
Upgrade Best Practices" : This recorded seminar provides an overview of tips and
best practices for upgrading your database to 11G. It starts with general Challenges
and Support Best Practices, but moves on to performance specifics such as AWR and
STATSPACK, SQL Plan Management, and Real Application Testing.
10. www.kerneltraining.com
COMMON PROBLEM TOPICS
This webcast can be found here: 11G Upgrade Best Practices 7/29/2015
Document 402983.1
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-
state=xpn86qnwb_201&id=402983.1 5/12 For advice on planning your upgrade see:
Document 785351.1 Oracle 11gR2 Upgrade Companion Document 601807.1 Oracle
11gR1 Upgrade Companion Document 466181.1 10g Upgrade Companion
Document 465787.1 Managing CBO Stats during an upgrade to 10g or 11g For more
general upgrade related information see: Document 1645862.1 Things to Consider
Before Upgrading to 11.2.0.4 to Avoid Poor Performance or Wrong Results
Document 1392633.1 Things to Consider before Upgrading to 11.2.0.3 to Avoid Poor
Performance or Wrong Results Document 1320966.1 Things to Consider before
Upgrading to 11.2.0.2 to Avoid Poor Performance or Wrong Results Document
1464274.1 Master Note for Real Application Testing Option Document 762540.1
Consolidated Reference List Of Notes For Migration / Upgrade Service Requests
PostUpgrade Performance Issue Resolution To troubleshoot performance issues
encountered after upgrading a database see: Document 1528847.1 Troubleshooting:
Avoiding and Resolving Database Performance Related Issues After
11. www.kerneltraining.com
COMMON PROBLEM TOPICS
Debugging Waits for Various Events There are numerous articles to help you
debug issues with wait event contention. The most common of these being: Library
Cache/Mutex Contention/Cursor Type Events: Document 1356828.1 FAQ: 'cursor:
mutex ..' / 'cursor: pin ..' / 'library cache: mutex ..' Type Wait Events Document
1377998.1 Troubleshooting: Waits for Mutex Type Events Document 1349387.1
Troubleshooting 'cursor: pin S wait on X' waits Document 1357946.1
Troubleshooting 'library cache: mutex X' waits. Other Types: Document 1376916.1
Troubleshooting: "Log File Sync" Waits Document 1342917.1 Troubleshooting 'latch:
cache buffers chains' Wait Contention Document 34405.1 WAITEVENT: "buffer busy
waits" Reference Note Document 223117.1 Troubleshooting I/Orelated waits
Document 1415999.1 Troubleshooting: Virtual Circuit Waits
12. www.kerneltraining.com
COMMON PROBLEM TOPICS
Common Causes of Performance issues Cursor Sharing/High Version Counts
for Cursors The failure to share cursors effectively can have a highly detrimental
affect on a database. For example, having hundreds of versions of the same cursor
is likely to be wasting space, is unlikely to promote good performance and, in
extreme cases, can cause contention issues. For issues linked to Cursor related waits
see: Document 1356828.1 FAQ: 'cursor: mutex ..' / 'cursor: pin ..' / 'library cache:
mutex ..' Type Wait Events Also, see the following articles for more information and
help with troubleshooting issues: Document 296377.1Troubleshooting: High
Version Count Issues Document 438755.1 High SQL Version Counts Script to
determine reason(s) Document 62143.1 Understanding and Tuning the Shared Pool
High CPU usage In cases where CPU usage of individual sessions or the whole
database is high, refer to the following: Document 352648.1 How to Diagnose High
CPU Usage Problems to the Module Level Document 273646.1 How to diagnose the
high CPU utilization of ORACLE.EXE in Windows environment Note that High
CPU usage can be caused by many factors including contention for cursors (see
above), inefficient SQL and may be associated with excessive buffer reads etc.
13. www.kerneltraining.com
COMMON PROBLEM TOPICS
Issues With waits for 'log file sync' If your system is waiting for the redo log files to
synchronise with the disk, the following articles can help: Document 1376916.1
Troubleshooting: "Log File Sync" Waits Document 34592.1 WAITEVENT: "log file sync"
Reference Note Document 857576.1 How to Minimise Waits for 'Log File Sync'? Document
1278149.1 High Log File Sync Wait Events, LGWR Posting Slow Write Times, Low IO Waits
Document 1064487.1 Script to Collect Log File Sync Diagnostic Information (lfsdiag.sql)
WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! This Issue occurs when the
database detects that a waiter has waited for a resource for longer than a particular threshold.
The message "WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK!" appears in the
alert log and trace and systemstates are dumped. 7/29/2015 Document 402983.1
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-
state=xpn86qnwb_201&id=402983.1 7/12 Typically this is caused by two (or more)
incompatible operations being run simltaneously. See: Document 278316.1 Troubleshooting:
"WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK!" Buffer Busy/Cache Buffers
Chains Latch waits Buffer Busy waits occur when a session wants to access a database block in
the buffer cache but it cannot as the buffer is "busy". Cache Buffers Chains Latch waits are
caused by contention where multiple sessions waiting to read the same block. Typical
solutions are to look for SQL that accesses the blocks in question and determine if the repeated
reads are necessary. Document 1342917.1 Troubleshooting 'latch: cache buffers chains' Wait
Contention Document 34405.1 WAITEVENT: "buffer busy waits" Reference Note Document
155971.1 Resolving Intense and "Random" Buffer Busy Wait Performance Problems:
Document 163424.1 How To Identify a Hot Block Within The Database Buffer Cache
14. www.kerneltraining.com
COMMON PROBLEM TOPICS
SYSAUX Issues SYSAUX is a mandatory tablespace in 10g which stores all
auxiliary database metadata related to various product options and features. Of
particular interest to Performance is the storage of Automatic Workload Repository
(AWR) data: Document 552880.1 General Guidelines for SYSAUX Space Issues
Document 1055547.1 SYSAUX Grows Because Optimizer Stats History is Not
Purged Document 387914.1 WRH$_ACTIVE_SESSION_HISTORY Does Not Get
Purged Document 554831.1 How to Control the Set of Top SQLs Captured During
AWR Snapshot Generation Document 1292724.1 SYSAUX Tablespace Grows
Rapidly
15. www.kerneltraining.com
Performance Diagnostics References
Performance Service Request Diagnostic Collection (SRDC) documents
Service Request Data Collection (SRDC) documents have been specially
designed to provide the reader with the necessary instructions to provide a
step by step guide to collecting information for a various common
Database Performance Issues. Document 1938786.1 List of Available
Database Performance Related SRDC Documents General Diagnostics
Overview Document 438452.1 Performance Tools Quick Reference Guide
7/29/2015 Document 402983.1
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-
state=xpn86qnwb_201&id=402983.1 8/12 Document 781198.1 Required
Diagnostics for Database Performance Issues
16. www.kerneltraining.com
Performance Diagnostics References
AWR/Statspack Document 1363422.1 AWR Reports Start Point
Document 94224.1 FAQ Statspack Complete Reference 10046 Trace
Document 376442.1 Recommended Method for Obtaining 10046 trace for
Tuning Systemstates/Hanganalyze Document 121779.1 Taking a
SYSTEMSTATE dump when you cannot CONNECT to Oracle. Document
175006.1 Steps to generate HANGANALYZE trace files Errorstacks for
Performance Issues Document 1364257.1 How to Collect Errorstacks for use
in Diagnosing Performance Issues PStack Document 70609.1 How To
Display Information About Processes on SUN Solaris
17. www.kerneltraining.com
Performance Diagnostics References
PL/SQL Profiler Document 243755.1 Implementing and Using the PL/SQL
Profiler OS Watcher OS Watcher (OSW) is a collection of UNIX shell scripts
intended to collect and archive operating system and network metrics to aid
support in diagnosing performance issues. Document 301137.1 OS Watcher User
Guide LTOM The Lite Onboard Monitor (LTOM) is a java program designed as a
proactive, realtime diagnostic platform and provides realtime automatic problem
detection and data collection. Document 352363.1 LTOM The OnBoard Monitor
User Guide 7/29/2015 Document 402983.1
https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-
state=xpn86qnwb_201&id=402983.1 9/12 Document 461050.1 The LTOM Graph
(LTOMg) User Guide Document 461228.1 The LTOM Graph FAQ Document
461052.1 LTOM System Profiler Sample Output Document 977975.1 Diagnose
LTOM Connection Problems Trace/Result Interpretation The following articles can
help with the interpretation of various diagnostics: Document 390374.1 Oracle
Performance Diagnostic Guide (OPDG) Document 39817.1 Interpreting Raw
SQL_TRACE and DBMS_SUPPORT.START_TRACE output
http://www.oracle.com/technetwork/database/focus-
areas/performance/statspackopm4134117.pdf
18. www.kerneltraining.com
Performance Diagnostics References
Performance (and other) Webcasts A series of Performance (and General Datbase
topic) related webcasts, including topics such as SQLHC, SQLTXPLAIN and
OSWBB, can be found here: Document 1456176.1 Oracle Database Advisor Webcast
Archives Performance and Scalability White Papers and Documentation
Performance and Scalability White Papers:
http://www.oracle.com/technetwork/database/focus-
areas/performance/index.html For White papers concerning Real Application
Testing (RAT) and SQL Performance Analyzer (SPA) see: Document 1546337.1 Real
Application Testing (RAT): Recommended White Papers Links to the main Tuning
and Performance documentation: Document 1195363.1 Database Performance and
SQL Tuning Documentation on OTN Interacting With Performance Support
Document 210014.1 How to Log a Good Performance Service Request Document
166650.1 Working Effectively With Global Customer Support Community:
Database Tuning A community has been set up for My Oracle Support (MOS) users
moderated by Oracle. The goal of this community is to exchange database related
Tuning knowledge and concepts. The community can be found via the following
article: Document 1383594.1 Collaborate With MOS Database Tuning Community
Members
20. www.kerneltraining.com
Contents
Introduction Query Tuning Enterprise Manager (SQL
Tuning Advisor) AWR SQL Report SQLTXPLAIN
TRCANLZR PL/SQ_ Profiler LTOM (Session Trace Collector)
OPDG OS Data OS_Watcher Database Tuning Enterprise
Manager ADDM ADDM Report STATSPACK AWR Report
OPDG Hang, Locking, and Transient Issues ASH Report
LTOM (Hang Detector, Data Recorder) HangFG Error/Crash
Issues Stackx ORA600/ORA7445 Troubleshooter RAC RDA
21. www.kerneltraining.com
Introduction
The purpose of this note is to provide a quick guide to
available tools that can be used for resolving performance
issue. A brief description and a link to a more detailed
document for 7/29/2015 Document 438452.1
https://support.oracle.com/epmos/faces/DocumentDisplay?
_adf.ctrlstate=xpn86qnwb_250&id=438452.1 2/7 each
available tool is provided below. NOTE: Most of the tools
referenced in this note can be installed and used at the time of
the performance issue. There are some exceptions however.
LTOM and OS Watcher need to be installed and running prior
to the performance. More details on these tools are provided
below.
22. www.kerneltraining.com
Query Tuning
A query tuning issue can manifest itself as: A particular
SQL statement or group of statements that run slowly at a time
when other statements run well. One or more sessions are
running slowly and most of the delay occurs during the
execution of a particular SQL statement. There are many tools
that are beneficial for tuning queries. Use one or more of the
below tools when troubleshooting query tuning issues.
25. www.kerneltraining.com
Hang, Locking, and Transient Issues
A "true" database hang can manifest itself several ways. Some examples
are: A database that is no longer allowing users to connect. A database that is
no longer performing work. "Select 1 from dual" does not produce output.
Create table does not complete. One or more sessions that have completely
stopped functioning is usually identified as a locking issue. Transient issues
related to performance are issues that generally occur for a time and then
disappear but, during the occurrence, the system is hung or very slow. LTOM
and HangFG are tools that assist in collecting the appropriate data when these
issues occur. Use these helpful tools and utilities when troubleshooting hang,
locking, and transient issues
28. www.kerneltraining.com
Error/Crash Issues
In this section, error and crash issues refer to ORA600, ORA7445 errors, and core
dumps. Use the tools below to assist in troubleshooting these types of issues.
30. www.kerneltraining.com
Legal Notices and Terms of Use
@keywords @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu @performance hang detection slow vmstat top unix 10046 statspack hanganalyze systemstate
memory cpu