K E L L Y N P O T ’ V I N
S R . T E C H N I C A L C O N S U L T A N T
EM12c Performance Diagnosis
and Tuning Outside the Box
Oracle ACE Director, Sr. Technical Specialist at
Specialize in performance and management of large
Board of directors for RMOUG, Director of Training
Days and Database Track Lead for KSCOPE 2013
Performance Diagnostics in EM12c
Simple access to performance, resource usage and
Data collection to investigate performance issues-
current, recent and historical.
Have the real answer, not assumptions.
Performance Out of the Box with EM12c
Tools at your Disposal
Top Activity, “The Grid”
Graphical display of performance usage.
15 second refresh, manual refresh or historical.
When to Worry
Out of the Ordinary Activity, (KNOW YOUR DB!)
Colors outside of green and [some] blue.
Large amounts of blue, (high IO)
Remember that pink, (unknown) red,
(concurrency/application) tan, (network) and
orange, (commit) in the grid should be investigated.
Brown or black? Run for the hills! (JK)
Here’s our spike, which waits?
Commonly, focus on pink,
orange, red and brown for
Network and queuing do
have opportunities for
tuning, as well.
Green and blue are expected,
but also part of problems
when over utilized.
We’re in the Red, (Orange, too!)
The Icing on the Cake
Duh, add some memory to the EM12c box!
SQL Monitor for Performance
• Elapsed Time
• SQL_ID, Beginning SQL Text.
• Parallel, Waits and Execution Time
• Choose your session, SQL_ID or SQL_Text
• Shows active, completed sessions for amount of time chosen.
• Shows high level wait events, dbtime, IO usage and duration.
By SQL_ID, we can inspect:
• DB Time
• PL/SQL Java time
• Wait Activity
• Buffer Gets
• IO Requests and IO Bytes
• If Exadata, Offload Efficiency
Monitoring Procedural Call
All SQL_ID’s called will show, along with
duration so it’s simple to pinpoint trouble
• Note that the SQL Statement, along with elapsed time is
• Data sources from Top Activity, not AWR data.
And More Detail
Session info, wait info, cursors and stats.
Along with the main stats-
Activity information on the statement.
The execution plan
If there is a SQL Plan or outline in place.
If there have been any tuning advisors run against the
And a direct link to SQL Monitoring
How to Use SQL Monitoring
Active Monitoring of database processing.
Investigation of performance.
Save off reports, which provide a graphical image of
performance differing from Top Activity or ASH
Distinct diagnosis at a session or statement level.
Future of Top Activity
Package installation to database.
Always on, non-impact of Top Activity performance
More defined, more accurate.
Historical data enhanced over Top Activity historical
Pick Your View
Ability to choose timelines by:
Custom Review Pane
• You can choose to change the overview pane to display data for any
amount of time.
• Just click on the pane and drag it to the area you are interested in or
extend it to cover the areas you are interested to investigate.
• Choose your filters or view all data and you are ready to go!
Similar to Top Activity when in “Activity”
Pick Your Poison
View data very similar to the SQL and Session data
in Top Activity.
All data is sourced by AWR data and dependent on
samples and AWR retention/interval info in the
Activity shows wait detail over time.
Processes, including parallel sessions involved
during shaded time.
Option to run AWR or ASH report.
The Rest of the Story
For standard SQL- Plan, Plan Control and Tuning
History is shown under individual tabs.
SQL Monitor is minimized access to the SQL
New Visual Way of Showing Data, Multiple
Data Break Down
Display offers incredible diversity in wait, resource
usage and other critical event choices.
ASH Analytics – When to Use It
Need the more defined ASH data for EM diagnostics.
Want a second way to present data to less “DBA”
centric groups, (load map)
Database level OR session/statement level
Dig down deep, present data in numerous formats to
get the most complete picture of a complex issue.
Can be used for Real-time or historical analysis.
Yes, it requires a PL/SQL installation for the view
Uses ADDM data for the source.
Always on, low to no impact.
Normal Mode or Emergency Mode when Emergency
Monitoring is required.
On Your Mark, Get Set…
This is a recorded ADDM session, beginning from
the time you click “Start”.
In Progress Data
Ability to stop
notifies of any
Once finished, verify no failures/errors occurred in
Use the tabs to investigate findings, activity, hang
data and statistics.
The number of findings are shown.
Example shows low priority SQL statements using
significant db time, but not other issues at this time.
If any issues are found that are high priority, will be
listed in red and details below the main pane, (low,
medium, high priority levels.)
Activity Data, but sourced from ADDM.
Similar output to Top Activity and ASH Analytics.
• By highlighting a wait link on the right, you can detail down to the actual wait
information for that wait event.
If a database hang situation occurred and the real-
time ADDM was used to diagnose, then the HANG
DATA tab will show any diagnostic data it has
collected during the collection.
Last but not Least…
Initialization Parameter data for the database
Any undocumented of non-recommended parameter
settings will be identified and listed in the findings
Compare Period ADDM
How is it different from Real-Time ADDM?
Ability to compare TWO snapshots in time, side by side of
Compares ADDM snapshots against each other, (dependent on
snapshot intervals and retention.)
All comparisons can be saved off or mailed from the console,
(mailed through EM12c settings)
• Clear comparison from previous day, same time to see performance issue vs.
the right hand side snapshot.
• Commonality comparison of the SQL for snapshots being compared.
• Note the concurrency, commits and increased application waits.
It’s all in the Details
First tab shows any configuration differences
between the two snapshots and what the
configuration parameter is.
Findings Summary Detail
Shows comparison increases or decreases in waits.
Lists the percentage of change between each period
Upon highlighting, details data regarding the
increase or decrease.
We can dig down into each of the SQL Statements
found to be the highest impacts to the system and
Finding Detail Descriptions
As shown above, the wait on Checkpoints to Tablespace are
describe below once you highlight the section in the findings
And for RAC, some waits can be broken down by instance.
Resource Usage: CPU
CPU Usage is viewable
by instance and total
If no CPU bound wait
issues were seen, its
stated by comparison
Resource Usage: Memory
• If you note, Memory has a warning alert by the tab to point you to it after the
comparison is completed.
• The base and comparison is in red, meaning that Virtual paging was an issue
in both snapshots.
• Data is separated by instance in RAC, showing clear usage for better
Resource Usage: IO
I/O is separated by Throughput and Single block
Again, if there was an issue, a warning would be on
the IO tab and the Base and Comparison would show
in red instead of green.
Resource Usage: Interconnect
As this is RAC, note that we also have an
interconnect tab with data on the speed and
Total vs. rate on throughput is viewed through a
radio button choice.
So What Changed?
The graphs show us where we need to focus:
How to Use the Comparison ADDM
Excellent to diagnose “what has changed”.
“Just the Facts” information on a comparison of
Dependent upon retention time settings and
intervals for AWR.
Historical data, can be set by date, custom, by
Will move to next snapshot window if mid-snapshot
time span is chosen.
Leighton Nelson- http://blogs.griddba.com/
Gokhan Atil- http://www.gokhanatil.com/
Martin Bach- http://martincarstenbach.wordpress.com
Niall Litchfield- http://orawin.info/blog/
Info for Me!
Company Website: www.enkitec.com
Linkedin: Kellyn Potvin and/or Rocky Mountain Oracle User Group
Email: email@example.com or firstname.lastname@example.org or
Kscope13 features more than 300
educational sessions, full-day
symposiums, hands-on training courses,
informal networking sessions, and a
plethora of chances to increase your
technical know-how by learning from the
• Application Express
• ADF and Fusion Dev.
• Developer's Toolkit
• The Database
• Building Better Software
• Business Intelligence
• Financial close
• EPM Reporting
• EPM Foundations and Data
• EPM Business Content http://kscope13.com/registration