1. ASH – Active Session History
Feel the Power
Kyle Hailey
http://perfvision.com
2. #.2
Copyright 2006 Kyle Hailey
ASH
Whole New Paradigm in
Technology
The Power of ASH lies in
Simplifying Performance Tuning
Totally new and exciting methodology
Cheaper, Quicker, Richer and better tasting too
3. #.3
Copyright 2006 Kyle Hailey
Why should you care?
Because ASH can Change your life …
10g immediately Accessible
Geeks: Via scripts in SQL (?/rdbms/admin/ashrpt.sql)
Managers : Graphical OEM 10g
Stuck on Oracle 7,8,9 ?
… my apologies because 10g rocks, but …
no worries, the data is accessible for you too via scripts
http://perfvision.com/sash.php
4. #.4
Copyright 2006 Kyle Hailey
ASH uses Sampling
New Paradigm
Clear Vision of the future
Use new sampling technology
Lets go of the need to know 100%
Statistical approximation
Cheaper
Richer Multidimensional Data
Much more powerful, identifies
SQL
Sessions
Objects
Machine
User
5. #.5
Copyright 2006 Kyle Hailey
To identify Players before ASH
Sessions
# sessions x (# wait events + statistics)
Example (150 x (800+200) = 150,000 )
SQL
v$sql
Could be 10000s
Takes out latches that compete with other sql executions
Objects
V$segstat 9i+
Could be 1000s of objects
Expensive !
6. #.6
Copyright 2006 Kyle Hailey
ASH – Intelligence for the new
Millennium
Intelligently Collects Data
It self adjusts for your needs
More activity, more data collected
Less activity, less data collected
Those old methods collected everything
Obfuscated the problem, too many statistics too late
Costly
Too Granular – once an hour ?! Give me a break
7. ASH Samples Session State
TIME?? ?? ?? ?? ??
Sessions change a lot quicker but can get the main
picture via sampling by sampling faster
Every secondEvery second
10:00:00 10:00:01 10:00:02 10:00:03 10:00:04 10:00:05
9. #.9
Copyright 2006 Kyle Hailey
If happens a lot or for long …
we’ll catch it, guaranteed
• Long running queries easily foundLong running queries easily found
• Short high execution queries found most of the timeShort high execution queries found most of the time
• Short low execution queries rarely foundShort low execution queries rarely found
Produces a good representation of load on DBProduces a good representation of load on DB
10. #.10
Copyright 2006 Kyle Hailey
Session States
IO CPU IdleWait
WorkWork
LatencyLatency
ContentionContention
11. #.11
Copyright 2006 Kyle Hailey
IDLE
Example : SQL*Net Message from Client
10g
9i or below – just have to keep track
Statspack STATS$IDLE_EVENT
select name
from v$event_name
where wait_class='Idle';
58 Rows
12. #.12
Copyright 2006 Kyle Hailey
CPU
ASH: SESSION_STATE = “ON CPU”
ASH: wait_time > 0
10g: v$session
9i: v$session_wait
w.wait_time != 0 /* on CPU */ and
s.status='ACTIVE' /* ACTIVE */
Identifying CPU via v$session_wait isIdentifying CPU via v$session_wait is
a huge break througha huge break through
13. #.13
Copyright 2006 Kyle Hailey
IO
ASH: SESSION_STATE=‘WAITING’
&
10g: WAIT_CLASS=‘User I/O’
9i:
Look at 10g for ideas
db file sequential read
db file scattered read
db file parallel read
direct path read
15. Session 1
Session 2
Session 3
Session 4
TIME
Samples for all users
10:15:00 10:15:01 10:15:02 10:15:03 10:15:04 10:15:05 10:15:06 10:15:07
16. #.16
Copyright 2006 Kyle Hailey
v$active_session_history
SESSION_ID NUMBER
SESSION_SERIAL# NUMBER
USER_ID NUMBER
SERVICE_HASH NUMBER
SESSION_TYPE VARCHAR2(10)
PROGRAM VARCHAR2(64)
MODULE VARCHAR2(48)
ACTION VARCHAR2(32)
CLIENT_ID VARCHAR2(64)
EVENT VARCHAR2(64)
EVENT_ID NUMBER
EVENT# NUMBER
SEQ# NUMBER
P1 NUMBER
P2 NUMBER
P3 NUMBER
WAIT_TIME NUMBER
TIME_WAITED NUMBER
CURRENT_OBJ# NUMBER
CURRENT_FILE# NUMBER
CURRENT_BLOCK# NUMBER0
SQL_ID VARCHAR2(13)
SQL_CHILD_NUMBER NUMBER
SQL_PLAN_HASH_VALUE NUMBER
SQL_OPCODE NUMBER
QC_SESSION_ID NUMBER
QC_INSTANCE_ID NUMBER
SAMPLE_ID NUMBER
SAMPLE_TIME TIMESTAMP(3) When
Session
SQL
Wait
SESSION_STATE VARCHAR2(7)
WAIT_TIME NUMBER
State
TIME_WAITED NUMBER
Duration
17. #.17
Copyright 2006 Kyle Hailey
Groupings – Top Consumer
SESSION_ID
SESSION_SERIAL# (signal SID reuse)
SESSION_TYPE (FOREGROUND,BACKGROUND)
CURRENT_OBJ#
CURRENT_FILE#
CURRENT_BLOCK#
USER_ID (SYS, SYSTEM, SCOTT etc)
SERVICE_HASH (OE,GL,HR)
PROGRAM (SQL, JDBC, Forms etc)
MODULE.ACTION(PLSQL tagging)
CLIENT_ID (identifying users in a session pool)
SQL_ID
QC_SESSION_ID - Query Coordinator
QC_INSTANCE_ID – RAC
EVENT + P1, P2, P3
18. #.18
Copyright 2006 Kyle Hailey
Amazing things YOU can do with
ASH
Consumers
Top Session
Top User
Top SQL
Top Object
Top Module.Action
Top Program
Top Service
Top Client
Top Wait
No machine - collect on logon trigger
X
Resources
• CPU
• Waits
• Event (800*)
• I/O
• File
• Block
• Time
Find top consumers by resource usage
19. #.19
Copyright 2006 Kyle Hailey
Select
session_id,
count(*)
from
v$active_session_history
where
session_state= ‘ON CPU‘ and
SAMPLE_TIME > sysdate – (5/(24*60))
group by
session_id
order by
count(*) desc;
Select
session_id,
count(*)
from
v$active_session_history
where
session_state= ‘ON CPU‘ and
SAMPLE_TIME > sysdate – (5/(24*60))
group by
session_id
order by
count(*) desc;
Top CPU Session
Who is the rogue session ?
22. #.22
Copyright 2006 Kyle Hailey
Select
session_id,
count(*)
from
v$active_session_history
where
session_state=‘WAITING’ and
SAMPLE_TIME > SYSDATE - (5/(24*60))
group by
session_id
order by
count(*) desc;
Select
session_id,
count(*)
from
v$active_session_history
where
session_state=‘WAITING’ and
SAMPLE_TIME > SYSDATE - (5/(24*60))
group by
session_id
order by
count(*) desc;
Top Waiting Session
Top Waiting Session in last 5 minutes
24. #.24
Copyright 2006 Kyle Hailey
Top SQL from ASH
select
ash.SQL_ID ,
sum(decode(ash.session_state,'ON CPU',1,0)) "CPU",
sum(decode(ash.session_state,'WAITING',1,0)) -
sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "WAIT" ,
sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "IO" ,
sum(decode(ash.session_state,'ON CPU',1,1)) "TOTAL"
from v$active_session_history ash,
v$event_name en
where SQL_ID is not NULL and en.event#=ash.event#
group by sql_id
order by sum(decode(session_state,'ON CPU',1,1)) desc
select
ash.SQL_ID ,
sum(decode(ash.session_state,'ON CPU',1,0)) "CPU",
sum(decode(ash.session_state,'WAITING',1,0)) -
sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "WAIT" ,
sum(decode(ash.session_state,'WAITING', decode(en.wait_class, 'User I/O',1,0),0)) "IO" ,
sum(decode(ash.session_state,'ON CPU',1,1)) "TOTAL"
from v$active_session_history ash,
v$event_name en
where SQL_ID is not NULL and en.event#=ash.event#
group by sql_id
order by sum(decode(session_state,'ON CPU',1,1)) desc
35. #.35
Copyright 2006 Kyle Hailey
Many Ways to Attack Problems
Confusing
How to Attack the problem?
Top SQL?
Top wait for that SQL?
Top Waiting Session ?
Top Waits for that Session
Top Waits for Database?
Top Session waiting for that wait
Top SQL for that wait
Solution - Graphics
36. #.36
Copyright 2006 Kyle Hailey
Mining Data is Non Trivial
Many Dimensions to consider
Constantly Varying time frames
Luckily 10g automates all of this functionality.
collects all of the data
analyses all the wait events
reports on bottlenecks
supplies solutions
Graphics add tremendous ease and power
45. #.45
Copyright 2006 Kyle Hailey
ASH
vs
Statistics
Statistics
are more expensive
have lag time
lack clear identification of culprits
46. #.46
Copyright 2006 Kyle Hailey
Statistic Lag Time
CountersCounters
SamplesSamples
Slight LagsSlight Lags
47. #.47
Copyright 2006 Kyle Hailey
CPU Lag Problem
ASH is the only way to see CPU usage
realtime
V$sysstat reports CPU but
is only updated at the end of the call.
Long calls look deceiving like no CPU is being
used
49. #.49
Copyright 2006 Kyle Hailey
ASH Sampling Cheap
Three Methods
1.With PL/SQL it’s less that 1 % CPU with 10
average active sessions
2.DMA - direct memory attach to SGA Using C
code 200x Cheaper
3.ASH in 10g should be even cheaper than
DMA because it’s done in the kernel
50. #.50
Copyright 2006 Kyle Hailey
ASH – In Memory
Collects active session data only
History v$session_wait + v$session + extras
Circular Buffer - 1M to 128M (~2% of SGA)
Flushed every hour to disk or when buffer 2/3
full (it protects itself so you can relax)
51. #.51
Copyright 2006 Kyle Hailey
ASH Sizing …
Avg row around 150bytes
3600 secs in an hour
~ ½ Meg per Active Session per hour
That’s generally over an hour of ASH
52. #.52
Copyright 2006 Kyle Hailey
How ASH works
V$session_wait
+ v$session +
extras
MMON/
MMNL
v$active_session_history
wrh$_active_session_history
Only writes out 1 in 10 rows
via direct path inserts
53. #.53
Copyright 2006 Kyle Hailey
Select reads
backwards
-No latching
-No read consistency
-Index on time Insert point
ASH buffer
Insert one direction
- Touch up wait times
54. #.54
Copyright 2006 Kyle Hailey
Family of ASH Tables
v$session_wait
v$active_session_history
wrh$active_session_history
V$session_wait_history
DBA_HIST_ACTIVE_SESS_HISTORY
56. #.56
Copyright 2006 Kyle Hailey
Wait Time vs Time Waited
SESSION_STATE
Waiting, on CPU
Based on WAIT_TIME
WAIT_TIME (v$session, v$session_wait, v$ash)
0 => waiting
>0 => CPU (value is time of last wait)
TIME_WAITED
Actual time waited for event
0 until wait finishes
Fix up values (no one else can do this)
57. #.57
Copyright 2006 Kyle Hailey
Oradebug
Dump to trace file
SQL> oradebug dump ash 5
SQL> Alter session set events ‘immediate
tracename ashdump level 5’;
level 5 = # of minutes
loader file rdbms/demo/ashldr.ctl
SQL> oradebug dump ashdump 5
SQL> Alter session set events ‘immediate
tracename ashdump level 5’;
58. #.58
Copyright 2006 Kyle Hailey
INIT.ORA
statistics_level = Typical (default)
PARAMETER SESSION_VALUE INSTANCE_VAL
----------------------- ---------- ------------
_ash_sampling_interval 1000 1000
milliseconds
_ash_size 1048618 1048618
ASH buffer size
_ash_enable TRUE TRUE
Turn on/off ASH sampling, flushing and the V$ views on ASH
_ash_disk_write_enable TRUE TRUE
Flush to disk
_ash_disk_filter_ratio 10 10
write 1 in 10 points
_ash_sample_all FALSE FALSE
Sample including idle waits
60. #.60
Copyright 2006 Kyle Hailey
10.2 Extras
10gR2 add fields to ASH
Blocking Session Id, serial# and state ***
XID, transaction ID
RAC event Fixup
Plan Hash Fix up
10gR1 - during parsing no plan, good way to find parsing
problems
10gR2 – get plan hash fixup – good but lose some ability to find
parsing problems
Wait Class, needed this for grouping
Force_matching_signature – sql profile id
64. #.64
Copyright 2006 Kyle Hailey
11g ASH extras
Run-time SQL row source information
Identifies current row source within plan
SQL execution ID
Is this same execution as last sample?
can see how long sql has been running
Operation bit vector
Capture non-timed operations
Examples: fetch, binding, close
Remote instance id for Cache transfers
Which instance sourced requested block?
65. #.65
Copyright 2006 Kyle Hailey
How Many Active Sessions?
How much data does ASH Collect ?
1 CPU means max 1 Avg Active Session unless
there is a bottleneck
Big site examples:
Oracle 4 way RAC internal apps
10,000 connected, 200 active
One Site
3000 connected, 30 Active
Site
12,000 connected, 100 active
66. #.66
Copyright 2006 Kyle Hailey
Simulating ASH
ASH is new in 10g
ASH data exist since V7
Need Diagnostic Pack License
Collect it yourself
SASH – Simulated ASH
http://perfvision.com/sash.php
Graphical Monitor
http://perfvision.com/ashmon.php
Consumes < 1% CPU for 10 active sessions (a lot)
67. #.67
Copyright 2006 Kyle Hailey
Sampling
Sampling is the future
Took clean vision to create
Bit of a leap of faith
Less Accurate but more powerful than past
Inaccuracies unimportant for the most part
Feasible since version 7
You can implement it yourself
Graphics are the only way to full harness the
power
68. #.68
Copyright 2006 Kyle Hailey
Summary
To solve hard problems you need Sampling
If you have 10g and a license you can use
V$active_session_history
OEM 10g (can access some 9 sampling)
?/rdbms/admim/ashrpt.sql
If you are on 7,8,9 or 10g without the license
Need custom sampling
SASH – Simulated ASH http://perfvision.com/ash.php
ASHMON – graphical http://perfvision.com/ashmon.php
69. #.69
Copyright 2006 Kyle Hailey
ASH Summary
Faster, Cheaper, instant (no lag) …
but most importantly
Rich & Multidimensional
Overview of system load
Drilldowns into problems
Sql
Session
Wait event
Details information on problem resolution
OEM simplifies mining of ASH data
Editor's Notes
Sampling is a leap in performance tuning technology. It took a clear vision of the future and a bit of a leap of faith to let go of the compulsive need to have exact measurements and instead embrace sampling a technology that statistically approximates the same value with little loss of accuracy but at the same time brings a much great wealth of information necessary for analyzing the performance of a database.
pmon timer
rdbms ipc message
wait for unread message on broadcast channel
wait for unread message on multiple broadcast channels
ges remote message
ges reconfiguration to start
gcs remote message
PX Deq: Par Recov Reply
PX Deq: Par Recov Execute
PX Deq: Par Recov Change Vector
wait for activate message
wakeup event for builder
wakeup event for preparer
wakeup event for reader
wait for transaction
parallel recovery coordinator waits for cleanup of slaves
smon timer
PX Deq: Txn Recovery Start
PX Deq: Txn Recovery Reply
statement suspended, wait error to be cleared
PX Deq: Index Merge Reply
PX Deq: Index Merge Execute
PX Deq: Index Merge Close
PX Deq: kdcph_mai
PX Deq: kdcphc_ack
virtual circuit status
dispatcher timer
jobq slave wait
pipe get
PX Deque wait
PX Idle Wait
PX Deq: Join ACK
PX Deq Credit: need buffer
PX Deq: Msg Fragment
PX Deq: Parse Reply
PX Deq: Execute Reply
PX Deq: Execution Msg
PX Deq: Table Q Normal
PX Deq: Table Q Sample
single-task message
SQL*Net message from client
SQL*Net message from dblink
PL/SQL lock timer
queue messages
wakeup time manager
AQ Proxy Cleanup Wait
Queue Monitor Wait
Queue Monitor Slave Wait
Queue Monitor Shutdown Wait
Queue Monitor IPC wait
STREAMS apply coord waiting for slave message
STREAMS fetch slave waiting for txns
STREAMS apply slave idle wait
STREAMS capture process filter callback wait for ruleset
waiting for subscribers to catch up
waiting for low memory condition to be resolved
HS message to agent
JS external job
WAIT_TIME is time of last wait, &gt; 0 , means on CPU
TIME_WAITED is the actual wait time
Select
session_id, session_serial# ,
count(*) ,
round((count(*)*100)/(5*60),0) &quot;%&quot;,
lpad(&apos;*&apos;,round((count(*)*10)/(5*60),0),&apos;*&apos;) &quot;Bar&quot;
from
v$active_session_history
where
session_state= &apos;ON CPU&apos; and
SAMPLE_TIME &gt; sysdate - (5/(24*60))
group by
session_id, session_serial#
order by
count(*) desc
/
Graphics – humans can process 400,000 times as much data graphically as textually
Think of a map of the US and all the counties with birth rates colored from low to high.
It’s a massive about of information textually but it can all be seen on one page of a map
Select
session_id, session_serial# ,
count(*) ,
round((count(*)*100)/(5*60),0) &quot;%&quot;,
lpad(&apos;*&apos;,round((count(*)*10)/(5*60),0),&apos;*&apos;) &quot;Bar&quot;
from
v$active_session_history
where
session_state= &apos;ON CPU&apos; and
SAMPLE_TIME &gt; sysdate - (5/(24*60))
group by
session_id, session_serial#
order by
count(*) desc
/
Size of Ash
--- Algorithm used to estimate ASH buffers sizememory_quota = max(2% of sga_target, 5% of shared_pool_size); /* sga_target = 0 when AUTO SGA is OFF */cpu_quota = 2MB * (# of CPUs);ash_size = min( cpu_quota, memory_quota );ash_size = max( 1MB, ash_size); /* atleast 1MB */ash_size = min( 128MB, ash_size); /* atmost 128MB */---
ASH
_ash_enable = false; [ A dynamic parameter will turn off ASH sampling, flushing and the V$ views on ASH ]
ADDM
_addm_auto_enable = false; [ A dynamic parameter to turn off automatic ADDM runs after every AWR snapshot ]
AWR
&quot;_swrf_mmon_flush&quot; = FALSE ; AWR
metrics
&quot;_swrf_mmon_metrics&quot; = FALSE ; METRICS
DB Feature Usage
&quot;_swrf_mmon_dbfus&quot; = FALSE ; DB Feature Usage
&quot;_swrf_on_disk_enabled&quot; = FALSE ; disable all (on disk, including manual) AWR operations:
set linesize 120
col entry_package for a25
col entry_procedure for a25
col cur_package for a25
col cur_procedure for a25
col calling_code for a70
select
procs1.object_name || decode(procs1.procedure_name,&apos;&apos;,&apos;&apos;,&apos;.&apos;)||
procs1.procedure_name ||&apos; &apos;||
decode(procs2.object_name,procs1.object_name,&apos;&apos;,procs2.object_name) ||
decode(procs2.procedure_name,procs1.procedure_name,&apos;&apos;,
decode(procs2.procedure_name,&apos;&apos;,&apos;&apos;,null,&apos;&apos;,&apos;.&apos;)||procs2.procedure_name)
&quot;calling_code&quot;,
sql_id,
count(*)
from v$active_session_history ash,
all_procedures procs1,
all_procedures procs2
where
ash.PLSQL_ENTRY_OBJECT_ID = procs1.object_id (+)
and ash.PLSQL_ENTRY_SUBPROGRAM_ID = procs1.SUBPROGRAM_ID (+)
and ash.PLSQL_OBJECT_ID = procs2.object_id (+)
and ash.PLSQL_SUBPROGRAM_ID = procs2.SUBPROGRAM_ID (+)
and ash.sample_time &gt; sysdate - &minutes/(60*24)
group by procs1.object_name, procs1.procedure_name,
procs2.object_name, procs2.procedure_name,sql_id
order by count(*)
/
ASH is a leap in performance tuning technology. It took a clear vision of the future and a bit of a leap of faith to let go of the compulsive need to have exact measurements and instead embrace sampling a technology that statistically approximates the same value with little loss of accuracy but at the same time brings a much great wealth of information necessary for analyzing the performance of a database.
ASH can be simulated on any version of Oracle since v7 with the simple act of repeatedly sampling and storing the values from v$session_wait into a historic table. Although the ideas is easy, the implementation turns out to be fairly complex. Luckily Oracle implements not only ASH in version 10g but more importantly Oracle implements a diagnostic package that reads the data, analyzes the data and report the findings and solutions automatically so you don’t even need to know all the esoteric wait analysis. The best part of all is OEM 10g exposes all the data and analysis graphically provide an interface that communicates quickly and clearly the date of the system.
