hitchhikers guide to free
Oracle tuning tools
a quick guide
Björn Rost
© 2015 Pythian Confidential2
• Consultant
– Oracle Database HA, Tuning, etc
– Linux&Solaris automation
• Oracle ACE Director
• (ex) president RAC SIG
• organizer OTN EMEA tour
© 2016 Pythian3
ABOUT ME
ABOUT PYTHIAN
Pythian’s 400+ IT professionals
help companies adopt
and manage disruptive
technologies to better compete
© 2016 Pythian. Confidential 4
© 2016 Pythian. Confidential 5
Systems currently
managed by Pythian
EXPERIENCED
Pythian experts
in 35 countries
GLOBAL
Millennia of experience
gathered and shared over
19 years
EXPERTS
11,800 2400
© 2016 Pythian. Confidential 6
TECHNICAL EXPERTISE
CLOUD
ADVANCED
ANALYTICS
DATABASES
BIG DATA
DEVOPS
Using the disruptive nature
of cloud for accelerated,
cost-effective growth
INFRASTRUCTURE
MANAGEMENT
Mining data for insights &
business transformation
using data science
Ensuring databases are
reliable, secure, available
and continuously optimized
Harnessing the
transformative power of
data on a massive scale
Providing critical velocity in
software deployment by
adopting DevOps practices
Transforming and managing
the IT infrastructure that
supports the business
WHY ORACLE?
7
• basic operations same
between DBs
• instrumentation makes the
difference
• allows to deeply investigate
and explain issues
© 2016 Pythian
(WRONG) TUNING CYCLE
© 2016 Pythian8
guess
change
hope
DON’T BE A STORMTROOPER DBA
© 2016 Pythian9
TUNING CYCLE
© 2016 Pythian10
understand
change
measure
TUNING IS SIMPLE, REALLY
11 © 2016 Pythian
LAST WEEK’S EXAMPLE
• increasing gap in standby db
– change max_archive_processes parameter
– change some other parameters
– try rman recovery vs MRP
– increase parallelism
– reduce parallelism
– try more silver bullets…
– have SRE look into IO issues
– have SRE look at CPU issues
12 © 2016 Pythian
guess
change
hope
LAST WEEK’S EXAMPLE
• select event from v$session
where program like '%MRP%’
– PX deq credit: send blkd
– PX deq credit: need buffer
• search for that on MOS
– 10.2 Parallel Standby Recovery Process Is Very Slow (Doc ID 420337.1)
• PARALLEL_EXECUTION_MESSAGE_SIZE=16k
13 © 2016 Pythian
understand
change
measure
TYPES
14
report real-time historic drill-down
© 2016 Pythian
ENTERPRISE MANAGER
• incredibly useful and thought through
• from vendor, perfectly integrated
• many common views and methods
– coloured AAS view
– Top-SQL view
– Real-Time SQL monitor
– blocking session tree
– AWR Reports
– ADDM and other advisors
16 © 2016 Pythian
ENTERPRISE MANAGER
© 2015 Pythian Confidential17
SO WHY USE CLI TOOLS?
• not licensed for Diag&Tuning
– Standard Edition
– EE without Diag&Tuning (a cardinal sin)
• very low throughput or no GUI access
• EM not setup/installed
– 12c EM express can get the job done
• you simply want to look like a nerd
18 © 2016 Pythian
WHY OPEN SOURCE
• see and learn
– you can actually look at the code you are running
• collaborate and contribute
– want a new feature? implement yourself!
• free(*) means
– no costs (duh)
– get up and running really fast
– don’t worry about audits
19 © 2016 Pythian
© 2015 Pythian Confidential20
RLSQLPLUS
• sqlplus on steroids
• edit sqlplus CLI
• history
– up/down arrow
– search with CTRL-r
• auto-completion
– against custom wordlist
21 © 2016 Pythian
RLSQLPLUS INSTALLATION STEP1
22
[root@kickpuppet ~]# yum install http://dl.fedoraproject.org/pub/epel/6/x86_64/rlwrap-0.42-1.el6.x86_64.rpm
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
rlwrap x86_64 0.41-1.el6 epel 92 k
Transaction Summary
================================================================================
Install 1 Package(s)
Installing : rlwrap-0.41-1.el6.x86_64 1/1
Verifying : rlwrap-0.41-1.el6.x86_64 1/1
Installed:
rlwrap.x86_64 0:0.41-1.el6
Complete!
© 2016 Pythian
RLSQLPLUS INSTALLATION STEP2
• optional: download or generate wordfile
– for autocompletion
23
$ curl -o wordfile.txt
http://blog.tanelpoder.com/files/scripts/setup/wordfile_11gR2.txt
http://blog.tanelpoder.com/files/scripts/setup/
© 2016 Pythian
RLSQLPLUS INSTALLATION STEP 3
• create aliases around sqlplus
– and rman
– and dgmgrl
• avoid overriding the original name
24
alias rlsqlplus='rlwrap -D2 -irc -b'''"@(){}[],+=&^%#;|''' -f ~/wordfile.txt
sqlplus'
alias rlrman='rlwrap -D2 -irc -b'''"@(){}[],+=&^%#;|''' rman'
© 2016 Pythian
SQLCL
• sqlplus “replacement” from SQLdeveloper team
– history and editor
– auto-completion
– auto-formatting
• ansiconsole for col widths
• json, xml, csv etc
– tons of good stuff
25 © 2016 Pythian
© 2015 Pythian Confidential26
AAS VIEW
• my favourite real-time DB dashboard
• one-glance DB activity overview
• colour-coded
– red -> blocking, app, concurrency
– blue -> IO, missing indexes, etc
– green -> CPU, executing too much?
– other -> weird, take a look
27 © 2016 Pythian
AAS VIEW
28 © 2016 Pythian
WEB-ASH
29
• simple and free
• reads v$session by default
• stores data in plain files
• lean javascript to display
http://datavirtualizer.com/web-ash-w-ash/
© 2016 Pythian
MOATS
• top-like output that refreshes
• excellent overview
• needs small installation on database
• samples v$-views while running
• improved forks
– MOATS 2.0
– SQL Dashboard
30 © 2016 Pythian
MOATS OUTPUT
31
MOATS: The Mother Of All Tuning Scripts v1.0 by Adrian Billington & Tanel Poder
http://www.oracle-developer.net & http://www.e2sn.com
+ INSTANCE SUMMARY ------------------------------------------------------------------------------------------+
| Instance: KICKPUPPET | Execs/s: 581.6 | sParse/s: 60.0 | LIOs/s: 4989.5 | Read MB/s: 0.7 |
| Cur Time: 19-Nov 21:11:09 | Calls/s: 164.2 | hParse/s: 0.0 | PhyRD/s: 86.7 | Write MB/s: 1.6 |
| History: 0h 9m 45s | Commits/s: 50.3 | ccHits/s: 578.8 | PhyWR/s: 112.7 | Redo MB/s: 0.2 |
+------------------------------------------------------------------------------------------------------------+
+ TOP SQL_ID (child#) -----+ TOP SESSIONS ---------+ + TOP WAITS -------------------------+ WAIT CLASS -+
| 29% | 7t0959msvyt5g (0) | 78,65 | | 29% | db file sequential read | User I/O |
| 14% | () | | | 29% | ON CPU | ON CPU |
| 14% | 147a57cxq3w5y (0) | 70 | | 14% | oracle thread bootstrap | Other |
| 14% | gh2g2tynpcpv1 (0) | 65 | | 14% | resmgr:cpu quantum | Scheduler |
| 14% | 4phvdvx32a3mf (0) | 73 | | 14% | db file parallel read | User I/O |
+--------------------------------------------------+ +--------------------------------------------------+
+ TOP SQL_ID ----+ PLAN_HASH_VALUE + SQL TEXT ---------------------------------------------------------------+
| 7t0959msvyt5g | 856749079 | SELECT ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORD |
| | | ER_TOTAL, SALES_REP_ID, PROMOTION_ID, WAREHOUSE_ID, DELIVERY_TYPE, COST |
+ ---------------------------------------------------------------------------------------------------------- +
| 147a57cxq3w5y | 0 | BEGIN :1 := orderentry.browseproducts(:2 ,:3 ,:4 ); END; |
+ ---------------------------------------------------------------------------------------------------------- +
| gh2g2tynpcpv1 | 0 | INSERT INTO CUSTOMERS ( CUSTOMER_ID , CUST_FIRST_NAME , CUST_LAST_NAME |
| | | , NLS_LANGUAGE , NLS_TERRITORY , CREDIT_LIMIT , CUST_EMAIL , ACCOUNT_MG |
+ ---------------------------------------------------------------------------------------------------------- +
| 4phvdvx32a3mf | 0 | begin prvt_ilm.stopjobs(-1,true,true,:1); end; |
+ ---------------------------------------------------------------------------------------------------------- +
© 2016 Pythian
MOATS 2.0 OUTPUT
32 © 2016 Pythian
SNAPPER
• realtime session monitor and drill-downs
• just one sqlscript to run
– no installation needed
• runs for sample period
– “snaps” v$session etc
– reports on diffs
• two modes
– “ash” for v$session
– stats for v$sesstat
33
http://tech.e2sn.com/oracle-scripts-and-tools/session-snapper
© 2016 Pythian
SNAPPER ARGUMENTS
• snapper ash 5 1 all
– mode can be ash, stat or custom column list
– seconds to run/sample
– number of runs
– “all” or filter on v$session columns
• sid=42
• username=brost
34 © 2016 Pythian
SNAPPER DEFAULT MODE
35
SQL> @snapper ash 5 1 all
Sampling SID all with interval 5 seconds, taking 1 snapshots...
-- Session Snapper v4.15 - by Tanel Poder ( http://blog.tanelpoder.com )
-- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet! :)
----------------------------------------------------------------------------------------------------
Active% | INST | SQL_ID | SQL_CHILD | EVENT | WAIT_CLASS
----------------------------------------------------------------------------------------------------
10% | 1 | | 0 | db file async I/O submit | System I/O
10% | 1 | | | log file sync | Commit
7% | 1 | 5ckxyqfvu60pj | 0 | db file sequential read | User I/O
7% | 1 | | 0 | log file parallel write | System I/O
5% | 1 | | | ON CPU | ON CPU
5% | 1 | 3fw75k1snsddx | 0 | db file sequential read | User I/O
5% | 1 | | 0 | ON CPU | ON CPU
5% | 1 | c13sma6rkr27c | 0 | db file parallel read | User I/O
5% | 1 | gzhkw1qu6fwxm | 0 | ON CPU | ON CPU
5% | 1 | 0w2qpuc6u2zsp | 0 | ON CPU | ON CPU
-- End of ASH snap 1, end=2016-04-24 15:24:37, seconds=5, samples_taken=42
© 2016 Pythian
AAS IN SNAPPER
36
SQL> @snapper ash=wait_class 5 1 all
Sampling SID all with interval 5 seconds, taking 1 snapshots...
-- Session Snapper v4.11 - by Tanel Poder ( http://blog.tanelpoder.com )
-- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet! 
-------------------------
Active% | WAIT_CLASS
-------------------------
44% | System I/O
19% | Commit
7% | Scheduler
7% | ON CPU
5% | User I/O
-- End of ASH snap 1, end=2014-11-19 16:23:06, seconds=5, samples_taken=43
© 2016 Pythian
SNAPPER STATS MODE
37
SQL> @snapper stats,gather=tw 15 1 sid=70
Sampling SID sid=70 with interval 15 seconds, taking 1 snapshots...
-- Session Snapper v4.11 BETA - by Tanel Poder ( http://blog.tanelpoder.com ) - Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet!
:)
---------------------------------------------------------------------------------------------------------------------------------------------------
SID, USERNAME , TYPE, STATISTIC , DELTA, HDELTA/SEC, %TIME, GRAPH , NUM_WAITS, WAITS/SEC, AVERAGES
---------------------------------------------------------------------------------------------------------------------------------------------------
70, SOE , TIME, repeated bind elapsed time , 300, 19.8us, .0%, [ ], , ,
70, SOE , TIME, parse time elapsed , 9272, 611.99us, .1%, [ ], , ,
70, SOE , TIME, PL/SQL execution elapsed time, 230999, 15.25ms, 1.5%, [# ], , ,
70, SOE , TIME, DB CPU , 569000, 37.56ms, 3.8%, [@ ], , ,
70, SOE , TIME, sql execute elapsed time , 1730975, 114.25ms, 11.4%, [## ], , ,
70, SOE , TIME, DB time , 2038543, 134.55ms, 13.5%, [## ], , , .88 % unaccounted
70, SOE , WAIT, log file sync , 235408, 15.54ms, 1.6%, [W ], 183, 12.08, 1.29ms average
70, SOE , WAIT, db file sequential read , 1123224, 74.14ms, 7.4%, [W ], 397, 26.2, 2.83ms average
70, SOE , WAIT, db file parallel read , 52763, 3.48ms, .3%, [ ], 3, .2, 17.59ms average
70, SOE , WAIT, latch: In memory undo latch , 499, 32.94us, .0%, [ ], 1, .07, 499us average
70, SOE , WAIT, resmgr:cpu quantum , 51794, 3.42ms, .3%, [ ], 3, .2, 17.26ms average
70, SOE , WAIT, SQL*Net message to client , 1458, 96.23us, .0%, [ ], 393, 25.94, 3.71us average
70, SOE , WAIT, SQL*Net message from client , 60177, 3.97ms, .4%, [ ], 393, 25.94, 153.12us average
70, SOE , WAIT, PL/SQL lock timer , 12918348, 852.66ms, 85.3%, [WWWWWWWWW ], 1328, 87.65, 9.73ms average
70, SOE , WAIT, events in waitclass Other , 960, 63.36us, .0%, [ ], 2, .13, 480us average
-- End of Stats snap 1, end=2014-11-20 08:04:39, seconds=15.2
© 2016 Pythian
TOP SQL
38
SQL> @snapper ash=sql_id+wait_class 15 1 user=soe
Sampling SID user=soe with interval 15 seconds, taking 1 snapshots...
-- Session Snapper v4.11 BETA - by Tanel Poder ( http://blog.tanelpoder.com )
-- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet! :)
-------------------------------------------
Active% | SQL_ID | WAIT_CLASS
-------------------------------------------
14% | | Commit
5% | 7hk2m2702ua0g | User I/O
4% | 5ckxyqfvu60pj | User I/O
3% | 7t0959msvyt5g | User I/O
3% | gh2g2tynpcpv1 | User I/O
2% | 0w2qpuc6u2zsp | Scheduler
2% | g81cbrq5yamf5 | User I/O
2% | 7ws837zynp1zv | User I/O
2% | f7rxuxzt64k87 | User I/O
2% | 1qf3b7a46jm3u | User I/O
-- End of ASH snap 1, end=2014-11-20 06:14:42, seconds=15, samples_taken=100
© 2016 Pythian
SQLDEVELOPER
39 © 2016 Pythian
SQLDEVELOPER
40 © 2016 Pythian
HISTORIC PERFORMANCE
• moats, snapper etc only sample when running
• EE (+diag/tuning) has ASH and AWR
41 © 2016 Pythian
AWR REPORTS -> STATSPACK
• good for a high-level overview
• AWR has a bit more “stuff” than statspack
• AWR is setup automatically (60min snaps)
– be mindful of averages
• but AWR needs EE and diag&tuning pack
42 © 2016 Pythian
STATSPACK IN 12C
• no problem in non-CDB
• with multitenant would make most sense in CDB
– but not really supported due to common user c#...
• there is a nasty and unsupported workaround
43
https://jonathanlewis.wordpress.com/2013/07/04/12c-statspack-hack/
© 2016 Pythian
ORASASH
• does pretty much what ASH does (but for free)
• sample and store information from v$-views
• mostly compatible with EE ASH
– ashmon
– ashmasters etc
• stores data from multiple sources in repo db
– minimal footprint in source, only view and dblink
44
http://pioro.github.io/orasash/
© 2016 Pythian
SQLT
• oracle supported scripts
• needs a schema an installation in db
– (this is annoying)
• will gather *everything* about a single SQL
– option to use diag&tuning
• output is one zipped html archive
45
Metalink Doc id 215187.1
© 2016 Pythian
SQLD360
• sql scope (just like SQLT)
• needs no installation
• one zipfile including html output
46
http://mauro-pagano.com/2015/02/16/sqld360-sql-diagnostics-collection-made-faster/
© 2016 Pythian
SQLD SCREENSHOTS
47 © 2016 Pythian
SQLD SCREENSHOTS
48 © 2016 Pythian
SQLD INCLUDES SQL
49
SELECT MIN(a.snap_id) snap_id,
TO_CHAR(a.begin_interval_time, 'YYYY-MM-DD HH24:MI') begin_time,
TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI') end_time,
NVL(TRUNC(SUM(b.elapsed_time_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) elapsed_time,
NVL(TRUNC(SUM(b.cpu_time_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) cpu_time,
NVL(TRUNC(SUM(b.iowait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) iowait,
NVL(TRUNC(SUM(b.clwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) clwait,
NVL(TRUNC(SUM(b.apwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) apwait,
NVL(TRUNC(SUM(b.ccwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) ccwait
FROM (SELECT snap_id, instance_number, elapsed_time_total, cpu_time_total,
iowait_total, clwait_total, apwait_total, ccwait_total, executions_total
FROM dba_hist_sqlstat
WHERE sql_id = 'gm9dkz4u1hwfv') b,
(SELECT snap_id, instance_number, begin_interval_time, end_interval_time
FROM dba_hist_snapshot
WHERE snap_id BETWEEN 0 AND 166142) a
WHERE a.snap_id = b.snap_id(+)
AND a.instance_number = b.instance_number(+)
AND 'Y' = 'Y'
AND a.instance_number = a.instance_number
GROUP BY
TO_CHAR(a.begin_interval_time, 'YYYY-MM-DD HH24:MI'),
TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI')
ORDER BY
TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI');
1850 rows selected.
© 2016 Pythian
EDB360
• database scope
• no installation needed!
– just run a sqlfile
• creates one zipfile with html for offline viewing
– also AWR etc
• T | D | N control for access packs
• convenient: graphs
50
http://www.enkitec.com/products/edb360
© 2016 Pythian
EDB SCREENSHOTS
51 © 2016 Pythian
TUNAS360
• best of sqld and snapper
• samples v$-views for period
– works without AWR/ASH license!
• produces output like sqld
52 © 2016 Pythian
TUNAS SCREENSHOTS
53 © 2016 Pythian
TUNAS SCREENSHOTS
54 © 2016 Pythian
GOOD OLD 10046 TRACE
• guaranteed to catch everything (unlike sampling)
– why not turn on instance-wide?
• overhead vs usefulness?
• no license needed at all
55 © 2016 Pythian
READING TRACEFILES
• sqldeveloper
• trcanlzr
– needs installing but now also part of SQLT
• Kyle Hailey’s script
– https://github.com/khailey/oracle_trace_parsing/
• Chris Antognini’s script
– https://antognini.ch/downloads/top2/chapter03/tvdxtat
_40beta10_20140630.zip
56 © 2016 Pythian
REVIEW
• make sqlplus great
again
– rlsqlplus
– sqlcl
• real-time analysis
– snapper
– moats
– web-ash
– sqldeveloper instance
viewer
– TUNAs360
57
• historical analysis
– Orasash / S-ASH
– statspack
– SQLT
– eDB360
– sqld360
• 10046 tracing
– raw
– various analyzers
© 2016 Pythian
59
@brost
http://www.pythian.com/blog
rost@pythian.com
© 2016 Pythian

Hitchhiker's Guide to free Oracle tuning tools

  • 1.
    hitchhikers guide tofree Oracle tuning tools a quick guide Björn Rost
  • 2.
    © 2015 PythianConfidential2
  • 3.
    • Consultant – OracleDatabase HA, Tuning, etc – Linux&Solaris automation • Oracle ACE Director • (ex) president RAC SIG • organizer OTN EMEA tour © 2016 Pythian3 ABOUT ME
  • 4.
    ABOUT PYTHIAN Pythian’s 400+IT professionals help companies adopt and manage disruptive technologies to better compete © 2016 Pythian. Confidential 4
  • 5.
    © 2016 Pythian.Confidential 5 Systems currently managed by Pythian EXPERIENCED Pythian experts in 35 countries GLOBAL Millennia of experience gathered and shared over 19 years EXPERTS 11,800 2400
  • 6.
    © 2016 Pythian.Confidential 6 TECHNICAL EXPERTISE CLOUD ADVANCED ANALYTICS DATABASES BIG DATA DEVOPS Using the disruptive nature of cloud for accelerated, cost-effective growth INFRASTRUCTURE MANAGEMENT Mining data for insights & business transformation using data science Ensuring databases are reliable, secure, available and continuously optimized Harnessing the transformative power of data on a massive scale Providing critical velocity in software deployment by adopting DevOps practices Transforming and managing the IT infrastructure that supports the business
  • 7.
    WHY ORACLE? 7 • basicoperations same between DBs • instrumentation makes the difference • allows to deeply investigate and explain issues © 2016 Pythian
  • 8.
    (WRONG) TUNING CYCLE ©2016 Pythian8 guess change hope
  • 9.
    DON’T BE ASTORMTROOPER DBA © 2016 Pythian9
  • 10.
    TUNING CYCLE © 2016Pythian10 understand change measure
  • 11.
    TUNING IS SIMPLE,REALLY 11 © 2016 Pythian
  • 12.
    LAST WEEK’S EXAMPLE •increasing gap in standby db – change max_archive_processes parameter – change some other parameters – try rman recovery vs MRP – increase parallelism – reduce parallelism – try more silver bullets… – have SRE look into IO issues – have SRE look at CPU issues 12 © 2016 Pythian guess change hope
  • 13.
    LAST WEEK’S EXAMPLE •select event from v$session where program like '%MRP%’ – PX deq credit: send blkd – PX deq credit: need buffer • search for that on MOS – 10.2 Parallel Standby Recovery Process Is Very Slow (Doc ID 420337.1) • PARALLEL_EXECUTION_MESSAGE_SIZE=16k 13 © 2016 Pythian understand change measure
  • 14.
    TYPES 14 report real-time historicdrill-down © 2016 Pythian
  • 15.
    ENTERPRISE MANAGER • incrediblyuseful and thought through • from vendor, perfectly integrated • many common views and methods – coloured AAS view – Top-SQL view – Real-Time SQL monitor – blocking session tree – AWR Reports – ADDM and other advisors 16 © 2016 Pythian
  • 16.
    ENTERPRISE MANAGER © 2015Pythian Confidential17
  • 17.
    SO WHY USECLI TOOLS? • not licensed for Diag&Tuning – Standard Edition – EE without Diag&Tuning (a cardinal sin) • very low throughput or no GUI access • EM not setup/installed – 12c EM express can get the job done • you simply want to look like a nerd 18 © 2016 Pythian
  • 18.
    WHY OPEN SOURCE •see and learn – you can actually look at the code you are running • collaborate and contribute – want a new feature? implement yourself! • free(*) means – no costs (duh) – get up and running really fast – don’t worry about audits 19 © 2016 Pythian
  • 19.
    © 2015 PythianConfidential20
  • 20.
    RLSQLPLUS • sqlplus onsteroids • edit sqlplus CLI • history – up/down arrow – search with CTRL-r • auto-completion – against custom wordlist 21 © 2016 Pythian
  • 21.
    RLSQLPLUS INSTALLATION STEP1 22 [root@kickpuppet~]# yum install http://dl.fedoraproject.org/pub/epel/6/x86_64/rlwrap-0.42-1.el6.x86_64.rpm ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: rlwrap x86_64 0.41-1.el6 epel 92 k Transaction Summary ================================================================================ Install 1 Package(s) Installing : rlwrap-0.41-1.el6.x86_64 1/1 Verifying : rlwrap-0.41-1.el6.x86_64 1/1 Installed: rlwrap.x86_64 0:0.41-1.el6 Complete! © 2016 Pythian
  • 22.
    RLSQLPLUS INSTALLATION STEP2 •optional: download or generate wordfile – for autocompletion 23 $ curl -o wordfile.txt http://blog.tanelpoder.com/files/scripts/setup/wordfile_11gR2.txt http://blog.tanelpoder.com/files/scripts/setup/ © 2016 Pythian
  • 23.
    RLSQLPLUS INSTALLATION STEP3 • create aliases around sqlplus – and rman – and dgmgrl • avoid overriding the original name 24 alias rlsqlplus='rlwrap -D2 -irc -b'''"@(){}[],+=&^%#;|''' -f ~/wordfile.txt sqlplus' alias rlrman='rlwrap -D2 -irc -b'''"@(){}[],+=&^%#;|''' rman' © 2016 Pythian
  • 24.
    SQLCL • sqlplus “replacement”from SQLdeveloper team – history and editor – auto-completion – auto-formatting • ansiconsole for col widths • json, xml, csv etc – tons of good stuff 25 © 2016 Pythian
  • 25.
    © 2015 PythianConfidential26
  • 26.
    AAS VIEW • myfavourite real-time DB dashboard • one-glance DB activity overview • colour-coded – red -> blocking, app, concurrency – blue -> IO, missing indexes, etc – green -> CPU, executing too much? – other -> weird, take a look 27 © 2016 Pythian
  • 27.
    AAS VIEW 28 ©2016 Pythian
  • 28.
    WEB-ASH 29 • simple andfree • reads v$session by default • stores data in plain files • lean javascript to display http://datavirtualizer.com/web-ash-w-ash/ © 2016 Pythian
  • 29.
    MOATS • top-like outputthat refreshes • excellent overview • needs small installation on database • samples v$-views while running • improved forks – MOATS 2.0 – SQL Dashboard 30 © 2016 Pythian
  • 30.
    MOATS OUTPUT 31 MOATS: TheMother Of All Tuning Scripts v1.0 by Adrian Billington & Tanel Poder http://www.oracle-developer.net & http://www.e2sn.com + INSTANCE SUMMARY ------------------------------------------------------------------------------------------+ | Instance: KICKPUPPET | Execs/s: 581.6 | sParse/s: 60.0 | LIOs/s: 4989.5 | Read MB/s: 0.7 | | Cur Time: 19-Nov 21:11:09 | Calls/s: 164.2 | hParse/s: 0.0 | PhyRD/s: 86.7 | Write MB/s: 1.6 | | History: 0h 9m 45s | Commits/s: 50.3 | ccHits/s: 578.8 | PhyWR/s: 112.7 | Redo MB/s: 0.2 | +------------------------------------------------------------------------------------------------------------+ + TOP SQL_ID (child#) -----+ TOP SESSIONS ---------+ + TOP WAITS -------------------------+ WAIT CLASS -+ | 29% | 7t0959msvyt5g (0) | 78,65 | | 29% | db file sequential read | User I/O | | 14% | () | | | 29% | ON CPU | ON CPU | | 14% | 147a57cxq3w5y (0) | 70 | | 14% | oracle thread bootstrap | Other | | 14% | gh2g2tynpcpv1 (0) | 65 | | 14% | resmgr:cpu quantum | Scheduler | | 14% | 4phvdvx32a3mf (0) | 73 | | 14% | db file parallel read | User I/O | +--------------------------------------------------+ +--------------------------------------------------+ + TOP SQL_ID ----+ PLAN_HASH_VALUE + SQL TEXT ---------------------------------------------------------------+ | 7t0959msvyt5g | 856749079 | SELECT ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORD | | | | ER_TOTAL, SALES_REP_ID, PROMOTION_ID, WAREHOUSE_ID, DELIVERY_TYPE, COST | + ---------------------------------------------------------------------------------------------------------- + | 147a57cxq3w5y | 0 | BEGIN :1 := orderentry.browseproducts(:2 ,:3 ,:4 ); END; | + ---------------------------------------------------------------------------------------------------------- + | gh2g2tynpcpv1 | 0 | INSERT INTO CUSTOMERS ( CUSTOMER_ID , CUST_FIRST_NAME , CUST_LAST_NAME | | | | , NLS_LANGUAGE , NLS_TERRITORY , CREDIT_LIMIT , CUST_EMAIL , ACCOUNT_MG | + ---------------------------------------------------------------------------------------------------------- + | 4phvdvx32a3mf | 0 | begin prvt_ilm.stopjobs(-1,true,true,:1); end; | + ---------------------------------------------------------------------------------------------------------- + © 2016 Pythian
  • 31.
    MOATS 2.0 OUTPUT 32© 2016 Pythian
  • 32.
    SNAPPER • realtime sessionmonitor and drill-downs • just one sqlscript to run – no installation needed • runs for sample period – “snaps” v$session etc – reports on diffs • two modes – “ash” for v$session – stats for v$sesstat 33 http://tech.e2sn.com/oracle-scripts-and-tools/session-snapper © 2016 Pythian
  • 33.
    SNAPPER ARGUMENTS • snapperash 5 1 all – mode can be ash, stat or custom column list – seconds to run/sample – number of runs – “all” or filter on v$session columns • sid=42 • username=brost 34 © 2016 Pythian
  • 34.
    SNAPPER DEFAULT MODE 35 SQL>@snapper ash 5 1 all Sampling SID all with interval 5 seconds, taking 1 snapshots... -- Session Snapper v4.15 - by Tanel Poder ( http://blog.tanelpoder.com ) -- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet! :) ---------------------------------------------------------------------------------------------------- Active% | INST | SQL_ID | SQL_CHILD | EVENT | WAIT_CLASS ---------------------------------------------------------------------------------------------------- 10% | 1 | | 0 | db file async I/O submit | System I/O 10% | 1 | | | log file sync | Commit 7% | 1 | 5ckxyqfvu60pj | 0 | db file sequential read | User I/O 7% | 1 | | 0 | log file parallel write | System I/O 5% | 1 | | | ON CPU | ON CPU 5% | 1 | 3fw75k1snsddx | 0 | db file sequential read | User I/O 5% | 1 | | 0 | ON CPU | ON CPU 5% | 1 | c13sma6rkr27c | 0 | db file parallel read | User I/O 5% | 1 | gzhkw1qu6fwxm | 0 | ON CPU | ON CPU 5% | 1 | 0w2qpuc6u2zsp | 0 | ON CPU | ON CPU -- End of ASH snap 1, end=2016-04-24 15:24:37, seconds=5, samples_taken=42 © 2016 Pythian
  • 35.
    AAS IN SNAPPER 36 SQL>@snapper ash=wait_class 5 1 all Sampling SID all with interval 5 seconds, taking 1 snapshots... -- Session Snapper v4.11 - by Tanel Poder ( http://blog.tanelpoder.com ) -- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet!  ------------------------- Active% | WAIT_CLASS ------------------------- 44% | System I/O 19% | Commit 7% | Scheduler 7% | ON CPU 5% | User I/O -- End of ASH snap 1, end=2014-11-19 16:23:06, seconds=5, samples_taken=43 © 2016 Pythian
  • 36.
    SNAPPER STATS MODE 37 SQL>@snapper stats,gather=tw 15 1 sid=70 Sampling SID sid=70 with interval 15 seconds, taking 1 snapshots... -- Session Snapper v4.11 BETA - by Tanel Poder ( http://blog.tanelpoder.com ) - Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet! :) --------------------------------------------------------------------------------------------------------------------------------------------------- SID, USERNAME , TYPE, STATISTIC , DELTA, HDELTA/SEC, %TIME, GRAPH , NUM_WAITS, WAITS/SEC, AVERAGES --------------------------------------------------------------------------------------------------------------------------------------------------- 70, SOE , TIME, repeated bind elapsed time , 300, 19.8us, .0%, [ ], , , 70, SOE , TIME, parse time elapsed , 9272, 611.99us, .1%, [ ], , , 70, SOE , TIME, PL/SQL execution elapsed time, 230999, 15.25ms, 1.5%, [# ], , , 70, SOE , TIME, DB CPU , 569000, 37.56ms, 3.8%, [@ ], , , 70, SOE , TIME, sql execute elapsed time , 1730975, 114.25ms, 11.4%, [## ], , , 70, SOE , TIME, DB time , 2038543, 134.55ms, 13.5%, [## ], , , .88 % unaccounted 70, SOE , WAIT, log file sync , 235408, 15.54ms, 1.6%, [W ], 183, 12.08, 1.29ms average 70, SOE , WAIT, db file sequential read , 1123224, 74.14ms, 7.4%, [W ], 397, 26.2, 2.83ms average 70, SOE , WAIT, db file parallel read , 52763, 3.48ms, .3%, [ ], 3, .2, 17.59ms average 70, SOE , WAIT, latch: In memory undo latch , 499, 32.94us, .0%, [ ], 1, .07, 499us average 70, SOE , WAIT, resmgr:cpu quantum , 51794, 3.42ms, .3%, [ ], 3, .2, 17.26ms average 70, SOE , WAIT, SQL*Net message to client , 1458, 96.23us, .0%, [ ], 393, 25.94, 3.71us average 70, SOE , WAIT, SQL*Net message from client , 60177, 3.97ms, .4%, [ ], 393, 25.94, 153.12us average 70, SOE , WAIT, PL/SQL lock timer , 12918348, 852.66ms, 85.3%, [WWWWWWWWW ], 1328, 87.65, 9.73ms average 70, SOE , WAIT, events in waitclass Other , 960, 63.36us, .0%, [ ], 2, .13, 480us average -- End of Stats snap 1, end=2014-11-20 08:04:39, seconds=15.2 © 2016 Pythian
  • 37.
    TOP SQL 38 SQL> @snapperash=sql_id+wait_class 15 1 user=soe Sampling SID user=soe with interval 15 seconds, taking 1 snapshots... -- Session Snapper v4.11 BETA - by Tanel Poder ( http://blog.tanelpoder.com ) -- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet! :) ------------------------------------------- Active% | SQL_ID | WAIT_CLASS ------------------------------------------- 14% | | Commit 5% | 7hk2m2702ua0g | User I/O 4% | 5ckxyqfvu60pj | User I/O 3% | 7t0959msvyt5g | User I/O 3% | gh2g2tynpcpv1 | User I/O 2% | 0w2qpuc6u2zsp | Scheduler 2% | g81cbrq5yamf5 | User I/O 2% | 7ws837zynp1zv | User I/O 2% | f7rxuxzt64k87 | User I/O 2% | 1qf3b7a46jm3u | User I/O -- End of ASH snap 1, end=2014-11-20 06:14:42, seconds=15, samples_taken=100 © 2016 Pythian
  • 38.
  • 39.
  • 40.
    HISTORIC PERFORMANCE • moats,snapper etc only sample when running • EE (+diag/tuning) has ASH and AWR 41 © 2016 Pythian
  • 41.
    AWR REPORTS ->STATSPACK • good for a high-level overview • AWR has a bit more “stuff” than statspack • AWR is setup automatically (60min snaps) – be mindful of averages • but AWR needs EE and diag&tuning pack 42 © 2016 Pythian
  • 42.
    STATSPACK IN 12C •no problem in non-CDB • with multitenant would make most sense in CDB – but not really supported due to common user c#... • there is a nasty and unsupported workaround 43 https://jonathanlewis.wordpress.com/2013/07/04/12c-statspack-hack/ © 2016 Pythian
  • 43.
    ORASASH • does prettymuch what ASH does (but for free) • sample and store information from v$-views • mostly compatible with EE ASH – ashmon – ashmasters etc • stores data from multiple sources in repo db – minimal footprint in source, only view and dblink 44 http://pioro.github.io/orasash/ © 2016 Pythian
  • 44.
    SQLT • oracle supportedscripts • needs a schema an installation in db – (this is annoying) • will gather *everything* about a single SQL – option to use diag&tuning • output is one zipped html archive 45 Metalink Doc id 215187.1 © 2016 Pythian
  • 45.
    SQLD360 • sql scope(just like SQLT) • needs no installation • one zipfile including html output 46 http://mauro-pagano.com/2015/02/16/sqld360-sql-diagnostics-collection-made-faster/ © 2016 Pythian
  • 46.
  • 47.
  • 48.
    SQLD INCLUDES SQL 49 SELECTMIN(a.snap_id) snap_id, TO_CHAR(a.begin_interval_time, 'YYYY-MM-DD HH24:MI') begin_time, TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI') end_time, NVL(TRUNC(SUM(b.elapsed_time_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) elapsed_time, NVL(TRUNC(SUM(b.cpu_time_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) cpu_time, NVL(TRUNC(SUM(b.iowait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) iowait, NVL(TRUNC(SUM(b.clwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) clwait, NVL(TRUNC(SUM(b.apwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) apwait, NVL(TRUNC(SUM(b.ccwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) ccwait FROM (SELECT snap_id, instance_number, elapsed_time_total, cpu_time_total, iowait_total, clwait_total, apwait_total, ccwait_total, executions_total FROM dba_hist_sqlstat WHERE sql_id = 'gm9dkz4u1hwfv') b, (SELECT snap_id, instance_number, begin_interval_time, end_interval_time FROM dba_hist_snapshot WHERE snap_id BETWEEN 0 AND 166142) a WHERE a.snap_id = b.snap_id(+) AND a.instance_number = b.instance_number(+) AND 'Y' = 'Y' AND a.instance_number = a.instance_number GROUP BY TO_CHAR(a.begin_interval_time, 'YYYY-MM-DD HH24:MI'), TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI') ORDER BY TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI'); 1850 rows selected. © 2016 Pythian
  • 49.
    EDB360 • database scope •no installation needed! – just run a sqlfile • creates one zipfile with html for offline viewing – also AWR etc • T | D | N control for access packs • convenient: graphs 50 http://www.enkitec.com/products/edb360 © 2016 Pythian
  • 50.
  • 51.
    TUNAS360 • best ofsqld and snapper • samples v$-views for period – works without AWR/ASH license! • produces output like sqld 52 © 2016 Pythian
  • 52.
  • 53.
  • 54.
    GOOD OLD 10046TRACE • guaranteed to catch everything (unlike sampling) – why not turn on instance-wide? • overhead vs usefulness? • no license needed at all 55 © 2016 Pythian
  • 55.
    READING TRACEFILES • sqldeveloper •trcanlzr – needs installing but now also part of SQLT • Kyle Hailey’s script – https://github.com/khailey/oracle_trace_parsing/ • Chris Antognini’s script – https://antognini.ch/downloads/top2/chapter03/tvdxtat _40beta10_20140630.zip 56 © 2016 Pythian
  • 56.
    REVIEW • make sqlplusgreat again – rlsqlplus – sqlcl • real-time analysis – snapper – moats – web-ash – sqldeveloper instance viewer – TUNAs360 57 • historical analysis – Orasash / S-ASH – statspack – SQLT – eDB360 – sqld360 • 10046 tracing – raw – various analyzers © 2016 Pythian
  • 58.