SQL Tuning 101 - Sep 2013

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Connor McDonald
OracleDBA
co.uk
1
@connor_mc_d
connormcdonald.wordpress.com

gratuitous book plug
4

bio slide
5

you should look cool
9

11
Dump of memory from 0x07246400 to 0x07248400
1064B4C00 06A20000 08C0000A 0A07C47E 00020506 [...........~....]
1064B4C10...

sounds sexy right ?
13

if you are tuning SQL ...
15

... you are failing
16

why ?
18

real example
19

21
"last transaction for all accounts"

problem
22

make it faster ...
24

"it’s a SQL problem"
25

26
SQL> desc ACCOUNT
Name Null? Type
----------------------------------- -------- ------------------------
ACCOUNT_NUM NOT...

existing code
27

28
SQL> select ACCOUNT_NUM, max(TXN_TS) LAST_TS
2 from TRANSACTIONS, ACCOUNT
3 where <joins etc>
4 group by ACCOUNT_NUM
5 ...

29
SQL> select ACCOUNT_NUM, max(TXN_TS) LAST_TS
2 from TRANSACTIONS
3 group by ACCOUNT_NUM
4 order by 1;
simplified

30
SQL> select ACCOUNT_NUM, max(TXN_TS) LAST_TS
2 from TRANSACTIONS t
3 group by ACCOUNT_NUM
4 order by 1;
---------------...

tuning attempt #1
31

32
SQL> select /*+ PARALLEL(t 16) */
2 ACCOUNT_NUM, max(TXN_TS) LAST_TS
3 from TRANSACTIONS t
4 group by ACCOUNT_NUM
5 ord...

worked...
33

... storage meltdown
34

tuning attempt #2
35

36
create index TRANSACTIONS_IX on TRANSACTIONS (
account_num, txn_ts )
SQL> select ACCOUNT_NUM, max(TXN_TS) LAST_TS
2 fro...

worked...
37

... oltp impact
38

tuning attempt #3
39

40
CREATE TABLE TRANSACTIONS
(
...
)
PARTITION BY RANGE (TXN_TS)
INTERVAL( NUMTOYMINTERVAL(1,'MONTH'))
(
PARTITION P1 VALU...

41
SQL> select ACCOUNT_NUM, max(TXN_TS) LAST_TS
2 from TRANSACTIONS t
3 where TXN_TS > add_months(sysdate,-12)
4 group by ...

42
---------------------------------------------------------------
| Id | Operation | Name |Pstart| Pstop |
--------------...

worked ...
43

... changed the function
44

tuning attempt #4
47

48
Jul Aug Sep Oct

49
SQL> insert into GTT_LAST_TXN
2 select ACCOUNT_NUM, max(TXN_TS) LAST_TS
3 from TRANSACTIONS t
4 where TXN_TS >= trunc(s...

worked ...
50

... complexity !
51

problem
52

53
SQL> select ...
2 from ...
3 where ...

the requirement
54

55
"we have an interest in recent
activity for account holders"
"last transaction for all accounts"

promotional material
56
"we haven't seen you in a while,
here's a discount on ...."

manual examination
58

tuning attempt #5
59

60
SQL> desc ACCOUNT
Name Null? Type
----------------------------------- -------- ---------------
ACCOUNT_NUM NOT NULL NUM...

optimistic locking
61

62
select ..., MODIFICATION_TS
into ..., :prev_modtstamp
from ACCOUNT
where ...
update ACCOUNT
set ...
where ACCOUNT_NUM =...

63
SQL> select ACCOUNT_NUM,
2 MODIFICATION_TS approx_last_usage
3 from ACCOUNT

that's it !
64

later ...
65

66
SQL> desc ACCOUNT
Name Null? Type
------------------------------- -------- --------------
ACCOUNT_NUM NOT NULL NUMBER(8...

67
update ACCOUNT
set ...
where ACCOUNT_NUM = :b1
and MODIFICATION_TS = :prev_modtstamp
LAST_TXN_TS = systimestamp
LARGEST...

68
SQL> desc ACCOUNT
Name Null? Type
----------------------------------- -------- ---------------
ACCOUNT_NUM NOT NULL NUM...

minimal cost
69

(semi) automate
70
"we haven't seen you in a while,
here's a discount on ...."

too focused on the SQL
71

so...
72

if you don't tune SQL ...
73

...what are you tuning ?
74

75
HERE

user experience
76

UX
77

screen...
78

... user experience
79

report....
80

... user experience
81

nightly batch...
82

... user experience
83

all solutions shown ...
84
denormalised
indexes
structure
horse power

... all valid
85

as long as
86

tuning the requirement
87

tuning the user experience
88

SQL as a means, not an end
89

</rant>
90

so if we're "failing"…
91

…why talk about
92

sql tuning is ...
95

stop the bleeding...
97

...until find a cure
98

but ... without
99

... side effects
100

every pain problem
101

102

... side effects
103

104
constipation
dizziness
drowsiness
headache
lightheadedness
nausea
mood swingssweating
vomitingrash
itching
difficulty ...

"This is not a complete list"
105

triage
106
find the (likely) pain
stop the bleeding
avoid side effects
work toward cure

find
107

108

where not too look ...
109
(maybe)

server
110

"server CPU is 45%"
111
"buffer cache is 99%"

too high level
112

user experience
113

"no problems last 15 minutes"
114

Statspack
115
(maybe)

hourly...
116

sql trace
117

definitive ... but
118

(maybe) intrusive
119

(maybe) user experience
120

"please repeat"
121

targetted
122

much easier
123

instrumented
124

125
procedure MY_PROC(...) is
begin
dbms_application_info.set_module('MY_PROC');
dai.set_client_info(' p_parm1 = '||p_parm...

diagnostic pack

v$active_session_history
127

amazing
128

every active session...
129

...every second
130

131
SQL> desc V$ACTIVE_SESSION_HISTORY
Name Null? Type
------------------------------- -------- -------------------
SAMPLE...

"This morning, my screen froze
for 3 mins running Pay Adjustment..."
132

133
SQL> select sql_id,
2 sql_exec_id,
3 count(*)
4 from v$active_session_history
5 where program = 'Finance.exe'
6 and mo...

134
SQL_ID SQL_EXEC_ID COUNT(*)
------------- ----------- ----------
087jnbt3bg3fn 33562711 178
9n2nx15wj6tb3 33512732 39
...

135
# cd $ORACLE_HOME/rdbms/admin
# sqlplus / as sysdba @ashrpt

136

very very cool
137

138
SQL> select sql_fulltext
2 from v$sql
3 where sql_id = '087jnbt3bg3fn';
SQL_FULLTEXT
---------------------------------...

139
latching

140
woeful metaphor

143
same with memory

SGA

145
latches

146
serialised access
aka, someone waits

SGA
protected by

SGA
protected by
1) get latch

SGA
protected by
2) access memory

SGA
protected by
3) release latch

151
what protects the latch ?

152
latch acquisition atomic

you can do better
153

154
SQL> select sql_fulltext
2 from v$sqlstats
3 where sql_id = '087jnbt3bg3fn';
SQL_FULLTEXT
----------------------------...

"The column definitions for columns in V$SQLSTATS are
identical to those in the V$SQL and V$SQLAREA views.
However, the V$...

156
found your sql...

SQL> select e.department_id, sum(salary)
2 from emp e,
3 job_hist j
4 where e.employee_id = j.employee_id
5 and extract(ye...

stop
158

optimizer got it wrong...
159

or
160

... it cannot run better
161
legitimate response

be like the optimizer
162
only smarter

what does it do ?
163

we can see !
164

10053
165

don't panic....
166

167
SQL> alter session set events =
2 '10053 trace name context forever, level 1';
Session altered.
SQL> explain plan for
...

168
SQL> select p.tracefile
2 from v$session s, v$process p
3 where s.paddr = p.addr
4 and s.sid = sys_context('USERENV','...

169
---------------------
QUERY BLOCK SIGNATURE
---------------------
signature (): qb_name=SEL$1 nbfros=2 flg=0
fro(0): f...

170
******************************
PARAMETERS
******************************
Compilation Environment Dump
optimizer_mode_h...

171
Considering Query Transformations
**************************
*************************
Common Subexpression eliminatio...

172
Predicate Move-Around (PM)
**************************
"E"."DEPTNO"="D"."DEPTNO" AND
"E"."ENAME" LIKE '%A%'
AND "D"."DE...

173
SINGLE TABLE ACCESS PATH
Single Table Cardinality Estimation for EMP[E]
Table: EMP Alias: E
Card: Original: 14.000000 ...

174
Access path analysis for DEPT
***************************************
SINGLE TABLE ACCESS PATH
Single Table Cardinalit...

175
Join order[1]: EMP[E]#0 DEPT[D]#1
***************
Now joining: DEPT[D]#1
***************
NL Join
Outer table: Card: 0....

176
SM Join
SM cost: 10.01
Outer table: EMP Alias: E
resc: 4.00 card 0.70 bytes: 13 deg: 1 resp: 4.00
Inner table: DEPT Al...

177
Join order[2]: DEPT[D]#1 EMP[E]#0
***************
Now joining: EMP[E]#0
***************
NL Join
...
SM Join
...
HA Joi...

178
------------------------------------------------+---------------+
| Id | Operation | Name | Rows | Cost |
------------...

the optimizer does ...
179

can I write it better ?
180
CONSIDERING QUERY TRANSFORMATIONS

where should I start ?
181
SINGLE TABLE ACCESS PATH

what should I join to (next) ?
182
Join order[1]: EMP[E]#0 DEPT[D]#1

how should I join ?
183
NL Join
HA Join
SM Join

we follow its lead
184

what is the real query
185

186
SQL> set autotrace traceonly stat
SQL> select * from STUFF
2 where CREATED > sysdate
Statistics
----------------------...

12c
187

188
SQL> variable c clob
SQL> begin
2 dbms_utility.expand_sql_text
3 ( 'select * from STUFF '||
4 'where created > sysdate...

189
SQL> create or replace
2 view STUFF as
3 select
4 o.owner,
5 o.created,
6 s.bytes,
7 s.tablespace_name
8 from
9 dba_se...

190
SELECT "A1"."OWNER" "OWNER",
"A1"."NAME" "NAME",
"A1"."CREATED" "CREATED",
"A1"."BYTES" "BYTES",
"A1"."TABLESPACE_NAME...

191
"A4"."HEADER_BLOCK",
"A4"."SEGMENT_TYPE_ID",
"A4"."BUFFER_POOL_ID",
"A4"."SEGMENT_FLAGS",
"A4"."SEGMENT_OBJD",
"A4"."B...

192
"A198"."SUBNAME" "PARTITION_NAME",
"A196"."OBJECT_TYPE" "SEGMENT_TYPE",
"A195"."TYPE#" "SEGMENT_TYPE_ID",DECODE(BIT
AN...

193
DECODE(BITAND("A209"."PROPERTY",8192),8192,'NESTED TABLE','TABLE') "OBJECT_TYPE",
2 "OBJECT_TYPE_ID",5 "SEGMENT_TYPE_I...

194
"A205"."TS#" "TS_NUMBER" FROM "SYS"."INDPART$" "A205") UNION ALL
(SELECT 'LOBSEGMENT' "OBJECT_TYPE",21 "OBJECT_TYPE_ID...

195
"A195"."FILE#"="A194"."RELFILE#") UNION ALL
(SELECT NVL("A193"."NAME",'SYS') "OWNER",
"A191"."NAME" "SEGMENT_NAME",NUL...

196
"A191"."TS#"="A189"."TS#" AND
"A191"."FILE#"="A189"."RELFILE#") UNION ALL
(SELECT NVL("A188"."NAME",'SYS') "OWNER",
TO...

197
"A186"."TYPE#"<>6 AND
"A186"."TYPE#"<>8 AND
"A186"."TYPE#"<>10 AND
"A186"."TYPE#"<>11 AND
"A186"."TS#"="A185"."TS#"
AN...

198
"SYS"."TS$" "A183",
"SYS"."SEG$" "A182",
"SYS"."FILE$" "A181" WHERE "A182"."TS#"="A183"."TS#" AND
"A182"."USER#"="A184...

16 ...
199
more
pages

reading a single table
200

should I use an index ?
201

full scan versus index
202

203
SQL> select *
2 from STUDENT
3 where gender = 'W';

full scan
(dba_extents)
File 1, Block 271, 512 blocks
File 2, Block 234, 256 blocks
etc

indexed access
W, File 1, Block 37, Row 4
W, File 1, Block 437, Row 11
Sue
Jane
POOL_IX (POOL_TYPE) STUDENT

what will a full scan cost ?
206

207
SQL> select blocks
2 from dba_tables
3 where owner = 'SCOTT'
4 and table_name = 'STUDENT';
BLOCKS
----------
9083

dbms_space
208

209
SQL> select value
2 from v$parameter
3 where name = 'db_file_multiblock_read_count';
VALUE
---------------
128

IO cost blocks / blocks at a time
210

adjusted downwards
211

adj = 1.6765 x orig0.6581
212
Ack: Wolfgang Brietling, 2001

128
213
41

214
SQL> set autotrace traceonly explain
SQL> select /*+ no_cpu_costing */ * from STUDENT;
-------------------------------...

215
SQL> set autotrace traceonly explain
SQL> select /*+ no_cpu_costing */ * from STUDENT;
-------------------------------...

"digest" the rows
216

217

218
SQL> set autotrace traceonly explain
SQL> select * from STUDENT;
-----------------------------------------------------...

what will a index scan cost ?
219

a few blocks +
220

221
SQL> select AVG_LEAF_BLOCKS_PER_KEY
2 from USER_INDEXES
3 where INDEX_NAME = 'STUDENT_IX'
AVG_LEAF_BLOCKS_PER_KEY
----...

approximation
222

the easiest way

run a few...
valid use of hints

understanding joins

nested loops
hash
sort merge
index join
and-equal
btree-bitmap conversion
bitmap join

other complexities
bloom filters
parallel

nested loop

"for each row in outer,
check for row in inner"

for (i=1; i<100; i++)
for (j=1; j<100; j++)
for (k=1; k<100; k++)
...
...

demo

cost

effort = L1 + M * L2
=
cost to Lookup rows
in outer table
number of rows Matching
criteria in outer table
cost of Lookup m...

High M ... no good
effort = L1 + M * L2
=

High L2 ... no good
effort = L1 + M * L2
=

High L1 ... no good maybe
effort = L1 + M * L2

sort merge

demo

merge

cost

effort = S1 + S2 + M
sort table 1 (hold result)
sort table 2 (hold result)
merge results (not held*)

risks

RAM
SQL> select value * 0.05 / 1024 sort_kb
2 from v$parameter
3 where name = 'pga_aggregate_target';
SORT_KB
----------
2...

single pass

really big sorts

multipass sorts

detection

SQL> select * from v$mystat / v$sesstat / v$sysstat
2 where name like 'work%';
NAME VALUE
--------------------------------...

why sort merge ?

non-equality joins
where t1.name > t2.name

hash joins

7.3

260
SQL> select value
2 from v$parameter
3 where name = 'hash_join_enabled';
VALUE
----------------------
true

261

"best" join
262

divide and conquer
263

demo
264

265

266

much larger hash
267

= and != only
268

nested loop
good selectivity into each table
small numbers of (result) rows
first row fast
269

hash join
the "best" join (theory)
270

sort merge
rarely seen
partial joins
271

272
back to our problem SQL

SQL> select e.department_id, sum(salary)
2 from emp e,
3 job_hist j
4 where e.employee_id = j.employee_id
5 and extract(ye...

274
v$sql_plan

-----------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |Temp...

276
now what ?

"where does it hurt ?"
277

278

279
where in the plan ...

SQL> select
2 DBMS_SQLTUNE.REPORT_SQL_MONITOR(
3 sql_id=>'d3ncuxj7629bf',
4 report_level=>'ALL',
5 type=>'HTML') as report...

options
284
rewrite the sql
hints
add indexes
remove indexes

rewrite the sql
285

ideally ... simply
286

287
dynamic sampling

288
cardinality is everything
288

289

290

291
same with Oracle
291

292
actual versus estimate
292

293
SQL> select /*+ GATHER_PLAN_STATISTICS */ count(*)
2 from VEHICLES
3 where MAKE = 'FORD'
4 and MODEL = 'FOCUS';
COUNT(...

294
SQL> SELECT *
2 FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(
3 NULL, NULL, 'ALLSTATS LAST'));
-------------------------------...

295
295
if the estimate is wrong

296
296
try dynamic sampling

297
SQL> create table TOUGH_DATA as
2 select
3 rownum pk,
4 dbms_random.string('U',10) str
5 from dual
6 connect by level ...

298
SQL> select count(*)
2 from TOUGH_DATA
3 where str like '%XX'
4 /
COUNT(*)
----------
1452
hard

299
SQL> select count(*)
2 from TOUGH_DATA
3 where str like '%XX'
4 /
-------------------------------------------------
| ...

300
SQL> select /*+ dynamic_sampling(t 2) */ count(*)
2 from TOUGH_DATA t
3 where str like '%XX'
4 /
---------------------...

more drastic measures
301

use hints
302

303
"optimizer hints should only be
used with extreme care"
Maria Colgan, Oct 2012
Optimizer product manager

awesome hints
304

305
gather_plan_statistics
dynamic_sampling
monitor
qb_name

query blocks
306

C#
PL/SQL
307

no comments
308

309

"C# is self-documenting"
310

for (int i = 0; i < w.Count; ++i)
{
if (w[i].v > 0 && w[i].p > mddh)
{
m = Math.Max(m, w[i].d);
}
}
for (int i = 0; i < w....

query blocks =
self- documenting SQL
312

select emp.*
from emp,
( select trunc(hiredate,'YYYY'), max(empno) empno
from emp
where empno > 0
group by trunc(hiredate,...

Id | Operation | Name |
----------------------------------------------|
0 | SELECT STATEMENT | |
1 | HASH JOIN | |
2 | TAB...

select emp.*
from emp,
( select /*+ QB_NAME(YR_HIRE) */
trunc(hiredate,'YYYY'), max(empno) empno
from emp
where empno > 0
...

Id | Operation | Name | Query Block
----------------------------------------------|--------------
0 | SELECT STATEMENT | |...

assist with other hints
317

select
/*+ QB_NAME(top)
INDEX(@yr_hire emp (empno))
FULL(@av_sal emp) */
emp.*
from emp,
( select /*+ QB_NAME(YR_HIRE) */
...

sidebar
319

320
select
/*+ QB_NAME(top)
INDEX(@yr_hire emp (empno))
FULL(@av_sal emp)
*/
INDEX(emp emp_ix)

cool hints
321

322
first_rows_n
all_rows
leading
cardinality
statement_queueing
opt_param

assisting optimizer
323

dangerous hints
324

everything else...
325

326
full
index
use_hash
use_nl
etc

suppressing the optimizer
327

undocumented hints
328

329
selectivity
no_trigger

an invalid hint
330

is a comment
331

332
SQL> select /*+ INDEX(accounts_pk) */
2 from ACCOUNTS
3 where ...
SQL> select /*+ INDEX(a) */
2 from ACCOUNTS
3 where ...

no error raised !
333

even if it IS valid ...
334

... the (in)famous ignore issue
335

336
SQL> select *
2 from emp e,
3 dept d
4 where e.deptno = d.deptno
5 and d.dname = 'SALES';
----------------------------...

337
"I want a hash join"

338
SQL> select /*+ use_hash(d) */ *
2 from emp e,
3 dept d
4 where e.deptno = d.deptno
5 and d.dname = 'SALES';
---------...

339
/*+ use_hash(d) */
if joining into "d"...
use a hash join

we're not...
340

341
--------------------------------------------------------
| Id | Operation | Name | Rows |
----------------------------...

342
if joining into "d"...
we must be starting with "e"

343
SQL> select /*+ leading(e) use_hash(d) */ *
2 from emp e,
3 dept d
4 where e.deptno = d.deptno
5 and d.dname = 'SALES'...

not enough hints
344

345
"hints are like violence…
if they do not work, use more"

346
SQL> select /*+ leading(e d b) use_hash(d) use_hash(b) */ *
2 from emp e,
3 dept d,
4 bonus b
5 where e.deptno = d.dep...

347
SQL> select /*+ leading(e d b) use_hash(d) use_hash(b) */ *
2 from emp e,
3 dept d,
4 bonus b
5 where e.deptno = d.dep...

348
---------------------------------------------
| Id | Operation | Name | Rows |
---------------------------------------...

join inputs
349

350
SQL> select /*+ leading(e d b)
2 use_hash(d)
3 use_hash(b)
4 no_swap_join_inputs(b)
5 */
6 *
7 from emp e,
8 dept d,
9...

problem ...
351

352
/*+ no_swap_join_inputs (b) */

option 1
353

fully specify hints
354

355
SQL> SELECT *
2 from table(dbms_xplan.display(.... ,
format=>'typical +OUTLINE))
/*+ BEGIN_OUTLINE_DATA
USE_HASH(@"SEL...

option 2
356

sql baseline
357

just hints … better in 12c
358

add / remove indexes
359

side effects
360

361
structural change

"an index will fix it..."
362

363
SQL> select *
2 from SALES
3 where SALES_DATE > ...

364

365
SQL> create index SALES_IX
2 on SALES( SALES_DATE );
Index created.

366
SQL> explain plan for
2 select * from SALES
3 where sales_date > ...;
------------------------------------------------...

367

368

369
your query

370
everyone elses queries

371
real example

372
SQL> create table T
2 as select * from all_objects;
Table created.
SQL> create index OBJ_IX on T ( OBJECT_ID);
Index c...

373
SQL> set autotrace traceonly explain
SQL> select * from T
2 where OWNER = 'SCOTT'
3 and created > sysdate - 1
4 /
----...

374
“Can I index OWNER..”

375
SQL> create index NEW_IX on T ( owner);
Index created.
SQL> select * from T
2 where OWNER = 'SCOTT'
3 and created > sy...

376
existing code ?

377
before the new index

378
SQL> set autotrace traceonly explain
SQL> select * from T
2 where OWNER = 'SYS'
3 and OBJECT_ID between 10 and 8000
4 ...

379
SQL> set autotrace traceonly stat
SQL> select * from T
2 where owner = 'SYS'
3 and object_id between 10 and 8000
4 /
4...

380
after the new index

381
SQL> select * from T
2 where owner = 'SYS'
3 and object_id between 10 and 8000
4 /
-----------------------------------...

382
SQL> set autotrace traceonly stat
SQL> select * from T
2 where owner = 'SYS'
3 and object_id between 10 and 8000
4 /
4...

383
invisible indexesconsider

384
SQL> alter index OWN_IX invisible;
Index altered.

385
SQL> alter session set
2 optimizer_use_invisible_indexes = true;
Session altered.
SQL> [MY QUERY]
SQL> alter session s...

wrap up
386

if you are tuning SQL ...
387
... you are failing

tuning the user experience
388

389
find
stop
side effects
cure

390

ORA-03113
connormcdonald.wordpress.com

Share Slideshare

LinkedIn
Facebook
Twitter

Embed

Size (px)

Show related Slideshows at end

WordPress Shortcode

Link

Share
Email

IMTC Miami Conference - Social Medi... by Michael Kent 966 views
Marketing and PR in Times of Turmoil by Pierpont Communic... 590 views
Dido Green: Expectations for Therap... by Beitissie1 624 views
Apresentacao by Airton Vilela 274 views
Optimizer Statistics by Connor McDonald 1727 views
CellCoverage by Brian Pfeiffer 211 views

View on Slideshare

1 of 391 Ad

View on Slideshare

1 of 391 Ad