The document discusses the risks and challenges of automatically gathering statistics in an Oracle database. It notes that while gathering statistics is intended to optimize SQL performance, it can sometimes have the unintended effect of adding more expensive SQL or invalidating cached execution plans, potentially slowing performance. The recommendation is that unless database performance is known to be poor, statistics should not be changed automatically and the risks of gathering statistics outweigh the potential benefits.

1. NOTE
itty bitty fonts in this
presentation
SQL> exec sample_font
Can you read this ?
1

2. Connor McDonald
OracleDBA
co.uk
2

3. 3

4. bio slide
4

5. Connor McDonald

6. a funny story
6

7. 7

8. 8

9. 9

10. why ?
10

12. 12

14. life was simple
14

15. 15

16. (good) cost optimizer
16

17. 17

18. version 7.0
18

19. optimizer_mode = CHOOSE
19

20. "we choose RULE"
20

21. SQL> analyze table SALES estimate statistics;
21

22. 22

23. eventually....
23

24. it got better
24

25. added more functionality
25

26. SQL> analyze table SALES estimate statistics
2 for table
3 for columns size 10
4 for ....;
26

27. SQL syntax engine
27

28. oracle 8i
28

29. DBMS_STATS
29

30. problem....
30

31. SQL> analyze table SALES estimate statistics;
41 characters
31

32. SQL> begin
2 dbms_stats.gather_table_stats(
3 ownname=>'HR',
4 tabname=>'SALES',
5 cascade=>true,
6 estimate_percent=>20
7 );
8 end;
128 characters
32

33. don't get chaining
33

34. s l o w e r
34

35. "I don't think so....."
35

36. times have changed ...
36

37. SQL> desc DBMS_STATS
PROCEDURE ALTER_DATABASE_TAB_MONITORING
PROCEDURE ALTER_SCHEMA_TAB_MONITORING
PROCEDURE ALTER_STATS_HISTORY_RETENTION
PROCEDURE CLEANUP_STATS_JOB_PROC
PROCEDURE CONVERT_RAW_VALUE
PROCEDURE CONVERT_RAW_VALUE
PROCEDURE CONVERT_RAW_VALUE
PROCEDURE CONVERT_RAW_VALUE
PROCEDURE CONVERT_RAW_VALUE
PROCEDURE CONVERT_RAW_VALUE_NVARCHAR
PROCEDURE CONVERT_RAW_VALUE_ROWID
PROCEDURE COPY_TABLE_STATS
FUNCTION CREATE_EXTENDED_STATS RETURNS VARCHAR
PROCEDURE CREATE_STAT_TABLE
PROCEDURE DELETE_COLUMN_STATS 37

38. 126 routines !
38

39. 39

40. profiles
tuning sets
baselines
features
bind peeking
adaptive cursor sharing
40

41. better stats needed ...
41

42. ... for the optimizer
42

43. hard to convince
43

44. 44

45. stop using analyze
45

46. ego
46

47. "good ol' days"
47

48. 48

49. 49

50. today....
50

51. optimizer is probably...
51

52. ... smarter than you
52

53. nested loop outer
sort merge outer
SQL> select count(e.hiredate)
2 from DEPT d, EMP e
3 where e.deptno = d.deptno(+)
4 and e.sal > 10;
hash hash anti
nested loop anti
53

54. -------------------------------------------
| Id | Operation | Name | Rows |
-------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
| 1 | SORT AGGREGATE | | 1 |
|* 2 | TABLE ACCESS FULL| EMP | 14 |
-------------------------------------------
no DEPT ?
54

55. not null
SQL> select count(e.hiredate)
2 from DEPT d, EMP e
3 where e.deptno = d.deptno(+)
4 and e.sal > 10;
foreign key
55 key preserved

56. Oracle's solution....
56

57. statistics by stealth...
57

58. 10g
58

59. SQL> select owner, job_name, enabled
2 from dba_scheduler_jobs
3 where owner = 'SYS';
OWNER JOB_NAME ENABLED
--------------- ------------------------------ -------
SYS PURGE_LOG TRUE
SYS FGR$AUTOPURGE_JOB TRUE
SYS GATHER_STATS_JOB FALSE
TRUE
SYS AUTO_SPACE_ADVISOR_JOB TRUE
59

60. 11g
60

61. SQL> select owner, job_name, enabled
2 from dba_scheduler_jobs
3 where owner = 'SYS';
OWNER JOB_NAME ENABLED
--------------- ------------------------------ -------
SYS AUTO_SPACE_ADVISOR_JOB FALSE
SYS BSLN_MAINTAIN_STATS_JOB TRUE
SYS DRA_REEVALUATE_OPEN_FAILURES TRUE
SYS FGR$AUTOPURGE_JOB FALSE
SYS GATHER_STATS_JOB FALSE
SYS HM_CREATE_OFFLINE_DICTIONARY FALSE
SYS ORA$AUTOTASK_CLEAN TRUE
SYS PURGE_LOG TRUE
SYS XMLDB_NFS_CLEANUP_JOB FALSE
61

62. stats STILL being collected
62

63. automatic "tasks"
"super stealth mode"
63

64. SQL> select client_name, status
2 from DBA_AUTOTASK_CLIENT;
CLIENT_NAME STATUS
------------------------------------ --------
auto optimizer stats collection ENABLED
auto space advisor ENABLED
sql tuning advisor ENABLED
64

65. stats every night
most sites
default options
65

66. in this session...
66

67. default behaviour
67

68. BAD
IDEA 68

69. collecting statistics
69

70. BAD
IDEA 70

71. but....
71

72. some default behaviour
72

73. GOOD
IDEA 73

74. collecting some statistics
74

75. GOOD
IDEA 75

76. 76

77. Inflammatory
statements
which will
alienate the
audience
1
Presentation Duration
77

78. we're all hypocrites
78

79. statistical hypocrisy
79

80. "I need to change some
reference data in my system"
80

81. wrong case

82. "nope...."
82

83. ITIL
83

84. Information
Technology
Infrastructure
Library
84

85. service
call
help
desk DONE !
problem
record

86. phew....
86

87. corrected

88. same site....
88

89. "Every night, I would like to
potentially change the
performance characteristics of
every single SQL statement in
the database"
89

90. "no problem...."
90

91. collecting stats = risk
91

92. Ensor's paradox
92

93. 1987
performance group
bstat/estat
tkprof
BMC patrol
afiedt.buf
93

94. "It is only safe to
gather statistics ..."
"...when to do so will
make no difference"
94

95. recommendation #1
95

96. 96

97. unless things are bad...
do not change statistics
97

98. "surely it can't hurt?"
98

99. 10g
99

100. options=>'GATHER STALE'
100

101. options=>'GATHER EMPTY'
101

102. options=>'GATHER AUTO'
102

103. "no plans changed"
103

104. "no queries ran slower"
104

105. still problems
105

106. problem # 1
106

107. dbms_stats
107

108. the goal of statistics
108

109. minimise expensive SQL....
109

110. added more expensive SQL !
110

111. SQL> alter session set sql_true = true;
Session altered.
SQL> begin
2 dbms_stats.gather_table_stats(
3 'DEMO',
4 'PEOPLE);
5 end;
6 /
111

112. SQL> select
2 count(distinct GENDER),
3 min(GENDER),
4 max(GENDER),
5 count(distinct NAME),
6 min(NAME),
7 max(NAME)
...
...
21 from PEOPLE;
112

113. hard !
113

115. problem #2
115

116. lingering pain
116

117. invalidation
117

119. library cache
119

120. select * insert into PEOPLE
from PEOPLE select * from ...
select ...
from PEOPLE,
DEPT
where ...
SQL> begin
2 dbms_stats.gather_table_stats(
3 'DEMO',
4 'PEOPLE);
5 end;
6 /
declare
delete from T v people.name%type;
where X in begin
( select PID ...
from PEOPLE )
120

121. 121

122. recap
122

123. gathering statistics
really hard core SQL
belted CPU with hard parsing
everything ran the same
123

124. oracle 9
124

125. SQL> desc DBMS_STATS
PROCEDURE GATHER_TABLE_STATS
Argument Name Type In/Out Default?
----------------------- -------------- ------ --------
OWNNAME VARCHAR2 IN
TABNAME VARCHAR2 IN
PARTNAME VARCHAR2 IN DEFAULT
ESTIMATE_PERCENT NUMBER IN DEFAULT
BLOCK_SAMPLE BOOLEAN IN DEFAULT
METHOD_OPT VARCHAR2 IN DEFAULT
DEGREE NUMBER IN DEFAULT
GRANULARITY VARCHAR2 IN DEFAULT
CASCADE BOOLEAN IN DEFAULT
STATTAB VARCHAR2 IN DEFAULT
STATID VARCHAR2 IN DEFAULT
STATOWN VARCHAR2 IN DEFAULT
NO_INVALIDATE BOOLEAN IN DEFAULT
STATTYPE VARCHAR2 IN DEFAULT
FORCE BOOLEAN IN DEFAULT
125

126. ?
no_invalidate => false | true
?
126

127. oracle 10
127

128. no_invalidate => auto
128

129. SQL> select
2 x.ksppinm name,
3 y.kspftctxvl value
4 from
5 sys.x$ksppi x,
6 sys.x$ksppcv2 y
7 where
8 x.indx+1 = y.kspftctxpn and
9 x.ksppinm = '_optimizer_invalidation_period'
NAME VALUE
------------------------------ ------------
_optimizer_invalidation_period 18000
129

130. select * insert into PEOPLE
from PEOPLE select * from ...
select ...
from PEOPLE,
DEPT
where ...
SQL> exec dbms_stats.gather_table_stats(
user,'PEOPLE');
declare
delete from T v people.name%type;
where X in begin
( select PID ...
from PEOPLE )
130

131. better ?
131

132. hard parse storm avoided
132

133. 5 hours before problems
133

134. 134

135. unless things are bad...
DO NOT CHANGE STATISTICS
135

136. Inflammatory
statements
which will
alienate the
audience
2
1
Presentation Duration
136

137. we don't know if things are bad
137

138. until its too late
138

139. users fed up
SQL's are slow
stats out of date
139

140. 140

141. 141

142. 2 hours to gather stats
142

143. 143

144. collect stats
144

145. in readiness
145

146. 146

147. pending statistics
147

148. "published" – optimizer uses
"pending" - private
148

149. SQL> begin
2 dbms_stats.set_schema_prefs(
3 ownname=>'DEMO',
4 pname=>'PUBLISH',
5 pvalue=>'FALSE');
6 end;
database,
table
149

150. SQL> alter session set
2 optimizer_use_pending_statistics = true;
150

151. SQL> begin
2 dbms_stats.publish_pending_stats(
3 ownname=>'DEMO',
4 tabname=>null);
5 end;
151

152. recommendation #2
152

153. when collecting statistics
153

154. PUBLISH = false
always....
154

155. even if you immediately publish
155

156. atomic publication
156

157. and don't wait 5 hours
157

158. _optimizer_invalidation_period
158

159. for most tables
159

160. dbms_stats.lock_table_stats
160

161. collecting system stats
161

162. different story
162

163. absolutely vital…
…to have them
163

164. maximize I/O throughput
164

165. SQL> set autotrace traceonly explain
SQL> select * from T;
---------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)|
---------------------------------------------------------------
| 0 | SELECT STATEMENT | | 53459 | 4489K| 113 (1)|
| 1 | TABLE ACCESS FULL| T | 53459 | 4489K| 113 (1)|
---------------------------------------------------------------
SQL> select value
2 from v$spparameter
3 where name = 'db_file_multiblock_read_count';
VALUE
-----------
16
165

166. SQL> alter session set events
2 = '10046 trace name context forever, level 8';
SQL> select * from T;
PARSING IN CURSOR #22 len=15 dep=0 uid=124 oct=3 lid=124 ...
select * from T
END OF STMT
PARSE #22:c=10000,e=108,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4
EXEC #22:c=0,e=65,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4
WAIT #22: nam='db file scattered read' ela= 3387
file#=5 block#=13122 blocks=16 obj#=199963
WAIT #22: nam='db file scattered read' ela= 3188
file#=5 block#=13234 blocks=16 obj#=199963
WAIT #22: nam='db file scattered read' ela= 3125
file#=5 block#=13250 blocks=16 obj#=199963
WAIT #22: nam='db file scattered read' ela= 3255
file#=5 block#=13266 blocks=16 obj#=199963
WAIT #22: nam='db file scattered read' ela= 3369
file#=5 block#=13282 blocks=16 obj#=199963
166

167. SSTIOMAX
167

168. 1 MB
168

169. db_block_size = 8192
169

170. db_file_multiblock_read_count = 128
170

171. ?
171

172. 172

173. the solution ?
173

174. system stats
174

175. SQL> select pname, pval1
2 from sys.aux_stats$
3 /
PNAME PVAL1
------------------------------ ----------
SREADTIM 4.065
MREADTIM 6.173
CPUSPEED 567
MBRC 12
MAXTHR 48203776
SLAVETHR -1
175

176. use MBRC to cost
176

177. use db_file_multiblock_read_count to read
177

178. Inflammatory
statements 3
which will
alienate the
audience
2
1
Presentation Duration
178

179. forget about system statistics
179

180. do not use them
180

181. 181

182. 182

183. absolutely vital…
…to have them
183

184. do not “use”* them
* = collect, change, set
184

185. problem #1
185

186. 186

187. problem #2
187

188. monitoring for bad....
188

189. optimizing for bad
189

190. recommendation #3
190

191. gather
into STAT table
191

192. monitor
192

193. set once
193

194. the defaults are probably fine
194

195. SQL> select pname, pval1
2 from sys.aux_stats$
3 /
PNAME PVAL1
---------------------- ----------
IOSEEKTIM 10 (ms)
IOTFRSPEED 4096 (bytes/ms)
SREADTIM = IOSEEKTIM + db_block_size / IOTFRSPEED = 10
MREADTIM = IOSEEKTIM + mbrc * db_block_size / IOTFRSPEED = 26
195

196. 10.2 and above
196

197. _db_file_optimizer_read_count
db_file_multiblock_read_count
_db_file_exec_read_count
197

198. dbms_resource_manager
to calibrate
198

199. so far...
199

200. not changing statistics
200

201. eventually....
201

202. the time will come...
202

203. one possible reason
203

204. even if stats unchanged …
204

205. … your plans might !
205

206. SQL> create table T as
2 select to_date('01-AUG-2011')+
3 trunc(dbms_random.value(0,7)) dte,
4 rpad('padding',20) padding
5 from dual
6 connect by level <= 100000;
Table created.

207. SQL> select dte, count(*)
2 from t
3 group by dte
4 order by 1;
DTE COUNT(*)
--------- ----------
01-AUG-11 14334
02-AUG-11 14222
03-AUG-11 14167
04-AUG-11 14510
05-AUG-11 14346
06-AUG-11 14349
07-AUG-11 14072

208. 15,000 rows per day always
208

209. SQL> exec dbms_stats.gather_table_stats(user,'T')
PL/SQL procedure successfully completed.
SQL> create index IX on T ( dte ) ;
Index created.

210. SQL> select * from t where dte = '03-AUG-2011';
--------------------------------------------------
| Id | Operation | Name | Rows | Bytes |
--------------------------------------------------
| 0 | SELECT STATEMENT | | 14286 | 404K|
|* 1 | TABLE ACCESS FULL| T | 14286 | 404K|
--------------------------------------------------

211. a week later…
211

212. SQL> insert into T
2 select to_date('08-AUG-2011')+
3 trunc(dbms_random.value(0,7)) dte,
4 rpad('padding',20) padding
5 from dual
6 connect by level <= 100000;
100000 rows created.

213. SQL> select dte, count(*)
2 from t
3 group by dte
4 order by 1;
DTE COUNT(*)
--------- ----------
01-AUG-11 14334
...
07-AUG-11 14072
08-AUG-11 14261
09-AUG-11 14106
10-AUG-11 14410
11-AUG-11 14289
12-AUG-11 14358
13-AUG-11 14252
14-AUG-11 14324

214. statistics unchanged
214

215. SQL> select * from t where dte = '12-AUG-2011';
----------------------------------------------------
| Id | Operation | Name | Rows |
----------------------------------------------------
| 0 | SELECT STATEMENT | | 2381 |
| 1 | TABLE ACCESS BY INDEX ROWID| T | 2381 |
|* 2 | INDEX RANGE SCAN | IX | 2381 |
----------------------------------------------------

216. SQL> select * from t where dte = '14-AUG-2011';
----------------------------------------------------
| Id | Operation | Name | Rows |
----------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
| 1 | TABLE ACCESS BY INDEX ROWID| T | 1 |
|* 2 | INDEX RANGE SCAN | IX | 1 |
----------------------------------------------------

217. why ?
217

218. SQL> select low_value, high_value
2 from user_tab_columns
3 where table_name = 'T'
4 and column_name = 'DTE';
LOW_VALUE HIGH_VALUE
-------------------- -----------------
786F0801010101 786F0807010101

219. SQL> set serverout on
SQL> declare
2 res date;
3 begin
4 dbms_stats.convert_raw_value(
5 '786F0801010101',res);
6 dbms_output.put_line(result);
7 dbms_stats.convert_raw_value(
8 '786F0806010101',res);
9 dbms_output.put_line(result);
10 end;
11 /
01-AUG-11
06-AUG-11

220. 01-AUG-11 06-AUG-11

221. when the time comes…
221

222. as accurate as possible
222

223. as cheaply as possible
223

224. statistics accurately
224

225. extended statistics
225

226. 226

227. 227

228. cardinality is everything
228
228

229. same with Oracle
229
229

230. some real(ish) data
230
230

231. SQL> desc VEHICLE
Name Null? Type
-------------------------- -------- -------------
ID NUMBER
MAKE VARCHAR2(6)
MODEL VARCHAR2(6)
SQL> select count(*)
2 from VEHICLE;
COUNT(*)
------------
2,157,079
231
231

232. default stats not enough
232
232

233. SQL> select count(*)
2 from VEHICLE
3 where MAKE = 'HOLDEN';
COUNT(*)
----------
415387
------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost |
------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 7 | 138|
| 1 | SORT AGGREGATE | | 1 | 7 | |
|* 2 | INDEX RANGE SCAN| MAKE_IX | 55310 | 378K| 138|
------------------------------------------------------------
233
233

234. histogram
234
234

235. SQL> begin
2 dbms_stats.gather_table_stats(user,'VEHICLE',
3 method_opt=>'for all columns size 1,'||
4 'for columns MAKE size 254,'||
5 'for columns MODEL size 254');
6 end;
7 /
PL/SQL procedure successfully completed.
235
235

236. SQL> select count(*)
2 from VEHICLE
3 where MAKE = 'HOLDEN';
COUNT(*)
----------
415387
-----------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost |
-----------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 7 | 1024|
| 1 | SORT AGGREGATE | | 1 | 7 | |
|* 2 | INDEX RANGE SCAN| MAKE_IX | 418K| 2859K| 1024|
-----------------------------------------------------------
236
236

237. make AND model
237
237

238. SQL> select count(*)
2 from VEHICLE
3 where MAKE = 'HOLDEN'
4 and MODEL = 'COMMODORE';
COUNT(*) 50% holdens are commodores
----------
214468
--------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |
---------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 14 |
| 1 | SORT AGGREGATE | | 1 | 14 |
| 2 | BITMAP CONVERSION COUNT | | 39527 | 540K|
| 3 | BITMAP AND | | | |
| 4 | BITMAP CONVERSION FROM ROWIDS| | | |
|* 5 | INDEX RANGE SCAN | MODEL_IX | | |
| 6 | BITMAP CONVERSION FROM ROWIDS| | | |
|* 7 | INDEX RANGE SCAN | MAKE_IX | | |
---------------------------------------------------------------------
238
238

239. two things
239
239

240. 240

241. 241
241

242. no correlation
10g and before
242
242

243. 11g
243
243

244. SQL> select
2 DBMS_STATS.CREATE_EXTENDED_STATS(
3 user, 'VEHICLE','(MAKE,MODEL)') tag
4 from dual;
TAG
----------------------------------
SYS_STU8QPK2S$PEWHARK2CP3#1F#G
244
244

245. SQL> begin
2 dbms_stats.gather_table_stats(user,'VEHICLE',
3 method_opt=>
4 'for columns (make,model) size 254');
5 end;
6 /
PL/SQL procedure successfully completed.
245
245

246. SQL> select count(*)
2 from VEHICLE
3 where MAKE = 'HOLDEN'
4 and MODEL = 'COMMODORE';
COUNT(*)
----------
214468
-------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost |
------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 14 | 1956|
| 1 | SORT AGGREGATE | | 1 | 14 | |
|* 2 | TABLE ACCESS FULL| VEHICLE | 220K| 3018K| 1956|
-------------------------------------------------------------
246
246

247. composite indexes
247
247

248. 11.2 column groups
248
248

249. SQL> exec DBMS_STATS.SEED_COL_USAGE(NULL,NULL,60);
249
249

250. implemented via virtual columns
250
250

251. various implications
251
251

252. accurate stats not always possible...
252

253. you can't have stats
253

254. on everything !
254

255. when stats wont do...
255

256. ... use the real data
256

257. dynamic sampling
257

258. SQL> drop table TOUGH_DATA purge;
Table dropped.
SQL> create table TOUGH_DATA nologging as
2 select
3 rownum pk,
4 dbms_random.string('U',10) str
5 from dual
6 connect by level < 1000000
7 /
Table created.
SQL> exec dbms_stats.gather_table_stats(
user,'TOUGH_DATA')
258

259. SQL> select count(*)
2 from TOUGH_DATA
3 where str like '%XX'
4 /
COUNT(*) hard
----------
1452
259

260. SQL> select count(*)
2 from TOUGH_DATA
3 where str like '%XX'
4 /
-------------------------------------------------
| Id | Operation | Name | Rows |
-------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
| 1 | SORT AGGREGATE | | 1 |
|* 2 | TABLE ACCESS FULL| TOUGH_DATA | 50000 |
-------------------------------------------------
5% assumption
260

261. SQL> select /*+ dynamic_sampling(t 2) */ count(*)
2 from TOUGH_DATA t
3 where str like '%XX'
4 /
-------------------------------------------------
| Id | Operation | Name | Rows |
-------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
| 1 | SORT AGGREGATE | | 1 |
|* 2 | TABLE ACCESS FULL| TOUGH_DATA | 1252 |
-------------------------------------------------
261

262. simple queries as well
262

263. 1,2,3,4,5,6...... A,A,A,A,A,B,B,B,B,.....
SQL> create table EASY_DATA nologging as
2 select
3 rownum pk,
4 chr(65+trunc(rownum/40000)) str
5 from dual
6 connect by level < 1000000
7 /
Table created.
SQL> exec dbms_stats.gather_table_stats(user,'EASY_DATA')
PL/SQL procedure successfully completed.
263

264. SQL> select count(*)
2 from EASY_DATA
3 where str = 'F'
4 and pk > 900000;
COUNT(*)
----------
0
264

265. SQL> set autotrace traceonly explain
SQL> select count(*)
2 from EASY_DATA
3 where str = 'F'
4 and pk > 900000;
------------------------------------------------
| Id | Operation | Name | Rows |
------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
| 1 | SORT AGGREGATE | | 1 |
|* 2 | TABLE ACCESS FULL| EASY_DATA | 4000 |
------------------------------------------------
265

266. SQL> set autotrace traceonly explain
SQL> select /*+ dynamic_sampling(t 2) */ count(*)
2 from EASY_DATA t
3 where str = 'F'
4 and pk > 900000;
------------------------------------------------
| Id | Operation | Name | Rows |
------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
| 1 | SORT AGGREGATE | | 1 |
|* 2 | TABLE ACCESS FULL| EASY_DATA | 23 |
------------------------------------------------
266

267. 11.2
267

268. parallel queries (maybe) sampled
268

269. SQL> select count(*)
2 from EASY_DATA t
3 where str = 'F'
4 and pk > 900000;
----------------------------------------------------
| Id | Operation | Name | Rows |
----------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
| 1 | SORT AGGREGATE | | 1 |
| 2 | PX COORDINATOR | | |
| 3 | PX SEND QC (RANDOM) | :TQ10000 | 1 |
| 4 | SORT AGGREGATE | | 1 |
| 5 | PX BLOCK ITERATOR | | 4 |
|* 6 | TABLE ACCESS FULL| EASY_DATA | 4 |
----------------------------------------------------
Note
-----
- dynamic sampling used for this statement (level=4)
269

270. low frequency
270

271. high cost
271

272. statistics cheaply
272

273. SQL> desc DBA_TABLES
Name Null? Type
----------------------------- -------- -------------
OWNER NOT NULL VARCHAR2(30)
TABLE_NAME NOT NULL VARCHAR2(30)
COLUMN_NAME NOT NULL VARCHAR2(30)
...
NUM_ROWS NUMBER
273

274. SQL> desc DBA_TAB_COLS
Name Null? Type
----------------------------- -------- -------------
OWNER NOT NULL VARCHAR2(30)
TABLE_NAME NOT NULL VARCHAR2(30)
COLUMN_NAME NOT NULL VARCHAR2(30)
...
NUM_DISTINCT NUMBER
LOW_VALUE RAW(32)
HIGH_VALUE RAW(32)
...
274

275. SQL> desc PEOPLE
Name Null? Type
----------------------------- -------- -------------
PID NUMBER
GENDER CHAR(1)
NAME VARCHAR2(47)
AGE NUMBER
DEATH_RATE NUMBER
275

276. SQL> alter session set sql_true = true;
Session altered.
SQL> begin
2 dbms_stats.gather_table_stats(
3 'DEMO',
4 'PEOPLE);
5 end;
6 /
276

277. select
/*+ no_parallel(t)
no_parallel_index(t)
dbms_stats
cursor_sharing_exact
use_weak_name_resl
dynamic_sampling(0)
no_monitoring
no_substrb_pad */
count(*),
...
count("GENDER")
count("GENDER"),
"GENDER")
count(distinct "GENDER"),
min("GENDER")
substrb(dump(min("GENDER"),16,0,32),1,120),
substrb(dump(max("GENDER"),16,0,32),1,120),
max("GENDER")
...
from "MCDONAC"."PEOPLE" t
277

278. count("GENDER") one pass
count(distinct "GENDER") hard
min("GENDER") one pass
max("GENDER") one pass
278

279. SQL> select count(*) from PEOPLE;
COUNT(*)
------------
500000000
Elapsed: 00:04:43.73
279

280. SQL> begin
2 dbms_stats.gather_table_stats(
3 user,
4 'PEOPLE',
5 estimate_percent=>25);
6 end;
7 /
ERROR at line 1:
ORA-01652: unable to extend temp
segment by 128 in tablespace TEMP
Elapsed: 00:16:37.12
280

281. 11g
281

282. one pass NDV
282

283. HASH
12 6 3 12 3 7 11 12 33 6 11 12 6 45 15 7 15 17 45 6 17
12 6 3 7 11 33 45 15 17
283

284. 12 6 3 7 11 33 45 15 17
NDV = 9
284

285. 91
11 34
12 7 15
6 64
33 56
92
what about large NDV ?
21
5 61
45
3
2
17
71
73 41
285

286. "Magic" HASH
286

287. NDV =
remaining hashes x
2^number of splits
287

288. demo
288

289. SQL> create table ONE_PASS nologging
2 as select substr(text,1,1) single_char
3 from DBA_SOURCE;
Table created.
SQL> select count(distinct single_char) ndv
2 from one_pass;
NDV
----------
83
289

290. 0 127
16 buckets...
290

291. 64 127
291

292. 96 127
292

293. SQL> set serverout on
SQL> declare
2 type t_bucket is table of varchar2(1);
3 l_synopsis t_bucket;
4 l_splits number := 0;
5 l_hash int;
6 l_min_val int := 0;
7 l_synopsis_size int := 16;
8 begin
9 for i in ( select single_char from one_pass ) loop
10 l_hash := ascii(i.single_char);
11
12 if l_synopsis.count = l_synopsis_size then
13 l_min_val :=
14 case
15 when l_min_val = 0 then 64
16 when l_min_val = 64 then 96
17 when l_min_val = 96 then 112
18 when l_min_val = 112 then 120
19 end;
20 l_splits := l_splits + 1;
293

294. 22
23 for j in 1 .. l_min_val loop
24 if l_synopsis.exists(j) then
25 l_synopsis.delete(j);
26 end if;
27 end loop;
28 end if;
29
30 if l_hash > l_min_val then
31 l_synopsis(l_hash) := 'Y';
32 end if;
33 end loop;
34 dbms_output.put_line(l_synopsis.count *
35 power(2,l_splits));
36 end;
37 /
Splitting, keeping entries above 64
Splitting, keeping entries above 96
Splitting, keeping entries above 112
88
294

295. 18,446,744,073,709,551,616 hash range
the reality
16384 bucket limit
295

296. SQL> begin
2 dbms_stats.gather_table_stats(
3 user,
4 'PEOPLE',
5 estimate_percent=>25);
6 end;
7 /
PL/SQL procedure successfully completed.
Elapsed: 00:05:39.82
296

297. recommendation #4
297

298. estimate_percent
use DEFAULT size (AUTO)
298

299. only for columns
without histograms
299

300. 4
Inflammatory
statements 3
which will
alienate the
audience
2
1
Presentation Duration
300

301. histograms...
301

302. ...SUCK
302

303. not really but...
303

304. 9i
304

306. SQL> set autotrace traceonly explain
SQL> select * from MY_TABLE_100K_ROWS
2 where r = :b1
3 /
------------------------------------------
| Id | Operation | Name | Rows |
------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
| 1 | INDEX UNIQUE SCAN| PK | 1 |
------------------------------------------
~ num rows / num distinct
306

307. SQL> set autotrace traceonly explain
SQL> select * from MY_TABLE_100K_ROWS
2 where r < :b1
3 /
-----------------------------------------
| Id | Operation | Name | Rows |
-----------------------------------------
| 0 | SELECT STATEMENT | | 5000 |
| 1 | INDEX RANGE SCAN| PK | 5000 |
-----------------------------------------
5%
307

308. 308

309. cool ?
309

310. 10g
310

311. method_opt
for all columns size auto
311

312. sys.col_usage$
312

313. SQL> desc SYS.COL_USAGE$
Name Null? Type
----------------------- -------- ---------
OBJ# NUMBER
INTCOL# NUMBER
EQUALITY_PREDS NUMBER
EQUIJOIN_PREDS NUMBER
NONEQUIJOIN_PREDS NUMBER
RANGE_PREDS NUMBER
LIKE_PREDS NUMBER
NULL_PREDS NUMBER
TIMESTAMP DATE
313

314. SQL> create table T ( 5 special
2 skew varchar2(10),
3 even number); values
Table created.
SQL> insert into T
2 select
3 case
4 when rownum > 99995 then 'SPECIAL'
5 else dbms_random.string('U',8)
6 end,
7 mod(rownum,200)
8 from dual
9 connect by level <= 100000 even
10 /
distribution
100000 rows created.
314

315. SQL> exec dbms_stats.gather_table_stats(
user,'T', estimate_percent=>null);
PL/SQL procedure successfully completed.
SQL> select COLUMN_NAME,NUM_DISTINCT,DENSITY,
2 ( select count(*)
3 from user_tab_histograms
4 where table_name = 'T'
5 and column_name = c.column_name ) hist_cnt
6 from user_tab_cols c
7 where table_name = 'T'
8 order by table_name,COLUMN_ID
9 /
COLUMN_NAME NUM_DISTINCT DENSITY HIST_CNT
------------------- ------------ ---------- ----------
SKEW 99996 .00001 2
EVEN 200 .005 2
315

316. SQL> select * from T where skew = 'SPECIAL';
SKEW VAL
---------- ----------
SPECIAL 196
SPECIAL 197
SPECIAL 198
SPECIAL 199
SPECIAL 0
5 rows selected.
SQL> select * from T where even = 5;
TAG VAL
---------- ----------
IBRXGVIE 5
[snip]
500 rows selected.
316

317. SQL> exec dbms_stats.gather_table_stats(
user,'T', estimate_percent=>null);
PL/SQL procedure successfully completed.
SQL> select COLUMN_NAME,NUM_DISTINCT,DENSITY,
2 ( select count(*)
3 from user_tab_histograms
4 where table_name = 'T'
5 and column_name = c.column_name ) hist_cnt
6 from user_tab_cols c
7 where table_name = 'T'
8 order by table_name,COLUMN_ID
9 /
COLUMN_NAME NUM_DISTINCT DENSITY HIST_CNT
------------------- ------------ ---------- ----------
SKEW 99996 .00001 2
EVEN 200 .000005 200
317

318. recommendation #5
318

319. set_schema_prefs
for all columns size 1
319

320. explicit control for each table
320

321. set_table_prefs
for column CCC size nnn
321

322. one more useful tool
322

323. 323

324. SQL> desc CONFERENCE_ATTENDEES
Name Null? Type
----------------------------- -------- --------------
PID NUMBER
GENDER CHAR(1)
NAME VARCHAR2(40)
AGE NUMBER
DEATH_RATE NUMBER
324

325. 325

326. SQL> select
2 count(distinct GENDER) NDV
3 from
4 CONFERENCE_ATTENDEES
NDV
----------------------
???
326

327. SQL> select
2 ntile(GENDER) over ( ... )
3 from
4 CONFERENCE_ATTENDEES
327

328. num_rows
set_table_stats
avg_row_len
328

329. high_value
set_column_stats
num_distinct
329

330. when all else fails ...
330

331. ... lie and cheat
331

332. 332

333. no
333

334. analyze ... validate structure
334

335. SQL> desc INDEX_STATS
Name Null? Type
----------------------------- -------- --------------
HEIGHT NUMBER
BLOCKS NUMBER
NAME VARCHAR2(30)
PARTITION_NAME VARCHAR2(30)
...
OPT_CMPR_COUNT NUMBER
OPT_CMPR_PCTSAVE NUMBER
335

336. wrap up
336

337. don't collect stats .... unless
don't collect system stats ... unless
don't collect histograms ... unless
default estimate size (NDV)
lie and cheat
337

338. Connor McDonald
OracleDBA
co.uk
338

339. ORA-00041
“active time limit exceeded - session terminated”
www.oracledba.co.uk 339

340. 11.2
340
340

341. actual versus estimate
341
341

342. SQL> select /*+ GATHER_PLAN_STATISTICS */ count(*)
2 from VEHICLE
3 where MAKE = 'HOLDEN'
4 and MODEL = 'COMMODORE';
COUNT(*)
----------
214468
SQL> SELECT *
2 FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(
3 NULL, NULL, 'ALLSTATS LAST'));
----------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows |
----------------------------------------------------------------
| 1 | SORT AGGREGATE | | 1 | 1 | 1 |
|* 2 | TABLE ACCESS FULL| VEHICLE | 1 | 220K| 214K|
-----------------------------------------------------------------
342
342

343. employ someone....
343
343

344. check every query...
344
344

345. SQL> select /*+ GATHER_PLAN_STATISTICS */ count(*)
2 from VEHICLE
3 where MAKE = 'HOLDEN'
4 and MODEL = 'COMMODORE';
COUNT(*)
----------
214468
SQL> SELECT *
2 FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(
3 NULL, NULL, 'ALLSTATS LAST'));
"ok"
-----------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows |
-----------------------------------------------------------------
| 1 | SORT AGGREGATE | | 1 | 1 | 1 |
|* 2 | TABLE ACCESS FULL| VEHICLE | 1 | 220K| 214K|
-----------------------------------------------------------------
345
345

346. 346
346

347. but just maybe ....
347
347

348. ... someone already is
348
348

349. very
very
cool
349

350. 100,000 rows
SQL> create table EMP as
2 select rownum empno,
3 mod(rownum,10) jobid,
4 mod(rownum,10)*1000 salary,
5 mod(rownum,50)+1 deptno
6 from dual
7 connect by rownum < 100000;
100 rows
SQL> create table DEPT as
2 select rownum deptno,
3 'dept'||rownum dname
4 from dual
5 connect by rownum <= 100;
350

351. SQL> exec dbms_stats.gather_table_stats(user,'EMP');
SQL> exec dbms_stats.gather_table_stats(user,'DEPT');
SQL> create index EMP_IX on EMP ( deptno );
SQL> create index DEPT_IX on DEPT ( deptno );
351

352. SQL> select e.empno, d.dname hard to
2 from emp e, dept d
3 where d.deptno = e.deptno optimize
4 and e.jobid = 1
5 and e.salary > 5000;
4 and e.jobid = 1
no rows selected
5 and e.salary > 5000;
----------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |
----------------------------------------------------------------
| 0 | SELECT STATEMENT | | | |
| 1 | MERGE JOIN | | 4444 | 104K |
| 2 | TABLE ACCESS BY INDEX ROWID| DEPT | 100 | 1000 |
| 3 | INDEX FULL SCAN | DEPT_IX | 100 | |
|* 4 | SORT JOIN | | 4444 | 62216 |
|* 5 | TABLE ACCESS FULL | EMP | 4444 | 62216 |
----------------------------------------------------------------
352

353. re-run the query
353

354. no changes to anything
354

355. SQL> select e.empno, d.dname
2 from emp e, dept d
3 where d.deptno = e.deptno
4 and e.jobid = 1
5 and e.salary > 5000;
no rows selected
---------------------------------------------------
| Id | Operation | Name | Rows | Bytes |
---------------------------------------------------
| 0 | SELECT STATEMENT | | | |
|* 1 | HASH JOIN | | 89 | 2136 |
| 2 | TABLE ACCESS FULL| DEPT | 1 | 10 |
|* 3 | TABLE ACCESS FULL| EMP | 4444 | 62216 |
---------------------------------------------------
355

356. 11.2
356

357. the optimizer knows what "hard" is
357

358. cardinality feedback loop
358

359. its not continuous
359

360. several restrictions
but still very cool
360