Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Databse & Technology 2 | Connor McDonald | Managing Optimiser Statistics - A better way.pdf

1,114 views

Published on

  • Be the first to comment

Databse & Technology 2 | Connor McDonald | Managing Optimiser Statistics - A better way.pdf

  1. 1. NOTE itty bitty fonts in this presentation SQL> exec sample_fontCan you read this ? 1
  2. 2. Connor McDonald OracleDBA co.uk 2
  3. 3. 3
  4. 4. bio slide 4
  5. 5. Connor McDonald
  6. 6. a funny story 6
  7. 7. 7
  8. 8. 8
  9. 9. 9
  10. 10. why ? 10
  11. 11. 12
  12. 12. life was simple 14
  13. 13. 15
  14. 14. (good) cost optimizer 16
  15. 15. 17
  16. 16. version 7.0 18
  17. 17. optimizer_mode = CHOOSE 19
  18. 18. "we choose RULE" 20
  19. 19. SQL> analyze table SALES estimate statistics; 21
  20. 20. 22
  21. 21. eventually.... 23
  22. 22. it got better 24
  23. 23. added more functionality 25
  24. 24. SQL> analyze table SALES estimate statistics 2 for table 3 for columns size 10 4 for ....; 26
  25. 25. SQL syntax engine 27
  26. 26. oracle 8i 28
  27. 27. DBMS_STATS 29
  28. 28. problem.... 30
  29. 29. SQL> analyze table SALES estimate statistics; 41 characters 31
  30. 30. SQL> begin 2 dbms_stats.gather_table_stats( 3 ownname=>HR, 4 tabname=>SALES, 5 cascade=>true, 6 estimate_percent=>20 7 ); 8 end; 128 characters 32
  31. 31. dont get chaining 33
  32. 32. s l o w e r 34
  33. 33. "I dont think so....." 35
  34. 34. times have changed ... 36
  35. 35. SQL> desc DBMS_STATSPROCEDURE ALTER_DATABASE_TAB_MONITORINGPROCEDURE ALTER_SCHEMA_TAB_MONITORINGPROCEDURE ALTER_STATS_HISTORY_RETENTIONPROCEDURE CLEANUP_STATS_JOB_PROCPROCEDURE CONVERT_RAW_VALUEPROCEDURE CONVERT_RAW_VALUEPROCEDURE CONVERT_RAW_VALUEPROCEDURE CONVERT_RAW_VALUEPROCEDURE CONVERT_RAW_VALUEPROCEDURE CONVERT_RAW_VALUE_NVARCHARPROCEDURE CONVERT_RAW_VALUE_ROWIDPROCEDURE COPY_TABLE_STATSFUNCTION CREATE_EXTENDED_STATS RETURNS VARCHARPROCEDURE CREATE_STAT_TABLEPROCEDURE DELETE_COLUMN_STATS 37
  36. 36. 126 routines ! 38
  37. 37. 39
  38. 38. profiles tuning setsbaselines features bind peeking adaptive cursor sharing 40
  39. 39. better stats needed ... 41
  40. 40. ... for the optimizer 42
  41. 41. hard to convince 43
  42. 42. 44
  43. 43. stop using analyze 45
  44. 44. ego 46
  45. 45. "good ol days" 47
  46. 46. 48
  47. 47. 49
  48. 48. today.... 50
  49. 49. optimizer is probably... 51
  50. 50. ... smarter than you 52
  51. 51. nested loop outer sort merge outer SQL> select count(e.hiredate) 2 from DEPT d, EMP e 3 where e.deptno = d.deptno(+) 4 and e.sal > 10; hash hash anti nested loop anti53
  52. 52. ------------------------------------------- | Id | Operation | Name | Rows | ------------------------------------------- | 0 | SELECT STATEMENT | | 1 | | 1 | SORT AGGREGATE | | 1 | |* 2 | TABLE ACCESS FULL| EMP | 14 | ------------------------------------------- no DEPT ?54
  53. 53. not null SQL> select count(e.hiredate) 2 from DEPT d, EMP e 3 where e.deptno = d.deptno(+) 4 and e.sal > 10; foreign key55 key preserved
  54. 54. Oracles solution.... 56
  55. 55. statistics by stealth... 57
  56. 56. 10g 58
  57. 57. SQL> select owner, job_name, enabled 2 from dba_scheduler_jobs 3 where owner = SYS;OWNER JOB_NAME ENABLED--------------- ------------------------------ -------SYS PURGE_LOG TRUESYS FGR$AUTOPURGE_JOB TRUESYS GATHER_STATS_JOB FALSE TRUESYS AUTO_SPACE_ADVISOR_JOB TRUE 59
  58. 58. 11g 60
  59. 59. SQL> select owner, job_name, enabled 2 from dba_scheduler_jobs 3 where owner = SYS;OWNER JOB_NAME ENABLED--------------- ------------------------------ -------SYS AUTO_SPACE_ADVISOR_JOB FALSESYS BSLN_MAINTAIN_STATS_JOB TRUESYS DRA_REEVALUATE_OPEN_FAILURES TRUESYS FGR$AUTOPURGE_JOB FALSESYS GATHER_STATS_JOB FALSESYS HM_CREATE_OFFLINE_DICTIONARY FALSESYS ORA$AUTOTASK_CLEAN TRUESYS PURGE_LOG TRUESYS XMLDB_NFS_CLEANUP_JOB FALSE 61
  60. 60. stats STILL being collected 62
  61. 61. automatic "tasks" "super stealth mode" 63
  62. 62. SQL> select client_name, status 2 from DBA_AUTOTASK_CLIENT; CLIENT_NAME STATUS ------------------------------------ -------- auto optimizer stats collection ENABLED auto space advisor ENABLED sql tuning advisor ENABLED64
  63. 63. stats every night most sites default options 65
  64. 64. in this session... 66
  65. 65. default behaviour 67
  66. 66. BADIDEA 68
  67. 67. collecting statistics 69
  68. 68. BADIDEA 70
  69. 69. but.... 71
  70. 70. some default behaviour 72
  71. 71. GOODIDEA 73
  72. 72. collecting some statistics 74
  73. 73. GOODIDEA 75
  74. 74. 76
  75. 75. Inflammatory statements which will alienate the audience 1 Presentation Duration 77
  76. 76. were all hypocrites 78
  77. 77. statistical hypocrisy 79
  78. 78. "I need to change somereference data in my system" 80
  79. 79. wrong case
  80. 80. "nope...." 82
  81. 81. ITIL 83
  82. 82. Information TechnologyInfrastructure Library 84
  83. 83. service call help desk DONE ! problem record
  84. 84. phew.... 86
  85. 85. corrected
  86. 86. same site.... 88
  87. 87. "Every night, I would like to potentially change theperformance characteristics of every single SQL statement in the database" 89
  88. 88. "no problem...." 90
  89. 89. collecting stats = risk 91
  90. 90. Ensors paradox 92
  91. 91. 1987performance group bstat/estat tkprof BMC patrol afiedt.buf 93
  92. 92. "It is only safe togather statistics ...""...when to do so willmake no difference" 94
  93. 93. recommendation #1 95
  94. 94. 96
  95. 95. unless things are bad...do not change statistics 97
  96. 96. "surely it cant hurt?" 98
  97. 97. 10g 99
  98. 98. options=>GATHER STALE 100
  99. 99. options=>GATHER EMPTY 101
  100. 100. options=>GATHER AUTO 102
  101. 101. "no plans changed" 103
  102. 102. "no queries ran slower" 104
  103. 103. still problems 105
  104. 104. problem # 1 106
  105. 105. dbms_stats 107
  106. 106. the goal of statistics 108
  107. 107. minimise expensive SQL.... 109
  108. 108. added more expensive SQL ! 110
  109. 109. SQL> alter session set sql_true = true;Session altered.SQL> begin 2 dbms_stats.gather_table_stats( 3 DEMO, 4 PEOPLE); 5 end; 6 / 111
  110. 110. SQL> select 2 count(distinct GENDER), 3 min(GENDER), 4 max(GENDER), 5 count(distinct NAME), 6 min(NAME), 7 max(NAME) ... ... 21 from PEOPLE; 112
  111. 111. hard ! 113
  112. 112. problem #2 115
  113. 113. lingering pain 116
  114. 114. invalidation 117
  115. 115. library cache 119
  116. 116. select * insert into PEOPLEfrom PEOPLE select * from ... select ... from PEOPLE, DEPT where ... SQL> begin 2 dbms_stats.gather_table_stats( 3 DEMO, 4 PEOPLE); 5 end; 6 / declare delete from T v people.name%type; where X in begin ( select PID ... from PEOPLE ) 120
  117. 117. 121
  118. 118. recap 122
  119. 119. gathering statistics really hard core SQL belted CPU with hard parsing everything ran the same 123
  120. 120. oracle 9 124
  121. 121. SQL> desc DBMS_STATSPROCEDURE GATHER_TABLE_STATS Argument Name Type In/Out Default? ----------------------- -------------- ------ -------- OWNNAME VARCHAR2 IN TABNAME VARCHAR2 IN PARTNAME VARCHAR2 IN DEFAULT ESTIMATE_PERCENT NUMBER IN DEFAULT BLOCK_SAMPLE BOOLEAN IN DEFAULT METHOD_OPT VARCHAR2 IN DEFAULT DEGREE NUMBER IN DEFAULT GRANULARITY VARCHAR2 IN DEFAULT CASCADE BOOLEAN IN DEFAULT STATTAB VARCHAR2 IN DEFAULT STATID VARCHAR2 IN DEFAULT STATOWN VARCHAR2 IN DEFAULT NO_INVALIDATE BOOLEAN IN DEFAULT STATTYPE VARCHAR2 IN DEFAULT FORCE BOOLEAN IN DEFAULT 125
  122. 122. ?no_invalidate => false | true ? 126
  123. 123. oracle 10 127
  124. 124. no_invalidate => auto 128
  125. 125. SQL> select 2 x.ksppinm name, 3 y.kspftctxvl value 4 from 5 sys.x$ksppi x, 6 sys.x$ksppcv2 y 7 where 8 x.indx+1 = y.kspftctxpn and 9 x.ksppinm = _optimizer_invalidation_periodNAME VALUE------------------------------ ------------_optimizer_invalidation_period 18000 129
  126. 126. select * insert into PEOPLEfrom PEOPLE select * from ... select ... from PEOPLE, DEPT where ... SQL> exec dbms_stats.gather_table_stats( user,PEOPLE); declare delete from T v people.name%type; where X in begin ( select PID ... from PEOPLE ) 130
  127. 127. better ? 131
  128. 128. hard parse storm avoided 132
  129. 129. 5 hours before problems 133
  130. 130. 134
  131. 131. unless things are bad...DO NOT CHANGE STATISTICS 135
  132. 132. Inflammatory statements which will alienate the audience 2 1 Presentation Duration 136
  133. 133. we dont know if things are bad 137
  134. 134. until its too late 138
  135. 135. users fed up SQLs are slow stats out of date 139
  136. 136. 140
  137. 137. 141
  138. 138. 2 hours to gather stats 142
  139. 139. 143
  140. 140. collect stats 144
  141. 141. in readiness 145
  142. 142. 146
  143. 143. pending statistics 147
  144. 144. "published" – optimizer uses "pending" - private 148
  145. 145. SQL> begin 2 dbms_stats.set_schema_prefs( 3 ownname=>DEMO, 4 pname=>PUBLISH, 5 pvalue=>FALSE); 6 end; database, table 149
  146. 146. SQL> alter session set 2 optimizer_use_pending_statistics = true; 150
  147. 147. SQL> begin 2 dbms_stats.publish_pending_stats( 3 ownname=>DEMO, 4 tabname=>null); 5 end; 151
  148. 148. recommendation #2 152
  149. 149. when collecting statistics 153
  150. 150. PUBLISH = false always.... 154
  151. 151. even if you immediately publish 155
  152. 152. atomic publication 156
  153. 153. and dont wait 5 hours 157
  154. 154. _optimizer_invalidation_period 158
  155. 155. for most tables 159
  156. 156. dbms_stats.lock_table_stats 160
  157. 157. collecting system stats 161
  158. 158. different story 162
  159. 159. absolutely vital… …to have them 163
  160. 160. maximize I/O throughput164
  161. 161. SQL> set autotrace traceonly explain SQL> select * from T; --------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| --------------------------------------------------------------- | 0 | SELECT STATEMENT | | 53459 | 4489K| 113 (1)| | 1 | TABLE ACCESS FULL| T | 53459 | 4489K| 113 (1)| --------------------------------------------------------------- SQL> select value 2 from v$spparameter 3 where name = db_file_multiblock_read_count; VALUE ----------- 16165
  162. 162. SQL> alter session set events 2 = 10046 trace name context forever, level 8; SQL> select * from T; PARSING IN CURSOR #22 len=15 dep=0 uid=124 oct=3 lid=124 ... select * from T END OF STMT PARSE #22:c=10000,e=108,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4 EXEC #22:c=0,e=65,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=4 WAIT #22: nam=db file scattered read ela= 3387 file#=5 block#=13122 blocks=16 obj#=199963 WAIT #22: nam=db file scattered read ela= 3188 file#=5 block#=13234 blocks=16 obj#=199963 WAIT #22: nam=db file scattered read ela= 3125 file#=5 block#=13250 blocks=16 obj#=199963 WAIT #22: nam=db file scattered read ela= 3255 file#=5 block#=13266 blocks=16 obj#=199963 WAIT #22: nam=db file scattered read ela= 3369 file#=5 block#=13282 blocks=16 obj#=199963166
  163. 163. SSTIOMAX167
  164. 164. 1 MB168
  165. 165. db_block_size = 8192169
  166. 166. db_file_multiblock_read_count = 128170
  167. 167. ?171
  168. 168. 172
  169. 169. the solution ?173
  170. 170. system stats174
  171. 171. SQL> select pname, pval1 2 from sys.aux_stats$ 3 / PNAME PVAL1 ------------------------------ ---------- SREADTIM 4.065 MREADTIM 6.173 CPUSPEED 567 MBRC 12 MAXTHR 48203776 SLAVETHR -1175
  172. 172. use MBRC to cost176
  173. 173. use db_file_multiblock_read_count to read177
  174. 174. Inflammatory statements 3 which will alienate the audience 2 1 Presentation Duration 178
  175. 175. forget about system statistics 179
  176. 176. do not use them 180
  177. 177. 181
  178. 178. 182
  179. 179. absolutely vital… …to have them 183
  180. 180. do not “use”* them * = collect, change, set 184
  181. 181. problem #1 185
  182. 182. 186
  183. 183. problem #2 187
  184. 184. monitoring for bad.... 188
  185. 185. optimizing for bad 189
  186. 186. recommendation #3 190
  187. 187. gather into STAT table 191
  188. 188. monitor 192
  189. 189. set once 193
  190. 190. the defaults are probably fine 194
  191. 191. SQL> select pname, pval1 2 from sys.aux_stats$ 3 / PNAME PVAL1 ---------------------- ---------- IOSEEKTIM 10 (ms) IOTFRSPEED 4096 (bytes/ms) SREADTIM = IOSEEKTIM + db_block_size / IOTFRSPEED = 10 MREADTIM = IOSEEKTIM + mbrc * db_block_size / IOTFRSPEED = 26195
  192. 192. 10.2 and above 196
  193. 193. _db_file_optimizer_read_countdb_file_multiblock_read_count _db_file_exec_read_count 197
  194. 194. dbms_resource_manager to calibrate 198
  195. 195. so far... 199
  196. 196. not changing statistics 200
  197. 197. eventually.... 201
  198. 198. the time will come... 202
  199. 199. one possible reason 203
  200. 200. even if stats unchanged … 204
  201. 201. … your plans might ! 205
  202. 202. SQL> create table T as 2 select to_date(01-AUG-2011)+ 3 trunc(dbms_random.value(0,7)) dte, 4 rpad(padding,20) padding 5 from dual 6 connect by level <= 100000;Table created.
  203. 203. SQL> select dte, count(*) 2 from t 3 group by dte 4 order by 1;DTE COUNT(*)--------- ----------01-AUG-11 1433402-AUG-11 1422203-AUG-11 1416704-AUG-11 1451005-AUG-11 1434606-AUG-11 1434907-AUG-11 14072
  204. 204. 15,000 rows per day always 208
  205. 205. SQL> exec dbms_stats.gather_table_stats(user,T)PL/SQL procedure successfully completed.SQL> create index IX on T ( dte ) ;Index created.
  206. 206. SQL> select * from t where dte = 03-AUG-2011;--------------------------------------------------| Id | Operation | Name | Rows | Bytes |--------------------------------------------------| 0 | SELECT STATEMENT | | 14286 | 404K||* 1 | TABLE ACCESS FULL| T | 14286 | 404K|--------------------------------------------------
  207. 207. a week later… 211
  208. 208. SQL> insert into T 2 select to_date(08-AUG-2011)+ 3 trunc(dbms_random.value(0,7)) dte, 4 rpad(padding,20) padding 5 from dual 6 connect by level <= 100000;100000 rows created.
  209. 209. SQL> select dte, count(*) 2 from t 3 group by dte 4 order by 1;DTE COUNT(*)--------- ----------01-AUG-11 14334 ...07-AUG-11 1407208-AUG-11 1426109-AUG-11 1410610-AUG-11 1441011-AUG-11 1428912-AUG-11 1435813-AUG-11 1425214-AUG-11 14324
  210. 210. statistics unchanged 214
  211. 211. SQL> select * from t where dte = 12-AUG-2011;----------------------------------------------------| Id | Operation | Name | Rows |----------------------------------------------------| 0 | SELECT STATEMENT | | 2381 || 1 | TABLE ACCESS BY INDEX ROWID| T | 2381 ||* 2 | INDEX RANGE SCAN | IX | 2381 |----------------------------------------------------
  212. 212. SQL> select * from t where dte = 14-AUG-2011;----------------------------------------------------| Id | Operation | Name | Rows |----------------------------------------------------| 0 | SELECT STATEMENT | | 1 || 1 | TABLE ACCESS BY INDEX ROWID| T | 1 ||* 2 | INDEX RANGE SCAN | IX | 1 |----------------------------------------------------
  213. 213. why ? 217
  214. 214. SQL> select low_value, high_value 2 from user_tab_columns 3 where table_name = T 4 and column_name = DTE;LOW_VALUE HIGH_VALUE-------------------- -----------------786F0801010101 786F0807010101
  215. 215. SQL> set serverout onSQL> declare 2 res date; 3 begin 4 dbms_stats.convert_raw_value( 5 786F0801010101,res); 6 dbms_output.put_line(result); 7 dbms_stats.convert_raw_value( 8 786F0806010101,res); 9 dbms_output.put_line(result); 10 end; 11 /01-AUG-1106-AUG-11
  216. 216. 01-AUG-11 06-AUG-11
  217. 217. when the time comes… 221
  218. 218. as accurate as possible 222
  219. 219. as cheaply as possible 223
  220. 220. statistics accurately 224
  221. 221. extended statistics 225
  222. 222. 226
  223. 223. 227
  224. 224. cardinality is everything228 228
  225. 225. same with Oracle229 229
  226. 226. some real(ish) data230 230
  227. 227. SQL> desc VEHICLE Name Null? Type -------------------------- -------- ------------- ID NUMBER MAKE VARCHAR2(6) MODEL VARCHAR2(6) SQL> select count(*) 2 from VEHICLE; COUNT(*) ------------ 2,157,079231 231
  228. 228. default stats not enough232 232
  229. 229. SQL> select count(*) 2 from VEHICLE 3 where MAKE = HOLDEN; COUNT(*)---------- 415387------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost |------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 7 | 138|| 1 | SORT AGGREGATE | | 1 | 7 | ||* 2 | INDEX RANGE SCAN| MAKE_IX | 55310 | 378K| 138|------------------------------------------------------------233 233
  230. 230. histogram234 234
  231. 231. SQL> begin 2 dbms_stats.gather_table_stats(user,VEHICLE, 3 method_opt=>for all columns size 1,|| 4 for columns MAKE size 254,|| 5 for columns MODEL size 254); 6 end; 7 / PL/SQL procedure successfully completed.235 235
  232. 232. SQL> select count(*) 2 from VEHICLE 3 where MAKE = HOLDEN; COUNT(*)---------- 415387-----------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost |-----------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 7 | 1024|| 1 | SORT AGGREGATE | | 1 | 7 | ||* 2 | INDEX RANGE SCAN| MAKE_IX | 418K| 2859K| 1024|-----------------------------------------------------------236 236
  233. 233. make AND model237 237
  234. 234. SQL> select count(*) 2 from VEHICLE 3 where MAKE = HOLDEN 4 and MODEL = COMMODORE; COUNT(*) 50% holdens are commodores---------- 214468--------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes |---------------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 14 || 1 | SORT AGGREGATE | | 1 | 14 || 2 | BITMAP CONVERSION COUNT | | 39527 | 540K|| 3 | BITMAP AND | | | || 4 | BITMAP CONVERSION FROM ROWIDS| | | ||* 5 | INDEX RANGE SCAN | MODEL_IX | | || 6 | BITMAP CONVERSION FROM ROWIDS| | | ||* 7 | INDEX RANGE SCAN | MAKE_IX | | |--------------------------------------------------------------------- 238 238
  235. 235. two things239 239
  236. 236. 240
  237. 237. 241 241
  238. 238. no correlation 10g and before242 242
  239. 239. 11g243 243
  240. 240. SQL> select 2 DBMS_STATS.CREATE_EXTENDED_STATS( 3 user, VEHICLE,(MAKE,MODEL)) tag 4 from dual; TAG ---------------------------------- SYS_STU8QPK2S$PEWHARK2CP3#1F#G244 244
  241. 241. SQL> begin 2 dbms_stats.gather_table_stats(user,VEHICLE, 3 method_opt=> 4 for columns (make,model) size 254); 5 end; 6 / PL/SQL procedure successfully completed.245 245
  242. 242. SQL> select count(*) 2 from VEHICLE 3 where MAKE = HOLDEN 4 and MODEL = COMMODORE; COUNT(*)---------- 214468-------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost |------------------------------------------------------------| 0 | SELECT STATEMENT | | 1 | 14 | 1956|| 1 | SORT AGGREGATE | | 1 | 14 | ||* 2 | TABLE ACCESS FULL| VEHICLE | 220K| 3018K| 1956|------------------------------------------------------------- 246 246
  243. 243. composite indexes247 247
  244. 244. 11.2 column groups248 248
  245. 245. SQL> exec DBMS_STATS.SEED_COL_USAGE(NULL,NULL,60); 249 249
  246. 246. implemented via virtual columns250 250
  247. 247. various implications251 251
  248. 248. accurate stats not always possible... 252
  249. 249. you cant have stats 253
  250. 250. on everything ! 254
  251. 251. when stats wont do... 255
  252. 252. ... use the real data 256
  253. 253. dynamic sampling 257
  254. 254. SQL> drop table TOUGH_DATA purge;Table dropped.SQL> create table TOUGH_DATA nologging as 2 select 3 rownum pk, 4 dbms_random.string(U,10) str 5 from dual 6 connect by level < 1000000 7 /Table created.SQL> exec dbms_stats.gather_table_stats( user,TOUGH_DATA) 258
  255. 255. SQL> select count(*) 2 from TOUGH_DATA 3 where str like %XX 4 / COUNT(*) hard---------- 1452 259
  256. 256. SQL> select count(*) 2 from TOUGH_DATA 3 where str like %XX 4 /-------------------------------------------------| Id | Operation | Name | Rows |-------------------------------------------------| 0 | SELECT STATEMENT | | 1 || 1 | SORT AGGREGATE | | 1 ||* 2 | TABLE ACCESS FULL| TOUGH_DATA | 50000 |------------------------------------------------- 5% assumption 260
  257. 257. SQL> select /*+ dynamic_sampling(t 2) */ count(*) 2 from TOUGH_DATA t 3 where str like %XX 4 /-------------------------------------------------| Id | Operation | Name | Rows |-------------------------------------------------| 0 | SELECT STATEMENT | | 1 || 1 | SORT AGGREGATE | | 1 ||* 2 | TABLE ACCESS FULL| TOUGH_DATA | 1252 |------------------------------------------------- 261
  258. 258. simple queries as well 262
  259. 259. 1,2,3,4,5,6...... A,A,A,A,A,B,B,B,B,.....SQL> create table EASY_DATA nologging as 2 select 3 rownum pk, 4 chr(65+trunc(rownum/40000)) str 5 from dual 6 connect by level < 1000000 7 /Table created.SQL> exec dbms_stats.gather_table_stats(user,EASY_DATA)PL/SQL procedure successfully completed. 263
  260. 260. SQL> select count(*) 2 from EASY_DATA 3 where str = F 4 and pk > 900000; COUNT(*)---------- 0 264
  261. 261. SQL> set autotrace traceonly explainSQL> select count(*) 2 from EASY_DATA 3 where str = F 4 and pk > 900000;------------------------------------------------| Id | Operation | Name | Rows |------------------------------------------------| 0 | SELECT STATEMENT | | 1 || 1 | SORT AGGREGATE | | 1 ||* 2 | TABLE ACCESS FULL| EASY_DATA | 4000 |------------------------------------------------ 265
  262. 262. SQL> set autotrace traceonly explainSQL> select /*+ dynamic_sampling(t 2) */ count(*) 2 from EASY_DATA t 3 where str = F 4 and pk > 900000;------------------------------------------------| Id | Operation | Name | Rows |------------------------------------------------| 0 | SELECT STATEMENT | | 1 || 1 | SORT AGGREGATE | | 1 ||* 2 | TABLE ACCESS FULL| EASY_DATA | 23 |------------------------------------------------ 266
  263. 263. 11.2 267
  264. 264. parallel queries (maybe) sampled 268
  265. 265. SQL> select count(*) 2 from EASY_DATA t 3 where str = F 4 and pk > 900000;----------------------------------------------------| Id | Operation | Name | Rows |----------------------------------------------------| 0 | SELECT STATEMENT | | 1 || 1 | SORT AGGREGATE | | 1 || 2 | PX COORDINATOR | | || 3 | PX SEND QC (RANDOM) | :TQ10000 | 1 || 4 | SORT AGGREGATE | | 1 || 5 | PX BLOCK ITERATOR | | 4 ||* 6 | TABLE ACCESS FULL| EASY_DATA | 4 |----------------------------------------------------Note----- - dynamic sampling used for this statement (level=4) 269
  266. 266. low frequency 270
  267. 267. high cost 271
  268. 268. statistics cheaply 272
  269. 269. SQL> desc DBA_TABLESName Null? Type----------------------------- -------- -------------OWNER NOT NULL VARCHAR2(30)TABLE_NAME NOT NULL VARCHAR2(30)COLUMN_NAME NOT NULL VARCHAR2(30)...NUM_ROWS NUMBER 273
  270. 270. SQL> desc DBA_TAB_COLSName Null? Type----------------------------- -------- -------------OWNER NOT NULL VARCHAR2(30)TABLE_NAME NOT NULL VARCHAR2(30)COLUMN_NAME NOT NULL VARCHAR2(30)...NUM_DISTINCT NUMBERLOW_VALUE RAW(32)HIGH_VALUE RAW(32)... 274
  271. 271. SQL> desc PEOPLEName Null? Type----------------------------- -------- -------------PID NUMBERGENDER CHAR(1)NAME VARCHAR2(47)AGE NUMBERDEATH_RATE NUMBER 275
  272. 272. SQL> alter session set sql_true = true;Session altered.SQL> begin 2 dbms_stats.gather_table_stats( 3 DEMO, 4 PEOPLE); 5 end; 6 / 276
  273. 273. select /*+ no_parallel(t) no_parallel_index(t) dbms_stats cursor_sharing_exact use_weak_name_resl dynamic_sampling(0) no_monitoring no_substrb_pad */ count(*), ... count("GENDER") count("GENDER"), "GENDER") count(distinct "GENDER"), min("GENDER") substrb(dump(min("GENDER"),16,0,32),1,120), substrb(dump(max("GENDER"),16,0,32),1,120), max("GENDER") ...from "MCDONAC"."PEOPLE" t 277
  274. 274. count("GENDER") one passcount(distinct "GENDER") hardmin("GENDER") one passmax("GENDER") one pass 278
  275. 275. SQL> select count(*) from PEOPLE; COUNT(*)------------ 500000000Elapsed: 00:04:43.73 279
  276. 276. SQL> begin 2 dbms_stats.gather_table_stats( 3 user, 4 PEOPLE, 5 estimate_percent=>25); 6 end; 7 /ERROR at line 1:ORA-01652: unable to extend tempsegment by 128 in tablespace TEMPElapsed: 00:16:37.12 280
  277. 277. 11g 281
  278. 278. one pass NDV 282
  279. 279. HASH12 6 3 12 3 7 11 12 33 6 11 12 6 45 15 7 15 17 45 6 1712 6 3 7 11 33 45 15 17 283
  280. 280. 12 6 3 7 11 33 45 15 17 NDV = 9 284
  281. 281. 91 11 34 12 7 15 6 64 33 56 92 what about large NDV ? 21 5 61 45 3 2 17 7173 41 285
  282. 282. "Magic" HASH 286
  283. 283. NDV =remaining hashes x2^number of splits 287
  284. 284. demo 288
  285. 285. SQL> create table ONE_PASS nologging 2 as select substr(text,1,1) single_char 3 from DBA_SOURCE;Table created.SQL> select count(distinct single_char) ndv 2 from one_pass; NDV---------- 83 289
  286. 286. 0 127 16 buckets... 290
  287. 287. 64 127 291
  288. 288. 96 127 292
  289. 289. SQL> set serverout onSQL> declare 2 type t_bucket is table of varchar2(1); 3 l_synopsis t_bucket; 4 l_splits number := 0; 5 l_hash int; 6 l_min_val int := 0; 7 l_synopsis_size int := 16; 8 begin 9 for i in ( select single_char from one_pass ) loop 10 l_hash := ascii(i.single_char); 11 12 if l_synopsis.count = l_synopsis_size then 13 l_min_val := 14 case 15 when l_min_val = 0 then 64 16 when l_min_val = 64 then 96 17 when l_min_val = 96 then 112 18 when l_min_val = 112 then 120 19 end; 20 l_splits := l_splits + 1; 293
  290. 290. 22 23 for j in 1 .. l_min_val loop 24 if l_synopsis.exists(j) then 25 l_synopsis.delete(j); 26 end if; 27 end loop; 28 end if; 29 30 if l_hash > l_min_val then 31 l_synopsis(l_hash) := Y; 32 end if; 33 end loop; 34 dbms_output.put_line(l_synopsis.count * 35 power(2,l_splits)); 36 end; 37 /Splitting, keeping entries above 64Splitting, keeping entries above 96Splitting, keeping entries above 11288 294
  291. 291. 18,446,744,073,709,551,616 hash range the reality 16384 bucket limit 295
  292. 292. SQL> begin 2 dbms_stats.gather_table_stats( 3 user, 4 PEOPLE, 5 estimate_percent=>25); 6 end; 7 /PL/SQL procedure successfully completed.Elapsed: 00:05:39.82 296
  293. 293. recommendation #4 297
  294. 294. estimate_percentuse DEFAULT size (AUTO) 298
  295. 295. only for columnswithout histograms 299
  296. 296. 4Inflammatory statements 3 which will alienate the audience 2 1 Presentation Duration 300
  297. 297. histograms... 301
  298. 298. ...SUCK 302
  299. 299. not really but... 303
  300. 300. 9i 304
  301. 301. SQL> set autotrace traceonly explainSQL> select * from MY_TABLE_100K_ROWS 2 where r = :b1 3 /------------------------------------------| Id | Operation | Name | Rows |------------------------------------------| 0 | SELECT STATEMENT | | 1 || 1 | INDEX UNIQUE SCAN| PK | 1 |------------------------------------------ ~ num rows / num distinct 306
  302. 302. SQL> set autotrace traceonly explainSQL> select * from MY_TABLE_100K_ROWS 2 where r < :b1 3 /-----------------------------------------| Id | Operation | Name | Rows |-----------------------------------------| 0 | SELECT STATEMENT | | 5000 || 1 | INDEX RANGE SCAN| PK | 5000 |----------------------------------------- 5% 307
  303. 303. 308
  304. 304. cool ? 309
  305. 305. 10g 310
  306. 306. method_optfor all columns size auto 311
  307. 307. sys.col_usage$ 312
  308. 308. SQL> desc SYS.COL_USAGE$Name Null? Type----------------------- -------- ---------OBJ# NUMBERINTCOL# NUMBEREQUALITY_PREDS NUMBEREQUIJOIN_PREDS NUMBERNONEQUIJOIN_PREDS NUMBERRANGE_PREDS NUMBERLIKE_PREDS NUMBERNULL_PREDS NUMBERTIMESTAMP DATE 313
  309. 309. SQL> create table T ( 5 special 2 skew varchar2(10), 3 even number); valuesTable created.SQL> insert into T 2 select 3 case 4 when rownum > 99995 then SPECIAL 5 else dbms_random.string(U,8) 6 end, 7 mod(rownum,200) 8 from dual 9 connect by level <= 100000 even 10 / distribution100000 rows created. 314
  310. 310. SQL> exec dbms_stats.gather_table_stats( user,T, estimate_percent=>null);PL/SQL procedure successfully completed.SQL> select COLUMN_NAME,NUM_DISTINCT,DENSITY, 2 ( select count(*) 3 from user_tab_histograms 4 where table_name = T 5 and column_name = c.column_name ) hist_cnt 6 from user_tab_cols c 7 where table_name = T 8 order by table_name,COLUMN_ID 9 /COLUMN_NAME NUM_DISTINCT DENSITY HIST_CNT------------------- ------------ ---------- ----------SKEW 99996 .00001 2EVEN 200 .005 2 315
  311. 311. SQL> select * from T where skew = SPECIAL;SKEW VAL---------- ----------SPECIAL 196SPECIAL 197SPECIAL 198SPECIAL 199SPECIAL 05 rows selected.SQL> select * from T where even = 5;TAG VAL---------- ----------IBRXGVIE 5[snip]500 rows selected. 316
  312. 312. SQL> exec dbms_stats.gather_table_stats( user,T, estimate_percent=>null);PL/SQL procedure successfully completed.SQL> select COLUMN_NAME,NUM_DISTINCT,DENSITY, 2 ( select count(*) 3 from user_tab_histograms 4 where table_name = T 5 and column_name = c.column_name ) hist_cnt 6 from user_tab_cols c 7 where table_name = T 8 order by table_name,COLUMN_ID 9 /COLUMN_NAME NUM_DISTINCT DENSITY HIST_CNT------------------- ------------ ---------- ----------SKEW 99996 .00001 2EVEN 200 .000005 200 317
  313. 313. recommendation #5 318
  314. 314. set_schema_prefsfor all columns size 1 319
  315. 315. explicit control for each table 320
  316. 316. set_table_prefsfor column CCC size nnn 321
  317. 317. one more useful tool 322
  318. 318. 323
  319. 319. SQL> desc CONFERENCE_ATTENDEES Name Null? Type ----------------------------- -------- -------------- PID NUMBER GENDER CHAR(1) NAME VARCHAR2(40) AGE NUMBER DEATH_RATE NUMBER 324
  320. 320. 325
  321. 321. SQL> select 2 count(distinct GENDER) NDV 3 from 4 CONFERENCE_ATTENDEES NDV---------------------- ??? 326
  322. 322. SQL> select 2 ntile(GENDER) over ( ... ) 3 from 4 CONFERENCE_ATTENDEES 327
  323. 323. num_rowsset_table_stats avg_row_len 328
  324. 324. high_valueset_column_stats num_distinct 329
  325. 325. when all else fails ... 330
  326. 326. ... lie and cheat 331
  327. 327. 332
  328. 328. no 333
  329. 329. analyze ... validate structure 334
  330. 330. SQL> desc INDEX_STATS Name Null? Type ----------------------------- -------- -------------- HEIGHT NUMBER BLOCKS NUMBER NAME VARCHAR2(30) PARTITION_NAME VARCHAR2(30) ... OPT_CMPR_COUNT NUMBER OPT_CMPR_PCTSAVE NUMBER 335
  331. 331. wrap up 336
  332. 332. dont collect stats .... unlessdont collect system stats ... unlessdont collect histograms ... unless default estimate size (NDV) lie and cheat 337
  333. 333. Connor McDonald OracleDBA co.uk 338
  334. 334. ORA-00041“active time limit exceeded - session terminated” www.oracledba.co.uk 339
  335. 335. 11.2340 340
  336. 336. actual versus estimate341 341
  337. 337. SQL> select /*+ GATHER_PLAN_STATISTICS */ count(*) 2 from VEHICLE 3 where MAKE = HOLDEN 4 and MODEL = COMMODORE; COUNT(*)---------- 214468SQL> SELECT * 2 FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR( 3 NULL, NULL, ALLSTATS LAST));----------------------------------------------------------------| Id | Operation | Name | Starts | E-Rows | A-Rows |----------------------------------------------------------------| 1 | SORT AGGREGATE | | 1 | 1 | 1 ||* 2 | TABLE ACCESS FULL| VEHICLE | 1 | 220K| 214K|----------------------------------------------------------------- 342 342
  338. 338. employ someone....343 343
  339. 339. check every query...344 344
  340. 340. SQL> select /*+ GATHER_PLAN_STATISTICS */ count(*) 2 from VEHICLE 3 where MAKE = HOLDEN 4 and MODEL = COMMODORE; COUNT(*)---------- 214468SQL> SELECT * 2 FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR( 3 NULL, NULL, ALLSTATS LAST)); "ok"-----------------------------------------------------------------| Id | Operation | Name | Starts | E-Rows | A-Rows |-----------------------------------------------------------------| 1 | SORT AGGREGATE | | 1 | 1 | 1 ||* 2 | TABLE ACCESS FULL| VEHICLE | 1 | 220K| 214K|----------------------------------------------------------------- 345 345
  341. 341. 346 346
  342. 342. but just maybe ....347 347
  343. 343. ... someone already is348 348
  344. 344. very very cool 349
  345. 345. 100,000 rowsSQL> create table EMP as 2 select rownum empno, 3 mod(rownum,10) jobid, 4 mod(rownum,10)*1000 salary, 5 mod(rownum,50)+1 deptno 6 from dual 7 connect by rownum < 100000; 100 rowsSQL> create table DEPT as 2 select rownum deptno, 3 dept||rownum dname 4 from dual 5 connect by rownum <= 100; 350
  346. 346. SQL> exec dbms_stats.gather_table_stats(user,EMP);SQL> exec dbms_stats.gather_table_stats(user,DEPT);SQL> create index EMP_IX on EMP ( deptno );SQL> create index DEPT_IX on DEPT ( deptno ); 351
  347. 347. SQL> select e.empno, d.dname hard to 2 from emp e, dept d 3 where d.deptno = e.deptno optimize 4 and e.jobid = 1 5 and e.salary > 5000; 4 and e.jobid = 1no rows selected 5 and e.salary > 5000;----------------------------------------------------------------| Id | Operation | Name | Rows | Bytes |----------------------------------------------------------------| 0 | SELECT STATEMENT | | | || 1 | MERGE JOIN | | 4444 | 104K || 2 | TABLE ACCESS BY INDEX ROWID| DEPT | 100 | 1000 || 3 | INDEX FULL SCAN | DEPT_IX | 100 | ||* 4 | SORT JOIN | | 4444 | 62216 ||* 5 | TABLE ACCESS FULL | EMP | 4444 | 62216 |---------------------------------------------------------------- 352
  348. 348. re-run the query 353
  349. 349. no changes to anything 354
  350. 350. SQL> select e.empno, d.dname 2 from emp e, dept d 3 where d.deptno = e.deptno 4 and e.jobid = 1 5 and e.salary > 5000;no rows selected---------------------------------------------------| Id | Operation | Name | Rows | Bytes |---------------------------------------------------| 0 | SELECT STATEMENT | | | ||* 1 | HASH JOIN | | 89 | 2136 || 2 | TABLE ACCESS FULL| DEPT | 1 | 10 ||* 3 | TABLE ACCESS FULL| EMP | 4444 | 62216 |--------------------------------------------------- 355
  351. 351. 11.2 356
  352. 352. the optimizer knows what "hard" is 357
  353. 353. cardinality feedback loop 358
  354. 354. its not continuous 359
  355. 355. several restrictions but still very cool 360

×