Feb14 successful development

839 views

Published on

Slides from Oracle Developer Day Perth

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
839
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Feb14 successful development

  1. 1. a good background in ...
  2. 2. successful application development...
  3. 3. ... in a mobile world
  4. 4. some controversy to start...
  5. 5. here's the problem... 7
  6. 6. ... it's us
  7. 7. a little digression on servers
  8. 8. a little bit of history ...
  9. 9. (not too) long ago
  10. 10. "front side bus" 13
  11. 11. any access to ... anything
  12. 12. CPU capped
  13. 13. why ?
  14. 14. hypertransport
  15. 15. memory access direct from CPU 22
  16. 16. "everyone wanted opteron" 25
  17. 17. ~2007
  18. 18. "Simply put, x86 systems are a lot more powerful than you think" - EMC presentation
  19. 19. source: OaktableWorld 2012
  20. 20. 50,000 SQL statements per second !
  21. 21. source: OaktableWorld 2012
  22. 22. read 5GB (bytes) I/O per second at 30% cpu load ! 33
  23. 23. source: twitter
  24. 24. write 500MB redo per second !
  25. 25. probably all you ever need 37
  26. 26. "why do I care?"
  27. 27. you can't blame the server 39
  28. 28. if you can't get that performance ...
  29. 29. ... it's you
  30. 30. 43
  31. 31. 44
  32. 32. all have their place...
  33. 33. "The sooner your company admits that you are not Google, the sooner you can get down to some real work" - Ted Dziuba 50
  34. 34. so what's holding you back ?
  35. 35. It all comes down to...
  36. 36. contention
  37. 37. and waste 54
  38. 38. this session ...
  39. 39. ... common causes
  40. 40. Part 1 sql processing
  41. 41. terminology 58
  42. 42. cursors
  43. 43. delete from MY_TABLE; drop table MY_TABLE; select * from EMPLOYEE where EMPNO > 1234; cursors declare cursor C(p number) is select * from DEPT where DEPTNO = p; begin for rec in C loop … end loop; end; begin MY_PROCEDURE(1,2,3); end;
  44. 44. all of them !
  45. 45. 3 phases
  46. 46. open process close 63
  47. 47. open gimme some memory process close 64
  48. 48. open gimme some memory process do some stuff using that memory, probably access other memory close 65
  49. 49. open gimme some memory process do some stuff using that memory, probably access other memory close here's your memory back 66
  50. 50. memory access = controlled access
  51. 51. woeful metaphor 61
  52. 52. 70
  53. 53. same with memory 64
  54. 54. SGA
  55. 55. latches 66
  56. 56. serialised access aka, someone waits 67 74
  57. 57. protected by SGA
  58. 58. 1) get latch protected by SGA
  59. 59. 1) get latch protected by SGA
  60. 60. 2) access memory protected by SGA
  61. 61. 2) access memory protected by SGA
  62. 62. 3) release latch protected by SGA 80
  63. 63. 3) release latch protected by SGA 81
  64. 64. latch contention
  65. 65. someone must wait ... protected by SGA
  66. 66. active wait
  67. 67. spinning 85
  68. 68. can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? spinning can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? 86
  69. 69. its importance to Oracle
  70. 70. early Oracle
  71. 71. 2000 times ! (_spin_count) 90
  72. 72. Zzzzzzzzzz....
  73. 73. yield mutex nowadays... posting CAS Good reading: http://andreynikolaev.wordpress.com/
  74. 74. latch contention....
  75. 75. hurts CPU... 94
  76. 76. hurts concurrency
  77. 77. you ... get ... nothing ... done
  78. 78. How Oracle runs your SQL
  79. 79. Syntax Validity Optimization Rowsourcing Execution (Fetch) 98
  80. 80. Syntax Validity Optimization SQL> Rowsourcing select * 2 frmo emp; Executionfrmo emp * ERROR at line 2: (Fetch) ORA-00923: FROM keyword not found where expected 99
  81. 81. Syntax Validity Optimization Rowsourcing Execution (Fetch) SQL> select empnoo 2 from emp; select empnoo * ERROR at line 1: ORA-00904: invalid column name
  82. 82. Syntax Validity Optimization Rowsourcing Execution (Fetch) PLAN ------------------------------------SELECT STATEMENT TABLE ACCESS BY INDEX ROWID EMP INDEX RANGE SCAN EMP_PK
  83. 83. Syntax Validity Optimization Rowsourcing Execution (Fetch) EMP_PK EMP
  84. 84. Syntax Validity Optimization Rowsourcing Execution (Fetch) 103
  85. 85. Syntax Validity Optimization Rowsourcing Execution (Fetch)
  86. 86. lots of preliminaries
  87. 87. “parsing”
  88. 88. dominant factor 107
  89. 89. execution time
  90. 90. CPU consumption
  91. 91. lots of memory access...
  92. 92. lots of latching ! 111
  93. 93. SQL> select name, gets 2 from v$latch 3 where name like 'library%' or name like 'shared%'; NAME GETS ------------------------------ ---------shared pool 333937 library cache load lock 0 shared pool simulator 7191 shared pool sim alloc 133 shared server configuration 6 shared server info 1
  94. 94. SQL> select name, gets 2 from v$latch 3 where name like 'library%' or name like 'shared%'; NAME GETS ------------------------------ ---------shared pool 333937 library cache load lock 0 shared pool simulator 7191 shared pool sim alloc 133 shared server configuration 6 shared server info 1 SQL> select 0 from dual; 0 ---------0
  95. 95. SQL> select name, gets 2 from v$latch 3 where name like 'library%' or name like 'shared%'; NAME GETS ------------------------------ ---------shared pool 333937 library cache load lock 0 shared pool simulator 7191 shared pool sim alloc 133 shared server configuration 6 shared server info 1 GETS ---------333991 0 7196 135 6 1 = 54 = 5 = 2 SQL> select 0 from dual; 0 ---------0 total = 945 !
  96. 96. impossible to avoid ?
  97. 97. library cache
  98. 98. previously executed statements 117
  99. 99. parse statement already in library cache ? reuse optimizer info reuse row source info
  100. 100. select surname, firstname from emp where empno = 123 probability of reuse low ? select * from dept where dname = ‘SALES’
  101. 101. select surname, firstname from emp where empno = 123 some queries are “nearly the same” select surname, firstname from emp where empno = 7632 120
  102. 102. binding
  103. 103. parse this... select surname, firstname from emp where empno = ? binding
  104. 104. parse this... select surname, firstname from emp where empno = ? binding now run it with ? = 7362 now run it with ? = 123
  105. 105. library cache select surname, firstname from emp where empno = ? select * from dept where deptno = ? select * from customer where locality = ? select prno from property where reference = ? select * from dept where deptno = ? select surname, firstname from emp where empno = ? select * from customer where locality = ? select * from organisation where officer = ? select surname, firstname from emp where empno = ? select * from dept where deptno = ? select offender from crimes where crno = ?
  106. 106. library cache select surname, firstname from emp where empno = ? select * from dept where deptno = ? select * from customer where locality = ? select prno from property where reference = ? select * from dept where deptno = ? select surname, firstname from emp where empno = ? select * from customer where locality = ? select * from organisation where officer = ? select surname, firstname from emp where empno = ? select * from dept where deptno = ? select offender from crimes where crno = ? Five full parses avoided
  107. 107. demo demo1
  108. 108. wow! 127
  109. 109. more complex = more parsing
  110. 110. SQL> declare 2 c number := dbms_sql.open_cursor; 3 begin 4 for i in 1 .. 10000 loop 5 dbms_sql.parse(c,' 6 select * from dba_objects o, dba_segments s 7 where object_id = '||i||' 8 and o.owner = s.owner 9 and o.object_name = s.segment_name' , 10 dbms_sql.native); 12 end loop; 12 dbms_sql.close_cursor(c); 13 end; more complex = more parsing Elapsed: 00:36:07.71 we aren’t even running the thing !
  111. 111. so binding really matters for high frequency SQL
  112. 112. SQL> 2 3 4 5 6 7 8 9 10 11 12 declare c number := dbms_sql.open_cursor; begin for i in 1 .. 6000 loop dbms_sql.parse(c,'select * from dba_objects where object_id = :x' , dbms_sql.native); dbms_sql.bind_variable(c,':x',i); end loop; dbms_sql.close_cursor(c); end; / PL/SQL procedure successfully completed. Elapsed: 00:00:00.57
  113. 113. we are STILL parsing 132
  114. 114. Syntax Validity Optimization Rowsourcing Execution (Fetch) Hard Parse not found in lib cache Soft Parse found
  115. 115. scott@mydb> select * 2 from EMP 3 where EMPNO = :b1 mike@mydb> select * 2 from EMP 3 where EMPNO = :b1
  116. 116. scott@mydb> select * 2 from EMP 3 where EMPNO = :b1 mike@mydb> select * 2 from EMP 3 where EMPNO = :b1 SQL> desc mike.emp Name Null? ----------------- -------EMPNO NOT NULL ENAME JOB Type -----------NUMBER(4) VARCHAR2(10) VARCHAR2(9)
  117. 117. parsing = memory access = latching
  118. 118. SQL> declare 2 c number; 3 begin 4 c := dbms_sql.open_cursor; 5 for i in 1 .. 1000 loop 6 dbms_sql.parse(c,'select * from dba_objects 7 where object_id = :x' , 8 dbms_sql.native); 9 dbms_sql.bind_variable(c,':x',i); 10 end loop; 12 dbms_sql.close_cursor(c); 12 end; latching hurts concurrency NAME COUNT ------------------------------ ---------library cache 10675 library cache pin 8430 library cache pin allocation 4145 shared pool 4390 137
  119. 119. option 1 lazy way :-)
  120. 120. session_cached_cursors = ...
  121. 121. lib cache select * from emp your SQL is always shared
  122. 122. lib cache select * from emp cursor your cursor points to it 141
  123. 123. lib cache select * from emp close cursor = lose pointer
  124. 124. cached cursor
  125. 125. close "ignored" 3rd execution
  126. 126. can we do even better ? 145
  127. 127. open gimme some memory process do some stuff using that memory, probably access other memory close here's your memory back
  128. 128. don’t close the cursor
  129. 129. hang on to the memory
  130. 130. remember what cursors we’ve used
  131. 131. reuse whenever possible
  132. 132. don’t exhaust memory
  133. 133. don’t allow invalid cursor reuse
  134. 134. sounds complicated ...
  135. 135. a free solution exists 154
  136. 136. plsql
  137. 137. create procedure MY_PROC is begin select * into … from dba_objects where object_id = my_plsql_variable ... end;
  138. 138. automatically uses bind variables static SQL in stored PLSQL hold’s cursors open (even if you close)
  139. 139. better cursor caching 158
  140. 140. procedure MY_PROC is begin select * into … from dept where … end; select * from dept …
  141. 141. procedure MY_PROC is begin select * into … from dept where … end; procedure MY_PROC is cursor c_emp is select * from emp; begin open c_emp; fetch c_emp into …; close c_emp; end; select * from dept …
  142. 142. procedure MY_PROC is begin select * into … from dept where … end; procedure MY_PROC is cursor c_emp is select * from emp; begin open c_emp; fetch c_emp into …; close c_emp; end; select * from dept … select * from emp
  143. 143. the future looks bleak Statement s = "select * from emp where empno = " + nEmpno
  144. 144. summary
  145. 145. avoid parsing costs for high frequency SQL 164
  146. 146. binding
  147. 147. using PLSQL is a good idea...
  148. 148. Part 2 PLSQL
  149. 149. here's the odd thing 168
  150. 150. plsql
  151. 151. 174
  152. 152. um ....
  153. 153. not true
  154. 154. packages
  155. 155. break the invalidation chain 178
  156. 156. proc A proc B proc C proc D pack A pack B pack C pack D body A body B body C body D
  157. 157. proc A proc B proc C proc D pack A pack B pack C pack D body A body B body C body D
  158. 158. proc A proc B proc C proc D pack A pack B pack C pack D body A body B body C body D
  159. 159. 11g and beyond
  160. 160. pack A pack B pack C body A body B pack D body C body D
  161. 161. edition based redefinition 12c 184
  162. 162. maybe a little bit true
  163. 163. ott since 9.2
  164. 164. client bind non-SQL datatypes 12.1
  165. 165. boolean records JPublisher / odp.net arrays 189
  166. 166. true …
  167. 167. only if you're IT shop is ignorant
  168. 168. 195
  169. 169. 196
  170. 170. 197
  171. 171. clone db free !
  172. 172. multitenant snapshot (not) free :-)
  173. 173. 3rd party solutions
  174. 174. there are no more excuses 201
  175. 175. other benefits
  176. 176. free dependency analysis DBA_DEPENDENCIES
  177. 177. definers / invokers Whitelists (12c) secure your data role enabled procedure (12c) encapsulation
  178. 178. Part 3 getting your data 205
  179. 179. efficient fetching
  180. 180. rows are stored in blocks
  181. 181. to read data, you read blocks...
  182. 182. from disk to memory to you 209
  183. 183. SQL> select * 2 from EMP 3 / Buffer Cache /d01/ts/my_data1.dat
  184. 184. why ?
  185. 185. disks are slow memory is fast
  186. 186. but memory access is complex 213
  187. 187. cache coherency
  188. 188. SQL> select * 2 from EMP 3 where … SQL> select max(EMP) 2 from DEPT SQL> delete 2 from DEPT 3 / SQL> update EMP 2 set … 3 where … SQL> select * 2 from EMP 3 where … SQL> insert into 2 EMP 3 values (…) SQL> select * 2 from CUSTOMER 3 /
  189. 189. we have to latch !
  190. 190. "give me a row" 217
  191. 191. "give me a row" • Get latch • Search along list of blocks in memory (or "chain") • Read block • Extract row of interest • Release latch • Give row back to client 218
  192. 192. lots of rows
  193. 193. lots of latching
  194. 194. typical program
  195. 195. OPEN c_emp_list LOOP FETCH c_emp_list INTO ... insert into EMP_COPY ( . . . ) values ( . . . ); END LOOP; 222
  196. 196. • • • • • • Get latch Walk along chain Get block Extract single row Release latch Give row back to client Row #1 OPEN c_emp_list LOOP FETCH c_emp_list INTO ... insert into EMP_COPY ( . . . ) values ( . . . ); END LOOP; 223
  197. 197. • • • • • • OPEN c_emp_list LOOP FETCH c_emp_list INTO ... insert into EMP_COPY ( . . . ) values ( . . . ); Get latch Walk along chain Get block Extract single row Release latch Give row back to client Row #1 • • • • • • Get latch (again) Walk along chain (again) Get block (again) Extract single row Release latch (again) Give row back to client (again) Row #2 END LOOP; 224
  198. 198. • • • • • • OPEN c_emp_list LOOP FETCH c_emp_list INTO ... insert into EMP_COPY ( . . . ) values ( . . . ); END LOOP; Get latch Walk along chain Get block Extract single row Release latch Give row back to client Row #1 • • • • • • Get latch (again) Walk along chain (again) Get block (again) Extract single row Release latch (again) Give row back to client (again) Row #2 • • • • • • Get latch (again) Walk along chain (again) Get block (again) Extract single row Release latch (again) Give row back to client (again) Row #3 etc etc etc 225
  199. 199. a better strategy.... “pinning”
  200. 200. "give me a row"
  201. 201. "give me a row" “by the way, I’ll be wanting several rows from this block”
  202. 202. "give me a row" “by the way, I’ll be wanting several rows from this block” • • • • • • • • • • • • Get latch Walk along chain Get block Pin the block Release latch Give row 1 back to client Give row 2 back to client … Give row n to client Get latch Walk along chain Remove my pin on the block Release latch
  203. 203. how to convey “I may want many rows”
  204. 204. SQL> set arraysize 10 231
  205. 205. PREFETCH=10
  206. 206. OCIAttrSet(… (ub4)OCI_ATTR_PREFETCH_ROWS, …)
  207. 207. FetchSize odp.net
  208. 208. “relational” not “rowlational” 235
  209. 209. easy way to do this ?
  210. 210. let plsql do it
  211. 211. bulk collect bulk bind
  212. 212. compare the performance 239
  213. 213. SQL> 2 3 4 5 6 7 8 9 10 11 12 13 14 15 declare n number; d date; cursor c is select empno, hiredate from big_emp; begin open c; loop fetch c into n,d; exit when c%notfound; end loop; close c; end; / 1,000,000 rows PL/SQL procedure successfully completed. Elapsed: 00:01:15.06
  214. 214. SQL> 2 3 4 5 6 7 8 9 10 11 12 13 declare cursor c is select empno, hiredate from big_emp; type c_list is table of c%rowtype; r c_list; begin open c; fetch c bulk collect into r; close c; end; / PL/SQL procedure successfully completed. Elapsed: 00:00:10.02
  215. 215. all that data goes somewhere 244
  216. 216. SQL> select * from v$mystats 2 where name like '%pga%' 3 / NAME VALUE ------------------------------ ---------session pga memory 101534672 session pga memory max 101534672
  217. 217. 100m of memory !
  218. 218. law of diminishing returns
  219. 219. pin rows on a block (8k) 248
  220. 220. SQL> declare . . . 9 begin 10 open c; 11 loop 12 fetch c 13 bulk collect 14 into r limit 100; 15 exit when c%notfound; . . . 19 / Elapsed: 00:00:08.03 NAME VALUE ------------------------------ ---------session pga memory 225912 session pga memory max 225912
  221. 221. it works both ways
  222. 222. compare the performance
  223. 223. SQL> begin 2 for i in 1 .. 50000 loop 3 insert into t1 values ( i ); 4 end loop; 5 end; 6 / PL/SQL procedure successfully completed. Elapsed: 00:00:11.03 252
  224. 224. SQL> 2 3 4 5 6 7 8 9 10 11 12 declare type num_list is table of number; n num_list := num_list(); begin n.extend(50000); for i in 1 .. 50000 loop n(i) := i; end loop; forall i in 1 .. 50000 insert into t1 values ( n(i) ); end; / PL/SQL procedure successfully completed. Elapsed: 00:00:00.76
  225. 225. remember our auditing trigger ?
  226. 226. SQL> 2 3 4 insert select from where into T owner, object_name all_objects rownum <= 10000; 10000 rows created. 213
  227. 227. SQL> insert into T 2 select owner, object_name insert into T 3 from all_objects select owner, object_name 4 all_objects rownum <= 10000; where from where rownum <= 10000 call cpu elapsed 10000 count ------- ---------rows created. ------- -----Parse 1 Execute 1 Fetch 0 ------- -----total 2 disk query current -------- --------- ---------0 0 0 88 123 10642 0 0 0 -------- --------- ---------88 123 10642 rows ---------0 10000 0 ---------10000 cpu elapsed disk query current ------- ---------- -------- --------- ---------0.00 0.00 0 0 0 0.79 0.97 2 109 10845 0.00 0.00 0 0 0 ------- ---------- -------- --------- ---------0.79 0.97 2 109 10845 rows ---------0 10000 0 ---------10000 0.01 0.00 3.10 3.05 0.00 0.00 ------- ---------3.12 3.06 INSERT INTO T_AUDIT VALUES (SYSDATE ,:B3 ,:B1 ,:B2 ) call count ------- -----Parse 1 Execute 10000 Fetch 0 ------- -----total 10001 213
  228. 228. create or replace package T_PKG is type each_row is record ( action varchar2(1), owner varchar2(30), object_name varchar2(30) ); type row_list is table of each_row index by pls_integer; g row_list; end; / 214 257
  229. 229. create or replace trigger AUDIT_TRG1 before insert or update or delete on T 215 258
  230. 230. create or replace trigger AUDIT_TRG1 before insert or update or delete on T begin t_pkg.g.delete; end; / 215 259
  231. 231. create or replace trigger AUDIT_TRG2 after insert or update or delete on T for each row 216
  232. 232. create or replace trigger AUDIT_TRG2 after insert or update or delete on T for each row begin if updating or inserting then t_pkg.g(t_pkg.g.count+1).owner t_pkg.g(t_pkg.g.count).object_name else t_pkg.g(t_pkg.g.count).owner t_pkg.g(t_pkg.g.count).object_name end if; end; / := :new.owner; := :new.object_name; := :old.owner; := :old.object_name; 216
  233. 233. create or replace trigger AUDIT_TRG3 after insert or update or delete on T 217
  234. 234. create or replace trigger AUDIT_TRG3 after insert or update or delete on T declare v_action varchar2(1) := case when updating then 'U' when deleting then 'D' else 'I' end; begin forall i in 1 .. t_pkg.g.count insert into T_AUDIT values ( sysdate, v_action, t_pkg.g(i).owner, t_pkg.g(i).object_name); t_pkg.g.delete; end; / 217
  235. 235. SQL> 2 3 4 insert select from where into T owner, object_name all_objects rownum <= 10000; 10000 rows created. 218
  236. 236. SQL> 2 insert 3 select 4 from where insert into T select owner, object_name into T from all_objects owner, object_name where all_objects rownum <= 10000; rownum <= 10000 call count cpu elapsed 10000 ------ ------- --------rows created. ------Parse 1 Execute 1 Fetch 0 ------- -----total 2 disk query current -------- ---------- ---------0 33 0 0 91 10653 0 0 0 -------- ---------- ---------0 124 10653 rows ---------0 10000 0 ---------10000 cpu elapsed disk query current ------- --------- -------- ---------- ---------0.00 0.00 0 0 0 0.04 0.03 0 90 478 0.00 0.00 0 0 0 ------- --------- -------- ---------- ---------0.04 0.03 0 90 478 rows ---------0 10000 0 ---------10000 218 0.00 0.00 0.56 0.58 0.00 0.00 ------- --------0.56 0.59 INSERT INTO T_AUDIT VALUES ( SYSDATE, :B1 , :B2 , :B3 ) call count ------- -----Parse 1 Execute 1 Fetch 0 ------- -----total 2
  237. 237. summary
  238. 238. fetch data in sets not rows
  239. 239. use bulk collect / bulk bind 268
  240. 240. a little known fetch 'feature'
  241. 241. dont forget this bit
  242. 242. SQL> set autotrace traceonly stat SQL> set arraysize 100 SQL> select object_type 2 from T; 89405 rows selected. Statistics --------------------------------------------4 recursive calls 0 db block gets 2452 consistent gets 0 physical reads 0 redo size 1198478 bytes sent via SQL*Net to client 271
  243. 243. SQL> select object_type 2 from T 3 order by object_type; 89405 rows selected. Statistics -------------------------------------------14 recursive calls 0 db block gets 1507 consistent gets 0 physical reads 0 redo size 618820 bytes sent via SQL*Net to client
  244. 244. SQL Net de-duplicate
  245. 245. ORDER BY matters !
  246. 246. Part 4 Read Consistency 275
  247. 247. “Readers don’t block writers" “Writers don't block readers"
  248. 248. Maximum concurrency… … very few locking worries
  249. 249. lack of understanding 279
  250. 250. corrupt your database
  251. 251. results are time consistent
  252. 252. “I’m a developer… …who cares about all that?”
  253. 253. So many ways…
  254. 254. 284
  255. 255. DIY replication
  256. 256. EMP DWEMP “every hour, populate data warehouse, copy of OLTP data” demo1a/b
  257. 257. DIY data integrity ... true story 287
  258. 258. Manufacturer Product
  259. 259. alter table add primary key Manufacturer Product alter table add foreign key
  260. 260. Manufacturer “ensure that every product corresponds to a valid manufacturer” Product
  261. 261. create or replace trigger PROD_MAN_CHECK before insert or update on PRODUCT for each row declare l_man_id number; begin select man_id into l_man_id from manufacturer where man_id = :new.man_id; exception when no_data_found then raise_application_error(-20000, 'Manufacturer is not valid'); end;
  262. 262. create or replace trigger MAN_PROD_CHECK before delete on MANUFACTURERS for each row declare l_prd_cnt number; begin select count(*) into l_prd_cnt from product where prd_id = :old.prd_id; if l_prd_cnt > 0 then raise_application_error(-20001, 'Manufacturer in use by product records'); end if; end;
  263. 263. testing “works” 293
  264. 264. SQL> insert into PRODUCT (...,MAN_ID,...) 2 values (...,100, ...); 1 row created. SQL> insert into PRODUCT (...,MAN_ID,...) 2 values (...,101,...); ERROR at line 1: ORA-20000: Manufacturer is not valid SQL> delete from MANUFACTURER 2 where man_id = 5768; ERROR at line 1: ORA-20001: Manufacturer in use by product records
  265. 265. one month later…
  266. 266. An unexpected error has occurred. Manufacturer record not found
  267. 267. SQL> select count(*) 2 from 3 ( select man_id from PRODUCT 4 minus 5 select man_id from MANUFACTURER ) 6 / COUNT(*) ---------86 297
  268. 268. DIY integrity = no integrity
  269. 269. delete MAN=7 Trigger fires - any PRD with MAN=7 - "No" (not yet) - OK commit Time create PRD with MAN=7 Trigger fires - does MAN=7 exist? - "Yes" - OK commit
  270. 270. referential integrity seems simple… “look up X before allowing Y” 300
  271. 271. referential integrity is complex read consistency, blocking / locking
  272. 272. “what percentage data integrity do we have?” - project manager 302
  273. 273. beware the module spec "When creating a product... validate registration date – ensure not in the future, validate … validate … validate manufacturer ID – must exist in Manufacturer table"
  274. 274. concurrency testing vital 304
  275. 275. summary
  276. 276. Oracle consistency model is …
  277. 277. very very cool… and
  278. 278. very very complex ! 308
  279. 279. (DIY) inter-table checking is hard to do right
  280. 280. (DIY) intra-table checking is hard to do right 310
  281. 281. let the database do it 311
  282. 282. Part 4 miscellaneous 312
  283. 283. excessive commit 313
  284. 284. "commits dont seem so bad" 314
  285. 285. SQL> begin 2 for i in 1 .. 30000 loop 3 insert into T1 4 values (i,i); 5 commit; 6 end loop; 7 end; 8 / PL/SQL procedure successfully completed. Elapsed: 00:00:05.40 315
  286. 286. another PLSQL trick no sync 316
  287. 287. SQL> begin 2 for i in 1 .. 30000 loop 3 insert into T1 4 values (i,i); 5 commit write immediate wait; 6 end loop; 7 end; 8 / PL/SQL procedure successfully completed. Elapsed: 00:00:24.73 317
  288. 288. commit to requirement 318
  289. 289. SQL> begin 2 for i in 1 .. 30000 loop 3 insert into T1 4 values (i,i); 6 7 8 end loop; end; / PL/SQL procedure successfully completed. Elapsed: 00:00:01.58 319
  290. 290. "special" values 320
  291. 291. SQL> create table T ( processed date ); Table created. SQL> 2 3 4 5 6 7 8 9 insert into T select case when mod(rownum,20) != 0 then to_date('01-JAN-11')+rownum/10 else null end from dual connect by level <= 10000; 10000 rows created. 321
  292. 292. SQL> select min(processed), max(processed), count(*) 2 from t; MIN(PROCE MAX(PROCE COUNT(*) --------- --------- ---------01-JAN-11 26-SEP-13 10000 10 rows per day across ~3 years 322
  293. 293. SQL> select * 2 from T 3 where processed between 4 '01-JUN-13' and '01-JUL-13'; -------------------------------------------| Id | Operation | Name | Rows | -------------------------------------------| 0 | SELECT STATEMENT | | 287 | |* 1 | FILTER | | | |* 2 | TABLE ACCESS FULL| T | 287 | -------------------------------------------- 323
  294. 294. SQL> select * 2 from T 3 where processed between 4 '01-JUN-13' and '01-JUL-13'; -------------------------------------------| Id | Operation | Name | Rows | -------------------------------------------| 0 | SELECT STATEMENT | | 287 | |* 1 | FILTER | | | |* 2 | TABLE ACCESS FULL| T | 287 | -------------------------------------------- reality = 285 324
  295. 295. "we don't like nulls"
  296. 296. SQL> truncate table T; Table truncated. SQL> 2 3 4 5 6 7 8 9 insert into T select case when mod(rownum,20) != 0 then to_date('01-JAN-11')+rownum/10 else to_date('31-DEC-3000') end from dual connect by level <= 10000; 10000 rows created. 326
  297. 297. SQL> select * 2 from T 3 where processed between 4 '01-JUN-13' and '01-JUL-13'; ------------------------------------------| Id | Operation | Name | Rows | ------------------------------------------| 0 | SELECT STATEMENT | | 3 | |* 1 | FILTER | | | |* 2 | TABLE ACCESS FULL| T | 3 | ------------------------------------------- uh oh...
  298. 298. if you're "stuck" 328
  299. 299. SQL> begin 2 dbms_stats.gather_table_stats('','T', 3 method_opt=>'for all columns size auto'); 4 end; 5 / PL/SQL procedure successfully completed. SQL> 2 3 4 select from where and HISTOGRAM user_tab_col_statistics table_name = 'T' column_name = 'PROCESSED'; HISTOGRAM --------------HYBRID
  300. 300. SQL> select * 2 from T 3 where processed between 4 '01-JUN-13' and '01-JUL-13'; ------------------------------------------| Id | Operation | Name | Rows | ------------------------------------------| 0 | SELECT STATEMENT | | 291 | |* 1 | FILTER | | | |* 2 | TABLE ACCESS FULL| T | 291 | ------------------------------------------- 330
  301. 301. lots of stuff we didn’t get to cover 331
  302. 302. wrap up 332
  303. 303. the hardware allows 333
  304. 304. "web scale" 334
  305. 305. minimise parsing intelligent commit as long as you are too set processing plsql 335
  306. 306. ORA-00041 “active time limit exceeded - session terminated” connormcdonald.wordpress.com 319

×