• Save
Feb14 successful development
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Feb14 successful development

on

  • 701 views

Slides from Oracle Developer Day Perth

Slides from Oracle Developer Day Perth

Statistics

Views

Total Views
701
Views on SlideShare
701
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Feb14 successful development Presentation Transcript

  • 1. a good background in ...
  • 2. successful application development...
  • 3. ... in a mobile world
  • 4. some controversy to start...
  • 5. here's the problem... 7
  • 6. ... it's us
  • 7. a little digression on servers
  • 8. a little bit of history ...
  • 9. (not too) long ago
  • 10. "front side bus" 13
  • 11. any access to ... anything
  • 12. CPU capped
  • 13. why ?
  • 14. hypertransport
  • 15. memory access direct from CPU 22
  • 16. "everyone wanted opteron" 25
  • 17. ~2007
  • 18. "Simply put, x86 systems are a lot more powerful than you think" - EMC presentation
  • 19. source: OaktableWorld 2012
  • 20. 50,000 SQL statements per second !
  • 21. source: OaktableWorld 2012
  • 22. read 5GB (bytes) I/O per second at 30% cpu load ! 33
  • 23. source: twitter
  • 24. write 500MB redo per second !
  • 25. probably all you ever need 37
  • 26. "why do I care?"
  • 27. you can't blame the server 39
  • 28. if you can't get that performance ...
  • 29. ... it's you
  • 30. 43
  • 31. 44
  • 32. all have their place...
  • 33. "The sooner your company admits that you are not Google, the sooner you can get down to some real work" - Ted Dziuba 50
  • 34. so what's holding you back ?
  • 35. It all comes down to...
  • 36. contention
  • 37. and waste 54
  • 38. this session ...
  • 39. ... common causes
  • 40. Part 1 sql processing
  • 41. terminology 58
  • 42. cursors
  • 43. delete from MY_TABLE; drop table MY_TABLE; select * from EMPLOYEE where EMPNO > 1234; cursors declare cursor C(p number) is select * from DEPT where DEPTNO = p; begin for rec in C loop … end loop; end; begin MY_PROCEDURE(1,2,3); end;
  • 44. all of them !
  • 45. 3 phases
  • 46. open process close 63
  • 47. open gimme some memory process close 64
  • 48. open gimme some memory process do some stuff using that memory, probably access other memory close 65
  • 49. open gimme some memory process do some stuff using that memory, probably access other memory close here's your memory back 66
  • 50. memory access = controlled access
  • 51. woeful metaphor 61
  • 52. 70
  • 53. same with memory 64
  • 54. SGA
  • 55. latches 66
  • 56. serialised access aka, someone waits 67 74
  • 57. protected by SGA
  • 58. 1) get latch protected by SGA
  • 59. 1) get latch protected by SGA
  • 60. 2) access memory protected by SGA
  • 61. 2) access memory protected by SGA
  • 62. 3) release latch protected by SGA 80
  • 63. 3) release latch protected by SGA 81
  • 64. latch contention
  • 65. someone must wait ... protected by SGA
  • 66. active wait
  • 67. spinning 85
  • 68. can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? spinning can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? 86
  • 69. its importance to Oracle
  • 70. early Oracle
  • 71. 2000 times ! (_spin_count) 90
  • 72. Zzzzzzzzzz....
  • 73. yield mutex nowadays... posting CAS Good reading: http://andreynikolaev.wordpress.com/
  • 74. latch contention....
  • 75. hurts CPU... 94
  • 76. hurts concurrency
  • 77. you ... get ... nothing ... done
  • 78. How Oracle runs your SQL
  • 79. Syntax Validity Optimization Rowsourcing Execution (Fetch) 98
  • 80. Syntax Validity Optimization SQL> Rowsourcing select * 2 frmo emp; Executionfrmo emp * ERROR at line 2: (Fetch) ORA-00923: FROM keyword not found where expected 99
  • 81. Syntax Validity Optimization Rowsourcing Execution (Fetch) SQL> select empnoo 2 from emp; select empnoo * ERROR at line 1: ORA-00904: invalid column name
  • 82. Syntax Validity Optimization Rowsourcing Execution (Fetch) PLAN ------------------------------------SELECT STATEMENT TABLE ACCESS BY INDEX ROWID EMP INDEX RANGE SCAN EMP_PK
  • 83. Syntax Validity Optimization Rowsourcing Execution (Fetch) EMP_PK EMP
  • 84. Syntax Validity Optimization Rowsourcing Execution (Fetch) 103
  • 85. Syntax Validity Optimization Rowsourcing Execution (Fetch)
  • 86. lots of preliminaries
  • 87. “parsing”
  • 88. dominant factor 107
  • 89. execution time
  • 90. CPU consumption
  • 91. lots of memory access...
  • 92. lots of latching ! 111
  • 93. SQL> select name, gets 2 from v$latch 3 where name like 'library%' or name like 'shared%'; NAME GETS ------------------------------ ---------shared pool 333937 library cache load lock 0 shared pool simulator 7191 shared pool sim alloc 133 shared server configuration 6 shared server info 1
  • 94. SQL> select name, gets 2 from v$latch 3 where name like 'library%' or name like 'shared%'; NAME GETS ------------------------------ ---------shared pool 333937 library cache load lock 0 shared pool simulator 7191 shared pool sim alloc 133 shared server configuration 6 shared server info 1 SQL> select 0 from dual; 0 ---------0
  • 95. SQL> select name, gets 2 from v$latch 3 where name like 'library%' or name like 'shared%'; NAME GETS ------------------------------ ---------shared pool 333937 library cache load lock 0 shared pool simulator 7191 shared pool sim alloc 133 shared server configuration 6 shared server info 1 GETS ---------333991 0 7196 135 6 1 = 54 = 5 = 2 SQL> select 0 from dual; 0 ---------0 total = 945 !
  • 96. impossible to avoid ?
  • 97. library cache
  • 98. previously executed statements 117
  • 99. parse statement already in library cache ? reuse optimizer info reuse row source info
  • 100. select surname, firstname from emp where empno = 123 probability of reuse low ? select * from dept where dname = ‘SALES’
  • 101. select surname, firstname from emp where empno = 123 some queries are “nearly the same” select surname, firstname from emp where empno = 7632 120
  • 102. binding
  • 103. parse this... select surname, firstname from emp where empno = ? binding
  • 104. parse this... select surname, firstname from emp where empno = ? binding now run it with ? = 7362 now run it with ? = 123
  • 105. library cache select surname, firstname from emp where empno = ? select * from dept where deptno = ? select * from customer where locality = ? select prno from property where reference = ? select * from dept where deptno = ? select surname, firstname from emp where empno = ? select * from customer where locality = ? select * from organisation where officer = ? select surname, firstname from emp where empno = ? select * from dept where deptno = ? select offender from crimes where crno = ?
  • 106. library cache select surname, firstname from emp where empno = ? select * from dept where deptno = ? select * from customer where locality = ? select prno from property where reference = ? select * from dept where deptno = ? select surname, firstname from emp where empno = ? select * from customer where locality = ? select * from organisation where officer = ? select surname, firstname from emp where empno = ? select * from dept where deptno = ? select offender from crimes where crno = ? Five full parses avoided
  • 107. demo demo1
  • 108. wow! 127
  • 109. more complex = more parsing
  • 110. SQL> declare 2 c number := dbms_sql.open_cursor; 3 begin 4 for i in 1 .. 10000 loop 5 dbms_sql.parse(c,' 6 select * from dba_objects o, dba_segments s 7 where object_id = '||i||' 8 and o.owner = s.owner 9 and o.object_name = s.segment_name' , 10 dbms_sql.native); 12 end loop; 12 dbms_sql.close_cursor(c); 13 end; more complex = more parsing Elapsed: 00:36:07.71 we aren’t even running the thing !
  • 111. so binding really matters for high frequency SQL
  • 112. SQL> 2 3 4 5 6 7 8 9 10 11 12 declare c number := dbms_sql.open_cursor; begin for i in 1 .. 6000 loop dbms_sql.parse(c,'select * from dba_objects where object_id = :x' , dbms_sql.native); dbms_sql.bind_variable(c,':x',i); end loop; dbms_sql.close_cursor(c); end; / PL/SQL procedure successfully completed. Elapsed: 00:00:00.57
  • 113. we are STILL parsing 132
  • 114. Syntax Validity Optimization Rowsourcing Execution (Fetch) Hard Parse not found in lib cache Soft Parse found
  • 115. scott@mydb> select * 2 from EMP 3 where EMPNO = :b1 mike@mydb> select * 2 from EMP 3 where EMPNO = :b1
  • 116. scott@mydb> select * 2 from EMP 3 where EMPNO = :b1 mike@mydb> select * 2 from EMP 3 where EMPNO = :b1 SQL> desc mike.emp Name Null? ----------------- -------EMPNO NOT NULL ENAME JOB Type -----------NUMBER(4) VARCHAR2(10) VARCHAR2(9)
  • 117. parsing = memory access = latching
  • 118. SQL> declare 2 c number; 3 begin 4 c := dbms_sql.open_cursor; 5 for i in 1 .. 1000 loop 6 dbms_sql.parse(c,'select * from dba_objects 7 where object_id = :x' , 8 dbms_sql.native); 9 dbms_sql.bind_variable(c,':x',i); 10 end loop; 12 dbms_sql.close_cursor(c); 12 end; latching hurts concurrency NAME COUNT ------------------------------ ---------library cache 10675 library cache pin 8430 library cache pin allocation 4145 shared pool 4390 137
  • 119. option 1 lazy way :-)
  • 120. session_cached_cursors = ...
  • 121. lib cache select * from emp your SQL is always shared
  • 122. lib cache select * from emp cursor your cursor points to it 141
  • 123. lib cache select * from emp close cursor = lose pointer
  • 124. cached cursor
  • 125. close "ignored" 3rd execution
  • 126. can we do even better ? 145
  • 127. open gimme some memory process do some stuff using that memory, probably access other memory close here's your memory back
  • 128. don’t close the cursor
  • 129. hang on to the memory
  • 130. remember what cursors we’ve used
  • 131. reuse whenever possible
  • 132. don’t exhaust memory
  • 133. don’t allow invalid cursor reuse
  • 134. sounds complicated ...
  • 135. a free solution exists 154
  • 136. plsql
  • 137. create procedure MY_PROC is begin select * into … from dba_objects where object_id = my_plsql_variable ... end;
  • 138. automatically uses bind variables static SQL in stored PLSQL hold’s cursors open (even if you close)
  • 139. better cursor caching 158
  • 140. procedure MY_PROC is begin select * into … from dept where … end; select * from dept …
  • 141. procedure MY_PROC is begin select * into … from dept where … end; procedure MY_PROC is cursor c_emp is select * from emp; begin open c_emp; fetch c_emp into …; close c_emp; end; select * from dept …
  • 142. procedure MY_PROC is begin select * into … from dept where … end; procedure MY_PROC is cursor c_emp is select * from emp; begin open c_emp; fetch c_emp into …; close c_emp; end; select * from dept … select * from emp
  • 143. the future looks bleak Statement s = "select * from emp where empno = " + nEmpno
  • 144. summary
  • 145. avoid parsing costs for high frequency SQL 164
  • 146. binding
  • 147. using PLSQL is a good idea...
  • 148. Part 2 PLSQL
  • 149. here's the odd thing 168
  • 150. plsql
  • 151. 174
  • 152. um ....
  • 153. not true
  • 154. packages
  • 155. break the invalidation chain 178
  • 156. proc A proc B proc C proc D pack A pack B pack C pack D body A body B body C body D
  • 157. proc A proc B proc C proc D pack A pack B pack C pack D body A body B body C body D
  • 158. proc A proc B proc C proc D pack A pack B pack C pack D body A body B body C body D
  • 159. 11g and beyond
  • 160. pack A pack B pack C body A body B pack D body C body D
  • 161. edition based redefinition 12c 184
  • 162. maybe a little bit true
  • 163. ott since 9.2
  • 164. client bind non-SQL datatypes 12.1
  • 165. boolean records JPublisher / odp.net arrays 189
  • 166. true …
  • 167. only if you're IT shop is ignorant
  • 168. 195
  • 169. 196
  • 170. 197
  • 171. clone db free !
  • 172. multitenant snapshot (not) free :-)
  • 173. 3rd party solutions
  • 174. there are no more excuses 201
  • 175. other benefits
  • 176. free dependency analysis DBA_DEPENDENCIES
  • 177. definers / invokers Whitelists (12c) secure your data role enabled procedure (12c) encapsulation
  • 178. Part 3 getting your data 205
  • 179. efficient fetching
  • 180. rows are stored in blocks
  • 181. to read data, you read blocks...
  • 182. from disk to memory to you 209
  • 183. SQL> select * 2 from EMP 3 / Buffer Cache /d01/ts/my_data1.dat
  • 184. why ?
  • 185. disks are slow memory is fast
  • 186. but memory access is complex 213
  • 187. cache coherency
  • 188. SQL> select * 2 from EMP 3 where … SQL> select max(EMP) 2 from DEPT SQL> delete 2 from DEPT 3 / SQL> update EMP 2 set … 3 where … SQL> select * 2 from EMP 3 where … SQL> insert into 2 EMP 3 values (…) SQL> select * 2 from CUSTOMER 3 /
  • 189. we have to latch !
  • 190. "give me a row" 217
  • 191. "give me a row" • Get latch • Search along list of blocks in memory (or "chain") • Read block • Extract row of interest • Release latch • Give row back to client 218
  • 192. lots of rows
  • 193. lots of latching
  • 194. typical program
  • 195. OPEN c_emp_list LOOP FETCH c_emp_list INTO ... insert into EMP_COPY ( . . . ) values ( . . . ); END LOOP; 222
  • 196. • • • • • • Get latch Walk along chain Get block Extract single row Release latch Give row back to client Row #1 OPEN c_emp_list LOOP FETCH c_emp_list INTO ... insert into EMP_COPY ( . . . ) values ( . . . ); END LOOP; 223
  • 197. • • • • • • OPEN c_emp_list LOOP FETCH c_emp_list INTO ... insert into EMP_COPY ( . . . ) values ( . . . ); Get latch Walk along chain Get block Extract single row Release latch Give row back to client Row #1 • • • • • • Get latch (again) Walk along chain (again) Get block (again) Extract single row Release latch (again) Give row back to client (again) Row #2 END LOOP; 224
  • 198. • • • • • • OPEN c_emp_list LOOP FETCH c_emp_list INTO ... insert into EMP_COPY ( . . . ) values ( . . . ); END LOOP; Get latch Walk along chain Get block Extract single row Release latch Give row back to client Row #1 • • • • • • Get latch (again) Walk along chain (again) Get block (again) Extract single row Release latch (again) Give row back to client (again) Row #2 • • • • • • Get latch (again) Walk along chain (again) Get block (again) Extract single row Release latch (again) Give row back to client (again) Row #3 etc etc etc 225
  • 199. a better strategy.... “pinning”
  • 200. "give me a row"
  • 201. "give me a row" “by the way, I’ll be wanting several rows from this block”
  • 202. "give me a row" “by the way, I’ll be wanting several rows from this block” • • • • • • • • • • • • Get latch Walk along chain Get block Pin the block Release latch Give row 1 back to client Give row 2 back to client … Give row n to client Get latch Walk along chain Remove my pin on the block Release latch
  • 203. how to convey “I may want many rows”
  • 204. SQL> set arraysize 10 231
  • 205. PREFETCH=10
  • 206. OCIAttrSet(… (ub4)OCI_ATTR_PREFETCH_ROWS, …)
  • 207. FetchSize odp.net
  • 208. “relational” not “rowlational” 235
  • 209. easy way to do this ?
  • 210. let plsql do it
  • 211. bulk collect bulk bind
  • 212. compare the performance 239
  • 213. SQL> 2 3 4 5 6 7 8 9 10 11 12 13 14 15 declare n number; d date; cursor c is select empno, hiredate from big_emp; begin open c; loop fetch c into n,d; exit when c%notfound; end loop; close c; end; / 1,000,000 rows PL/SQL procedure successfully completed. Elapsed: 00:01:15.06
  • 214. SQL> 2 3 4 5 6 7 8 9 10 11 12 13 declare cursor c is select empno, hiredate from big_emp; type c_list is table of c%rowtype; r c_list; begin open c; fetch c bulk collect into r; close c; end; / PL/SQL procedure successfully completed. Elapsed: 00:00:10.02
  • 215. all that data goes somewhere 244
  • 216. SQL> select * from v$mystats 2 where name like '%pga%' 3 / NAME VALUE ------------------------------ ---------session pga memory 101534672 session pga memory max 101534672
  • 217. 100m of memory !
  • 218. law of diminishing returns
  • 219. pin rows on a block (8k) 248
  • 220. SQL> declare . . . 9 begin 10 open c; 11 loop 12 fetch c 13 bulk collect 14 into r limit 100; 15 exit when c%notfound; . . . 19 / Elapsed: 00:00:08.03 NAME VALUE ------------------------------ ---------session pga memory 225912 session pga memory max 225912
  • 221. it works both ways
  • 222. compare the performance
  • 223. SQL> begin 2 for i in 1 .. 50000 loop 3 insert into t1 values ( i ); 4 end loop; 5 end; 6 / PL/SQL procedure successfully completed. Elapsed: 00:00:11.03 252
  • 224. SQL> 2 3 4 5 6 7 8 9 10 11 12 declare type num_list is table of number; n num_list := num_list(); begin n.extend(50000); for i in 1 .. 50000 loop n(i) := i; end loop; forall i in 1 .. 50000 insert into t1 values ( n(i) ); end; / PL/SQL procedure successfully completed. Elapsed: 00:00:00.76
  • 225. remember our auditing trigger ?
  • 226. SQL> 2 3 4 insert select from where into T owner, object_name all_objects rownum <= 10000; 10000 rows created. 213
  • 227. SQL> insert into T 2 select owner, object_name insert into T 3 from all_objects select owner, object_name 4 all_objects rownum <= 10000; where from where rownum <= 10000 call cpu elapsed 10000 count ------- ---------rows created. ------- -----Parse 1 Execute 1 Fetch 0 ------- -----total 2 disk query current -------- --------- ---------0 0 0 88 123 10642 0 0 0 -------- --------- ---------88 123 10642 rows ---------0 10000 0 ---------10000 cpu elapsed disk query current ------- ---------- -------- --------- ---------0.00 0.00 0 0 0 0.79 0.97 2 109 10845 0.00 0.00 0 0 0 ------- ---------- -------- --------- ---------0.79 0.97 2 109 10845 rows ---------0 10000 0 ---------10000 0.01 0.00 3.10 3.05 0.00 0.00 ------- ---------3.12 3.06 INSERT INTO T_AUDIT VALUES (SYSDATE ,:B3 ,:B1 ,:B2 ) call count ------- -----Parse 1 Execute 10000 Fetch 0 ------- -----total 10001 213
  • 228. create or replace package T_PKG is type each_row is record ( action varchar2(1), owner varchar2(30), object_name varchar2(30) ); type row_list is table of each_row index by pls_integer; g row_list; end; / 214 257
  • 229. create or replace trigger AUDIT_TRG1 before insert or update or delete on T 215 258
  • 230. create or replace trigger AUDIT_TRG1 before insert or update or delete on T begin t_pkg.g.delete; end; / 215 259
  • 231. create or replace trigger AUDIT_TRG2 after insert or update or delete on T for each row 216
  • 232. create or replace trigger AUDIT_TRG2 after insert or update or delete on T for each row begin if updating or inserting then t_pkg.g(t_pkg.g.count+1).owner t_pkg.g(t_pkg.g.count).object_name else t_pkg.g(t_pkg.g.count).owner t_pkg.g(t_pkg.g.count).object_name end if; end; / := :new.owner; := :new.object_name; := :old.owner; := :old.object_name; 216
  • 233. create or replace trigger AUDIT_TRG3 after insert or update or delete on T 217
  • 234. create or replace trigger AUDIT_TRG3 after insert or update or delete on T declare v_action varchar2(1) := case when updating then 'U' when deleting then 'D' else 'I' end; begin forall i in 1 .. t_pkg.g.count insert into T_AUDIT values ( sysdate, v_action, t_pkg.g(i).owner, t_pkg.g(i).object_name); t_pkg.g.delete; end; / 217
  • 235. SQL> 2 3 4 insert select from where into T owner, object_name all_objects rownum <= 10000; 10000 rows created. 218
  • 236. SQL> 2 insert 3 select 4 from where insert into T select owner, object_name into T from all_objects owner, object_name where all_objects rownum <= 10000; rownum <= 10000 call count cpu elapsed 10000 ------ ------- --------rows created. ------Parse 1 Execute 1 Fetch 0 ------- -----total 2 disk query current -------- ---------- ---------0 33 0 0 91 10653 0 0 0 -------- ---------- ---------0 124 10653 rows ---------0 10000 0 ---------10000 cpu elapsed disk query current ------- --------- -------- ---------- ---------0.00 0.00 0 0 0 0.04 0.03 0 90 478 0.00 0.00 0 0 0 ------- --------- -------- ---------- ---------0.04 0.03 0 90 478 rows ---------0 10000 0 ---------10000 218 0.00 0.00 0.56 0.58 0.00 0.00 ------- --------0.56 0.59 INSERT INTO T_AUDIT VALUES ( SYSDATE, :B1 , :B2 , :B3 ) call count ------- -----Parse 1 Execute 1 Fetch 0 ------- -----total 2
  • 237. summary
  • 238. fetch data in sets not rows
  • 239. use bulk collect / bulk bind 268
  • 240. a little known fetch 'feature'
  • 241. dont forget this bit
  • 242. SQL> set autotrace traceonly stat SQL> set arraysize 100 SQL> select object_type 2 from T; 89405 rows selected. Statistics --------------------------------------------4 recursive calls 0 db block gets 2452 consistent gets 0 physical reads 0 redo size 1198478 bytes sent via SQL*Net to client 271
  • 243. SQL> select object_type 2 from T 3 order by object_type; 89405 rows selected. Statistics -------------------------------------------14 recursive calls 0 db block gets 1507 consistent gets 0 physical reads 0 redo size 618820 bytes sent via SQL*Net to client
  • 244. SQL Net de-duplicate
  • 245. ORDER BY matters !
  • 246. Part 4 Read Consistency 275
  • 247. “Readers don’t block writers" “Writers don't block readers"
  • 248. Maximum concurrency… … very few locking worries
  • 249. lack of understanding 279
  • 250. corrupt your database
  • 251. results are time consistent
  • 252. “I’m a developer… …who cares about all that?”
  • 253. So many ways…
  • 254. 284
  • 255. DIY replication
  • 256. EMP DWEMP “every hour, populate data warehouse, copy of OLTP data” demo1a/b
  • 257. DIY data integrity ... true story 287
  • 258. Manufacturer Product
  • 259. alter table add primary key Manufacturer Product alter table add foreign key
  • 260. Manufacturer “ensure that every product corresponds to a valid manufacturer” Product
  • 261. create or replace trigger PROD_MAN_CHECK before insert or update on PRODUCT for each row declare l_man_id number; begin select man_id into l_man_id from manufacturer where man_id = :new.man_id; exception when no_data_found then raise_application_error(-20000, 'Manufacturer is not valid'); end;
  • 262. create or replace trigger MAN_PROD_CHECK before delete on MANUFACTURERS for each row declare l_prd_cnt number; begin select count(*) into l_prd_cnt from product where prd_id = :old.prd_id; if l_prd_cnt > 0 then raise_application_error(-20001, 'Manufacturer in use by product records'); end if; end;
  • 263. testing “works” 293
  • 264. SQL> insert into PRODUCT (...,MAN_ID,...) 2 values (...,100, ...); 1 row created. SQL> insert into PRODUCT (...,MAN_ID,...) 2 values (...,101,...); ERROR at line 1: ORA-20000: Manufacturer is not valid SQL> delete from MANUFACTURER 2 where man_id = 5768; ERROR at line 1: ORA-20001: Manufacturer in use by product records
  • 265. one month later…
  • 266. An unexpected error has occurred. Manufacturer record not found
  • 267. SQL> select count(*) 2 from 3 ( select man_id from PRODUCT 4 minus 5 select man_id from MANUFACTURER ) 6 / COUNT(*) ---------86 297
  • 268. DIY integrity = no integrity
  • 269. delete MAN=7 Trigger fires - any PRD with MAN=7 - "No" (not yet) - OK commit Time create PRD with MAN=7 Trigger fires - does MAN=7 exist? - "Yes" - OK commit
  • 270. referential integrity seems simple… “look up X before allowing Y” 300
  • 271. referential integrity is complex read consistency, blocking / locking
  • 272. “what percentage data integrity do we have?” - project manager 302
  • 273. beware the module spec "When creating a product... validate registration date – ensure not in the future, validate … validate … validate manufacturer ID – must exist in Manufacturer table"
  • 274. concurrency testing vital 304
  • 275. summary
  • 276. Oracle consistency model is …
  • 277. very very cool… and
  • 278. very very complex ! 308
  • 279. (DIY) inter-table checking is hard to do right
  • 280. (DIY) intra-table checking is hard to do right 310
  • 281. let the database do it 311
  • 282. Part 4 miscellaneous 312
  • 283. excessive commit 313
  • 284. "commits dont seem so bad" 314
  • 285. SQL> begin 2 for i in 1 .. 30000 loop 3 insert into T1 4 values (i,i); 5 commit; 6 end loop; 7 end; 8 / PL/SQL procedure successfully completed. Elapsed: 00:00:05.40 315
  • 286. another PLSQL trick no sync 316
  • 287. SQL> begin 2 for i in 1 .. 30000 loop 3 insert into T1 4 values (i,i); 5 commit write immediate wait; 6 end loop; 7 end; 8 / PL/SQL procedure successfully completed. Elapsed: 00:00:24.73 317
  • 288. commit to requirement 318
  • 289. SQL> begin 2 for i in 1 .. 30000 loop 3 insert into T1 4 values (i,i); 6 7 8 end loop; end; / PL/SQL procedure successfully completed. Elapsed: 00:00:01.58 319
  • 290. "special" values 320
  • 291. SQL> create table T ( processed date ); Table created. SQL> 2 3 4 5 6 7 8 9 insert into T select case when mod(rownum,20) != 0 then to_date('01-JAN-11')+rownum/10 else null end from dual connect by level <= 10000; 10000 rows created. 321
  • 292. SQL> select min(processed), max(processed), count(*) 2 from t; MIN(PROCE MAX(PROCE COUNT(*) --------- --------- ---------01-JAN-11 26-SEP-13 10000 10 rows per day across ~3 years 322
  • 293. SQL> select * 2 from T 3 where processed between 4 '01-JUN-13' and '01-JUL-13'; -------------------------------------------| Id | Operation | Name | Rows | -------------------------------------------| 0 | SELECT STATEMENT | | 287 | |* 1 | FILTER | | | |* 2 | TABLE ACCESS FULL| T | 287 | -------------------------------------------- 323
  • 294. SQL> select * 2 from T 3 where processed between 4 '01-JUN-13' and '01-JUL-13'; -------------------------------------------| Id | Operation | Name | Rows | -------------------------------------------| 0 | SELECT STATEMENT | | 287 | |* 1 | FILTER | | | |* 2 | TABLE ACCESS FULL| T | 287 | -------------------------------------------- reality = 285 324
  • 295. "we don't like nulls"
  • 296. SQL> truncate table T; Table truncated. SQL> 2 3 4 5 6 7 8 9 insert into T select case when mod(rownum,20) != 0 then to_date('01-JAN-11')+rownum/10 else to_date('31-DEC-3000') end from dual connect by level <= 10000; 10000 rows created. 326
  • 297. SQL> select * 2 from T 3 where processed between 4 '01-JUN-13' and '01-JUL-13'; ------------------------------------------| Id | Operation | Name | Rows | ------------------------------------------| 0 | SELECT STATEMENT | | 3 | |* 1 | FILTER | | | |* 2 | TABLE ACCESS FULL| T | 3 | ------------------------------------------- uh oh...
  • 298. if you're "stuck" 328
  • 299. SQL> begin 2 dbms_stats.gather_table_stats('','T', 3 method_opt=>'for all columns size auto'); 4 end; 5 / PL/SQL procedure successfully completed. SQL> 2 3 4 select from where and HISTOGRAM user_tab_col_statistics table_name = 'T' column_name = 'PROCESSED'; HISTOGRAM --------------HYBRID
  • 300. SQL> select * 2 from T 3 where processed between 4 '01-JUN-13' and '01-JUL-13'; ------------------------------------------| Id | Operation | Name | Rows | ------------------------------------------| 0 | SELECT STATEMENT | | 291 | |* 1 | FILTER | | | |* 2 | TABLE ACCESS FULL| T | 291 | ------------------------------------------- 330
  • 301. lots of stuff we didn’t get to cover 331
  • 302. wrap up 332
  • 303. the hardware allows 333
  • 304. "web scale" 334
  • 305. minimise parsing intelligent commit as long as you are too set processing plsql 335
  • 306. ORA-00041 “active time limit exceeded - session terminated” connormcdonald.wordpress.com 319