The document discusses optimizing SQL application performance by emphasizing the importance of efficient SQL processing, compiling statements, and managing memory access. It highlights key concepts like cursor management, the significance of binding user inputs to prevent security vulnerabilities, and strategies to minimize parsing costs. The session also includes various technical demonstrations and best practices for database interaction to improve overall efficiency.

1. 26/11/2019
1
1
a good background in ...
2
... SQL processing, aka
3
... successful application
development
4
Connor McDonald
1 2
3 4

2. 26/11/2019
2
Copyright© 2018, Oracleand/or its affiliates. All rights reserved.
5
Copyright© 2018, Oracleand/or its affiliates. All rights reserved.
6
Getting in touch is easy...
@connor_mc_d
https://linktr.ee/connor
Copyright© 2018, Oracleand/or its affiliates. All rights reserved.
8https://asktom.oracle.com
5 6
7 8

3. 26/11/2019
3
Copyright© 2018, Oracleand/or its affiliates. All rights reserved.https://asktom.oracle.com/officehours10
250 hours free access so far
11
assumptions
1. You roughly know databases (tables and columns)
2. You can write a SQL statement
12
SQL is like any language
9 10
11 12

4. 26/11/2019
4
13
compile it
on the fly
14
run it
15
get | store results
16
for successful applications...
13 14
15 16

5. 26/11/2019
5
17
compile it ... fast
18
run it ... fast
19
get | store results ... fast
20
that's it!
17 18
19 20

6. 26/11/2019
6
21
what everyone talks about
22
compile
execute
store/fetch results
23
this session
24
compile
execute
store/fetch results
21 22
23 24

7. 26/11/2019
7
25
before we begin ...
26
some controversy to start
27
here's the problem...
28
... it's us
25 26
27 28

8. 26/11/2019
8
29
a small digression
30
a little bit of history on servers ...
31
(not too) long ago
29 30
31 32

9. 26/11/2019
9
33
FSB = "front side bus"
34
any access to ... anything
35
CPU capped
33 34
35 36

10. 26/11/2019
10
37
hypertransport
39
memory access direct from CPU
37 38
39 40

11. 26/11/2019
11
~2007
42
"why do I care?"
43
you can't blame the server (anymore)
44
if you can't get that performance ...
41 42
43 44

12. 26/11/2019
12
45
... it's you
47
"The sooner your company admits that you are not
Google, the sooner you can get down to some real
work"
- Ted Dziuba
48
so what's holding you back ?
45 46
47 48

13. 26/11/2019
13
49
It all comes down to...
50
too much work or ...
51
... not being able to do work
52
PART #1
5
2
compiling SQL
49 50
51 52

14. 26/11/2019
14
53
... how to avoid work when processing SQL
54
terminology
55
cursorsdeclare
cursor C(p number) is
select * from DEPT
where DEPTNO = p;
begin
for rec in C loop
…
end loop;
end;
select *
from EMPLOYEE
where EMPNO > 1234;
delete from MY_TABLE;
drop table MY_TABLE;
begin
MY_PROCEDURE(1,2,3);
end;
56
all of them !
53 54
55 56

15. 26/11/2019
15
57
3 phases
58
open
process
close
gimme some memory
do some stuff using that memory,
maybe access other memory
here's your memory back
59
memory access = controlled access
60
a quick primer on (database) memory
57 58
59 60

16. 26/11/2019
16
61 62
metaphor
63
61 62
63 64

17. 26/11/2019
17
67
limited resource
lots of people want it
concurrent access causes problems
it's a complex system
65 66
67 68

18. 26/11/2019
18
69
same with memory
70
SGA
71
SGA
protected by
72
SGA
protected by
1) get latch
69 70
71 72

19. 26/11/2019
19
73
SGA
protected by
2) access memory
74
SGA
protected by
3) release latch
75
latch contention
76
someone must wait ...
SGA
protected by
73 74
75 76

20. 26/11/2019
20
77
active wait
78
spinning
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
can I have the latch ?
77 78
79 80

21. 26/11/2019
21
81
latch contention....
82
hurts CPU...
83
hurts concurrency
YOU
81 82
83 84

22. 26/11/2019
22
GET NOTHING
DONE
88
"Errrr.... weren't we talking SQL?"
85 86
87 88

23. 26/11/2019
23
89
to run a SQL statement
90
Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
SQL> select *
2 frmo emp;
frmo emp
*
ERROR at line 2:
ORA-00923: FROM keyword not found where expected
91
Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
SQL> select empnoo
2 from emp;
select empnoo
*
ERROR at line 1:
ORA-00904: invalid column name
92
PLAN
-------------------------------------
SELECT STATEMENT
TABLE ACCESS BY INDEX ROWID EMP
INDEX RANGE SCAN EMP_PK
Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
89 90
91 92

24. 26/11/2019
24
93
EMP_PK EMP
Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
94
Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
95
Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
96
lots of preliminaries
93 94
95 96

25. 26/11/2019
25
97
parsing
"compile"
98
lots of memory access
99
lots of latching !
100
impossible to avoid ?
97 98
99 100

26. 26/11/2019
26
101
library cache
102
previously executed
statements
103
parse statement
already in library cache ?
reuse optimizer info
reuse row source info
select * from emp where empno = 123
syntactically, semantics OK
104
select surname, firstname from emp where empno = 123
select * from dept where deptno = 4567
select * from customer where locality = 17
select prno from property where reference = 'X123G'
select * from dept where deptno = 23
select surname, firstname from emp where empno = 123
select * from customer where locality = 256
select * from organisation where officer = 'OFF876'
select surname, firstname from emp where empno = 7632
select * from dept where deptno = 4567
select offender from crimes where crno = 247462
Two full parses avoided
library
cache
101 102
103 104

27. 26/11/2019
27
105
select surname, firstname from emp where empno = 123
select * from dept where dname = 'SALES'
probability of reuse low ?
106
some queries are “nearly the same”
select surname, firstname from emp where empno = 123
select surname, firstname from emp where empno = 456
107
binding
parse this...
now run it with ? = 123
now run it with ? = 456
select surname, firstname from emp where empno = ?
108
select surname, firstname from emp where empno = ?
select * from dept where deptno = ?
select * from customer where locality = ?
select prno from property where reference = ?
select * from dept where deptno = ?
select surname, firstname from emp where empno = ?
select * from customer where locality = ?
select * from organisation where officer = ?
select surname, firstname from emp where empno = ?
select * from dept where deptno = ?
select offender from crimes where crno = ?
Five full parses avoided
library
cache
105 106
107 108

28. 26/11/2019
28
109
demo
ParseDemo
ParseDemoBind
110
"performance still looks ok"
111
let's make it real
ParseDemo2 nn
ParseDemo2Bind nn
112
much more serious
109 110
111 112

29. 26/11/2019
29
113
building SQL by concatenation
114
you'll get hacked
115
select ename
from emp
where empno = 6543
select ename
from emp
where empno = 6543
and 1=0
union all
select table_name
from all_tables
where table_name like '%SECURITY%'
select ename
from emp
where empno = 6543
and 1=0
union all
select username
from app_security
where ...
116
it takes 5 minutes to hack you
113 114
115 116

30. 26/11/2019
30
117
for fast, secure SQL ...
118
... always bind user input
119
"I'm an APEX developer... not an issue"
120
117 118
119 120

31. 26/11/2019
31
121
other common culprits
item protection
create_collection_from_query
122
while we are talking binding
123
parse statement
already in library cache ?
reuse optimizer info
reuse row source info
select * from emp where empno = 123
syntactically, semantics OK
Yes!
Yes!
???
???
124
still some parsing to do
121 122
123 124

32. 26/11/2019
32
125
Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
Hard
Parse
Soft
Parse
foundnot found
in lib cache
126
scott@mydb> select *
2 from EMP
3 where EMPNO = :b1
sue@mydb> select *
2 from EMP
3 where EMPNO = :b1
SQL> desc sue.emp
Name Null? Type
----------------- -------- ------------
EMPNO NOT NULL NUMBER(4)
ENAME VARCHAR2(10)
JOB VARCHAR2(9)
127
(even soft) parsing =
memory access =
latching
128
can we do better ?
125 126
127 128

33. 26/11/2019
33
129
your SQL is always shareable
library cache
select *
from emp
130
your cursor points to it
cursor
library cache
select *
from emp
131
close cursor = lose pointer
cursor
library cache
select *
from emp
132
next time we parse
soft parse
129 130
131 132

34. 26/11/2019
34
133
what if we don't close the cursor
134
emplist(String[] args) {
if "first time I'm using this" {
else
"reset the pointer, re-execute"
}
library cache
select *
from emp
select *
from dept
select *
from emp
where ...
pstmt_all_emp =
con.prepareStatement("select * from emp");
cursor
cursor
cursor
135
don’t close the cursorhang on to the memoryremember what cursors we’ve usedreuse whenever possibledon’t exhaust memorydon’t allow invalid cursor reuse
136
sounds complicated ...
133 134
135 136

35. 26/11/2019
35
137
a free solution exists
138
plsql
139
create procedure MY_PROC is
begin
select empno, ename, ...
into …
from emp
where empno = my_plsql_variable
...
end;
140
static SQL in stored PLSQL
automatically uses bind variables
hold’s cursors open
(even if you close)
137 138
139 140

36. 26/11/2019
36
141
procedure MY_PROC is
cursor c_emp is
select * from emp;
begin
open c_emp;
fetch c_emp
into …;
close c_emp;
end;
select * from emp
procedure MY_PROC is
begin
select *
into …
from dept
where …
end;
select * from dept …
142
PART #2
get | store results
we'll come back to running SQL
143 144
start with reading data
141 142
143 144

37. 26/11/2019
37
145
querying data
146
data is stored in blocks | pages
147
to read data, you read blocks...
148
SQL> select *
2 from EMP
3 where ...
/u01/my_data1.dat
memory
145 146
147 148

38. 26/11/2019
38
149
but (database) memory access is complex
150
SQL> select *
2 from EMP
3 where …
SQL> update EMP
2 set …
3 where …
SQL> select *
2 from EMP
3 where …
SQL> insert into
2 EMP
3 values (…)
SQL> select *
2 from CUSTOMER
3 /
SQL> select max(EMPNO)
2 from EMP
SQL> delete
2 from DEPT
3 /
151
we have to latch !
152
• Get latch
• Search along list of blocks in memory
• Read block
• Extract row of interest
• Release latch
• Give row back to client
"fetch a row"
149 150
151 152

39. 26/11/2019
39
153
lots of rows
154
lots of latching
155
typical program
156
rs = stmt.executeQuery("...");
while(rs.next()) {
v1 = rs.getInt(1);
v2 = rs.getString(2);
...
}
• Get latch
• Walk along list
• Get block
• Extract single row
• Release latch
• Give row back to client
• Get latch (again)
• Walk along list (again)
• Get block (again)
• Extract single row
• Release latch (again)
• Give row back to client (again)
Row #1
Row #2
• Get latch (again)
• Walk along list (again)
• Get block (again)
• Extract single row
• Release latch (again)
• Give row back to client (again)
Row #3
153 154
155 156

40. 26/11/2019
40
157
a better strategy.... “pinning”
158
"fetch a row"
• Get latch
• Walk along list
• Get block
• Pin the block
• Release latch
• Give row 1 back to client
• Give row 2 back to client
…
• Give row n to client
• Get latch
• Walk along list
• Remove my pin on the block
• Release latch
“and .. I’ll will need
several rows from
this block”
159
how do you say
“I may want many rows” ?
160
responsibility of the client
157 158
159 160

41. 26/11/2019
41
161
FetchDemo nn
162
focus on blocks
163
focus on roundtrips
164
all that data goes somewhere
161 162
163 164

42. 26/11/2019
42
165
SQL> declare
cursor C is select ... (500m table)
...
9 begin
10 open c;
11 loop
12 fetch c
13 into r;
14 exit when c%notfound;
. . .
19 /
Elapsed: 00:01:11.72
166
SQL> declare
cursor C is select ... (500m table)
. . .
9 begin
10 open c;
11
12 fetch c
13 bulk collect
14 into r;
15
. . .
19 /
Elapsed: 00:00:08.93
167
SQL> select * from v$mystats
2 where name like '%pga%'
3 /
NAME VALUE
------------------------------ ----------
session pga memory 501534672
session pga memory max 501534672
168
500m of memory !
165 166
167 168

43. 26/11/2019
43
169
law of diminishing returns
170
pin rows on a block (8k)
171
SQL> declare
. . .
9 begin
10 open c;
11 loop
12 fetch c
13 bulk collect
14 into r limit 100;
15 exit when c%notfound;
. . .
19 /
Elapsed: 00:00:08.17
NAME VALUE
------------------------------ ----------
session pga memory 625912
session pga memory max 625912
172
SQL> declare
. . .
9 begin
10 for i in c loop
12 ...
16 end loop;
17 end;
18 /
Elapsed: 00:00:08.21
169 170
171 172

44. 26/11/2019
44
173
not just your code, but our code too
174
175 176
< 18 18,19 19.2
Classic
IR
IG
173 174
175 176

45. 26/11/2019
45
177
PART #2(a)
storing data
178
the same rules apply
179
think blocks not rows
when appropriate
180
commit when it makes sense to
177 178
179 180

46. 26/11/2019
46
181
demo
InsertDemo3
InsertDemo4 nn
182
summary
183
avoid parsing costs
for high frequency SQL
184
use binding and don't get hacked !
181 182
183 184

47. 26/11/2019
47
185
using PLSQL is golden
186
and you can do it all .... for free
oracle.com/gbtour
New Free Tier
Always Free
Oracle Cloud Infrastructure
Services you can use for unlimited time
30-Day Free Trial
Free credits you can use for more services
+
Thanks for coming!
connor-mcdonald.com
https://bit.ly/techguysong
185 186
187 188