Latin America Tour 2019 - slow data and sql processing

Copyright © 2018, Oracle and/or its affiliates. All rights reserved.
bienvenido

Lo siento, no hablo español :-(

But I will do my best to help
Pero haré todo lo posible para ayudar

Connor McDonald

6

7

8

Me
connor-mcdonald.com
https://linktr.ee/connor

10https://asktom.oracle.com

https://asktom.oracle.com/officehours

11

12

no matter how many layers ...
13
no importa cuántas capas

somewhere you are doing ...
14
XXXalgun lado tú lo estás haciendo

... data access with SQL
15
acceso a datos con SQL

SQL is like any language
13
SQL es como ninguna idioma

compile it
14
compilarlo

compile it
14
on the fly
sobre la marcha

run it
15
ejecutarlo

get | store results
16
obtener | almacenar resultados

for successful applications...
17
para aplicaciones exitosas

compile it ... fast
18
rápido

run it ... fast
19
rápido

get | store results ... fast
20
rápido

that is all!
21
¡Eso es!

what everyone talks about
22
de lo que todos hablan

compile
execute
store/fetch results
23

this session
24
esta sesión

compile
execute
store/fetch results
25

before we begin ...
antes de que comencemos

Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | connor_mc_dconnor mcdonald.com
controversy
31
controversia

here is the problem...
32
aqui esta el problema

a small digression
una pequeña digresión

a little bit of history on servers ...
un poco de historia en los servidores

(not too) long ago
no hace mucho

FSB = "front side bus"
38

any access to ... anything
cualquier acceso a ... cualquier cosa

CPU capped

hypertransport

memory access direct from CPU
44
acceso a memoria directo desde la CPU

"why do I care?"
"¿Porqué me importa?"

you can't blame the server (anymore)
48
ya no puedes culpar al servidor

if you can't get that performance ...
si no puedes obtener ese rendimiento

... it is our fault
está nuestro culpa

good code
buen código

so what's holding us back ?
¿qué nos está frenando?

It all comes down to...
Todo se reduce a

too much work or ...
demasiado trabajo

... not being able to do work
58
no poder hacer el trabajo

PART #1
56
compiling SQL

... how to avoid work when processing SQL
cómo evitar el trabajo excesivo al procesar SQL

terminology
61

cursors
declare
cursor C(p number) is
select * from DEPT
where DEPTNO = p;
begin
for rec in C loop
…
end loop;
end;
select *
from EMPLOYEE
where EMPNO > 1234;
delete from MY_TABLE;
drop table MY_TABLE;
begin
MY_PROCEDURE(1,2,3);
end;

all of them !
todos ellos !

3 phases

open
process
close
give me some memory
do some stuff using that memory,
maybe access other memory
here's your memory back
66
Dame un poco de memoria
usa la memoria
devolver la memoria

memory access = controlled access
acceso a memoria = acceso controlado

a quick primer on (database) memory
una cartilla rápida en memoria

71
limited resource
lots of people want it
concurrent access causes problems
it's a complex system
recurso limitado
mucha gente lo quiere
el acceso concurrente causa problemas
es un systema complejo

73
same with memory
lo mismo con la memoria

SGA
protected by
2) access memory

SGA
protected by
3) release latch

SGA
protected by
someone must wait ...
alguien debe esperar

spinning
can I have the latch ?
can I have the latch ?can I have the latch ?
90

hurts concurrency
duele la concurrencia

YOU

GET

NOTHING

DONEno logras nada

"When are we talking about SQL?"
"¿Cuándo estamos hablando de SQL?

to run a SQL statement
para ejecutar una declaración SQ

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
107

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
SQL> select *
2 frmo emp;
frmo emp
*
ERROR at line 2:
ORA-00923: FROM keyword not found where expected
108

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
SQL> select empnoo
2 from emp;
select empnoo
*
ERROR at line 1:
ORA-00904: invalid column name

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
PLAN
-------------------------------------
SELECT STATEMENT
TABLE ACCESS BY INDEX ROWID EMP
INDEX RANGE SCAN EMP_PK

EMP_PK EMP
Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
112

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)

lots of preliminary steps
muchos pasos preliminares

lots of memory access
116
mucha memoria de acceso

impossible to avoid ?
imposible de evitar?

library cache

previously executed statements
120
declaraciones ejecutadas previamente

parse statement select * from emp where empno = 123

parse statement
already in library cache ?
select * from emp where empno = 123

parse statement
syntactically, semantics OK

parse statement
reuse optimizer info

parse statement
reuse row source info

select surname, firstname from emp where empno = 123
select * from dept where deptno = 4567
select * from customer where locality = 17
select prno from property where reference = 'X123G'
select * from customer where locality = 256
select * from organisation where officer = 'OFF876'
select offender from crimes where crno = 247462
Two full parses avoided
library
cache

select * from dept where dname = 'SALES'
probability of reuse low ?
probabilidad de reutilización baja?

some queries are “nearly the same”
128
algunas consultas son "casi iguales"

binding

binding
parse this...
select surname, firstname from emp where empno = ?

binding
parse this...
now run it with ? = 123
now run it with ? = 456

select * from dept where deptno = ?
select * from customer where locality = ?
select prno from property where reference = ?
select * from customer where locality = ?
select * from organisation where officer = ?
select offender from crimes where crno = ?
Five full parses avoided
library
cache

"performance still looks ok"
"el rendimiento todavía se ve bien"

let us use real queries
hagámoslo real

let's make it real
ParseDemo2 nn
ParseDemo2Bind nn

much more serious
mucho Más grave

building SQL by concatenation
construir SQL por concatenación

select ename
from emp
where empno = 6543

select ename
from emp
where empno = 6543
select ename
from emp
where empno = 6543
and 1=0
union all
select table_name
from all_tables
where table_name like '%SECURITY%'

select ename
from emp
where empno = 6543
select ename
from emp
where empno = 6543
and 1=0
union all
select table_name
from all_tables
where table_name like '%SECURITY%'
select ename
from emp
where empno = 6543
and 1=0
union all
select username
from app_security
where ...

you'll get hacked ...
... really easily!
muy fácil!

it takes 5 minutes to hack you
se necesita 5 minutos

for fast, secure SQL ...
para rápido seguro SQL

... always bind user input
siempre vincula la entrada del usuario

while we are talking about binding
mientras hablamos de encuadernación

parse statement
reuse row source info
Yes!
Yes!
???
???

still some parsing to do
156

Syntax
Validity
Optimization
Rowsourcing
Execution
(Fetch)
Hard
Parse
Soft
Parse
foundnot found
in lib cache

scott@mydb> select *
2 from EMP
3 where EMPNO = :b1
mike@mydb> select *
2 from EMP
3 where EMPNO = :b1

scott@mydb> select *
2 from EMP
3 where EMPNO = :b1
anna@mydb> select *
2 from EMP
3 where EMPNO = :b1
SQL> desc anna.emp
Name Null? Type
----------------- -------- ------------
EMPNO NOT NULL NUMBER(4)
ENAME VARCHAR2(10)
JOB VARCHAR2(9)

(even soft) parsing =
memory access =
latching

can we do better ?
podemos hacerlo mejor?

recall
recuerda

open
process
close
give me some memory
do some stuff using that memory,
maybe access other memory
here's your memory back
163
Dame un poco de memoria
usa la memoria
devolver la memoria

your SQL is always shareable
library cache
select *
from emp
tu SQL siempre se puede compartir

your cursor points to it
library cache
select *
from emp
cursor
165

close cursor = lose pointer
library cache
select *
from emp
cursor cercano = perder puntero
cursor

next time we parse
soft parse
la próxima vez

impossible to avoid ?
168
imposible de evitar?

what if we don't close the cursor
Y si nos no lo hagas cierra el cursor

library cache
select *
from emp
cursor 1
select *
from dept
cursor 2
select *
from emp
where ...
cursor 3

library cache
select *
from emp
cursor 1
select *
from dept
cursor 2
select *
from emp
where ...
cursor 3
pstmt_all_emp =
con.prepareStatement("select * from emp");

emplist(String[] args) {
if "first time" {
else
"reset the pointer, re-execute"
}
library cache
select *
from emp
cursor 1
select *
from dept
cursor 2
select *
from emp
where ...
cursor 3
pstmt_all_emp =
con.prepareStatement("select * from emp");

don’t close the cursor
no cierres el cursor

hang on to the memory
mantener la memoria

remember what cursors we’ve used
recuerda qué cursores hemos usado

reuse whenever possible
reutilizar siempre que sea posible

don’t exhaust memory
no agotar la memoria

don’t allow invalid cursor reuse
no permitir la reutilización inválida del cursor

sounds complicated ...
Suena complicado

a free solution exists
gratis la solución

plsql

create procedure MY_PROC is
begin
select empno, ename, ...
into …
from emp
where empno = my_plsql_variable
...
end;

static SQL in stored PLSQL
automatically uses bind variables
hold’s cursors open
(even if you close)

better cursor caching
mejor almacenamiento en caché del cursor

procedure MY_PROC is
begin
select *
into …
from dept
where …
end;
select * from dept …

cursor c_emp is
select * from emp;
begin
open c_emp;
fetch c_emp
into …;
close c_emp;
end;
select * from emp
begin
select *
into …
from dept
where …
end;
select * from dept …

plsql in the database

171
PART #2
get | store results

172

start with reading data
173
Empezar con leyendo datos

querying data

data is stored in blocks | pages
los datos se almacenan en bloques | paginas

to read data, you read blocks...
para leer datos, lees bloques

SQL> select *
2 from EMP
3 where ...
/u01/my_data1.dat
memory

but (database) memory access is complex
195
pero el acceso a la memoria es complejo

SQL> select *
2 from EMP
3 where …
SQL> update EMP
2 set …
3 where …
SQL> select *
2 from EMP
3 where …
SQL> insert into
2 EMP
3 values (…)
SQL> select *
2 from CUSTOMER
3 /
SQL> select max(EMP)
2 from DEPT
SQL> delete
2 from DEPT
3 /

we have to latch !

• Get latch
• Search along list of blocks in memory
• Read block
• Extract row of interest
• Release latch
• Give row back to client
"fetch a row"
198
Obtener el pestillo
Buscar en la lista de bloques en la memoria
Leer bloquear
Extraer fila de interés
Liberar el pestillo
Devolver la fila al cliente

lots of rows
muchas filas

lots of latching

typical program

rs = stmt.executeQuery("...");
while(rs.next()) {
v1 = rs.getInt(1);
v2 = rs.getString(2);
...
}
202

rs = stmt.executeQuery("...");
while(rs.next()) {
v1 = rs.getInt(1);
v2 = rs.getString(2);
...
}
• Get latch
• Walk along list
• Get block
• Extract single row
• Release latch
• Give row back to client
• Get latch (again)
• Walk along list (again)
• Get block (again)
• Release latch (again)
• Give row back to client (again)
Row #1
Row #2
• Get latch (again)
• Walk along list (again)
• Get block (again)
• Release latch (again)
• Give row back to client (again)
Row #3
203

a better strategy.... “pinning”
una mejor estrategia

"fetch a row"

"fetch a row"
“and .. I may need
many rows from this
block”
Puedo necesitar muchos
filas de esto bloquear"

"fetch a row"
•Get latch
•Walk along list
•Get block
•Pin the block
•Release latch
•Give row 1 back to client
…
•Give row n to client
•Get latch
•Walk along list
•Remove my pin on the block
•Release latch
“and .. I may need
many rows from this
block”

how do you say ?
“I may want many rows”
Cómo se dice ?
"Puedo querer muchas filas"

responsibility of the client
responsabilidad del cliente

190

190
FetchDemo nn

focus on blocks
191

focus on roundtrips
192

nothing in life is free !
192
nada en la vida es gratis

all that data goes somewhere
220
todos esos datos van algun lado

SQL> declare
. . .
9 begin
10 open c;
11 loop
12 fetch c
13 into r;
14 exit when c%notfound;
. . .
19 /
Elapsed: 00:01:11.72

SQL> declare
. . .
9 begin
10 open c;
11
12 fetch c
13 bulk collect
14 into r;
15
. . .
19 /
Elapsed: 00:00:08.93

SQL> select * from v$mystats
2 where name like '%pga%'
3 /
NAME VALUE
------------------------------ ----------
session pga memory 501534672
session pga memory max 501534672

500m of memory !

pin rows on a block (8k)
225

SQL> declare
. . .
9 begin
10 open c;
11 loop
12 fetch c
13 bulk collect
14 into r limit 100;
15 exit when c%notfound;
. . .
19 /
Elapsed: 00:00:08.17
NAME VALUE
------------------------------ ----------
session pga memory 625912
session pga memory max 625912

SQL> declare
. . .
9 begin
10 for i in c loop
12 ...
16 end loop;
17 end;
18 /
Elapsed: 00:00:08.21

PART #2(a)
storing data

the same rules apply
205
se aplican las mismas reglas

think blocks not rows
206
when appropriate
pensar bloques no filas

demo
208
InsertDemo3
InsertDemo4 nn

summary

avoid parsing costs
for high frequency SQL
evitar costos de análisis
para SQL de alta frecuencia

use binding and don't get hacked !
¡usa el enlace y no te hackeen!

PLSQL in the database is golden
PLSQL en la base de datos es dorado

Thank you!
connor-mcdonald.com
https://linktr.ee/connor
Gracias!

Latin America Tour 2019 - slow data and sql processing

Recommended

Recommended

More Related Content

What's hot

What's hot (15)

Similar to Latin America Tour 2019 - slow data and sql processing

Similar to Latin America Tour 2019 - slow data and sql processing (20)

More from Connor McDonald

More from Connor McDonald (19)

Recently uploaded

Recently uploaded (20)

Latin America Tour 2019 - slow data and sql processing