TeamStation AI System Report LATAM IT Salaries 2024
Fatkulin hotsos 2014
1. Leveraging In-Memory Storage to
Overcome Oracle PGA Memory Limits
March 5, 2014
Presented by: Alex Fatkulin
Senior Consultant
2. Who am I ?
Senior Technical Consultant at Enkitec
12 years using Oracle
Clustered and HA solutions
Database Development and Design
Technical Reviewer
Blog at http://afatkulin.blogspot.com
3
4. Data Growth and Processing
Data Volumes UP
Processing Power UP
Database Design (in general) DOWN
5
Data Volume
Processing Power
Database Design
BetterWorse
6. “640K” Problems
7
Processing patterns change
Things that didn’t matter before matter now
Legacy RDBMS code vs 64-bit systems
KIWI (Kill It With Iron)
8. PGA Memory (Dedicated Server)
9
Sort AreaHash Area Bitmap Area
SQL Work Areas
Session Memory Private SQL Area
Process Memory
9. Query Execution Work Areas
10
Work Area Size
Manual Management Auto Management
hash_area_size
sort_area_size
bitmap_merge_area_size
create_bitmap_area_size
pga_aggregate_target
11. Manual Work Area Management
12
SQL> alter session set workarea_size_policy=manual;
Session altered
SQL> alter session set hash_area_size=4294967296; -- 4GB
alter session set hash_area_size=4294967296
ORA-02017: integer value required
11.2.0.4 Linux x64
SQL> alter session set hash_area_size=2147483648; -- 2GB
alter session set hash_area_size=2147483648
ORA-02017: integer value required
SQL> alter session set hash_area_size=2147483647; -- 2GB - 1
Session altered
32-bit Signed Integer Limit
12. Auto Work Area Management
13
SQL> alter system set pga_aggregate_target=1024g; -- 1TB
System altered
11.2.0.4 Linux x64
Where is the catch?
PGA_AGGREGATE_TARGET
_smm_max_size/_smm_max_size_static
Maximum work area size per process (px/serial)
_smm_px_max_size/_smm_px_max_size_static
Maximum work area size per query (px)
_pga_max_size
Maximum PGA size per process (px/serial)
16. Auto Work Area Management
17
PGA_AGGREGATE_TARGET >= 10G
_smm_max_size[_static] = 1G (maximum value)
_pga_max_size = 2G (maximum value)
If you’re bumping into per process limits further rising
pga_aggregate_target will not help
19. Run With Higher DOP?
20
Exadata X3-8
2TB RAM (per node)
80 CPU cores (per node)
PGA_AGGREGATE_TARGET = 1536G
Would require at least DOP 768 (_pga_max_size)
9.6x core count
Concurrency issues
Manageability issues
PX data distribution/algorithm issues
20. Run With Higher DOP?
21
DOP vs CPU Cores Used
5 (spill) 8 (spill)
64 (no spill)
0
16
32
48
64
16 32 64
CPUCoresUsed
DOP
21. Run With Higher DOP?
22
PX Algorithm Issues (ex.: median function)
22. Play With Underscores?
23
MOS Doc ID 453540.1
Allows _smm_max_size=2097151
Allows _pga_max_size > 2GB with patch 17951233
Can get 4G per process limit
Can get > 4G per process limit
/proc/sys/vm/max_map_count (OS page count)
_realfree_heap_pagesize_hint (allocator page size)
Weird behavior and likely not fully supported for work
areas (MOS Doc ID 453540.1)
23. Radically Improve TEMP I/O?
24
TEMP I/O (in general)
Doesn’t need redundancy
Doesn’t need persistency
Doesn’t need recoverability
In-Memory FS (tmpfs, etc.)
SAN/NAS LUN with write-back cache
24. Linux tmpfs (via loop device)
25
10.1B rows / 416G HCC (QUERY HIGH)
SELECT /*+ parallel(t,16) */
CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D
FROM TRANS T
WHERE DATE_ID between to_date('01012012', 'ddmmyyyy') and to_date('31122013', 'ddmmyyyy')
GROUP BY CUST_ID, DATE_ID;
Exadata X3-8 TEMP
tmpfs TEMP
13.8x faster TEMP I/O (237G)
25. Linux tmpfs (via loop device)
26
10.1B rows / 416G HCC (QUERY HIGH)
SELECT /*+ parallel(t,16) */
CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D
FROM TRANS T
WHERE DATE_ID between to_date('01012012', 'ddmmyyyy') and to_date('31122013', 'ddmmyyyy')
GROUP BY CUST_ID, DATE_ID;
26. ZFSSA (Infiniband SRP)
27
7.3B rows / 300G HCC (QUERY HIGH)
SELECT /*+ parallel(t,32) */
CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D
FROM TRANS T
GROUP BY CUST_ID, DATE_ID;
Exadata X3-8 TEMP
ZFSSA TEMP
(write-back cache)
64.3x faster TEMP I/O (141G)
27. ZFSSA (Infiniband SRP)
28
7.3B rows / 300G HCC (QUERY HIGH)
SELECT /*+ parallel(t,32) */
CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D
FROM TRANS T
GROUP BY CUST_ID, DATE_ID;
28. Summary
29
Remember about per process limits
Larger PGA_AGGREGATE_TARGET may do nothing
Balance PGA/DOP/CPU
It’s possible to increase TEMP I/O performance by 10x-
50x and more
Oracle PGA code does not fully embrace 64-bit systems
at the moment
29. Q & A
Email: afatkulin@enkitec.com
Blog: http://afatkulin.blogspot.com
30