Your SlideShare is downloading. ×
Fatkulin hotsos 2014
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Fatkulin hotsos 2014

297
views

Published on

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
297
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Leveraging In-Memory Storage to Overcome Oracle PGA Memory Limits March 5, 2014 Presented by: Alex Fatkulin Senior Consultant
  • 2. Who am I ?  Senior Technical Consultant at Enkitec  12 years using Oracle  Clustered and HA solutions  Database Development and Design  Technical Reviewer  Blog at http://afatkulin.blogspot.com 3
  • 3. Why This Presentation? 4
  • 4. Data Growth and Processing  Data Volumes UP  Processing Power UP  Database Design (in general) DOWN 5 Data Volume Processing Power Database Design BetterWorse
  • 5. 6 How many people have systems with more than 128G of RAM?
  • 6. “640K” Problems 7  Processing patterns change  Things that didn’t matter before matter now  Legacy RDBMS code vs 64-bit systems  KIWI (Kill It With Iron)
  • 7. PGA Memory 8
  • 8. PGA Memory (Dedicated Server) 9 Sort AreaHash Area Bitmap Area SQL Work Areas Session Memory Private SQL Area Process Memory
  • 9. Query Execution Work Areas 10 Work Area Size Manual Management Auto Management hash_area_size sort_area_size bitmap_merge_area_size create_bitmap_area_size pga_aggregate_target
  • 10. PGA Memory Limits 11
  • 11. Manual Work Area Management 12 SQL> alter session set workarea_size_policy=manual; Session altered SQL> alter session set hash_area_size=4294967296; -- 4GB alter session set hash_area_size=4294967296 ORA-02017: integer value required  11.2.0.4 Linux x64 SQL> alter session set hash_area_size=2147483648; -- 2GB alter session set hash_area_size=2147483648 ORA-02017: integer value required SQL> alter session set hash_area_size=2147483647; -- 2GB - 1 Session altered  32-bit Signed Integer Limit
  • 12. Auto Work Area Management 13 SQL> alter system set pga_aggregate_target=1024g; -- 1TB System altered  11.2.0.4 Linux x64  Where is the catch?  PGA_AGGREGATE_TARGET  _smm_max_size/_smm_max_size_static  Maximum work area size per process (px/serial)  _smm_px_max_size/_smm_px_max_size_static  Maximum work area size per query (px)  _pga_max_size  Maximum PGA size per process (px/serial)
  • 13. Auto Work Area Management 14  PGA_AGGREGATE_TARGET 16M – 512M 0 100 200 300 400 500 600 16 32 64 128 256 512 pga_aggregate_target _smm_max_size _smm_px_max_size _pga_max_size  _pga_max_size = 200M  _smm_max_size[_static] = 20%  _smm_px_max_size[_static] = 50%
  • 14. Auto Work Area Management 15  PGA_AGGREGATE_TARGET 1G – 10G  _pga_max_size = 20%  _smm_max_size[_static] = 10%  _smm_px_max_size[_static] = 50% 0 2000 4000 6000 8000 10000 12000 pga_aggregate_target _smm_max_size _smm_px_max_size _pga_max_size
  • 15. Auto Work Area Management 16  PGA_AGGREGATE_TARGET 10G – 16G  _pga_max_size = 2G  _smm_max_size[_static] = 2G  _smm_px_max_size[_static] = 50% 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 10240 10752 11264 11776 12288 12800 13312 13824 14336 14848 15360 15872 16384 pga_aggregate_target _smm_max_size _smm_px_max_size _pga_max_size
  • 16. Auto Work Area Management 17  PGA_AGGREGATE_TARGET >= 10G  _smm_max_size[_static] = 1G (maximum value)  _pga_max_size = 2G (maximum value)  If you’re bumping into per process limits further rising pga_aggregate_target will not help
  • 17. Per Process Limit 18  PGA_AGGREGATE_TARGET = 192G / DOP = 16  PGA_AGGREGATE_TARGET = 512G / DOP = 16
  • 18. Per Process Limit 19  PGA_AGGREGATE_TARGET = 192G / DOP = 16  PGA_AGGREGATE_TARGET = 512G / DOP = 16  _pga_max_size * DOP
  • 19. Run With Higher DOP? 20  Exadata X3-8  2TB RAM (per node)  80 CPU cores (per node)  PGA_AGGREGATE_TARGET = 1536G  Would require at least DOP 768 (_pga_max_size)  9.6x core count  Concurrency issues  Manageability issues  PX data distribution/algorithm issues
  • 20. Run With Higher DOP? 21  DOP vs CPU Cores Used 5 (spill) 8 (spill) 64 (no spill) 0 16 32 48 64 16 32 64 CPUCoresUsed DOP
  • 21. Run With Higher DOP? 22  PX Algorithm Issues (ex.: median function)
  • 22. Play With Underscores? 23  MOS Doc ID 453540.1  Allows _smm_max_size=2097151  Allows _pga_max_size > 2GB with patch 17951233  Can get 4G per process limit  Can get > 4G per process limit  /proc/sys/vm/max_map_count (OS page count)  _realfree_heap_pagesize_hint (allocator page size)  Weird behavior and likely not fully supported for work areas (MOS Doc ID 453540.1)
  • 23. Radically Improve TEMP I/O? 24  TEMP I/O (in general)  Doesn’t need redundancy  Doesn’t need persistency  Doesn’t need recoverability  In-Memory FS (tmpfs, etc.)  SAN/NAS LUN with write-back cache
  • 24. Linux tmpfs (via loop device) 25  10.1B rows / 416G HCC (QUERY HIGH) SELECT /*+ parallel(t,16) */ CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D FROM TRANS T WHERE DATE_ID between to_date('01012012', 'ddmmyyyy') and to_date('31122013', 'ddmmyyyy') GROUP BY CUST_ID, DATE_ID; Exadata X3-8 TEMP tmpfs TEMP  13.8x faster TEMP I/O (237G)
  • 25. Linux tmpfs (via loop device) 26  10.1B rows / 416G HCC (QUERY HIGH) SELECT /*+ parallel(t,16) */ CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D FROM TRANS T WHERE DATE_ID between to_date('01012012', 'ddmmyyyy') and to_date('31122013', 'ddmmyyyy') GROUP BY CUST_ID, DATE_ID;
  • 26. ZFSSA (Infiniband SRP) 27  7.3B rows / 300G HCC (QUERY HIGH) SELECT /*+ parallel(t,32) */ CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D FROM TRANS T GROUP BY CUST_ID, DATE_ID; Exadata X3-8 TEMP ZFSSA TEMP (write-back cache)  64.3x faster TEMP I/O (141G)
  • 27. ZFSSA (Infiniband SRP) 28  7.3B rows / 300G HCC (QUERY HIGH) SELECT /*+ parallel(t,32) */ CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D FROM TRANS T GROUP BY CUST_ID, DATE_ID;
  • 28. Summary 29  Remember about per process limits  Larger PGA_AGGREGATE_TARGET may do nothing  Balance PGA/DOP/CPU  It’s possible to increase TEMP I/O performance by 10x- 50x and more  Oracle PGA code does not fully embrace 64-bit systems at the moment
  • 29. Q & A Email: afatkulin@enkitec.com Blog: http://afatkulin.blogspot.com 30