Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Fatkulin hotsos 2014

1,871 views

Published on

Published in: Technology
  • Be the first to comment

Fatkulin hotsos 2014

  1. 1. Leveraging In-Memory Storage to Overcome Oracle PGA Memory Limits March 5, 2014 Presented by: Alex Fatkulin Senior Consultant
  2. 2. Who am I ?  Senior Technical Consultant at Enkitec  12 years using Oracle  Clustered and HA solutions  Database Development and Design  Technical Reviewer  Blog at http://afatkulin.blogspot.com 3
  3. 3. Why This Presentation? 4
  4. 4. Data Growth and Processing  Data Volumes UP  Processing Power UP  Database Design (in general) DOWN 5 Data Volume Processing Power Database Design BetterWorse
  5. 5. 6 How many people have systems with more than 128G of RAM?
  6. 6. “640K” Problems 7  Processing patterns change  Things that didn’t matter before matter now  Legacy RDBMS code vs 64-bit systems  KIWI (Kill It With Iron)
  7. 7. PGA Memory 8
  8. 8. PGA Memory (Dedicated Server) 9 Sort AreaHash Area Bitmap Area SQL Work Areas Session Memory Private SQL Area Process Memory
  9. 9. Query Execution Work Areas 10 Work Area Size Manual Management Auto Management hash_area_size sort_area_size bitmap_merge_area_size create_bitmap_area_size pga_aggregate_target
  10. 10. PGA Memory Limits 11
  11. 11. Manual Work Area Management 12 SQL> alter session set workarea_size_policy=manual; Session altered SQL> alter session set hash_area_size=4294967296; -- 4GB alter session set hash_area_size=4294967296 ORA-02017: integer value required  11.2.0.4 Linux x64 SQL> alter session set hash_area_size=2147483648; -- 2GB alter session set hash_area_size=2147483648 ORA-02017: integer value required SQL> alter session set hash_area_size=2147483647; -- 2GB - 1 Session altered  32-bit Signed Integer Limit
  12. 12. Auto Work Area Management 13 SQL> alter system set pga_aggregate_target=1024g; -- 1TB System altered  11.2.0.4 Linux x64  Where is the catch?  PGA_AGGREGATE_TARGET  _smm_max_size/_smm_max_size_static  Maximum work area size per process (px/serial)  _smm_px_max_size/_smm_px_max_size_static  Maximum work area size per query (px)  _pga_max_size  Maximum PGA size per process (px/serial)
  13. 13. Auto Work Area Management 14  PGA_AGGREGATE_TARGET 16M – 512M 0 100 200 300 400 500 600 16 32 64 128 256 512 pga_aggregate_target _smm_max_size _smm_px_max_size _pga_max_size  _pga_max_size = 200M  _smm_max_size[_static] = 20%  _smm_px_max_size[_static] = 50%
  14. 14. Auto Work Area Management 15  PGA_AGGREGATE_TARGET 1G – 10G  _pga_max_size = 20%  _smm_max_size[_static] = 10%  _smm_px_max_size[_static] = 50% 0 2000 4000 6000 8000 10000 12000 pga_aggregate_target _smm_max_size _smm_px_max_size _pga_max_size
  15. 15. Auto Work Area Management 16  PGA_AGGREGATE_TARGET 10G – 16G  _pga_max_size = 2G  _smm_max_size[_static] = 2G  _smm_px_max_size[_static] = 50% 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 10240 10752 11264 11776 12288 12800 13312 13824 14336 14848 15360 15872 16384 pga_aggregate_target _smm_max_size _smm_px_max_size _pga_max_size
  16. 16. Auto Work Area Management 17  PGA_AGGREGATE_TARGET >= 10G  _smm_max_size[_static] = 1G (maximum value)  _pga_max_size = 2G (maximum value)  If you’re bumping into per process limits further rising pga_aggregate_target will not help
  17. 17. Per Process Limit 18  PGA_AGGREGATE_TARGET = 192G / DOP = 16  PGA_AGGREGATE_TARGET = 512G / DOP = 16
  18. 18. Per Process Limit 19  PGA_AGGREGATE_TARGET = 192G / DOP = 16  PGA_AGGREGATE_TARGET = 512G / DOP = 16  _pga_max_size * DOP
  19. 19. Run With Higher DOP? 20  Exadata X3-8  2TB RAM (per node)  80 CPU cores (per node)  PGA_AGGREGATE_TARGET = 1536G  Would require at least DOP 768 (_pga_max_size)  9.6x core count  Concurrency issues  Manageability issues  PX data distribution/algorithm issues
  20. 20. Run With Higher DOP? 21  DOP vs CPU Cores Used 5 (spill) 8 (spill) 64 (no spill) 0 16 32 48 64 16 32 64 CPUCoresUsed DOP
  21. 21. Run With Higher DOP? 22  PX Algorithm Issues (ex.: median function)
  22. 22. Play With Underscores? 23  MOS Doc ID 453540.1  Allows _smm_max_size=2097151  Allows _pga_max_size > 2GB with patch 17951233  Can get 4G per process limit  Can get > 4G per process limit  /proc/sys/vm/max_map_count (OS page count)  _realfree_heap_pagesize_hint (allocator page size)  Weird behavior and likely not fully supported for work areas (MOS Doc ID 453540.1)
  23. 23. Radically Improve TEMP I/O? 24  TEMP I/O (in general)  Doesn’t need redundancy  Doesn’t need persistency  Doesn’t need recoverability  In-Memory FS (tmpfs, etc.)  SAN/NAS LUN with write-back cache
  24. 24. Linux tmpfs (via loop device) 25  10.1B rows / 416G HCC (QUERY HIGH) SELECT /*+ parallel(t,16) */ CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D FROM TRANS T WHERE DATE_ID between to_date('01012012', 'ddmmyyyy') and to_date('31122013', 'ddmmyyyy') GROUP BY CUST_ID, DATE_ID; Exadata X3-8 TEMP tmpfs TEMP  13.8x faster TEMP I/O (237G)
  25. 25. Linux tmpfs (via loop device) 26  10.1B rows / 416G HCC (QUERY HIGH) SELECT /*+ parallel(t,16) */ CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D FROM TRANS T WHERE DATE_ID between to_date('01012012', 'ddmmyyyy') and to_date('31122013', 'ddmmyyyy') GROUP BY CUST_ID, DATE_ID;
  26. 26. ZFSSA (Infiniband SRP) 27  7.3B rows / 300G HCC (QUERY HIGH) SELECT /*+ parallel(t,32) */ CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D FROM TRANS T GROUP BY CUST_ID, DATE_ID; Exadata X3-8 TEMP ZFSSA TEMP (write-back cache)  64.3x faster TEMP I/O (141G)
  27. 27. ZFSSA (Infiniband SRP) 28  7.3B rows / 300G HCC (QUERY HIGH) SELECT /*+ parallel(t,32) */ CUST_ID, DATE_ID, COUNT(DISTINCT STORE_ID) D FROM TRANS T GROUP BY CUST_ID, DATE_ID;
  28. 28. Summary 29  Remember about per process limits  Larger PGA_AGGREGATE_TARGET may do nothing  Balance PGA/DOP/CPU  It’s possible to increase TEMP I/O performance by 10x- 50x and more  Oracle PGA code does not fully embrace 64-bit systems at the moment
  29. 29. Q & A Email: afatkulin@enkitec.com Blog: http://afatkulin.blogspot.com 30

×