Exadata and the Oracle Optimizer: The Untold Story

6,483 views

Published on

Published in: Technology
0 Comments
16 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,483
On SlideShare
0
From Embeds
0
Number of Embeds
26
Actions
Shares
0
Downloads
0
Comments
0
Likes
16
Embeds 0
No embeds

No notes for slide

Exadata and the Oracle Optimizer: The Untold Story

  1. 1. Exadata And The Oracle OptimizerThe untold storyCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  2. 2. Oracle safe harbor statementAll information provided outlines our general product direction. Its intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code or functionality, and should not be relied upon in making a purchasing decision. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle.Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  3. 3. Exadata and the OptimizerHistory•  Originally plan was the Optimizer shouldn’t change –  Optimizer should not cost smart scan –  Same plan should be chosen regardless of platform –  Easier for migrations –  Exadata will speed up all scan operations automatically by offloading them to the storageCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  4. 4. Exadata and the Optimizer What Happens when a SQL statement is issued? 1 User 2 Oracle Database Syntax Check SQL Execution Parsing Semantic Check 4 Shared Pool checkShared Pool 3 Query TransformationLibrary Cache Plan Generator Cost EstimatorShared SQL Area … OptimizerC1 C2 Cn Code Generator Decision on what processing is offloaded occurs here Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  5. 5. Exadata and the OptimizerHistory•  So what operations can actually be offloaded? –  Full table scan –  Fast full index scan of a B-Tree or bitmap index –  Bloom filters•  But not all predicates can be offloaded –  V$SQLFN_METADATA has a column called offloadable –  Indicates if SQL function is offloadable or not –  With each new release more function can be offloaded –  For example TRUNC is but RANK is not offloadableCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  6. 6. Exadata and the OptimizerHistory•  The plan always shows an eligible operation is offloaded The display of the word STORAGE is actually controlled by the parameter cell_offload_plan_display it is not an indication that offloading will or has occurred, just that it is possibleCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  7. 7. Exadata and the OptimizerHistory•  The plan always shows an eligible operation is offloaded The word STORAGE is in the predicate information is controlled by another internal optimizer parameterCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  8. 8. Exadata and the Optimizer Ideally I want an Exadata aware Optimizer Actual TranslationOptimizer should pick operations that perform best on Exadata Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  9. 9. Exadata and the Optimizer In reality I want an Exadata aware Optimizer Actual TranslationOptimizer should pick a full table scan instead of index access Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  10. 10. Exadata and the Optimizer Overview Optimizer selects the wrong plan because Incorrect Not accounting Not accountingcardinality correctly for for HCC estimate speed of scans decompression Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  11. 11. Exadata and the Optimizer Why the Optimizer picks the wrong •  More than 85% of the plan plan broken down by the percentage problems reported on Exadata are caused by Incorrect cardinality estimates Incorrect cardinality •  Fix cardinality estimates estimate before trying any other tuning techniques Not accounting correctly for speed of •  Typically fixing the cardinality scans estimates achieves the Not accounting for desired plan HCC decompressionCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  12. 12. Exadata and the Optimizer Overview Optimizer selects the wrong plan because Incorrect Not accounting Not accountingcardinality correctly for for HCC estimate speed of scans decompression Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  13. 13. Check Cardinality EstimatesSELECT /*+ gather_plan_statistics */ p.prod_name, SUM(s.quantity_sold)FROM sales s, products pWHERE s.prod_id =p.prod_id GROUP By p.prod_name ;SELECT * FROM table ( DBMS_XPLAN.DISPLAY_CURSOR(FORMAT=>ALLSTATS LAST));Compare the estimated number of rows returned for each operation in the plan to actual rows returned Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  14. 14. Checking Cardinality estimates for PESELECT /*+ gather_plan_statistics */ p.prod_name, SUM(s.quantity_sold)FROM sales s, products pWHERE s.prod_id =p.prod_id GROUP By p.prod_name ;SELECT * FROM table ( ALLSTATS ALL DBMS_XPLAN.DISPLAY_CURSOR(FORMAT=>ALLSTATS LAST)); Note: a lot of the data is zero in the A-rows column because we only show last executed cursor which is the QC. Need to use ALLSTATS ALL to see info on all parallel server cursorsCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  15. 15. Check Cardinality using SQL MonitorSQL Monitor is the easiest way to compare the estimated number of rows returned for each operationin a parallel plan to actual rows returned Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  16. 16. The Optimizer and Exadata Incorrect cardinality estimatesCommon causes for cardinality issues SolutionsStale or missing statistics DBMS_STATS using AUTO_SAMPLE_SIZEData Skew Re-gather statistics to get histogramsMultiple single column predicates on a Create a column group usingtable DBMS_STATS.CREATE_EXTENDED_STATSFunction wrapped column Create statistics on the function wrapped column using DBMS_STATS.CREATE_EXTENDED_STATSMultiple columns used in a join Create a column group on join columns using DBMS_STATS.CREATE_EXTENDED_STATComplicated expression containing Use dynamic sampling level 4 or highercolumns from multiple tables Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  17. 17. The Optimizer and ExadataDynamic Sampling In Oracle Database 11g Release 2•  Optimizer automatically decide if dynamic sampling will be useful & what level for parallel SQL statements –  Decision is based on size of the tables in the statement –  The complexity of the predicates•  If OPTIMIZER_DYNAMIC_SAMPLING is explicitly set then that specified value will be honored•  You can tell if dynamic sampling kicks in by looking in the note section of the execution planCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  18. 18. Exadata and the Optimizer Overview Optimizer selects the wrong plan because Incorrect Not accounting Not accounting cardinality correctly for for HCC estimate speed of scans decompressionStale Incomplete ComplexStats stats Expressions Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  19. 19. Exadata and the Optimizer Overview Optimizer selects the wrong plan because Incorrect Not accounting Not accounting cardinality correctly for for HCC estimate speed of scans decompressionStale Incomplete ComplexStats stats Expressions Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  20. 20. Exadata and the Optimizer Not accounting for speed of scans •  On a non-Exadata system the Non-Exadata System: Indexes vs. FTS cost of a full table scan far out weighs the cost of an index range scan or even a fast full index scanCost •  Typically the elapse times follows suit Full Table Scan Index Scan Rows Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  21. 21. Exadata and the Optimizer Not accounting for speed of scans •  On an Exadata system the Exadata System: Indexes vs. FTS cost of the full table scan still excesses the cost of an index scan •  BUT in reality the elapse timeTimeCost can often be much faster Full Table Scan Index Scan Rows Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  22. 22. Exadata and the OptimizerNot accounting correctly for the speed of scan•  TEST_TBL table contains 20 million rows –  22 Columns consisting of numbers and string data types•  Compressed using HCC query high•  Index created on UK column –  This is a unique index –  Values range from 1 to 20 millionCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  23. 23. Exadata and the OptimizerNot accounting for speed of scansQuery: Create table <name> as Select * from test_tbl t where UK <= :VAL;Default plan is index range scanElapse time is 13.33 secondsCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  24. 24. Exadata and the OptimizerNot accounting for speed of scansQuery: Create table <name>as Select /*+ full(t) */ * from test_tbl t where UK <= :VAL;Forced plan is full table scanElapse time is 5.8 secondsCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  25. 25. Exadata and the OptimizerNot accounting for speed of scans•  So the answer is simple •  Drop all of the index •  Everything will be faster via a full table scan•  Not so fast •  What happens when you have multiple concurrent user?Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  26. 26. Exadata and the Optimizer Not accounting for speed of scans •  With Exadata the number of Performance of concurrent Full Table scan concurrent scan operations that can be sustained before the elapse time gets too high is greater due to offloadTime •  Predicate push-down Non-Exadata •  Column projection Exadata •  But at some point it stops scaling •  Optimizer needs to be aware of this too # of concurrent scans Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  27. 27. Exadata and the OptimizerNot accounting for speed of scans•  How can you influence the Optimizer to pick a full table scan even with an index present?•  DO NOT use OPTIMIZER_INDEX_COST_ADJ •  It doesn’t influence full table scan cost •  It is a multiplier on the cost of an index access •  Setting it to 400 makes the index cost jump 4X from 2028 to 8114Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  28. 28. Exadata and the OptimizerNot accounting correctly for the speed of scan•  Better to look at statistics that influence Optimizer’s decisions•  Cost of scan determined by Number of blocks to read MBRC•  What is Multi Block Read Count (MBRC)? –  Number of db blocks read in a single OS read command –  On Exadata one OS read is 1MB of data or 128 8KB db blocks But the Optimizer always uses MBRC of 8Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  29. 29. Exadata and the Optimizer Not accounting correctly for the speed of scan •  By default Optimizer uses MBRC of 8 instead 128 •  In 8.1.6. wanted to decouple Optimizer MBRC from the parameter DB_FILE_MULTI_BLOCK_READ_COUNT –  This was to allowed the database to request larger IOs without changing any of the existing execution plans •  Cost of full table scan becomes expensive to soon on ExadataHow do you tell the Optimizer what IO a hardware system can do? Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  30. 30. Exadata and the OptimizerNot accounting correctly for the speed of scan •  System statistics tell the Optimizer about the speed of IO on a system •  By default system statistics does not gather MBRC •  Only buffer IO are monitored •  Direct path IO are excludedCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  31. 31. Exadata and the OptimizerAccounting correctly for the speed of scan •  New GATHER_SYSTEM_STATS mode exclusively for EXADATA •  Monitors and captures MBRC •  For 8KB block size captures 128 •  New method for monitoring IOTFRSPEED •  Captures roughly 200MB per second per process •  Requires patch for Bug 10248538Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  32. 32. Exadata and the OptimizerAccounting for speed of scansDefault plans afterexec dbms_stats.gather_system_stats(EXADATA); Cost of full table can is now lower than the index range scan of 21120Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  33. 33. Exadata and the OptimizerAccounting for speed of scansQuery: Select * from test_tbl where UK <= :VAL;Full table scan plan with default system statisticsFull table scan plan with EXADATA system statisticsCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  34. 34. Exadata and the Optimizer Accounting for speed of scans •  With the correct system Exadata System: Indexes vs. FTS statistics in place the cost for a full table scan more accurately reflects the expected performanceCost •  This will not prevent the Optimizer from selecting an index •  It just lowers the inflection Full Table Scan point where the plan will switch from an index access to Index Scan a full table scan Rows Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  35. 35. Exadata and the Optimizer Overview Optimizer selects the wrong plan because Incorrect Not accounting Not accounting cardinality correctly for for HCC estimate speed of scans decompressionStale Incomplete Complex By defaultStats stats Expressions Optimizer assumes MBRC = 8 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  36. 36. Exadata and the Optimizer Overview Optimizer selects the wrong plan because Incorrect Not accounting Not accounting cardinality correctly for for HCC estimate speed of scans decompressionStale Incomplete Complex By defaultStats stats Expressions Optimizer assumes MBRC = 8 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  37. 37. Exadata and the Optimizer How HCC decompression works Buffer Cache •  For single block access decompression happens on Compression Unit database server Column 1 Column 2 Column 3 •  Compression Unit is read from disk and stored compressed in buffer cache •  Decompression occurs in the PGA and decompressed data PGA is not sharedCompressionUnit •  Only one Compression UnitColumn 1 Column 2 Column 3 Decompression can be decompressed at a time Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  38. 38. Exadata and the Optimizer Not accounting for HCC decompressionQuery: Select * from test_tbl where UK < 96;Plan: TEST_TBL Access by rowid UK_IND Range Scan Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  39. 39. Exadata and the Optimizer Data in UK_ID HCC decompression with index access index ordered sequentially Buffer Cache Index on UK Compression Compression 1 Compression Unit Unit 1 Unit 2 is read from 1 2 Column 1 Column 2 Column 1 Column 2 2 disk & stored 3 4 compressed in 5 6 3 7 8 buffer cache 4 TEST_TBL TABLE PGA 5Query high HCC Compression Compression Compression 6sorts data before Unit 1 Unit 2 Unit 2 1compressing it odd 1 2 2 Column 1 Column 2 Column 1 Column 2 1 Column 1 Column 2 Column 1 Column 2 7numbers in one 3 4 Decompress 4 3Better yet OptimizerHow does how do we prevent the repeatedcompression unit & 7 8 Unit 2 decompression? 5 account for the repeated decompression? 6 1 6 5 8 8even in the other 7 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  40. 40. Exadata and the Optimizer Overview Optimizer selects the wrong plan because Incorrect Not accounting Not accounting cardinality correctly for for HCC estimate speed of scans decompressionStale Incomplete Complex By default Decompress isStats stats Expressions Optimizer CPU intensive assumes &results are MBRC = 8 not shared Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  41. 41. Exadata and the Optimizer Summary Exadata System baseline Accounting for speed of Scan Accounting for decompression Cost CostCost Full Table Scan Full Table Scan Full Table Scan Index Scan Index Scan Index Scan Rows Rows Rows Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  42. 42. Exadata and the Optimizer Summary•  If you are getting an index access instead of a full table scan –  Ensure the cardinality estimates are accurate –  Confirm the full table scan is actually more performant •  If so apply patch for 10248538 •  Gather system stats with EXADATA option –  Should be available in Exadata patch bundles 18 on 11.2.0.2 (11.2.0.2.18 –  Should be available in Exadata patch bundles 8 on 11.2.0.3 (11.2.0.3.8)•  If the index access is needed on a HCC table –  Apply patch for 12780479 (available soon) Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  43. 43. More Information•  White papers –  Technical overview of the Exadata Database Machine –  What to expect from the Oracle Optimizer in 11g•  Optimizer Blog –  http://blogs.oracle.com/optimizer•  Oracle.com –  http://www.oracle.com/us/products/database/exadata/overview/ index.html –  http://www.oracle.com/technetwork/database/focus-areas/bi- datawarehousing/dbbi-tech-info-optmztn-092214.htmlCopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  44. 44. Q&ACopyright © 2012, Oracle and/or its affiliates. All rights reserved.
  45. 45. Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

×