Oracle 12c Optimizer IOUG

2 Copyright © 2012, Oracle and/or its affiliates. All rights
reserved.
Insert Information Protection Policy Classification from Slide 8
Oracle Optimizer:
12c New Capabilities
Thomas Kyte
http://asktom.oracle.com/
ORACLE
PRODUCT
LOGO
Wifi sid: guest-access
Wifi username: Q7D3KcRn
Wifi password: 3dF2DneR

Oracle safe harbor statement
The following is intended to outline our general product direction. It is
intended for information purposes only, and may not be incorporated into
any contract. It is not a commitment to deliver any material, code, or
functionality, and should not be relied upon in making purchasing
decisions. The development, release, and timing of any features or
functionality described for Oracle's products remains at the sole
discretion of Oracle.

Optimizer evolution
In the beginning
there were rules
Optimizer evolved to
be cost based
CBO
CBO
Optimizer proactively adapts
to become self-learning
Rule are not enough
Databases became more
feature rich
Reactive tuning with
the use of advisors
and auto jobs
As environment changes
Potential for plan changes
Databases become
more real-time, ad-hoc
environments
Reactive tuning not enough

Agenda
 Adaptive Query Optimization
 Optimizer Statistics enhancements
 SQL Plan Management enhancements

Adaptive Plans
Join methods
Parallel distribution methods
Adaptive Statistics
Discovered at compile time
Discovered at run time
Adaptive Query Optimization
Overview
Adaptive Query
Optimization
Adaptive Plans
Adaptive
Statistics
Join
Methods
Parallel
distribution
Methods
At compile
time
At run
time

Query: Find all of the
products with a unit price of
15 that we have sold more
that 1 of
Two possible join methods
for this query
Nested Loops
Hash Join
Adaptive Execution Plans
Adapt join methods

Alternative sub-plans are
pre-computed
Sub-plans stored in the
cursor
Stats collect inserted before
join
Rows buffered until final
decision is made
Adapt join methods
Table scan
Order _items
NESTED LOOPS
Index Scan
Prod_info_ind
Statistics
Collector
Default Plan is a nested loops join
HASH JOIN
Table scan
Prod_info
Rows coming out of order_items table are buffered up
to a point If row count is less than the threshold use
nested Loops otherwise switch to hash join

Number of rows seen in
statistics collector exceeds
threshold
Plan switches to hash join
Statistics collect disabled
Plan resolved on first
execution & remains the
same for subsequent
executions
Adapt join methods
Final Plan is a hash join
Statistics collector disabled after decision is
made and becomes a pass through operation
Table scan
Order _items
NESTED LOOPS
Index Scan
Prod_info_ind
HASH JOIN
Table scan
Prod_info
Statistics
Collector

Explain plan command
always shows default plan
Example shows a nested
loops join as default plan
No statistics collector shown
in plan
Displaying the default plan

After the statement has
completed use
DBMS_XPLAN.DISPLAY_CURSOR
to see the final plan selected
Example shows that hash
join picked at execution time
Again the statistics collector
is not visible in the plan
Displaying the final plan

Full adaptive plan displayed
when format parameter
‘+adaptive’ is set
Example shows both the
nested loops and hash join
in the plan
Displaying the full adaptive plan

Additional information
displayed on why operations
are inactive can be seen
with format parameter
‘+report’
Displaying plan with +adaptive & +report formats

New column in V$SQL
IS_RESOLVED_ADAPTIVE_PLAN
Indicates statement had an
adaptive plan which was
resolved on first execution
Resolved plan is used for
subsequent executions
Statistics collectors and
buffering is disabled
Indicator in V$SQL

Adaptive plans are enabled by
default
Can be put in reporting mode
OPTIMIZER_ADAPTIVE_REPORTING_ONLY
Reporting mode shows what
would have happened during
execution in the plan
Reporting mode

 Join method decision deferred until runtime
– Default plan is computed using available statistics
– Alternate sub-plans are pre-computed and stored in the cursor
– Statistic collectors are inserted at key points in the plan
 Final decision is based on statistics collected during execution
– Default plan and sub-plans have a valid range for statistics collected
– If statistics prove to be out of range, the sub-plans will be swapped
– Requires buffering near the swap point to avoid returning rows to user
 Possible sub-plans are nested loop joins or hash joins and vice versa
Adaptive join methods

Adaptive Plans
Join methods
Adaptive Statistics
Overview
Adaptive Query
Optimization
Adaptive Plans
Adaptive
Statistics
Join
Methods
Parallel
distribution
Methods
At
compile
time
At run
time

Dynamic Statistics
 During compilation optimizer decides if statistics are sufficient to
generate a good plan or not
 Dynamic statistics are used to compensate for missing, stale, or
incomplete statistics
 They can be used for table scans, index access, joins and group bys
 One type of dynamic statistics is dynamic sampling

Dynamic sampling has a
new level 11(AUTO)
Decision to use dynamic
sampling depends on the
complexity of predicate,
existing statistics and
total execution time
Dynamic statistics shared
among queries
Dynamic Statistics
Dynamic Sampling

Dynamic Statistics SGA
SHARED_POOL
Select *
From product_information
Where list_price-min_price=29
And category_id not in (11,22)
And prod_name like ‘Smart%’;
Optimizer checks
for existing statistics
in data dictionary
2
SHARED DYNAMIC STATISTICS
DATA DICTIONARY
SQL statement
is submitted
1
Optimizer determines
plan using dynamic
statistics
5
Cursor 0: Select * from product_information …
Dynamic sampling
occurs on small number of
blocks from table
Resulting dynamic statistics
are stored in cache
4
Statistics found but need to be
augmented due to complex predicates
3
Table cardinality estimate for prod_info

Dynamic Statistics SGA
SHARED_POOL
Select supplier_id, prod_name
From product_information
Where list_price-min_price=29
And category_id not in (11,22)
And prod_name like ‘Smart%’;
Optimizer checks
for existing statistics
in data dictionary
7
SHARED DYNAMIC STATISTICS
DATA DICTIONARY
Different SQL
statement is
submitted with
same predicates
6
plan using dynamic
statistics
10
Cursor 0: Select supplier_id, prod_name …
Necessary Dynamic
statistics found in shared
cache
9
Statistics found but need to be
augmented due to complex predicates
8
Table cardinality estimate for prod_info

Adaptive Statistics
 During execution optimizer estimates are compared to execution statistics
 If statistics vary significantly then a new plan will be chosen for
subsequent executions based on execution statistics
 Re-optimization uses statistics gathered from previous executions
 First introduced as Cardinality Feedback in 11.2
Re-optimization

Adaptive Statistics
 Statistics gathered about data volume and data type seen during execution
 If execution statistics vary significantly statement will be hard parsed on
the next execution using the execution statistics instead
 Statements are only monitored once if they don’t show significant
differences initially they won’t change in the future
 Only individual table cardinalities and group by estimates examined
 Information is stored in the cursor only and is lost if cursor ages out
Cardinality Feedback pre 12c

Initial execution of a query
is monitored
Example shows initial plan
is hash join between sales
and customers
Cardinality estimates 8X off
Adaptive Statistics
Initial Cardinality estimates are more than 8X off

Execution statistics used to
reparse the statement on
the second execution
New plan shows correct
cardinality estimates and a
new join order
Information learnt is stored
in the cursor only and is lost
if cursor ages out
Adaptive Statistics
Execution Plan after the second execution
Estimates
are now
correct

Adaptive Statistics
 Join statistics are also monitored
 Works with adaptive cursor sharing for statement with binds
 New Column in V$SQL IS_REOPTIMIZABLE
 Information found at execution time is persisted as SQL Plan Directives
Re-optimization in 12c

New column in V$SQL
IS_REOPTIMIZABLE
Indicates that the statement
will be re-parsed on the next
execution
Adaptive Statistics
Re-optimization – indicator in V$SQL

Adapt join method example
showed the join change from
nested loops to hash join
But cursor is also marked
IS_REOPTIMIZABLE
Why?
Re-optimization will occur on
second execution because
cardinality estimates off
Adaptive Statistics
Adaptive Plans & Re-optimization working together
Initial Cardinality estimates are off

On second execution
statement is re-parsed
using execution statistics
New child cursor created
with new plan
New plan changes the join
order
Adaptive Statistics
Adaptive Plans & Re-optimization working together

Adaptive Statistics
 Directives are additional information used during optimization to
generate a better plan
– For example, when table T1 is joined to T2 use dynamic statistics to get
accurate cardinality estimate
– Directives are collected on query expressions not at a statement level
 Allows for directives to be used for multiple statements
– Persisted on disk in the SYSAUX tablespace
– Directives will be automatically maintained
– Managed using the new package DBMS_SPD
SQL Plan Directives

SQL Plan Directives SGA
SHARED_POOL
Select *
FROM customers
where state =‘CA’
AND country=‘USA’;
SQL statement
is submitted
Cursor 0: Select * from customers
……..
plan
2
DIRECTIVE CACHE
During execution
cardinality estimate
discovered to be wrong
and directive created
3
IS_REOPTIMIZABLE = Y
Directive: Use DS for customers table
when column city, country are used
1
Sysaux Tablespace

SQL Plan Directives
Select *
FROM customers
where state=‘CA’
SQL Directives
Sysaux Tablespace
SGA
This execution and
subsequent execution
use child cursor 1
6
Directive: Use DS for customer table
when column city, country are used
SHARED_POOL
Cursor 0: Select * from customers …..
IS_REOPTIMIZABLE = Y
new plan using
execution statistics
5
Cursor 1: Select * from customers …..
IS_REOPTIMIZABLE = N
Directives periodically
persisted to disk in the
sysaux tablespace
Extension Binds Stats Sqlid
SQL Directives
DIRECTIVE CACHEDIRECTIVE CACHE
Same SQL
statement is
submitted
again
4

SQL Plan Directives
Select email
FROM customers
where state=‘MA’
New SQL
statement is
submitted on
customers table
7
Optimizer
checks for
directives on
customers table
and finds one on
the columns state
and country
8
Cursor 0: Select email from customers
plan with help of
directive
9
Optimizer adds
column group creation
for state & country
columns to next statistics
gather on customer table
10
SGA
DIRECTIVE CACHE
SHARED_POOL
Sysaux Tablespace
Extension Binds Stats Sqlid
SQL Directives
DIRECTIVE CACHE

Number of SQL Plan
Directives used for a
statement is displayed in
the notes section of the
execution plan
In this case one SQL
Plan Directive was used
SQL Plan Directives
In use

SQL Plan Directives
Monitoring
Auto flushed every 15 minutes

SQL Plan Directive state
was “MISSING_STATS”
Indicates Oracle can
automatically gather
statistics to improve the
original mis-estimation
In this case a column
group on STATE and
COUNTRY will address
the problem
SQL Plan Directives
Gather statistics to allow automatic creation of missing statistics

SQL Plan Directives
Monitoring

Correct statistics in place
Optimizer uses statistics
instead of directives
Accurate cardinality
estimate seen in the plan
Statement not marked for
re-optimization
SQL Plan Directives
Final plan no longer needs the directive

Adaptive Plans
Join methods
Adaptive Statistics
Overview
Adaptive Query
Optimization
Adaptive Plans
Adaptive
Statistics
Join
Methods
Parallel
distribution
Methods
At
compile
time
At run
time

Summary
 Optimizer begins to learn from its mistakes
– Adaptive Plans
 Adapt join methods
 Adapt parallel distribution methods
– Adaptive Statistics
 Dynamic statistics
 Re-optimization
 SQL Plan Directives
 And lots of other great features!

Agenda

Statistics
Enhancements
 New types of histograms
 Online statistics gathering
 Session level statistics for GTTs
 Enhanced incremental statistics
 Concurrent statistics gathering
 Automatic detection of column groups
 Statistic gathering reporting

Histograms
 Histograms tell Optimizer about the data distribution in a Column
 Creation controlled by METHOD_OPT parameter
 Default create histogram on any column that has been used in the
WHERE clause or GROUP BY of a statement AND has a data skew
 Relies on column usage information gathered at compilation time and
stored in SYS.COL_USAGE$
 Four types of histograms
– Frequency
– Top-Frequency
Oracle Confidential
– Height balanced
– Hybrid

Histograms
 A frequency histogram is only created if the number of distinct values in
a column (NDV) is less than 254 values
Frequency Histograms (FREQUENCY)
Frequency histogram
Oracle Confidential

Histograms
 A height balanced histogram is created if the number of distinct values
in a column (NDV) is greater than 254 values
Height Balanced Histograms (HEIGHT BALANCED)
Height balanced histogram

Histograms
 Traditionally a frequency histogram is only created if NDV < 254
 But if a small number of values occupies most of the rows (>99% rows)
 Creating a frequency histograms on that small set of values is very
useful even though NDV is greater than 254
• Ignores the unpopular values to create a better quality histogram for
popular values
 Built using the same technique used for frequency histograms
 Only created with AUTO_SAMPLE_SIZE
Top Frequency (TOP-FREQUENCY)

Histograms
 A Hybrid histogram is created if the number of distinct values in a column
(NDV) is greater than 254 values and Top Frequency requirements are not met
Hybrid Histograms (HYBRID)
Hybrid histogram

Histograms
 Similar to height balanced histogram as created if the NDV >254
 Store the actual frequencies of bucket endpoints in histograms
 No values are allowed to spill over multiple buckets
 More endpoint values can be squeezed in a histogram
 Achieves the same effect as increasing the # of buckets
 Only created with AUTO_SAMPLE_SIZE
Hybrid Histograms (HYBRID)

Statistics
Enhancements

Online Statistics Gathering
 Statistics gathered as part of the direct path load operations
– Create Table As Select or Insert As Select commands
 Statistics available directly after load
 No additional table scan required to gather statistics
 All internal maintenance operations that use CTAS benefit from this
 Note only occurs on IAS if table is empty

New table SALES2 is
created using a CTAS
command
Both table and column
level statistics are
available immediately
after table has been
created
Online Statistics Gathering Example
Statistics immediately available

Histogram & index
statistics not gathered
To gather these statistics
without re-gathering
basic statistics set option
parameter of
GATHER_TABLE_STATS
to GATHER AUTO

Histogram & index
statistics gathered using
new GATHER AUTO
option without
regathering base column
statistics

Statistics
Enhancements

Session private statistics for GTT
 Traditionally statistics gathered on GTT were shared by all sessions
 Share statistics are not always optimal
 Now each session can have its own version of statistics for GTT
 Controlled by new preference GLOBAL_TEMP_TABLE_STATS
 Default value is SESSION (non shared)
 To force sharing (as in 11g) set table preference to SHARED
Overview

Statistics gathered on a
GTT are no longer
shared by all sessions
To restore shared
statistics change the
table preference
GLOBAL_TEMP_TABLE_STATS
to SHARED
Session private statistics for GTT
By default statistics on GTT are session-private

Statistics
Enhancements

Enhanced Incremental Statistics
 Incremental statistics allows global level statistics to accurately
generated from partition level statistics
 NDV statistics can now be accurately aggregated by the introduction of
the synopsis
 The synopses are stored in the Sysaux tablespace
 In 12c reduced the space required to store synopses on disk

NDV statistics can now
be accurately aggregated
The synopses are stored
in the Sysaux tablespace
12c reduces the space
required to store
synopses on disk
Enhanced Incremental Statistics
Sales Table
May 22nd 2012
May 23rd 2012
May 18th 2012
May 19th 2012
May 20th 2012
May 21st 2012
Sysaux Tablespace
1. Partition level stats are
gathered & synopsis created
2. Global stats
generated by
aggregating partition
level statistics and
synopsis

Enhanced Incremental Statistics for Partition
Exchange
DBA
1. Create external table for flat files
3. Set INCREMENTAL to true &
INCREMENTAL_LEVEL to TABLE
4. Gather Statistics
Sales Table
May 22nd 2012
May 23rd 2012
May 18th 2012
May 19th 2012
May 20th 2012
May 21st 2012
2. Use CTAS command to create non-partitioned
table TMP_SALES
TMP_SALES
May 24th 2012
Sysaux Tablespace
6. Global stats generated
by aggregating partition
level statistics for existing
partition with stats on new
partition
5. Alter table Sales exchange partition May_24_2012 with
table tmp_sales
May 24th 2012

During data load some rows
going to the older partitions
In 11g any DML on older
partitions triggered partition
statistics to be re-gathered
New DBMS_STATS preference
INCREMENTAL_STALENESS
When set to
USE_STALE_PERCENT DML on less
than 10% of rows in older
partitions will not trigger re-
gather
Enhanced Incremental Statistics Staleness
Tolerance
Sales Table
May 22nd 2012
May 23rd 2012
May 18th 2012
May 19th 2012
May 20th 2012
May 21st 2012
Sysaux Tablespace
1. Partition level stats are
gathered & synopsis created
2. Global stats
generated by
aggregating partition
level statistics and
synopsis

Statistics
Enhancements

Statistics Enhancements
• Gather statistics on multiple objects at the same time
• Controlled by DBMS_STATS preference, CONCURRENT
• Uses Database Scheduler and Advanced Queuing
• Number of concurrent gather operations controlled by
job_queue_processes parameter
• Each gather operation can still operate in parallel
Concurrent Statistics Gathering – Originally introduced in 11.2

A statistics gathering job is
created for each table and
partition in the schema
Level 1 contain statistics
gathering jobs for all non-
partitioned tables and a
coordinating job for each
partitioned table
Level 2 contain statistics
gathering jobs for each
partition in the partitioned
tables
Concurrent Statistics Gathering for SH Schema
Exec DBMS_STATS.GATHER_SCHEMA_STATS(‘SH’);

The number of concurrent
gathers is controlled by the
parameter
job_queue_processes
In this example it is set to 3
Remember each concurrent
gather operates in parallel
In this example parallel
degree is 4
Concurrent Statistics Gathering on Partition
Table
Exec DBMS_STATS.GATHER_TABLE_STATS(‘SH’,’SALES);
Sales Table
Partition 1
May 18th 2012
Partition 2
May 19th 2012
Partition 3
May 20th 2012
Job1
Job2
Job3

Statistics Enhancements
 Batch manager to batch smaller jobs together to reduce scheduling
over head
 Auto statistics gather job can now use concurrency
 Cap to limit the resource available to the job via resource manager

Statistics
Enhancements

Extended Statistics
 Two types of Extended Statistics
– Column groups statistics
 Column group statistics useful when multiple column from the same table are
used in where clause predicates or group by clause
– Expression statistics
 Expression statistics useful when a column is used as part of a complex
expression in where clause predicate
 Automatically maintained when statistics are gathered on the table
 Candidates for columns groups can be manually or automatically
determined

Oracle can automatically
detect column group
candidates based on an
STS or by monitoring a
workload
Uses DBMS_STATS procedure
SEED_COL_USAGE
If the first two arguments are
set to NULL the current
workload will be monitored
The third argument is the
time limit in seconds
Automatic Column Group Detection
1. Start column group usage capture

Actual number of rows
returned by this query is 932
Optimizer under-estimates
the cardinality as it assumes
each where clause
predicate will reduce
number of rows returned
Optimizer is not aware of
real-world relations between
city, state, & country
2. Run your workload

Actual number of rows
returned by this query is 145
Optimizer over-estimates
the cardinality as it is not
aware of the real-world
relations between state &
country
2. Run your workload

3. Check column usage information recorded for our table
COLUMN USAGE REPORT FOR SH.CUSTOMERS
1. COUNTRY_ID : EQ
2. CUST_CITY : EQ
3. CUST_STATE_PROVINCE : EQ
4. (CUST_CITY, CUST_STATE_PROVINCE, COUNTRY_ID) : FILTER
5. (CUST_STATE_PROVINCE, COUNTRY_ID) : GROUP_BY
SELECT dbms_stats.report_col_usage(user, 'customers') FROM dual;
EQ means column was used in
equality predicate in query 1
GROUP_BY columns used in group
by expression in query 2
FILTER means columns used together
as filter predicates rather than join etc.
Comes from query 1

4. Create extended stats for customers based on usage
EXTENSIONS FOR SH.CUSTOMERS
1. (CUST_CITY, CUST_STATE_PROVINCE, COUNTRY_ID):
SYS_STUMZ$C3AIHLPBROI#SKA58H_N created
2. (CUST_STATE_PROVINCE, COUNTRY_ID) :
SYS_STU#S#WF25Z#QAHIHE#MOFFMM_ created
Column group statistics will now be automatically maintained
every time you gather statistics on this table
SELECT dbms_stats.create_extended_stats(user, 'customers') FROM dual;

Statistics
Enhancements

DBMS_STATS.REPORT_STATS_OPERATIONS

DBMS_STATS.REPORT_GATHER_SCHEMA_STATS

Agenda

SQL Plan Management
How it works
Parse
HJ
HJ
GB
Plan history
Plan baseline
Execute
Plan Acceptable
HJ
HJ
GB
Users
Actual execution plans stored in SQL
plan baseline in Oracle Database 12c

SQL Plan Management
How it works
Parse
NL
NL
GB
Plan history
Plan baseline
HJ
HJ
GB
Users
NL
NL
GB
Note you do not need to be in auto-
capture mode to have a new plan
added to an existing SQL plan baseline
Note additional fields such as fetches,
row processed etc. are not populated
because new plan never executed

SQL Plan Management
How it works
Parse
Plan history
Plan baseline
Execute
Plan Acceptable
HJ
HJ
GB
Users
NL
NL
GB
HJ
HJ
GB

Adaptive SQL Plan Management
 New evolve auto task running in the maintenance window
– Ranks all non-accepted plans and runs evolve process for them
 Newly found plans are ranked the highest
– If new plan performs better than existing plan it is automatically accepted
– If new plan performs worse than existing plan it will remain unaccepted
– Poor performing plans will not be retried for 30 days and then only if the
statement is active
SPM Evolve Advisor

Automatic Plan Evolution
Plan history
Plan baseline
NL
NL
GB
HJ
HJ
GB
Nightly verification task checks for
unaccepted plans
It then test execute
to check if new plan is
as good as or better
than old plan
1
2
Plan history
HJ
HJ
GB
Plan baseline
NL
NL
GB
Plans which perform better than
original plan are recommended
to be added to the plan baseline
3
L
Plans which don’t perform as good
as the original plan stay in the
plan history & are marked
unaccepted but last_verified updated
NL
NL
GB
4

SPM Evolve Advisor
New nightly auto task
called
SYS_AUTO_SPM_EVOLVE_TASK
Information on task found
in DBA_ADVISOR_TASKS
Adaptive SQL Plan Management
You can review what happening during
the task using DBMS_SPM.
REPORT_AUTO_EVOLVE_TASK

Reports are persisted
and can be review any
time
Clearer findings section
summaries the results of
the evolve task
New recommendation
section that clearly states
how to implement the
recommendations
proposed
Reviewing the evolve report
Same
comparison
criteria as in 11g

If you can hint it, baseline it
• It is not always possible to add hints to third party applications
• Hints can be extremely difficult to manage over time
• Once added never removed
Alternative approach to hints
Solution
 Use SQL Plan Management (SPM)

Influencing plan without adding hints
• Simple two table join between the SALES and PRODUCTS
Example Overview
Group By
HASH JOIN
TABLE ACCESS
SALES
TABLE ACCESS
PRODUCTS
Group By
HASH JOIN
TABLE ACCESS
SALESINDEX RANGE SCAN
PROD_SUPP_ID_INDX
Current Plan Desired Plan

Step 1. Execute the non-hinted SQL statement

Default plan uses full table scans followed by a hash join

Step 2. Find the SQL_ID for the non-hinted statement in V$SQL

Step 3. Create a SQL plan baseline for the non-hinted SQL statement

Step 4. Captured Plan is not our desired plan so it should be disabled

Step 5. Modify the SQL statement to use the hint(s) & execute it

Step 6. Find SQL_ID & PLAN_HASH_VALUE for hinted SQL stmt

Step 7. Associate hinted plan with original SQL stmt’s SQL HANDLE
Sql_id & plan_hash_value
belong to hinted statement
sql_handle is for
the non-hinted
statement

Step 8. Confirm SQL statement has two plans in its SQL plan baseline
Hinted plan only enabled plan for
non-hinted SQL statement

Step 9. Confirm hinted plan is being used
Non-hinted SQL text but
it is using the plan hash
value for the hinted
statement
Note section also
confirms SQL plan
baseline used for
statement

Oracle 12c Optimizer IOUG

Recommended

Recommended

More Related Content

What's hot

What's hot (13)

Similar to Oracle 12c Optimizer IOUG

Similar to Oracle 12c Optimizer IOUG (20)

More from Connor McDonald

More from Connor McDonald (20)

Recently uploaded

Recently uploaded (20)

Oracle 12c Optimizer IOUG