Successfully reported this slideshow.
Upcoming SlideShare
×

OHarmony - How the Optimiser works

So you are young and single – you are standing in a crowded room and you are looking to connect with all the people that match the criteria of your perfect partner. How do you identify and grab the details of those individuals as quickly as possible, without spending too much time talking to people you will never get on with. This presentation will use a situation we have all been familiar with to explain how the optimizer works, the steps in the optimisation process and some of the rules that help it to score a match as efficiently as possible.

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

OHarmony - How the Optimiser works

1. 1. www.sagecomputing.com.au penny@sagecomputing.com.au OHarmony – Finding Your Perfect Match How the Optimizer Works Penny Cookson SAGE Computing Services SAGE Computing Services Customised Oracle Training Workshops and Consulting
2. 2. WARNING This presentation contains material which is not politically correct Includes adult concepts May contain strong language
3. 3. SAGE Computing Services Customised Oracle Training Workshops and Consulting Penny Cookson Managing Director and Principal Consultant Working with Oracle products since 1987 Oracle Magazine Educator of the Year 2004 www.sagecomputing.com.au penny@sagecomputing.com.au
4. 4. Optimiser Step 1 - Transformation Query rewrite for materialised views Transitive conditions OR expansion View merging Predicate pushing Join factorisation
5. 5. COL1 = :B1 AND COL2 >= :B2 AND COL2 < :B3 AND COL3 >= :B4 AND COL4 = :B5 ……………………………………. Col1 Col2 Col3 Col4 Col5 Col6 How many of these are there likely to be? How many rows will be returned? There are 23 million rows in this table
6. 6. ATTRIBUTE1 = :B1 AND ATTRIBUTE2 >= :B2 AND ATTRIBUTE2 < :B3 AND ATTRIBUTE3 >= :B4 AND ATTRIBUTE4 = :B5 How many of these are there likely to be? Looking for your perfect match There are 11 million males in Australia
7. 7. MARRIED = ‘N’ AND AGE >=25 AND AGE <30 AND HEIGHT >= 6ft 2 in AND JOB=‘DBA’ How many of these are there likely to be? There are 11 million males in Australia
8. 8. The attribute Married has two distinct values Yes or No 50% 50% We assume 50% of each How many people satisfy the criteria Married = ‘N’? There are 11 million males in Australia Number of Unmarried males is 11,000,000/2 = 5,500,000
9. 9. How many people satisfy the criteria Married = ‘N’?
10. 10. More Statistics 6 in every 10 males are married So - Number of Unmarried males is 4,400,000
11. 11. This is what we did originally begin dbms_stats.gather_schema_stats (ownname=>'AUSOUG', method_opt => 'for all columns size 1' ); end; How can Oracle get better statistics?
12. 12. begin dbms_stats.gather_table_stats (ownname=>'AUSOUG', tabname=>'MEN', method_opt => 'for all columns size 1, for columns size auto married' ); end; Create a histogram only for Married This is no longer relevant Because we have this
13. 13. Cummulative
14. 14. http://jonathanlewis.wordpress.com/2010/10/05/frequency-histogram-4/
15. 15. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM men WHERE married = 'N‘; SELECT dbms_xplan.display_cursor ('5zyycq4y22j9g',format=>'ALLSTATS LAST') FROM dual; Now it gets it right
16. 16. MARRIED = ‘N’ AND AGE >=25 AND AGE <30 AND HEIGHT >= 6ft 2 in AND JOB=‘DBA’ How many of these are there likely to be? There are 11 million males in Australia MARRIED = ‘N’  40% 
17. 17. Age statistics 8.2% of men are BETWEEN 25 and 29
18. 18. Age statistics
19. 19. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM men WHERE age >=25 and age < 30;
20. 20. create or replace function raw_to_num(i_raw raw) return number as m_n number; begin dbms_stats.convert_raw_value(i_raw,m_n); return m_n; end; / create or replace function raw_to_date(i_raw raw) return date as m_n date; begin dbms_stats.convert_raw_value(i_raw,m_n); return m_n; end; / create or replace function raw_to_varchar2(i_raw raw) return varchar2 as m_n varchar2(20); begin dbms_stats.convert_raw_value(i_raw,m_n); return m_n; end; / http://jonathanlewis.wordpress.com/2006/11/29/low_value-high_value/
21. 21. SELECT column_name, decode(data_type, 'VARCHAR2',to_char(raw_to_varchar2(low_value)), 'DATE',to_char(raw_to_date(low_value)), 'NUMBER',to_char(round(raw_to_num(low_value),2)) ) low_value, decode(data_type, 'VARCHAR2',to_char(raw_to_varchar2(high_value)), 'DATE',to_char(raw_to_date(high_value)), 'NUMBER',to_char(round(raw_to_num(high_value),2)) ) high_value FROM user_tab_columns WHERE table_name='MEN'; http://jonathanlewis.wordpress.com/2006/11/29/low_value-high_value/
22. 22. 5.10204… % * 11,000,000 = 561,224 0 98 25 30 SELECT TRUNC(11000000*(30-25)/(98)),trunc((30-25)/(98)*100,6) from dual
23. 23. Add a histogram begin dbms_stats.gather_table_stats(ownname=>'AUSOUG', tabname=>'MEN', method_opt => 'for all columns size 1 for columns size auto married, for columns size 254 age ' ); end;
24. 24. SELECT trunc(h.endpoint_value,2) ,h.endpoint_repeat_count from user_tab_histograms h, user_tables t WHERE t.table_name = h.table_name AND h.table_name = 'MEN' AND h.column_name = 'AGE' ORDER BY endpoint_value
25. 25. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM men WHERE age >=25 and age < 30; SELECT dbms_xplan.display_cursor('6aub3fn962277',format=>'ALLSTATS LAST') FROM dual Not perfect but better
26. 26. MARRIED = ‘N’ AND AGE >=25 AND AGE <30 AND HEIGHT >= 6ft 2 in (188cm) AND JOB=‘DBA’ How many of these are there likely to be? There are 11 million males in Australia MARRIED = ‘N’  40% AND AGE >=25 AND <30  8.2%  
27. 27. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM men WHERE height >= 188 SELECT dbms_xplan.display_cursor('bf4wbtgb9zptq', format=>'ALLSTATS LAST') FROM dual How many men are >= 188cm (we have gathered a histogram)
28. 28. MARRIED = ‘N’ AND AGE >=25 AND AGE <30 AND HEIGHT >= 6ft 2 in (188cm) AND JOB=‘DBA’ How many of these are there likely to be? There are 11 million males in Australia MARRIED = ‘N’  40% AND AGE >=25 AND <30  8.2% AND HEIGHT >= 6ft 2 in (188cm)  2.9%   
29. 29. 11000000*40/100 *8.2/100 * 2.9/100 =10,463 MARRIED = ‘N’  40% AND AGE >=25 AND <30  8.2% AND HEIGHT >= 6ft 2 in (188cm)  2.9% SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM men WHERE married = 'N' AND age >=25 AND age < 30 AND height >= 188
30. 30. Height Statistics
31. 31. 5.3% of males are >= 6ft 2inches Are these statistics out of date? - do select of last_analyzed
32. 32. 5.3% of males are >= 6ft 2inches Note the correlation between gender, age and height
33. 33. SELECT c.column_name, t.num_rows "Number of Rows", c.num_distinct "Distinct Values", c.histogram "Histogram" FROM user_tables t, user_tab_col_statistics c WHERE t.table_name = c.table_name AND t.table_name = 'MEN'
34. 34. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM men WHERE age =6 AND height = 174
35. 35. BEGIN dbms_stats.gather_table_stats(ownname=>'AUSOUG', tabname=>'MEN', estimate_percent=>NULL, method_opt => 'for all columns size 1 for columns size auto married height, for columns size 254 age, for columns (age, height) size 2048 ' ); END; Gather extended statistics
36. 36. SELECT * FROM dba_stat_extensions WHERE table_name = 'MEN'
37. 37. SELECT c.column_name, t.num_rows "Number of Rows", c.num_distinct "Distinct Values", c.histogram "Histogram" FROM user_tables t, user_tab_col_statistics c WHERE t.table_name = c.table_name AND t.table_name = 'MEN'
38. 38. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM men WHERE age =6 AND height = 174
39. 39. ALTER SESSION SET EVENTS ' 10053 trace name context forever ‘; EXPLAIN PLAN FOR …………; ALTER SESSION SET EVENTS ' 10053 trace name context off ‘;
40. 40. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM men WHERE age = 30 AND height = 174
41. 41. SELECT /*+ GATHER_PLAN_STATISTICS */ COUNT(*) FROM men WHERE married = 'N' AND age >=25 AND age < 30 AND height >= 188 When we combine range checks it gets it wrong
42. 42. 10053 trace file – its not looking at the extended stats ALTER SESSION SET EVENTS ' 10053 trace name context forever ‘; EXPLAIN PLAN FOR …………; ALTER SESSION SET EVENTS ' 10053 trace name context off ‘;
43. 43. BEGIN DBMS_STATS.DROP_EXTENDED_STATS('AUSOUG', 'MEN', '("AGE","HEIGHT","MARRIED")') ; END; BEGIN dbms_stats.purge_stats(sysdate); END;
44. 44. What about Statistics Feedback?
45. 45. Execute again
46. 46. begin dbms_spd.flush_sql_plan_directive; end; Clear any existing SQL Plan Directives SELECT d.directive_id, d.type, d.state, d.reason, d.created, o.object_name, o.subobject_name, o.notes, o.owner FROM dba_sql_plan_directives d, dba_sql_plan_dir_objects o WHERE o.directive_id = d.directive_id AND o.owner NOT IN ('XDB','SYS','SYSTEM') ORDER BY directive_id desc; begin dbms_spd.drop_sql_plan_directive(3579438123315094543); end;
47. 47. ALTER SYSTEM FLUSH shared_pool; ALTER SESSION SET statistics_level = ALL; Clear any existing plans and gather plan statistics
48. 48. Execute each of these twice
49. 49. Only the last two use statistics feedback
50. 50. UNMARRIED = ‘Y’ AND AGE >=25 AND AGE <30 AND HEIGHT >= 6ft 2 in (188cm) AND JOB=‘DBA’ How many of these are there likely to be? There are 11 million males in Australia UNMARRIED = ‘Y’  40% AND AGE >=25 AND <30  8.2% AND HEIGHT >= 6ft 2 in (188cm)  2.9% We have no idea what percentage of males are DBAs   
51. 51. begin dbms_stats.delete_schema_stats(ownname=>'AUSOUG' ); end; SELECT COUNT(*) FROM men WHERE job = 'DBA' Actual value is 1,000,014
52. 52. Now we know how many matches to expect what is the best way to get to them
53. 53. Two main types: Pretty cruisy really – almost anyone will do Really very picky – he must be just right
54. 54. The Oracle approach to Pretty cruisy really – almost anyone will do SELECT COUNT(*) FROM men WHERE age >=20 AND age < 50 AND married = 'N' AND job != 'LAWYER'
55. 55. Full table scan WHERE col1 = ' bbbbb ' aaaaa bbbbb ccccc ddddd eeeee aaaaa ggggg ccccc ddddd eeeee aaaaa kkkkk ccccc ddddd eeeee aaaaa bbbbb ccccc ddddd eeeee aaaaa bbbbb bbbbb ddddd eeeee Multi block read
56. 56. SELECT COUNT(*) FROM men WHERE age >=20 AND age < 50 AND married = 'N' AND job != 'LAWYER'
57. 57. Really very picky – he must be just right OHarmony I tell you what I want and you just give me the phone numbers where I can contact them
58. 58. OHarmony
59. 59. OHarmony Brown DBA 188 120,000 N 25Age From Age To 30 IQ (min) Married Smoker Eyes Height From Salary (min annual) 120 N 195Height To Preferred Job Type
60. 60. OHarmony Brown DBA 188 120,000 N 25Age From Age To 30 IQ (min) Married Smoker Eyes Height From Salary (min annual) 120 N 195Height To Preferred Job Type John Smith 9999 9999
61. 61. Index range scan a z bbbb rowid bbbb rowid aaaaa bbbbb ccccc ddddd eeeee aaaaa ggggg ccccc ddddd eeeee aaaaa kkkkk ccccc ddddd eeeee aaaaa bbbbb ccccc ddddd eeeee WHERE col1 = ' bbbbb '
62. 62. SELECT id, surname, firstname FROM men WHERE iq = 155;
63. 63. SELECT COUNT(surname) FROM men WHERE iq > 154; SELECT COUNT(surname) FROM men WHERE iq > 153; 9 rows 0.00008% 137,488 rows 1.25%
64. 64. OHarmony DEVELOPER 195 120,000 N 25Age From Age To 30 IQ (min) Married Smoker Eyes Height From Salary (min annual) N Height To Preferred Job Type
65. 65. Oharmony has provided 14 matches in 14 locations Poorly clustered
66. 66. Oharmony has provided 14 matches in 2 locations Well clustered
67. 67. SELECT i.index_name, i.distinct_keys, i.num_rows, i.clustering_factor, t.blocks FROM user_indexes i, user_tables t WHERE t.table_name = i.table_name AND t.table_name = 'MEN';
68. 68. CREATE TABLE men2 AS SELECT * FROM men ORDER BY IQ; CREATE INDEX MEN2_IQ_N4 ON men2(iq); begin dbms_stats.gather_table_stats(ownname=>'AUSOUG', tabname=>'MEN2', estimate_percent => null, cascade=>true, method_opt => 'for all columns size 1 for columns size 2000 iq ' ); end;
69. 69. SELECT i.distinct_keys, i.num_rows, i.clustering_factor, t.blocks FROM user_indexes i, user_tables t WHERE t.table_name = i.table_name AND i.index_name = 'MEN2_IQ_N4'; previously
70. 70. previously
71. 71. Optimizer Mode
72. 72. Optimizer Mode All_Rows First_Rows Tends towards:
73. 73. SELECT id, surname, firstname FROM men WHERE married = 'N' AND age_range = '25-29' AND height = 188 AND iq = 120 AND job = 'DBA'; Bitmap indexes
74. 74. With Bitmap indexes
75. 75. Sorting With B*Tree indexes SELECT id, surname, firstname FROM men WHERE married = 'N' AND age_range = '25-29‘ AND height = 188 AND iq = 120 AND job= 'DBA';
76. 76. access conditions number of rows? access method? access conditions number of rows? access method? access conditions number of rows? access method? Identify each join path How many rows am I likely to get? What is the best join method? All I have talked about so far is Access to one table – how does it JOIN them together ?
77. 77. access conditions number of rows? access method? access conditions number of rows? access method? access conditions number of rows? access method? 1 2 3
78. 78. access conditions number of rows? access method? access conditions number of rows? access method? access conditions number of rows? access method? 1 2 3
79. 79. access conditions number of rows? access method? access conditions number of rows? access method? access conditions number of rows? access method? 1 2 3 for each join path for each join – what is the best JOIN METHOD?
80. 80. Joins methods – Nested Loop with index 1 2 A.COL1 = B.COL2 A B COL1 = 1 B.COL2 index ACCESS ROWS WHERE COL2 = 1 COL2 = 1 COL2 = 1 COL2 = 1
81. 81. Joins methods – Hash join A B HASH TABLE1
82. 82. Joins methods – cartesian A B
83. 83. SELECT COUNT(*) FROM men WHERE married = 'N' AND age < 12 AND height >= 188 The wrong plan 0 rows
84. 84. 0 rows ALTER SESSION SET OPTIMIZER_INDEX_COST_ADJ = 5; SELECT COUNT(*) FROM men WHERE married = 'N' AND age < 12 AND height >= 188 This is even worse
85. 85. SELECT /*+ INDEX_COMBINE(MEN MEN_AGE_N1,MEN_HEIGHT_N1) */ COUNT(*) FROM men WHERE married = 'N' AND age < 12 AND height >= 188 0 rowsBetter
86. 86. SELECT COUNT(surname) FROM men WHERE height between 159.4 AND 160 26188 rows SELECT /*+ FULL(MEN) */ COUNT(surname) FROM men WHERE height between 159.4 AND 160
87. 87. 10,299,997 rowsSELECT COUNT(surname), COUNT( start_date) FROM men m, events e WHERE e.men_id (+) = m.id AND m.iq = 156 SELECT /*+ USE_HASH(M,E) */ COUNT(surname), COUNT( start_date) FROM men m, events e WHERE e.men_id (+) = m.id AND m.iq = 156
88. 88. www.sagecomputing.com.au penny@sagecomputing.com.au Questions? Penny Cookson SAGE Computing Services SAGE Computing Services Customised Oracle Training Workshops and Consulting