Indexing in Exadata

12,225 views

Published on

Richard Foote

Published in: Technology
0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
12,225
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
0
Comments
0
Likes
12
Embeds 0
No embeds

No notes for slide

Indexing in Exadata

  1. 1. Indexing In Exadata Richard Foote Oracle Corporation
  2. 2. Richard Foote• Working in IT for 25+ years (scary stuff)• Working with Oracle Database for 15+ years• Spent 19 years employed in Australian Federal Government in various IT roles, last 9 years as a senior DBA with the Australian Federal Police• Responsible for many large scale, mission critical, “life-dependant” classified Oracle systems• Previously a DBA Instructor with Oracle Corporation (1996 and 2002)• Since Nov 2011, re-joined Oracle Corporation as a Technical Solutions Consultant based in sunny Canberra• Spent much of this time playing with Exadata (one of the reasons I rejoined) • Oracle OakTable Member since 2002 and Oracle ACE Director since 2008 (now Oracle Employee Ace)• Richard Foote’s Oracle Blog: http://richardfoote.wordpress.com/• Spend as much free time as possible listening to the music of David Bowie !! 2 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  3. 3. Exadata Innovations Exadata Storage Server Software  Intelligent storage • Hybrid Columnar Compression – Smart Scan query offload – 10x compression for warehouses – Scale-out storage – 15x compression for archives Data + + + remains compressed Uncompressed for scans • Smart Flash Cache and in Flash – Accelerates random I/O up to 30x primary backup – Doubles data scan rate Benefits test Multiply standby dev’t Compressed3 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  4. 4. Things can run so much faster in Exadata …With Non-Exadata … SQL> select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200; 266176 rows selected. Elapsed: 00:04:43.38 ------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 266K| 23M| 2348K (1)| 07:49:45 | |* 1 | TABLE ACCESS FULL| DWH_BOWIE | 266K| 23M| 2348K (1)| 07:49:45 | ------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42) Statistics ---------------------------------------------------------- 1 recursive calls 0 db block gets 8644512 consistent gets 8625479 physical reads 0 redo size 12279634 bytes sent via SQL*Net to client 195719 bytes received via SQL*Net from client 17747 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 266176 rows processed4 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  5. 5. Things can run so much faster in Exadata …With Exadata, what previously took minutes can take seconds … SQL> select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200; 266176 rows selected. Elapsed: 00:00:03.97 --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 266K| 23M| 2348K (1)| 07:49:45 | |* 1 | TABLE ACCESS STORAGE FULL| DWH_BOWIE | 266K| 23M| 2348K (1)| 07:49:45 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42) filter("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 8626963 consistent gets 8625479 physical reads 0 redo size 12279634 bytes sent via SQL*Net to client 195719 bytes received via SQL*Net from client 17747 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 266176 rows processed 5 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  6. 6. Indexing In Exadata …Are indexes needed anymore ???6 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  7. 7. Worse - Indexes can get in the way … SQL> create bitmap index dwh_album_artist_i on dwh_bowie(artist_id, album_id); Index created. SQL> select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200; 266176 rows selected. Elapsed: 00:00:23.11 --------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 266K| 23M| 78267 (1)| 00:15:40 | | 1 | TABLE ACCESS BY INDEX ROWID | DWH_BOWIE | 266K| 23M| 78267 (1)| 00:15:40 | | 2 | BITMAP CONVERSION TO ROWIDS| | | | | | |* 3 | BITMAP INDEX RANGE SCAN | DWH_ALBUM_ARTIST_I | | | | | --------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 3 - access("ARTIST_ID">=42 AND "ALBUM_ID"=42 AND "ARTIST_ID"<=4200) filter("ARTIST_ID">=42 AND "ARTIST_ID"<=4200 AND "ALBUM_ID"=42) Statistics ---------------------------------------------------------- 1 recursive calls 0 db block gets 291129 consistent gets 25337 physical reads 29728 redo size 28836174 bytes sent via SQL*Net to client 195719 bytes received via SQL*Net from client 17747 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 266176 rows processed7 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  8. 8. Indexing In Exadata …Are indexes evil? Should they just be dropped?Mr. Index, or Dr. Evil !! 8 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  9. 9. What to do? Important Decision• Many suggest simply drop indexes once you move to Exadata• Indexes take up much storage (important consideration, especially with ¼ Exadata racks where storage is more limited)• A redundant index wastes space and resources to maintain• Indexes critical to performance (or they certainly used to be)• But can “get in the way” in Exadata• Once dropped, tricky to keep track of what indexes previously existed• As usual, the decision to drop or keep indexes not a straightforward one … 9 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  10. 10. Let’s Go Back To The BasicsCost of using an Index:Basic Cost = blevel + ceil(effective index selectivity x leaf_blocks) + ceil(effective table selectivity x clustering_factor) + a bit of CPU …In Exadata, this is basically the same for uncompressed tables.With EHCC tables, there’s some additional CPU costs requiredto decompress rows.Therefore actual cost of using an Index can increase in Exadata 10 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  11. 11. Let’s Go Back To The BasicsSimplistically, cost of Full Table Scan:Cost = (((No. of table blocks / effective multi block read count) x multi block read time) + a bit of CPU) / single block read timeThe costing is still calculated the same in Exadata, however thismay no longer represent the true costs… 11 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  12. 12. Exadata Intelligent Storage Grid  Exadata cells implement smart scans to greatly reduce the data that needs to be processed by database – Only return relevant rows and columns to database – Offload predicate evaluation  Data reduction can be very large – Column and row reduction often decrease data to be returned to the database by 10x12 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  13. 13. Exadata Smart Scans Full Scans (Tables/Indexes/MVs/Partitions) Direct Path Reads (By-Passing Buffer Cache) Smart Scan Optimizations, some of which include: – Column Projection – Predicate Filtering – Simple Joins – Function Off-Loading – Virtual Column Evaluation – HCC Decompression – Decryption See MOS Notes 793845.1 and 50415.1 for more info on Direct Path Reads 13 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  14. 14. Exadata Smart Scans - Benefits CPU utilization on database nodes improves due to offload Less data shipped to database nodes (generally …) Query fully encrypted database by moving decryption from database to storage cell hardware Full Table Scan performance can dramatically improve Costs associated with Full Table Scan operation can therefore significantly decrease, especially with Storage Indexes kicking in With index costs perhaps increasing and FTS costs perhaps decreasing, indexes in Exadata become less viable … Not looking good for indexes !!! 14 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  15. 15. Non-Exadata Chart: Indexes vs. FTSCost Rows Returned Full Table Scan Index Scan 15 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  16. 16. Non-Exadata Chart: Indexes vs. FTS (DWH) xx x x xx xx x xx xx xx xxCost x x xx x x x x x x x xx Rows Returned Full Table Scan x Queries using indexes Index Scan 16 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  17. 17. Non-Exadata Chart: Indexes vs. FTS (OLTP) x xCost x x x x x xx xx x x x x xx x Rows Returned Full Table Scan Index Scan x Queries using indexes 17 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  18. 18. Exadata Chart: Indexes vs. FTSCost Rows Returned Full Table Scan Index Scan (on HCC tables) 18 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  19. 19. Exadata Chart: Indexes vs. FTS (DWH) x xx x x x xx xxxx x x xx x xCost x xx x x x x x x x xx Rows Returned Full Table Scan Index Scan x Queries using indexes 19 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  20. 20. Exadata Chart: Indexes vs. FTS (OLTP) x xCost x x x x x xxx xx x xx x x xx Rows Returned Full Table Scan Index Scan x Queries using indexes 20 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  21. 21. Exadata Indexes vs. FTS (CBO Costs) x xx xx xx x x xx xx xx xxCost x x xx x x x x x x x xx Rows Returned Full Table Scan Index Scan x Queries using indexes CBO Costs 21 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  22. 22. Exadata Indexes vs. FTS (CBO Costs) x x x xx x x x xxxx x x x x xCost x x xxx x x x x x x x xx Full Table Scan Rows Returned Index Scan x Queries using indexes FTS CBO Cost 22 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  23. 23. Without Exadata Smart ScansGoing back to the first example … SQL> select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200; 266176 rows selected. Elapsed: 00:04:43.38 ------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 266K| 23M| 2348K (1)| 07:49:45 | |* 1 | TABLE ACCESS FULL| DWH_BOWIE | 266K| 23M| 2348K (1)| 07:49:45 | ------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42) Statistics ---------------------------------------------------------- 1 recursive calls 0 db block gets 8644512 consistent gets 8625479 physical reads 0 redo size 12279634 bytes sent via SQL*Net to client 195719 bytes received via SQL*Net from client 17747 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 266176 rows processed23 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  24. 24. Smart Scan Example SQL> select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200; 266176 rows selected. Elapsed: 00:00:03.97 --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 266K| 23M| 2348K (1)| 07:49:45 | |* 1 | TABLE ACCESS STORAGE FULL| DWH_BOWIE | 266K| 23M| 2348K (1)| 07:49:45 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42) filter("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 8626963 consistent gets 8625479 physical reads 0 redo size 12279634 bytes sent via SQL*Net to client 195719 bytes received via SQL*Net from client 17747 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 266176 rows processed24 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  25. 25. Smart Scan – How can you tell ? SQL> select sql_text, io_cell_offload_eligible_bytes, io_interconnect_bytes, io_cell_uncompressed_bytes, io_cell_offload_returned_bytes from v$sql where sql_id = admdhphbh1a1c; SQL_TEXT -------------------------------------------------------------------------------- IO_CELL_OFFLOAD_ELIGIBLE_BYTES IO_INTERCONNECT_BYTES IO_CELL_UNCOMPRESSED_BYTES ------------------------------ --------------------- -------------------------- IO_CELL_OFFLOAD_RETURNED_BYTES ------------------------------ select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200 1.4132E+11 55217792 1.6718E+10 55217792 SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 59414.9453 cell physical IO interconnect bytes returned by smart scan 26.32989525 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  26. 26. Non-Smart Scan – CBO Costs The Same SQL> alter session set cell_offload_processing=false; Session altered. SQL> select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200; 266176 rows selected. Elapsed: 00:03:03.38 --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 266K| 23M| 2348K (1)| 07:49:45 | |* 1 | TABLE ACCESS STORAGE FULL| DWH_BOWIE | 266K| 23M| 2348K (1)| 07:49:45 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter("ALBUM_ID"=42 AND "ARTIST_ID"<=4200 AND "ARTIST_ID">=42) Statistics ---------------------------------------------------------- 1 recursive calls 0 db block gets 8644512 consistent gets 8625479 physical reads 0 redo size 12279634 bytes sent via SQL*Net to client 195719 bytes received via SQL*Net from client 17747 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 266176 rows processedCBO is not aware of smart scan benefits and FTS costs remain the same … 26 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  27. 27. Less Efficient Index Chosen By CBO SQL> select * from dwh_bowie where album_id = 42 and artist_id between 42 and 4200; 266176 rows selected. Elapsed: 00:00:23.11 --------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 266K| 23M| 78267 (1)| 00:15:40 | | 1 | TABLE ACCESS BY INDEX ROWID | DWH_BOWIE | 266K| 23M| 78267 (1)| 00:15:40 | | 2 | BITMAP CONVERSION TO ROWIDS| | | | | | |* 3 | BITMAP INDEX RANGE SCAN | DWH_ALBUM_ARTIST_I | | | | | --------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 3 - access("ARTIST_ID">=42 AND "ALBUM_ID"=42 AND "ARTIST_ID"<=4200) filter("ARTIST_ID">=42 AND "ARTIST_ID"<=4200 AND "ALBUM_ID"=42) Statistics ---------------------------------------------------------- 1 recursive calls 0 db block gets 291129 consistent gets 25337 physical reads 29728 redo size 28836174 bytes sent via SQL*Net to client 195719 bytes received via SQL*Net from client 17747 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 266176 rows processedTherefore existing indexes although maybe slower, appear much cheaper… 27 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  28. 28. CBO and Smart Scans The decision to go for Direct Reads and use a Smart Scan is not made by the CBO The CBO is not aware of potential efficiencies provided by Smart Scans and costs Full Scans similar to Non-Exadata Bug 10248538: OPTIMIZER COST MODEL NOT EXADATA-AWARE Therefore, Full Scans are generally over-costed in Exadata, favoring Index Scan related execution paths Simply dropping all indexes however is not the answer (to be discussed …) Consider setting MBRC system stat to 128, assuming 8K block size to compensate (1MB default storage region size in storage cell) Consider v$sql_plan, dba_hist_sql_plan searching for index plans with high cardinality values Storage Indexes can make cost differentials even worse … 28 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  29. 29. Storage Index – Basic ExampleTable Index • Exadata Storage Indexes maintain summary information about table data in memoryA B C D – Store MIN and MAX values of columns 1 – Typically index entry for every MB of disk 3 Min B = 1 • Eliminates disk I/Os if MIN and MAX can never Max B = 5 match “where” clause of a query 5 5 • Completely automatic and transparent 8 Min B = 3 • Example: Select count(*) from table where B = 1; Max B =8 3 • FTS no longer really a FTS … 29 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  30. 30. Storage Indexes – Demo SetupSQL> create table big_bowie (id number not null, album_id number not null, artist_id number not null, format_idnumber, release_date date, total_sales number, description varchar2(100));Table created.SQL> create sequence bowie_seq order;Sequence created.SQL> create or replace procedure pop_big_bowie as 2 begin 3 for v_album_id in 1..100 loop 4 for v_artist_id in 1..100000 loop 5 insert into big_bowie values (bowie_seq.nextval, v_album_id, v_artist_id,ceil(dbms_random.value(0,5)) * 2, 6 trunc(sysdate-ceil(dbms_random.value(0,10000))), ceil(dbms_random.value(0,500000)),THE RISE AND FALL OF ZIGGY STARDUST AND THE SPIDERS FROM MARS); 7 end loop; 8 commit; 9 end loop; 10 commit; 11 end; 12 /Procedure created.SQL> exec pop_big_bowiePL/SQL procedure successfully completed. 30 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  31. 31. Storage Indexes – Demo Setup SQL> update big_bowie set format_id = 3 where mod(id,10000)=0; 1000 rows updated. SQL> commit; Commit complete. SQL> update big_bowie set format_id = 5 where id between 424242 and 425241; 1000 rows updated. SQL> commit; Commit complete. SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>BIG_BOWIE‘, method_opt=>FOR ALL COLUMNS SIZE 1); PL/SQL procedure successfully completed. SQL> select blocks from dba_tables where table_name = BIG_BOWIE; BLOCKS ---------- 134809 This equates to approx. 1054 MB (a “typical” OLTP table) The DWH_BOWIE table is simply 64 times the BIG_BOWIE table31 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  32. 32. Storage Indexes – “Clustering Factor”ALBUM_ID column data is well clustered … SQL> select * from big_bowie where album_id = 42; 100000 rows selected. Elapsed: 00:00:00.27 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 100K| 8984K| 36663 (1)| 00:07:20 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 100K| 8984K| 36663 (1)| 00:07:20 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ALBUM_ID"=42) filter("ALBUM_ID"=42) Statistics ---------------------------------------------------------- 1 recursive calls 0 db block gets 134834 consistent gets 134809 physical reads 0 redo size 4345496 bytes sent via SQL*Net to client 73850 bytes received via SQL*Net from client 6668 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 100000 rows processed32 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  33. 33. Storage Indexes – “Clustering Factor”SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# andn.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved bystorage index);NAME MB---------------------------------------------------------------- ----------cell physical IO bytes saved by storage index 1042.24219cell physical IO interconnect bytes returned by smart scan 9.56161499The Storage Index has managed to avoid having to readapprox. 99% of the table.Having data well clustered can result in Partition-Pruninglike efficiencies when using an associated Storage Index … 33 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  34. 34. Storage Indexes – “Clustering Factor”Note: Conventional database indexes love well clustered data as well … SQL> select * from big_bowie where album_id = 42; 100000 rows selected. Elapsed: 00:00:01.07 Execution Plan ---------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ---------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 100K| 8984K| 1550 (1)| 00:00:19 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 100K| 8984K| 1550 (1)| 00:00:19 | |* 2 | INDEX RANGE SCAN | BIG_BOWIE_ALBUM_ID_I | 100K| | 199 (1)| 00:00:03 | ---------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("ALBUM_ID"=42) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 1590 consistent gets 1550 physical reads 0 redo size 9689267 bytes sent via SQL*Net to client 733 bytes received via SQL*Net from client 21 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 100000 rows processedBut is taking twice as long as FTS, despite “reported” physical reads much reduced… 34 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  35. 35. Storage Indexes – “Clustering Factor”Even when all the data is cached … SQL> select * from big_bowie where album_id = 42; 100000 rows selected. Elapsed: 00:00:00.27 Execution Plan ---------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ---------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 100K| 8984K| 1550 (1)| 00:00:19 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 100K| 8984K| 1550 (1)| 00:00:19 | |* 2 | INDEX RANGE SCAN | BIG_BOWIE_ALBUM_ID_I | 100K| | 199 (1)| 00:00:03 | ---------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("ALBUM_ID"=42) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 1590 consistent gets 0 physical reads 0 redo size 9689267 bytes sent via SQL*Net to client 733 bytes received via SQL*Net from client 21 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 100000 rows processedIndex is no more efficient than FTS, even though the index is extremely efficient … 35 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  36. 36. Storage Indexes – “Clustering Factor”The TOTAL_SALES column however is not well clustered … SQL> select album_id, artist_id from big_bowie where total_sales between 42 and 142; 2009 rows selected. Elapsed: 00:00:01.25 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2040 | 26520 | 36700 (1)| 00:07:21 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 2040 | 26520 | 36700 (1)| 00:07:21 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("TOTAL_SALES"<=142 AND "TOTAL_SALES">=42) filter("TOTAL_SALES"<=142 AND "TOTAL_SALES">=42) Statistics ---------------------------------------------------------- 1 recursive calls 0 db block gets 134834 consistent gets 134809 physical reads 0 redo size 47506 bytes sent via SQL*Net to client 1987 bytes received via SQL*Net from client 135 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 2009 rows processed 36 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  37. 37. Storage Indexes – “Clustering Factor”SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# andn.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved bystorage index);NAME MB---------------------------------------------------------------- ----------cell physical IO bytes saved by storage index 72.65625cell physical IO interconnect bytes returned by smart scan .383415222The Storage Index has only managed to avoid having to read approx. 7%of the table.The clustering of the data can make a huge impact to the effectiveness ofStorage Indexes.Even though this query is only reading a fraction of the data as theprevious query, it’s taking twice as long.The smart scan though is only returning a small fraction of the overalldata back to the database. 37 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  38. 38. Storage Indexes – “Clustering Factor”Note: Conventional indexes also hate poorly clustered tables … SQL> select album_id, artist_id from big_bowie where total_sales between 42 and 142; 2009 rows selected. Elapsed: 00:00:01.45 Execution Plan ------------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes |Cost (%CPU)| Time | ------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2040 | 26520 | 2048 (1)| 00:00:25 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 2040 | 26520 | 2048 (1)| 00:00:25 | |* 2 | INDEX RANGE SCAN | BIG_BOWIE_TOTAL_SALES_I | 2040 | | 7 (0)| 00:00:01 | ------------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("TOTAL_SALES">=42 AND "TOTAL_SALES"<=142) Statistics ---------------------------------------------------------- 1 recursive calls 0 db block gets 2150 consistent gets 2005 physical reads 0 redo size 43311 bytes sent via SQL*Net to client 1987 bytes received via SQL*Net from client 135 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 2009 rows processedThis index is taking longer than the equivalent Exadata FTS … 38 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  39. 39. Storage Indexes – “Clustering Factor”But on subsequent re-runs, when most of the data has been cached … SQL> select album_id, artist_id from big_bowie where total_sales between 42 and 142; 2009 rows selected. Elapsed: 00:00:00.02 Execution Plan ------------------------------------------------------------------------------------------------------ | Id | Operation | Name | Rows | Bytes |Cost (%CPU)| Time | ------------------------------------------------------------------------------------------------------ | 0 | SELECT STATEMENT | | 2040 | 26520 | 2048 (1)| 00:00:25 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 2040 | 26520 | 2048 (1)| 00:00:25 | |* 2 | INDEX RANGE SCAN | BIG_BOWIE_TOTAL_SALES_I | 2040 | | 7 (0)| 00:00:01 | ------------------------------------------------------------------------------------------------------ Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("TOTAL_SALES">=42 AND "TOTAL_SALES"<=142) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 2150 consistent gets 0 physical reads 0 redo size 43308 bytes sent via SQL*Net to client 1987 bytes received via SQL*Net from client 135 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 2009 rows processedThe index is now significantly faster than the FTS which performs consistently … 39 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  40. 40. Database vs. Storage IndexesHave much in common:• Both can eliminate reading unnecessary data within a table• Efficiencies of both are directly impacted by the clustering of the data within the table• Both may need a period of “warming” up before optimal use• Both can combine multiple indexes to further increase efficiencies40 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  41. 41. Storage Indexes – Warm upOn the first execution of this query since table creation: SQL> select artist_id, total_sales from big_bowie where album_id = 42; 100000 rows selected. Elapsed: 00:00:00.89 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 100K| 1269K| 36663 (1)| 00:07:20 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 100K| 1269K| 36663 (1)| 00:07:20 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ALBUM_ID"=42) filter("ALBUM_ID"=42) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 114.601563 cell physical IO interconnect bytes returned by smart scan 1.77637482 41 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  42. 42. Storage Indexes – Warm upSubsequent executions, many more IO bytes saved by “warm” storage index SQL> select artist_id, total_sales from big_bowie where album_id = 42; 100000 rows selected. Elapsed: 00:00:00.27 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 100K| 1269K| 36663 (1)| 00:07:20 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 100K| 1269K| 36663 (1)| 00:07:20 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ALBUM_ID"=42) filter("ALBUM_ID"=42) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 1156.84375 cell physical IO interconnect bytes returned by smart scan 3.41764832 42 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  43. 43. Database Indexes – Cache Sensitive SQL> select album_id, artist_id from big_bowie where total_sales between 42 and 142; 2009 rows selected. Elapsed: 00:00:01.45 Execution Plan ------------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes |Cost (%CPU)| Time | ------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2040 | 26520 | 2048 (1)| 00:00:25 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 2040 | 26520 | 2048 (1)| 00:00:25 | |* 2 | INDEX RANGE SCAN | BIG_BOWIE_TOTAL_SALES_I | 2040 | | 7 (0)| 00:00:01 | ------------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("TOTAL_SALES">=42 AND "TOTAL_SALES"<=142) Statistics ---------------------------------------------------------- 1 recursive calls 0 db block gets 2150 consistent gets 2005 physical reads 0 redo size 43311 bytes sent via SQL*Net to client 1987 bytes received via SQL*Net from client 135 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 2009 rows processed43 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  44. 44. Database Indexes – Cache SensitiveBut on subsequent re-runs, when most of the data has been cached … SQL> select album_id, artist_id from big_bowie where total_sales between 42 and 142; 2009 rows selected. Elapsed: 00:00:00.02 Execution Plan ------------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes |Cost (%CPU)| Time | ------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 2040 | 26520 | 2048 (1)| 00:00:25 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 2040 | 26520 | 2048 (1)| 00:00:25 | |* 2 | INDEX RANGE SCAN | BIG_BOWIE_TOTAL_SALES_I | 2040 | | 7 (0)| 00:00:01 | ------------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("TOTAL_SALES">=42 AND "TOTAL_SALES"<=142) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 2150 consistent gets 0 physical reads 0 redo size 43308 bytes sent via SQL*Net to client 1987 bytes received via SQL*Net from client 135 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 2009 rows processedThe index is now significantly faster than previously 44 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  45. 45. Exadata Smart Flash Cache  Improves Database Index performance  Random reads against tables and indexes are likely to have subsequent reads and normally will be cached and have their data subsequently delivered from the flash cache  10x -100x better performance than disk  Both table and index blocks cached45 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  46. 46. Storage Indexes – CombineFirst, we run a query using a storage index on the ALBUM_ID column SQL> select * from big_bowie where album_id <= 10; 1000002 rows selected. Elapsed: 00:00:06.41 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1009K| 88M| 36667 (1)| 00:07:21 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 1009K| 88M| 36667 (1)| 00:07:21 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ALBUM_ID"<=10) filter("ALBUM_ID"<=10) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 947.367188 cell physical IO interconnect bytes returned by smart scan 94.2925186 46 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  47. 47. Storage Indexes – CombineNext, we run a query using a storage index on the ARTIST_ID column SQL> select * from big_bowie where artist_id <= 10000; 1000002 rows selected. Elapsed: 00:00:06.85 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1000K| 87M| 36676 (1)| 00:07:21 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 1000K| 87M| 36676 (1)| 00:07:21 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ARTIST_ID"<=10000) filter("ARTIST_ID"<=10000) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 850.140625 cell physical IO interconnect bytes returned by smart scan 94.4599609 47 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  48. 48. Storage Indexes – CombineFinally, we run a query using storage indexes on both columns SQL> select * from big_bowie where album_id <= 10 and artist_id <= 10000; 100002 rows selected. Elapsed: 00:00:00.52 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 100K| 9066K| 36675 (1)| 00:07:21 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 100K| 9066K| 36675 (1)| 00:07:21 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ARTIST_ID"<=10000 AND "ALBUM_ID"<=10) filter("ARTIST_ID"<=10000 AND "ALBUM_ID"<=10) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 1031.4375 cell physical IO interconnect bytes returned by smart scan 9.31826019IO bytes saved by storage indexes is now greater than previous examples 48 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  49. 49. Storage Index Advantages  Are created automatically and transparently  Used automatically if present during Smart Scans  Do not take up any physical storage (memory only)  Have negligible impact on DML statements  Effective in processing IS NULL predicates  Might make corresponding Database indexes redundant  As they’re used during Smart Scans, when used can return just the rows/columns of interest to the database49 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  50. 50. Storage Index Disadvantages Limited to a max of 8 columns per table May not exist on a particular column at a point in time Not as “focused” (1MB storage region default vs. 1 data block) Has limitations, can only use Min/Max as referenced data Database indexes determines specific location of data while a SI can only determine where data can’t exist Therefore SI can still access unnecessarily storage of no interest Can not cache data in database buffer cache for general re-usability The CBO is not aware of Storage Indexes savings Not as general purpose as database indexes (can’t police constraints, avoid sorts, provide CBO statistics, etc.)50 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  51. 51. Storage Indexes: 8 Column LimitSQL> create table radiohead (id number, col1 number, col2 number, col3 number, col4 number, col5 number, col6 number, col7number, col8 number, col9 number, col10 number, col11 number, col12 number, some_text varchar2(50));Table created.SQL> insert into radiohead select rownum, mod(rownum,10), mod(rownum,100), mod(rownum,1000), mod(rownum,10000),mod(rownum,100000), mod(rownum,1000000), ceil(dbms_random.value(0,10)), ceil(dbms_random.value(0,100)),ceil(dbms_random.value(0,1000)), ceil(dbms_random.value(0,10000)), ceil(dbms_random.value(0,100000)),ceil(dbms_random.value(0,1000000)), OK COMPUTER from dual connect by level <=2000000;2000000 rows created.SQL> commit;Commit complete.SQL> insert/*+ append */ into radiohead select * from radiohead;2000000 rows created.SQL> commit;Commit complete.SQL> insert/*+ append */ into radiohead select * from radiohead;4000000 rows created.SQL> commit;Commit complete.SQL> insert/*+ append */ into radiohead select * from radiohead;8000000 rows created.SQL> commit;Commit complete.SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>RADIOHEAD, estimate_percent=>null, method_opt=> FOR ALLCOLUMNS SIZE 1); 51 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  52. 52. Storage Indexes: 8 Column Limit SQL> select * from radiohead where id = 42; 8 rows selected. Elapsed: 00:00:00.05 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 8 | 416 | 42425 (1)| 00:08:30 | |* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 8 | 416 | 42425 (1)| 00:08:30 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ID"=121212) filter("ID"=121212) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 1333.99219 cell physical IO interconnect bytes returned by smart scan .164878845Storage index created on the ID column …52 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  53. 53. Storage Indexes: 8 Column LimitRepeat with predicates based on each of the table columns SQL> select * from radiohead where col1=42; no rows selected Elapsed: 00:00:00.01 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 52 | 42440 (1)| 00:08:30 | |* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 1 | 52 | 42440 (1)| 00:08:30 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("COL1"=42) filter("COL1"=42) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 2546.90625 cell physical IO interconnect bytes returned by smart scan .341438293The storage index on the COL1 column is used as well (can be useful in situationswhen the required value doesn’t exist)… 53 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  54. 54. Storage Indexes: 8 Column LimitA Storage index is also used on COL4 although because of the distribution of data,the saved I/Os is limited SQL> select * from radiohead where col4=42; 1600 rows selected. Elapsed: 00:00:00.68 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1600 | 83200 | 42486 (1)| 00:08:30 | |* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 1600 | 83200 | 42486 (1)| 00:08:30 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("COL4"=42) filter("COL4"=42) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 2612.64063 <= + 66MB cell physical IO interconnect bytes returned by smart scan 32.3048401 54 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  55. 55. Storage Indexes: 8 Column LimitA Storage index is also used on, COL5, COL6, COL7, Col8 and COL9 making atotal of 8 storage indexes defined on this table SQL> select * from radiohead where col9=0; no rows selected Elapsed: 00:00:00.02 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 15984 | 811K| 42561 (1)| 00:08:31 | |* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 15984 | 811K| 42561 (1)| 00:08:31 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("COL9"=0) filter("COL9"=0) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 8792.94531 cell physical IO interconnect bytes returned by smart scan 45.3872757 55 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  56. 56. Storage Indexes: 8 Column LimitHowever although a smart scan is performed, a viable storage index is not nowcreated on COL10 column with 8 existing storage indexes SQL> select * from radiohead where col10=42; 1536 rows selected. Elapsed: 00:00:00.73 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1600 | 83200 | 42577 (1)| 00:08:31 | |* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 1600 | 83200 | 42577 (1)| 00:08:31 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("COL10"=42) filter("COL10"=42) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 8792.94531  No change cell physical IO interconnect bytes returned by smart scan 45.9288864 56 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  57. 57. Storage Indexes: 8 Column LimitOn this much more selective query on COL12, a new storage index is created SQL> select * from radiohead where col12=42; 8 rows selected. Elapsed: 00:00:00.39 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 19 | 988 | 42607 (1)| 00:08:32 | |* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 19 | 988 | 42607 (1)| 00:08:32 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("COL12"=42) filter("COL12"=42) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 9526.01563 cell physical IO interconnect bytes returned by smart scan 53.5218124 57 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  58. 58. Storage Indexes: 8 Column LimitIf we go back to the first query on the ID column, the storage index is still used … SQL> select * from radiohead where id=42; 8 rows selected. Elapsed: 00:00:00.05 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 8 | 416 | 42425 (1)| 00:08:30 | |* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 8 | 416 | 42425 (1)| 00:08:30 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ID"=42) filter("ID"=42) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 10726.9844 cell physical IO interconnect bytes returned by smart scan 53.5264816 58 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  59. 59. Storage Indexes: 8 Column LimitBut if we go back to the not so selective query on the COL4 column, the storageindex is no longer used as it was previously … SQL> select * from radiohead where col4 = 4242; 1600 rows selected. Elapsed: 00:00:00.73 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1600 | 83200 | 42486 (1)| 00:08:30 | |* 1 | TABLE ACCESS STORAGE FULL| RADIOHEAD | 1600 | 83200 | 42486 (1)| 00:08:30 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("COL4"=4242) filter("COL4"=4242) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 10726.9844  No Change !! cell physical IO interconnect bytes returned by smart scan 54.1522598The Storage Index on COL4 is replaced by the Storage Index on COL12 … 59 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  60. 60. “Hidden” Values of InterestReturning to the BIG_BOWIE table, the FORMAT_ID column has values 2 – 10, butonly a handful with a value of 3 and 5 … SQL> update big_bowie set format_id = 3 where mod(id,10000)=0; 1000 rows updated. SQL> commit; Commit complete. SQL> update big_bowie set format_id = 5 where id between 424242 and 425241; 1000 rows updated. SQL> commit; Commit complete. SQL> create index big_bowie_format_id_i on big_bowie(format_id); Index created. SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>BIG_BOWIE, method_opt=> FOR COLUMNS FORMAT_ID SIZE 10); PL/SQL procedure successfully completed. 60 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  61. 61. “Hidden” Values of InterestWith a histogram in place, the CBO uses an index very effectively … SQL> select album_id, total_sales from big_bowie where format_id = 3; 1000 rows selected. Elapsed: 00:00:00.01 Execution Plan ----------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ----------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1000 | 11000 | 72 (0)| 00:00:01 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 1000 | 11000 | 72 (0)| 00:00:01 | |* 2 | INDEX RANGE SCAN | BIG_BOWIE_FORMAT_ID_I | 1000 | | 4 (0)| 00:00:01 | ----------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("FORMAT_ID"=3) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 1072 consistent gets 0 physical reads 0 redo size 22984 bytes sent via SQL*Net to client 1250 bytes received via SQL*Net from client 68 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 1000 rows processed 61 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  62. 62. “Hidden” Values of InterestWhereas value 3 was littered throughout the table, values 5 are all clusteredtogether, making the index even more effective … SQL> select album_id, total_sales from big_bowie where format_id = 5; 1000 rows selected. Elapsed: 00:00:00.00 Execution Plan ----------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ----------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1000 | 11000 | 72 (0)| 00:00:01 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 1000 | 11000 | 72 (0)| 00:00:01 | |* 2 | INDEX RANGE SCAN | BIG_BOWIE_FORMAT_ID_I | 1000 | | 4 (0)| 00:00:01 | ----------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("FORMAT_ID"=5) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 154 consistent gets 0 physical reads 0 redo size 22685 bytes sent via SQL*Net to client 1250 bytes received via SQL*Net from client 68 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 1000 rows processed 62 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  63. 63. “Hidden” Values of InterestSmart Scan performed but a storage index is totally ineffective as the required valuesare “hidden” between the min/max (2 – 10) range found in most 1MB storage regions SQL> alter index big_bowie_format_id_i invisible; Index altered. SQL> select album_id, total_sales from big_bowie where format_id = 3; 1000 rows selected. Elapsed: 00:00:00.80 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1000 | 11000 | 36682 (1)| 00:07:21 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 1000 | 11000 | 36682 (1)| 00:07:21 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("FORMAT_ID"=3) filter("FORMAT_ID"=3) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 0 cell physical IO interconnect bytes returned by smart scan .397476196 63 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  64. 64. “Hidden” Values of InterestIn both cases, Storage Index is not used with Database index considerably faster … SQL> select album_id, total_sales from big_bowie where format_id = 5; 1000 rows selected. Elapsed: 00:00:00.89 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1000 | 11000 | 36682 (1)| 00:07:21 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 1000 | 11000 | 36682 (1)| 00:07:21 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("FORMAT_ID"=5) filter("FORMAT_ID"=5) SQL> select name , value/1024/1024 MB from v$statname n, v$mystat s where n.statistic# = s.statistic# and n.name in (cell physical IO interconnect bytes returned by smart scan, cell physical IO bytes saved by storage index); NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 0 cell physical IO interconnect bytes returned by smart scan .565643311 64 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  65. 65. OLTP Environments In OLTP environments, most queries and processes are based on highly selective data where database indexes are more viable Indexes used for more than just “selecting” data, for example: – Police Primary and Unique Key constraints – Reduce locking implications with Foreign Keys – Reduce sorting requirements – Reduce function-based processing – Provide additional selectivity data to the CBO 65 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  66. 66. Storage Indexes - Unique ScanWith a storage index in place, a FTS is no longer really a FTS … SQL> select album_id, artist_id from big_bowie where id = 42; Elapsed: 00:00:00.02 Execution Plan --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 14 | 36653 (1)| 00:07:20 | |* 1 | TABLE ACCESS STORAGE FULL| BIG_BOWIE | 1 | 14 | 36653 (1)| 00:07:20 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - storage("ID"=42) filter("ID"=42) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 134834 consistent gets 134809 physical reads 0 redo size 600 bytes sent via SQL*Net to client 524 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 1 rows processed NAME MB ---------------------------------------------------------------- ---------- cell physical IO bytes saved by storage index 1052.22656 cell physical IO interconnect bytes returned by smart scan .000419617 66 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  67. 67. Unique ScanHowever, an index is even faster and potentially far more scalable … SQL> select album_id, artist_id from big_bowie where id = 42; Elapsed: 00:00:00.00 Execution Plan -------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | -------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 14 | 3 (0)| 00:00:01 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 1 | 14 | 3 (0)| 00:00:01 | |* 2 | INDEX UNIQUE SCAN | BIG_BOWIE_PK | 1 | | 2 (0)| 00:00:01 | -------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("ID"=42) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 4 consistent gets 0 physical reads 0 redo size 600 bytes sent via SQL*Net to client 524 bytes received via SQL*Net from client 2 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 1 rows processed 67 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  68. 68. Primary and Unique KeysThen there’s the issue of removing an index that enforces either a Primary or UniqueKey constraint …SQL> drop index big_bowie_pk;drop index big_bowie_pk *ERROR at line 1:ORA-02429: cannot drop index used for enforcement of unique/primary keySQL> alter table big_bowie drop primary key;Table alteredA Storage Index can not be used to enforce constraints. 68 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  69. 69. Foreign Key ConstraintsIndexes can be critical in preventing locking issues with regard to Foreign KeyConstraints…SQL> create table albums (id number, name varchar2(50));Table created.SQL> insert into albums select rownum, table_name from dba_tables where rownum <=101;101 rows created.SQL> commit;Commit complete.SQL> alter table albums add constraint albums_pk primary key (id);Table altered.SQL> exec dbms_stats.gather_table_stats(ownname=>user, tabname=>ALBUMS, estimate_percent=>null,method_opt=> FOR ALL COLUMNS SIZE 1);PL/SQL procedure successfully completed.SQL> alter table big_bowie add constraint albums_fk foreign key (album_id) referencing albums(id);Table altered. 69 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  70. 70. Foreign Key Constraints In Session 1 (no commit)SQL> insert into big_bowie values (10000001, 1, 1, 1, sysdate, 1, TEST);1 row created. In Session 2 (hangs !!)SQL> delete albums where id = 101; In Session 3 (hangs !!)SQL> insert into big_bowie values (10000002, 1,1,1,sysdate,1,TEST2);A Storage Index can not prevent Foreign Key locking related issues … 70 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  71. 71. Foreign Key ConstraintsIn Session 1 (no commit)SQL> create index big_bowie_album_id_i on big_bowie(album_id);Index created.SQL> insert into big_bowie values (10000001, 1, 1, 1, sysdate, 1, TEST);1 row created.In Session 2 (no longer hangs)SQL> delete albums where id = 101;1 row deleted.In Session 3 (no longer hangs)SQL> insert into big_bowie values (10000002, 1,1,1,sysdate,1,TEST2);1 row created. 71 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.
  72. 72. Database Indexes and SortsDatabase indexes return data in a sorted manner and so can avoid expensive sortoperations … SQL> select * from big_bowie where album_id between 1 and 50 order by album_id; 5000002 rows selected. Elapsed: 00:00:13.08 Execution Plan ---------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ---------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 5049K| 443M| 79536 (1)| 00:15:55 | | 1 | TABLE ACCESS BY INDEX ROWID| BIG_BOWIE | 5049K| 443M| 79536 (1)| 00:15:55 | |* 2 | INDEX RANGE SCAN | BIG_BOWIE_ALBUM_ID_I | 5049K| | 9916 (1)| 00:01:59 | ---------------------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 2 - access("ALBUM_ID">=1 AND "ALBUM_ID"<=50) Statistics ---------------------------------------------------------- 0 recursive calls 0 db block gets 80506 consistent gets 0 physical reads 0 redo size 483419167 bytes sent via SQL*Net to client 11524 bytes received via SQL*Net from client 1002 SQL*Net roundtrips to/from client 0 sorts (memory) 0 sorts (disk) 5000002 rows processed 72 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

×