This document discusses an issue where the same SQL statement is using two different execution plans on the PRODUCTION database. The SQL is performing an update on the TBL_XXX table. One plan uses a full table scan while the other uses an index range scan on the primary key index TBL_XXX_PK. Statistics show the primary key index has a very high clustering factor, indicating data is scattered across blocks, likely due to row migration from updates. A test case confirms around 200 rows span multiple blocks. Adjusting an optimizer parameter reduces the cost of the index plan, but a better solution is needed to address the underlying data distribution issue causing row migration.
Modern query optimisation features in MySQL 8.Mydbops
MySQL 8 (a huge leap forward), indexing capabilities, execution plan enhancements, optimizer improvements, and many other current query tweak features are covered in the slides.
Indexing Strategies for Oracle Databases - Beyond the Create Index StatementSean Scott
B-tree indexes are the most common type of index and order data within the index in branches and leaves. Composite indexes consist of more than one column to improve performance. When choosing indexes, consider columns frequently used in queries, primary keys, and foreign keys. Index maintenance includes rebuilding, coalescing, and shrinking indexes.
The document discusses adaptive query optimization in Oracle 12c. It begins by describing drawbacks of the optimizer in pre-12c versions, such as insufficient statistics triggering dynamic sampling. It then outlines the key features of adaptive query optimization in 12c, including adaptive/dynamic plans using techniques like adaptive parallel distribution and adaptive joins. It also discusses automatic re-optimization using feedback from initial executions. The document provides illustrations of these techniques using example queries and optimizer statistics.
About Flexible Indexing
Postgres’ rich variety of data structures and data-type specific indexes can be confusing for newer and experienced Postgres users alike who may be unsure when and how to use them. For example, gin indexing specializes in the rapid lookup of keys with many duplicates — an area where traditional btree indexes perform poorly. This is particularly useful for json and full text searching. GiST allows for efficient indexing of two-dimensional values and range types.
To listen to the recorded presentation with Bruce Momjian, visit Enterprisedb.com > Resources > Webcasts > Ondemand Webcasts.
For product information and subscriptions, please email sales@enterprisedb.com.
[PGDay.Seoul 2020] PostgreSQL 13 New Featureshyeongchae lee
This document discusses the new features in PostgreSQL version 13, including:
- Improved performance from de-duplicating B-tree index entries and parallel vacuuming of indexes.
- Better query planning for queries using aggregates, partitions, and extended statistics.
- Changes required for migrating to version 13, including incompatibilities.
The document summarizes how SQL Plan Directives in Oracle 12c can help address issues caused by cardinality misestimation in the optimizer. It provides an example where the optimizer underestimates the number of rows returned by a query on a table due to not having statistics on correlated columns. In 12c, a SQL Plan Directive is automatically generated after the first execution to capture this misestimation. On subsequent queries, the directive can be used to provide more accurate cardinality estimates through automatic reoptimization or dynamic sampling.
Modern query optimisation features in MySQL 8.Mydbops
MySQL 8 (a huge leap forward), indexing capabilities, execution plan enhancements, optimizer improvements, and many other current query tweak features are covered in the slides.
Indexing Strategies for Oracle Databases - Beyond the Create Index StatementSean Scott
B-tree indexes are the most common type of index and order data within the index in branches and leaves. Composite indexes consist of more than one column to improve performance. When choosing indexes, consider columns frequently used in queries, primary keys, and foreign keys. Index maintenance includes rebuilding, coalescing, and shrinking indexes.
The document discusses adaptive query optimization in Oracle 12c. It begins by describing drawbacks of the optimizer in pre-12c versions, such as insufficient statistics triggering dynamic sampling. It then outlines the key features of adaptive query optimization in 12c, including adaptive/dynamic plans using techniques like adaptive parallel distribution and adaptive joins. It also discusses automatic re-optimization using feedback from initial executions. The document provides illustrations of these techniques using example queries and optimizer statistics.
About Flexible Indexing
Postgres’ rich variety of data structures and data-type specific indexes can be confusing for newer and experienced Postgres users alike who may be unsure when and how to use them. For example, gin indexing specializes in the rapid lookup of keys with many duplicates — an area where traditional btree indexes perform poorly. This is particularly useful for json and full text searching. GiST allows for efficient indexing of two-dimensional values and range types.
To listen to the recorded presentation with Bruce Momjian, visit Enterprisedb.com > Resources > Webcasts > Ondemand Webcasts.
For product information and subscriptions, please email sales@enterprisedb.com.
[PGDay.Seoul 2020] PostgreSQL 13 New Featureshyeongchae lee
This document discusses the new features in PostgreSQL version 13, including:
- Improved performance from de-duplicating B-tree index entries and parallel vacuuming of indexes.
- Better query planning for queries using aggregates, partitions, and extended statistics.
- Changes required for migrating to version 13, including incompatibilities.
The document summarizes how SQL Plan Directives in Oracle 12c can help address issues caused by cardinality misestimation in the optimizer. It provides an example where the optimizer underestimates the number of rows returned by a query on a table due to not having statistics on correlated columns. In 12c, a SQL Plan Directive is automatically generated after the first execution to capture this misestimation. On subsequent queries, the directive can be used to provide more accurate cardinality estimates through automatic reoptimization or dynamic sampling.
The document discusses Exadata Smart Scan, which offloads SQL processing from the database layer to the storage cell layer. Smart Scan passes only the required data blocks filtered by predicates to the database server instead of all blocks. Testing shows that with Smart Scan, a count query takes 0.05 seconds versus 15.68 seconds without Smart Scan. Analysis of statistics with Smart Scan shows most blocks are filtered at the storage layer with only a small amount of data, about 0.18MB, returned to the database layer.
[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스PgDay.Seoul
This document summarizes how to set up and use Citus, an open-source PostgreSQL-based distributed database. It explains how to install Citus, add worker nodes, create distributed tables, and use features like reference tables to perform distributed queries across the cluster.
This one is about advanced indexing in PostgreSQL. It guides you through basic concepts as well as through advanced techniques to speed up the database.
All important PostgreSQL Index types explained: btree, gin, gist, sp-gist and hashes.
Regular expression indexes and LIKE queries are also covered.
The document discusses histograms in Oracle databases before and after Oracle 12c. It describes the different types of histograms including frequency histograms, height-balanced histograms, top frequency histograms, and hybrid histograms. It highlights issues with histograms in Oracle 11g and how Oracle 12c introduced new histogram types to improve cardinality estimates.
Character Encoding - MySQL DevRoom - FOSDEM 2015mushupl
Character encoding configuration in MySQL has always been a bit confusing. With too many options to set, unclear relationships between them, and the default settings that make MySQL incompatible with most languages, it is a headache to many users, many of whom end up with broken data. This lecture will provide an overview of the character set support in MySQL, guidelines on how to use it correctly, and will demonstrate several methods of detecting and repairing mangled data.
This document summarizes a talk on optimizer hints in databases. It begins with introducing the speaker and their background. It then covers the basics of query optimization in databases and how hints can provide additional information to the optimizer. Specifically, it discusses query hints to force a plan, statistics hints to provide join selectivity, and data hints about column dependencies. It notes that PostgreSQL does not support hints directly but similar control is possible through configuration parameters. It concludes by listing some drawbacks of hints.
MariaDB: Engine Independent Table Statistics, including histogramsSergey Petrunya
EITS (Engine Independent Table Statistics) provides new statistics for MariaDB query optimization, including histograms for column value distributions. These statistics must be manually collected with ANALYZE TABLE and enabled to improve estimates for queries using non-indexed columns and ranges. With EITS, the optimizer can better estimate row counts and filtering for joins and WHERE clauses.
Basic Query Tuning Primer - Pg West 2009mattsmiley
Intro to query tuning in Postgres, for beginners or intermediate software developers. Lists your basic toolkit, common problems, a series of examples. Assumes the audience knows basic SQL but has little or no experience with reading or adjusting execution plans. Accompanies 45-90 minute talk; meant to encourage Q/A.
SQL Plan Management with Oracle Database provides tools to manage SQL performance and stability. It includes SQL profiles, stored outlines, SQL patches, and SQL baselines that can capture and enforce execution plans. New features in Oracle 12c include adaptive plans, which automatically choose join methods and parallel distribution, as well as adaptive statistics using dynamic sampling to improve cardinality estimates. Bind variable peeking and cardinality feedback also help the optimizer select optimal plans.
- The document discusses using various Oracle diagnostic tools like AWR, ASH, and SQL Monitoring for SQL tuning. It focuses on scenarios where multiple data sources are needed to fully understand performance issues.
- Historical ASH and AWR data can provide different perspectives on SQL executions over time that help identify problems like long-running queries or concurrent executions. However, ASH only samples a subset of data so it may miss short queries.
- GV$SQL and AWR reports aggregate performance metrics over all executions of a SQL, so they do not show the full user experience if a query runs intermittently. ASH sampled data can help determine how database time relates to clock time in such cases.
-
The Query Optimizer is the “brain” of your Postgres database. It interprets SQL queries and determines the fastest method of execution. Using the EXPLAIN command , this presentation shows how the optimizer interprets queries and determines optimal execution.
This presentation will give you a better understanding of how Postgres optimally executes their queries and what steps you can take to understand and perhaps improve its behavior in your environment.
To listen to the webinar recording, please visit EnterpriseDB.com > Resources > Ondemand Webcasts
If you have any questions please email sales@enterprisedb.com
The document discusses table partitioning and sharding in PostgreSQL as approaches to improve performance and scalability as data volumes grow over time. Table partitioning involves splitting a master table into multiple child tables or partitions based on a partition function to distribute data. Sharding distributes partitions across multiple database servers. The document provides steps to implement table partitioning and sharding in PostgreSQL using the Citus extension to distribute a sample sales table across a master and worker node.
New Tuning Features in Oracle 11g - How to make your database as boring as po...Sage Computing Services
One of the key problems that have haunted Oracle sites since the introduction of the cost based optimiser is the ability to provide a stable level of performance over time. The very responsiveness of the CBO to factors such as changes in statistics and initialisation parameters can lead to sudden changes in performance levels. Oracle 11g is set to introduce a number of features that will assist the DBA in providing a stable environment for mission critical applications. Excitement is for out of work time, (and for developers). The aim of most database administrators is to have as boring a working life as possible. Oracle 11g may help us achieve those aims.
This presentation discusses some of those features including:
Capture and replay of workload
Automatic SGA tuning
Managing and fixing plans
The 11g Automatic Tuning Advisor
The document discusses how the PostgreSQL query planner works. It explains that a query goes through several stages including parsing, rewriting, planning/optimizing, and execution. The optimizer or planner has to estimate things like the number of rows and cost to determine the most efficient query plan. Statistics collected by ANALYZE are used for these estimates but can sometimes be inaccurate, especially for n_distinct values. Increasing the default_statistics_target or overriding statistics on columns can help address underestimation issues. The document also discusses different plan types like joins, scans, and aggregates that the planner may choose between.
This document summarizes new features in the MariaDB 10.0 query optimizer, including:
1. Engine-independent statistics like histograms that are collected via ANALYZE TABLE instead of random sampling.
2. New subquery optimizations that convert EXISTS subqueries to inner joins and trivially correlated EXISTS to IN.
3. EXPLAIN improvements like SHOW EXPLAIN to see EXPLAIN plans for running queries, and logging EXPLAIN output in the slow query log.
This document compares the NOT EXISTS and NOT IN operators in SQL queries. It notes that NOT EXISTS is more efficient because it can stop evaluating the subquery after finding a single match, while NOT IN must scan the entire subquery. The document also shows that NOT IN performs a full table scan of the subquery for each row in the main query, while NOT EXISTS only scans tables twice total. It demonstrates these differences through examples run on two tables, comparing the session statistics and trace files. New features in Oracle 11g are also discussed that help optimize NOT IN queries.
The document discusses different methods for updating a target table based on values from a source table: correlated update, merge, and update from select. It finds that correlated update has poor performance without an index on the source table, as it must do a full table scan for each target row. Merge and update from select leave unmatched rows unchanged, while correlated update updates them to empty strings. Update from select requires a unique index on the source table. Overall, update from select has better performance when updating a small percentage of rows and the target table has an index.
The document discusses Exadata Smart Scan, which offloads SQL processing from the database layer to the storage cell layer. Smart Scan passes only the required data blocks filtered by predicates to the database server instead of all blocks. Testing shows that with Smart Scan, a count query takes 0.05 seconds versus 15.68 seconds without Smart Scan. Analysis of statistics with Smart Scan shows most blocks are filtered at the storage layer with only a small amount of data, about 0.18MB, returned to the database layer.
[Pgday.Seoul 2019] Citus를 이용한 분산 데이터베이스PgDay.Seoul
This document summarizes how to set up and use Citus, an open-source PostgreSQL-based distributed database. It explains how to install Citus, add worker nodes, create distributed tables, and use features like reference tables to perform distributed queries across the cluster.
This one is about advanced indexing in PostgreSQL. It guides you through basic concepts as well as through advanced techniques to speed up the database.
All important PostgreSQL Index types explained: btree, gin, gist, sp-gist and hashes.
Regular expression indexes and LIKE queries are also covered.
The document discusses histograms in Oracle databases before and after Oracle 12c. It describes the different types of histograms including frequency histograms, height-balanced histograms, top frequency histograms, and hybrid histograms. It highlights issues with histograms in Oracle 11g and how Oracle 12c introduced new histogram types to improve cardinality estimates.
Character Encoding - MySQL DevRoom - FOSDEM 2015mushupl
Character encoding configuration in MySQL has always been a bit confusing. With too many options to set, unclear relationships between them, and the default settings that make MySQL incompatible with most languages, it is a headache to many users, many of whom end up with broken data. This lecture will provide an overview of the character set support in MySQL, guidelines on how to use it correctly, and will demonstrate several methods of detecting and repairing mangled data.
This document summarizes a talk on optimizer hints in databases. It begins with introducing the speaker and their background. It then covers the basics of query optimization in databases and how hints can provide additional information to the optimizer. Specifically, it discusses query hints to force a plan, statistics hints to provide join selectivity, and data hints about column dependencies. It notes that PostgreSQL does not support hints directly but similar control is possible through configuration parameters. It concludes by listing some drawbacks of hints.
MariaDB: Engine Independent Table Statistics, including histogramsSergey Petrunya
EITS (Engine Independent Table Statistics) provides new statistics for MariaDB query optimization, including histograms for column value distributions. These statistics must be manually collected with ANALYZE TABLE and enabled to improve estimates for queries using non-indexed columns and ranges. With EITS, the optimizer can better estimate row counts and filtering for joins and WHERE clauses.
Basic Query Tuning Primer - Pg West 2009mattsmiley
Intro to query tuning in Postgres, for beginners or intermediate software developers. Lists your basic toolkit, common problems, a series of examples. Assumes the audience knows basic SQL but has little or no experience with reading or adjusting execution plans. Accompanies 45-90 minute talk; meant to encourage Q/A.
SQL Plan Management with Oracle Database provides tools to manage SQL performance and stability. It includes SQL profiles, stored outlines, SQL patches, and SQL baselines that can capture and enforce execution plans. New features in Oracle 12c include adaptive plans, which automatically choose join methods and parallel distribution, as well as adaptive statistics using dynamic sampling to improve cardinality estimates. Bind variable peeking and cardinality feedback also help the optimizer select optimal plans.
- The document discusses using various Oracle diagnostic tools like AWR, ASH, and SQL Monitoring for SQL tuning. It focuses on scenarios where multiple data sources are needed to fully understand performance issues.
- Historical ASH and AWR data can provide different perspectives on SQL executions over time that help identify problems like long-running queries or concurrent executions. However, ASH only samples a subset of data so it may miss short queries.
- GV$SQL and AWR reports aggregate performance metrics over all executions of a SQL, so they do not show the full user experience if a query runs intermittently. ASH sampled data can help determine how database time relates to clock time in such cases.
-
The Query Optimizer is the “brain” of your Postgres database. It interprets SQL queries and determines the fastest method of execution. Using the EXPLAIN command , this presentation shows how the optimizer interprets queries and determines optimal execution.
This presentation will give you a better understanding of how Postgres optimally executes their queries and what steps you can take to understand and perhaps improve its behavior in your environment.
To listen to the webinar recording, please visit EnterpriseDB.com > Resources > Ondemand Webcasts
If you have any questions please email sales@enterprisedb.com
The document discusses table partitioning and sharding in PostgreSQL as approaches to improve performance and scalability as data volumes grow over time. Table partitioning involves splitting a master table into multiple child tables or partitions based on a partition function to distribute data. Sharding distributes partitions across multiple database servers. The document provides steps to implement table partitioning and sharding in PostgreSQL using the Citus extension to distribute a sample sales table across a master and worker node.
New Tuning Features in Oracle 11g - How to make your database as boring as po...Sage Computing Services
One of the key problems that have haunted Oracle sites since the introduction of the cost based optimiser is the ability to provide a stable level of performance over time. The very responsiveness of the CBO to factors such as changes in statistics and initialisation parameters can lead to sudden changes in performance levels. Oracle 11g is set to introduce a number of features that will assist the DBA in providing a stable environment for mission critical applications. Excitement is for out of work time, (and for developers). The aim of most database administrators is to have as boring a working life as possible. Oracle 11g may help us achieve those aims.
This presentation discusses some of those features including:
Capture and replay of workload
Automatic SGA tuning
Managing and fixing plans
The 11g Automatic Tuning Advisor
The document discusses how the PostgreSQL query planner works. It explains that a query goes through several stages including parsing, rewriting, planning/optimizing, and execution. The optimizer or planner has to estimate things like the number of rows and cost to determine the most efficient query plan. Statistics collected by ANALYZE are used for these estimates but can sometimes be inaccurate, especially for n_distinct values. Increasing the default_statistics_target or overriding statistics on columns can help address underestimation issues. The document also discusses different plan types like joins, scans, and aggregates that the planner may choose between.
This document summarizes new features in the MariaDB 10.0 query optimizer, including:
1. Engine-independent statistics like histograms that are collected via ANALYZE TABLE instead of random sampling.
2. New subquery optimizations that convert EXISTS subqueries to inner joins and trivially correlated EXISTS to IN.
3. EXPLAIN improvements like SHOW EXPLAIN to see EXPLAIN plans for running queries, and logging EXPLAIN output in the slow query log.
This document compares the NOT EXISTS and NOT IN operators in SQL queries. It notes that NOT EXISTS is more efficient because it can stop evaluating the subquery after finding a single match, while NOT IN must scan the entire subquery. The document also shows that NOT IN performs a full table scan of the subquery for each row in the main query, while NOT EXISTS only scans tables twice total. It demonstrates these differences through examples run on two tables, comparing the session statistics and trace files. New features in Oracle 11g are also discussed that help optimize NOT IN queries.
The document discusses different methods for updating a target table based on values from a source table: correlated update, merge, and update from select. It finds that correlated update has poor performance without an index on the source table, as it must do a full table scan for each target row. Merge and update from select leave unmatched rows unchanged, while correlated update updates them to empty strings. Update from select requires a unique index on the source table. Overall, update from select has better performance when updating a small percentage of rows and the target table has an index.
Subquery factoring, introduced in Oracle 9i, allows queries to be rewritten using a WITH clause to simplify queries and improve performance in some cases. For a query using a full table scan that scanned the same table twice, subquery factoring reduced logical and physical I/O by half by scanning the table once and storing results in memory. For a query using unique index scans, subquery factoring increased I/O due to creating a temporary table, showing there is no single best approach and testing is required.
Hash join is a type of join operation that uses a hash table to perform the join. There are three types of hash joins - optimal, onepass, and multipass. Optimal hash join performs the join entirely in memory, while onepass and multipass hash joins spill data to temporary storage due to insufficient memory. The size of the build table can impact the performance and memory requirements of the hash join, with smaller build tables generally requiring less memory but potentially more disk reads. The best build table depends on the relative sizes of the tables and available memory.
This document discusses improvements to nested loop joins (NLJs) in Oracle 11g. A new technique called table batching is introduced that creates two NLJs - one for the index and another for the table. This performs similarly to the prefetching technique in 9i. Testing shows that consistent gets are reduced from 42,000 to 34,000 from 10g to 11g for all techniques. The batching technique is fastest while classic NLJ is slowest. Array size also impacts consistent gets by determining the number of network round trips. In conclusion, prefetching benefits non-unique indexes, and 11g optimizations improve performance versus 10g for all join techniques.
The document discusses several useful features of analytical functions in Oracle including ranking, lag/lead, windowing, and flashback queries. It provides examples of ranking functions like RANK, DENSE_RANK, and ROW_NUMBER. It also discusses LAG and LEAD functions, windowing, and how to rewrite queries using EXISTS instead of IN for better performance. Flashback features allow querying past states of the database, tables, and transactions.
The document provides an introduction to the Oracle optimizer. It discusses key concepts like cost, selectivity, cardinality, clustering factor, access methods including table scans, index scans and joins. It covers partitioning, subqueries and hints. The examples show how statistics, histograms and indexes impact the optimizer's choice of access paths and joins.
This document discusses various Oracle SQL concepts including query optimization, execution plans, joins, indexes, and full table scans. It provides guidance on understanding how Oracle processes and executes SQL queries, the importance of statistics and selectivity, and techniques for writing efficient queries such as predicate pushing and query transformations. The goal is to help readers gain a conceptual understanding of Oracle's internals to formulate more efficient SQL.
Matt Smiley
This is a basic primer aimed primarily at developers or DBAs new to Postgres. The format is a Q/A style tour with examples, based on common questions and pitfalls. Begin with a quick tour of relevant parts of the postgres catalog, with an aim to answer simple but important questions like:
How many rows does the optimizer think my table has?
When was it last analyzed?
Which other tables also have a column named "foo"?
How often is this index used?
Understanding Query Optimization with ‘regular’ and ‘Exadata’ OracleGuatemala User Group
The document discusses query optimization with regular Oracle databases and Exadata databases. It explains what happens when a SQL statement is issued, including parsing, optimization, and execution. It describes what an execution plan is and how it can be generated and displayed. It discusses how operations can be offloaded to storage cells on Exadata and factors the optimizer considers for determining a good execution plan.
Dimensional performance benchmarking of SQLBrendan Furey
This document summarizes a presentation on dimensional performance benchmarking of SQL. It introduces a framework for defining data sets in terms of parameters (x,y) and running queries across a 2D grid to capture performance metrics. Six examples are described that demonstrate different SQL techniques for problems like bursting, hierarchies, and string splitting. The results show how performance varies based on data size and shape. The framework allows rigorous and automated testing of SQL performance.
Informix Warehouse Accelerator (IWA) features in version 12.1Keshav Murthy
The document discusses enhancements made to Informix Warehouse Accelerator (IWA) in version 12.10. Key points include:
- IWA now supports operations like creating, deploying, loading, enabling, and disabling data marts on secondary nodes in MACH11 and high availability environments, in addition to the primary/standard server node.
- New procedures like dropPartMart and loadPartMart allow refreshing partitions in a partitioned fact table within a data mart.
- Performance of SQL queries involving UNIONs, derived tables, and DISTINCT aggregates was improved.
- Additional OLAP functions and options like NULLS FIRST/LAST in ORDER BY were added for enhanced analytical querying.
In this first of a series of presentations, we'll overview the differences between SQL and PL/SQL, and the first steps in optimization, as understanding RULE vs. COST, and how to slash 90% response time in data extractions running in SQL*Plus.
On version 12c Oracle introduced new features to allow Adaptive optimizations: Adaptive Plans and Adaptive Statistics. After a quick presentation of concepts, this session will explore the interaction of these features with other performance management techniques using examples, like SPM and SQL profiles. Attendees will get an updated picture of tools available to troubleshoot performance issues, and how to get the most of these new features.
Kellyn Pot’Vin-Gorman presented a comparison of indexing in Oracle and SQL Server databases. They loaded test data into indexes in each platform, altered the fill factor/pctfree settings, and measured performance of data loads, updates, deletes and index storage. SQL Server exhibited page splits when indexes became fragmented, while Oracle showed leaf block splits. Rebuilding indexes with high pctfree in Oracle had significantly worse performance than rebuilding with lower fillfactor in SQL Server. Overall, Oracle generally outperformed SQL Server for indexing and was less prone to fragmentation issues.
This document discusses various diagnostic tools and query tuning techniques in PostgreSQL, including:
- PostgreSQL workload monitoring tools like Mamonsu and Zabbix Agent for collecting metrics.
- Extensions for tracking resource-intensive queries like pg_stat_statements, pg_stat_kcache, and auto_explain.
- Using pg_profile to detect resource-consuming queries and view statistics on execution time, shared blocks fetched, and I/O waiting time.
- Query tuning techniques like replacing DISTINCT and window functions with LIMIT, optimizing queries using GROUP BY, and creating appropriate indexes.
The document discusses Oracle database performance tuning. It covers identifying and resolving performance issues through tools like AWR and ASH reports. Common causes of performance problems include wait events, old statistics, incorrect execution plans, and I/O issues. The document recommends collecting specific data when analyzing problems and provides references and scripts for further tuning tasks.
MySQL 8 -- A new beginning : Sunshine PHP/PHP UK (updated)Dave Stokes
MySQL 8 has many new features and this presentation covers the new data dictionary, improved JSON functions, roles, histograms, and much more. Updated after SunshinePHP 2018 after feedback
This document provides 9 hints for optimizing Oracle database performance:
1. Take a methodical and empirical approach to tuning by focusing on root causes, measuring performance before and after changes, and avoiding "silver bullets".
2. Design databases and applications with performance in mind from the beginning.
3. Index wisely by only creating useful indexes that improve performance without excessive overhead.
4. Leverage built-in Oracle tools like DBMS_XPLAN and SQL Trace to measure performance.
5. Tune the optimizer by adjusting parameters and statistics to encourage better execution plans.
6. Focus SQL and PL/SQL tuning on problem queries, joins, sorts, and DML statements.
7. Address
The document discusses several topics related to SQL:
1) SQLNet compression - How ordering data in a query can significantly reduce the amount of data sent over the network by compressing repeated values. Ordering by additional columns further improves compression.
2) NULLs and indexes - There is a misconception that indexes cannot be used with queries involving NULL values, but indexes can support queries searching for NULL values.
3) Subquery caching - Repeated scalar subqueries are cached and evaluated only once to improve performance of queries containing subqueries.
SQL Macros - Game Changing Feature for SQL Developers?Andrej Pashchenko
SQL Macros are functions that return SQL statements as text. When called in a SQL statement, the returned SQL text is parsed and optimized rather than executing the function at runtime. This avoids context switches to PL/SQL and allows the optimizer to see the full SQL. Table SQL macros can be called in the FROM clause and act like views or inline queries, except they allow parameters to make the views polymorphic. Scalar parameters in the returned SQL text are substituted like bind variables to make the macros more reusable and flexible.
This paper describes the evolution of the Plan table and DBMSX_PLAN in 11g and some of the features that can be used to troubelshoot SQL performance effectively and efficiently.
Tony jambu (obscure) tools of the trade for tuning oracle sq lsInSync Conference
There are several tools available for SQL tuning in Oracle, including those that generate explain plans, analyze trace files, and provide real-time SQL monitoring. The document discusses tuning methodology, generating explain plans with SQL*Plus and Autotrace, tracing using parameters and DBMS_MONITOR, and tools like DBMS_XPLAN, TRCA, SQLTXPLAIN, Oracle Active Report, and Toad. It provides examples of using many of these tools to analyze SQL performance.
Using PostgreSQL statistics to optimize performance Alexey Ermakov
The document discusses using statistics in PostgreSQL to optimize performance. It describes how the planner estimates row counts in tables and selectivity of query conditions. Default estimators are used if no statistics are collected. Statistics are gathered on tables and indexes to estimate selectivity. Partial indexes can be useful when not all values need to be indexed. Monitoring and diagnosing performance issues is also covered.
Tony Jambu (obscure) tools of the trade for tuning oracle sq lsInSync Conference
This document provides an overview of various tools that can be used for tuning Oracle SQL statements. It discusses tuning methodology, generating explain plans and traces, and tools like SQL*Plus autotrace, DBMS_XPLAN, TRCA trace analyzer, and SQLTXPLAIN. Demo examples are provided for many of the tools to analyze SQL performance.
Similar to Checking clustering factor to detect row migration (20)
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Checking clustering factor to detect row migration
1. Today, while checking PRODUTION DB, I saw 1 SQL_ID with 2 different execution plans as below:
SQL Plan
Plan hash value: 561816070
------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
------------------------------------------------------------------------|
0 | UPDATE STATEMENT
|
|
|
|
78 |
|
1 | UPDATE
| TBL_XXX|
|
|
|
|
2 |
TABLE ACCESS FULL| TBL_XXX|
236 | 14632 |
78 |
------------------------------------------------------------------------Plan hash value: 2757418408
-------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
-------------------------------------------------------------------------------------|
0 | UPDATE STATEMENT
|
|
|
|
1 |
|
1 | UPDATE
| TBL_XXX
|
|
|
|
|
2 |
TABLE ACCESS BY INDEX ROWID| TBL_XXX
|
1 |
72 |
1 |
|
3 |
INDEX RANGE SCAN
| TBL_XXX_PK |
1 |
|
1 |
--------------------------------------------------------------------------------------
I/O Comparison for SELECT Statement
SQL> select /*+ full(TBL_XXX) */ * from TBL_XXX where site_id=234;
8996 rows selected.
Elapsed: 00:00:00.51
Execution Plan
---------------------------------------------------------Plan hash value: 1988479474
-----------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
-----------------------------------------------------------------------|
0 | SELECT STATEMENT |
| 8463 |
520K|
493 |
|* 1 | TABLE ACCESS FULL| TBL_XXX
| 8463 |
520K|
493 |
-----------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------1 - filter("SITE_ID"=234)
Statistics
---------------------------------------------------------1 recursive calls
2. 0
8668
0
0
542896
7081
601
0
0
8996
db block gets
consistent gets
physical reads
redo size
bytes sent via SQL*Net to client
bytes received via SQL*Net from client
SQL*Net roundtrips to/from client
sorts (memory)
sorts (disk)
rows processed
SQL> select /*+ index(TBL_XXX,TBL_XXX_pk) */ * from TBL_XXX where site_id=234;
8996 rows selected.
Elapsed: 00:00:01.38
Execution Plan
---------------------------------------------------------Plan hash value: 2363445147
------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
------------------------------------------------------------------------------------|
0 | SELECT STATEMENT
|
| 8463 |
520K|
499 |
|
1 | TABLE ACCESS BY INDEX ROWID| TBL_XXX
| 8463 |
520K|
499 |
|* 2 |
INDEX RANGE SCAN
| TBL_XXX_PK | 8463 |
|
5 |
------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------2 - access("SITE_ID"=234)
Statistics
---------------------------------------------------------1 recursive calls
0 db block gets
6179 consistent gets
67 physical reads
0 redo size
534539 bytes sent via SQL*Net to client
7081 bytes received via SQL*Net from client
601 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
8996 rows processed
If we see the average execution time, this is fine (still in milliseconds). And also it doesn’t execute too
frequently (232 times in every 15 minutes).
First plan (2757418408) uses Index Range Scan on its PK while the other (561816070) uses FTS.
Checking the data dictionary, I came to know that the FTS is being used by the tables which have statistics
while the Index Range Scan is being used by the tables with no statistics on it.
Table and Index Statistics
SQL> select owner,clustering_factor,NUM_ROWS,LEAF_BLOCKS,DISTINCT_KEYS,last_analyzed from
dba_indexes where index_name='TBL_XXX_1IX';
OWNER
CLUSTERING_FACTOR
NUM_ROWS LEAF_BLOCKS DISTINCT_KEYS LAST_ANAL
------------------------------ ----------------- ---------- ----------- ------------- --------SCHEMA1
21490
592461
5606
592461 11-MAR-13
SCHEMA2
SCHEMA3
SCHEMA4
SCHEMA5
3. SCHEMA6
SCHEMA7
SCHEMA8
SCHEMA9
SCHEMA10
SCHEMA11
105009
8853
3179805
343400
26048
887
3179805 08-NOV-12
343400 15-SEP-11
11 rows selected.
SQL> select owner,clustering_factor,NUM_ROWS,LEAF_BLOCKS,DISTINCT_KEYS,last_analyzed from
dba_indexes where index_name='TBL_XXX_PK';
OWNER
CLUSTERING_FACTOR
NUM_ROWS LEAF_BLOCKS DISTINCT_KEYS LAST_ANAL
------------------------------ ----------------- ---------- ----------- ------------- --------SCHEMA1
335430
575315
2986
575315 11-MAR-13
SCHEMA2
53205
118276
502
118276 30-JUN-10
SCHEMA3
SCHEMA4
SCHEMA5
SCHEMA6
1783417
3093008
13264
3093008 08-NOV-12
SCHEMA7
53205
118276
502
118276 30-JUN-10
SCHEMA8
8 rows selected.
SQL> select OWNER,BLOCKS,NUM_ROWS,LAST_ANALYZED from dba_tables where table_name='TBL_XXX';
OWNER
BLOCKS
NUM_ROWS LAST_ANAL
------------------------------ ---------- ---------- --------SCHEMA10
SCHEMA9
SCHEMA8
SCHEMA7
SCHEMA6
1258
235533 30-JUN-10
SCHEMA5
27445
3122013 08-NOV-12
SCHEMA4
SCHEMA3
SCHEMA2
SCHEMA2
SCHEMA1
8065
575315 11-MAR-13
I was trying to solve this issue by creating SQL Profile (using my normal template) for this SQL_ID to force Index
Range Scan but it doesn’t workfor SCHEMA1account
Based on below data distribution, index scan should be much more efficient (it will scan approx. 1.5%of total
population; there are 64 distinct values for SITE_ID with “evenly” distributed data)
Also I have updated the statistics but still it doesn’t work
The question is WHY???
Data Distribution
SQL> select SITE_ID,count(*) from schema1.TBL_XXX group by SITE_ID;
SITE_ID
COUNT(*)
---------- ---------100
8987
201
8993
202
8993
203
8993
204
8992
...cut here to safe the space...
257
258
289
292
295
1000
8987
8987
8987
8987
8987
1
4. 65 rows selected.
SQL Profile
begin
dbms_sqltune.import_sql_profile(
name =>'tbl_xxx_063hrvz3s17vk',
sql_text =>
'UPDATE TBL_XXX SET EXP_DATE = :closingTime, SYS_UPDATE_DATE =
sysdate,OPERATOR_ID = :OPERATOR_ID,APPLICATION_ID =
:APPLICATION_ID,DL_SERVICE_CODE = :DL_SERVICE_CODE,DL_UPDATE_STAMP =
:DL_UPDATE_STAMP WHERE EXP_DATE IS NULL AND SITE_ID = :siteID',
profile => sqlprof_attr(
'INDEX_RS_ASC(@"UPD$1" "TBL_XXX"@"UPD$1"',
'
("TBL_XXX"."SITE_ID" "TBL_XXX"."INTERVAL_ID"))',
'OUTLINE_LEAF(@"UPD$1")',
'IGNORE_OPTIM_EMBEDDED_HINTS'),
force_match => TRUE
);
end;
/
SQL>conn schema1/passwd
Connected.
SQL> explain plan for
2 UPDATE TBL_XXX SET EXP_DATE = :closingTime, SYS_UPDATE_DATE =
3 sysdate,OPERATOR_ID = :OPERATOR_ID,APPLICATION_ID =
4 :APPLICATION_ID,DL_SERVICE_CODE = :DL_SERVICE_CODE,DL_UPDATE_STAMP =
5 :DL_UPDATE_STAMP WHERE EXP_DATE IS NULL AND SITE_ID = :siteID;
Explained.
SQL> select * from table(dbms_xplan.display);
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------Plan hash value: 561816070
------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
------------------------------------------------------------------------|
0 | UPDATE STATEMENT
|
|
1 |
37 |
493 |
|
1 | UPDATE
| TBL_XXX |
|
|
|
|* 2 |
TABLE ACCESS FULL| TBL_XXX |
1 |
37 |
493 |
------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------2 - filter("EXP_DATE" IS NULL AND "SITE_ID"=TO_NUMBER(:SITEID))
Note
----- cpu costing is off (consider enabling it)
- SQL profile "tbl_xxx_063hrvz3s17vk" used for this statement
Going back to the table and index statistics, I am curious with the value of clustering_factorand it gives me
some clues.
It is the number time (if you are subscribing for “Baby First TV” channel, then you will familiar with that
song )
OWNER
---------SCHEMA1
SCHEMA1
INDEX_NAME
CLUSTERING_FACTOR
NUM_ROWS LEAF_BLOCKS DISTINCT_KEYS LAST_ANAL
---------------------- ----------------- ---------- ----------- ------------- --------TBL_XXX_1IX
21490
592461
5606
592461 11-MAR-13
TBL_XXX_PK
335430
575315
2986
575315 11-MAR-13
5. OWNER
TABLE_NAME
BLOCKS
NUM_ROWS LAST_ANAL
---------- ---------------------- ---------- ---------- --------SCHEMA1
TBL_XXX
8065
575315 11-MAR-13
There are 8,065 blocks in TBL_XXX with 575,315 rows (approx. 71 rows per block)
If we take a look on the index statistics, the clustering_factor of TBL_XXX_PK is 335,430 (it’s close to the
number of rows rather than blocks).
P.S. For the clustering_factor topic, I have explained it (very small pieces) in my “Introduction to Oracle
Optimizer” PPT or better googling it.
What I can say after seeing those statistics is that the data in the table are scattered from TBL_XXX_PK point of
few.
The side effect is that Oracle think it will requires more I/O to get the data when we use index access compare
to FTS (you can confirmed that by looking at the output from below explain plan, the cost of FTS [493] is
smaller than cost of Index Range Scan [530]). The word think here is referring to Oracle cost based optimizer
calculation. You can check how the COST is being calculated for FTS and Index Range Scan in the web (googling)
or my above PPT.
Explain Plan Output
SQL>
2
3
4
5
6
explain plan for
UPDATE /*+ index(TBL_XXX, TBL_XXX_PK) */ TBL_XXX
SET EXP_DATE = :closingTime, SYS_UPDATE_DATE =
sysdate,OPERATOR_ID = :OPERATOR_ID,APPLICATION_ID =
:APPLICATION_ID,DL_SERVICE_CODE = :DL_SERVICE_CODE,DL_UPDATE_STAMP =
:DL_UPDATE_STAMP WHERE EXP_DATE IS NULL AND SITE_ID = :siteID;
Explained.
Elapsed: 00:00:00.05
SQL> select * from table(dbms_xplan.display(null,null,'ADVANCED'));
PLAN_TABLE_OUTPUT
-------------------------------------------------------------------------------------Plan hash value: 2757418408
-------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
-------------------------------------------------------------------------------------|
0 | UPDATE STATEMENT
|
|
1 |
37 |
530 |
|
1 | UPDATE
| TBL_XXX
|
|
|
|
|* 2 |
TABLE ACCESS BY INDEX ROWID| TBL_XXX
|
1 |
37 |
530 |
|* 3 |
INDEX RANGE SCAN
| TBL_XXX_PK | 8989 |
|
5 |
--------------------------------------------------------------------------------------
SQL> explain plan for
2 UPDATE TBL_XXX
3 SET EXP_DATE = :closingTime, SYS_UPDATE_DATE =
4 sysdate,OPERATOR_ID = :OPERATOR_ID,APPLICATION_ID =
5 :APPLICATION_ID,DL_SERVICE_CODE = :DL_SERVICE_CODE,DL_UPDATE_STAMP =
6 :DL_UPDATE_STAMP WHERE EXP_DATE IS NULL AND SITE_ID = :siteID;
select * from table(dbms_xplan.display);
Explained.
Elapsed: 00:00:00.04
SQL>
PLAN_TABLE_OUTPUT
------------------------------------------------------------------------Plan hash value: 561816070
------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
-------------------------------------------------------------------------
6. |
0 | UPDATE STATEMENT
|
|
1 |
37 |
493 |
|
1 | UPDATE
| TBL_XXX|
|
|
|
|* 2 |
TABLE ACCESS FULL| TBL_XXX|
1 |
37 |
493 |
-------------------------------------------------------------------------
For the temporary solution, I have recreated the SQL profile using additional information as below yellow
highlighted (setting optimizer_index_cost_adj parameter to half of previous value, it was 10) to tell Oracle that
index access is half expensive than before
And cost is reduced from 530 (Test Cast section) to 265 (green highlighted)
Simple math will be (5 / 10) * 530 = 265
Where:
5 is new value of optimizer_index_cost_adj
10 is previous value of optimizer_index_cost_adj
530 is previous COST of Index Range Scan
exec dbms_sqltune.drop_sql_profile('tbl_xxx_063hrvz3s17vk');
/
begin
dbms_sqltune.import_sql_profile(
name => 'tbl_xxx_063hrvz3s17vk',
sql_text =>
'UPDATE TBL_XXX SET EXP_DATE = :closingTime, SYS_UPDATE_DATE =
sysdate,OPERATOR_ID = :OPERATOR_ID,APPLICATION_ID =
:APPLICATION_ID,DL_SERVICE_CODE = :DL_SERVICE_CODE,DL_UPDATE_STAMP =
:DL_UPDATE_STAMP WHERE EXP_DATE IS NULL AND SITE_ID = :siteID',
profile => sqlprof_attr(
'INDEX_RS_ASC(@"UPD$1" "TBL_XXX"@"UPD$1"',
'
("TBL_XXX"."SITE_ID" "TBL_XXX"."INTERVAL_ID"))',
'OUTLINE_LEAF(@"UPD$1")',
'OPT_PARAM(''optimizer_index_cost_adj'' 5)',
'IGNORE_OPTIM_EMBEDDED_HINTS')
);
end;
SQL> conn schema1/passwd
SQL> explain plan for
2
UPDATE TBL_XXX
3
SET EXP_DATE = :closingTime, SYS_UPDATE_DATE =
4
sysdate,OPERATOR_ID = :OPERATOR_ID,APPLICATION_ID =
5
:APPLICATION_ID,DL_SERVICE_CODE = :DL_SERVICE_CODE,DL_UPDATE_STAMP =
6
:DL_UPDATE_STAMP WHERE EXP_DATE IS NULL AND SITE_ID = :siteID;
SQL> select * from table(dbms_xplan.display);
Explained.
PLAN_TABLE_OUTPUT
-------------------------------------------------------------------------------------Plan hash value: 2757418408
-------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
-------------------------------------------------------------------------------------|
0 | UPDATE STATEMENT
|
|
1 |
37 |
265 |
|
1 | UPDATE
| TBL_XXX
|
|
|
|
|* 2 |
TABLE ACCESS BY INDEX ROWID| TBL_XXX
|
1 |
37 |
265 |
|* 3 |
INDEX RANGE SCAN
| TBL_XXX_PK | 8989 |
|
3 |
--------------------------------------------------------------------------------------
BUT, setting optimizer_index_cost_adj to a very low value (as what I did), for.ex. 5 is not “a proper way” if we
don’t have any idea about data distribution. Of course we can set the value to 6 or 7 or any value less than 10
only to reduce the calculated COST of Index Scan.
So I will continue my investigation by focusing on the huge value of clustering factor.
7. When I first time see that huge clustering factor, there is only one thing in my mind, this is a common issue
with application design (table structure, red) and we might hit “ROW MIGRATION” issue
How come a Primary Key has a huge clustering factor, it should be less (close to number of block) if we design it
“properly” (pctree, initrans, etc)
This table is having “default” value of PCTFREE (10) and seems the application is behaving like this (I am
guessing here, so need to check with application team):
- The initial insert only set a value for “several” column (NULL for almost not-null columns)
- Next update will set the “previous-empty” column to some value
With above behavior, a lot of “short anduncompleted” rows will be there in every single block and once the
UPDATE comes, that block will not be sufficient anymore to hold that row and then the row will be migrated
into other blocks.
Let’s the “NUMBER” confirmed it by doing below test case. The purpose of this case is to check the value of 2
system/ session statistics: “table fetch by rowid” and “table fetch continued row”
1. First I will check the initial value of below 2 statistics (maroon colored font)
2. Then I will do FTS on TBL_XXX and some “small” increment on the “table fetch continued row”,
probably we have chained-row here (green colored font)
3. Finally, I force Index Range Scan on TBL_XXX_PK and the “table fetch continued row” increase from 14
to 217 (blue colored font).
It confirmed that there are, at least, approx. 200 rows with more than 1 block (migrated)
The Test Case
Initial Value
SQL>
2
3
4
select name,class,value
from v$sesstat a,v$statname b
where a.STATISTIC#=b.STATISTIC# and name like 'table fetch%'
and sid=9868;
NAME
CLASS
VALUE
---------------------------------------------- ---------- ---------table fetch by rowid
64
64
table fetch continued row
64
0
Full Table Scan
SQL> select * from TBL_XXX where site_id=100;
9129 rows selected.
Execution Plan
---------------------------------------------------------Plan hash value: 1988479474
-----------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
-----------------------------------------------------------------------|
0 | SELECT STATEMENT |
| 9751 |
599K|
493 |
|* 1 | TABLE ACCESS FULL| TBL_XXX | 9751 |
599K|
493 |
-----------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------1 - filter("SITE_ID"=100)
8. Statistics
---------------------------------------------------------267 recursive calls
0 db block gets
8736 consistent gets
4744 physical reads
0 redo size
541965 bytes sent via SQL*Net to client
7180 bytes received via SQL*Net from client
610 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
9129 rows processed
SQL>
2
3
4
select name,class,value
from v$sesstat a,v$statname b
where a.STATISTIC#=b.STATISTIC# and name like 'table fetch%'
and sid=9868;
NAME
CLASS
VALUE
---------------------------------------------- ---------- ---------table fetch by rowid
64
1014
table fetch continued row
64
14
Index Range Scan
SQL> select /*+ index(TBL_XXX, TBL_XXX_pk) */ * from TBL_XXX where site_id=100;
9129 rows selected.
Execution Plan
---------------------------------------------------------Plan hash value: 2363445147
------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
------------------------------------------------------------------------------------|
0 | SELECT STATEMENT
|
| 9751 |
599K|
574 |
|
1 | TABLE ACCESS BY INDEX ROWID| TBL_XXX
| 9751 |
599K|
574 |
|* 2 |
INDEX RANGE SCAN
| TBL_XXX_PK | 9751 |
|
6 |
------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------2 - access("SITE_ID"=100)
Statistics
---------------------------------------------------------1 recursive calls
0 db block gets
6278 consistent gets
591 physical reads
0 redo size
533312 bytes sent via SQL*Net to client
7180 bytes received via SQL*Net from client
610 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
9129 rows processed
SQL>
2
3
4
select name,class,value
from v$sesstat a,v$statname b
where a.STATISTIC#=b.STATISTIC# and name like 'table fetch%'
and sid=9868;
NAME
CLASS
VALUE
---------------------------------------------- ---------- ---------table fetch by rowid
64
10143
table fetch continued row
64
217
9. From Oracle documentation you will get the explanation of those 2 statistics:
The sensible solution (align with above documentation):
1. Alter or reorganize the table by specifying bigger value for PCT_FREE (this will give more room for any
rows to grow, because of UPDATE statement)
2. Create backup of TBL_XXX, truncate the original table and reinsert the data (ORDER BY SITE_ID) so that
data is ordered by SITE_ID, then clustering factor of the Primary Key will be reduced
3. Rebuild the indexes
If we only move/rebuild the table/index without changing PCT_FREE, there is a chance that this issue will
happened again in the future.
In the below Final Result, we can see good result of this tuning:
- I/O is decreased from 6K to 2K
- No row migration (from the value of “table fetch continued row” statistic)
- Without any SQL Hint or SQL Profile, it goes for Index Range San
Proposed Permanent Solution
Size of PCTFREE should be calculated based on block size and average row length (all columns should be filled),
so that we can have good value for average row per block. Below “PCTFREE 10” is default value and being used
for testing only.
SQL> create table dbc_TBL_XXXpctfree 10as select * from schema1.TBL_XXX order by site_id;
Table created.
SQL> create index idx_dbc_TBL_XXX on dbc_TBL_XXX(site_id, interval_id);
Index created.
SQL> exec dbms_stats.gather_table_stats(USER, 'DBC_TBL_XXX');
PL/SQL procedure successfully completed.
SQL> select clustering_factor, last_analyzed from user_indexes where
index_name='IDX_DBC_TBL_XXX';
10. CLUSTERING_FACTOR LAST_ANAL
----------------- --------107754 17-MAR-13
SQL> select blocks,num_rows,last_analyzed from user_tables where table_name='DBC_TBL_XXX';
BLOCKS
NUM_ROWS LAST_ANAL
---------- ---------- --------5579
572435 17-MAR-13
SQL> select * from dbc_TBL_XXX where site_id=100;
8941 rows selected.
Execution Plan
-----------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
-----------------------------------------------------------------------------------------|
0 | SELECT STATEMENT
|
| 8944 |
550K|
172 |
|
1 | TABLE ACCESS BY INDEX ROWID| DBC_TBL_XXX| 8944 |
550K|
172 |
|* 2 |
INDEX RANGE SCAN
| IDX_DBC_TBL_XXX| 8944 |
|
3 |
-----------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------2 - access("SITE_ID"=100)
Statistics
---------------------------------------------------------1 recursive calls
0 db block gets
2764 consistent gets
3 physical reads
0 redo size
495600 bytes sent via SQL*Net to client
7048 bytes received via SQL*Net from client
598 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
8941 rows processed
Final Result
SQL> select distinct sid from v$mystat;
SID
---------12359
SQL>
SQL>
SQL>
2
3
4
set pages 200 lines 200
col name for a35
select name,class,value
from v$sesstat a,v$statname b
where a.STATISTIC#=b.STATISTIC# and name like 'table fetch%'
and sid=12359;
NAME
CLASS
VALUE
----------------------------------- ---------- ---------table fetch by rowid
64
15
table fetch continued row
64
0
SQL> select * from dbc_TBL_XXX where site_id=100;
8941 rows selected.
Execution Plan
----------------------------------------------------------
11. -----------------------------------------------------------------------------------------| Id | Operation
| Name
| Rows | Bytes | Cost |
-----------------------------------------------------------------------------------------|
0 | SELECT STATEMENT
|
| 8944 |
550K|
172 |
|
1 | TABLE ACCESS BY INDEX ROWID| DBC_TBL_XXX| 8944 |
550K|
172 |
|* 2 |
INDEX RANGE SCAN
| IDX_DBC_TBL_XXX| 8944 |
|
3 |
-----------------------------------------------------------------------------------------Predicate Information (identified by operation id):
--------------------------------------------------2 - access("SITE_ID"=100)
Statistics
---------------------------------------------------------0 recursive calls
0 db block gets
2764 consistent gets
0 physical reads
0 redo size
495600 bytes sent via SQL*Net to client
7049 bytes received via SQL*Net from client
598 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
8941 rows processed
SQL>
2
3
4
select name,class,value
from v$sesstat a,v$statname b
where a.STATISTIC#=b.STATISTIC# and name like 'table fetch%'
and sid=12359;
NAME
CLASS
VALUE
----------------------------------- ---------- ---------table fetch by rowid
64
8961
table fetch continued row
64
0