Downloaded 241 times
![57
Parse
Builds a tree structure based upon the logical operators in the query.
For example:
SELECT
SSOD.[SalesOrderID],
PP.[Name],
PP.[Weight],
SSOD.[UnitPrice]
FROM [Sales].[SalesOrderDetail] SSOD
INNER JOIN [Production].[Product] PP
ON SSOD.ProductID = PP.ProductID
WHERE PP.Weight > 100
Project
Filter
Join
Product
Sales
Order
Detail
LogicalOperations
Nodes](https://image.slidesharecdn.com/sqlserverperformancetuning-150826164928-lva1-app6891/85/Sql-server-performance-tuning-57-320.jpg)
![58
Bind
• Series of validation steps
• Schema validation
• Table validation
• Attribute validation
• Permission validation
SELECT
SSOD.[SalesOrderID],
PP.[Name],
PP.[Weight],
SSOD.[UnitPrice]
FROM [Sales].[SalesOrderDetail] SSOD
INNER JOIN [Production].[Product] PP
ON SSOD.ProductID = PP.ProductID
WHERE PP.Weight > 100](https://image.slidesharecdn.com/sqlserverperformancetuning-150826164928-lva1-app6891/85/Sql-server-performance-tuning-58-320.jpg)
![62
Intro to execution plans – a simple example
• Execution Plan shows how SQL Server compiled & executes this
query
– Ctl-L in SSMS for “estimated” plan (without running query)
– Ctl-M displays “actual” plan after query has completed
• Read Execution Plans from top right to bottom left
– In this case, plan starts with Clustered Index Scan of [SalesOrderHeader]
– Then, for every row returned, performs an index seek into [Customers]
Plan node cost
shown as % of total
plan “cost”
Table / index access
methods displayed
(Scan, Seek etc)
Join physical
operators displayed
(Loops, Merge, Hash)
Thickness of intra node rows
denotes estimated / actual number
of rows carried between nodes in
the plan](https://image.slidesharecdn.com/sqlserverperformancetuning-150826164928-lva1-app6891/85/Sql-server-performance-tuning-62-320.jpg)
![74
What Is an Index?
‣ One of the best ways to reduce disk I/O is to use an index
‣ Allows SQL Server to find data in a table without scanning the
entire table
‣ Example
SELECT TOP 10 p.ProductID, p.[Name], p.StandardCost, p.
[Weight], ROW_NUMBER() OVER (ORDER BY p.Name DESC)
AS RowNumber FROM Production.Product p ORDER BY
p.Name DESC;](https://image.slidesharecdn.com/sqlserverperformancetuning-150826164928-lva1-app6891/85/Sql-server-performance-tuning-73-320.jpg)
More Related Content
Similar to Sql server performance tuning
Sql server performance tuning
- 1. 1 SQL Server PerformanceTuning (Tips & Tricks) By Nitin K niting123@gmail.com
- 2. 2 Agenda ‣ SQL ServerConcepts/Structure ‣ Performance Measuring & Troubleshooting Tools ‣ Locking ‣ Performance Problem : CPU ‣ Performance Problem : Memory ‣ Performance Problem : I/O ‣ Performance Problem : Blocking ‣ Query Tuning ‣ Indexing
- 3. 3 Who am I ‣12 + years of experience on Microsoft Technologies ‣ Data Architect on SQL Server Platform ‣ Working as a Datawarehouse Architect from last 5 years ‣ Playing a role of BI-DBA ‣ Certified MCSE- SQL Server 2012 Data track ‣ Passionate about performance tuning on Both database as well as Business Intelligence Area
- 4. 4 Performance Tuning inSQL Server ‣ Why is performance tuning necessary?
- 5. 5 Why is PerformanceTuning Necessary? ‣ Allowing your system to scale •Adding more customers •Adding more features ‣ Improve overall system performance ‣ Save money but not wasting resources ‣ The database is typically one of the most expensive resources in a datacenter
- 6. 6 Performance Focus ‣ ResponseTime Time interval when a request is submitted and when 1st character of response is received. ‣ Throughput Number of transaction processed in fixed unit of time ‣ Scalability Throughput and response time changes as we add more hardware resources
- 7. 7 ‣ Best Practices: Optimizefor real- world workloads Monitor/review performance regularly Focus on specific issues Performance Monitoring Approaches
- 8. 8 Overview of PerformanceMonitoring Tools and Methods
- 9. 9 Performance Tools: NoExtra cost ‣ SQL Server Profiler ‣ System Monitor (windows performance monitor) ‣ Dynamic Management Views (DMV) : SQL 2005+ ‣ Database tuning advisor (DTA) ‣ Microsoft Free tools (SQLDiag, PSSDiag, SQL Nexus – www.codeplex.com) ‣ MDW – Management Datawarehouse
- 10.
- 11. 11 Major Performance Killers ‣Insufficient indexing ‣ Inaccurate statistics ‣ Improper query design ‣ Poorly generated execution plans ‣ Excessive blocking and deadlocks ‣ Non-set-based operations, usually T-SQL cursors ‣ Inappropriate database design ‣ Excessive fragmentation ‣ Nonreusable execution plans ‣ Frequent recompilation of queries ‣ Improper use of cursors ‣ Improper configuration of the database transaction log ‣ Excessive use or improper configuration of tempdb
- 12. 12 Monitoring SQL Server Usingvarious SQL Server features to monitor database activity
- 13. 13 ‣ Overviews ofSQL Server usage Can export to Excel or PDF ‣ Server-Level Report Examples: Server Dashboard Memory Consumption Activity – All Block Transactions Activity – Top Sessions Performance – Batch Execution Statistics Performance – Top Queries by Average CPU Object Execution Statistics SQL Server Management Studio Reports
- 14. 14 ‣ Examples: Disk Usage AllTransactions All Blocking Transactions Index Usage Statistics Top Transactions by Age Schema Changes History ‣ New reports added in Service Packs Ability to use custom reports Database-Level Reports
- 15. 15 ‣ Windows EventLogs / Event Viewer Application and System Event Logs ‣ SQL Server Management Studio SQL Server Logs ‣ Can configure max. # of log files SQL Server Agent Error logs ‣ Can configure logging levels (Errors, Warnings, Information) ‣ Using the Log File Viewer Can Export / Load log information Can search for specific errors/messages Monitoring SQL Server Logs
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26. 26 Transaction Isolation Levels LevelDefinition Read Un-Committed Don’t need a lock to read a data Read Committed “Default” Read only committed data otherwise wait. Request ‘S’ lock to read. No guarantee that read is repeatable Repeatable read Guarantees that data read in a transaction will not change for it duration. Holds the ‘S’ lock until the duration of transaction. Serializable Prevent phantoms In any of above transaction, an ‘X’ lock acquired is held for total duration of transaction.
- 27. 27 Locks ‣ Lock types/modes X(exclusive), S (shared) , U(Update), IX (Intent Exclusive), IS….. imposed on DB (database), RID (row id), PAG(page), TAB(table) ‣ Locks Compatibility (matrix) ‣ Lock Hints Select * from employee with (Nolock) More - Rowlock, HoldLock, Tablock……. ‣ Locks Hierarchy Database (DB) ->Table (TB) -> Page (PG)-> Row (key) Lock modes Shared(S) Exclusive (X) Shared (S) – READ LOCKS OK NO Exclusive(X) NO NO
- 28. 28 What happens whenyou CRUD ‣ Two places get affected Memory: Data looked into Memory. If doesn’t exist, Page brought into Memory(RAM). Operation performed on Page in memory Transaction Log: Entry in transaction log file (.ndf file) ‣ No “instant” changes to mdf file (data file) ⇒Latest Changes are in Memory and Transaction Log file ⇒Backup transaction Log file in case of SQL failure ‣ Checkpoint : Lazywriter write to data changes in memory to disk
- 29.
- 30.
- 31. 31 Tools to Identifythe Performance Metric ‣ Performance counters ‣ Activity Monitor ‣ Task Manager ‣ SQL Server DMV’s
- 32. 32 Memory Performance Analysis ‣The basics of the Performance Monitor tool ‣ Some of the dynamic management objects used to observe system behavior ‣ How and why hardware resources can be bottlenecks ‣ Methods of observing and measuring memory use within SQL Server and Windows ‣ Possible resolutions to memory bottlenecks
- 33.
- 34.
- 35. 35 DMO’s and DMV’s ‣SELECT dopc.cntr_value, dopc.cntr_type FROM sys.dm_os_performance_counters AS dopc WHERE dopc.object_name = 'SQLServer:General Statistics' AND dopc.counter_name = 'Logins/sec';
- 36. 36 SQL Server MemoryManagement ‣ Max Memory ‣ Min Memory
- 37. 37 Additional Memory MonitoringTools ‣ DBCC MEMORYSTATUS ‣ Sys.dm_os_memory_brokers ‣ Sys.dm_os_memory_clerks ‣ Sys.dm_os_ring_buffers
- 38. 38 Resolution for CommonMemory Problems ‣ Optimizing application workload ‣ Allocating more memory to SQL Server ‣ Moving in-memory tables back to standard storage ‣ Increasing system memory ‣ Changing from a 32-bit to a 64-bit processor ‣ Enabling 3GB of process space ‣ Compressing data
- 39. 39 Disk Performance Analysis ‣Using system counters to gather disk performance metrics ‣ Using other mechanisms of gathering disk behavior ‣ Resolving disk performance issues
- 40.
- 41. 41 DMV’s and DMO’s ‣Sys.dm_io_virtual_file_stats ‣ Sys.dm_os_wait_stats ‣ Example SELECT * FROM sys.dm_os_wait_stats AS dows WHERE wait_type LIKE 'PAGEIOLATCH%'
- 42. 42 Resolution for commonDisk bottlenecks ‣ Optimizing application workload ‣ Using a faster I/O path ‣ Using a RAID array ‣ Using a SAN system ‣ Using Solid State Drives
- 43. 43 Resolution for commonDisk bottlenecks ‣ Aligning disks properly ‣ Using a battery-backed controller cache ‣ Adding system memory ‣ Creating multiple files and filegroups ‣ Moving the log files to a separate physical drive ‣ Using partitioned tables
- 44. 44 CPU Performance Analysis ‣How to gather metrics on the processor ‣ Additional metrics available through T-SQL queries ‣ Methods for resolving processor bottlenecks
- 45.
- 46. 46 DMV’s ‣ Sys.dm_os_wait_stats ‣ Sys.dm_os_workersand Sys.dm_os_schedulers
- 47. 47 Resolution for commonbottlenecks ‣ Optimizing application workload ‣ Eliminating or reducing excessive compiles/recompiles ‣ Using more or faster processors ‣ Not running unnecessary software
- 48.
- 49. 49 Costly Queries ‣ Identifycostly queries using - SQL Server DMV’s(sys.dm_exec_query_stats) - Extended events
- 50. 50 Query Tuning ‣ Detection Profiler ‣Look for Queries/Stored Proc with High reads, CPU, & Duration. These are candidates of tuning. ‣ Look for Stored proc that’s Recompiling (it’s an event) DMV’s ‣ Find Queries with missing indexes ‣ Find tables that are defragmented ‣ Find TempDB database bottlenecks
- 51. 51 Query Tuning cont. ‣Troubleshoot : Query Execution Plan Operator types ‣ Seek (Best and preferred) ‣ Scan (not preferred) ‣ Bookmark lookup (better than scan and mostly with non- clustered index) Join type ‣ Nested ‣ Merge ‣ Hash (Avoid) Graphical Execution Plan Icons : http://msdn2.microsoft.com/en-us/library/ms175913.aspx http://www.sql-server-performance.com/articles/per/select_indexes_p1.aspx
- 52.
- 53. 53 Why do weneed an optimizer? The Query Optimizer •T-SQL is a “What” not “how” language. •We write “logical” requests. •SQL Optimizer Engine converts logical requests into physical plans.
- 54. 54 The job ofthe SQL Optimizer is to find “the best plan possible”. The Query Optimizer X What is the goal of the Optimizer?
- 55. 55 Query optimization explainedsimply 1. Query submitted 2. Magic happens 3. Shedload of data returned
- 56. 56 Optimizer steps Query Optimization(in a bit more detail) Bind Execute Optimize Parse
- 57. 57 Parse Builds a treestructure based upon the logical operators in the query. For example: SELECT SSOD.[SalesOrderID], PP.[Name], PP.[Weight], SSOD.[UnitPrice] FROM [Sales].[SalesOrderDetail] SSOD INNER JOIN [Production].[Product] PP ON SSOD.ProductID = PP.ProductID WHERE PP.Weight > 100 Project Filter Join Product Sales Order Detail LogicalOperations Nodes
- 58. 58 Bind • Series ofvalidation steps • Schema validation • Table validation • Attribute validation • Permission validation SELECT SSOD.[SalesOrderID], PP.[Name], PP.[Weight], SSOD.[UnitPrice] FROM [Sales].[SalesOrderDetail] SSOD INNER JOIN [Production].[Product] PP ON SSOD.ProductID = PP.ProductID WHERE PP.Weight > 100
- 59. 59 Optimize Works though manyrules and heuristics. These Include: •Commutativity •Substitution rules •Exploration rules •Implementation rules
- 60. 60 SELECT prod_category, AVG(amount_sold) FROMo_sales s, o_products p WHERE p.prod_id = s.prod_id GROUP BY prod_category;
- 61. 61 SQL: set basedexpression / serial execution • SQL syntax based on “set based” expressions (no processing rules) • Query execution is serial – SQL Server “compiles” query into a series of sequential steps which are executed one after the other – Individual steps also have internal sequential processing • (eg table scans are processed one page after another & per row within page) Returns CustID, OrderID & OrderDate for orders > 1st Jan 2005 No processing rules included in SQL statement, just the “set” of data to be returned • Execution Plans Display these steps
- 62. 62 Intro to executionplans – a simple example • Execution Plan shows how SQL Server compiled & executes this query – Ctl-L in SSMS for “estimated” plan (without running query) – Ctl-M displays “actual” plan after query has completed • Read Execution Plans from top right to bottom left – In this case, plan starts with Clustered Index Scan of [SalesOrderHeader] – Then, for every row returned, performs an index seek into [Customers] Plan node cost shown as % of total plan “cost” Table / index access methods displayed (Scan, Seek etc) Join physical operators displayed (Loops, Merge, Hash) Thickness of intra node rows denotes estimated / actual number of rows carried between nodes in the plan
- 63. 63 Execution plan nodeproperties • Mouse over execution plan node reveals extra properties.. Search predicate. WHERE filter in this case, but can also be join filter Number of rows returned shown in Actual Execution Plan Name of Schema object accessed to physically process query – typically an index, but also possibly a heap structure Ordered / Unordered – displays whether scan operation follows page “chain” linked list (next / previous page # in page header) or follows Index Allocation Map (IAM) page
- 64. 64 “Heap” Table Storage •Query execution example: Select FName, Lname, PhNo from Customers where Lname = ‘Smith’ No b-tree with HEAPs, so no lookup method available unless other indexes are present. Only option is to scan heap No physical ordering of table rows (despite this display) Scan cannot complete just because a row is located. Because data is not ordered, scan must continue through to end of table (heap) • Table storage structure used when no clustered index on table – Rarely used as CIXs added to PKs by default – Oracle uses Heap storage by default (even with PKs) • No physical ordering of rows – Stored in order of insertion – New pages added to end of “heap” as needed • NO B-Tree index nodes (no “index”)
- 65.
- 66.
- 67.
- 68.
- 69. 70 Blocking and Non-blockingOperators • Operators / Iterators can be put in two categories: 1. Blocking 2. Non-blocking • Having a blocking operator in your plan means other operators further down the line are sitting idle. This will reduce the overall performance of your query • Some examples…
- 70. 71 Blocking example Blocking andNon-blocking operators • An example using the sort operator: Row 1 Row 2 Row 3 Row 4 Row 5 ? Sort Desc
- 71. 72 Hints can beplaced in SQL to force optimizer to follow our desired retrieval path rather then calculated by the optimizer Select /* +RULE */ From emp , dept Where….. Select statement instructs the optimizer to use the rule based optimizer rather than the cost based optimizer. Delete /*+RULE*/ . . . . . . . . Update /*+RULE*/ . . . . . . . . 72
- 72. 73 Index Tuning ‣ Whatan index is ‣ The benefits and overhead of an index ‣ General recommendations for index design ‣ Clustered and nonclustered index behavior and comparisons ‣ Recommendations for clustered and nonclustered indexes
- 73. 74 What Is anIndex? ‣ One of the best ways to reduce disk I/O is to use an index ‣ Allows SQL Server to find data in a table without scanning the entire table ‣ Example SELECT TOP 10 p.ProductID, p.[Name], p.StandardCost, p. [Weight], ROW_NUMBER() OVER (ORDER BY p.Name DESC) AS RowNumber FROM Production.Product p ORDER BY p.Name DESC;
- 74. 75 Types of Index ‣Clustered Index –Primary Key Default (but not necessary) –Data is stored at the leaf level –Data is ordered by the key ‣ Non-clustered Index –Uses cluster key or RID of a heap –INCLUDE stored at leaf ‣ And the rest – outside the scope of this session
- 75. 76 Index Rules ‣ ClusteredIndex Choose wisely. Only one per table possible Primary key by default is clustered. Evaluate default behaviour ‣ Non-Clustered Index More than one possible. Foreign keys are always good candidate for non-clustered index (because of joins) ‣ Evaluate ‘Included Columns’ in Indexing. Every NonClustered index contains Clustered Keys ‣ Choose Index Fill Factor wisely. ‣ Find out tables with large rowcount but no indexing. May be it needs index.
- 76. 77 Index design recommendations Examinethe WHERE clause and JOIN criteria columns. Use narrow indexes. Examine column uniqueness. Examine the column data type. Consider column order. Consider the type of index (clustered versus nonclustered).
- 77. 78 Lookups & Joins ‣Key ‣ RID
- 78. 79 Joins optimization ‣ Hashjoins ‣ Merge joins ‣ Nested loop joins
- 79. 80 Join Operators (intra-tableoperators) • Nested Loop Join – Original & only join operator until SQL Server 7.0 – “For Each Row…” type operator – Takes output from one plan node & executes another operation “for each” output row from that plan node • Merge Join – Scans both sides of join in parallel – Ideal for large range scans where joined columns are indexed • If joined columns aren’t indexed, requires expensive sort operation prior to Merge • Hash Join – “Hashes” values of join column/s from one side of join • Usually smaller side – “Probes” with the other side • Usually larger side – Hash is conceptually similar to building an index for every execution of a query • Hash buckets not shared between executions – Worst case join operator – Useful for large scale range scans which occur infrequently
- 80. 81 Hash Join ‣ Ahash join uses the two join inputs as a build input and a probe input. ‣ The build input is shown as the top input in the execution plan, and the probe input is shown as the bottom input. ‣ Usually the smaller of the two inputs serves as the build input because it's going to be stored on the system, so the optimizer attempts to minimize the memory used. ‣ The hash join performs its operation in two phases: the build phase and the probe phase.
- 81. 82 Hash Join –Example ‣ SELECT p.* FROM Production.Product p JOIN Production.ProductCategory pc ON p.ProductSubcategoryID = pc.ProductCategoryID;
- 82. 83 Merge Join ‣ Amerge join requires both join inputs to be sorted on the merge columns, as defined by the join criterion ‣ Since each join input is sorted, the merge join gets a row from each input and compares them for equality ‣ A matching row is produced if they are equal. This process is repeated until all rows are processed
- 83. 84 Merge Join -Example
- 84. 85 Nested Loop Join ‣A nested loop join uses one join input as the outer input table and the other as the inner input table ‣ The outer input table is shown as the top input in the execution plan, and the inner input table is shown as the bottom input table ‣ The inner loop, executed for each outer row, searches for matching rows in the inner input table ‣ Nested loop joins are highly effective if the outer input is quite small and the inner input is larger but indexed
- 85. 86 Nested Loop Join- Example
- 86.
- 87. 88 Statistics, Data Distribution,and Cardinality ‣ The role of statistics in query optimization ‣ The importance of statistics on columns with indexes ‣ The importance of statistics on non-indexed columns used in join and filter criteria ‣ Analysis of single-column and multicolumn statistics, including the computation of selectivity of a column for indexing ‣ Statistics maintenance ‣ Effective evaluation of statistics used in a query execution
- 88. 89 Statistics :Query Optimizer Thequery optimizer in SQL Server is cost-based. It includes: 1. Cost for using different resources (CPU and IO) 2. Total execution time It determines the cost by using: ‣ Cardinality: The total number of rows processed at each level of a query plan with the help of histograms , predicates and constraint ‣ Cost model of the algorithm: To perform various operations like sorting, searching, comparisons etc.
- 89. 90 Statistics Analysis ‣ Thequery optimizer uses statistics to create query plans that improve query performance ‣ A correct statistics will lead to high-quality query plan. ‣ The query optimizer determines when statistics might be out-of-date by counting the number of data modifications since the last statistics update and comparing the number of modifications to a threshold.
- 90. 91 Auto create statistics ‣Default setting of auto create statistics is ON. ‣ It creates when: ‣ Clustered and non clustered Index is created ‣ Select query is executed. ‣ Auto create and updates applies strictly to single-column statistics.
- 91. 92 Why query 2is performing better ‣ If we perform following operations on field of any table in query predicate: 1. Using any system function or user defined function 2. Scalar operation like addition, multiplication etc. 3. Type casting ‣ In this situation sql server query optimizer is not able to estimate correct cardinality using statistics.
- 92. 93 To improve cardinality ‣If possible, simplify expressions with constants in them. ‣ If possible, don't perform any operation on the any field of a table in WHERE Clause, ON Clause, HAVING Clause ‣ Don't use local variables in WHERE Clause, ON Clause, HAVING Clause. ‣ If there is any cross relationship among fields or there is a complex expression in a field in a query predicates, it is better to create a computed column and then create a non- clustered index on it.
- 93. 94 Statistics Tools andcommands ‣ Create Statistics ‣ Sp_Updatestats ‣ Sp_autostats ‣ Sp_helpstats ‣ DBCC Show_statistics
- 94. 95 Statistics Maintenance ‣ AutoCreate Statistics(DB level) ‣ Auto Update Statistics ‣ Manual Maintenance
- 95. 96 Index fragmentation ‣ Thecauses of index fragmentation, including an analysis of page splits caused by INSERT and UPDATE statements ‣ The overhead costs associated with fragmentation ‣ How to analyze the amount of fragmentation ‣ Techniques used to resolve fragmentation ‣ The significance of the fill factor in helping to control fragmentation
- 96. 97 Cause for Fragmentation ‣Fragmentation occurs when data is modified in a table. ‣ Page splits cause database fragmentation ‣ A new leaf page will then be added that contains part of the original page and maintains the logical order of the rows in the index key ‣ New leaf page maintains the logical order of the data rows in the original page, this new page usually won't be physically adjacent to the original page on the disk. ‣ The logical key order of the index doesn't match the physical order within the file
- 97. 98 Identify Fragmentation &Resolution ‣ Checking the fragmentation using sys.dm_db_index_physical_stats ‣ Only rebuild or reorganize indexes that are fragmented ‣ Rebuild heavily fragmented indexes ‣ Reorganize moderately fragmented indexes
- 98. 99 Reorganize Index ‣ Ifdatabase fragmentation is less than 10%, no action is required ‣ 20 – 30% requires you to reorganize indexes ‣ Use ALTER INDEX REORGANIZE USE AdventureWorks ALTER INDEX PK_ProductPhoto_ProductPhotoID ON Production.ProductPhoto REORGANIZE reorganize the PK_Product_Product-PhotoID index on the Production.ProductPhoto table
- 99. 100 Rebuild Index ‣ Morethan 30% fragmentation requires you to rebuild indexes ‣ There are two methods CREATE INDEX WITH DROP EXISTING ALTER INDEX REBUILD
- 100. 101 Significance of theFill Factor ‣ SQL Server allows you to control the amount of free space within the leaf pages of the index by using the fill factor ‣ If there will be enough INSERT queries on the table or UPDATE queries on the index key columns, then you can pre- add free space to the index leaf page using the fill factor to minimize page splits
- 101. 102 Query parsing Parsing Binding Query optimization Executionplan generation, caching, and hash plan generation Query execution
- 102.
- 103.
- 104. 105 Optimization Techniques ‣ Syntax-basedoptimization of the query ‣ Trivial plan match to avoid in-depth query optimization for simple queries ‣ Index and join strategies based on current distribution statistics ‣ Query optimization in stepped phases to control the cost of optimization ‣ Execution plan caching to avoid the regeneration of query plans
- 105.
- 106. 107 Query – ExecutionPlan Cache ‣ Saves the plans created in a memory space on the server called the plan cache. ‣ SELECT * FROM sys.dm_exec_cached_plans;
- 107.
- 108. 109 Plan Reusability ofan Ad Hoc Workload ‣ Optimize for an Ad Hoc Workload ‣ Simple Parameterization(default) ‣ Forced Parameterization
- 109. 110 Plan Reusability ofa Prepared Workload ‣ Stored Procedures ‣ sp_executesql ‣ Prepare/Execute Model
- 110. 111 Stored Procedure –why? ‣ Standard technique for improving the effectiveness of plan caching ‣ When the stored procedure is compiled the generated execution plan is cached for future reuse. This plan is used for future execution ‣ Performance Benefits - Reduced network traffic - Business logic is close to the data
- 111. 112 sp_executesql ‣ sp_executesql isa system stored procedure that provides a mechanism to submit one or more queries as a prepared workload ‣ It allows the variable parts of the query to be explicitly parameterized, and it can therefore provide execution plan reusability as effective as a stored procedure
- 112. 113 Continued ‣ DECLARE @queryNVARCHAR(MAX), @paramlist NVARCHAR(MAX); SET @query = N'SELECT soh.SalesOrderNumber ,soh.OrderDate ,sod.OrderQty ,sod.LineTotal FROM Sales.SalesOrderHeader AS soh JOIN Sales.SalesOrderDetail AS sod ON soh.SalesOrderID = sod.SalesOrderID WHERE soh.CustomerID = @CustomerID AND sod.ProductID = @ProductID'; SET @paramlist = N'@CustomerID INT, @ProductID INT'; EXEC sp_executesql @query,@paramlist,@CustomerID = 29690,@ProductID = 711;
- 113. 114 Prepare/Execute Model ‣ ODBCand OLEDB provide a prepare/execute model to submit queries as a prepared workload ‣ Like sp_executesql, this model allows the variable parts of the queries to be parameterized explicitly ‣ The prepare phase allows SQL Server to generate the execution plan for the query and return a handle of the execution plan to the application ‣ This execution plan handle is used by the execute phase to execute the query with different parameter values
- 114. 115 Query stats andQuery Hash ‣ With SQL Server 2008, new functionality around execution plans and the cache was introduced called the query plan hash and the query hash ‣ You can retrieve the query plan hash and the query hash from sys.dm_exec_query_stats
- 115. 116 Recommendations ‣ Explicitly parameterizevariable parts of a query. ‣ Use stored procedures to implement business functionality. ‣ Use sp_executesql to avoid stored procedure maintenance. ‣ Use the prepare/execute model to avoid resending a query string. ‣ Avoid ad hoc queries. ‣ Use sp_executesql over EXECUTE for dynamic queries. ‣ Parameterize variable parts of queries with care. ‣ Avoid modifying environment settings between connections. ‣ Avoid the implicit resolution of objects in queries
- 116. 117 Top 10 forBuilding Efficient Queries 1 Favor set-based logic over procedural or cursor logic • The most important factor to consider when tuning queries is how to properly express logic in a set-based manner. •Cursors or other procedural constructs limit the query optimizer’s ability to generate flexible query plans. •Cursors can therefore reduce the possibility of performance improvements in many situations
- 117. 118 Top 10 forBuilding Efficient Queries 2. Test query variations for performance •The query optimizer can often produce widely different plans for logically equivalent queries. •Test different techniques, such as joins or subqueries, to find out which perform better in various situations.
- 118. 119 Top 10 forBuilding Efficient Queries 3. Avoid query hints. •You must work with the SQL Server query optimizer, rather than against it, to create efficient queries. •Query hints tell the query optimizer how to behave and therefore override the optimizer’s ability to do its job properly. •If you eliminate the optimizer’s choices, you might limit yourself to a query plan that is less than ideal. •Use query hints only when you are absolutely certain that the query optimizer is incorrect. .
- 119. 120 Top 10 forBuilding Efficient Queries ‣ 4. Use correlated subqueries to improve performance. • Since the query optimizer is able to integrate subqueries into the main query flow in a variety of ways, subqueries might help in various query tuning situations. • Subqueries can be especially useful in situations in which you create a join to a table only to verify the existence of correlated rows. For better performance, replace these kinds of joins with correlated subqueries that make use of the EXISTS operator
- 120. 121 Top 10 forBuilding Efficient Queries ‣ 4. Continued .
- 121. 122 Top 10 forBuilding Efficient Queries 5. Avoid using a scalar user-defined function in the WHERE clause. ‣Scalar user-defined functions, unlike scalar subqueries, are not optimized into the main query plan. ‣Instead, you must call them row-by-row by using a hidden cursor. ‣This is especially troublesome in the WHERE clause because the function is called for every input row. ‣Using a scalar function in the SELECT list is much less problematic because the rows have already been filtered in the WHERE clause
- 122. 123 Top 10 forBuilding Efficient Queries ‣ 6. Use table-valued user-defined functions as derived tables. ‣ In contrast to scalar user-defined functions, table-valued functions are often helpful from a performance point of view when you use them as derived tables. ‣ The query processor evaluates a derived table only once per query. ‣ If you embed the logic in a table-valued user-defined function, you can encapsulate and reuse it for other queries
- 123. 124 Top 10 forBuilding Efficient Queries ‣ 6. Continued .
- 124. 125 Top 10 forBuilding Efficient Queries ‣ 7 Avoid unnecessary GROUP BY columns ‣ Use a subquery instead. ‣ •The process of grouping rows becomes more expensive as you add more columns to the GROUP BY list. ‣ •If your query has few column aggregations but many non- aggregated grouped columns, you might be able to refactor it by using a correlated scalar subquery. ‣ •This will result in less work for grouping in the query and therefore possibly better overall query performance.
- 125. 126 Top 10 forBuilding Efficient Queries ‣ 7 Continued .
- 126. 127 Top 10 forBuilding Efficient Queries ‣ 8 .Use CASE expressions to include variable logic in a query ‣ The CASE expression is one of the most powerful logic tools available to T-SQL programmers. ‣ •Using CASE, you can dynamically change column output on a row-by-row basis. ‣ •This enables your query to return only the data that is absolutely necessary and therefore reduces the I/O operations and network overhead that is required to assemble and send large result sets to clients.
- 127. 128 Top 10 forBuilding Efficient Queries ‣ 9 Divide joins into temporary tables when you query very large tables. ‣ The query optimizer’s main strategy is to find query plans that satisfy queries by using single operations. ‣ •Although this strategy works for most cases, it can fail for larger sets of data because the huge joins require so much I/O overhead. ‣ •In some cases, a better option is to reduce the working set by using temporary tables to materialize key parts of the query. You can then join the temporary tables to produce a final result.
- 128. 129 Stored Procedure Best Practices ‣•Avoid using “sp_” as name prefix ‣ •Avoid stored procedures that accept parameters for table names ‣ •Use the SET NOCOUNT ON option in stored procedures ‣ •Limit the use of temporary tables and table variables in stored procedures ‣ •If a stored procedure does multiple data modification operations, make sure to enlist them in a transaction. ‣ •When working with dynamic T-SQL, use sp_executesqlinstead of the EXEC statement
- 129. 130 Views Best Practices ‣ Useviews to abstract complex data structures ‣ •Use views to encapsulate aggregate queries ‣ •Use views to provide more user-friendly column names ‣ •Think of reusability when designing views ‣ •Avoid using the ORDER BY clause in views that contain a TOP 100 PERCENT clause. ‣ •Utilize indexes on views that include aggregate data
- 130. 131 Top 10 forBuilding Efficient Queries ‣ 10. Refactoring Cursors into Queries.. ‣ Rebuild logic as multiple queries ‣ •Rebuild logic as a user-defined function ‣ •Rebuild logic as a complex query with a case expression
Editor's Notes
- #5 Query performance tuning remains an important part of today's database applications. Yes, hardware performance is constantly improving. Upgrades to SQL Server—especially to the optimizer, which helps determine how a query is executed, and the query engine, which executes the query—lead to better performance all on their own. At the same time, SQL Server instances are being put on virtual machines, either locally or in hosted environments, where the hardware behavior is not guaranteed. Databases are going to platform as a service systems such as Amazon RDS and Windows Azure SQL Database. You still have to deal with fundamental database design and code generation. In short, query performance tuning remains a vital mechanism for improving the performance of your database management systems. The beauty of query performance tuning is that, in many cases, a small change to an index or a SQL query can result in a far more efficient application at a very low cost. In those cases, the increase in performance can be orders of magnitude better than that offered by an incrementally faster CPU or a slightly better optimizer.
- #8 How will you establish the Baseline? How will you identify the bottlenecks? Why once change at a time? How will you measure the Performance
- #22 DBCC Loginfo
- #27 --demo
- #28 Locking types/modes: ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/108297fa-35fc-4cbe-a1ac-369aabd8cf44.htm Locking & compatibility: ms-help://MS.SQLCC.v9/MS.SQLMobile.v3.en/SSMMain3/html/fb7c1c79-e392-444f-873f-888ad0556631.htm Locking Hints: ms-help://MS.SQLCC.v9/MS.SQLMobile.v3.en/SSMMain3/html/7657c311-2121-4c45-8a36-4a6579384f24.htm Show sp_lock demo and not been able to delete database when different connection has put shared lock
- #32 Select * from sys.configurations
- #37 Minimum(MB), also known as min server memory, works as a floor value for the memory pool. Once the memory pool reaches the same size as the floor value, SQL Server can continue committing pages in the memory pool, but it can't be shrunk to less than the floor value. Note that SQL Server does not start with the min server memory configuration value but commits memory dynamically, as needed. Maximum(MB), also known as max server memory, serves as a ceiling value to limit the maximum growth of the memory pool. These configuration settings take effect immediately and do not require a restart. In SQL Server 2014 the lowest maximum memory is 64MB for a 32-bit system and 128MB for a 64-bit system.
- #38 Sys.dm_os_memory_brokers While most of the memory within SQL Server is allocated to the buffer cache, there are a number of processes within SQL Server that also can, and will, consume memory. These processes expose their memory allocations through this DMO. You can use this to see what processes might be taking resources away from the buffer cache in the event you have other indications of a memory bottleneck. Sys.dm_os_memory_clerks A memory clerk is the process that allocates memory within SQL Server. Looking at what these processes are up to allows you to understand whether there are internal memory allocation issues going on within SQL Server that might rob the procedure cache of needed memory. If the Performance Monitor counter for Private Bytes is high, you can determine which parts of the system are being consumed through the DMV. If you have a database using in-memory OLTP storage, you can use sys.dm_db_xtp_table_memory_stats to look at the individual database objects. But if you want to look at the allocations of these objects across the entire instance, you'll need to use sys.dm_os_memory_clerks. Sys.dm_os_ring_buffers This DMV is not documented within Books Online, so it is subject to change or removal. It changed between SQL Server 2008R2 and SQL Server 2012. The queries I normally run against it still seem to work for SQL Server 2014, but you can't count on that. This DMV outputs as XML. You can usually read the output by eye, but you may need to implement XQuery to get really sophisticated reads from the ring buffers.
- #47 Wait statistics are a good way to understand whether there are bottlenecks on the system. You can't simply say something greater than x is a bad number, though. Faulty Processor
- #51 Demo : use profiler to identify slow running queries. Look for CPU, Reads column and Duration column DMV demo – missing Index, DMV
- #52 Demo :
- #60 Heuristics quote from WikiPedia - The objective of a heuristic is to produce quickly enough a solution that is good enough for solving the problem at hand. This solution may not be the best of all the actual solutions to this problem, or it may simply approximate the exact solution. But it is still valuable because finding it does not require a prohibitively long time. Commutativity – In mathematics Commutativity is where sums can be rewritten and still provide the same answer, 10+100=110 & 100+10=110. The optimizer knows that if you are asking for an inner join (that is a join where the same data must be found on either side) then it would be more efficient to join the large table to the smaller one. This would result in less work for the storage engine. Let’s take the figures 10 & 100 again Table A will be 100 records, table b will be 10. Join A to b would require 100 searches into table b. join b to a would require 10 searches into table a – quite a saving and still the same logical outcome. In other words this part helps to work out the most efficient join order for your query. It’s been some time since we have had to worry about the order of our joins, the optimizer now figures that out for us. Substitution rules & Exploration rules - Substitution & Exploration rules are rules that use heuristics or mathematic rules to generate new tree shapes, this may provide the opportunity for more optimization rules. Implementation rules - Implementation rules convert the logical trees into physical trees, at this point they can be optimized further and become the eventual execution plan.
- #67 Parameters should be considered for differentiating the good and bad execution plan
- #68 Index Unique Scan : Only one row will be returned by unique index Index Range Scan : =,<,>,LIKE (NON UNIQUE INDEX) Order BY clause has all the columns present in the index and order same as index
- #75 There is a index on ProductID column will index be used here?
- #77 Demo :
- #82 In the most commonly used form of hash join, the in-memory hash join, the entire build input is scanned or computed, and then a hash table is built in memory. Each row from the outer input is inserted into a hash bucket depending on the hash value computed for the hash key (the set of columns in the equality predicate). A hash is just a mathematical construct run against the values in question and used for comparison purposes.
- #83 Smaller input as build input because it's going to be stored on the system, so the optimizer attempts to minimize the memory used.
- #85 In situations where the data is ordered by an index, a merge join can be one of the fastest join operations, but if the data is not ordered and the optimizer still chooses to perform a merge join, then the data has to be ordered by an extra operation, a sort. This can make the merge join slower and more costly in terms of memory and I/O resources.
- #86 A loop join can be fast because it uses memory to take a small set of data and compare it quickly to a second set of data. A merge join similarly uses memory and a bit of tempdb to do its ordered comparisons. A hash join uses memory and tempdb to build out the hash tables for the join. Although a loop join can be faster at small data sets, it can slow down as the data sets get larger or there aren't indexes to support the retrieval of the data. That's why SQL Server has different join mechanisms.
- #105 SQL Server uses a cost-based optimization technique to determine the processing strategy of a query. The optimizer considers both the metadata of the database objects, such as unique constraints or index size, and the current distribution statistics of the columns referred to in the query when deciding which index and join strategies should be used.
- #107 Parallel Plan Optimization The optimizer considers various factors while evaluating the cost of processing a query using a parallel plan. Some of these factors are as follows: Number of CPUs available to SQL Server SQL Server edition Available memory Cost threshold for parallelism Type of query being executed Number of rows to be processed in a given stream Number of active concurrent connections