Sql Server Performance Tuning


Published on

This session is for you if you want to learn tips and techniques that are used to optimize database development with special emphasis on SQL Server 2005. If you write lot of stored procedures and want to learn the tools of a DBA, this is the session for you. If you are new to SQL Server development environment, you will learn how the various constructs compare to each other and better performance can be produced every time with a brief introduction to understanding Execution Plans.

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Designed for SMP Buffer cache avoid physical IO (tens of thousands of times faster) Disk Design for throughput and fault tolerance (reliability) Many small drives, not few big ones Raid 0 striping faster but not fault tolerant Raid 1 mirroring fault tolerant but not faster for writes (is for reads since can read from either) Multiple Raid 1 faster but if have a hot spot (all writes to one table) then all writes to one disk so no speed increase. (File groups help across a database and Yukon partitioning will address table hot spots) Raid 1 good for transaction logs since writes are sequential. Raid 5 = striping and parity (any one drive fails can get data from remaining 4) has a write performance penalty and a read enhancement Raid 10 = raid 1&0 (e.g configure stripes and then mirror using W2K) (Raid 01 configure mirrors and then stripe
  • Performance Monitor Counters Memory Page Reads/sec Page Writes/sec Page Input/sec Page Output/sec Network Interface Bytes Received/sec Bytes Sent/sec Bytes Total/sec Current Bandwidth Output Queue Length Objects All Paging File All Physical Disk – set ‘disk perf – y’ in DOS and reboot % Disk Read Time % Disk Write Time % Idle Time Avg Disk Bytes/Read Avg Disk Bytes/Transfer Avg Disk Bytes/Write Avg Disk Queue Length Current Disk Queue Length Process % Privileged Time % Processor Time % User Time Processor % Privileged Time % Processor Time % User Time Server Work Queues Active Threads Available Threads Queue Length Total Bytes/sec Total Operations/sec SQLServer:Access Methods Full Scans Page Splits/sec Table Lock Escalations/sec SQLServer:Cache Manager Cache Hit Ratio - _Total Cache Pages - _Total SQLServer:Databases Transactions/sec SQLServer:General Statistics Logins/sec Logouts/sec User Connections SQLServer:Locks Number of Deadlocks/sec
  • sys.sysprocesses (sysprocesses in SQL2K) provides up to the second data on CPU, IO PRIOR to query completion can be joined to DMVs via sql_handle to obtain executing query data SQL2k options DBCC INPUTBUFFER() fn_getsql() sys.dm_os_workers DMV provides further info from thread perspective
  • You can deal with capacity issues by tuning query workload, or increasing hardware, but tuning workload is most effective & cheaper Memory is the most significant infrastructure component to size correctly Unless utilization genuinely increases significantly or memory is actually reduced, memory problems are typically consequences of other problems. If query workload efficiency has degraded (increased reads), usually better to tune queries (source of problem) than simply add more memory. Requires problem query identification (Profiler, Trace, DMVs) Might not be “tunable” (eg vendor applications)
  • Decreases represent either: Inefficient query workload (new changes / optimization issues) Increased utilization
  • all cases listed above occur on a per-session basis, so many users can be causing each of the disk IO workloads concurrently all cases listed above are highly disk WRITE oriented in nature temp table & cursor population, resultset sorting & versioning all WRITE to disk often causes significantly higher random, concurrent disk activity than user databases hard drive disk heads can only be physically in one place at any point in time tempdb‘s random, concurrent, highly write intensive disk activity can generate enormous queued disk workloads
  • Testing / Live results Customer testing & live deployment of SDD on tempdb alone confirms significant improvement in system performance large-scale financial services online system 19,000% reduction in IO stalls in batch processing
  • It’s doing “include” fields for us. How many know what that is? What does this NOT show? The impact of inserts/updates/deletes by adding these indexes. If we went and added all of these indexes right now, would that help? How can we tell? Are there existing indexes we could tweak instead? We could test it by capturing a load test with Profiler, then making our changes on a dev box, replaying those traces with/without our changes. But that’s getting into an ACTIVE method of tuning. More on that later.
  • Putting the clustered index on the primary key of OLTP tables reduces pages splits. Don’t do this on earlier versions of SQL Server where row-level locking is unavailable or on SQL2K servers where row-level locking is disabled.
  • Any time you rebuild the clustered index, you also automatically rebuild all non-clustered indexes on the table.
  • Can set FILL FACTOR at a server level or specify with each clustered index. A good rule of thumb setting is 75-80% Should not set at the server level since some tables perform worse with fill factor of less than 100%. Naturally, this option strongly affects the amount of space that a table and its indexes will consume. However, disk is cheap!
  • Avoiding wildcards in the SELECT list (as in, SELECT * FROM foo ) offers several advantages: reduces on network activity since unneed columns are not to the client improves self-documentation of the code, since unacquainted coders can more easily discern the important data in the query alleviates the need for SQL Server to rebuild the query plan on procedures, views, triggers, functions or even frequently run queries any time the underlying table structure is changed 4. narrows the the query engine to the most pertinent index choices
  • Execution Plan Is Strategy determined by SQL Query Optimizer Can be influenced by developers Key Decisions are made: Which indexes to use? How to perform JOIN operations? How to order and group data? In what order tables should be processed? If cached data and previously compiled plans can be reused? High-percentage bookmark lookups are bad if your query is already running slow… Consider covering indices Thick lines coming into the operation and thin lines coming out – Bad and Ugly because it means that SQL Server is reading a lot of data initially (high disk I/O) but it’s throwing most of it away at the end because of the query clauses. Similar to reading 1GB file in memory and using only 100k of data. Database Developer’s role in Query Optimization Process Is: Apply iterative process of changing queries and database objects so that SQL Query Optimizer can make better decisions and do it faster Identify plan deficiencies and make changes to force SQL Server to correct them Correct one problem at a time Capture performance statistics “before” and “after” – use Database Load Testing tool To test our applications with production-like data load before rolling it out As amount of data increases, to proactively monitor application performance in order to catch problems early and correct them before customer sees them – work with DBA team SQL Server performs sort, intersect, union, and difference operations using in-memory sorting and hash join technology. Using this type of query plan, SQL Server supports vertical table partitioning, sometimes called columnar storage. SQL Server employs three types of join operations: Nested loops joins Merge joins Hash joins If one join input is quite small (such as fewer than 10 rows) and the other join input is fairly large and indexed on its join columns, index nested loops are the fastest join operation because they require the least I/O and the fewest comparisons. For more information about nested loops, see Understanding Nested Loops Joins. If the two join inputs are not small but are sorted on their join column (for example, if they were obtained by scanning sorted indexes), merge join is the fastest join operation. If both join inputs are large and the two inputs are of similar sizes, merge join with prior sorting and hash join offer similar performance. However, hash join operations are often much faster if the two input sizes differ significantly from each other. For more information, see Understanding Merge Joins. Hash joins can process large, unsorted, nonindexed inputs efficiently. They are useful for intermediate results in complex queries because: Intermediate results are not indexed (unless explicitly saved to disk and then indexed) and often are not produced suitably sorted for the next operation in the query plan. Query optimizers estimate only intermediate result sizes. Because estimates can be an order of magnitude wrong in complex queries, algorithms to process intermediate results not only must be efficient but also must degrade gracefully if an intermediate result turns out to be much larger than anticipated. The hash join allows reductions in the use of denormalization to occur. Denormalization is typically used to achieve better performance by reducing join operations, in spite of the dangers of redundancy, such as inconsistent updates. Hash joins reduce the need to denormalize. Hash joins allow vertical partitioning (representing groups of columns from a single table in separate files or indexes) to become a viable option for physical database design. For more information, see Understanding Hash Joins.
  • Not optimally written queries and flawed database schema Missing indexes Unnecessary joins Too much data returned (SELECT *) Unnecessary ORDER BY clauses Inadequate disk performance Disk I/O can’t keep up with needs of relational engine Disk fragmentation of db files (sometimes called external fragmentation) Index and data pages fragmentation within db files (sometimes called internal fragmentation) Memory pressure and low page life expectancy Data pages frequently accessed can not fit in SQL Server memory which causes more disk I/O Low cache/hit ratio and page life expectancy Long running queries (blocking and locking) Reports or massive batch inserts or updates Long running transactions Parsing is a step during which syntax of the statement is validated and clauses are converted into internal compiler structures. Execution tree is produced. Execution tree is a structure that describes the logical steps needed to transform the source data into the format required by the result set. Normalization is a step during which objects are verified and bound to, views are replaced with their definitions and implicit type conversions are performed (when column/variable types allow implicit conversions). Optimization is a most important step, during which execution plan (optimized, final version of execution tree) is formed. Execution plan is a detailed strategy of query execution – see next slides for details. Execution plans are reused and cached in memory. If SQL query engine finds a suitable execution plan that is already cached, it will use it. By the same token, “aged” execution plans are ejected from cache. After that, execution plan is cached is a specially allocated buffer called procedure cache. Please notice that percentage of memory allocated for procedure cache fluctuates depending on number of plans that need to be kept in memory. Therefore, having too many execution plans (common scenario when raw SQL statements are used) may cause SQL server to start ejecting data and index pages from cache, which is not good. After that, the relational engine begins executing the execution plan. As steps that need data from the base tables are processed, the relational engine uses OLE DB to request that the storage engine pass up data from the rowsets requested from the relational engine
  • Source: Inside SQL Server Book Phase 1: Trivial Plan Optimization. Cost-based optimization is expensive to do when there really is only one viable plan for the SQL statement. Example 1: a query that consists of an INSERT statement with a VALUES clause. There is only one possible plan. Example 2: SELECT statement where all the columns are in a unique covering index and there is no other index that has that set of columns in it. The trivial plan optimizer finds the really obvious plans, which are typically very inexpensive. As a result, optimizer doesn't spend a lot of time searching for a good plan. Phase 2: Syntactical Transformations Looking for commutative properties Looking for operations that can be rearranged Constant folding Other operations that don't require looking at the cost or analyzing indexes Phase 3: Full Optimization SQL Server then loads up the statistics information on indexes and columns, and enters the final major part of optimization, which is the cost based optimizer
  • How many indexes are there already? How many rows are in the table? Is it write-intensive? Do we have fast enough storage for our writes? Query to Identifying Missing Indexes SELECT statement AS [database.scheme.table], column_id , column_name, column_usage, migs.user_seeks, migs.user_scans, migs.last_user_seek, migs.avg_total_user_cost, migs.avg_user_impact FROM sys.dm_db_missing_index_details AS mid CROSS APPLY sys.dm_db_missing_index_columns (mid.index_handle) INNER JOIN sys.dm_db_missing_index_groups AS mig ON mig.index_handle = mid.index_handle INNER JOIN sys.dm_db_missing_index_group_stats  AS migs ON mig.index_group_handle=migs.group_handle ORDER BY mig.index_group_handle, mig.index_handle, column_id GO
  • The Index Tuning Wizard is invoked in SQL Enterprise Manager on the Tools menu >> Wizards >> Management >> Index Tuning Wizard. We feed it a load – could be a single query, could be a trace file It actively runs these queries against our server, and it changes schema objects while it works to figure out what indexes and statistics will be the fastest. The Index Tuning Wizard can consume significant CPU resources on the server where it is run so you might want to: A) avoid running it on production servers, B) run it on a separate computer, C) run it on small subsets of the tables in the database, and/or D) disable the Perform thorough analysis option. Rename Each Index If we follow its recommendations and just hit apply, this is what we get – a bunch of new indexes we can’t identify. CREATE NONCLUSTERED INDEX [_dta_index_Activity_11_1977058079__K1_K4_K7_K5_K3] ON [dbo].[Activity] ( [ServerName] ASC, [ActivityTypeID] ASC, [StatusTypeID] ASC, [StartTime] ASC, [DatabaseID] ASC )
  • DMF & DMV New with SQL Server 2005 Gathers information continuously Data does disappear with restarts You can walk in and start tuning immediately with little preparation Now, when we get a list of index recommendations from the wizard, we can compare it against our schema to see what we’ve got, and what we need to add. -- Possible bad Indexes (writes > reads) DECLARE @dbid int SELECT @dbid = db_id() SELECT 'Table Name' = object_name(s.object_id), 'Index Name' =i.name, i.index_id, 'Total Writes' = user_updates, 'Total Reads' = user_seeks + user_scans + user_lookups, 'Difference' = user_updates - (user_seeks + user_scans + user_lookups) FROM sys.dm_db_index_usage_stats AS s INNER JOIN sys.indexes AS i ON s.object_id = i.object_id AND i.index_id = s.index_id WHERE objectproperty(s.object_id,'IsUserTable') = 1 AND s.database_id = @dbid AND user_updates > (user_seeks + user_scans + user_lookups) ORDER BY 'Difference' DESC, 'Total Writes' DESC, 'Total Reads' ASC
  • You can pick which individual indexes you want to compress. The smaller they are, the faster they’re read off disk. How much faster? Well, SQL gives us a tool to find that out.
  • Table fragmentation is similar to hard disk fragmentation caused by frequent file creation, deletion and modification. Database tables and indexes need occasional defragmentation to stay efficient. The most efficient allocation for read-heavy tables is when all pages occupy a contiguous area in the database, but after weeks of use, a table may become scattered across the disk drive. The more pieces it is broken into – the less efficient the table becomes. T-SQL Code As Microsoft SQL Server 2000 maintains indexes to reflect updates to their underlying tables, these indexes can become fragmented. Depending on workload characteristics, this fragmentation can adversely affect workload performance. This white paper provides information to help you determine whether you should defragment table indexes to benefit workload performance. To defragment indexes, SQL Server 2000 provides several statements. This white paper compares two of those statements: DBCC DBREINDEX and DBCC INDEXDEFRAG. /* Determine which indexes to defrag using our user-defined parameters */ INSERT INTO #indexDefragList SELECT database_id AS databaseID , QUOTENAME(DB_NAME(database_id)) AS 'databaseName' , [OBJECT_ID] AS objectID , index_id AS indexID , partition_number AS partitionNumber , avg_fragmentation_in_percent AS fragmentation , page_count , 0 AS 'defragStatus' /* 0 = unprocessed, 1 = processed */ , Null AS 'schemaName' , Null AS 'objectName' , Null AS 'indexName' FROM sys.dm_db_index_physical_stats (@databaseID, OBJECT_ID(@tableName), Null , Null, @scanMode) WHERE avg_fragmentation_in_percent >= @minFragmentation And index_id > 0 -- ignore heaps And page_count > 8 -- ignore objects with less than 1 extent OPTION (MaxDop 1); /* Grab the most fragmented index first to defrag */ SELECT TOP 1 @objectID = objectID , @indexID = indexID , @databaseID = databaseID , @databaseName = databaseName , @fragmentation = fragmentation , @partitionNumber = partitionNumber , @pageCount = page_count FROM #indexDefragList WHERE defragStatus = 0 ORDER BY fragmentation DESC; /* If the index is heavily fragmented and doesn't contain any partitions or LOB's, rebuild it */ IF @fragmentation >= @rebuildThreshold And IsNull(@containsLOB, 0) != 1 And @partitionCount <= 1 BEGIN /* Set online rebuild options; requires Enterprise Edition */ IF @onlineRebuild = 1 And @editionCheck = 1 SET @rebuildCommand = N' Rebuild With (Online = On'; ELSE SET @rebuildCommand = N' Rebuild With (Online = Off'; /* Set processor restriction options; requires Enterprise Edition */ IF @maxDopRestriction IS Not Null And @editionCheck = 1 SET @rebuildCommand = @rebuildCommand + N', MaxDop = ' + CAST(@maxDopRestriction AS VARCHAR(2)) + N')'; ELSE SET @rebuildCommand = @rebuildCommand + N')'; SET @sqlCommand = N'Alter Index ' + @indexName + N' On ' + @databaseName + N'.' + @schemaName + N'.' + @objectName + @rebuildCommand; END; EXECUTE SP_EXECUTESQL @sqlCommand; http://technet.microsoft.com/en-us/library/cc966523.aspx http://sqlserverpedia.com/wiki/Index_Maintenance
  • Object names can include table name, table id, view name or view id (where an index exists on the view), and/or an optional index name or index ID. The WITH option allows you to control how much data comes back. - FAST skips a leaf (data) level read and returns minimal information. These columns: Pages Scanned, Extent Switches, Scan Density [Best Count:Actual Count], Logical Scan Fragmentation. - TABLERESULTS returns the data in table format. You could then store it in a temp table if you wanted. It also returns a few additional columns: ExtentSwitches, AverageFreeBytes, AveragePageDensity, ScanDensity, BestCount, ActualCount, LogicalFragmentation, ExtentFragmentation. - ALL_INDEXES returns data for all indexes on a table, even when an individual index is identified - ALL_LEVELS only usable with TABLERESULTS (and not with FAST), produces results for each level of the index processed. Otherwise, only the index leaf level or table data level are processed. There are several especially important points to check here. Pages Scanned : Number of database pages used by the table (when you specify indid of 1 or 0) or a non-clustered index (when you specify indid > 1). Extent Switches : All pages of a table or an index are linked into a chain. Access to the table or index is more efficient when all pages of each extent are linked together into a segment of this chain. DBCC command scans the chain of pages and counts the number of times it has to switch between extents. If the number of extent switches exceeds the number of pages divided by 8, then there is a room for optimization. Extents switched compared to extents scanned gives us the scan density value. Avg. Pages per Extent : Space for each table is reserved in extents of 8 pages. Some pages are unused, because the table has never grown to use them or because rows have been deleted from a page. The closer this number is to 8 – the better. A lower number indicates that there is a lot of unused pages that decrease performance of table access. Scan Density [Best Count: Actual Count] : Scan Density shows how contiguous the table is. The closer the number is to 100% – the better. Anything less than 100% indicate fragmentation. Best Count shows the ideal number of extent switches that could be achieved on this table. Actual Count shows the actual number of extent switches. Logical Scan Fragmentation : The Percentage of out-of-order pages returned from scanning the leaf pages of an index. This reading is not relevant for heaps (tables without indexes of any kind) and text indexes. A page is considered out of order when the next page in the Index Allocation Map (IAM) is different than the page indicated by the next page pointer in the leaf page. Extent Scan Fragmentation : Percentage of out-of-order extents in scanning the leaf pages of an index, excluding heaps. An extent is considered out-of-order when the extent containing the current index page is not physically next after the extent holding the previous index page. Logical and Extent scan fragmentation should be as low as possible. Extent scan will usually be higher. Logical scan of 0% to 10% is usually acceptible. Avg. Bytes free per page : The average number of free bytes per page used by the table or index. The lower the number – the better. High numbers indicate inefficient space usage . The highest possible number of free space is 2014 (on SQL 7.0) – the size of a database page minus overhead. This or a close number will be displayed for empty tables. For tables with large rows this number may be relatively high even after optimization. For example, if row size is 1005 bytes, then only one row will fit per page. DBCC will report average free space also as 1005 bytes, but don’t expect another row to fit into the same page. In order to fit a row of 1005 bytes you’d also need additional room for row system overhead. Avg. Page density (full) : How full is an average page. Numbers close to 100% are better. This number is tied to the previous one and depends on the row size as well as on clustered index fillfactor. Transactions performed on table rows change this number because they delete, insert or move rows around by updating keys. SQL BOL has a good way to defragment all indexes in a database that is fragmented above a declared threshold under the topic DBCC SHOWCONTIG.
  • When defining which database, table, view, or index you would like to defragment, you may use either the name of the object or its object ID. (When using a zero instead of the database name or database ID, the current database is assumed.) For example: DBCC INDEXDEFRAG (Pubs, Authors, Aunmind) GO Results: Pages Scanned Pages Moved Pages Removed ------------- ----------- ------------- 359 346 8
  • If either index_name or fillfactor is specified, all preceding parameters must also be specified.
  • Create another pagefile.sys files on every separate physical drives (Except drive contains the Windows NT system directory). Spreading paging files across multiple disk drives and controllers improves performance on most disk systems because multiple disks can process input/output requests concurrently. If you have a lot of RAM, you can configure your Windows NT server to never page out drivers and system code to the pagefile that are in the pageable memory area. Run regedit and choose: HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlSession ManagerMemory Management Set DisablePagingExecutive to 1 and reboot the server box. Set the "Maximize Throughput for Network Applications" option. This can increase SQL Server performance, because Windows NT will allocate more RAM to SQL Server than to its file cache. To set this option, you can do the following: 1. Double-click the Network icon in Control Panel. 2. Click the Services tab. 3. Click Server to select it, and then click the Properties button. 4. Click Maximize Throughput for Network Applications, and then click OK. 5. Restart the computer. Allow the tempdb database to automatically expand as needed. This ensures that queries that generate larger than expected intermediate result sets stored in the tempdb database are not terminated before execution is complete. Set the original size of the tempdb database files to a reasonable size to avoid the files from automatically expanding as more space is needed. If the tempdb database expands too frequently, performance can be affected. Set the file growth increment percentage to a reasonable size to avoid the tempdb database files from growing by too small a value. If the file growth is too small compared to the amount of data being written to the tempdb database, then tempdb may need to constantly expand, thereby affecting performance. Place the tempdb database on a fast I/O subsystem to ensure good performance. Stripe the tempdb database across multiple disks for better performance. Place the tempdb database on disks different from those used by user databases. To optimize system memory use for SQL Server, you should limit the amount of memory that is used by the system for file caching. To limit the file system cache, make sure that Maximize data throughput for file sharing is not selected. You can specify the smallest file system cache by selecting Minimize memory used or Balance . To change the server optimization settings in Windows Server 2003 In Control Panel , choose Network Connections , and then click Local Area Connection . On the General tab of the Local Area Connection Status dialog box, click Properties . On the General tab of the Local Area Connection Properties dialog box, select File and Printer Sharing for Microsoft Networks , and click Properties . In the File and Printer Sharing for Microsoft Networks dialog box, choose one of the following options: Maximize data throughput for file sharing Maximize data throughput for network applications (the option that SQL Server automatically sets) While the default setting of 1.5 x RAM is sufficient for pagefile size for some server roles, if your machine is running disk- and memory-intensive applications like Exchange or SQL Server then you may want to increase the initial pagefile size to 2 x RAM or even larger. Microsoft doesn't recommend increasing the pagefile to more than 3 x RAM, however, and the maximum allowed size for any one instance of pagefile.sys on a machine is limited to 4GB. So, if you have a powerful machine with 4GB of RAM then you have to split the pagefile to overcome this limit. This splitting can be done in two ways: create separate pagefiles on different volumes (done in the usual way) or create multiple pagefiles on a single volume. (See KB 237740 for how to do this.) On a more modest machine with 1GB of RAM though, you would probably be best served to set the initial pagefile size to 2GB (2 x RAM) and the maximum pagefile size to 3GB (3 x RAM).
  • This tasks are superior to the Database Maintenance wizard. But, at least, do the DBMaint wizard if nothing else. http://www.devarticles.com/c/a/SQL-Server/How-to-Perform-a-SQL-Server-Performance-Audit/
  • 1 0 Most Frequent Top-Level SQL Queries 1 0 Most Frequent Stored Procedures Average Execution Time by Stored Procedure Average Execution Time by SQL Query Long Running SQL Queries CPU Utilization by Application CPU Utilization by Login SQL Agent Jobs CPU Utilization CPU Utilization Bottlenecks and Timing CPU Utilization Trends Hourly Average CPU Utilization Daily Average CPU Utilization I/O Utilization by Application I/O Utilization by Login I/O Utilization Trends Deadlocks Lock Timeouts Active Sessions Trends Applications Running in SQL Server Cumulative SQL Server Load Tables Experiencing Full-Table Scans Backup Duration Trends Latest Backup Timing and Overlap Overlapping SQL Agent Jobs Database Errors and Exceptions
  • SQL Queries Whose Execution Time Has Changed Over 1 0% Stored Procedures Whose Execution Time Has Changed Over 1 0% Stored Procedures Execution Time Trends Stored Procedures Cache Hits and Misses (Details) Stored Procedures Cache Hits and Misses (Summary) Data Output Trends Free/Used Data Space Usage Trends Free/Used Log Space Usage Trends Data Space Allocation Trends - Comparison by Database Log Space Allocation Trends - Comparison by Database Row Count Trends by Database Top 1 0 Fastest Growing Tables by Space Top 1 0 Fastest Growing Tables by Row Count
  • Educated Guess Users notify Help Desk of system issues Help desk scrambles IT to find the problem IT frantically searches for the problem Network, Windows, SQL Server, front end application, logs, etc Unable to find issue  report to Help Desk User escalation to management IT monitor for symptoms and make changes to benefit the users, but cannot validate Problem = Lack of information Hunt and Peck Approach Ask users where problems exist Monitor SQL Server to capture data Review data to determine issues Change SQL Server based on data Re-design, code changes, indexing, etc. Monitor to determine improvement Problem = Information Overload 24x7 Performance Monitoring Install, configure and baseline Review data from integrated tools Current and historical view of system Proactively and intuitively review Focus IT Team on objects requiring the most resources Correct and validate improvement
  • Performance Studio concepts (data collection, management data warehouse) How to monitor/troubleshoot performance issues using Performance Studio New performance monitoring features in SQL Server® 2008 A framework that ties together collection, analysis, troubleshooting, and persistence of SQL Server diagnostics information. It consists of a suite of tools for: Low overhead data collection Performance monitoring, troubleshooting, tuning Persistence of diagnostics data Reporting Short term goals: Provide enhanced data collection and reports out of the box In many cases, when a problem occurs, you get a call later that same day or even the next day, saying, “There is a problem, we don’t know what’s happening, but could you please look into it?” To correctly fix the issue, you need the ability to centrally store performance data to go back in time to see exactly what happened during that period of time and, hopefully, you’ll be able to figure out what the problem was. With Performance Studio, you could use the performance data to analyze and write policies to prevent future issues to your system. For example, a policy if the CPU utilization goes over 85 percent for more than 15 minutes, then take this action or enable a specific type of data collection. But, you want to be able to apply more general policies to your system. Server Side: Data Collector Extensible data collection infrastructure Includes out of the box data collections required to identify and troubleshoot the most common problems for the relational engine Support for SQL Server relational engine only, but other SQL Server services can be added in the future Server Side: Management Data Warehouse Data repository for baseline and historical comparisons Aggregated reporting for multiple SQL Server instances Client Side: Data collection configuration UI Management data warehouse properties General data collection properties Collection set configuration SQL Server dashboard based on system collection sets reports Performance monitoring and troubleshooting Historical data analysis based on warehouse information Data Collector Concepts data collection should be always on and have low overhead. Overhead is a very tricky question because for some people, anything above zero is overhead. For some other people, 5 percent is OK. The overhead level is up to you. Out of the box, a lot of testing has been done—running against TPC-C and all sorts of other benchmarks—to ensure that the basic overhead on our system collection sets is always below 5 percent. But it’s really up to the user. It’s disabled by default, so you have to enable it and run the collection sets if you want data collection on. Data Provider Source of information (for example, SQL Trace, Perform counters, DMVs, T-SQL queries, logs)
  • http://www.devarticles.com/c/a/SQL-Server/How-to-Perform-a-SQL-Server-Performance-Audit/
  • Sql Server Performance Tuning

    1. 1. SQL Server performance monitoring and tuning Bala Subra [email_address]
    2. 2. Make SQL Server faster <ul><li>Methodology </li></ul><ul><ul><li>Look at SQL Server from a holistic standpoint. </li></ul></ul><ul><ul><li>How to baseline a SQL Server system. </li></ul></ul><ul><ul><li>How to track it from a “landscape perspective.” </li></ul></ul><ul><li>Evaluate the system now and going forward </li></ul>
    3. 3. Performance Tuning Challenges <ul><li>Difficult to create a baseline and compare over time to current status </li></ul><ul><li>Exorbitant amount of data to manage and analyze = ‘analysis paralysis’ </li></ul><ul><li>Multiple tools to manage the tiers and they are not compatible </li></ul><ul><li>Difficult to distinguish the issue at an object level or tier = ‘finger pointing’ </li></ul>
    4. 4. Challenges - Continued <ul><li>Cannot further impact the performance of the production system </li></ul><ul><li>Understand production, not simulation </li></ul><ul><li>Need for real time and historical view of performance data in a single system </li></ul><ul><li>Record of changes over time and subsequent impact once implemented </li></ul><ul><li>Throw hardware at the problem as a quick fix </li></ul>
    5. 5. Phases of performance tuning <ul><li>Define components </li></ul><ul><li>Evaluate objects </li></ul><ul><li>Interpret findings </li></ul><ul><li>Create an action plan </li></ul>
    6. 6. Performance tracking <ul><li>Use tracking tool of your choice </li></ul><ul><ul><li>Word </li></ul></ul><ul><ul><li>Excel </li></ul></ul><ul><ul><li>Database </li></ul></ul><ul><li>Methodology works on any platform </li></ul>
    7. 7. Define components <ul><li>A holistic view of the landscape </li></ul><ul><ul><li>Path determination </li></ul></ul><ul><ul><li>Systems </li></ul></ul><ul><ul><li>Software </li></ul></ul><ul><ul><li>Hardware </li></ul></ul>
    8. 8. The landscape <ul><li>“Literally everything” </li></ul><ul><ul><li>Server itself </li></ul></ul><ul><ul><li>Clustering components, if clustered </li></ul></ul><ul><ul><li>Networking, cards and driver levels </li></ul></ul><ul><ul><li>Routers and switches </li></ul></ul><ul><ul><li>Client workstations </li></ul></ul><ul><ul><li>Etc. </li></ul></ul><ul><li>An entire representation of your environment </li></ul>
    9. 9. Define components <ul><li>A holistic view of the landscape </li></ul><ul><ul><li>Path determination </li></ul></ul><ul><ul><li>Systems </li></ul></ul><ul><ul><li>Software </li></ul></ul><ul><ul><li>Hardware </li></ul></ul>
    10. 10. The path <ul><li>Determine how data gets from a fairly unique client machine to the server. </li></ul><ul><li>Diagram the path: </li></ul><ul><ul><li>Paint </li></ul></ul><ul><ul><li>PowerPoint </li></ul></ul><ul><ul><li>Visio </li></ul></ul><ul><ul><li>Network tools </li></ul></ul><ul><li>Determine areas of slowdown. </li></ul>
    11. 11. Define components <ul><li>A holistic view of the landscape </li></ul><ul><ul><li>Path determination </li></ul></ul><ul><ul><li>Systems </li></ul></ul><ul><ul><li>Software </li></ul></ul><ul><ul><li>Hardware </li></ul></ul>
    12. 12. The system <ul><li>Document the architecture </li></ul><ul><ul><li>Two tier – client and a server </li></ul></ul><ul><ul><li>Three tier – client, middle layer and a server </li></ul></ul><ul><ul><li>N tier – multiple systems </li></ul></ul><ul><ul><li>SOA – lots of moving parts </li></ul></ul><ul><ul><li>SQL Server Edition </li></ul></ul><ul><ul><ul><li>Memory 32TB? - Enterprise Edition (64-bit) </li></ul></ul></ul><ul><ul><ul><li>32 CPUs </li></ul></ul></ul><ul><ul><ul><li>Clustering </li></ul></ul></ul>
    13. 13. Define components <ul><li>A holistic view of the landscape </li></ul><ul><ul><li>Path determination </li></ul></ul><ul><ul><li>Systems </li></ul></ul><ul><ul><li>Software </li></ul></ul><ul><ul><li>Hardware </li></ul></ul>
    14. 14. The software <ul><li>Document software drivers, interfaces and code </li></ul><ul><ul><li>Only concerned with representative systems. </li></ul></ul><ul><ul><li>Avoid making immediate changes; if you change the test, you can’t determine the exact issue. </li></ul></ul><ul><ul><li>Do take care of security issues. </li></ul></ul><ul><ul><li>Graphical representation of your system </li></ul></ul>
    15. 15. Define components <ul><li>A holistic view – the landscape </li></ul><ul><ul><li>Path determination </li></ul></ul><ul><ul><li>Systems </li></ul></ul><ul><ul><li>Software </li></ul></ul><ul><ul><li>Hardware </li></ul></ul>
    16. 16. The hardware <ul><li>Document hardware </li></ul><ul><ul><li>Networking </li></ul></ul><ul><ul><li>Memory </li></ul></ul><ul><ul><li>Input/Output </li></ul></ul><ul><ul><ul><li>hard drives </li></ul></ul></ul><ul><ul><ul><li>storage area networks (SANs) </li></ul></ul></ul><ul><ul><ul><ul><li>Health </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Activity </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Performance impact </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Communication channels </li></ul></ul></ul></ul><ul><ul><ul><li>network-attached storage (NAS) devices </li></ul></ul></ul><ul><ul><ul><li>Network Monitoring </li></ul></ul></ul><ul><ul><ul><li>VMWare (Virtualization) </li></ul></ul></ul>
    17. 17. Evaluate objects <ul><li>Tools </li></ul><ul><li>Working with a baseline </li></ul><ul><li>Working without a baseline </li></ul><ul><li>Don’t fix anything yet! </li></ul>
    18. 18. Tools <ul><li>Enterprise Manager </li></ul><ul><ul><li>Primary tool to manage all SQL Server functionality across the enterprise </li></ul></ul><ul><ul><li>Features – Ability to manage database processes, locking, indexes, configurations, etc. </li></ul></ul><ul><li>Performance Monitor </li></ul><ul><ul><li>Capture a macro view of the servers with the ability to configure counters with specific sample rates save to a log file or real time monitor </li></ul></ul><ul><ul><li>Counters </li></ul></ul><ul><ul><ul><li>Memory, Processors </li></ul></ul></ul><ul><ul><ul><li>SQL Server </li></ul></ul></ul><ul><ul><ul><li>Network Activity, Disk Drives </li></ul></ul></ul><ul><ul><ul><li>System Statistics (Threads, Context Switching, Queuing, etc.) </li></ul></ul></ul><ul><li>SQL Server Profiler </li></ul><ul><ul><li>Micro view of all SQL Server transactions saved to a log file or database table </li></ul></ul><ul><ul><li>Filters – Ability to capture a subset of the transactions based on the transaction type, user, application, etc. </li></ul></ul><ul><ul><li>Concerns – High system overhead </li></ul></ul><ul><li>Query Analyzer – Query Plan </li></ul><ul><ul><li>Ability to graphically review the query execution plan </li></ul></ul>
    19. 19. SQL Server Query Processing basics Data Cache Proc Cache MTL 256Mb Select * from authors where au_lname = ‘White’ au_id au_lname au_fname phone address city state 172-32-1176 White Johnson 408-496-7223 10932 Bigge Rd. Oakland CA Pages read from disk - slow Pages read from cache –fast! Lookup Pages In Data Cache Lookup Exec Plan in Proc Cache Execution Plan Found? Yes? Execute.. No? Compile & Execute.. SQL Server stores table rows & columns (Authors Table in pubs db has ~26rows and is approx 6kb total size) Table rows are stored on Disk in 8kb units, named “ pages”. When loaded into memory pages are referred to as “ buffers” Data volume (HDD) Physical Memory (RAM) Buffer Manager
    20. 20. SQL Profiler Event Data <ul><li>Cursors </li></ul><ul><ul><li>CursorOpen </li></ul></ul><ul><ul><li>CursorExecute </li></ul></ul><ul><ul><li>CursorClose </li></ul></ul><ul><li>Errors and Warnings </li></ul><ul><ul><li>Hash Warning </li></ul></ul><ul><ul><li>Missing Column Statistics </li></ul></ul><ul><li>Locks </li></ul><ul><ul><li>Lock:Deadlock </li></ul></ul><ul><ul><li>Lock:Timeout </li></ul></ul><ul><li>TSQL </li></ul><ul><ul><li>Unprepare SQL </li></ul></ul><ul><li>Parallelism </li></ul><ul><ul><li>Degree of Parallelism – All Counters </li></ul></ul><ul><ul><li>Execution Plan </li></ul></ul><ul><ul><li>Show Plan All </li></ul></ul><ul><ul><li>Show Plan Statistics </li></ul></ul><ul><ul><li>Show Plan Text </li></ul></ul><ul><li>Stored Procedure </li></ul><ul><ul><li>SP:Starting </li></ul></ul><ul><ul><li>SP:Completed </li></ul></ul><ul><ul><li>SP:Recompile </li></ul></ul><ul><ul><li>SP:StmtCompleted </li></ul></ul><ul><ul><li>SP:StmtStarting </li></ul></ul>
    21. 21. Identifying Query Bottlenecks <ul><li>SQL Server Profiler </li></ul><ul><ul><li>Collect RPC:SPCompleted / TSQL:BatchCompleted events </li></ul></ul><ul><ul><ul><li>Filter with Reads > 10,000 at first, reduce to 1,000 </li></ul></ul></ul><ul><ul><li># of reads = # of pages “read” from cache (disk if not cached) </li></ul></ul><ul><ul><ul><li>CPU, Duration, Writes & RowCount also interesting, but reads is the best representation of source workload </li></ul></ul></ul><ul><ul><li>Relies on queries completing </li></ul></ul><ul><ul><ul><li>On a swamped server, queries might be piling up without completing, therefore not showing up in Profiler as completed events as fast as they are starting. </li></ul></ul></ul><ul><li>SQL Trace </li></ul><ul><ul><li>Same as Profiler, but runs in background </li></ul></ul><ul><ul><ul><li>Far lower performance impact that Profiler GUI </li></ul></ul></ul><ul><ul><li>Requires post analysis of .trc log files collected </li></ul></ul><ul><ul><ul><li>3rd Party Tools – SQLBenchmarkPro (continuous) / Cleartrace (ad-hoc) </li></ul></ul></ul><ul><ul><li>Can be scripted from GUI Profiler </li></ul></ul>
    22. 22. Identifying Query Bottlenecks (cont..) <ul><li>DMVs </li></ul><ul><ul><li>Gives only a current snapshot of query / procedure cache </li></ul></ul><ul><ul><ul><li>All data lost between restarts </li></ul></ul></ul><ul><ul><ul><li>Similar to SQL Trace Profiler in that updates only occur POST query completion. Therefore not quite up to the second information. </li></ul></ul></ul><ul><ul><li>Important DMVs: </li></ul></ul><ul><ul><ul><li>sys.dm_exec_query_stats – reads / time by sql_handle </li></ul></ul></ul><ul><ul><ul><li>sys.dm_exec_query_plan() – execution plan by sql_handle </li></ul></ul></ul><ul><ul><ul><li>sys.dm_exec_sql_text() – query text by sql_handle </li></ul></ul></ul><ul><ul><li>Identify slow queries by joining above three DMVs together </li></ul></ul>
    23. 23. What about query blocking? <ul><li>Use Profiler / SQL Trace – “Blocked Process Report” Event </li></ul><ul><li>Must configure “Blocked Process Threshold” </li></ul><ul><li>configuration set in seconds (# of seconds blocked) </li></ul><ul><li>trace events continually raised every x seconds </li></ul>
    24. 24. What about query blocking? (cont..) <ul><li>Blocked queries are usually caused by inefficient queries taking more locks than necessary </li></ul><ul><li>Blocked queries are usually a consequence of other poorly performing queries </li></ul><ul><li>Still worth monitoring with Blocked Process Report trace to identify (other) inefficient queries for tuning </li></ul><ul><li>Snapshot isolation level provides an alternative to readers being blocked by writers </li></ul><ul><ul><li>readers see previous committed value and read past rather than be blocked by writers. </li></ul></ul>
    25. 25. Infrastructure bottlenecks New features released <ul><li>Logical Page Reads / sec shows TOTAL number of query reads / sec. </li></ul><ul><ul><li>Increases represent either: </li></ul></ul><ul><ul><ul><li>New features, possibly not well tuned (this case) </li></ul></ul></ul><ul><ul><ul><li>Query optimization problems </li></ul></ul></ul><ul><ul><ul><li>Increased utilization </li></ul></ul></ul>
    26. 26. Infrastructure bottlenecks <ul><li>Buffer Life Expectancy shows average time (secs) page buffers survive in data cache before being forced out by pressure from other queries </li></ul><ul><ul><li>High Number (> 1000 secs for OLTPs) is good (low cache cycling) </li></ul></ul>
    27. 27. Special case - tempdb <ul><li>Temp Tables AND Table Variables are created on disk </li></ul><ul><li>Version store is materialized in tempdb </li></ul><ul><ul><li>Under snapshot isolation, db updates are written to disk in tempdb, allowing other queries to read previously committed results </li></ul></ul><ul><li>Large resultset query sorting (ORDER BY) on disk </li></ul><ul><ul><li>Turns SELECT queries from pure disk reads (in user db), to read + write + read </li></ul></ul>
    28. 28. Solid State Drives (SSDs) <ul><li>SSDs are similar in nature to RAM. </li></ul><ul><ul><li>No physically moving parts </li></ul></ul><ul><ul><li>Concurrent access </li></ul></ul><ul><ul><li>Extremely high speed </li></ul></ul><ul><li>SSDs are ideal for tempdb, given tembdb’s disk oriented workload </li></ul><ul><li>SSDs have lower mean time between failures than HDDs </li></ul><ul><ul><li>no moving parts to wear down </li></ul></ul><ul><ul><li>HDDs involve physically moving metal at high speed </li></ul></ul>
    29. 29. Solid State Drives (SSDs) <ul><li>Even if SSD fails, having tempdb on it creates no risk </li></ul><ul><ul><li>tempdb persists no transactional data </li></ul></ul><ul><ul><li>tempdb is totally rebuilt upon every reboot of SQL Server </li></ul></ul><ul><ul><li>even if device totally fails, tempdb can be relocated on HDD during restart of SQL Server </li></ul></ul><ul><li>Hard drive disk heads can only be physically in one place at any point in time </li></ul><ul><ul><li>tempdb‘s random, concurrent, highly write intensive disk activity can generate enormous queued disk workloads </li></ul></ul>
    30. 30. Common SQL Server Performance Problems <ul><li>High CPU Utilization </li></ul><ul><ul><li>Identification </li></ul></ul><ul><ul><ul><li>Guess – Task Manager figures </li></ul></ul></ul><ul><ul><ul><li>Hunt – Perfmon counters </li></ul></ul></ul><ul><ul><ul><li>24x7 – CPU usage by time, statement </li></ul></ul></ul><ul><ul><li>Resolution </li></ul></ul><ul><ul><ul><li>Add additional CPUs </li></ul></ul></ul><ul><ul><ul><li>Identify statement(s) with high CPU </li></ul></ul></ul><ul><ul><ul><li>Move processes to another server or to off peak times </li></ul></ul></ul>
    31. 31. High Disk I/O <ul><li>Identification </li></ul><ul><ul><li>Guess – Disk drive lights or drive churning </li></ul></ul><ul><ul><li>Hunt – Avg Disk Queue Length, % Disk Time </li></ul></ul><ul><ul><li>24x7 – Review IO wait types and consumption </li></ul></ul><ul><li>Resolution </li></ul><ul><ul><li>Add additional physical drives </li></ul></ul><ul><ul><li>Separate tables, indexes, file groups </li></ul></ul><ul><ul><li>Separate databases on physical disks </li></ul></ul><ul><ul><li>Appropriate RAID (database 1, 10, 5 - log 1) </li></ul></ul><ul><ul><li>Add additional indexes and/or re-index tables </li></ul></ul>
    32. 32. Poor Performing Statements <ul><li>Identification </li></ul><ul><ul><li>Guess – User perception and input </li></ul></ul><ul><ul><li>Hunt – Profiler statement analysis </li></ul></ul><ul><ul><li>24x7 – Statements by resource, time, user </li></ul></ul><ul><li>Resolution </li></ul><ul><ul><li>Review database design and query plans </li></ul></ul><ul><ul><li>Review table access order for JOINs </li></ul></ul><ul><ul><li>Recommend indexes based on data access </li></ul></ul><ul><ul><li>Short transactions with regular commits </li></ul></ul>
    33. 33. The Index Impact <ul><li>Identification </li></ul><ul><ul><li>Guess – User perception and input </li></ul></ul><ul><ul><li>Hunt – Review query plans for entire application </li></ul></ul><ul><ul><li>24x7 – Index recommendations </li></ul></ul><ul><li>Resolution </li></ul><ul><ul><li>Use Index Tuning Wizard </li></ul></ul><ul><ul><li>CRUD chart to determine needed indexes </li></ul></ul><ul><ul><li>Review code to determine columns in JOIN, WHERE, ORDER BY, GROUP BY, etc clauses </li></ul></ul><ul><ul><li>Leverage correct index based on needs </li></ul></ul><ul><ul><li>Maintain indexes and statistics per object </li></ul></ul>
    34. 34. Clustered Indexes <ul><li>Clustered indexes are the actual physically written records. </li></ul><ul><ul><li>A SELECT statement with no ORDER BY clause will return data in the clustered index order. </li></ul></ul><ul><ul><li>1 clustered index per table, 249 non-clustered indexes per table. </li></ul></ul><ul><li>Highly recommended for every table! </li></ul><ul><li>Very useful for columns sorted on GROUP BY and ORDER BY clauses, as well as those filtered by WHERE clauses. </li></ul>
    35. 35. Non-Clustered Indexes <ul><li>Useful for retrieving a single record or a range of records. </li></ul><ul><li>Maintained in a separate structure and maintained as changes are made to the base table. </li></ul><ul><li>Tend to be much narrower than the base table, so they can locate the exact record(s) with much less I/O. </li></ul><ul><li>Has at least one more intermediate level than the clustered index, but are much less valuable if table doesn’t have a clustered index. </li></ul>
    36. 36. Fill Factor <ul><li>When SQL Server creates indexes, every page is nearly 100% full. </li></ul><ul><ul><li>No room on the leaf or intermediate pages for INSERTs, UPDATEs, or DELETEs. </li></ul></ul><ul><ul><li>Default can cause costly page splits on certain tables. </li></ul></ul><ul><ul><li>Promotes table fragmentation. </li></ul></ul><ul><li>SQL Server allows you to specify amount of free space in leaf pages with FILL FACTOR, an option in the CREATE INDEX statement. </li></ul>
    37. 37. Stored Procedure Optimization <ul><li>SET NOCOUNT OFF improves performance when coding stored procedures, triggers, and functions. </li></ul><ul><li>Turns of the N rows affected verbiage and eliminates messages from the server to the client for each step in a stored procedure. </li></ul><ul><li>CREATE PROC xyz </li></ul><ul><li>AS </li></ul><ul><li>SET NOCOUNT ON </li></ul><ul><li>< stored procedure code > </li></ul><ul><li>SET NOCOUNT OFF </li></ul><ul><li>GO </li></ul><ul><li>Mixing DDL and DML operations causes a recompile </li></ul><ul><li>Certain operations on temporary tables cause a recompile </li></ul><ul><ul><li>Refer to temp tables created locally </li></ul></ul><ul><ul><li>Don’t declare cursors that reference a temp table </li></ul></ul><ul><ul><li>Don’t create temp tables while in a loop </li></ul></ul>
    38. 38. Querying against Composite Keys <ul><li>Composite keys are only useful from the leftmost column to the rightmost column, in the order they appeared in the CREATE INDEX statement. Example: </li></ul><ul><ul><li>CREATE NONCLUSTERED INDEX ndx_foo ON foo(a, b, c, d) </li></ul></ul><ul><li>The following WHERE clauses will access the NDX_FOO: </li></ul><ul><ul><li>WHERE a = @a </li></ul></ul><ul><ul><li>WHERE a = @a AND b = @b </li></ul></ul><ul><li>The following WHERE clauses will access only part of NDX_FOO: </li></ul><ul><ul><li>WHERE a = @a AND d = @d </li></ul></ul><ul><ul><li>WHERE a = @a AND c = @c AND b = @b </li></ul></ul><ul><li>The following WHERE clauses invalidate NDX_FOO: </li></ul><ul><ul><li>WHERE b = @b AND c = @c </li></ul></ul><ul><ul><li>WHERE b = @b AND a = @a </li></ul></ul>
    39. 39. Queries with LIKE <ul><li>Queries on production systems should NOT use SELECT * FROM… </li></ul><ul><ul><li>Main reason is that any time the underlying table is changed, all query plans stored in the cache must be rebuilt </li></ul></ul><ul><ul><li>The SQL tools allow very quick scripting – so no excuses! </li></ul></ul><ul><li>Queries that use the LIKE clause have two simple rules: </li></ul><ul><ul><li>LIKE can use indexes if the pattern starts with a character string, such as WHERE lname LIKE ‘w%’ </li></ul></ul><ul><ul><li>LIKE cannot use an index if the pattern starts with a leading wildcard, such as WHERE lname LIKE ‘%alton’ </li></ul></ul>
    40. 40. Queries with Functions & Calculations in the WHERE clause <ul><li>Avoid using functions or calculations on the column in a WHERE clause because it causes SQL Server to ignore any index on the column: </li></ul><ul><ul><li>WHERE qty * 12 > 10000 </li></ul></ul><ul><ul><li>WHERE ISNULL(ord_date, ‘Jan 01,2001’) > ‘Jan 01, 2002 12:00:00 AM’ </li></ul></ul><ul><li>Instead, move the function or calculation to the SARG: </li></ul><ul><ul><li>WHERE qty > 10000/12 </li></ul></ul><ul><ul><li>WHERE ord_date IS NOT NULL </li></ul></ul><ul><ul><li>AND ord_date > ‘Jan 01, 2002 12:00:00 AM’ </li></ul></ul>
    41. 41. Query Tuning <ul><li>Use SHOWPLAN_TEXT or Graphic Query Plan to analyze queries. </li></ul><ul><li>Joins perform better than sub queries </li></ul><ul><li>Beware queries that have SCAN but not SEEK operations </li></ul><ul><li>Beware join queries that have HASH but not NESTED LOOP operations </li></ul><ul><li>Remember that constraints put lots of overhead on INSERT and UPDATE statements </li></ul>
    42. 42. Execution Plan Notation <ul><li>The Good, the Ugly, the Bad </li></ul><ul><ul><li>Table Scans and Index Scans – Bad and Ugly </li></ul></ul><ul><ul><li>Sorts – generally Bad and Ugly </li></ul></ul><ul><ul><li>Hash Joins – Bad and Ugly </li></ul></ul><ul><ul><li>Thick lines coming into the operation and thin lines coming out – Bad and Ugly </li></ul></ul><ul><ul><li>Merge Joins – Good without big sort </li></ul></ul><ul><ul><li>Index Seeks and Clustered Index Seeks – Good </li></ul></ul><ul><ul><li>Nested Loop Joins – Good </li></ul></ul><ul><ul><li>Bookmark Lookups – “it depends” </li></ul></ul><ul><li>Join Conditions </li></ul><ul><ul><li>Nested loops are used when one of the inputs is smaller then other. Extremely effective </li></ul></ul><ul><ul><li>Merge joins are used when both inputs are roughly the same size. Requires presorted data and therefore can be dangerous </li></ul></ul><ul><ul><li>Hash joins are used to process un-sorted data using in-memory hashing – generally the slowest way </li></ul></ul>
    43. 43. Optimization Process Spiral
    44. 44. Inside SQL Server Query Optimization
    45. 45. Reduce Contention <ul><li>Keep transactions short </li></ul><ul><ul><li>Don’t get user input in the middle of a transaction </li></ul></ul><ul><ul><li>Process all rows </li></ul></ul><ul><li>Good Indexes </li></ul><ul><ul><li>Reduce time to identify rows to update </li></ul></ul><ul><ul><li>More granular locks </li></ul></ul><ul><li>Monitor Locks and Deadlocks </li></ul><ul><ul><li>Enterprise Manager, syslockinfo, sysprocesses, trace </li></ul></ul><ul><li>Manage Locks and Deadlocks </li></ul><ul><ul><li>Balance deadlocks and performance </li></ul></ul>
    46. 46. Deadlocks <ul><li>Choose Appropriate Isolation Levels </li></ul><ul><li>Cyclic Deadlocks </li></ul><ul><ul><li>Ensure consistent update sequences </li></ul></ul><ul><li>Conversion Deadlocks </li></ul><ul><ul><li>Serialize Access (UPDLOCK hint) </li></ul></ul>
    47. 47. Missing Index Query
    48. 48. Database Tuning Advisor
    49. 49. Index Tuning <ul><li>Passive Tuning with DMVs </li></ul><ul><li>Active Tuning </li></ul><ul><ul><li>Don’t just click apply </li></ul></ul><ul><ul><li>Use smart names </li></ul></ul><ul><ul><li>Look for overlaps </li></ul></ul><ul><ul><li>Go passive first </li></ul></ul>
    50. 50. SQL 2008 Data Compression <ul><li>Estimating Compression </li></ul><ul><ul><li>sp_estimate_data_compression_savings </li></ul></ul><ul><ul><ul><li>@schema_name </li></ul></ul></ul><ul><ul><ul><li>@object_name </li></ul></ul></ul><ul><ul><ul><li>@index_id </li></ul></ul></ul><ul><ul><ul><li>@partition_number </li></ul></ul></ul><ul><ul><ul><li>@data_compression </li></ul></ul></ul><ul><li>Index Compression Drawbacks </li></ul><ul><ul><li>Enterprise Edition only </li></ul></ul><ul><ul><li>No inheritance </li></ul></ul><ul><ul><li>No automation </li></ul></ul>
    51. 51. Index Defragmentation Best Practices <ul><li>13% to 460% Faster </li></ul>
    52. 52. DBCC SHOWCONTIG <ul><li>Use either table name and index name, or table ID and index ID numbers. </li></ul><ul><li>DBCC SHOWCONTIG ( [Order Details], OrderID ) </li></ul><ul><li>Results : </li></ul><ul><li>DBCC SHOWCONTIG scanning 'Order Details' table... </li></ul><ul><li>Table: 'Order Details' (325576198); index ID: 2, database ID: 6 </li></ul><ul><li>LEAF level scan performed. </li></ul><ul><li>- Pages Scanned................................: 5 </li></ul><ul><li>- Extents Scanned..............................: 2 </li></ul><ul><li>- Extent Switches..............................: 1 </li></ul><ul><li>- Avg. Pages per Extent........................: 2.5 </li></ul><ul><li>- Scan Density [Best Count:Actual Count].......: 50.00% [1:2] </li></ul><ul><li>- Logical Scan Fragmentation ..................: 0.00% </li></ul><ul><li>- Extent Scan Fragmentation ...................: 50.00% </li></ul><ul><li>- Avg. Bytes Free per Page.....................: 2062.0 </li></ul><ul><li>- Avg. Page Density (full).....................: 74.52% </li></ul><ul><li>DBCC execution completed. If DBCC printed error messages, contact your system administrator. </li></ul>
    53. 53. DBCC INDEXDEFRAG <ul><li>DBCC INDEXDEFRAG is a great way to rebuild the leaf level of index in one step </li></ul><ul><ul><li>Performs on-line index reconstruction </li></ul></ul><ul><ul><li>Can be interrupted without losing the work already completed </li></ul></ul><ul><ul><li>Fully logged </li></ul></ul><ul><ul><li>Can take longer than rebuilding the index and is not quite as effective </li></ul></ul><ul><li>Syntax: </li></ul><ul><li>DBCC INDEXDEFRAG ( { database | 0 } ,{ table | 'view' } ,{ index }   ) </li></ul><ul><li>     [ WITH NO_INFOMSGS ] </li></ul>
    54. 54. DBCC DBREINDEX <ul><li>DBCC DBREINDEX was introduced in version 7.0 to enable DBAs to rebuild indexes without having to drop and recreate PRIMARY KEY and UNIQUE constraints </li></ul><ul><ul><li>Locks the table for the duration of the operation </li></ul></ul><ul><ul><li>Can offer additional optimizations than a series of individual DROP INDEX and CREATE INDEX statements on a single table </li></ul></ul><ul><li>Syntax: </li></ul><ul><li>DBCC DBREINDEX     ( ['database.owner.table_name' [,index_name [,fillfactor] ] ]     )    [ WITH NO_INFOMSGS ] </li></ul>
    55. 55. Implement Best Practices <ul><li>Be sure to set “Maximize throughput for network applications”. </li></ul><ul><li>Make sure PAGEFILE.SYS is adequately sized. </li></ul><ul><li>Add additional pagefiles on separate physical drives and/or segregate them from SQL Server files. </li></ul><ul><li>Tempdb </li></ul><ul><ul><li>is too small by default. </li></ul></ul><ul><ul><li>automatic file growth is much too small by default </li></ul></ul><ul><ul><li>In high OLTP systems, it should be on a physically separate and fast I/O system </li></ul></ul>
    56. 56. Automate DBA Maintenance Tasks <ul><li>A daily task to perform DBCC checks and dump each of the major system databases. </li></ul><ul><li>A weekly task to reinitialize the indexes and restore fill factors on major datbases. </li></ul><ul><li>A nightly task to update index statistics. </li></ul><ul><li>A nightly task to update the Sysindexes table of each database. </li></ul>
    57. 57. Unknown SQL Server Changes <ul><li>Identification </li></ul><ul><ul><li>Guess – Broken application </li></ul></ul><ul><ul><li>Hunt – Query sysobjects </li></ul></ul><ul><ul><li>24x7 – Schema change report </li></ul></ul><ul><li>Resolution </li></ul><ul><ul><li>Appropriate security based on duties </li></ul></ul><ul><ul><li>Solidified change management process </li></ul></ul><ul><ul><li>Open communication among the team </li></ul></ul>
    58. 58. SQL Server Trending <ul><li>Identification </li></ul><ul><ul><li>Guess – Change in user complaints </li></ul></ul><ul><ul><li>Hunt – Perfmon and Profiler changes </li></ul></ul><ul><ul><li>24x7 – Performance metrics over time </li></ul></ul><ul><li>Benefits </li></ul><ul><ul><li>Proactive approach for future planning </li></ul></ul><ul><ul><li>Justification for hardware and software </li></ul></ul><ul><ul><li>Capacity planning </li></ul></ul>
    59. 59. Evaluate objects <ul><li>Tools </li></ul><ul><li>Working with a baseline </li></ul><ul><li>Working without a baseline </li></ul><ul><li>Don’t fix anything yet! </li></ul>
    60. 60. Gather a baseline <ul><li>Working with a baseline </li></ul><ul><ul><li>Collect data when the problem doesn’t exist. </li></ul></ul><ul><ul><li>Gather a lot of detail. </li></ul></ul><ul><li>Working without a baseline </li></ul><ul><ul><li>Start broad and zero in on problems. </li></ul></ul><ul><ul><li>Look at wider counters (i.e. CPU performance). </li></ul></ul>
    61. 61. Evaluate objects <ul><li>Tools </li></ul><ul><li>Working with a baseline </li></ul><ul><li>Working without a baseline </li></ul><ul><li>Don’t fix anything yet! </li></ul>
    62. 62. Interpret findings <ul><li>Gather subject matter experts </li></ul><ul><ul><li>You can’t do it all – don’t try </li></ul></ul><ul><li>Gather their thoughts </li></ul><ul><ul><li>Make everyone come up with what they think </li></ul></ul><ul><li>Agree on common interpretations </li></ul><ul><ul><li>Don’t sweat the small stuff </li></ul></ul><ul><li>Table differences </li></ul><ul><li>Don’t fix anything yet! </li></ul>
    63. 63. Create an action plan <ul><li>Decide on the fixes </li></ul><ul><li>Decide who should implement </li></ul><ul><li>Decide risks and rewards </li></ul><ul><li>Detail timelines </li></ul><ul><li>Create backup plan </li></ul><ul><li>Implement </li></ul><ul><li>Monitor for change, report </li></ul>
    64. 64. SQL Server Performance Tuning Process Automation SQL Server Performance Tuning Process Automation Educated Guess Manual Automated Hunt and Peck Tool Set Entire company Reactive approach No tools Entire company Reactive approach Disjointed tools Entire company Proactive approach Integrated tools People: Process: Technology:
    65. 65. Performance Tuning Redefined with SQL 2008
    66. 66. Performance Tuning Best Practices <ul><li>Focus on performance needs from the project scope to maintenance </li></ul><ul><li>Design and develop for high performance </li></ul><ul><li>Hardware, Windows, SQL Server and application </li></ul><ul><li>System baseline with ongoing comparisons </li></ul><ul><li>Monitor, analyze, alert and report </li></ul><ul><li>Solidified change management process </li></ul><ul><li>Properly designate permissions based on duties </li></ul><ul><li>Work towards continuous improvements </li></ul>
    67. 67. Methodology review <ul><li>Gather component list </li></ul><ul><li>Evaluate objects </li></ul><ul><li>Interpret findings </li></ul><ul><li>Create an action plan </li></ul>
    68. 68. Thank You! <ul><li>SearchSQLServer.com Performance and Tuning: http://searchSQLServer.com/r/0,,59918,00.htm? </li></ul><ul><li>InformIT.com: http://www.informit.com </li></ul><ul><li>(Click on Reference Guides, then SQL Server) </li></ul><ul><li>SQL-Server-Performance.com: http://sql-server-performance.com </li></ul><ul><li>Books with excellent performance tuning content </li></ul><ul><ul><li>“ SQL Server Query Performance Tuning Distilled”, Sajal Dam </li></ul></ul><ul><ul><ul><li>http:// www.apress.com/book/bookDisplay.html?bID =371 </li></ul></ul></ul><ul><ul><li>“ SQL Server 2005 Performance Tuning”, various </li></ul></ul><ul><ul><ul><li>http:// www.wrox.com </li></ul></ul></ul><ul><ul><li>“ Guru’s Guide to SQL Server Architecture & Internals”, Ken Henderson </li></ul></ul><ul><ul><ul><li>http://www.amazon.com/exec/obidos/tg/detail/-/0201700476/ref=pd_bxgy_img_2/104-7280867-1941549?v= glance&s =books </li></ul></ul></ul><ul><ul><li>“ SQL Server 2005 Practical Troubleshooting”, Ken Henderson </li></ul></ul><ul><ul><ul><li>http://safari.oreilly.comamazon.com/0321447743 </li></ul></ul></ul>