The document discusses various techniques for optimizing SQL Server performance, including handling index fragmentation, optimizing files and partitioning tables, effective use of SQL Profiler and Performance Monitor, a methodology for performance troubleshooting, and a 10 step process for performance optimization. Some key points covered are determining and resolving index fragmentation, partitioning tables across multiple file groups, capturing traces with SQL Profiler and Performance Monitor counters to diagnose issues, and ensuring proper indexing through query execution plans and the SQL Server tuning advisor.

1. Presented By : M.M.Al-Farooque http://www.linkedin.com/in/thisisshubho (Shubho)

2. Topics Handling Index fragmentation Optimizing files and partitioning tables Effective use of SQL Profiler Performance Monitor Performance troubleshooting methodology 10 steps for performance optimization

3. Handling Index Fragmentations What is Index Fragmentation? Fragmentation occurs in index/data pages as a result of data insert/update/delete operations in the tables. If indexes have high fragmentations, either scanning/seeking the indexes takes much time or the indexes are not used at all (Resulting in table scan). Two types fragmentation can occur: Internal Fragmentation: Occurs due to the data deletion/update operation in the index pages which ends up in distribution of data as sparse matrix in the index/data pages. Also results in increase of index/data pages that increase query execution time. External Fragmentation: Occurs due to the data insert/update operation in the index/data pages which ends up in page splitting and allocation of new index/data pages that are not contiguous. That reduces performance in determining range and read-ahead operations.

4. Handling Index Fragmentations How to determine whether index fragmentation occurred or not? Execute the following SQL: select object_name(dt.object_id) Tablename,si.name IndexName,dt.avg_fragmentation_in_percent as ExternalFragmentation,dt.avg_page_space_used_in_percent as InternalFragmentation from ( select object_id,index_id,avg_fragmentation_in_percent,avg_page_space_used_in_percent from sys.dm_db_index_physical_stats (db_id('AdventureWorks'),null,null,null,'DETAILED' ) where index_id <> 0) as dt inner join sys.indexes si on si.object_id=dt.object_id and si.index_id=dt.index_id and dt.avg_fragmentation_in_percent>10 and dt.avg_page_space_used_in_percent<75 order by avg_fragmentation_in_percent desc ExternalFragmentation value > 10 indicates External fragmentation occurred for corresponding index InternalFragmentation value < 75 indicates Internal fragmentation occurred for corresponding index

5. Handling Index Fragmentations Defragment index fragmentation Use “ALTER INDEX ALL ON TableName RECOGNIZE” when External Fragmentation Between 10-15 and Internal Fragmentation between 60-75 Use “ALTER INDEX ALL ON TableName REBUILD WITH (FILLFACTOR=90,ONLINE=ON)” when External Fragmentation >15 and Internal Fragmentation < 60

6. Optimizing files and partitioning tables Consider creating a &quot;User defined file group&quot; for your database if total database size is over 100 MB. Make this as &quot;Default&quot; file group. Consider creating multiple files per &quot;User defined file group&quot; so that, SQL server can concurrently access those files while retrieving data Place heavily accessed big fat tables in one filegroup and place the table's indexes and/or text/image columns in a different filegroup on different physical disks. Because logging is more write-intensive, place the log files on different physical disk(s) than data files. Consider assigning the &quot;Read only&quot; tables into a file group that is marked as &quot;Read Only&quot; Consider creating &quot;History tables&quot; that contains archived read only data where select operations will be applied. Use SQL server Maintanence feature to populate those history tables. Consider partitioning big fat tables into different file groups so that the table spans across different files and SQL server engine can concurrently access the table data.

7. Partitioning tables Table partitioning Partitioning lets us split a table (And,Indexes) across multiple filegroups, based upon a user specification. Partitioning enables database engine to read/write data, and, also to calculate (Say, calculating aggregate functions() in parallel) data operations faster. Data could be transferred for analysis purpose based upon age, so, data volume for OLTP operation remains minimal. Partitioning enables easy/manageable/faster way of archive/backup/recovery

8. Partitioning tables Following tasks are to be performed to partition a table or index 1. Add user defined file groups to the database 2. Create partition function 3. Create a partition schema 4. Create/Modify the table or index on the partition schema.

9. Partitioning tables 1. Add user defined file groups to the database ALTER DATABASE AdventureWorks ADD FILEGROUP [2003Q3] ALTER DATABASE AdventureWorks ADD FILE (NAME = N'2003Q3', FILENAME = N'C:\AdventureWorks\2003Q3.ndf', SIZE = 5MB, MAXSIZE = 100MB, FILEGROWTH = 5MB) TO FILEGROUP [2003Q3]

10. Partitioning tables 2. Create Partition function A partition function is an object that defines the boundary points for partitioning data. Following Command creates a partition function CREATE PARTITION FUNCTION OrderDateRangePFN(datetime) AS RANGE LEFT FOR VALUES ('20000930 23:59:59.997', '20001231 23:59:59.997', '20010331 23:59:59.997', '20010630 23:59:59.997')

11. Partitioning tables 3. Create Partition Scheme The partition scheme defines the file groups that will be used with a partition function to partition the table/index Following command creates a partition schema CREATE PARTITION SCHEME OrderDatePScheme AS PARTITION OrderDateRangePFN TO ([2000Q3], [2000Q4], [2001Q1], [2001Q2], [PRIMARY])

12. Partitioning tables 4. Create the table or index on the partition schema. With the partition function (the logical structure) and the partition scheme (the physical structure) defined, the table can be created to take advantage of them. The table defines which scheme should be used, and the scheme defines the function ALTER TABLE Orders ADD CONSTRAINT OrdersPK PRIMARY KEY CLUSTERED ON OrderDatePScheme (OrderDate)

13. Partitioning tables Partitioning Index and indexed views Following command creates a partitioned index CREATE NONCLUSTERED INDEX indxCustomerAddress on dbo.CustomerAddress(City) on partscheme(CustomerAddressID) In case of an existing index, drop the index first Partitioning indexed view means partitioning the index of the view

14. Effective use of SQL Profiler Tips & Tricks 1. Use existing templates, but, create your own template when in need. 2. Save profiler trace into table for further analysis 3. Export particular events (Say, TSQLs) from profiler trace for particular analysis. 4. Create Replay trace and replay the trace on test server to diagnose problems. 5. Capture deadlock events on production and diagnose on test server. 6. Capture ShowPlan to include SQL plans in the profiler. 7. Capture Table Scan events in profiler.

15. Performance Monitor Profiler enables us to capture long running queries but, cannot provide the context to explain the reasons. (e.g. Same query takes 3 seconds in production but, takes 30 ms in test server, cause, it lacks context). Performance Monitor tool (System monitor) could be used to generate a counter log that can be used to provide the profiler trace with the context. Performance Monitor gathers statistical data related to hardware and software metrics.

16. Performance Monitor Windows has lots of built in objects with their performance counters. When installation, Performance counters for SQL server also get installed. Create a Performance Counter log by incorporating the following common performance counters: (Next slide)

17. Performance Monitor Include the following common counters Network Interface\Output Queue length Processor\%Processor Time SQL Server:Buffer Manager\Buffer Cache Hit Ratio SQL Server:Buffer Manager\Page Life Expectancy SQL Server:SQL Statistics\Batch Requests/Sec SQL Server:SQL Statistics\SQL Compilations SQL Server:SQL Statistics\SQL Re-compilations/Sec

18. Performance troubleshooting methodology SQL server dynamically generates different query plans based on: --Volume of Data --Statistics --Index variation --Parameter value in TSQL --Load on server So, when diagnosing any SQL performance related problem , a methodology should be followed to simulate the production environment in Test server.

19. Performance troubleshooting methodology Performance troubleshoot methodology --Capture trace at production server and include ShowPlan in the trace. Do the same in the Test server and compare the plans. --Create Replay Trace in production and replay the trace on test server to create a similar load. --In the Database Engine Tuning Advisor too, use the production trace as the workload to view tuning suggestions.

20. Performance troubleshooting methodology --Take performance counter log in the production server and correlate it with the SQL profiler trace to diagnose the bottleneck. --Analyze the Query plans in the production SQL profiler trace to diagnose whether indexes are properly utilized in the SQL’s --Execute the TSQL (Provider earlier) to view the External/Internal fragmentation values for indexes in the target database.

21. 10 steps for performance optimization 1. Make sure that, indexing is properly done in the database (All tables has primary keys, and, tables has appropriate non-clustered indexes on columns which -Are used as search criteria fields -Are used to join other tables -Are used as foreign key fields -Have low selectivity -Are used in the ORDER BY clause While creating indexes, create appropriate &quot;Covering indexes&quot; involving frequently accessed fields Use &quot;SQL server management studio&quot; to view the query execution plan and identify the scopes for improvement in the SQLs Use &quot;SQL server tuning advisor&quot; to get help from SQL server in creating indexes.

22. 10 steps for performance optimization 2. Consider moving your SQL's from application to Stored Procedures/Views if there is any 3. Re-factor the SQL's used in Stored procedures/Views/Triggers and apply the best practices to write optimized queries 4. Consider using the &quot;Full text search&quot; feature to perform search on textual columns. 5. Re-build indexes if fragmentation occurs. 6. Create &quot;Indexed Views&quot; with expensive select queries that internally saves result sets on queries 7. Consider refactoring your table design and apply De-normalizations to improve select operations.

23. 10 steps for performance optimization 8. Consider creating different &quot;User defined file groups&quot; and use these based upon -Frequency of table access -Read/write nature of tables -Types of objects (Tables/Index) 9. Consider creating &quot;History tables&quot; that contains archived read only data where select operations will be applied. Use SQL server Maintenance feature to populate those history tables. 10. Consider partitioning big fat tables into different file groups so that the table spans across different files and SQL server engine can concurrently access the table data.