Tracing Sql Server 2005


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Tracing Sql Server 2005

    1. 1. ISQL Users Group May 2009 Tracing and Profiling SQL Server [email_address]
    2. 2. <ul><li>Over the next 1 hour: </li></ul><ul><li>SQL Trace Architecture and Terminology </li></ul><ul><li>Security and Permissions </li></ul><ul><li>Profiler </li></ul><ul><li>Server side Tracing </li></ul><ul><li>Saving and Replaying Traces </li></ul><ul><li>Querying Server-Side Trace Metadata </li></ul>
    3. 3. <ul><li>Introduction </li></ul><ul><li>Query tuning, optimization and troubleshooting are all possible by the ability to view what's going on inside SQL Server </li></ul><ul><li>SQL Trace provides a real-time view into what’s going on inside the database engine and at a very granular level </li></ul>
    4. 4. SQL Trace Architecture and Terminology
    5. 5. <ul><li>SQL Trace is an SQL Server database engine technology </li></ul><ul><li>SQL Server Profiler is a .NET application that uses system stored procedures exposing the functionality of SQL Trace </li></ul><ul><li>Microsoft.SqlServer.Management.Trace </li></ul><ul><li> </li></ul>
    6. 6. <ul><li>When tracing we monitor for specific events which are generated when various actions occur in the database engine </li></ul><ul><ul><li>For example, a user login or the execution of a query are each actions that cause events to fire </li></ul></ul><ul><li>Each event has an associated collection of columns that contain the collected data when the event fires </li></ul><ul><li>SQL Server uses a model that selectively enables data collection as required </li></ul><ul><ul><li>Data is never collected until someone asks for it </li></ul></ul><ul><li>There are around 171 events and 65 columns to select from </li></ul>
    7. 7. <ul><li>The main component of the SQL Trace architecture is the trace controller which is a shared resource that manages all traces created by any consumer </li></ul><ul><ul><li>There are various event producers in the database engine for example in the query processor, lock manager, and cache manager </li></ul></ul><ul><ul><li>Each of these producers is responsible for generating events that relates to certain categories of server activity </li></ul></ul><ul><ul><li>All producers are disabled by default and therefore generate no data </li></ul></ul><ul><ul><li>When a trace is started for a certain event a global bitmap in the trace controller is updated letting the event producer know that at least one trace is listening, and causing the event to begin firing </li></ul></ul><ul><ul><li>Managed along with this bitmap is a secondary list of which traces are monitoring which events </li></ul></ul><ul><ul><li>Once an event fires, its data is routed into a global event sink, which queues the event data for distribution to each trace that is actively listening </li></ul></ul><ul><ul><li>The trace controller routes the data to each listening trace based on its internal list of traces and watched events </li></ul></ul><ul><ul><li>In addition to the trace controller's own lists each individual trace also keeps track of which events it is monitoring along with which columns are actually being used as well as what filters are in place </li></ul></ul><ul><ul><li>The event data returned by the trace controller to each trace is filtered and the data columns are trimmed down as necessary before the data is routed to an I/O provider </li></ul></ul>
    8. 8. <ul><li>Trace I/O providers send the data to the final destination </li></ul><ul><li>The available output formats for trace data are either a file on the database server (or a network share) or a rowset to a client </li></ul><ul><li>Both providers use internal buffers to ensure that if the data is not consumed quickly enough (written to disk or read from the rowset) it will be queued </li></ul>
    9. 9. <ul><li>The file provider is designed with a guarantee that no event data will be lost </li></ul><ul><li>If an I/O slowdown occurs so disk writes are not occurring quickly enough the internal buffers begin to fill </li></ul><ul><li>To monitor for these waits watch the SQLTRACE_LOCK and IO_COMPLETION wait types </li></ul>
    10. 10. <ul><li>The rowset provider is not designed for data loss guarantee </li></ul><ul><li>If data is not being consumed quickly enough and its internal buffers fill it waits up to 20 seconds before it begins dropping events in order to free buffers for the sake of getting things moving </li></ul><ul><li>SQL Server Profiler will then send an information message informing that some events had been lost and had not been captured </li></ul><ul><li>You can find out if you're headed in that direction by monitoring SQL Server's TRACEWRITE wait type which is incremented as threads are waiting for buffers to free up </li></ul>
    11. 11. <ul><li>A background trace management thread is started whenever at least one trace is active on the server </li></ul><ul><ul><li>SELECT * FROM sysprocesses </li></ul></ul><ul><ul><li>WHERE cmd = N'TRACE QUEUE TASK' ; </li></ul></ul><ul><li>The background thread is responsible for: </li></ul><ul><ul><li>Flushing file provider buffers (done every 4 seconds) </li></ul></ul><ul><ul><li>Can be seen as a wait_type of SQLTRACE_BUFFER_FLUSH </li></ul></ul><ul><ul><li>Closing rowset-based traces that are considered to be expired (dropping events for more than 10 minutes) </li></ul></ul><ul><li>There is no provider that writes trace data directly to a table </li></ul>
    12. 12. <ul><ul><li>Security and Permissions </li></ul></ul>
    13. 13. <ul><li>SQL Server 2005 introduces a new permission called ALTER TRACE. This is a server-level permission granted to a login principal and allows access to start, stop, or modify a trace (in addition to being able to generate user-defined events) </li></ul><ul><li>GRANT ALTER TRACE TO <LoginName> ; </li></ul><ul><li>SQL Trace has a couple of built-in security features to keep passwords secured </li></ul><ul><ul><li>SQL Trace will automatically omit data if an event contains a call to a password-related stored procedure or statement. For example, a call to CREATE LOGIN including the WITH PASSWORD option will be blanked out by SQL Trace. </li></ul></ul><ul><ul><li>SQL Trace will not return statement text or query plans generated within an encrypted stored procedure, user-defined function, or view </li></ul></ul>
    14. 14. Profiler
    15. 15. <ul><li>The General tab allows you to control how the trace will be processed by the consumer </li></ul><ul><li>The default setting is to use the rowset provider displaying the events in real time in the SQL Server Profiler tool window </li></ul><ul><li>Other available options are </li></ul><ul><ul><li>Save the events to a table </li></ul></ul><ul><ul><li>Save the events to a file (on either the server or the client) </li></ul></ul><ul><li>When saving to a table Profiler uses the rowset provider and routes the data back into a table </li></ul><ul><li>When saving to a server side file Profiler actually starts two equivalent traces—one using the rowset provider and the other using the file provider </li></ul><ul><li>Saving to a client-side file does not use the file provider at all. Rather, the data is routed to the Profiler tool via the rowset provider and then saved from there to a file. </li></ul><ul><ul><li>This is more efficient than using Profiler to write to a server-side file </li></ul></ul><ul><ul><li>Remember that doing so you do not take benefit of the lossless data guarantee offered by the file provider </li></ul></ul>
    16. 16. <ul><li>The Events Selection tab allows you to select events that you'd like to trace along with their associated data columns </li></ul><ul><li>In order to narrow your scope and help ensure that tracing does not cause performance issues SQL Trace offers the ability to filter the events based on various criteria </li></ul><ul><li>In SQL Server Profiler filtration is exposed via the “Column Filters” tab </li></ul><ul><li>Remember to check the “Show All Columns” checkbox in order to see the complete list of columns </li></ul><ul><li>Once you click the Run button data will begin streaming immediately and be displayed at the Profiler window (this is because Profiler uses the rowset provider) </li></ul><ul><ul><li>If you find that data is coming in too quickly for you to be able to read it you may disable scrolling using the Auto Scroll icon on the SQL Server Profiler toolbar </li></ul></ul><ul><ul><li>If you are on a slow remote connection and your trace is not well filtered you may be in a situation that the icons in the Profiler tool bar are seen as if they were inactive thus disallowing you to stop the trace. The only way out here is to stop the trace using sp_trace_setstatus </li></ul></ul>
    17. 17. <ul><li>Events that do not produce data for a specific column cannot be used to filter on that column </li></ul><ul><ul><li>For example, the SQL:BatchStarting event does not produce duration data and therefore a filter defined on that column for this event will not filter anything </li></ul></ul><ul><ul><li>Note that the SQL:BatchStarting event will still return even though it lacks the Duration output column. To modify this behavior, check the “Exclude Rows That Do Not Contain Values” checkbox in the “Edit Filter” dialog box for the column you'd like to change the setting for </li></ul></ul>
    18. 18. Saving and Replaying Traces
    19. 19. <ul><li>Profiler ships with eight predefined templates </li></ul><ul><ul><li>A template is a collection of event and column selections, filters, and other settings that you can save to create a reusable trace definitions </li></ul></ul><ul><li>TSQL_Replay template selects a variety of columns for 15 events that are all required for Profiler to be able to play back (Replay) a collected trace at a later time. </li></ul><ul><ul><li>Collecting data using this template allows you to reproduce a problem experienced on a production system by replaying events collected on the production system at a test environment </li></ul></ul><ul><ul><li>Very useful for support departments </li></ul></ul><ul><li>Collected trace data has to be saved and then reopened before a replay can begin </li></ul><ul><li>SQL Server Profiler offers the following options for saving trace data available from the File menu: </li></ul><ul><ul><li>The Trace File option is used to save the data to a file formatted using a proprietary binary format. This is generally the fastest way to save the data, and also the smallest in terms of bytes on disk </li></ul></ul><ul><ul><li>The Trace Table option is used to save the data to a new or previously created table in a database. This option is useful if you need to manipulate or report on the data using T-SQL </li></ul></ul><ul><ul><li>The Trace XML File option saves the data to a text file formatted as XML </li></ul></ul><ul><ul><li>The Trace XML File For Replay option also saves the data to an XML text file, but only those events and columns needed for replay functionality are saved </li></ul></ul>
    20. 20. <ul><li>Once the data has been saved to a file or table the original trace window can be closed and the file or table reopened via SQL Server Profiler's File menu </li></ul><ul><li>A trace reopened in this way will have a Replay menu on the Profiler tool bar allowing you to start replaying the trace, stop the replay, or set a breakpoint </li></ul><ul><li>During the course of the replay, the same events used to produce the trace being replayed will be traced from the server on which you replay </li></ul><ul><ul><li>The ”Save To File” and “Save To Table” options are used for a client-side save. No server-side option exists for saving playback results </li></ul></ul><ul><li>The trace will be replayed on multiple threads (2005 only), corresponding to at most the “Number Of Replay Threads” specified </li></ul><ul><ul><li>The “Replay Events In The Order They Were Traced” option ensures that all events will be played back in exactly the order in which they occurred as based upon the EventSequence column. Multiple threads will still be used to simulate multiple spids. </li></ul></ul><ul><ul><li>The “Replay Events Using Multiple Threads” option allows SQL Server Profiler to reorder the order in which each spid starts to execute events, in order to enhance playback performance. However, the order of events will remain consistent with the EventSequence within a given spid </li></ul></ul>
    21. 21. Server-Side Tracing
    22. 22. <ul><li>sp_trace_create is used to define a trace and specify an output file location. This stored procedure returns a handle to the created trace in the form of an integer TraceId </li></ul><ul><li>sp_trace_setevent is used to add/remove event/column combinations to traces based on the TraceId </li></ul><ul><li>sp_trace_setfilter is used to define event filters based on trace columns </li></ul><ul><li>sp_trace_setstatus is called to start, stop and and delete a trace. </li></ul><ul><ul><li>Traces can be started and stopped multiple times over their lifespan </li></ul></ul>
    23. 23. <ul><ul><li>sp_trace_setfilter </li></ul></ul><ul><ul><ul><li>@TraceID </li></ul></ul></ul><ul><ul><ul><li>,@columnid </li></ul></ul></ul><ul><ul><ul><li>,@logical_operator </li></ul></ul></ul><ul><ul><ul><li>,@comparison_operator </li></ul></ul></ul><ul><ul><ul><li>,@value </li></ul></ul></ul><ul><ul><li>/* </li></ul></ul><ul><ul><li>@logical_operator: AND (0) or OR (1) </li></ul></ul><ul><ul><li>@comparison_operator: </li></ul></ul><ul><ul><li>0 = Equal, 1 = Not equal, 2 = Greater than, 3 = Less than, 4 = Greater than or equal, </li></ul></ul><ul><ul><li>5 = Less than or equal, , 6 = Like, 7 = Not like </li></ul></ul><ul><ul><li>*/ </li></ul></ul>
    24. 24. <ul><li>SQL Trace offers both &quot;and&quot; and &quot;or&quot; logical operators that can be combined if multiple filters are used. However, there is no way to indicate parentheses or other grouping constructs, meaning that the order of operations is limited to left-to-right evaluation. This means that an expression such as A and B or C and D is logically evaluated by SQL Trace as (((A and B) or C) and D). However, SQL Trace will internally break the filters into groups based on columns being filtered. So the expression Column1=10 or Column1=20 and Column3=15 or Column3=25 will actually be evaluated as (Column1=10 or Column1=20) and (Column3=15 or Column3=25). Not only is this somewhat confusing, but it can make certain conditions difficult or impossible to express. </li></ul>
    25. 25. Querying Server-Side Trace Metadata
    26. 26. <ul><li>Get Traces </li></ul><ul><li>SELECT * </li></ul><ul><li>FROM sys.traces WHERE [status] = 1 ; </li></ul><ul><ul><li>--This query returns the trace status, which will be 1 (started) or 0 (stopped); the server-side path to the trace file (or NULL if the trace is using the rowset provider); the maximum file size (or again, NULL in the case of the rowset provider); information about how many buffers of what size are in use for processing the I/O; the number of events captured; and the number of dropped events (NULL if your trace is using the file provider(loseless guarantee) ). </li></ul></ul>
    27. 27. <ul><li>Get Events </li></ul><ul><li>SELECT </li></ul><ul><li> AS [e vent ] , </li></ul><ul><li> AS [c olumn ] </li></ul><ul><li>FROM fn_trace_geteventinfo( @TraceId ) ei </li></ul><ul><li>INNER JOIN sys.trace_events e </li></ul><ul><li>ON ei.eventid = e.trace_event_id </li></ul><ul><li>INNER JOIN sys.trace_columns c </li></ul><ul><li>ON ei.columnid = c.trace_column_id; </li></ul>
    28. 28. <ul><li>Get Filters </li></ul><ul><li>SELECT </li></ul><ul><li> columnid </li></ul><ul><li>, AS [c olumn ] </li></ul><ul><li>,logical_operator </li></ul><ul><li>,comparison_operator </li></ul><ul><li>,value </li></ul><ul><li>FROM fn_trace_getfilterinfo( TraceId ) ei </li></ul><ul><li>INNER JOIN sys.trace_columns c </li></ul><ul><li>ON ei.columnid = c.trace_column_id; </li></ul>
    29. 29. <ul><li>Get Events/Columns combination </li></ul><ul><li>SELECT </li></ul><ul><li> AS [event] </li></ul><ul><li>, AS [column] </li></ul><ul><li>FROM sys.trace_event_bindings b </li></ul><ul><li>INNER JOIN sys.trace_events e </li></ul><ul><li>ON e.trace_event_id = b.trace_event_id </li></ul><ul><li>INNER JOIN sys.trace_columns c </li></ul><ul><li>ON c.trace_column_id = b.trace_column_id </li></ul><ul><li>ORDER BY ; </li></ul>
    30. 30. <ul><li>Get SubClass values </li></ul><ul><li>SELECT </li></ul><ul><li> AS [column] </li></ul><ul><li>, AS [event] </li></ul><ul><li>,s.subclass_value </li></ul><ul><li>,s.subclass_name </li></ul><ul><li>FROM sys.trace_columns c </li></ul><ul><li>INNER JOIN sys.trace_subclass_values s </li></ul><ul><li>ON c.trace_column_id = s.trace_column_id </li></ul><ul><li>INNER JOIN sys.trace_events e </li></ul><ul><li>ON e.trace_event_id = s.trace_event_id </li></ul><ul><li>WHERE LIKE 'Audit Login'; ` </li></ul>
    31. 31. <ul><li>Reading trace files </li></ul><ul><li>SELECT * FROM fn_trace_gettable(‘B:TracesTraceErrors.trc', 1); </li></ul><ul><ul><li>This function takes two parameters: The path + name to the trace file and the number of trace files to read. If the second parameter is DEFAULT then all available trace files will be read </li></ul></ul><ul><li>SELECT * INTO ProfilerErrors </li></ul><ul><li>FROM fn_trace_gettable(‘B:TracesTraceErrors.trc', DEFAULT); </li></ul><ul><ul><li>Loading the file to a table allows querying the data. </li></ul></ul><ul><ul><li>It also consumes less resources as it is a one time operation and benefits from the fact that the SELCT..INTO command is a BULK operation where as just performing queries against the trace loads sql server’s memory and if it is a large set of data spilling begins thus incurring additional I/O </li></ul></ul><ul><li>SELECT IDENTITY(int, 1, 1) AS RowNumber, * </li></ul><ul><li>INTO dbo.ProfilerErrors </li></ul><ul><li>FROM fn_trace_gettable(‘B:TracesTraceErrors.trc', DEFAULT); </li></ul>
    32. 32. <ul><li>Controlling trace state </li></ul><ul><li>EXEC sp_trace_setstatus @TraceId, 1; --start </li></ul><ul><li>EXEC sp_trace_setstatus @TraceId, 0; --stop </li></ul><ul><li>EXEC sp_trace_setstatus @TraceId, 2; –remove </li></ul><ul><li>A trace in a stopped state can be started without the need to recreate the trace. This allows to modify the Events/Columns selection and the selected Filters </li></ul><ul><li>A restart of the SQL Server service removes all trace definitions </li></ul><ul><ul><li>Zeros the EventSequence column </li></ul></ul><ul><ul><li>If you need your trace always up and running you can launch it from a start up stored procedure </li></ul></ul>
    33. 33. <ul><li>Import Performance Monitor Data </li></ul><ul><li>Default Trace </li></ul><ul><li>Balackbox Trace </li></ul>
    34. 34. ?
    35. 35. SQL Server BOL (Books Online) sys.dm_os_wait_stats Troubleshooting and Analysis with Traces Introducing SQL Trace SQL Profiler Events SQL Profiler Data Columns Inside Microsoft SQL Server 2005
    36. 36. Trace Errors