Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Watch Re-runs
OnYour SQL Server!
David Cobb
sql@davidcobb.net
All slides and content available
on my blog:
http://bit.ly/1...
David Cobb
SQLTrainer / Consultant
MCT, MCSE Data Platform for SQL 2012
sql@davidcobb.net
daveslog.com
Presenter
Past Role...
Free 8-part series
THURSDAYS, 6:00 PM – 9:00 PM
Session 3: Legal & IP, 3d Printing (May 28)
Session 4: Design UIUX & Machi...
What Are RML Utilities
 Microsoft internal tool used by the SQL support team
 Precursor to Distributed Replay Client int...
WhyWould I UseThem?
 PRE-SQL 2012 (otherwise look at the Distributed Replay Client,
it IS supported)
 Testing effect on ...
Where Do I Start?
 RML Utilities Quickstart - Install the tool, (http://bit.ly/1IPGGfJ)locate the RML
Help.pdf and walk t...
How Would I UseThisTo Capture And Replay
Production Workloads?
 Workflow diagram
 Before you start:
 Install RML Utilit...
Overview OfThe RML Replay Process:
 RML Utilities BasicWorkflow Diagram (github link)
BASELINE CAPTURE:
On Production Server (SQL1)
 Take a full backup in advance, plus tlog backups of all relevant user data...
BASELINE CAPTURE : Continued..
 Start simulated workload if testing, or wait for production activity
 When capture of pr...
DEMO: BASELINE CAPTURE
 From RML Help.pdf, pg 17
DEMO : ReadTrace
 I’m performing the demos that take some time so I can talk while they run..
PROCESS : OnTest Server (SQL2)
Using the ReadTrace tool
 Use readtrace.exe from RML Cmd Prompt on the test server (or
any...
ReadTrace Continued..
 ReadTrace -I<PATH_TO_FIRST_TRACE_FILE> -o<PATH_TO_RML_OUTPUT_FOLDER>
-S<DBSERVER> -d<NEW_PERF_ANAL...
REPLAY : OnTest Server (SQL2)
 Prepare environment on test server
 Ensure you won't have access to / affect production!
...
DEMO : Ostress
 Again, I’m performing the demos that take some time so I can talk while they run..
OSTRESS :The Replay Command
 OStress is a scalable, ODBC based application which can replay database commands in two mode...
Use OstressTo Start Replay
 Start replay with ostress.exe from RML Cmd Prompt
 ostress -E -SSQL2 -iC:RMLDemooutputrmlrun...
PROCESS : OnTest Server (SQL2)
 Now we need to analyze the results of the replay on the test server.
 We use readtrace.e...
ANALYZE : OnTest Server (SQL2)
 reporter.exe runs once readtrace.exe is
complete. From Properties tab, enter the names
of...
BUTWAIT,THERE'S MORE!
 SQLNexus
 Visual reports from SQL traces (generated using PSSDIAG script) and SQLDIAG
 http://sq...
QUESTIONS?
sql@davidcobb.net
Daveslog.com
RESOURCE LINKS
 Description of the Replay Markup Language (RML) Utilities for SQL Server -
https://support.microsoft.com/...
Watch Re-runs on your SQL Server with RML Utilities
Watch Re-runs on your SQL Server with RML Utilities
Upcoming SlideShare
Loading in …5
×

Watch Re-runs on your SQL Server with RML Utilities

844 views

Published on

RML Utilities provide command line tools and interactive reports enabling you to:
Take SQL trace files (captured with SQL Profiler, sp_trace or extended events in SQL 2012+),
Process them into replayable RML files (using readtrace.exe),
Play them back in a different SQL environment (using ostress.exe),
And compare the performance at a granular level (using reporter.exe or custom queries).

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Watch Re-runs on your SQL Server with RML Utilities

  1. 1. Watch Re-runs OnYour SQL Server! David Cobb sql@davidcobb.net All slides and content available on my blog: http://bit.ly/1HicQw9http://www.imdb.com/find?s=all&q=good%20times
  2. 2. David Cobb SQLTrainer / Consultant MCT, MCSE Data Platform for SQL 2012 sql@davidcobb.net daveslog.com Presenter Past Roles: Tech Support, Network Admin, Web Developer, DBA Current: Systems Architect for CheckAlt.com Training SQL Since: 2002 FavoriteTechs of the Moment: PowerShell & Azure
  3. 3. Free 8-part series THURSDAYS, 6:00 PM – 9:00 PM Session 3: Legal & IP, 3d Printing (May 28) Session 4: Design UIUX & Machine Learning (Jun 4) Session 5: Finance & Programming services for IoT Session 6: KPIs & Test-to-Manufacture Session 7: Distribution & Programming Drones Session 8: Funding/Investments & Programming Wearables Details and Registration: aka.ms/micmeetup
  4. 4. What Are RML Utilities  Microsoft internal tool used by the SQL support team  Precursor to Distributed Replay Client introduced in SQL 2012  Has both replay and stress test modes.  Supports SQL Server versions 2005 thru SQL 2014.  Not officially supported! Good email responsiveness from the support team though. sstlbugs@microsoft.com
  5. 5. WhyWould I UseThem?  PRE-SQL 2012 (otherwise look at the Distributed Replay Client, it IS supported)  Testing effect on SQL statement performance from environment changes using real production query load (including inserts/updates/deletes) .  Test with new version of SQL (or hotfix, CU, or SP)  Test with different server or database settings (MAXDOP changes, RCSI)  Test with different files or disks (additional files, filegroups, different RAID or LUN, SSDs)  Test index changes  Etc.
  6. 6. Where Do I Start?  RML Utilities Quickstart - Install the tool, (http://bit.ly/1IPGGfJ)locate the RML Help.pdf and walk through the quickstart in a test environment to get familiar with it.  Documentation is thorough, covering many use cases, but often at a high level of technical detail.
  7. 7. How Would I UseThisTo Capture And Replay Production Workloads?  Workflow diagram  Before you start:  Install RML Utilities and requirements (ReportViewer 2008) on test server  Create folders for config files, scripts, backups, traces and RML files on test server and production  Optional: Install SQLNexus, PSSDIAG (To be covered in a future presentation  )
  8. 8. Overview OfThe RML Replay Process:  RML Utilities BasicWorkflow Diagram (github link)
  9. 9. BASELINE CAPTURE: On Production Server (SQL1)  Take a full backup in advance, plus tlog backups of all relevant user databases databases.  Note the time  Using a customized copy of SD_Detailed.XML (which is generated by running SQLDIAG ), with ProfilerCollector disabled, start a SQLDIAG session to newly created folder.  sqldiag.exe /OC:RMLDemooutputsqldiagrun01 /IC:RMLDemoconfigSD_Detailed.XML  Also start capture of a SQL trace using sp_trace to a newly created folder.  using TraceCaptureDef.sql saving to a named folder (i.e. trace01 or trace0521).  exec @rc = sp_trace_create @TraceID output, 2 /* rollover*/, N'C:RMLDemooutputtracerun01replay01' , @maxfilesize, NULL  Alternately, use StartSQLDiagForReplay2008.cmd from PSSDIAG  DON'T USE REPLAYTRACETEMPLATE IN SQL PROFILER!
  10. 10. BASELINE CAPTURE : Continued..  Start simulated workload if testing, or wait for production activity  When capture of production load is complete (timeframe up to you, 15 minutes, 1 hour?) stop the trace and SQLDIAG (ctrl-C in SQLDIAG window)  Take final tlog backups, then copy the following from production to the test server (or other server that will analyze the results):  Trace files  SQLDIAG output  Full andTLog backups
  11. 11. DEMO: BASELINE CAPTURE  From RML Help.pdf, pg 17
  12. 12. DEMO : ReadTrace  I’m performing the demos that take some time so I can talk while they run..
  13. 13. PROCESS : OnTest Server (SQL2) Using the ReadTrace tool  Use readtrace.exe from RML Cmd Prompt on the test server (or any other), to process the trace files into replayable RML files  ReadTrace consumes as input trace (.TRC) or (.XEL) files and .CAB or .ZIP files which contain .TRC files from a SQLTrace or Extended Events  It generates .RML (Replay Mark-up Language) files and the Performance Analysis (PerfAnalysis) database. (A populated performance analysis database is required for reporter.exe functionality.)  TIP! Save your command line statements in text file and paste to console.  Track what statements and parameters were used exactly, and when.  Minimize typos.
  14. 14. ReadTrace Continued..  ReadTrace -I<PATH_TO_FIRST_TRACE_FILE> -o<PATH_TO_RML_OUTPUT_FOLDER> -S<DBSERVER> -d<NEW_PERF_ANALYSIS_DB_NAME>  Note: No space between the parameter flag and the value in any RML Utilities commands!  Using ReadTrace this way creates RML files and stores performance in a new performance analysis database  Many options for ReadTrace.exe  limit start time (-b) or end time (-e)  limit databases (-C), application (-A) or host (-H)  trace flags, MARS support, etc.. 5 pages of parameters in RML Help pdf, pg 35-40  ReadTrace –IC:RMLDemoFROMSQL1outputtracetrace01trace01.trc –oC:RMLDemooutputrmlrun01-1 –SSQL2 –dPerfAnalysis_run01_1 -CAdventureworks2008R2  Upon completion, will open the result report using reporter.exe  Also review contents of RML output folder
  15. 15. REPLAY : OnTest Server (SQL2)  Prepare environment on test server  Ensure you won't have access to / affect production!  Restore databases up to point in time that you began the trace  Start SQLDIAG Session, just like on production  sqldiag.exe /OC:RMLDemooutputsqldiagrun01replay01 /IC:RMLDemoconfigSD_Detailed.XML  Also start capture of a replay trace using sp_trace, just like on production  exec @rc = sp_trace_create @TraceID output, 2 /rollover*/, N'C:RMLDemooutputtracerun01' , @maxfilesize, NULL
  16. 16. DEMO : Ostress  Again, I’m performing the demos that take some time so I can talk while they run..
  17. 17. OSTRESS :The Replay Command  OStress is a scalable, ODBC based application which can replay database commands in two modes:  Replay mode orchestrates the order of the commands, to accurately simulate OLTP activity. (i.e. Insert table 1, when done then update table 2, then delete table 3..)  Note that replay mode can take around 5 times longer that original capture, due to orchestration overhead  Stress mode plays back SQL statements as quickly as possible, based on parameters you provide. (i.e. Firehose mode)  No respecting order of statements, not useful for Inserts/Updates/Deletes with Foreign Key relationships.  Many parameters for this tool as well  RML Help.pdf pg 93
  18. 18. Use OstressTo Start Replay  Start replay with ostress.exe from RML Cmd Prompt  ostress -E -SSQL2 -iC:RMLDemooutputrmlrun01*.rml -cC:RMLDemoconfigostress_config.ini -oC:RMLDemooutputostressrun01replay01 -mreplay  (again, no spaces between parameter flags and their values)  The ostress command and its parameters:  -E,-S server info  -i path to RML files to replay  -c Configuration file (copied from RML samples)  -o Output path for replay result  -m Mode is replay.This plays back in order. Also can be stress mode, aka fire hose mode  Once replay complete, stop the trace and SQLDIAG (ctrl-C in SQLDIAG window)
  19. 19. PROCESS : OnTest Server (SQL2)  Now we need to analyze the results of the replay on the test server.  We use readtrace.exe from RML Cmd Prompt again, but this time use -f parameter just to create the performance analysis database, not to create another set of replay files.  ReadTrace -IC:RMLDemooutputtracerun01replay01.trc -f -SSQL2 -dPerfAnalysis_run01_replay01 -C"AdventureWorks2008R2"
  20. 20. ANALYZE : OnTest Server (SQL2)  reporter.exe runs once readtrace.exe is complete. From Properties tab, enter the names of both the performance analysis databases (Baseline and test) for reports that compare results.  Review ER Schema of the PerfAnalysis database. Understand how we can query these tables for our own analysis  Use Performance Analysis Comparison Script.sql  Dave is on github! https://github.com/dave-007/RML-Utilities-Demo
  21. 21. BUTWAIT,THERE'S MORE!  SQLNexus  Visual reports from SQL traces (generated using PSSDIAG script) and SQLDIAG  http://sqlnexus.codeplex.com/  PSSDIAG Manager  RML Utilities big brother. More features  https://diagmanager.codeplex.com/  PAL  Gathers performance monitor logs and generates visual reports.  Compares metrics to thresholds defined by Microsoft geniuses!  https://pal.codeplex.com/
  22. 22. QUESTIONS? sql@davidcobb.net Daveslog.com
  23. 23. RESOURCE LINKS  Description of the Replay Markup Language (RML) Utilities for SQL Server - https://support.microsoft.com/en-us/kb/944837/  Download RML Utilities for SQL Server (x64) CU4 - https://www.microsoft.com/en-us/download/details.aspx?id=4511  Dependency: Download ReportViewer 2008 SP1 - https://www.microsoft.com/en-us/download/confirmation.aspx?id=3841  SQL Nexus - http://sqlnexus.codeplex.com/  SQL Server 2012: RML, XEventViewer and Distributed Replay - http://blogs.msdn.com/b/psssql/archive/2012/04/06/sql-server-2012-rml-xevent-viewer-and-distributed- replay.aspx  AdventureWorks RandomWorkload from Jonathan Kehayias - https://www.sqlskills.com/blogs/jonathan/the-adventureworks2008r2-books-online-random-workload- generator/  Distributed Replay walkthrough - http://blogs.msdn.com/b/mspfe/archive/2012/11/08/using-distributed-replay-to-load-test-your-sql-server-part-1.aspx  RML Demo files on Github: https://github.com/dave-007/RML-Utilities-Demo

×