Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Shopping Around for Query Optimizations? Try the Query Store!

404 views

Published on

Watch the full webinar here: http://bit.ly/1RnbVib

This session will discuss the fantastic new capabilities of Query Store with SQL Server 2016. It will illustrate what it is, why we want it, how we would use it, and how it is BETTER than any other available performance tuning tools. This session will walk through enabling this feature, leveraging its capabilities, and modifying some of the available options. If attendees walk away with nothing else from this session, remember the LIVE Query Statistics (or Execution Plan). This is a game changer and will greatly impact any on-demand query optimization.

Published in: Technology
  • Be the first to comment

Shopping Around for Query Optimizations? Try the Query Store!

  1. 1. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM SQL Server 2016 The Query Store Author: Sean Werick Company: Pragmatic Works
  2. 2. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Who is Pragmatic Works? o 100% Microsoft SQL Server Focused o Team of Microsoft MVPs, Authors, Presenters o Concentrated on Optimizing your Data Lifecycle What is Pragmatic Works? o Software Solutions o Training Offerings o Professional Services Why Pragmatic Works? o Microsoft Gold Partner – Data Platform & Business Intelligence o Microsoft’s only “National Solution Provider” for Business Intelligence o Microsoft Analytics Platform System Implementation Partner of the Year o INC Magazine’s Fastest 5000 Growing Companies o Leader in offerings centered around Microsoft’s emerging technologies About Pragmatic Works
  3. 3. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM • Principal Consultant • Community Speaker • Blogs: www.seanwerick.com • Email: SWerick@pragmaticworks.com About Sean
  4. 4. SQL Server 2016 New Innovations
  5. 5. MobileReports Natural language queryDashboardsApplications StreamingRelational Internal & external  Non-relational NoSQL Orchestration Machine learningModeling Information management Complex event processingData The Microsoft data platform
  6. 6. Cloud first Speed Agility Proven Feedback
  7. 7. All of this results in a better on-premises SQL Server Announcing SQL Server 2016
  8. 8. Do more. Achieve more.
  9. 9. 0100101010110 SQL Server OLTP SQL Server data warehouse ETL In-memory ColumnStore In-memory OLTP Real-time fraud detection In-memory enhancements Operational analytics & enhanced performance Fraud detected 2-24 hrs
  10. 10. dbo.Patients Jane Doe Name 243-24-9812 SSN USA Country Jim Gray 198-33-0987 USA John Smith 123-82-1095 USA dbo.Patients Jane Doe Name 1x7fg655se2e SSN USA Jim Gray 0x7ff654ae6d USA John Smith 0y8fj754ea2c USA Country Result Set Jim Gray Name Jane Doe Name 1x7fg655se2e SSN USA Country Jim Gray 0x7ff654ae6d USA John Smith 0y8fj754ea2c USA dbo.Patients SQL Server ciphertext Query Always Encrypted Help protect data at rest and in motion, on-premises & cloud TrustedApps SELECT Name FROM Patients WHERE SSN=@SSN @SSN='198-33-0987' Result Set Jim Gray Name SELECT Name FROM Patients WHERE SSN=@SSN @SSN=0x7ff654ae6d Column Encryption Key Enhanced ADO.NET Library Column Master Key Client side
  11. 11. Mission critical performance Performance Operational analytics In-memory OLTP for more applications Query data store Native JSON Temporal database support Security Always Encrypted Row Level Security Dynamic Data Masking Availability Enhanced AlwaysOn • 3synchronousreplicasfor autofailoveracrossdomains • Roundrobinloadbalancing ofreplicas • DTCfortransactionalintegrity acrossdatabaseinstances withAlwaysOn Enhanced online operations Scalability Enhanced database caching Support for Windows Server 2016 • 12TBmemorysupport
  12. 12. PolyBase Query relational and non-relational data with T-SQL T-SQL query SQL Server Hadoop Quote: ************************ ********************** ********************* ********************** *********************** $658.39 Jim Gray Name 11/13/58 DOB WA State Ann Smith 04/29/76 ME
  13. 13. Data Scientist Interact directly with data Built-in to SQL Server Data Developer/DBA Manage data and analytics together Built-in advanced analytics In-database analytics at massive scale Example Solutions • Salesforecasting • Warehouse efficiency • Predictive maintenance Relational Data Analytic Library T-SQL Interface Extensibility ? R RIntegration 010010 100100 010101 Microsoft Azure Marketplace New R scripts 010010 100100 010101 010010 100100 010101 010010 100100 010101 010010 100100 010101 010010 100100 010101 • Credit risk protection
  14. 14. SQL Server Report Builder SQL Server BI “Reporting Services” • Power BI Desktop • Mobile reports • Paginated reports Power BI • Power BI Desktop • Microsoft Excel • Mobile reports • Paginated reports Power BI Web Portal Power BI Desktop CONSUMEMANAGECREATE Microsoft Excel “Datazen” Publisher** Report Designer in SQL Server Data Tools Windows App Android App iOS App Report Server Web portal (all BI report types) SharePoint Web
  15. 15. Deeper insights across data Access any data PolyBase Power Query for analytics and reporting Enhanced SSIS • Designersupportforprevious SSISversions • SupportforPowerQuery Scale and manage Enterprise-grade Analysis Services New single SSDT in Visual Studio 2015 Enhanced MDS • Exceladd-in15xfaster • Moregranularsecurityroles • Archivaloptionsfortransactionlogs • Reuseentitiesacrossmodels Powerful insights Built-in advanced analytics Business insights through rich visualizations on mobile devices Enhanced Reporting Services
  16. 16. Order history Name SSN Date Jane Doe cm61ba906fd 2/28/200 5 Jim Gray ox7ff654ae6d 3/18/200 5 John Smith i2y36cg776rg 4/10/200 5 Bill Brown nx290pldo90l 4/27/200 5 5/12/200 Order history Name SSN Date Jane Doe cm61ba906fd 2/28/200 5 Jim Gray ox7ff654ae6d 3/18/200 5 John Smith i2y36cg776rg 4/10/200 5 Bill Brown nx290pldo90l 4/27/200 5 Customer data Product data Order History Stretch to cloud Stretch SQL Server into Azure Stretch warm and cold tables to Azure with remote query processing App Query Microsoft Azure  Jim Gray ox7ff654ae6d 3/18/2005
  17. 17. Hyperscale cloud Hybrid solutions Stretch tables into Azure Power BI with on-premises data Hybrid scenarios with SSIS Enhanced backup to Azure Simplicity Easy migration of on-premises SQL Server Simplified Add Azure Replica Wizard Consistency Common development, management and identity tools Consistent experience from on-premises to Azure
  18. 18. Cortana Analytics Suite Transform data into intelligent action
  19. 19. Introducing Azure SQL Data Warehouse Integration with Power BI, ADF, and Machine Learning services Separate storage & compute Elastic scale Scale-out relational data warehouse
  20. 20. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM What is the Query Store? A SQL Server 2016 (and SQL DB) Feature that can be Enabled • Provides insight on query plan choice and performance • Simplifies performance troubleshooting by quickly finding performance differences caused by query plan changes • Automatically captures a history of queries, plans, and runtime statistics, and retains these for review • Separates data by time windows to better illustrate database usage patterns and understand when query plan changes happened on the server
  21. 21. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Why Use the Query Store? Query Performance Insight Helps Satisfy 2 Main Goals Performance Cost
  22. 22. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM WHAT IS THE REAL COST? Why Use the Query Store? All about the DTUs (PaaS) A Database Transaction Unit (DTU) is a blended measure of the resources allocated at each performance level. • Processor - % Processor Time • Logical Disk - Disk Reads/sec • Logical Disk - Disk Writes/sec • Database - Log Bytes Flushed/sec • Review the list of top resource-consuming queries • Select an individual query to view its details • Click Settings to customize how data is displayed or to show a different time period • Open Index Advisor* (PaaS) and check if any recommendations are available Performance Cost * Index Advisor - https://azure.microsoft.com/en-us/documentation/articles/sql-database-index-advisor/
  23. 23. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM How the Query Store “Works”
  24. 24. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Does It Work with In-Memory OLTP? • SQL Server Query Store allows you to monitor the performance of natively compiled code for workloads running in-memory OLTP • Compile and runtime statistics are collected and exposed the same way as for disk-based workloads • When you migrate to in-memory OLTP you can continue using Query Store views in SQL Server Management Studio as well as custom scripts you have developed for disk-based workloads before migration • This saves your investment in learning Query Store technology and makes it generally usable for troubleshooting all type of workloads
  25. 25. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM How to Use the Query Store In Just 3 Easy Steps, You TOO can be a DBA!! Okay, okay. This may take a bit more effort and knowledge.
  26. 26. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM There are 2 Ways to Enable Query Store By Using the Query Store Page in SSMS Setting up the Query Store In Object Explorer, right-click a database, and then click Properties
  27. 27. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM By Using the Query Store Page in SSMS Setting up the Query Store In the Database Properties dialog box, select the Query Store page Select related attributes • General o Operation Mode (Read or Read/Write) • Monitoring o Data Flush Interval (Manual Entry in Minutes) o Statistics Collection (5 mins, 10 mins, 15 mins, 30 mins, 1 hr, 1 day) • Max Size o Query Store Capture Mode (All or Auto) o Size Based Cleanup Mode (Off or Auto) o Stale Query Threshold (Manual Entry in Days)
  28. 28. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM By Using T-SQL Statements • Use the ALTER DATABASE statement ALTER DATABASE AdventureWorks2012 SET QUERY_STORE = ON; Ability to Setting up the Query Store
  29. 29. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM GUI Option T-SQL Option Operation Mode (Requested) OPERATION_MODE Data Flush Interval (Minutes) DATA_FLUSH_INTERVAL_SECONDS Statistics Collection Interval INTERVAL_LENGTH_MINUTES Max Size (MB) MAX_STORAGE_SIZE_MB Query Store Capture Mode QUERY_CAPTURE_MODE Size Based Cleanup Mode SIZE_BASED_CLEANUP_MODE Stale Query Threshold (Days) CLEANUP_POLICY (STALE_QUERY_THRESHOLD_DAYS) MAX_PLANS_PER_QUERY Query Store Options with T-SQL ALTER DATABASE [db_name] SET QUERY_STORE ( [Query Store options] ); -- Example ALTER DATABASE AdventureWorks SET QUERY_STORE ( MAX_STORAGE_SIZE_MB = 250 );
  30. 30. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM sys.query_store_plansys.query_store_runtime_stats sys.query_context_settings sys.query_store_runtime_stats_interval sys.query_store_query sys.query_store_query_text plan_id runtime_stats_interval_id query_id context_settings_id query_text_id DATA MODEL sys.database_query_store_options What Information Does It Provide?
  31. 31. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM What Information Does It Provide? *Additional Detail - https://msdn.microsoft.com/en-us/library/dn818146.aspx Query Store View Description sys.database_query_store_options Query Store options for this database sys.query_context_settings Semantics affecting context settings associated with a query (The same query text compiled under different settings may produce different results [depending on the underlying data]) sys.query_store_plan Each execution plan associated with a query sys.query_store_query Information about the query and its associated overall aggregated runtime execution statistics sys.query_store_query_text Transact-SQL text and the SQL handle of the query sys.query_store_runtime_stats Information about runtime execution statistics information for the query sys.query_store_runtime_stats_interval Information about the start and end time of each interval over which runtime execution statistics information for a query has been collected
  32. 32. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Query Store Best Practices  Information and performance are generally a give and take, but there ARE some best practices  Every single setting in the Query Store has associated costs and benefits • Treat Query Store performance tuning like an agile project with Sprints o Sprint 1 – Keep Operation Mode in READ/WRITE mode for information collection  This reduces write time, and subsequent overall database performance, for data collection as no reporting is occurring o Sprint 2 – Change Operation Mode to READ mode for analysis and actual performance tuning  This it increases report performance because no writes are occurring. This also results in no “dirty reads” because of the asynchronous writes
  33. 33. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Query Store Best Practices • Flush the data at specific business case intervals. For example, if performance of the DB seems to be poor around end of the month due financial reporting, consider using the Agile methodology to complete one iteration, then flush the data before the next iteration. • Keep the Max Size small enough to keep the reporting relatively snappy, however, keep enough data to capture all of the business needs • Keep an accurate Cleanup Policy as the business and reporting solution evolves • Collect statistics as frequently as possible to see exactly which execution plans are being executed. If the statistics are not updated for some time, you may be looking at an old execution plan because the statistics have changed  Information and performance are generally a give and take, and it ultimately relies on each and every business and its processes
  34. 34. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Query Store Reporting Azure SQL DB • Cloud offering (PaaS) • Learn and adapt dynamically with your app • Scale performance on the fly, without app downtime • Build multitenant apps with customer isolation and efficiency • Work within your preferred development environments • Help protect and secure your app data
  35. 35. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Query Store Reporting The top queries view opens and the top CPU consuming queries are listed • The top line shows overall DTU% for the database, while the bars show CPU% consumed by the selected queries during the selected interval • The bottom grid represents aggregated information for the visible queries o Average CPU per query during observable interval o Total duration per query o Total number of executions for a particular query
  36. 36. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Query Store Reporting Viewing Individual Query Details Click any query in the list of top queries. The details view opens and the queries CPU consumption is broken down over time. Click around the chart for details. The top line is overall DTU%, and the bars are CPU% consumed by the selected query. Review the data to see detailed metrics including duration, number of executions, and resource utilization percentage for each interval the query was running.
  37. 37. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Query Store On Premise Reporting On Premise Solution?
  38. 38. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Query Store Dashboards SSMS
  39. 39. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Custom Views and SQL HOW? sys.query_store_plansys.query_store_runtime_stats sys.query_context_settings sys.query_store_runtime_stats_interval sys.query_store_query sys.query_store_query_text plan_id runtime_stats_interval_id query_id context_settings_id query_text_id sys.database_query_store_options
  40. 40. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Query Store Admin with T-SQL Stored Procedure Description sp_query_store_flush_db Flushes the in-memory portion of the Query Store data to disk sp_query_store_reset_exec_stats Clears the runtime stats for a specific query plan from the query store sp_query_store_force_plan Enables forcing a particular plan for a particular query. When a plan is forced for a particular query, every time SQL Server encounters the query, it tries to force the plan in the optimizer. If plan forcing fails, a XEvent is fired and the optimizer is instructed to optimize in the normal way. sp_query_store_unforce_plan Enables unforcing a particular plan for a particular query sp_query_store_remove_plan Removes a single plan from the query store sp_query_store_remove_query Removes the query, as well as all associated plans and runtime stats from the query store
  41. 41. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Forcing Execution Plans OLD WAY 1. Run SET SHOWPLAN_XML ON to return the XML Execution Plan, or alternatively, grab the XML Plan from the sys.dm_exec_text_query_plan DMF) 2. Create the Plan Guide using the sp_create_plan_guide Stored Procedure
  42. 42. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Forcing Execution Plans NEW WAY 1. Click the “Force Plan” button inside one of the Query Store reports 2. Use the sp_query_store_force_plan SP. EXEC sp_query_store_force_plan 136, 146;
  43. 43. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Error State Remember to verify intermittently that the Query Store is still collecting data SELECT actual_state_desc, desired_state_desc, current_storage_size_mb, max_storage_size_mb, readonly_reason, interval_length_minutes, stale_query_threshold_days, size_based_cleanup_mode_desc, query_capture_mode_desc FROM sys.database_query_store_options; If there is a difference between the actual_state_desc and desired_state_desc, this indicates that a change of operations mode occurred automatically. The most common change is for the Query Store to silently switch to read-only mode. In extremely rarely circumstances, Query Store can end up in the ERROR state because of internal errors. When the actual state is read-only, use the readonly_reason column to determine the root cause. Typically, the Query Store transitioned to read-only mode because the size quota was exceeded. In that case the readonly_reason is set to 65536.
  44. 44. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Error State How to Fix Reactive Increase the maximum storage size by using the MAX_STORAGE_SIZE_MB option of ALTER DATABASE Clean up Query Store data ALTER DATABASE [QueryStoreDB] SET QUERY_STORE CLEAR; Change Operation Mode back to READ/WRITE ALTER DATABASE [QueryStoreDB] SET QUERY_STORE (OPERATION_MODE = READ_WRITE); Proactive Apply best practices, specifically related to Query Store size Make sure that most recent data is retained, configure time-based policy to remove stale information regularly Consider setting Query Capture Mode to Auto as it filters out queries that are usually less relevant for your workload
  45. 45. INTELLIGENT DATA SOLUTIONS WWW.PRAGMATICWORKS.COM Questions and Answers

×