- Distributed Replay allows replaying a captured workload from multiple client computers to better simulate production loads.
- A controller coordinates the replay across clients to reproduce the original query rates or run in stress test mode faster than original rates.
- It improves on SQL Server Profiler for application compatibility testing, performance debugging, capacity planning, and benchmarking.
- Events are replayed in synchronization mode to match original order, or unsynchronized to stress test without timing constraints.
Introducing 3 FREE Smart solutions for SQL Server (Adi Sapir, Docco Labs)
As Database experts, we work with SQL Server Databases on a daily basis. We face the same problems every SQL Administrator and/or developer does. And – we spend our time writing solutions for these problems! In this session Adi will introduce the following 3, totally FREE solutions:
· ClipTable – A revolutionary new *anything* to SQL Table importer
· Database File Explorer – a much easier way to explore our database->filegroups->files->storage mapping
· Log Table Viewer – a complete client/server logger solution for SQL Server
Introducing 3 FREE Smart solutions for SQL Server (Adi Sapir, Docco Labs)
As Database experts, we work with SQL Server Databases on a daily basis. We face the same problems every SQL Administrator and/or developer does. And – we spend our time writing solutions for these problems! In this session Adi will introduce the following 3, totally FREE solutions:
· ClipTable – A revolutionary new *anything* to SQL Table importer
· Database File Explorer – a much easier way to explore our database->filegroups->files->storage mapping
· Log Table Viewer – a complete client/server logger solution for SQL Server
Hardware planning & sizing for sql serverDavide Mauri
Purchasing a dedicated server to SQL Server is still a necessary operation. The cloud is a great choice but if you need to create a data warehouse of non-trivial size or if you have the need for optimal performance and control of your production database server, the choice of on-premise server is still an optimal choice. So, how not to throw away money on unnecessary hardware? In this session we will see how each component works together to form a balanced hardware (this is the key word!), without bottlenecks, maximizing the investment made. We'll talk about SAN, CPU, HBA, Fibre Channel, Memory and everything you thought you knew well...
Deep dive into Clustered Columnstore structures with information on compression algorithms, compression types, locking and dictionaries, as well as the Batch Processing Mode.
En este diapositivas der Microsoft podemos ver qué aporta SQL 2014 en áreas como: Tablas optimizadas en memòria, Cambios en estimacion de la cardinalidad, Cifrado de los Backups, Mejoras en arquitectures, Always On, Cambios en Resource Governor, Data files en Azure.
MySQL Performance Tuning: Top 10 Tips intended for PHP, Ruby and Java developers on performance tuning and optimization of MySQL. We will cover the deadly mistakes to be avoided. We will take real life examples of optimizing application many times. Here is the summary of what we intend to cover:
• Selection of Storage Engine
• Schema Optimization
• Server Tuning
• Hardware Selection and Tuning
• Effective uses of Index, when to use and when not to use.
• Partitions
• Speeding up using Stored Procedures
• Implementing prepared statements?
• Deadly Sins to be avoided
• Performance Tuning and Benchmarking Tools
Introduction into the world of Clustered Columnstore Indexes in SQL Server 2014, with explanations of the basic structures and functionalities.
Available data types, limitations and differences to SQL Server 2012 Nonclustered Columnstore Indexes are all described here
Database as a Service on the Oracle Database Appliance PlatformMaris Elsins
Speaker: Marc Fielding, Co-speaker: Maris Elsins.
Oracle Database Appliance provides a robust, highly-available, cost-effective, and surprisingly scalable platform for database as a service environment. By leveraging Oracle Enterprise Manager's self-service features, databases can be provisioned on a self-service basis to a cluster of Oracle Database Appliance machines. Discover how multiple ODA devices can be managed together to provide both high availability and incremental, cost-effective scalability. Hear real-world lessons learned from successful database consolidation implementations.
Hardware planning & sizing for sql serverDavide Mauri
Purchasing a dedicated server to SQL Server is still a necessary operation. The cloud is a great choice but if you need to create a data warehouse of non-trivial size or if you have the need for optimal performance and control of your production database server, the choice of on-premise server is still an optimal choice. So, how not to throw away money on unnecessary hardware? In this session we will see how each component works together to form a balanced hardware (this is the key word!), without bottlenecks, maximizing the investment made. We'll talk about SAN, CPU, HBA, Fibre Channel, Memory and everything you thought you knew well...
Deep dive into Clustered Columnstore structures with information on compression algorithms, compression types, locking and dictionaries, as well as the Batch Processing Mode.
En este diapositivas der Microsoft podemos ver qué aporta SQL 2014 en áreas como: Tablas optimizadas en memòria, Cambios en estimacion de la cardinalidad, Cifrado de los Backups, Mejoras en arquitectures, Always On, Cambios en Resource Governor, Data files en Azure.
MySQL Performance Tuning: Top 10 Tips intended for PHP, Ruby and Java developers on performance tuning and optimization of MySQL. We will cover the deadly mistakes to be avoided. We will take real life examples of optimizing application many times. Here is the summary of what we intend to cover:
• Selection of Storage Engine
• Schema Optimization
• Server Tuning
• Hardware Selection and Tuning
• Effective uses of Index, when to use and when not to use.
• Partitions
• Speeding up using Stored Procedures
• Implementing prepared statements?
• Deadly Sins to be avoided
• Performance Tuning and Benchmarking Tools
Introduction into the world of Clustered Columnstore Indexes in SQL Server 2014, with explanations of the basic structures and functionalities.
Available data types, limitations and differences to SQL Server 2012 Nonclustered Columnstore Indexes are all described here
Database as a Service on the Oracle Database Appliance PlatformMaris Elsins
Speaker: Marc Fielding, Co-speaker: Maris Elsins.
Oracle Database Appliance provides a robust, highly-available, cost-effective, and surprisingly scalable platform for database as a service environment. By leveraging Oracle Enterprise Manager's self-service features, databases can be provisioned on a self-service basis to a cluster of Oracle Database Appliance machines. Discover how multiple ODA devices can be managed together to provide both high availability and incremental, cost-effective scalability. Hear real-world lessons learned from successful database consolidation implementations.
Take an in-depth look at data warehousing with Amazon Redshift and get answers to your technical questions. We will cover performance tuning techniques that take advantage of Amazon Redshift's columnar technology and massively parallel processing architecture. We will also discuss best practices for migrating from existing data warehouses, optimizing your schema, loading data efficiently, and using work load management and interleaved sorting.
AWS June 2016 Webinar Series - Amazon Redshift or Big Data AnalyticsAmazon Web Services
Analyzing big data quickly and efficiently requires a data warehouse optimized to handle and scale for large datasets. Amazon Redshift is a fast, petabyte-scale data warehouse that makes it simple and cost-effective to analyze big data for a fraction of the cost of traditional data warehouses. By following a few best practices, you can take advantage of Amazon Redshift’s columnar technology and parallel processing capabilities to minimize I/O and deliver high throughput and query performance. This webinar will cover techniques to load data efficiently, design optimal schemas, and tune query and database performance.
Learning Objectives:
Get an inside look at Amazon Redshift's columnar technology and parallel processing capabilities
Learn how to migrate from existing data warehouses, optimize schemas, and load data efficiently
Learn best practices for managing workload, tuning your queries, and using Amazon Redshift's interleaved sorting features
Take an in-depth look at data warehousing with Amazon Redshift and get answers to your technical questions. We will cover performance tuning techniques that take advantage of Amazon Redshift's columnar technology and massively parallel processing architecture. We will also discuss best practices for migrating from existing data warehouses, optimizing your schema, loading data efficiently, and using work load management and interleaved sorting.
A quick tour in 16 slides of Amazon's Redshift clustered, massively parallel database.
Find out what differentiates it from the other database products Amazon has, including SimpleDB, DynamoDB and RDS (MySQL, SQL Server and Oracle).
Learn how it stores data on disk in a columnar format and how this relates to performance and interesting compression techniques.
Contrast the difference between Redshift and a MySQL instance and discover how the clustered architecture may help to dramatically reduce query time.
Optimizing Query is very important to improve the performance of the database. Analyse query using query execution plan, create cluster index and non-cluster index and create indexed views
Geek Sync I Need for Speed: In-Memory Databases in Oracle and SQL ServerIDERA Software
You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/S6MG50A5ok5
Microsoft introduced IN-MEMORY OLTP, widely referred to as “Hekaton” in SQL Server 2014. Hekaton allows for the creation of fully transactionally consistent memory-resident tables designed for high concurrency and no blocking. With SQL 2016, many of the original restrictions and limitations of this feature have been reduced. IDERA’s Vicky Harp will give an overview of this feature, including how to compile T-SQL code into machine code for an even greater performance boost.
There’s also been a lot of buzz about Oracle 12c’s new IN-MEMORY COLUMN STORE. Oracle ACE Bert Scalzo will cover this new feature, how it works, it’s benefits, scripts to measure/monitor it and more. He will also touch on performance observations from benchmarking this new feature against more traditional SGA memory allocations plus Oracle 11g R2’s Database Smart Flash Cache. All findings, scripts and conclusions from this exercise will be shared. In addition, two very popular database benchmarking tools will be highlighted.
Oracle 12c New Features For Better PerformanceZohar Elkayam
Oracle 12cR1 and 12cR2 came with some great features for better performance and scaling. In this session we will talk about some of the new features that might improve performance greatly: Optimizer changes, adaptive plans improvements, changes to statistics gathering and we'll get to know Oracle 12cR2 new sharding option
On the agenda:
- Oracle Database In Memory (Column Store)
- Oracle Sharding (12.2.0.1)
- Optimizer changes in 12c
- Statistics changes in 12c.
Presented first at ilOUG - Israel Oracle User Group meetup in February 2017.
[including promised hidden slide.. :) ]
Big Data and New Challenges for DBAs (Michael Naumov, LivePerson)
Hadoop has become a popular platform for managing large datasets of structured and unstructured data. It does not replace existing infrastructures, but instead augments them. Most companies will still use relational databases for transactional processing and low-latency queries, but can benefit from Hadoop for reporting, machine learning or ETL. This session will cover:
What is Hadoop and why do I care?
What do people do with Hadoop?
How can SQL Server DBAs add Hadoop to their architecture?
Common Errors That Effect Performance (Adi Cohen, Naya-Tech)
There are a few common errors that have a negative effect on performance. In this session we will review some of them, see why they impact performance and provide alternative solutions. Among the issues we will cover are:
· Misunderstanding of the query plan when using procedures
· Query plan differences between procedures and ad-hoc batches
· The differences between a temporary table and a table variable
· And many more…
Who is afraid of Columnstore Indexes? (Michael Zilberstein, DB-Art)
This talk describes new SQL Server 2012 feature called "columnstore index". In this session we will learn about the differences between columnstore indexes and B-Tree indexes we are used to work with. We will see when it is best to use and when not to use this new index. We will cover limitations that columnstore index imposes on the tables that use it and how to live with those limitations. Like in all my sessions, I won't let you go without some internals – how columnstore index is organized on a physical level and how Query Processor works this new type of index. And of course Demos, Demos, Demos…
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
3. Improved Data Warehouse Query performance
• Columnstore indexes provide an easy way
to significantly improve data warehouse
and decision support query performance
against very large data sets
• Performance improvements for “typical”
data warehouse queries from 10x to 100x
• Ideal candidates include queries against
star schemas that use filtering,
aggregations and grouping against very
large fact tables
3
4. What Happens When…
• You need to execute high performance DW
queries against very large data sets?
• In SQL Server 2008 and SQL Server 2008 R2
• OLAP (SSAS) MDX solution
• ROLAP and T-SQL + intermediate summary tables,
indexed views and aggregate tables
• Inherently inflexible
4
5. What Happens When…
• You need to execute high performance DW
queries against very large data sets?
• In SQL Server 2012
• You can create a columnstore index on a very large fact table
referencing all columns with supporting data types
• Utilizing T-SQL and core Database Engine functionality
• Minimal query refactoring or intervention
• Upon creating the columnstore index, your table becomes
“read only” – but you can still use partitioning to switch in
and out data OR drop/rebuild indexes periodically
5
6. How Are These Performance
Gains Achieved?
• Two complimentary technologies:
• Storage
• Data is stored in a compressed columnar data format
(stored by column) instead of row store format (stored
by row).
• Columnar storage allows for less data to be accessed when
only a sub-set of columns are referenced
• Data density/selectivity determines how compression friendly
a column is – example “State” / “City” / “Gender”
• Translates to improved buffer pool memory usage
6
7. How Are These Performance
Gains Achieved?
• Two complimentary technologies:
• New “batch mode” execution
• Data can then be processed in batches (1,000 row
blocks) versus row-by-row
• Depending on filtering and other factors, a query may
also benefit by “segment elimination” - bypassing
million row chunks (segments) of data, further reducing
I/O
7
9. Batch Mode
• Allows processing of 1,000 row blocks as an
alternative to single row-by-row operations
• Enables additional algorithms that can reduce CPU
overhead significantly
• Batch mode “segment” is a partition broken into
million row chunks with associated statistics used
for Storage Engine filtering
9
10. Batch Mode
• Batch mode can work to further improve query
performance of a columnstore index, but this
mode isn’t always chosen:
• Some operations aren’t enabled for batch mode:
• E.g. outer joins to columnstore index table / joining strings /
NOT IN / IN / EXISTS / scalar aggregates
• Row mode might be used if there is SQL Server
memory pressure or parallelism is unavailable
• Confirm batch vs. row mode by looking at the
graphical execution plan
1
11. Columnstore format + batch
mode Variations
• Performance gains can come from a
combination of:
• Columnstore indexing alone + traditional row
mode in QP
• Columnstore indexing + batch mode in QP
• Columnstore indexing + hybrid of batch and
traditional row mode in QP
1
13. Good Candidates for
Columnstore Indexing
• Table candidates:
• Very large fact tables (for example – billions of
rows)
• Larger dimension tables (millions of rows) with
compression friendly column data
• If unsure, it is easy to create a columnstore index
and test the impact on your query workload
1
14. Good Candidates for
Columnstore Indexing
• Query candidates (against table with a columnstore index):
• Scan versus seek (columnstore indexes don’t support seek
operations)
• Aggregated results far smaller than table size
• Joins to smaller dimension tables
• Filtering on fact / dimension tables – star schema pattern
• Sub-set of columns (being selective in columns versus returning
ALL columns)
• Single-column joins between columnstore indexed table and
other tables
1
15. Defining the Columnstore
Index
• Index type
• Columnstore indexes are always non-clustered and non-unique
• They cannot be created on views, indexed views, sparse columns
• They cannot act as primary or foreign key constraints
• Column selection
• Unlike other index types, there are no “key columns”
• Instead you choose the columns that you anticipate will be used in your queries
• Up to 1,024 columns – and the ordering in your CREATE INDEX doesn’t matter
• No concept of “INCLUDE”
• No 900 byte index key size limit
• Column ordering
• Use of ASC or DESC sorting not allowed – as ordering is defined via columnstore
compression algorithms
1
16. Supported Data Types
• Supported data types
• Char / nchar / varchar / nvarchar
• (max) types, legacy LOB types and FILESTREAM are not supported
• Decimal/numeric
• Precision greater than 18 digits NOT supported
• Tinyint, smallint, int, bigint
• Float/real
• Bit
• Money, smallmoney
• Date and time data types
• Datetimeoffset with scale > 2 NOT supported
1
18. Maintaining Data in a
Columnstore Index
• Once built, the table becomes “read-only” and
INSERT/UPDATE/DELETE/MERGE is no longer allowed
• ALTER INDEX REBUILD / REORGANIZE not allowed
• Other options are still supported:
• Partition switches (IN and OUT)
• Drop columnstore index / make modifications / add
columnstore index
• UNION ALL (but be sure to validate performance)
1
19. Limitations
• Columnstore indexes cannot be used in conjunction with
• Change Data Capture and Change Tracking
• Filestream columns (supported columns from same table are supported)
• Page, row and vardecimal storage compression
• Replication
• Sparse columns
• Data type limitations
• Binary / varbinary / ntext / text / image / varchar (max) / nvarchar (max) /
uniqueidentifier / rowversion / sql_variant / decimal or numeric with precesion > 18
digits / CLR types / hierarchyid / xml / datetimeoffset with scale > 2
• You can prevent a query from using the columnstore index using the
IGNORE_NONCLUSTERED_COLUMNSTORE_INDEX query hint
1
21. Summary
• SQL Server 2012 offers significantly faster query performance for
data warehouse and decision support scenarios
• 10x to 100x performance improvement depending on the schema and
query
• I/O reduction and memory savings through columnstore compressed storage
• CPU reduction with batch versus row processing, further I/O reduction if
segmentation elimination occurs
• Easy to deploy and requires less management than some legacy
ROLAP or OLAP methods
• No need to create intermediate tables, aggregates, pre-processing and cubes
• Interoperability with partitioning
• For the best interactive end-user BI experience, consider Analysis
Services, PowerPivot and Crescent
2
23. Improvements for Table Partitioning
• SQL Server 2012 RTM supports up to 15,000 partitions
• No need for a service pack to gain functionality
• Partition statistics are created using a row sub-set sampling
when an index is rebuilt or created - versus scanning all rows to
create the statistics
• Additional partition management wizard options can assist
with executing or scripting out common partition operations
• Partitioning can be used in conjunction with tables that have a
columnstore index in order to switch in and out data
2
26. What Happens When…
• You need to partition data by day for 3 years of
data or more? Or you need to partition data by
hour for a year’s worth of data?
• In SQL Server 2008 & SQL Server 2008 R2
• Limited to 1,000
• Unless you installed 2008 SP2 or 2008 R2 SP1 – which allowed for
15,000 partitions when enabled via sp_db_increased_partitions
• This prevented moving from 2008 SP2 to 2008 R2 RTM
• Also prevented moving SQL Server 2008 SP2 database with
15,000 partitions enabled to SQL Server 2008 or 2008 SP1
• Created other restrictions for Log Shipping, Database
Mirroring, Replication, SSMS manageability
2
27. What Happens When…
• You need to partition data by day for 3 years
of data or more? Or you need to partition
data by hour for a year’s worth of data?
• In SQL Server 2012
• 15,000 partitions are supported in RTM (no SP
required)
2
28. 15,000 Partitions
• You now have the option – as appropriate
• Flexibility to partition based on common data
warehousing increments (hours / days / months)
without hitting the limit
• This doesn’t remove the need for an archiving strategy or
mindful planning
• You have native support for log shipping, availability
groups, database mirroring, replication and SSMS
management
2
29. 15,000 Partitions
• Exceptions:
• > 1000 partitions for x86 is permitted but not
supported
• > 1000 partitions for non-aligned indexes is
permitted but not supported
• For both exceptions – the risk is in degraded
performance and insufficient memory
2
30. What Happens When…
• Your partitioned index is rebuilt or created:
• In SQL Server 2008 and SQL Server 2008 R2
• All table rows are scanned in order to create the
statistics histogram
• In SQL Server 2012
• A default sampling algorithm is used instead
• May or may not have an impact on performance
• You can still choose to scan all rows by using CREATE STATISTICS
or UPDATE STATISTICS with FULLSCAN
3
35. Summary
• SQL Server 2012 offers
• An increased number of partitions, helping address
common data warehouse requirements
• Prevention of lock starvation during SWITCH
operations
• Reduced statistics generation footprint (not scanning
ALL rows by default)
• An enhanced manageability experience, enabling
wizard-based SWITCH IN and SWITCH OUT assistance
3
37. OVER Clause Windowing
• Some existing queries do not optimize well
• Example: Details of orders and days since
previous order of each product
38. OVER Clause Windowing
-- Traditional approach
SELECT rs.ProductKey,rs.OrderDateKey,rs.SalesOrderNumber,
rs.OrderDateKey-(
SELECT TOP(1) prev.OrderDateKey
FROM dbo.FactResellerSales AS prev
WHERE rs.ProductKey=prev.ProductKey
AND prev.OrderDateKey<=rs.OrderDateKey
AND prev.SalesOrderNumber<rs.SalesOrderNumber
ORDER BY prev.OrderDateKey DESC,prev.SalesOrderNumber
DESC)
AS DaysSincePrevOrder
FROM dbo.FactResellerSales AS rs
ORDER BY rs.ProductKey,rs.OrderDateKey,rs.SalesOrderNumber;
39. OVER Clause Windowing
-- Windowed approach
SELECT ProductKey,OrderDateKey,SalesOrderNumber,
• Some existing queries do not optimize well
OrderDateKey-LAG(OrderDateKey)
OVER (PARTITION BY ProductKey
• Example: Details of orders and days since
ORDER BY OrderDateKey,SalesOrderNumber)
AS DaysSincePrevOrder
previous order of each product
FROM dbo.FactResellerSales AS rs
ORDER BY ProductKey,OrderDateKey,SalesOrderNumber;
40. OVER Clause Windowing
• Some existing queries do not optimize well
• Example: Details of orders and days since
previous order of each product
42. Sequences
•CREATE SEQUENCE Booking.BookingID AS INT
User-defined object
START WITH 20001
• INCREMENT BY 10; particular table
Not tied to any
•CREATE TABLE Booking.FlightBooking in a single table
Not restricted to being used
(FlightBookingID INT PRIMARY KEY CLUSTERED
•DEFAULT (NEXT VALUE FOR Booking.BookingID),
...
Eases migration from other database engines
43. Sequence: Cache or No Cache
• Cache can increases performance for
applications that use sequence objects by
minimizing the number of disk IOs that are
required to generate sequence numbers
44. Cache Example
• If the Database Engine is stopped after you use 22 numbers, the next
intended sequence number in memory (23) is written to the system
tables, replacing the previously stored number.
• If the Database Engine stops abnormally for an event such as a power
failure, the sequence restarts with the number read from system tables
(39). Any sequence numbers allocated to memory (but never requested
by a user or application) are lost. This functionality may leave gaps, but
guarantees that the same value will never be issued two times for a
single sequence object unless it is defined as CYCLE or is manually
restarted.
45. Cache default?
• If the cache option is enabled without
specifying a cache size, the Database Engine
will select a size
• “Don’t count on it being consistent. Microsoft
might change the method of calculating the
cache size without notice.”
48. Improved Benchmarking and
Testing
• Benchmarking and testing are improved through
implementation of:
• Distributed Replay Controller
• Support for multiple Distributed Replay Clients
49. What happens when…
• The business needs to perform application
compatibility testing prior to performing an
upgrade, performance debugging of a highly
concurrent workload, system capacity planning,
or benchmark analysis of a database workload
• In SQL Server 2008
• SQL Server Profiler may be used to replay a captured trace
against an upgraded test environment from a single computer
• Event replay does not follow original query rates
50. What happens when…
• In SQL Server 2012
• Distributed Replay can be used to replay a workload
from multiple computers and better simulate a mission-
critical workload
• Replay can be configured to reproduce original query
rates, or to run in stress test mode where the rate of
replay occurs faster than the original query rate
51. Distributed Replay
Components
• Administration Tool
• Command line application that talks to the Replay Controller
• Replay Controller
• Computer running the “SQL Server Distributed Replay Controller” service which is used to control
the Replay Clients
• Replay Clients
• One or more computers which run the “SQL Server Distributed Replay Client” service
• Target Server
• SQL Server instance that the replay is directed towards
DBA
DBA Replay
Replay SQL Server
Workstation Controller Target SQL
Workstation Controller Instance
Server
Replay
Replay Instance
Clients
Clients
52. Distributed Replay Process
• Event Capture
• Events are captured using SQL Server Profiler or a server side trace
based on the Replay Trace template
• Preprocessing
• Trace data is parsed into an intermediate file for replay
• Specifies whether system session activities are included in the replay
and the max idle time setting
• Event Replay
• Intermediate file divided among the replay clients
• After clients receive dispatch data, controller launches and
synchronizes the replay operation
• Each client can record the replay activity to a local result trace file
53. Distributed Replay Sequencing
Modes
• Synchronization mode
(Application compatibility and performance
testing)
• Events are replayed in the order in which they were
submitted during the capture, within and across
connections based on the events timestamp
• The replay engine will try to emulate the original query
rate observed during the capture
54. Distributed Replay Sequencing
Modes
• Stress mode
(Stress testing and capacity planning or forecasting)
• No order or time synchronization across clients
• Submit order is only maintained within each connection allowing the
replay engine to drive more load against SQL Server than in
synchronization mode
• ConnectTimeScale and ThinkTimeScale parameters adjust the degree of
stress during replay
• Actual connect time is multiplied by ConnectTimeScale/100 to determine replay
connect time
• Actual think time is multiplied by ThinkTimeScale/100 to determine replay think
time
55. Summary
• SQL Server 2012 offers better benchmarking and
testing
• Distributed Replay supports multiple replay clients
allowing for higher scalability during the replay process
• Replay operations can match original query rates for
more accurate analysis of changes to the environment
57. What happens when…
• The business wants to:
• Make use of the mostly-unused
failover server(s) for reporting
• Against real-time business data
58. SQL Server 2008 R2 or prior
• Database mirroring required snapshot management of
the mirrored databases for reporting purposes
• Snapshot data does not change requiring a new snapshot to
keep data up to date, plus connection migration to the new
snapshot
• Snapshots exist until cleaned up, even after failover occurs
• Reporting workload can block database mirroring process
• Log shipping using RESTORE … WITH STANDBY provides
near real-time access to business data
• Log restore operations require exclusive access to the database
59. In SQL Server 2012
• In SQL Server 2012
• AlwaysOn Readable Secondaries enable read-only
access for offloading reporting workloads
• Read workload on Readable Secondaries does not
interfere with data transfer from primary replica
• Readable Secondaries can be used for offloading
backup operations
61. Readable Secondary
Client Connectivity
• Client connection behavior determined by the Availability Group Replica
option
• Replica option determines whether a replica is enabled for read access when in a
secondary role and which clients can connect to it
• Choices are:
• No connections
• Only connections specifying Application Intent=ReadOnly connection property
• All connections
• Read-only Routing enables redirection of client connection to new
readable secondary after a failover
• Connection specifies the Availability Group Listener Virtual Name plus Application
Intent=ReadOnly in the connection string
• Possible for connections to go to different readable secondaries if available to
balance read-only access
62. Readable secondary
Readonly routing
Availability Group
Listener
• Client connects to the
Availability Group Listener
virtual name
• Standard connections are
routed to the Primary server
for read/write operations
• ReadOnly connections are
routed to a readable
secondary based on ReadOnly
routing configuration
63. Query Performance on the Secondary
• Challenges:
• Query workloads typically require index/column statistics so the query optimizer can formulate an
efficient query plan
• Read-only workloads on a secondary replica may require different statistics than the workload on the
primary replica
• Users cannot create different statistics themselves (secondaries can’t be modified)
• Solution:
• SQL Server will automatically create required statistics, but store them as temporary statistics in
tempdb on the secondary node
• If different indexes are required by the secondary workload, these must be
created on the primary replica so they will be present on the secondaries
• Care should be taken when creating additional indexes that maintenance overhead does not affect
the workload performance on the primary replica
64. Offloading Backups To a Secondary
• Backups can be done on any replica of a database to offload I/O
from primary replica
• Transaction log backups, plus COPY_ONLY full backups
• Backup jobs can be configured on all replicas and preferences set
so that a job only runs on the preferred replica at that time
• This means no script/job changes are required after a failover
• Transaction log backups done on all replicas form a single log chain
• Database Recovery Advisor tool helps with restoring backups from
multiple Secondaries
65. Workload Impact on the Secondary
• Read-only workloads on mirror database using traditional database mirroring can
block replay of transactions from the principal
• Using Readable Secondaries, the reporting workload uses snapshot isolation to
avoid blocking the replay of transactions
• Snapshot isolation avoids read locks which could block the REDO background
thread
• The REDO thread will never be chosen as the deadlock victim, if a deadlock
occurs
• Replaying DDL operations on the secondary may be blocked by schema locks held by
long running or complex queries
• XEvent fires which allows programmatic termination/resumption of reporting
• sqlserver.lock_redo_blocked event
66. Summary
• SQL Server 2012 allows more efficient use of IT infrastructure
• Failover servers are available for read-only workloads
• Read-only secondaries are updated continuously from the primary
without having to disconnect the reporting workload
• SQL Server 2012 can improve performance of workloads
• Reporting workloads can be offloaded to failover servers, improving
performance of the reporting workload and the main workload
• Backups can be offloaded to failover servers, improving performance of
the main workload