This document describes using SQL stored procedures to automate performance checks across multiple SQL Server instances. It provides a code sample to collect wait statistics from linked servers and databases. The summary explains how to set up the necessary permissions and infrastructure to execute the stored procedure across multiple servers from a central location on a scheduled basis.
Over the years, Core Data has gained a pretty bad reputation amongst developers who prefer to use another service like Realm for their local persistence. In this talk I will make an argument for using Core Data and why it's not so bad. I will share some examples of where it's easy to go wrong with Core Data, and how to avoid those pitfalls. I will also quickly go over setting up Core Data in an app and by the end, the audience should have a couple of simple rules that should help them safely integrate Core Data in their apps.
The objective of this tutorial is to demonstrate the implementation of Mule caching strategy with REDIS cache using Spring Data Redis module. Mule caching strategy is associated with Mule Cache scope and it is used to define the actions a cache scope takes when a message enters its subflow. In this tutorial, we will be using a simple use case to show the steps require to cache the query results of an Oracle database table into Redis cache using Spring Data Redis module.
Change tracking is a lightweight solution that provides an efficient change tracking mechanism for applications. Typically, to enable applications to query for changes to data in a database and access information that is related to the changes, application developers had to implement custom change tracking mechanisms. Creating these mechanisms usually involved a lot of work and frequently involved using a combination of triggers, timestamp columns, new tables to store tracking information, and custom cleanup processes.
Over the years, Core Data has gained a pretty bad reputation amongst developers who prefer to use another service like Realm for their local persistence. In this talk I will make an argument for using Core Data and why it's not so bad. I will share some examples of where it's easy to go wrong with Core Data, and how to avoid those pitfalls. I will also quickly go over setting up Core Data in an app and by the end, the audience should have a couple of simple rules that should help them safely integrate Core Data in their apps.
The objective of this tutorial is to demonstrate the implementation of Mule caching strategy with REDIS cache using Spring Data Redis module. Mule caching strategy is associated with Mule Cache scope and it is used to define the actions a cache scope takes when a message enters its subflow. In this tutorial, we will be using a simple use case to show the steps require to cache the query results of an Oracle database table into Redis cache using Spring Data Redis module.
Change tracking is a lightweight solution that provides an efficient change tracking mechanism for applications. Typically, to enable applications to query for changes to data in a database and access information that is related to the changes, application developers had to implement custom change tracking mechanisms. Creating these mechanisms usually involved a lot of work and frequently involved using a combination of triggers, timestamp columns, new tables to store tracking information, and custom cleanup processes.
PostgreSQL High-Performance Cheat Sheets contains quick methods to find performance issues.
Summary of the course so that when problems arise, you are able to easily uncover what are the performance bottlenecks.
MySql Triggers Tutorial - The Webs Academythewebsacademy
A SQL trigger is a set of SQL statements stored in the database catalog. A SQL trigger is executed or fired whenever an event associated with a table occurs e.g., insert, update or delete.
A walkthrough on implementing Always Encrypted Encryption on sensitive information to reduce your attack surface area and develop an active data security posture.
Cassandra v3.0 at Rakuten meet-up on 12/2/2015datastaxjp
Cassandra v3.0 sessions at Cassandra Meet-up at Rakuten Tokyo, Fall 2015. New Functionality (support of JSON, new storage engine, Mview, UDF, UDA etc..)
Take Control of your Integration Testing with TestContainersNaresha K
Slides from my Java2Days conference talk - Take Control of your Integration Testing with TestContainers.
How easy is it to write and maintain integration tests when your system under test interacts with databases, message stores, and other external systems? It can be quite challenging, can’t it? For example, the lack of control over the setup of databases can increase the cost of integration testing. Sometimes we take the route of using an in-memory database instead of the one we employ in the production environment, making the tests less effective. With Docker containers, you can simplify this to some extent by running the setup before invoking your tests. Imagine if you get to control the database environment right from within the test code. TestContainers bring in this convenience.
TestContainers is a tool that you can invoke from your test code. It provides lightweight, throwaway instances of common databases, web browsers(Selenium tests), or anything else that can run in a Docker container. In this presentation, I walk you through how TestContainers is the most effective way to take advantage of Docker containers for integration testing. I present through several use-cases and demonstrate how TestContainers simplifies integration testing.
Inexpensive Datamasking for MySQL with ProxySQL — Data Anonymization for Deve...Ontico
HighLoad++ 2017
Зал «Кейптаун», 8 ноября, 16:00
Тезисы:
http://www.highload.ru/2017/abstracts/3115.html
During this session we will cover the last development in ProxySQL to support regular expressions (RE2 and PCRE) and how we can use this strong technique in correlation with ProxySQL's query rules to anonymize live data quickly and transparently. We will explain the mechanism and how to generate these rules quickly. We show live demo with all challenges we got from the Community and we finish the session by an interactive brainstorm testing queries from the audience.
This is one of the 15 minute "TED" style talk presented as part of the Database Symposium at the ODTUG Kscope18 conference. In this presentation @SQLMaria shares details on 4 useful supplied PL/SQL package with the Oracle Database
PostgreSQL High-Performance Cheat Sheets contains quick methods to find performance issues.
Summary of the course so that when problems arise, you are able to easily uncover what are the performance bottlenecks.
MySql Triggers Tutorial - The Webs Academythewebsacademy
A SQL trigger is a set of SQL statements stored in the database catalog. A SQL trigger is executed or fired whenever an event associated with a table occurs e.g., insert, update or delete.
A walkthrough on implementing Always Encrypted Encryption on sensitive information to reduce your attack surface area and develop an active data security posture.
Cassandra v3.0 at Rakuten meet-up on 12/2/2015datastaxjp
Cassandra v3.0 sessions at Cassandra Meet-up at Rakuten Tokyo, Fall 2015. New Functionality (support of JSON, new storage engine, Mview, UDF, UDA etc..)
Take Control of your Integration Testing with TestContainersNaresha K
Slides from my Java2Days conference talk - Take Control of your Integration Testing with TestContainers.
How easy is it to write and maintain integration tests when your system under test interacts with databases, message stores, and other external systems? It can be quite challenging, can’t it? For example, the lack of control over the setup of databases can increase the cost of integration testing. Sometimes we take the route of using an in-memory database instead of the one we employ in the production environment, making the tests less effective. With Docker containers, you can simplify this to some extent by running the setup before invoking your tests. Imagine if you get to control the database environment right from within the test code. TestContainers bring in this convenience.
TestContainers is a tool that you can invoke from your test code. It provides lightweight, throwaway instances of common databases, web browsers(Selenium tests), or anything else that can run in a Docker container. In this presentation, I walk you through how TestContainers is the most effective way to take advantage of Docker containers for integration testing. I present through several use-cases and demonstrate how TestContainers simplifies integration testing.
Inexpensive Datamasking for MySQL with ProxySQL — Data Anonymization for Deve...Ontico
HighLoad++ 2017
Зал «Кейптаун», 8 ноября, 16:00
Тезисы:
http://www.highload.ru/2017/abstracts/3115.html
During this session we will cover the last development in ProxySQL to support regular expressions (RE2 and PCRE) and how we can use this strong technique in correlation with ProxySQL's query rules to anonymize live data quickly and transparently. We will explain the mechanism and how to generate these rules quickly. We show live demo with all challenges we got from the Community and we finish the session by an interactive brainstorm testing queries from the audience.
This is one of the 15 minute "TED" style talk presented as part of the Database Symposium at the ODTUG Kscope18 conference. In this presentation @SQLMaria shares details on 4 useful supplied PL/SQL package with the Oracle Database
This presentation covers:
The main components in the connectivity path.
Best practices in your code to avoid connectivity issues.
How to solve most common connectivity problems.
common_schema, DBA's framework for MySQLShlomi Noach
An introduction to common_schema, looking at the concepts behind the project and some notable features.
Slides from my talk at Percona Live London, Dec 2012
TROOPERS 20 - SQL Server Hacking Tips for Active Directory EnvironmentsScott Sutherland
During this presentation, I’ll cover common ways to target, exploit, and escalate domain privileges through SQL Servers in Active Directory environments. I’ll also share a msbuild.exe project file that can be used as an offensive SQL Client during red team engagements when tools like PowerUpSQL are too overt.
Where there is Active Directory, there are SQL Servers. In dynamic enterprise environments it’s common to see both platforms suffer from misconfigurations that lead to unauthorized system and sensitive data access. During this presentation, I’ll cover common ways to target, exploit, and escalate domain privileges through SQL Servers in Active Directory environments. I’ll also share a msbuild.exe project file that can be used as an offensive SQL Client during red team engagements when tools like PowerUpSQL are too overt.
ProxySQL and the Tricks Up Its Sleeve - Percona Live 2022.pdfJesmar Cannao'
ProxySQL is well-affirmed into thousands of production environments for the features we all know: multiplexing, query routing, and rewriting to name a few.
Let's go through those use cases which maybe are the least common: from keeping malicious eyes away from your production data to rebuilding your non-production environment, from stopping having hanging transactions to monitor your instance, from query firewalling to changing ProxySQL configuration without a single line of SQL!
When the sky is the limit, ProxySQL gives you some boost!
Microsoft SQL Azure - Developing And Deploying With SQL Azure WhitepaperMicrosoft Private Cloud
SQL Azure is built on the SQL Server’s core engine, so developing against SQL Azure is very similar to developing against on-premise SQL Server. While there are certain features that are not compatible with SQL Azure, most T-SQL syntax is compatible. The MSDN link http://msdn.microsoft.com/en-us/library/ee336281.aspx provides a comprehensive description of T-SQL features that are supported, not supported and partially supported in SQL Azure.
2° Ciclo Microsoft CRUI 3° Sessione: l'evoluzione delle piattaforme tecnologi...Jürgen Ambrosi
L’obiettivo è quello di fare una panoramica dello stato dell’arte sulle tecnologie a supporto dei database. Alcuni esempi sono la tecnologia in-memory integrata con le funzionalità di analisi operative in tempo reale e della tecnologia Always Encrypted per la protezione dei dati utilizzati in locale o durante gli spostamenti. La tecnologia in-memory consente di migliorare di 30 volte le performance delle transazioni utilizzando hardware standard di settore. Inoltre i Big Data e l'analisi sono diventati un importante fattore di differenziazione competitivo, ma la gestione delle enormi quantità di dati correlate a un tempo di attività 24 ore su 24 continua a essere una sfida per l'IT. Oggi è più importante che mai soddisfare a livello aziendale l'esigenza di prestazioni, disponibilità e sicurezza efficace per gestire carichi di lavoro mission-critical a un costo contenuto. Le soluzioni Microsoft fissano un nuovo standard nelle performance mission-critical.
2° Ciclo Microsoft CRUI 3° Sessione: l'evoluzione delle piattaforme tecnologi...
SSMS-waitstats
1. Contact: EliasBlake
prosperdatasystems@gmail.com
www.prosperidata.com
Using SQL Stored Procedures to automate performance checks
Code Sample from SQL Server Management Studio to demonstrate collecting wait statistics.
use YourDatabaseHere;
DECLARE @server nvarchar(100);
DECLARE @db nvarchar(100);
DECLARE @command nvarchar(4000);
DECLARE DBLINKS CURSOR FOR select SERVER,db from YourDatabaseHere.dbo.TableofLinkedServers
--Open the cursor.
OPEN DBLINKS;
-- Loop through the partitions.SQL_Latin1_General_CP1_CI_AS
WHILE (1=1)
BEGIN;
FETCH NEXT
FROM DBLINKS
INTO @server,@db;
IF @@FETCH_STATUS < 0 BREAK;
SET @command = N'INSERT INTO YourDatabaseHere.dbo.waits
(Server,session_id,task_state,wait_duration_ms,max_wait_time_ms,wait_type,waiting_tasks_count,stamp
)
select CASE when a3.session_id != null THEN'''+ @server + ''' END
server,a3.session_id,a3.task_state,a2.wait_duration_ms,a1.max_wait_time_ms,a1.wait_type,a1.waiting_
tasks_count,CASE when a3.session_id != null THEN getdate() ELSE getdate() END as [stamp]
from ' + '['+@server+ ']'+ '.['+@db+ N'].sys.dm_os_waiting_tasks a2 inner join sys.dm_os_tasks a3
on a2.session_id = a3.session_id
inner join sys.dm_os_wait_stats a1
on a2.wait_type = a1.wait_type collate SQL_Latin1_General_CP1_CI_AS
where a2.wait_type in (''CXPACKET'',''ASYNC_IO_COMPLETION'',''ASYNC_NETWORK_IO'')
OR a2.wait_type LIKE ''LCK%'' OR a2.wait_type like ''%LATCH%''
order by a2.wait_duration_ms desc,a2.session_id desc';
EXEC sp_executesql @command;
PRINT N'Executed: ' + @command;
END
CLOSE DBLINKS;
DEALLOCATE DBLINKS;
Thiscode sample assumesanenvironmentwithmultipleSQLServerInstancesbeyondthe 30 that can be monitored
usingthe UCP service (UtilityControl Point) introduced inSQLServer2008 R2 enterprise.Alsohave abackgroundin
Oracle thisis somewhatequivalenttotakingsnapshotsinAWR. AswithUCP there isa requirementfora“super”service
account withsysadmaccesson monitoredinstances butalsolinkedservers andservergroupsregisteredin management
studio. Thiswill notworkinenvironmentswherethe rpcand rpcoutare restricted. Itispossible to install otherinstances
to run UCP on a single serverbut UCPonlyprovidesawindow intothe hardware bydisplayinggrowthof datafiles, CPU
and memory.If more detail isneededforrootcause analysis inside the database andsp_whodoesn’tshedlight, the
waitstats fromthe example canprovide awindow into the processesand sessionID/usersthatcreate unsavory wait
types. Althoughthe SQLServerextendedeventsandProfilerService cancreate histories thisscriptcanbe a goodway to
target whatserversneed attention inacentral way withoutrandomlyrunningtracesandextendedsessionevents. Then
hopefullythe DBA can avoidthe performance impacts of collectingperformance metricswhere it’snotneeded. If using
SQL Server
2. Using SSMS to construct a wait loop or other looping script
The firststepis to putthe scriptintoa storedprocedure as shownina database;as withadventure worksinthe sample.
A SQL AgentJob(notmy preference)withappropriate“exec”permissions couldbe usedinsteadof astoredprocedure.
Althoughthe code can be ploppedintoSQLAgent thatadds a layerof additional permissionheadaches.Creatingthe
storedprocedure alsoallows automationusingsqlcmdfilesand scheduledtasksif youcan’tuse SQL Agent.(Another
article).
Set links and a dedicated domain user
Second addthe superuseraccount to all instancesthatwill be linked.Servergroupswill allow the additionof the super
account to SQL Serversandverificationof permissionsasshowninthe examples.
Note above the querywindowisopenonSQLGRP.
3. Belowthe helploginsextendedprocshowsthe new service accountwasadded.
The linksto remote databasesare authenticatedasthe “super”useror anothersysadmaccount as shownor usingthe
“current securitycontext”.
The last linkpermissionbelow “Be made usingthissecuritycontext”wouldbe usedtoauthenticate againstexternal
database accountsfor Oracle,MySQL,etc.
One the firstdatabase linkiscompletedandsaveditcanbe scriptedtoa querywindow (rightclick)andusingfindand
replace forthe each newSQL ServerInstance couldspeedupthe process. Alsousingawhile loopto pull servernames
fromthe a “MyLinksTable”asshownin the nextstep couldwork to “auto create”links.Thisstepis what allowsthe
creationof a single central table versestablesoneachinstance. Iwill lookmore atlinksandODBC connectorsforOracle
queries andPL/SQLanothertime.
4. Using Server Groups or Central Managed Server
Thirdusingyour registeredservergrouptocreate a querywithselect@@SERVERNAME + name from sys.databases,you
can populate atable or a tempdbtable tocreate a listof sql instances anddatabases.
If you don’twantlinksyoucan create a “waits”table ineach instance andqueryagainstthe servergroup. Alsoyoucould
add sp_configure commandto enable anddisable linkaccessatthe instance level.
5. Reference Table for links and database connect strings
The MyLinksTable will supplythe variables inthe storedprocto run the waitstats scriptagainstthe linkeddatabases
and insertdataintothe inMy in WaitsTable atthe “central” server. Before creatingthe storedprocbelowitisbestto
take the @command scriptSQL and attempta testinsertto the central table.Thishelpsavoidastray bracketor quote
and the testrows can thenbe truncatedif needed.
The excerptabove wouldbe tweakedbasedonsecuritycontrolsandpolicymanagementorfacetsthatmay restrict
remote functionsorsysadmaccess.It ispossible thateachinstance mayneeditsownwaitstable whichcouldbe
queriedlater.
6. SQL Agent and scheduling
Lastlyschedule SQLAgent,taskschedulerorpowershelltorun your storedproc. Manually runningthe storedprocusing
@spYourProcName inthe scriptwindowisagood final test. The resultswill show the following:
Thisprocess takessome doing initially butsetsthe stage forcollectingotherimportantmetrics withstoredprocstopull
fromDMV’s for databases like db_index_XXX_xxxx,db_missing_index_xxxx.Withsome modificationstoremove the
@db database variable serverlevelOSperformance metrics canbe collected fromdm_osviewsand
dm_io_virtual_file_stats views. Thistechniquecanbe valuable tokeepwait,memory,schedulerorIOtrendsovertime.
It has alsobeeninvaluable inindex reorg/rebuild,ADchecks,diskmovesandmore. Alsoif SQLServerstandard,express
or olderversionsare used, standardizingmonitoringwithstoredprocedurescancompensate forlackof enterprise
featuressuchas Resource Governor,BackupCompressionsandUCPand SQL Agent(use sqlcmd)