Full blown of useful and highly technical information on SQL Server 2005 and 2008 High Availabilty features.
Included benchmarks results on: SQL Server 2005 vs. 2008 and Conventional vs. Solid State Devices.
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
SQL Server 2008 Migration Workshop 04/29/2009
1. Database Mirroring
Database Snapshots
SQL Server 2008
USA & France
Migrating to 2008
Benchmarks 2005 vs 2008
Benchmarking 2008 on SSD 1
Copyright 2009 all rights reserved
2. Agenda
Database Mirroring – SQL Server 2008
Database Mirroring 2005 vs 2008 benchmark
Database Mirroring 2008 SSD benchmark
Database Snapshots 2008 (we might get to it)
SQL Performance & Tuning Tool (SQL Shot)
3. Database Architechs
Principals
Paul Bertucci
Founder Database Architechs
Former Chief Data Architect – Veritas & Symantec
Currently Chief Architect - Autodesk
Author of SQL Server 2000, 2005 & 2008 Unleashed!
Co-Author of ADO.NET in 24 hours
Author MS SQL Server High Availability
Author Sybase Performance & Tuning
Author Sybase Physical DB Design
Veritas SQL Server Performance Series
pbertucci@dbarchitechs.com
Thierry Gerardin
• Managing Principal - Database Architechs
- Management, Research and Development, DB Design and P&T for
MS SQL Server, Sybase and Oracle
• European focus until recently/company growth leader – multiple years
• Expanding DB ARCH consulting practices to USA and Asia
• Sybase P&T and DBA Certified Professional
• Contributor to SQL Server 2005 Unleashed!
tgerardin@dbarchitechs.com 3
Copyright 2009 all rights reserved
4. Database Architechs
Primary Services
Data Analysis
& Modeling
Database Design
Business Intelligence &
Data Delivery
Platforms
Data Architecture
Distributed Data &
Data Replication
Performance &
Tuning
Master Data
Management
High Availability
Database Education
& Training
4
Copyright 2009 all rights reserved
5. Past and Current
Customers
Intel Applied Materials
Cisco Apple Computers
Charles Schwab PG&E
Aplia Thomson Learning
AGIS Sybase
Veritas Symantec Corporation
Honda Motors Nissan Motors
Toshiba Computers Breg International
Juniper Networks CSAA
Safeway Stores Federal Express
Wells Fargo Bank of America
Robert Half Merrill Lynch
Metalinc many others….
6. Database Mirroring?
=
Adventure Adventure
Works DB Works DB
A “complete” copy of a database that is created and
maintained with as up to the second completeness
that is possible a mirror image.
What’s new or improved?
Compression in log transactions across topology
Various other performance enhancements
3 years of production execution success!
6
Copyright 2009 all rights reserved
7. Log Shipping
“Source”
“Destination”
SQL Server 2008
SQL Server 2008
TxnLog
Primary CallOne DB Secondary
backups
Server Server
translog
CallOne DB
BackupCallOne_tlog_200405141120.TRN
Last log shipped
Delay Answer
LogShareCallOne_tlog_200405141120.TRN TxnLog
TxnLog
Restores
Copies
“Monitor”
SQL Server 2008 Delay between
logs loaded
Delay Answer
Monitor
MSDB DB
Server
7
Don’t buildCopyright 2009 all rights reserved deprecated !
on this, it is being
8. Client Client Client Client
Network
A
SQL Server 2008
SQL Server 2008
DB Mirror Server
MirroringAdventure D
Principal Adventure D
Works DB
B Works DB
Server
translog
translog
C
SQL Server xyz
Witness
MSDB DB
Server
8
Copyright 2009 all rights reserved
9. Copy-on-write Technology
The new “copy-on-write” technology that Microsoft has created, is at the core of
the database mirroring capability. A transaction from a client connection to the
principal server (arrow label A) is written to the adventure works database
(label D). Once the transaction is written to the principal servers transaction log,
it is immediately copied (arrow label B) and written to the Mirror Server (also
labeled D). When this physical log record is written to the mirror server, it sends
back an acknowledgement (arrow label C) to the principal of its write success.
This is the “copy-on-write” technology. The end result is that the mirror server
is in the exact same state as the principal server (when the physical log record
has been successfully written on the mirror side).
9
Copyright 2009 all rights reserved
10. DB Mirroring Terms
The Principal database server - is the source of the mirroring. You can mirror one
or more databases on a single SQL Server instance to another SQL Server instance.
You cannot mirror a database on one SQL Server instance to itself (the same SQL
Server instance).
The Mirror database server - will be the recipient of the mirroring from the principal
database server. This mirrored database will be kept in a hot standby mode and
cannot be used directly in any way. In fact, once you have configured database
mirroring, this database will show its status as in continuous “restore mode”.
The Witness database server - is used when you want to be continuously checking
to see if any failures have occurred to the primary database server and to help make
the decision to failover to the mirror database server. It is optional, but a sound way
to configure database mirroring. If you do not identify a witness server, the principal
and mirror are left on their own to decide to fail-over or not.
10
Copyright 2009 all rights reserved
11. DB Mirroring Modes
High-availability mode, High-protection mode, and High-performance mode.
[you must sacrifice levels of protection for performance]
Synchronous operation - a committed transaction will be committed (written) on
both partners of the database mirroring pair. This obviously adds some latency cost
to a complete transaction (it is across two servers). High-availability mode and
High-protection mode use synchronous operations (termed High safety)
Asynchronous operation - transactions commit without waiting for the mirror server
to write the log to disk. This can speed up performance significantly.
High-performance mode uses asynchronous operations.
Whether the operations are asynchronous or synchronous depends on the
“transaction safety” setting. This is controlled by the SAFETY option when
configuring with Transact-SQL commands (Set to FULL for synchronous
operations, set to OFF for asynchronous operations).
11
Copyright 2009 all rights reserved
12. Fail-over methods
Automatic failover - is enabled with a three-server configuration; a principal, a
mirror, and a witness server. Synchronous operations are required and the mirror
database must already be synchronized (in sync with the transactions as they are
being written to the principal). Role switching is done automatically. This is for high-
availability mode.
Manual failover - is needed when there is no witness server and is also doing
synchronous operations. The principal and the mirror are connected to each other
and the mirror database must already be synchronized. Role switching is done
manually. This is for high-protection mode.
Forced Service - in the case where there is a mirror server available, but it might
not be synchronized. It can be forced to take over when the principal server has
failed. This will possibly have data loss, since the transactions were not
synchronized. This is for either high-protection or high-performance mode.
12
Copyright 2009 all rights reserved
13. Rule-of-Thumb
You should use database mirroring:
• If you need to increase the availability of the database layer
• If you need to have automatic data protection (redundant storage of data)
• If you need to decrease the downtime that would normally be required to
do upgrades
And very often:
• If you need to off load reporting (periodic data snapshots) without impacting
the transactional system use database mirroring with database snapshots.
• Combo Needs: If you need data distribution, high availability and high data
resiliency - use data replication with database mirroring.
13
Copyright 2009 all rights reserved
14. Cannot be used with:
Database Mirroring cannot be used for any of SQL Server’s internal
databases TempDB, Master DB, MSDB, or Model DB.
Database Mirroring is NOT supported in SQL Server Workgroup
Edition or Express Edition. However, these server editions could be
used as Witness servers.
You cannot have database mirroring for databases enabled with
FILESTREAM storage.
14
Copyright 2009 all rights reserved
15. Instance: SQL08DE01 Instance: SQL08DE02
Endpoint name: “EndPoint4DBMirroring1430” Endpoint name: “EndPoint4DBMirroring1440”
Role: PARTNER Role: PARTNER
Example 2008 SQL Server 2008
SQL Server
Mirror Server
Configuratio
Principal Adventure Adventure
n Works DB Works DB
Server
translog translog
TCP: Listener_Port: 1430 TCP: Listener_Port: 1440
SQL Server 2008
Witness
MSDB DB
Server
TCP: Listener_Port: 1450
Instance: SQL08DE03
Endpoint name: “EndPoint4DBMirroring1450”
Role: WITNESS 15
Copyright 2009 all rights reserved
19. Creating Endpoints
-- create endpoint for PRINCIPAL server --
CREATE ENDPOINT [EndPoint4DBMirroring1430]
STATE=STARTED
AS TCP (LISTENER_PORT = 1430, LISTENER_IP = ALL)
FOR DATA_MIRRORING (ROLE = PARTNER,
AUTHENTICATION = WINDOWS NEGOTIATE
, ENCRYPTION = REQUIRED ALGORITHM RC4)
SQL Server Log entry
01/05/2009 13:17:23,spid55,Unknown,The Database Mirroring protocol transport is now listening for
connections.
01/05/2009 13:17:23,spid55,Unknown,Server is listening on [ 'any' <ipv4> 1430].
19
Copyright 2009 all rights reserved
20. Creating Endpoints
-- create endpoint for MIRROR server --
CREATE ENDPOINT [EndPoint4DBMirroring1440]
STATE=STARTED
AS TCP (LISTENER_PORT = 1440, LISTENER_IP = ALL)
FOR DATA_MIRRORING (ROLE = PARTNER,
AUTHENTICATION = WINDOWS NEGOTIATE
, ENCRYPTION = REQUIRED ALGORITHM RC4)
-- create endpoint for WITNESS server --
CREATE ENDPOINT [EndPoint4DBMirroring1450]
STATE=STARTED
AS TCP (LISTENER_PORT = 1450, LISTENER_IP = ALL)
FOR DATA_MIRRORING (ROLE = WITNESS,
AUTHENTICATION = WINDOWS NEGOTIATE
, ENCRYPTION = REQUIRED ALGORITHM RC4)
20
Copyright 2009 all rights reserved
21. Backup Principal
BACKUP DATABASE [AdventureWorks]
TO DISK = N'C:Program FilesMicrosoft SQL
N'C: Files
ServerMSSQL10.SQL08DE01MSSQLBackupAdventureWorks4Mirror.bak'
Server MSSQL10.SQL08DE01 MSSQL Backup
WITH FORMAT
GO
-- Processed 17672 pages for database 'AdventureWorks', file 'AdventureWorks_Data' on file 1.
'AdventureWorks_Data'
-- Processed 3 pages for database 'AdventureWorks', file 'AdventureWorks_Log' on file 1.
'AdventureWorks_Log'
-- BACKUP DATABASE successfully processed 17675 pages in 7.718 seconds (17.890 MB/sec).
seconds
BACKUP LOG [AdventureWorks] TO
DISK = N'C:Program FilesMicrosoft SQL
N'C: Files
ServerMSSQL10.SQL08DE01MSSQLBackupAdventureWorks4MirrorLog.bak'
Server MSSQL10.SQL08DE01 MSSQL Backup
GO
-- Processed 1849 pages for database 'AdventureWorks', file 'AdventureWorks_Log' on file 1.
'AdventureWorks_Log'
-- BACKUP LOG successfully processed 1849 pages in 0.862 seconds (16.757 MB/sec).
(16.757
22. Restore to Mirror
RESTORE FILELISTONLY
FROM DISK = N'C:Program FilesMicrosoft SQL
N'C: Files
ServerMSSQL10.SQL08DE01MSSQLBackupAdventureWorks4Mirror.bak'
Server MSSQL10.SQL08DE01 MSSQL Backup
RESTORE DATABASE AdventureWorks
FROM DISK = N'C:Program FilesMicrosoft SQL
N'C: Files
ServerMSSQL10.SQL08DE01MSSQLBackupAdventureWorks4Mirror.bak'
Server MSSQL10.SQL08DE01 MSSQL Backup
WITH NORECOVERY,
MOVE 'AdventureWorks_Data' TO 'C:Program FilesMicrosoft SQL
'C: Files
ServerMSSQL10.SQL08DE02MSSQLDATAAdventureWorks_Data.mdf',
Server MSSQL10.SQL08DE02 MSSQL DATA
MOVE 'AdventureWorks_Log' TO 'C:Program FilesMicrosoft SQL
'C: Files
ServerMSSQL10.SQL08DE02MSSQLDATAAdventureWorks_Log.ldf'
Server MSSQL10.SQL08DE02 MSSQL DATA
GO
-- Processed 17672 pages for database 'AdventureWorks', file 'AdventureWorks_Data' on file 1.
'AdventureWorks_Data'
-- Processed 3 pages for database 'AdventureWorks', file 'AdventureWorks_Log' on file 1.
'AdventureWorks_Log'
-- RESTORE DATABASE successfully processed 17675 pages in 6.716 seconds (20.560 MB/sec).
seconds
RESTORE LOG AdventureWorks
FROM DISK = N'C:Program FilesMicrosoft SQL
N'C: Files
ServerMSSQL10.SQL08DE01MSSQLBackupAdventureWorks4MirrorLog.bak'
Server MSSQL10.SQL08DE01 MSSQL Backup
WITH FILE = 1, NORECOVERY;
GO
-- Processed 0 pages for database 'AdventureWorks', file 'AdventureWorks_Data' on file 1.
'AdventureWorks_Data'
-- Processed 1849 pages for database 'AdventureWorks', file 'AdventureWorks_Log' on file 1.
'AdventureWorks_Log'
-- RESTORE LOG successfully processed 1849 pages in 1.083 seconds (13.338 MB/sec).
29. SQL Server 2008
SQL Server 2008
Mirror Server
10
Principal
AdventureWorks
Server AdventureWorks
DB DB
translog translog
Database
Database
Snapshot
Snapshot
becomes the mirror.
If this server node
SQL Server 2008
Witness
rk
rk
s
oo
er
MSDB DB
tw
Server
tw
Us
Ne
Ne
g
n
rti
po
Re
29
Copyright 2009 all rights reserved
31. Instance: SQL08DE01 Instance: SQL08DE02
Endpoint name: “EndPoint4DBMirroring1430” Endpoint name: “EndPoint4DBMirroring1440”
Role: PARTNER Role: PARTNER
Example 2008 SQL Server 2008
SQL Server
Mirror Server
Configuratio
Principal Adventure Adventure
n Works DB Works DB
Server
translog translog
TCP: Listener_Port: 1430 TCP: Listener_Port: 1440
SQL Server 2008
Witness
MSDB DB
Server
TCP: Listener_Port: 1450
Instance: SQL08DE03
Endpoint name: “EndPoint4DBMirroring1450”
Role: WITNESS 31
Copyright 2009 all rights reserved
40. DB Mirroring Benchmark Summary
Overall send rate 41 % faster (2005 vs 2008)
Overall restore rate 52% faster (2005 vs 2008)
Overall availability topology inherits the restore rate ~ 50% more available
Example:
743 ms Transaction (in 2005 mirroring configuration)
379 ms Transaction (in 2008 mirroring configuration)
In failover, what would have been 20 seconds before mirror becomes principal
turns into roughly 8-10 seconds (or 3seconds turns into 1.5 seconds, so on).
40
Copyright 2009 all rights reserved
41. DB Mirroring 2008
Solid State Disks
Benchmarks
DB Mirroring 2008 Conventional Disk configuration (SCSI/Raid10)
DB Mirroring 2008 Solid State Disk configuration (Stripping/Dynamic Volume)
** SSD’s courtesy of Solid Data corporation
42. Instance: SQL08DE01 Instance: SQL08DE02
Endpoint name: “EndPoint4DBMirroring1430” Endpoint name: “EndPoint4DBMirroring1440”
Role: PARTNER Role: PARTNER
SQL Server 2008
SQL Server 2008
Benchmark Mirror Server
Configuration
Principal Adventure Adventure
Works DB Works DB
Server
translog translog
TCP: Listener_Port: 1430 TCP: Listener_Port: 1440
SQL Server 2008
Witness
MSDB DB
Server
TCP: Listener_Port: 1450
Instance: SQL08DE03
Endpoint name: “EndPoint4DBMirroring1450”
Role: WITNESS 42
Copyright 2009 all rights reserved
46. Mirroring Conventional Disk
Client Execution Time 5:11:59 AM
Query Profile Statistics
Number of transactions 100000
Network Statistics
Number of server roundtrips 1
TDS packets sent from client 1
TDS packets received from server 4912
Bytes sent from client 92
Bytes received from server 20117100
Time Statistics
Client processing time 622782
Total execution time 622953
Wait time on server replies 171
Conventional Disk DB Mirroring
49. Mirroring Solid State Disk
Client Execution Time 3:53:14 AM
Query Profile Statistics
Number of transactions 100000
Network Statistics
Number of server roundtrips 1
TDS packets sent from client 1
TDS packets received from server 4951
Bytes sent from client 92
Bytes received from server 20277550
Time Statistics
Client processing time 517844
Total execution time 517953
Wait time on server replies 109
For Mirroring execution SSD's TO SSD's
50. DB Mirroring SSD Benchmark Summary
Overall send rate 26 % faster (2008 Conventional versus SSD)
Overall restore rate 17% faster (2008 Conventional versus SSD)
Overall db mirroring topology garners ~ 20% more availability
10 min 17 seconds to complete
> 4:32:37am 4:43:00am (principal send’s)
> 4:32:48 am 4:43:05am (mirror restore’s)
8 min 38 seconds to complete
> 3:45:19am 3:53:20am (principal send’s)
> 3:45:20am 3:53:20am (mirror restore’s)
100% Transparent (zero db & application changes) !
50
Copyright 2009 all rights reserved
52. Snapshots
52
Copyright 2009 all rights reserved
53. Tr
an
s
Us acti
er on
s al
SQL Server 2008
01 AdventureWork
Source CREATE DB ‘xyz’
s DB
Server AS SNAPSHOT OF
AdventureWorks
translog
Database
Snapshot
--- Reverting --- if needed ---
RESTORE DB
AdventureWorks
FROM DATABASE_SNAPSHOT = ‘xyz’
d- U e
y) s
ea ing tim
nl r
O se
(R rt in-
po nt-
Re Poi
54. Source Data
Pages
Source
02
SQL Server 2008
Sparse
SQL
Server File
Pages Snapshot
AdventureWork AdventureWork
s DB s DB
Empty Sparse file
of a Snapshot just created
(no updates to original
data pages have occurred yet)
54
Copyright 2009 all rights reserved
55. Source
Data
Pages
Source
03
SQL Server 2008
Sparse
SQL
Server File
Pages Snapshot
AdventureWork AdventureWork
s DB s DB
System Catalog Copy of original
of changed pages pages for snapshot
only when a page is changed
(Copy-on-write)
55
Copyright 2009 all rights reserved
56. Snapshot
Users
Source Data
Pages
SELECT …..data…….
FROM AdventureWorks
SNAPSHOT
04
SQL Server 2008
SQL
Server
Snapshot
AdventureWork AdventureWork
s DB s DB
System Catalog Sparse
of changed pages File
Pages
56
Copyright 2009 all rights reserved
57. Transactional
Users
Data
Pages
A
C
Update Row
Committed
SQL Server 2008
05
Sparse
SQL
Server File
B Pages Snapshot
AdventureWork
B AdventureWork
s DB s DB
translog
B
System Catalog
of changed pages Copy of original
pages for snapshot use
B
only when a page is changed
(Copy-on-write)
58. Us All
er
s
SQL Server 2008
06 UPDATE AWSource.tableX
AdventureWorks
Source DB set xyz = …
Server FROM AWSnapshot6:00AM.tableX
translog
Database
6:00AM Snapshot
Snapshot
Database
12:00PM Snapshot
Snapshot
Restore from Database
6:00PM Snapshot
Snapshot
Any Point-in-time
Database 12:00AM Snapshot
Snapshot if needed Snapshot
59. Us
er
Generating s
Mass Changes
SQL Server 2008
07
Source UPDATE AWSource.tableX
AdventureWorks set xyz = …
Server DB FROM AWSafegaurd6:00AM.tableX
Database
SAFEGUARD Snapshot
Snapshot
(Before the mass changes)
Restore from snapshot
(if changes are not correct)
60. Te
Us stin
er g
s
SQL Server 2008
08
Source
AdventureWorks
Server DB
Database
BEFORE TEST Snapshot
Snapshot
Restore from the BEFORE snapshot
after testing cycle is complete and can
continue with next test.
61. Tr
an
s
Us act
er ion
s al
Re Poi
po nt-i
(R rt n-
e a i n g ti m
d- U e
O se
SQL Server 2008
nl r s
y)
09 AdventureWorks
Source DB
Server
translog
Database
6:00AM Reporting Snapshot
Snapshot
Database
12:00PM Reporting Snapshot
Snapshot
Database
6:00PM Reporting Snapshot
Snapshot
Database 12:00AM Reporting Snapshot
Snapshot
62. SQL Server 2008
SQL Server 2008
Mirror Server
10
Principal
AdventureWorks
Server AdventureWorks
DB DB
translog translog
Database
Database
Snapshot
Snapshot
becomes the mirror.
If this server node
SQL Server 2008
Witness
rk
rk
s
oo
er
MSDB DB
tw
Server
tw
Us
Ne
Ne
g
n
rti
po
Re
62
Copyright 2009 all rights reserved
63. Instance: SQL2008xyz
Endpoint Name: “endpoint4mirroring”
Role: PARTNER PH Topology With Snapshots
SQL Server 2008
Instance: SQL2008zzz
Principal
Endpoint Name: “endpoint4mirroring”
Server
Role: PARTNER
OLTP Application
SQL Server 2008
Active Mirror Server
Clustered
Replication
Adventure
Adventure
Works DB
Works DB
translog
translog
Database
Passive
PH Snapshot
Network
Topology Network
Principal
Server
SQL Server 2008
63
Reporting Users
Copyright 2009 all rights reserved
64. DB Mirror and
Publisher
Replication
SQL Server 2008 SQL Server 2008
Principal Mirror SQL Server 2008
Server Server
Subscriber
Distributor
SQL Server 2008 SQL Server 2008
Principal Mirror
Server Server
SQL Server 2008
SQL Server 2008
Witness
Server
Subscriber
64
Copyright 2009 all rights reserved
65. Distributing Data
Sites
Data Access Latency Autonomy Frequency Network Machines Owner Other REPLICATION
(locations)
Each site only
Read Only Central Publisher
fast/ 1 Transactional repl
short high many high 1 OLTP needs regional
Reporting
Database Mirroring stable server/site site filter by region
data
Each site only
Read Only Central Publisher
fast/ 1 Snapshot repl
long high many low 1 OLTP needs regional
Reporting
Database Mirroring stable server/site site filter by region
data
Regional updates Central Publisher
Read Mostly
fast/ 1 Transactional repl
A few updates short high < 10 medium 1 OLTP on one table
stable server/site site Updating Subs
Regional update
Read Mostly Central Publisher
slow/ 1 All Merge repl
A few updates medium high < 10 medium all tables
unreliab server/site update
Regional update
Read equal Peer-to-Peer
fast/ 1 All Transactional
Equal updates short high < 10 medium all tables
stable server/site update repl
Each site only
Inserts 1 Central Subscriber
fast/ 1 Transactional repl
(new orders) short high many high needs regional
stable server/site report data
site
Central Publisher
Hot/Warm Very fast/ 1 Fail-over Remote Distributor
high <2 high 1 OLTP
Spare short stable server/site site Transactional repl
Database Mirroring
65
Copyright 2009 all rights reserved
66. Questions
Is there any time left????
Send your questions to:
pbertucci@dbarchitechs.com
or
tgerardin@dbarchitechs.com
Copyright 2009 all rights reserved
67. SQL Clustering basic configuration
SQL
CLUSTER1
Connections
C:
Local
Binaries
Windows 2003 EE
MSCS Cluster Group
Resources
SQL Server 2008 (physical)
MS DTC
S: SQL Agent
SQL Full Text
Search
Master DB
SQL Server 2008
SCSI
TempDB
(Virtual SQL Server)
Appl 1 DB
VSQLSERVER2008
Quorum
Q:
Disk
SQL Server 2008 (physical)
MSCS
Windows 2003 EE
CLUSTER2
C:
Local
67
Binaries
Copyright 2009 all rights reserved