In this session, you'll gain invaluable guidance for optimizing your backup strategies. Actually that is a lie, I will be advocating the mind shift from backup strategies and the devastating affect they can have on a company and instead move you to thinking about implementing restore strategies. Things become much simpler when you consider the purpose of a backup, and the effects of the different recovery models and backup options have on your customers and effectively your livelihood. Using trace flags the session will also cover how SQL Server manages its own backup options and how you can tune them to make sure that you meet the time constraints of your enforced maintenance windows.
4. 6 SQLBits XIV - The Big Backup Theory
The goal of
this
presentation
The Cooper-Hofstadter Polarization - S1 E9
5. 7 SQLBits XIV - The Big Backup Theory
I do work for a vendor,
there may be some
subliminal messages in
this presentation
6. 8 SQLBits XIV - The Big Backup Theory
Don’t have a
“Backup
Strategy”
7. 9 SQLBits XIV - The Big Backup Theory
Agenda
• Why do we need backups?
• Backups available in SQL Server
• Understanding the Transaction Log
• Backup Strategy vs. Restore Strategy
• How to make your backups faster
• How to make your restores faster
8. 10 SQLBits XIV - The Big Backup Theory
Have a
“Restore
Strategy”
9. 11 SQLBits XIV - The Big Backup Theory
If we haven’t met yet…
• Richard Douglas
• Sales Engineer / SQL Evangelist
• SQL Server MCITPro
• SQL User Group Leader
• Blog: http://SQL.RichardDouglas.co.uk
• Twitter: @SQLRich
• Email: Richard.Douglas@Software.Dell.com
• Slides: http://www.Slideshare.net/SQLRich
10. 12 SQLBits XIV - The Big Backup Theory
Seriously,
think about it.
A “Restore
Strategy” makes
so much sense!
11. 13 SQLBits XIV - The Big Backup Theory
This time
Sheldon is
wrong…
Backups are
Essential!
The Cooper-Hofstadter
Polarization S1 E9
12. 14 SQLBits XIV - The Big Backup Theory
Why do we need backups?
• Protect valuable data
• Provides the potential to return to a “point in time”
• Confusion between Disaster Recovery and High Availability
• Backup misconceptions
Image backups are sufficient
Snapshots are sufficient
Differential backups are incremental
System databases only need to be backed up once a week
Only need to keep my last backup
Buffer pool usage
13. 15 SQLBits XIV - The Big Backup Theory
Have a
“Restore
Strategy”
14. 16 SQLBits XIV - The Big Backup Theory
Recovery Models
• Simple
• Full
• Bulk Logged
Data
File
Log
File
SQL
Database
Automated administration Manual Administration
Note: Recovery model will be defaulted to the one in the Model database
15. 17 SQLBits XIV - The Big Backup Theory
Backup Types
• Full
• Differential
• Log
• File / Filegroup
• Tail of the log
• Database Snapshot
16. 18 SQLBits XIV - The Big Backup Theory
Have a
“Restore
Strategy”
17. 20 SQLBits XIV - The Big Backup Theory
Transaction Log Internals
• The Transaction Log is segmented into pieces called
Virtual Log Files (VLF’s)
• There will always be at least one “active” VLF
• The number of VLF’s is determined by the size at creation / growth time.
– Under 64MB 4VLF’s
– >=64 and > 1GB 8 VLF’s
– >= 1GB 16 VLF’s
• But wait, there’s more. VLF’s aren’t the smallest segment of the Transaction
Log …
Active
VLF
Inactive
VLF
Inactive
VLF
Inactive
VLF
Inactive
VLF
18. 21 SQLBits XIV - The Big Backup Theory
Log Blocks
• Each VLF will have a number of “Log Blocks” inside it
• These blocks vary in size:
– Anywhere between 512bytes and 60KB
• A new block is created when:
– The data inside the block hits the 60KB limit
– A commit or abort occurs within a transaction
• Each log block contains a parity bit of 64 or 128 which is flipped every time it
is used.
• Inside each log block we have…
19. 22 SQLBits XIV - The Big Backup Theory
Explaining the
transaction log
is a bit like
nesting dolls
20. 23 SQLBits XIV - The Big Backup Theory
Log Records
• Log records are identified by Log Sequence Numbers or LSN’s
– VLF Sequence nbr : Log block nbr : Log record nbr
• Each data page contains the LSN of the last record that made changes to
that page.
• Log records are written in the order they occur.
i.e. SQL Server will not wait for a transaction to finish before writing away
another concurrent transaction.
• Log records often store before and after images of changes.
• They contain a list of pages that this change has affected.
• Log records are stored in the buffer pool (log cache) until flushed to disk
Look for WAITLOG wait type.
This will become clear when we
cover the restore process
21. 24 SQLBits XIV - The Big Backup Theory
Checkpoint
• Data pages are marked as “dirty” when they are changed.
• The checkpoint process writes all dirty pages to disk and marks them as
cleared.
• Checkpoint is an optimisation process, it reduces the amount of writes to
disk to keep data up to date.
22. 26 SQLBits XIV - The Big Backup Theory
Have a
“Restore
Strategy”
23. 27 SQLBits XIV - The Big Backup Theory
Restore Strategy main considerations
• Time taken to restore data
aka Recovery Time Objective (RTO)
• Data loss acceptability
aka Recovery Point Objective (RPO)
• Cost associated with system
unavailable
Backup Strategy main considerations
• Time taken to perform a backup
• Cost associated with longer
maintenance window
• Cost associated with storing
backup files
Backup Strategy vs Restore Strategy
24. 28 SQLBits XIV - The Big Backup Theory
Showing you
why you need to
have a “Restore
Strategy”
33. 37 SQLBits XIV - The Big Backup Theory
Figured out it’s
safer to have a
“Restore
Strategy” yet?
34. 38 SQLBits XIV - The Big Backup Theory
Optimizing backups
Server
Architecture
Database
Architecture
Backup
Options
35. 39 SQLBits XIV - The Big Backup Theory
Have a
“Restore
Strategy”
36. 40 SQLBits XIV - The Big Backup Theory
Server
Architecture
37. 41 SQLBits XIV - The Big Backup Theory
What
computer do
you have? And
please don’t
say “a white
one”.
The Peanut Reaction – S1 E16
38. 42 SQLBits XIV - The Big Backup Theory
• IO Subsystem
– RAID Levels
– Disk Partition Alignment
– Dedicated drives
• Network cards
– Add more NICs
– Use different subnets
– Use different switches
Optimizing backups – Server Architecture
The DBA team start negotiations
with the Infrastructure team
http://www.flickr.com/photos/jakesutton/859193891/in/photostream
39. 43 SQLBits XIV - The Big Backup Theory
Have a
“Restore
Strategy”
40. 44 SQLBits XIV - The Big Backup Theory
Optimizing backups – Database Architecture
• Table design
– See Natural Born Killers webcast November 2012 (http://dell.to/15RwEtz)
– Blogged at http://bit.ly/GHuUrH
– Space that a table uses is not completely released after you use a DELETE
statement to delete data from the table in SQL Server:
http://support.microsoft.com/kb/913399
• Index Design
– See indexing webcasts October 2012 (http://dell.to/19urm8s)
• File placement
• Compression
41. 45 SQLBits XIV - The Big Backup Theory
End of
subliminal
messages
42. 46 SQLBits XIV - The Big Backup Theory
Actually, I
wouldn’t mind
a caramel
macchiato
43. 47 SQLBits XIV - The Big Backup Theory
Optimizing Backups
• Compression
• BufferCount
• MaxTransferSize
• BlockSize
• Multiple backup devices
Other Backup Options of note
• Init / NoInit
• Format / NoFormat
• Copy Only
• Checksum
• Continue After Error
Backup Options
For a full list of all the backup options refer to Books Online (BOL)
http://msdn.microsoft.com/en-us/library/ms186865.aspx
44. 48 SQLBits XIV - The Big Backup Theory
Backup Options Continued
• Memory used is accessed from outside of the bufferpool
• You can calculate memory usage:
Buffercount * MaxTransferSize
Setting these values too high may cause out of memory conditions
Buffercount calculations for Full / Diff backups:
• 2000
– (NumberofBackupDevices * GetSuggestedIoDepth)
+ NumberofBackupDevices + (DatabaseDeviceCount)
• 2005 and above
– (NumberofBackupDevices * GetSuggestedIoDepth)
+ NumberofBackupDevices + (2*DatabaseDeviceCount)
45. 49 SQLBits XIV - The Big Backup Theory
Calculating BufferCount
• GetSuggestedIODepth
– Disk = 3
– Tape = 3
– VDI = 1 default and 4 max based on device spanning needs http://bit.ly/1i69GUG
• DatabaseDeviceCount = The number distinct drives the database files reside
on.
• I think the formula for 2008+ is wrong. Waiting for CSS to return my email.
The following seems to work:
(NumberofBackupDevices * GetSuggestedIoDepth [4])
+ NumberofBackupDevices + (2*DatabaseDeviceCount)
47. 51 The Big Backup Theory
Things that
break the log
chain
48. 52 SQLBits XIV - The Big Backup Theory
Restore Problems
• Broken Transaction Log Chain
– Clearing the transaction log (using WITH TRUNCATE ONLY or WITH NO LOG)
– Changing the database to SIMPLE recovery model (checkpoint will clear log))
– Reverting from a database snapshot
– Deleting a log file in the chain
– Backups to NUL devices
49. 53 SQLBits XIV - The Big Backup Theory
Backups
are like
Schrodinger’s
cat
50. 54 SQLBits XIV - The Big Backup Theory
Restore Myths
• Downgrading a backup
• No need to test backups
• Restore time will be the same as the backup time
• Table restores
51. 55 SQLBits XIV - The Big Backup Theory
The Restore Process
A restore operation can have up to four phases:
1. File creation and initialization
2. Data and/or log copy
3. REDO phase of recovery
4. UNDO phase of recovery
52. 56 SQLBits XIV - The Big Backup Theory
The Restore Process
File creation and initialization
• In the beginning there was file creation.
• Then there was the zeroing of the files.
• If the database already exists overwrite using the “WITH REPLACE” clause,
do not drop and restore.
• Use “Instant File Initialization” (IFI) if possible.
Remember the log file still needs to be zeroed out.
• There is a security risk.
• TF 3004 will provide more information about IFI
• TF 1806 will disable IFI
53. 57 SQLBits XIV - The Big Backup Theory
The Restore Process
Data and/or log copy
• In this stage SQL Server:
– Reads the data from the backup file(s)
– Writes the data to the database files
• Having multiple data files and backup files can improve I/O throughput.
• Using native or third party compression reduces the amount of data to be
read (and potentially copied across the network).
Compression vs DeDuplication
• Whilst deduplication does reduce storage costs it will be rehydrated before
copying back across the network and so will not aid the speed of a restore.
54. 58 SQLBits XIV - The Big Backup Theory
The Restore Process
REDO phase of recovery (1)
• The database being recovered has it’s boot page (page 9) checked.
• This page will provide the details of the most recent checkpoint.
• Checkpoint log records are examined to see if there is a list of uncommitted
transactions
• Recovery will start at the oldest uncommitted transaction for the most recent
checkpoint
55. 59 SQLBits XIV - The Big Backup Theory
The Restore Process
REDO phase of recovery (2)
• Each log record is tied to an LSN and contains page numbers of records it has altered
• Each data page’s page header contains the last LSN which made a change to it
• The redo phase compares the two LSN’s
• The redo phase uses something akin to the following formula:
IF PageHeaderLSN < LogRecordLSN
BEGIN
Replay Log Record –Record has not been updated
END
ELSE
DoNothing –Record has this change and possibly some others
56. 60 SQLBits XIV - The Big Backup Theory
The Restore Process
REDO phase of recovery (3)
Finding the end of the log
There are two ways:
1. If the log has not been wrapped then it checks for the occurrence of 512 byte
block full of zeros.
2. If it has been wrapped then it reads through all the active VLF’s in the order of
their sequence number and reads through each log block until it finds one with
the wrong flipped parity bit
57. 61 SQLBits XIV - The Big Backup Theory
The Restore Process
UNDO phase of recovery
• The redo phase keeps track of uncommitted transactions. If there are
uncommitted transactions during recovery then the undo phase kicks in
• As mentioned in the redo section the data page page header contains an
LSN. This is compared to the Log Record LSN
IF PageHeaderLSN >= LogRecordLSN
BEGIN
Undo Log Record –Record requires rollback
Create a “compensation” log record
Update PageHeader LSN with compensation log record LSN
END
58. 63 SQLBits XIV - The Big Backup Theory
Further ways to optimize restores
• Using “Instant File Initialization”
• Using the correct restore strategy
• Server Architecture
• Database Architecture
• Backup compression
• Restore options
– Buffercount
– MaxTransferSize
60. Thank you for attending
Richard Douglas
Richard.Douglas@Software.Dell.com
@SQLRich
Editor's Notes
Single point in time, look at how much data could be lost!
2 places to restore to. Less data loss
Restore to any point in time up to the last transaction log backup.
Data loss is delta between now and the last transaction log backup.
Buffercount – Total number of I/O buffers to be used.
MaxTransferSize – Largest unit of transfer size used in bytes
BlockSize - The physical block size of the backup
From John Huang’s blog: http://www.sqlnotes.info/2011/10/31/page-type/
Type 1 – Data page.
Data records in heap
Clustered index leaf-level
Location can be random
Type 2 – Index page
Non-clustered index
Non-leave-level clustered index
Location can be random
Type 3 – Text Mixed Page
Small LOB value(s), multiple types and rows.
Location can be random
Type 4 – Text Page
LOB value from a single column value
Location can be random
Type 7 – Sort Page
Temporary page for sort operation.
Usually tempdb, but can be in user database for online operations.
Location can be random
Type 8 – GAM Page
Global Allocation Map, track allocation of extents.
One bit for each extent, if the bit is 1, means the extent is free, otherwise means the extent is allocated (not necessary full).
The first GAM page in each file is page 2
Type 9 – SGAM Page
Shared Global Allocation Map, track allocation of shared extents
One bit for each extent, if the bit is 1, means the extent is allocated but has free space, otherwise means the extent is full
The first SGAM page in each file is page 3
Type 10 – IAM Page
Index Allocation Map. Extent allocation in a GAM interval for an index or heap table.
Location can be random.
Type 11 – PFS Page
Page Free Space. Byte map, keeps track of free space of pages
The first PFS is page 1 in each file.
Type 13 – Boot Page
Information about the page
Only page 9 in file 1.
Type 14 – Server Configuration Page (It may not be the official name)
Part of information returned from sp_configure.
It only exists in master database, file 1, page 10
SQL Server 2008 Only
Type 15 – File Header Page
Information about the file.
It’s always page 0 every data page.
Type 16 – Differential Changed map
Extents in GAM interval have changed since last full or differential backup
The first Differential Changed Page is page 6 in each file
Type 17 – Bulk Change Map
Extents in GAM interval modified by bulk operations since last backup
The first Bulk Change Map page is page 7 in each file.