The document discusses different techniques for database recovery including rollback/undo recovery, commit/redo recovery, and checkpoint recovery. Rollback/undo recovery undoes the changes of an uncommitted transaction using log records, while commit/redo recovery reapplies the changes of a committed transaction using log records to restore the most recent consistent state. Checkpoint recovery periodically saves the database state to a checkpoint file to speed up recovery. The document also discusses transaction failure, system crash, log-based recovery using write-ahead logging, and shadow paging recovery using two page tables.

1. Database Recovery and Indexing

2. Database recovery in DBMS
• Used to restore a database to a consistent state after a failure
or error has occurred.
• The main goal to:
1. Ensure data integrity and
2. Consistency and prevent data loss.
• There are mainly two types of recovery techniques used in
DBMS:
1. Rollback/Undo Recovery Technique
2. Commit/Redo Recovery Technique
In addition to these two techniques, there is also a third
technique called checkpoint recovery.
3. Checkpoint recovery

3. 1.The Rollback/Undo Recovery Technique
• This technique is based on the principle of backing out or
undoing the effects of a transaction that has not completed
successfullydue to a system failure or error.
• This technique is accomplished by undoing the changes made
by the transaction using the log records stored in the
transaction log.
• The transaction log contains a record of all the transactions
that have been performed on the database.
• The system uses the log records to undo the changes made by
the failed transaction and restore the database to its previous
state.

4. 2. Commit/Redo Recovery Technique
• This technique is based on the principle of reapplying the
changes made by a transaction that has been completed
successfully to the database.
• This technique is accomplished by using the log records
stored in the transaction log to redo the changes made by the
transaction that was in progress at the time of the failure or
error.
• The system uses the log records to reapply the changes made
by the transaction and restore the database to its most recent
consistent state.

5. 3. Checkpoint recovery
• Checkpoint recovery is a technique used to reduce the
recovery time by periodically saving the state of the
database in a checkpoint file.
• In the event of a failure, the system can use the checkpoint
file to restore the database to the most recent consistent
state before the failure occurred, rather than going through
the entire log to recover the database.

6. Failure Classification:
• To see where the problem has occurred, generalize the failure
into various categories:
1. TRANSACTION FAILURE : When a transaction is failed to
execute or it reaches a point after which it cannot be completed
successfully it has to abort. This is called transaction failure.
Where only few transaction or process are hurt.
• Reason for transaction failure could be:
1. Logical errors:
o where a transaction cannot complete because of it has some code
error or any internal error condition.
2. System errors:
o Where the database system itself terminates an active transaction
because DBMS is not able to execute it or it has to stop because of
some system condition.
o For example, in case of deadlock or resource unavailability systems
aborts an active transaction.

7. SYSTEM CRASH
2. SYSTEM CRASH:
• There are problems, which are external to the system,
which may cause the system to stop abruptly and cause
the system to crash.
• Ex: Interruption in power supply, underlying hardware
or software failure.
• Ex: may include operating system errors.
1.Disk failure:
2.Storage structure:

8. SYSTEM CRASH CONTD…
• Disk failure:
o Where hard disk drives or storage drives used to fail frequently.
- In early days of technologyevolution, it was a common problem
o Disk failures include formation of bad sectors, unreachability to the disk,
disk head crash or any other failure, which destroys all or part of disk
storage.
• Storage Structure:
The storage structure can be divided in various categories:
o Volatile storage: data is loss if power is off
✓ As name suggests, this storage does not survive system crashes and
mostly placed very closed to CPU by embedding them onto the chipset
itself. Examples: main memory, cache memory.
✓ They are fast but can store a small amount of information.
o Non-volatile storage: retain stored information even after power is
removed.
✓ These memories are made to survivesystem crashes.
✓ They are huge in data storage capacity but slower in accessibility.
✓ Ex:hard disks, magnetic tapes, flash memory, non-volatile (battery backed
up) RAM.

9. Recovery and Atomicity:
➢ When a system crashes, it may have several transactions being
executed and various files opened for them to modifying data items.
o Transactions are made of various operations, which are atomic in
nature.
o But according to ACID properties , atomicity of transactions as a
whole must be maintained that is, either all operations are executed
or none.
➢ When DBMS recovers from a crash it should maintain the
following:
1. It should check the states of all transactions, which were being
executed.
2. A transaction may be in the middle of some operation; DBMS
must ensure the atomicity of transaction in this case.
3. It should check whether the transaction can be completed now
or needs to be rolled back.
4. No transactions would be allowed to left DBMS in
inconsistent state.

10. Types of recovery techniques
There are two types of techniques, in recovering as well as
maintaining the atomicity of transaction:
1. Maintaining the logs (logs: It contains information about the
start &end of each transaction &any updates occurs during transaction)
of each transaction, and writing them onto some stable
storage before actually modifying the database.
2. Maintaining shadow paging, where are the changes are
done on a volatile memory and later the actual database is
updated.

11. 1. Log-Based Recovery
• Log is a sequence of records, which maintains the records of actions
performed by a transaction.
o LOG: It contains information about the start & end of each
transaction & any updatesoccurs during transaction
• The logs are written prior to actual modification & stored on a stable
storage media, which is failsafe.
• Log based recovery works as follows:
o The log file is kept on stable storage media -start_transaction(T)
o When a transactionenters the system and starts execution, it writes a log
about it.
o When the transaction modifies an item X, it write logs as follows:
✓ It reads Tn has changed the value of X, from V1 to V2.
✓ When transaction finishes, it logs:
<Tn, commit>
• Database can be modified using two approaches:
1. Deferred database modification:
2. Immediatedatabase modification:

12. Log based recovery works as follows:
1. The log is kept on disk start_transaction(T): This log entry
records that transaction T starts the execution.
2. read_item(T, X): This log entry records that transaction T reads the
value of database item X.
3. write_item(T, X, old_value, new_value): This log entry records that
transaction T changes the value of the database item X from old_value
to new_value. The old value is sometimes known as a before an
image of X, and the new value is known as an afterimage of X.
4. commit(T): This log entry records that transaction T has completed all
accesses to the database successfully and its effect can be committed
(recorded permanently) to the database.
5. abort(T): This records that transaction T has been aborted.
6. checkpoint: Checkpoint is a mechanism where all the previous logs
are removed from the system and stored permanently in a storage disk.
Checkpoint declares a point before which the DBMS was in a
consistent state, and all the transactionswere committed.

13. Database modification approach
➢ The techniques used to recover the lost data due to system
crashes, transaction errors, viruses; catastrophic failure,
incorrect commands execution, etc. are database recovery
techniques.
➢ So to prevent data loss recovery techniques based on
1. Deferred update : All logs are written on to the stable
storage and database is updated when transaction
commits.
2. Immediate update or backing up data : Each log
follows an actual database modification. That is, database
is modified immediately after every operation.

14. Shadow Paging
➢ Shadow Paging is recovery technique that is used to recover database.
➢ In this recovery technique, database is considered as made up of fixed size
of logical units of storage which are known as pages.
➢ Pages are mapped into physical blocks of storage, with help of the page
table which allowone entry for each logical page of database.
➢ This method uses two page tables named current page table and shadow
page table.
✓ The entries which are present in current page table are used to point
to most recent databasepages on disk.
✓ Another table i.e., Shadow page table is used when the transaction
starts which is copying current page table.
✓ After this, shadow page table gets saved on disk and current page table
is going to be used for transaction.
✓ Entries present in current page table may be changed during
execution but in shadow page table it never get changed.
➢ After transaction, both tables become identical. This technique is also
known as Cut-of-Place updating.

15. Concept of shadow paging
Now let see the concept of shadow paging step by step −
Step 1 − Page is a segment of memory. Page table is an index of pages. Each
table entry points to a page on the disk.
Step 2 − Two page tables are used during the life of a transaction: the current
page table and the shadow page table. Shadow page table is a copy
of the current page table.
Step 3 − When a transaction starts, both the tables look identical, the current
table is updated for each write operation.
Step 4 − The shadow page is never changed during the life of the transaction.
Step 5 − When the current transaction is committed, the shadow page entry
becomes a copy of the current page table entry and the disk block
with the old data is released.
Step 6 − The shadow page table is stored in non-volatile memory. If the
system crash occurs, then the shadow page table is copied to the
current page table.
The shadow paging is represented diagrammatically as follows −

16. Shadow paging
➢ Firstly, search start for availablefree block in disk blocks.
➢ After finding free block, it copies page 3 to free block which is represented by
Page 3 (New).
➢ Now current page table points to Page 3 (New) on disk but shadow page
table pointsto old page 3 because it is not modified.
➢ The changes are now propagated to Page 3 (New) which is pointed by current
page table.
➢ To understand concept,
consider figure.
➢ In this two (2) write
operations are
performed on page 3
and 5.
➢ Before start of write
operation on page 3,
current page table
points to old page 3.
➢ When write operation
starts following steps
are performed :

17. COMMIT Operation
• COMMIT Operation : To commit transaction following steps
should be done :
1. All the modifications which are done by transaction which
are present in buffers are transferred to physical database.
2. Output current page table to disk.
3. Disk address of current page table output to fixed location
which is in stable storage containing address of shadow
page table.
This operation overwrites address of old shadow page table.
With this current page table becomes same as shadow page
table and transaction is committed.

18. Failure
• Failure : If system crashes during execution of transaction but
before commit operation, With this, it is sufficient only to free
modified database pages and discard current page table.
• Before execution of transaction, state of database get
recovered by reinstalling shadow page table.
✓ If the crash of system occur after last write operation then it
does not affect propagation of changes that are made by
transaction.
✓ These changes are preserved and there is no need to
perform redo operation.

19. Advantages and Disadvantages
• Advantages
1. No need for log records.
2. This method require fewer disk accesses to perform
operation.
3. No undo/ Redo algorithm.
4. In this method, recovery from crash is inexpensive and
quite fast.
5. There is no need of operations like- Undo and Redo.

20. Advantage of shadow paging
• Improved fault tolerance: Shadow paging provides improved fault
tolerance since it isolates transactions from each other. This means
that if one transaction fails, it does not affect the other transactions
that are currently executing.
• Increased concurrency: Since modifications made during a
transaction are written to the shadow copy instead of the actual
database, multiple transactions can be executed concurrently
without interfering with each other. This leads to increased
concurrency and better performance.
• Simplicity: Shadow paging is a relatively simple technique to
implement. It requires minimal modifications to the existing
database system, making it easier to integrate into existing systems.
• No need for log files: In traditional database systems, log files are
used to maintain a record of all changes made to the database.
Shadow paging eliminates the need for log files since all changes are
made to the shadow copy. This reduces the overhead associated
with maintaining log files and makes the system more efficient.

21. Disadvantages of shadow paging
1. Data is fragmented or scattered.
2. Garbage collection problem. Database pages containing old
versions of modified data need to be garbage collected
after every transaction.
3. Concurrent transactions are difficult to execute.
4. Garbage collection
5. Performance overhead
6. Limited concurrency control
7. Difficult to implement for some systems
8. Limited fault toleranc

22. Disadvantages of shadow paging
• Due to location change on disk , due to update database it is
quite difficult to keep related pages in database closer on disk.
• During commit operation, changed blocks are going to be
pointed by shadow page table which have to be returned to
collection of free blocks otherwise they become accessible.
• The commit of single transaction requires multiple blocks
which decreases execution speed.
• To allow this technique to multiple transactions concurrently,
it is difficult.
• Data fragmentation: The main disadvantage of this technique
is the updated Data will suffer from fragmentation as the data
is divided up into pages that may or not be in linear order for
large sets of related hence, complex storage management
strategies.

23. Disadvantages of shadow paging
• Garbage collection: Garbage will accumulate in the pages on the
disk as data is updated and pages lose any references. For example
if I have a page that contains a data item X that is replaced with a
new value then a new page will be created. Once the shadow page
table is updated nothing will reference the old value of X. The
operation to migrate between current and shadow directories must
be implemented as an atomic mode.
• Performance overhead: Since modifications made during a
transaction are written to the shadow copy, there is a performance
overhead associated with copying the changes back to the actual
database once the transaction is committed. This can impact the
overall performance of the system.
• Limited concurrency control: Shadow paging does not provide
strong concurrency control mechanisms. While it allows for multiple
transactions to execute concurrently, it does not prevent conflicts
between transactions. This means that transactions can interfere
with each other, leading to inconsistencies in the database.

24. Disadvantages of shadow paging
• Difficult to implement for some systems: Shadow paging
can be difficult to implement for some systems that have
complex data structures or use a lot of shared memory. In these
cases, it may not be possible to maintain a shadow copy of the
entire database.
• Limited fault tolerance: While shadow paging does provide
improved fault tolerance in some cases, it does not provide
complete fault tolerance. In the event of a crash, there is still a
risk of data loss if the changes made during a transaction are
not properly copied to the actual database.

25. Recovery with concurrent transactions
• Recovery with concurrent transactions:
o When more than one transactions are being executed in parallel,
the logs are interleaved.
o At the time of recovery it would become hard for recovery
system to backtrack all logs, and then start recovering.
o To ease this situation most modern DBMS use the concept of
'checkpoints'.
• Checkpoint recovery is a technique used to reduce the recovery
time by periodically saving the state of the database in a
checkpoint file.
• In the event of a failure, the system can use the checkpoint file to
restore the database to the most recent consistent state before the
failure occurred, rather than going through the entire log to
recover the database.

26. CHECKPOINT:
➢ CHECKPOINT: Keeping and maintaining logs in real time
and in real environment may fill out all the memory space
available in the system.
➢ At time passes log file may be too big to be handled at all.
✓ Checkpoint is a mechanism where all the previous logs are
removed from the system and stored permanently in
storage disk.
✓ Checkpoint declares a point before which the DBMS
was in consistent state and all the transactions were
committed.
✓ RECOVERY
• When system with concurrent transaction crashes and
recovers, it does behave in the following manner:

27. RECOVERY
The recovery system reads the logs
backwards from the end to the last
Checkpoint.
▪ It maintains two lists, undo-list
and redo-list.
▪ If the recovery system sees a log
with <Tn,start>& <Tn, commit>
or just < Tn, commit> , it puts the
transaction in redo-list.
▪ If the recovery system sees a log
with <Tn, start> but no commit or
abort log found, it puts the
transaction in undo-list.
▪ All transactions in undo-list are then undone and their logs are removed.
▪ All transaction in redo-list, their previous logs are removed and then
redone again and log saved.

28. Introduction to ARIES(Algorithm for Recovery and
Isolation Exploiting Semantics)
➢ Recovery techniques are heavily dependent upon the
existence of a special file known as a system log.
➢ It contains information about the start and end of each
transaction and any updates which occur during the
transaction–called system log.
➢ The log keeps track of all transaction operations that affect
the values of database items.
➢ This information is needed to recover from transaction
failure.
➢ ARIES is a recovery algorithm designed to work with a no-
force, steal database approach.
➢ It is used by IBM Db2, Microsoft SQL Server and many other
database systems.

29. ARIES Recovery Algorithm
• The previous sections answered the question: what extra
information do we maintain to, upon crash, reconstruct the
database to the state it was in prior to the crash? In this
section, we answer the algorithm question: how would we
use this information to actually reconstruct the database?
• When a database crashes, all we have are the logs that got
saved to disk. The ARIES recovery algorithm has 3 phases that
occur in the following order:
• Analysis: reconstructtransaction and dirty page tables
• Redo: repeat operations (for durability)
• Undo: undo operations from transactions in-progress during
the crash (for atomicity)

30. Introduction to ARIES: Crash Recovery:
• Algorithm for Recovery and Isolation Exploiting Semantics
(ARIES) is based on the Write Ahead Log (WAL) protocol.
• Every update operation writes a log record which is one of the
following :
1. Undo-only log record: Only the before image is logged.
Thus, an undo operation can be done to retrieve the old data.
2. Redo-only log record: Only the after image is logged. Thus,
a redo operation can be attempted.
3. Undo-redo log record: Both before images and after images
are logged.

31. Introduction to ARIES:
• In ARIES algorithm every log record is assigned a unique and
monotonically increasing Log Sequence Number (LSN).
• Every data page has a page LSN field that is set to the LSN of
the log record corresponding to the last update on the page.
• WAL(write ahead Log) requires that the log record corresponding
to an update make it to stable storage before the data page
corresponding to that update is written to disk.
✓ For performance reasons, each log write is not immediately
forced to disk.
✓ A log tail is maintained in main memory to buffer log writes.
✓ The log tail is flushed to disk when it gets full.
✓ A transaction cannot be declared committed until the commit
log record makes it to disk.

32. Working of ARIES
The ARIES recovery algorithm has 3 phases that occur in the following
order:
1. Analysis: (reconstruct transaction and dirty page tables)
o The recovery subsystem determines the earliest log record from
which the next pass must start.
o It also scans the log forward from the checkpoint record to
construct a snapshot of what the system looked like at the instant
of the crash.
2. Redo: repeat operations (for durability)
Starting at the earliest LSN, the log is read forward and each update
redone.
3. Undo: undo operations from transactions in-progress during the
crash (for atomicity)
The log is scanned backward and updates corresponding to loser
transactions are undone.

33. Analysis
• In the analysis phase, we scan through the log from the start to
reconstruct the transaction and dirty page tables.
• For each record in the log, we act according to the following
rules:
• If the record is not an END:
– Add transactionto the transaction table if it does not exist
– Set transaction’slast LSN to the record’sLSN
• If the record is an UPDATE:
– If the modified page is not in the DPT, add it to the DPT
– Set the page’s recLSN to the record’sLSN
• If the record is a COMMIT or an ABORT:
– Change the transaction’sstatus accordingly in the transaction table
• If the record is an END:
– Remove the transactionfrom the transaction table

34. Redo
• Once we finish the analysis phase, we start the redo phase to get
durability.
• We begin at the smallestrecLSN in the dirty page table and redo following
transactions.
• Intuitively, in this phase, we want to redo all operations that did not make
it to disk before the crash. The metadatain the transaction and dirty page
tables will help us determine whether the operation made it to disk before
the crash.
• We redo all UPDATEand CLR operations that do not meet any of the
following criteria:
• The page is not in the DPT — meaning the change must have been
flushed to disk before the crash
• The corresponding pageLSN on disk >= current record’s LSN —
meaning at least the current operation (and possibly some future
operation) must have made it to disk before the crash
• The corresponding recLSN in the dirty page table > current record’s
LSN — meaning the first operation that dirtied the page occurred after
the current operation, so the current operation must have made it to
disk before the crash

35. Undo
• Finally, we can do the undo phase to ensure atomicity.
• We start at the end of the log and work our way back to the
start. Intuitively, we want to undo any UPDATEfor any running
or aborting transaction at the time of crash.
• We will only undo UPDATEsthat do not have corresponding
CLRs (meaning, we will not undo UPDATEsthat have already
been undone by a CLR).
• We have one extra piece of metadata to track in this phase:
the undoNextLSN.
✓ This field stores the LSN of the operation that we want to
next undo for that transaction (derived from the prevLSN
of the operation currently being undone).
✓ Once we undo all operations for a transaction, we can
write an END record for that transaction.

36. check pointing
• In production DBs, it can be inefficient to reconstruct the transaction and
dirty page tables from the absolute beginning of the log.
• So ARIES uses check pointing to speed up the analysis phase, which
periodically writes the contents of the transaction and dirty page tables to
the log, so we can read from the latest checkpoint instead of
reconstructing from scratch.
• With check pointing, <BEGIN_CHECKPOINT> and <END_CHECKPOINT>
records get written to the log, with the contents of the transaction and
dirty page tables in between.
• When reconstructing, we need to start reading operations from the
<BEGIN_CHECKPOINT> — because the tables saved to the log can be the
state at any point between the <BEGIN_CHECKPOINT> and
<END_CHECKPOINT> records.

37. List of LSNs

38. RAID (Redundant Arrays of Independent Disks)
• Pre-Requisite: Data Redundancy in DBMS
• RAID is a technique that makes use of a combination of multiple
disks instead of using a single disk for increased performance, data
redundancy, or both.
• The term was coined by David Patterson, Garth A. Gibson, and
Randy Katz at the University of California, Berkeley in 1987.
• Why Data Redundancy?
• Data redundancy: although taking up extra space, adds to disk
reliability.
• This means, in case of disk failure, if the same data is also backed up
onto another disk, we can retrieve the data and go on with the
operation.
• On the other hand, if the data is spread across just multiple disks
without the RAID technique, the loss of a single disk can affect the
entire data.

39. Key Evaluation Points for a RAID System
➢ Reliability: How many disk faults can the system tolerate?
➢ Availability: What fraction of the total session time is a
system in uptime mode, i.e. how available is the system for
actual use?
➢ Performance: How good is the response time? How high is
the throughput (rate of processing work)? Note that
performance contains a lot of parameters and not just the two.
➢ Capacity: Given a set of N disks each with B blocks, how
much useful capacity is available to the user?
1. RAID is very transparent to the underlying system. This
means, to the host system, it appears as a single big disk
presenting itself as a linear array of blocks.
2. This allows older technologies to be replaced by RAID
without making too many changes to the existing code.

40. Different RAID Levels
• RAID-0 (Stripping)
• RAID-1 (Mirroring)
• RAID-2 (Bit-Level Stripping with Dedicated Parity)
• RAID-3 (Byte-Level Stripping with Dedicated Parity)
• RAID-4 (Block-Level Stripping with Dedicated Parity)
• RAID-5 (Block-Level Stripping with Distributed Parity)
• RAID-6 (Block-Level Stripping with two Parity Bits)

42. RAID-0 (Stripping)
• Blocks are “stripped” across disks.
In the figure, blocks “0,1,2,3” form a stripe.
Instead of placing just one block into a disk at a time, we can
work with two (or more) blocks placed into a disk before
moving on to the next one.