BUFFER
MANAGEMENT
BY
G.KARTHIGA M.SC IT
NADAR SARASWATHI COLLEGE OF ARTS
&SCIENCE
THENI
DATABASE BUFFERING
O Database Is Stored In Disk.
O For Read And Write Database Operation
Must Reside In Memory.
O Force Policy:
Modified Block Is Output To Disk
Once A Transaction Commits.
O No Force Policy:
Modified Block Is Not Output To
Disk Immediately If A Transaction Commits
DB BUFFER MANAGEMENT –
CONTENTS
O Goal:
realization of efficient, page-based in-memory
processing.
• maximum possible avoidance of physical I/O.
• replacement algorithms without and with
context knowledge.
O Role of DB buffer management:
course of access to the DB buffer.
• comparison with similar functionality in
operating systems (OS) .
O Locality:
• various measures for locality.
• characterization by LRU-stack-depth
distributions and reference density.
DBMS LAYER
Recovery
Manager
Transaction
Manager
Lock
Manager
DB
Query optimization
and execution
Relational operators
File and access method
Buffer management
Disk space
management
queries
SATISFYING PAGE REQUESTS
 For each frame in the pool, the DBMS buffer
manager maintains
 The pin_count variable: # of users of a page
 The dirty variable: whether a page has been
modified or not
 If a page is requested and not in the pool, the DBMS
buffer manager
O Chooses a frame for replacement and increments its
pin_count (a process known as pinning)
O If frame is dirty, writes it back to disk
O Reads the requested page into chosen frame
BUFFER MANAGEMENT
ARCHITECTURE
O The buffer manager controls main memory directly, as
in many relational DBMS’s
O The buffer manager allocates buffers in virtual memory,
allowing the operating system to decide which buffers
are actually in main memory at any time and which are
in the “swap space” on disk that the operating system
manages. Many “main-memory” DBMS’s and
“object-oriented” DBMS’s operate this way.
BUFFER MANAGEMENT STRATEGIES
O The critical choice that the buffer manager
must make is what block to throw out of the
buff pool when the buffer is needed for newly
requested block. There are some common
strategies, such as in operating systems. These
include:
• LRU
• FIFO
• Clock
• System Control
O LRU – Least recently used
• Throw out block that has not been read or
written for the longest time.
O FIFO – First in first out
• The oldest block in the buffer is emptied for
the new block
O Clock – Let blocks in buffer have second
chance to live.
• Clock wisely, buffer manager loops through
blocks in buffer and looks for block that
hasn’t been accessed for two rounds. Then,
replace this block with new block.
BUFFER MANAGEMENT STRATEGIES
0
0
1
1
1
0
0
Start point to search a
0 flag
the buffer with
a 0 flag will be
replaced The flag will
be set to 0
By next time the hand reaches
it, if the content of this buffer
is not accessed, i.e. flag=0,
this buffer will be replaced.
That’s “Second Chance”.
1
THE RELATIONSHIP BETWEEN PHYSICAL
OPERATOR SELECTION AND BUFFER
MANAGEMENT
O The buffer manager may not be willing or able to
guarantee the availability of M buffers when the query
is executed
• i.e multiple queries
O Two questions arise
• Can the algorithm adapt to the changes in the value
of M, the number of main-memory buffers
available?
• When the expected M buffers are not available, and
some blocks that are expected to be in the memory
have actually been moved to disk by the buffer
manager, how does the buffer-replacement strategy
used by the buffer manager impact the number of
additional I/O’s that must be performed?
BUFFER ALLOCATION FOR
CONCURRENT TRANSACTIONS
OThe buffer allocation algorithm of the
buffer management component
distributes the available buffer frames
among the concurrent database
transactions. It is closely related to the
page replacement algorithm; in some
cases, there is only one algorithm used
both to distribute buffer frames to
transactions and to make
CLASSIFICATION OF BUFFER
ALLOCATION ALGORITHMS
O Buffer allocation algorithms can be subdivided
into local and global algorithms.
O The main disadvantage of static allocation
(whether transaction oriented or page-type
oriented) is its inflexibility in situations where
the DBMS load changes frequently.
DYNAMIC BUFFER ALLOCATION
USING LOCALALGORITHMS
O The best-known dynamic storage allocation algorithm
is Denning’s working-set algorithm as defined in which
can be used to describe locality in the reference
behavior of programs or database transactions.
O The working-set W(t, T) of a transaction is defined as
the set of pages referenced by the transaction during the
last 7 references at time t. 7 is called window size. w (c,
7) = 1 lV(t, 7) 1 is the working-set size at time t. Figure
6 shows examples of working-sets and their sizes.
PROBLEMS RELATED TO DBMS
BUFFER MANAGEMENT
O DBMS Buffer Management Under a Virtual OS :
Embedding a DBMS and an OS environment, in which it
is usually treated like a normal application program, can result in
aggravating effects on buffer management.
The following kinds of fault occur:
Page faults:
The required page is contained in the DBMS buffer but
is not currently in main memory. It has to be read by the OS from
the paging device.
Buffer faults:
The requested page is not found within the buffer. The
buffer page selected for replacement.
Double-page faults:
A logical reference to a database page fails and the
buffer page to be replaced is not in main memory.
CONCLUSIONS
O The interface requirements of a DBMS buffer
manager and introduced the concept of fixing
pages in a buffer to prevent their uncontrolled
replacement. The spectrum of possible
strategies for searching the buffer was then
discussed; hash techniques on buffer
information tables with overflow chaining are
recommended as the most efficient
implementation alternative for the buffer
search function.

Buffer management

  • 1.
    BUFFER MANAGEMENT BY G.KARTHIGA M.SC IT NADARSARASWATHI COLLEGE OF ARTS &SCIENCE THENI
  • 2.
    DATABASE BUFFERING O DatabaseIs Stored In Disk. O For Read And Write Database Operation Must Reside In Memory. O Force Policy: Modified Block Is Output To Disk Once A Transaction Commits. O No Force Policy: Modified Block Is Not Output To Disk Immediately If A Transaction Commits
  • 3.
    DB BUFFER MANAGEMENT– CONTENTS O Goal: realization of efficient, page-based in-memory processing. • maximum possible avoidance of physical I/O. • replacement algorithms without and with context knowledge. O Role of DB buffer management: course of access to the DB buffer. • comparison with similar functionality in operating systems (OS) . O Locality: • various measures for locality. • characterization by LRU-stack-depth distributions and reference density.
  • 4.
    DBMS LAYER Recovery Manager Transaction Manager Lock Manager DB Query optimization andexecution Relational operators File and access method Buffer management Disk space management queries
  • 5.
    SATISFYING PAGE REQUESTS For each frame in the pool, the DBMS buffer manager maintains  The pin_count variable: # of users of a page  The dirty variable: whether a page has been modified or not  If a page is requested and not in the pool, the DBMS buffer manager O Chooses a frame for replacement and increments its pin_count (a process known as pinning) O If frame is dirty, writes it back to disk O Reads the requested page into chosen frame
  • 6.
    BUFFER MANAGEMENT ARCHITECTURE O Thebuffer manager controls main memory directly, as in many relational DBMS’s O The buffer manager allocates buffers in virtual memory, allowing the operating system to decide which buffers are actually in main memory at any time and which are in the “swap space” on disk that the operating system manages. Many “main-memory” DBMS’s and “object-oriented” DBMS’s operate this way.
  • 7.
    BUFFER MANAGEMENT STRATEGIES OThe critical choice that the buffer manager must make is what block to throw out of the buff pool when the buffer is needed for newly requested block. There are some common strategies, such as in operating systems. These include: • LRU • FIFO • Clock • System Control
  • 8.
    O LRU –Least recently used • Throw out block that has not been read or written for the longest time. O FIFO – First in first out • The oldest block in the buffer is emptied for the new block O Clock – Let blocks in buffer have second chance to live. • Clock wisely, buffer manager loops through blocks in buffer and looks for block that hasn’t been accessed for two rounds. Then, replace this block with new block.
  • 9.
    BUFFER MANAGEMENT STRATEGIES 0 0 1 1 1 0 0 Startpoint to search a 0 flag the buffer with a 0 flag will be replaced The flag will be set to 0 By next time the hand reaches it, if the content of this buffer is not accessed, i.e. flag=0, this buffer will be replaced. That’s “Second Chance”. 1
  • 10.
    THE RELATIONSHIP BETWEENPHYSICAL OPERATOR SELECTION AND BUFFER MANAGEMENT O The buffer manager may not be willing or able to guarantee the availability of M buffers when the query is executed • i.e multiple queries O Two questions arise • Can the algorithm adapt to the changes in the value of M, the number of main-memory buffers available? • When the expected M buffers are not available, and some blocks that are expected to be in the memory have actually been moved to disk by the buffer manager, how does the buffer-replacement strategy used by the buffer manager impact the number of additional I/O’s that must be performed?
  • 11.
    BUFFER ALLOCATION FOR CONCURRENTTRANSACTIONS OThe buffer allocation algorithm of the buffer management component distributes the available buffer frames among the concurrent database transactions. It is closely related to the page replacement algorithm; in some cases, there is only one algorithm used both to distribute buffer frames to transactions and to make
  • 12.
    CLASSIFICATION OF BUFFER ALLOCATIONALGORITHMS O Buffer allocation algorithms can be subdivided into local and global algorithms. O The main disadvantage of static allocation (whether transaction oriented or page-type oriented) is its inflexibility in situations where the DBMS load changes frequently.
  • 13.
    DYNAMIC BUFFER ALLOCATION USINGLOCALALGORITHMS O The best-known dynamic storage allocation algorithm is Denning’s working-set algorithm as defined in which can be used to describe locality in the reference behavior of programs or database transactions. O The working-set W(t, T) of a transaction is defined as the set of pages referenced by the transaction during the last 7 references at time t. 7 is called window size. w (c, 7) = 1 lV(t, 7) 1 is the working-set size at time t. Figure 6 shows examples of working-sets and their sizes.
  • 14.
    PROBLEMS RELATED TODBMS BUFFER MANAGEMENT O DBMS Buffer Management Under a Virtual OS : Embedding a DBMS and an OS environment, in which it is usually treated like a normal application program, can result in aggravating effects on buffer management. The following kinds of fault occur: Page faults: The required page is contained in the DBMS buffer but is not currently in main memory. It has to be read by the OS from the paging device. Buffer faults: The requested page is not found within the buffer. The buffer page selected for replacement. Double-page faults: A logical reference to a database page fails and the buffer page to be replaced is not in main memory.
  • 15.
    CONCLUSIONS O The interfacerequirements of a DBMS buffer manager and introduced the concept of fixing pages in a buffer to prevent their uncontrolled replacement. The spectrum of possible strategies for searching the buffer was then discussed; hash techniques on buffer information tables with overflow chaining are recommended as the most efficient implementation alternative for the buffer search function.