The document discusses distributed transactions and two-phase commit protocol. It begins by defining a transaction and the ACID properties that must be ensured in a distributed system. It then describes two-phase commit protocol used to ensure atomic commits across multiple sites. The protocol involves two phases - in phase 1 the coordinator obtains a preliminary decision from participants to commit or abort, and in phase 2 it informs all participants of the final decision. The document also discusses how failures of sites and the coordinator are handled during the protocol. It concludes by covering alternative models of transaction processing using persistent messaging.
In transaction processing, databases, and computer networking, the two-phase commit protocol (2PC) is a type of atomic commitment protocol (ACP). ... The protocol achieves its goal even in many cases of temporary system failure (involving either process, network node, communication, etc. failures), and is thus widely used.
Trafodion brings a completely distributed scalable transaction management implementation integrated into HBase. It does not suffer from the scale and performance limitations of other transaction managers on HBase.
This presentation reviews the elegant architecture and how this architecture is leveraged to provide full ACID SQL transactional capabilities across multiple rows, tables, statements, and region servers. It discusses the life of a transaction from BEGIN WORK, to updates, to ABORT WORK, to COMMIT WORK, and then discusses recovery and high availability capabilities provided. An accompanying white paper goes into depth explaining this animated presentation in more detail.
Given the increasing interest for transaction managers on Hadoop, or to provide transactional capabilities for NoSQL users when needed, the Trafodion community can certainly open up this Distributed Transaction Management support to be leveraged by implementations other than Trafodion.
In transaction processing, databases, and computer networking, the two-phase commit protocol (2PC) is a type of atomic commitment protocol (ACP). ... The protocol achieves its goal even in many cases of temporary system failure (involving either process, network node, communication, etc. failures), and is thus widely used.
Trafodion brings a completely distributed scalable transaction management implementation integrated into HBase. It does not suffer from the scale and performance limitations of other transaction managers on HBase.
This presentation reviews the elegant architecture and how this architecture is leveraged to provide full ACID SQL transactional capabilities across multiple rows, tables, statements, and region servers. It discusses the life of a transaction from BEGIN WORK, to updates, to ABORT WORK, to COMMIT WORK, and then discusses recovery and high availability capabilities provided. An accompanying white paper goes into depth explaining this animated presentation in more detail.
Given the increasing interest for transaction managers on Hadoop, or to provide transactional capabilities for NoSQL users when needed, the Trafodion community can certainly open up this Distributed Transaction Management support to be leveraged by implementations other than Trafodion.
Transaction Processing; Concurrency control; ACID properties; Schedule and Discoverability; Serialization; Concurrency control and Recovery; Two Phase locking; Deadlock Shadow Paging
Transaction Processing Monitors represent an early type of middleware that is still widely used for performing distributed transactions involving multiple databases.
Usually TPMs employ the two phase commit protocol that ensures ACID properties (Atomicity, Consistency, Isolation, Durability) as in relational databases.
Distributed database system is collection of loosely coupled sites that are independeant of each other.
Distributed transaction model
Concurrency control
2 phase commit protocol
Transaction Processing; Concurrency control; ACID properties; Schedule and Discoverability; Serialization; Concurrency control and Recovery; Two Phase locking; Deadlock Shadow Paging
Transaction Processing Monitors represent an early type of middleware that is still widely used for performing distributed transactions involving multiple databases.
Usually TPMs employ the two phase commit protocol that ensures ACID properties (Atomicity, Consistency, Isolation, Durability) as in relational databases.
Distributed database system is collection of loosely coupled sites that are independeant of each other.
Distributed transaction model
Concurrency control
2 phase commit protocol
Introduction to transaction processing concepts and theoryZainab Almugbel
Modified version of Chapter 21 of the book Fundamentals_of_Database_Systems,_6th_Edition with review questions
as part of database management system course
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptxR&R Consult
CFD analysis is incredibly effective at solving mysteries and improving the performance of complex systems!
Here's a great example: At a large natural gas-fired power plant, where they use waste heat to generate steam and energy, they were puzzled that their boiler wasn't producing as much steam as expected.
R&R and Tetra Engineering Group Inc. were asked to solve the issue with reduced steam production.
An inspection had shown that a significant amount of hot flue gas was bypassing the boiler tubes, where the heat was supposed to be transferred.
R&R Consult conducted a CFD analysis, which revealed that 6.3% of the flue gas was bypassing the boiler tubes without transferring heat. The analysis also showed that the flue gas was instead being directed along the sides of the boiler and between the modules that were supposed to capture the heat. This was the cause of the reduced performance.
Based on our results, Tetra Engineering installed covering plates to reduce the bypass flow. This improved the boiler's performance and increased electricity production.
It is always satisfying when we can help solve complex challenges like this. Do your systems also need a check-up or optimization? Give us a call!
Work done in cooperation with James Malloy and David Moelling from Tetra Engineering.
More examples of our work https://www.r-r-consult.dk/en/cases-en/
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
Courier management system project report.pdfKamal Acharya
It is now-a-days very important for the people to send or receive articles like imported furniture, electronic items, gifts, business goods and the like. People depend vastly on different transport systems which mostly use the manual way of receiving and delivering the articles. There is no way to track the articles till they are received and there is no way to let the customer know what happened in transit, once he booked some articles. In such a situation, we need a system which completely computerizes the cargo activities including time to time tracking of the articles sent. This need is fulfilled by Courier Management System software which is online software for the cargo management people that enables them to receive the goods from a source and send them to a required destination and track their status from time to time.
Quality defects in TMT Bars, Possible causes and Potential Solutions.PrashantGoswami42
Maintaining high-quality standards in the production of TMT bars is crucial for ensuring structural integrity in construction. Addressing common defects through careful monitoring, standardized processes, and advanced technology can significantly improve the quality of TMT bars. Continuous training and adherence to quality control measures will also play a pivotal role in minimizing these defects.
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
Vaccine management system project report documentation..pdfKamal Acharya
The Division of Vaccine and Immunization is facing increasing difficulty monitoring vaccines and other commodities distribution once they have been distributed from the national stores. With the introduction of new vaccines, more challenges have been anticipated with this additions posing serious threat to the already over strained vaccine supply chain system in Kenya.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
Event Management System Vb Net Project Report.pdfKamal Acharya
In present era, the scopes of information technology growing with a very fast .We do not see any are untouched from this industry. The scope of information technology has become wider includes: Business and industry. Household Business, Communication, Education, Entertainment, Science, Medicine, Engineering, Distance Learning, Weather Forecasting. Carrier Searching and so on.
My project named “Event Management System” is software that store and maintained all events coordinated in college. It also helpful to print related reports. My project will help to record the events coordinated by faculties with their Name, Event subject, date & details in an efficient & effective ways.
In my system we have to make a system by which a user can record all events coordinated by a particular faculty. In our proposed system some more featured are added which differs it from the existing system such as security.
Democratizing Fuzzing at Scale by Abhishek Aryaabh.arya
Presented at NUS: Fuzzing and Software Security Summer School 2024
This keynote talks about the democratization of fuzzing at scale, highlighting the collaboration between open source communities, academia, and industry to advance the field of fuzzing. It delves into the history of fuzzing, the development of scalable fuzzing platforms, and the empowerment of community-driven research. The talk will further discuss recent advancements leveraging AI/ML and offer insights into the future evolution of the fuzzing landscape.
Automobile Management System Project Report.pdfKamal Acharya
The proposed project is developed to manage the automobile in the automobile dealer company. The main module in this project is login, automobile management, customer management, sales, complaints and reports. The first module is the login. The automobile showroom owner should login to the project for usage. The username and password are verified and if it is correct, next form opens. If the username and password are not correct, it shows the error message.
When a customer search for a automobile, if the automobile is available, they will be taken to a page that shows the details of the automobile including automobile name, automobile ID, quantity, price etc. “Automobile Management System” is useful for maintaining automobiles, customers effectively and hence helps for establishing good relation between customer and automobile organization. It contains various customized modules for effectively maintaining automobiles and stock information accurately and safely.
When the automobile is sold to the customer, stock will be reduced automatically. When a new purchase is made, stock will be increased automatically. While selecting automobiles for sale, the proposed software will automatically check for total number of available stock of that particular item, if the total stock of that particular item is less than 5, software will notify the user to purchase the particular item.
Also when the user tries to sale items which are not in stock, the system will prompt the user that the stock is not enough. Customers of this system can search for a automobile; can purchase a automobile easily by selecting fast. On the other hand the stock of automobiles can be maintained perfectly by the automobile shop manager overcoming the drawbacks of existing system.
Overview of the fundamental roles in Hydropower generation and the components involved in wider Electrical Engineering.
This paper presents the design and construction of hydroelectric dams from the hydrologist’s survey of the valley before construction, all aspects and involved disciplines, fluid dynamics, structural engineering, generation and mains frequency regulation to the very transmission of power through the network in the United Kingdom.
Author: Robbie Edward Sayers
Collaborators and co editors: Charlie Sims and Connor Healey.
(C) 2024 Robbie E. Sayers
2. 2
Lecture Outcome
After this lecture, the students will be able to learn the following:
1. The concept of distributed transactions and transactional
properties
2. The requirement of COMMIT protocol in distributed transaction
and the COMMIT protocols
3. The application of COMMIT protocol in distributed transaction in
failure handling
4. Transaction processing using persistent messaging
3. 3
Transaction Concept
A transaction is a unit of program execution that accesses and
possibly updates various data items.
E.g., transaction to transfer $50 from account A to account B:
1. read(A)
2. A := A – 50
3. write(A)
4. read(B)
5. B := B + 50
6. write(B)
Two main issues to deal with:
Failures of various kinds, such as hardware failures and
system crashes
Concurrent execution of multiple transactions
4. 4
Required Properties of a Transaction
Consider a transaction to transfer $50 from account A to account B:
1. read(A)
2. A := A – 50
3. write(A)
4. read(B)
5. B := B + 50
6. write(B)
Atomicity requirement
If the transaction fails after step 3 and before step 6, money will be
“lost” leading to an inconsistent database state
Failure could be due to software or hardware
The system should ensure that updates of a partially executed
transaction are not reflected in the database
Durability requirement — once the user has been notified that the
transaction has completed (i.e., the transfer of the $50 has taken place), the
updates to the database by the transaction must persist even if there are
software or hardware failures.
5. 5
Required Properties of a Transaction (Cont.)
Consistency requirement in above example:
The sum of A and B is unchanged by the execution of the transaction
In general, consistency requirements include
Explicitly specified integrity constraints such as primary keys and
foreign keys
Implicit integrity constraints
– e.g., sum of balances of all accounts, minus sum of loan
amounts must equal value of cash-in-hand
A transaction, when starting to execute, must see a consistent database.
During transaction execution the database may be temporarily
inconsistent.
When the transaction completes successfully the database must be
consistent
Erroneous transaction logic can lead to inconsistency
6. 6
Required Properties of a Transaction (Cont.)
Isolation requirement — if between steps 3 and 6 (of the fund transfer
transaction) , another transaction T2 is allowed to access the partially
updated database, it will see an inconsistent database (the sum A + B
will be less than it should be).
T1 T2
1. read(A)
2. A := A – 50
3. write(A)
read(A), read(B), print(A+B)
4. read(B)
5. B := B + 50
6. write(B
Isolation can be ensured trivially by running transactions serially
That is, one after the other.
However, executing multiple transactions concurrently has significant
benefits, as we will see later.
7. 7
ACID Properties
Atomicity. Either all operations of the transaction are properly reflected
in the database or none are.
Consistency. Execution of a transaction in isolation preserves the
consistency of the database.
Isolation. Although multiple transactions may execute concurrently,
each transaction must be unaware of other concurrently executing
transactions. Intermediate transaction results must be hidden from other
concurrently executed transactions.
That is, for every pair of transactions Ti and Tj, it appears to Ti that
either Tj, finished execution before Ti started, or Tj started execution
after Ti finished.
Durability. After a transaction completes successfully, the changes it
has made to the database persist, even if there are system failures.
A transaction is a unit of program execution that accesses and possibly
updates various data items. To preserve the integrity of data the database
system must ensure:
8. 8
Distributed Transactions
Transaction may access data at several sites.
Each site has a local transaction manager responsible for:
Maintaining a log for recovery purposes
Participating in coordinating the concurrent execution of the
transactions executing at that site.
Each site has a transaction coordinator, which is responsible for:
Starting the execution of transactions that originate at the site.
Distributing subtransactions at appropriate sites for execution.
Coordinating the termination of each transaction that originates at
the site, which may result in the transaction being committed at all
sites or aborted at all sites.
10. 10
Lecture Outcome
After this lecture, the students will be able to learn the following:
1. The concept of distributed transactions and transactional
properties
2. The requirement of COMMIT protocol in distributed transaction
and the COMMIT protocols
3. The application of COMMIT protocol in distributed transaction in
failure handling
4. Transaction processing using persistent messaging
11. 11
System Failure Modes
Failures unique to distributed systems:
Failure of a site.
Loss of massages
Handled by network transmission control protocols such as
TCP-IP
Failure of a communication link
Handled by network protocols, by routing messages via
alternative links
Network partition
A network is said to be partitioned when it has been split into
two or more subsystems that lack any connection between
them
– Note: a subsystem may consist of a single node
Network partitioning and site failures are generally indistinguishable.
12. 12
Commit Protocols
Commit protocols are used to ensure atomicity across sites
a transaction which executes at multiple sites must either be
committed at all the sites, or aborted at all the sites.
not acceptable to have a transaction committed at one site and
aborted at another
The two-phase commit (2PC) protocol is widely used
The three-phase commit (3PC) protocol is more complicated and
more expensive, but avoids some drawbacks of two-phase commit
protocol. This protocol is not used in practice.
13. 13
Two Phase Commit Protocol (2PC)
Assumes fail-stop model – failed sites simply stop working, and do
not cause any other harm, such as sending incorrect messages to
other sites.
Execution of the protocol is initiated by the coordinator after the last
step of the transaction has been reached.
The protocol involves all the local sites at which the transaction
executed
Let T be a transaction initiated at site Si, and let the transaction
coordinator at Si be Ci
14. 14
Phase 1: Obtaining a Decision
Coordinator asks all participants to prepare to commit transaction Ti.
Ci adds the records <prepare T> to the log and forces log to
stable storage
sends prepare T messages to all sites at which T executed
Upon receiving message, transaction manager at site determines if it
can commit the transaction
if not, add a record <no T> to the log and send abort T message
to Ci
if the transaction can be committed, then:
add the record <ready T> to the log
force all records for T to stable storage
send ready T message to Ci
15. 15
Phase 2: Recording the Decision
T can be committed of Ci received a ready T message from all the
participating sites: otherwise T must be aborted.
Coordinator adds a decision record, <commit T> or <abort T>, to the
log and forces record onto stable storage. Once the record stable
storage it is irrevocable (even if failures occur)
Coordinator sends a message to each participant informing it of the
decision (commit or abort)
Participants take appropriate action locally.
16. 16
Lecture Outcome
After this lecture, the students will be able to learn the following:
1. The concept of distributed transactions and transactional
properties
2. The requirement of COMMIT protocol in distributed transaction
and the COMMIT protocols
3. The application of COMMIT protocol in distributed transaction in
failure handling
4. Transaction processing using persistent messaging
17. 17
Handling of Failures - Site Failure
When site Si recovers, it examines its log to determine the fate of
transactions active at the time of the failure.
Log contain <commit T> record: txn had completed, nothing to be done
Log contains <abort T> record: txn had completed, nothing to be done
Log contains <ready T> record: site must consult Ci to determine the
fate of T.
If T committed, redo (T); write <commit T> record
If T aborted, undo (T)
The log contains no log records concerning T:
Implies that Sk failed before responding to the prepare T message
from Ci
since the failure of Sk precludes the sending of such a response,
coordinator C1 must abort T
Sk must execute undo (T)
18. 18
Handling of Failures- Coordinator Failure
If coordinator fails while the commit protocol for T is executing then
participating sites must decide on T’s fate:
1. If an active site contains a <commit T> record in its log, then T must be
committed.
2. If an active site contains an <abort T> record in its log, then T must be
aborted.
3. If some active participating site does not contain a <ready T> record in its
log, then the failed coordinator Ci cannot have decided to commit T.
Can therefore abort T; however, such a site must reject any
subsequent <prepare T> message from Ci
4. If none of the above cases holds, then all active sites must have a <ready
T> record in their logs, but no additional control records (such as <abort
T> of <commit T>).
In this case active sites must wait for Ci to recover, to find decision.
Blocking problem: active sites may have to wait for failed coordinator to
recover.
19. 19
Handling of Failures - Network Partition
If the coordinator and all its participants remain in one partition, the
failure has no effect on the commit protocol.
If the coordinator and its participants belong to several partitions:
Sites that are not in the partition containing the coordinator think
the coordinator has failed, and execute the protocol to deal with
failure of the coordinator.
No harm results, but sites may still have to wait for decision
from coordinator.
The coordinator and the sites are in the same partition as the
coordinator think that the sites in the other partition have failed, and
follow the usual commit protocol.
Again, no harm results
20. 20
Recovery and Concurrency Control
In-doubt transactions have a <ready T>, but neither a
<commit T>, nor an <abort T> log record.
The recovering site must determine the commit-abort status of such
transactions by contacting other sites; this can slow and potentially
block recovery.
Recovery algorithms can note lock information in the log.
Instead of <ready T>, write out <ready T, L> L = list of locks held
by T when the log is written (read locks can be omitted).
For every in-doubt transaction T, all the locks noted in the
<ready T, L> log record are reacquired.
After lock reacquisition, transaction processing can resume; the
commit or rollback of in-doubt transactions is performed concurrently
with the execution of new transactions.
21. 21
Three Phase Commit (3PC)
Assumptions:
No network partitioning
At any point, at least one site must be up.
At most K sites (participants as well as coordinator) can fail
Phase 1: Obtaining Preliminary Decision: Identical to 2PC Phase 1.
Every site is ready to commit if instructed to do so
Phase 2 of 2PC is split into 2 phases, Phase 2 and Phase 3 of 3PC
In phase 2 coordinator makes a decision as in 2PC (called the pre-commit
decision) and records it in multiple (at least K) sites
In phase 3, coordinator sends commit/abort message to all participating
sites,
Under 3PC, knowledge of pre-commit decision can be used to commit despite
coordinator failure
Avoids blocking problem as long as < K sites fail
Drawbacks:
higher overheads
assumptions may not be satisfied in practice
22. 22
Lecture Outcome
After this lecture, the students will be able to learn the following:
1. The concept of distributed transactions and transactional
properties
2. The requirement of COMMIT protocol in distributed transaction
and the COMMIT protocols
3. The application of COMMIT protocol in distributed transaction in
failure handling
4. Transaction processing using persistent messaging
23. 23
Alternative Models of Transaction
Processing
Notion of a single transaction spanning multiple sites is inappropriate
for many applications
E.g. transaction crossing an organizational boundary
No organization would like to permit an externally initiated
transaction to block local transactions for an indeterminate period
Alternative models carry out transactions by sending messages
Code to handle messages must be carefully designed to ensure
atomicity and durability properties for updates
Isolation cannot be guaranteed, in that intermediate stages are
visible, but code must ensure no inconsistent states result due
to concurrency
Persistent messaging systems are systems that provide
transactional properties to messages
Messages are guaranteed to be delivered exactly once
Will discuss implementation techniques later
24. 24
Alternative Models (Cont.)
Motivating example: funds transfer between two banks
Two phase commit would have the potential to block updates on the
accounts involved in funds transfer
Alternative solution:
Debit money from source account and send a message to other
site
Site receives message and credits destination account
Messaging has long been used for distributed transactions (even
before computers were invented!)
Atomicity issue
once transaction sending a message is committed, message must
guaranteed to be delivered
Guarantee as long as destination site is up and reachable, code to
handle undeliverable messages must also be available
– e.g. credit money back to source account.
If sending transaction aborts, message must not be sent
25. 25
Error Conditions with Persistent
Messaging
Code to handle messages has to take care of variety of failure situations
(even assuming guaranteed message delivery)
E.g. if destination account does not exist, failure message must be
sent back to source site
When failure message is received from destination site, or
destination site itself does not exist, money must be deposited back
in source account
Problem if source account has been closed
– get humans to take care of problem
User code executing transaction processing using 2PC does not have to
deal with such failures
There are many situations where extra effort of error handling is worth
the benefit of absence of blocking
E.g. pretty much all transactions across organizations
26. 26
Persistent Messaging and Workflows
Workflows provide a general model of transactional processing
involving multiple sites and possibly human processing of certain
steps
E.g. when a bank receives a loan application, it may need to
Contact external credit-checking agencies
Get approvals of one or more managers
and then respond to the loan application
We study workflows in Chapter 25
Persistent messaging forms the underlying infrastructure for
workflows in a distributed environment
27. 27
Implementation of Persistent Messaging
Sending site protocol.
When a transaction wishes to send a persistent message, it writes a
record containing the message in a special relation
messages_to_send; the message is given a unique message
identifier.
A message delivery process monitors the relation, and when a new
message is found, it sends the message to its destination.
The message delivery process deletes a message from the relation
only after it receives an acknowledgment from the destination site.
If it receives no acknowledgement from the destination site, after
some time it sends the message again. It repeats this until an
acknowledgment is received.
If after some period of time, that the message is undeliverable,
exception handling code provided by the application is invoked
to deal with the failure.
Writing the message to a relation and processing it only after the
transaction commits ensures that the message will be delivered if and
only if the transaction commits.
28. 28
Implementation of Persistent Messaging
(Cont.)
Receiving site protocol.
When a site receives a persistent message, it runs a transaction that
adds the message to a received_messages relation
provided message identifier is not already present in the relation
After the transaction commits, or if the message was already present
in the relation, the receiving site sends an acknowledgment back to
the sending site.
Note that sending the acknowledgment before the transaction
commits is not safe, since a system failure may then result in loss
of the message.
In many messaging systems, it is possible for messages to get
delayed arbitrarily, although such delays are very unlikely.
Each message is given a timestamp, and if the timestamp of a
received message is older than some cutoff, the message is
discarded.
All messages recorded in the received messages relation that are
older than the cutoff can be deleted.
29. 29
Concurrency Control
Learning Outcome:
After this lecture, the students will be able to learn and apply the following:
1. The basic concept of concurrency control using lock-based protocol
2. How can lock-based protocol be used in distributed database using –
Single lock manager approach
Distributed Lock manager approach
Primary copy
Majority protocol
Biased protocol
How can transaction serialazability be maintained using time-stamp
ordering protocol
How can the deadlock be handled using wait-for graphs
30. 30
Concurrency Control
Modify concurrency control schemes for use in distributed environment.
We assume that each site participates in the execution of a commit
protocol to ensure global transaction automicity.
We assume all replicas of any item are updated
Will see how to relax this in case of site failures later
31. 31
Atomocity Using Lock-Based Protocols
A lock is a mechanism to control concurrent access to a data
item
Data items can be locked in two modes :
1. exclusive (X) mode. Data item can be both read as well as
written. X-lock is requested using lock-X instruction.
2. shared (S) mode. Data item can only be read. S-lock is
requested using lock-S instruction.
Lock requests are made to the concurrency-control manager
by the programmer. Transaction can proceed only after
request is granted.
32. 32
Lock-Based Protocols (Cont.)
Lock-compatibility matrix
A transaction may be granted a lock on an item if the requested
lock is compatible with locks already held on the item by other
transactions
Any number of transactions can hold shared locks on an item,
But if any transaction holds an exclusive on the item no other
transaction may hold any lock on the item.
If a lock cannot be granted, the requesting transaction is made to
wait till all incompatible locks held by other transactions have
been released. The lock is then granted.
33. 33
Lock-Based Protocols (Cont.)
Example of a transaction performing locking:
T2: lock-S(A);
read (A);
unlock(A);
lock-S(B);
read (B);
unlock(B);
display(A+B)
Locking as above is not sufficient to guarantee serializability
— if A and B get updated in-between the read of A and B,
the displayed sum would be wrong.
A locking protocol is a set of rules followed by all
transactions while requesting and releasing locks. Locking
protocols restrict the set of possible schedules.
34. 34
Automatic Acquisition of Locks
A transaction Ti issues the standard read/write instruction,
without explicit locking calls.
The operation read(D) is processed as:
if Ti has a lock on D
then
read(D)
else begin
if necessary wait until no other
transaction has a lock-X on D
grant Ti a lock-S on D;
read(D)
end
35. 35
Automatic Acquisition of Locks (Cont.)
write(D) is processed as:
if Ti has a lock-X on D
then
write(D)
else begin
if necessary wait until no other transaction has any lock on D,
if Ti has a lock-S on D
then
upgrade lock on D to lock-X
else
grant Ti a lock-X on D
write(D)
end;
All locks are released after commit or abort
Example of T1 (Q1), T2(Q2), T3(Update) ……
36. 36
Concurrency Control
Learning Outcome:
After this lecture, the students will be able to learn and apply the following:
1. The basic concept of concurrency control using lock-based protocol
2. How can lock-based protocol be used in distributed database using –
Single lock manager approach
Distributed Lock manager approach
Primary copy
Majority protocol
Biased protocol
How can transaction serialazability be maintained using time-stamp
ordering protocol
How can the deadlock be handled using wait-for graphs
37. 37
Single-Lock-Manager Approach
System maintains a single lock manager that resides in a single
chosen site, say Si
When a transaction needs to lock a data item, it sends a lock request
to Si and lock manager determines whether the lock can be granted
immediately
If yes, lock manager sends a message to the site which initiated
the request
If no, request is delayed until it can be granted, at which time a
message is sent to the initiating site
38. 38
Single-Lock-Manager Approach (Cont.)
The transaction can read the data item from any one of the sites at
which a replica of the data item resides.
Writes must be performed on all replicas of a data item
Advantages of scheme:
Simple implementation
Simple deadlock handling
Disadvantages of scheme are:
Bottleneck: lock manager site becomes a bottleneck
Vulnerability: system is vulnerable to lock manager site failure.
39. 39
Concurrency Control
Learning Outcome:
After this lecture, the students will be able to learn and apply the following:
1. The basic concept of concurrency control using lock-based protocol
2. How can lock-based protocol be used in distributed database using –
Single lock manager approach
Distributed Lock manager approach
Primary copy
Majority protocol
Biased protocol
How can transaction serializability be maintained using time-stamp
ordering protocol
How can the deadlock be handled using wait-for graphs
40. 40
Distributed Lock Manager
In this approach, functionality of locking is implemented by lock
managers at each site
Lock managers control access to local data items
But special protocols may be used for replicas
Advantage: work is distributed and can be made robust to failures
Disadvantage: deadlock detection is more complicated
Lock managers cooperate for deadlock detection
More on this later
Several variants of this approach
Primary copy
Majority protocol
Biased protocol
Quorum consensus
41. 41
Concurrency Control
Learning Outcome:
After this lecture, the students will be able to learn and apply the following:
1. The basic concept of concurrency control using lock-based protocol
2. How can lock-based protocol be used in distributed database using –
Single lock manager approach
Distributed Lock manager approach
Primary copy
Majority protocol
Biased protocol
How can transaction serialazability be maintained using time-stamp
ordering protocol
How can the deadlock be handled using wait-for graphs
42. 42
Primary Copy
Choose one replica of data item to be the primary copy.
Site containing the replica is called the primary site for that data
item
Different data items can have different primary sites
When a transaction needs to lock a data item Q, it requests a lock at
the primary site of Q.
Implicitly gets lock on all replicas of the data item
Benefit
Concurrency control for replicated data handled similarly to
unreplicated data - simple implementation.
Drawback
If the primary site of Q fails, Q is inaccessible even though other
sites containing a replica may be accessible.
43. 43
Concurrency Control
Learning Outcome:
After this lecture, the students will be able to learn and apply the following:
1. The basic concept of concurrency control using lock-based protocol
2. How can lock-based protocol be used in distributed database using –
Single lock manager approach
Distributed Lock manager approach
Primary copy
Majority protocol
Biased protocol
How can transaction serialazability be maintained using time-stamp
ordering protocol
How can the deadlock be handled using wait-for graphs
44. 44
Majority Protocol
Local lock manager at each site administers lock and unlock requests
for data items stored at that site.
When a transaction wishes to lock an unreplicated data item Q
residing at site Si, a message is sent to Si ‘s lock manager.
If Q is locked in an incompatible mode, then the request is delayed
until it can be granted.
When the lock request can be granted, the lock manager sends a
message back to the initiator indicating that the lock request has
been granted.
45. 45
Majority Protocol (Cont.)
In case of replicated data
If Q is replicated at n sites, then a lock request message must be
sent to more than half of the n sites in which Q is stored.
The transaction does not operate on Q until it has obtained a lock
on a majority of the replicas of Q.
When writing the data item, transaction performs writes on all
replicas.
Benefit
Can be used even when some sites are unavailable
details on how handle writes in the presence of site failure later
Drawback
Requires 2(n/2 + 1) messages for handling lock requests, and (n/2
+ 1) messages for handling unlock requests.
Potential for deadlock even with single item - e.g., each of 3
transactions may have locks on 1/3rd of the replicas of a data.
Example: Sites-S1, S2, S3 and S4, Transactions T1 and T2
required X-lock. T1 get S1 & S3 and wait for one. T2 gets S2 & S4
and wait for one---- Deadlock.
46. 46
Concurrency Control
Learning Outcome:
After this lecture, the students will be able to learn and apply the following:
1. The basic concept of concurrency control using lock-based protocol
2. How can lock-based protocol be used in distributed database using –
Single lock manager approach
Distributed Lock manager approach
Primary copy
Majority protocol
Biased protocol
How can transaction serializability be maintained using time-stamp
ordering protocol
How can the deadlock be handled using wait-for graphs
47. 47
Biased Protocol
Local lock manager at each site as in majority protocol, however,
requests for shared locks are handled differently than requests for
exclusive locks.
Shared locks. When a transaction needs to lock data item Q, it simply
requests a lock on Q from the lock manager at one site containing a
replica of Q.
Exclusive locks. When transaction needs to lock data item Q, it
requests a lock on Q from the lock manager at all sites containing a
replica of Q.
Advantage - imposes less overhead on read operations.
Disadvantage - additional overhead on writes
48. 48
Concurrency Control
Learning Outcome:
After this lecture, the students will be able to learn and apply the following:
1. The basic concept of concurrency control using lock-based protocol
2. How can lock-based protocol be used in distributed database using –
Single lock manager approach
Distributed Lock manager approach
Primary copy
Majority protocol
Biased protocol
Quorum Consensus Protocol
How can transaction serializability be maintained using time-stamp
ordering protocol
How can the deadlock be handled using wait-for graphs
49. 49
Quorum Consensus Protocol
A generalization of both majority and biased protocols
Each site is assigned a weight.
Let S be the total of all site weights
Choose two values read quorum Qr and write quorum Qw
Such that Qr + Qw > S and 2 * Qw > S
Quorums can be chosen (and S computed) separately for each
item
Each read must lock enough replicas that the sum of the site weights
is >= Qr
Each write must lock enough replicas that the sum of the site weights
is >= Qw
For now we assume all replicas are written
Extensions to allow some sites to be unavailable described later
50. 50
Concurrency Control
Learning Outcome:
After this lecture, the students will be able to learn and apply the following:
1. The basic concept of concurrency control using lock-based protocol
2. How can lock-based protocol be used in distributed database using –
Single lock manager approach
Distributed Lock manager approach
Primary copy
Majority protocol
Biased protocol
Quorum Consensus Protocol
How can transaction serialazability be maintained using time-stamp
ordering protocol
How can the deadlock be handled using wait-for graphs
51. 51
Timestamping
Timestamp based concurrency-control protocols can be used in
distributed systems
Each transaction must be given a unique timestamp
Main problem: how to generate a timestamp in a distributed fashion
Each site generates a unique local timestamp using either a logical
counter or the local clock.
Global unique timestamp is obtained by concatenating the unique
local timestamp with the unique identifier.
52. 52
Timestamping (Cont.)
A site with a slow clock will assign smaller timestamps
Still logically correct: serializability not affected
But: “disadvantages” transactions
To fix this problem
Define within each site Si a logical clock (LCi), which generates
the unique local timestamp
Require that Si advance its logical clock whenever a request is
received from a transaction Ti with timestamp < x,y> and x is
greater that the current value of LCi.
In this case, site Si advances its logical clock to the value x + 1.
53. 53
Concurrency Control
Learning Outcome:
After this lecture, the students will be able to learn and apply the following:
1. The basic concept of concurrency control using lock-based protocol
2. How can lock-based protocol be used in distributed database using –
Single lock manager approach
Distributed Lock manager approach
Primary copy
Majority protocol
Biased protocol
Quorum Consensus Protocol
How can transaction serialazability be maintained using time-stamp
ordering protocol
How can the deadlock be handled using wait-for graphs
54. 54
Deadlock Handling
Consider the following two transactions and history, with item X and
transaction T1 at site 1, and item Y and transaction T2 at site 2:
T1: write (X)
write (Y)
T2: write (Y)
write (X)
X-lock on X
write (X) X-lock on Y
write (Y)
wait for X-lock on X
Wait for X-lock on Y
Result: deadlock which cannot be detected locally at either site
55. 55
Centralized Approach
A global wait-for graph is constructed and maintained in a single site;
the deadlock-detection coordinator
Real graph: Real, but unknown, state of the system.
Constructed graph:Approximation generated by the controller
during the execution of its algorithm .
the global wait-for graph can be constructed when:
a new edge is inserted in or removed from one of the local wait-
for graphs.
a number of changes have occurred in a local wait-for graph.
the coordinator needs to invoke cycle-detection.
If the coordinator finds a cycle, it selects a victim and notifies all sites.
The sites roll back the victim transaction.
58. 58
False Cycles (Cont.)
Suppose that starting from the state shown in figure,
1. T2 releases resources at S1
resulting in a message remove T1 T2 message from the
Transaction Manager at site S1 to the coordinator)
2. And then T2 requests a resource held by T3 at site S2
resulting in a message insert T2 T3 from S2 to the coordinator
Suppose further that the insert message reaches before the delete
message
this can happen due to network delays
The coordinator would then find a false cycle
T1 T2 T3 T1
The false cycle above never existed in reality.
False cycles cannot occur if two-phase locking is used.
59. 59
Unnecessary Rollbacks
Unnecessary rollbacks may result when deadlock has indeed
occurred and a victim has been picked, and meanwhile one of the
transactions was aborted for reasons unrelated to the deadlock.
Unnecessary rollbacks can result from false cycles in the global wait-
for graph; however, likelihood of false cycles is low.
61. 61
Distributed Query Processing
For centralized systems, the primary criterion for measuring the cost
of a particular strategy is the number of disk accesses.
In a distributed system, other issues must be taken into account:
The cost of a data transmission over the network.
The potential gain in performance from having several sites
process parts of the query in parallel.
62. 62
Query Transformation
Translating algebraic queries on fragments.
It must be possible to construct relation r from its fragments
Replace relation r by the expression to construct relation r from its
fragments
Consider the horizontal fragmentation of the account relation into
account1 = branch_name = “Hillside” (account )
account2 = branch_name = “Valleyview” (account )
The query branch_name = “Hillside” (account ) becomes
branch_name = “Hillside” (account1 account2)
which is optimized into
branch_name = “Hillside” (account1) branch_name = “Hillside” (account2)
63. 63
Example Query (Cont.)
Since account1 has only tuples pertaining to the Hillside branch,
we can eliminate the selection operation.
Apply the definition of account2 to obtain
branch_name = “Hillside” ( branch_name = “Valleyview” (account )
This expression is the empty set regardless of the contents of the
account relation.
Final strategy is for the Hillside site to return account1 as the result
of the query.
64. 64
Simple Join Processing
Consider the following relational algebra expression in which the three
relations are neither replicated nor fragmented
account depositor branch
account is stored at site S1
depositor at S2
branch at S3
For a query issued at site SI, the system needs to produce the result at
site SI
65. 65
Possible Query Processing Strategies
Ship copies of all three relations to site SI and choose a strategy for
processing the entire locally at site SI.
Ship a copy of the account relation to site S2 and compute temp1 =
account depositor at S2. Ship temp1 from S2 to S3, and compute
temp2 = temp1 branch at S3. Ship the result temp2 to SI.
Devise similar strategies, exchanging the roles S1, S2, S3
Must consider following factors:
amount of data being shipped
cost of transmitting a data block between sites
relative processing speed at each site
66. 66
Semijoin Strategy
Let r1 be a relation with schema R1 stores at site S1
Let r2 be a relation with schema R2 stores at site S2
Evaluate the expression r1 r2 and obtain the result at S1.
1. Compute temp1 R1 R2 (r1) at S1.
2. Ship temp1 from S1 to S2.
3. Compute temp2 r2 temp1 at S2
4. Ship temp2 from S2 to S1.
5. Compute r1 temp2 at S1. This is the same as r1 r2.
67. 67
Formal Definition
The semijoin of r1 with r2, is denoted by:
r1 r2
it is defined by:
R1 (r1 r2)
Thus, r1 r2 selects those tuples of r1 that contributed to r1 r2.
In step 3 above, temp2=r2 r1.
For joins of several relations, the above strategy can be extended to a
series of semijoin steps.
68. 68
Join Strategies that Exploit Parallelism
Consider r1 r2 r3 r4 where relation ri is stored at site Si. The result
must be presented at site S1.
r1 is shipped to S2 and r1 r2 is computed at S2: simultaneously r3 is
shipped to S4 and r3 r4 is computed at S4
S2 ships tuples of (r1 r2) to S1 as they produced;
S4 ships tuples of (r3 r4) to S1
Once tuples of (r1 r2) and (r3 r4) arrive at S1 (r1 r2) (r3 r4) is
computed in parallel with the computation of (r1 r2) at S2 and the
computation of (r3 r4) at S4.