Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
DISTRIBUTED SYSTEMS
B. Tech IV/IV, II Semester
COMPUTER SCIENCE & ENGINEERING
2/6/2017 1@Copyright Material
UNIT VII
2/6/2017 2
Distributed Systems
Foundation Middleware Support Systems
Algorithms &
Shared Data
Characterization
of...
What we learnt till NOW
1. Characterization of a Distributed System
1. Need for a Distributed System
2. Evolution of a Dis...
What we learnt till NOW
5. Operating System, Types & Layers
1. Types of Operating System
2. Various Layers of OS
3. Protec...
What we have learnt in Unit VII
1. Introduction
1. Failure Assumption
2. Failure Detectors
2. Distributed Mutual Exclusion...
Topic of Contents #1
1. Introduction to Transactions
1. ACID Properties
2. Concurrency Control
3. Recovery from Aborts
2. ...
Topic of Contents #2
2/6/2017 7@Copyright Material
5. Replication
1. Introduction
2. System Model
3. The role of Group Com...
Learning Objective
Upon completion of this unit, students will be able to:
• Learn the need for a Transaction & the proper...
1. Introduction to Transaction
Transaction is specified by a client as a set of operations on objects
to be performed as a...
1. Introduction to Transaction
An example to understand more. Its terminology
• Each account is represented by a remote ob...
1. Introduction to Transaction
A basic Atomic Operation at the Server
• If client operations were synchronous, without tra...
1. Introduction to Transaction
Failure Model for Transactions
• Lampson’s failure model for distributed transactions deals...
1. Introduction to Transaction
Failure Model for Transactions
– There may be an arbitrary delay before a message arrives.
...
1. Introduction to Transaction
Failure Model for Transactions
• The fault model for permanent storage, processors and
comm...
1. Introduction to Transaction
Transactions:
In some situations, clients require a sequence of separate
requests to a serv...
1. Introduction to Transaction
ACID Properties
• Atomicity: a transaction must be all or nothing;
• Consistency: a transac...
1. Introduction to Transaction
Operations in Coordinator interface
• openTransaction() trans;
Starts a new transaction an...
1. Introduction to Transaction
2. CONCURRENCY CONTROL
• For the understanding, lets learn 2 situations, which
arise with i...
1. Introduction to Transaction
2. CONCURRENCY CONTROL
LOST UPDATE PROBLEM
A, B, C have initial balances of $100, $200, $30...
1. Introduction to Transaction
2. CONCURRENCY CONTROL
LOST UPDATE PROBLEM
If both the transactions have executed like show...
1. Introduction to Transaction
2. CONCURRENCY CONTROL
INCONSISTENT RETRIEVAL PROBLEM
A, B have initial balances of $200, $...
1. Introduction to Transaction
2. CONCURRENCY CONTROL
INCONSISTENT RETRIEVAL PROBLEM
If both the transactions were allowed...
1. Introduction to Transaction
2. CONCURRENCY CONTROL
• In the Lost Update Problem, the U’s update is lost
because T overw...
1. Introduction to Transaction
2. CONCURRENCY CONTROL
• In the Inconsistent Retrieval Problem, W’s retrievals
are inconsis...
1. Introduction to Transaction
2. CONCURRENCY CONTROL
Conflicting Operations
• When we say that a pair of operations confl...
2. CONCURRENCY CONTROL
Conflicting Operations
Are these transactions Serially Equivalent??
NO
• the ordering is not serial...
3. RECOVERY FROM ABORTS
• If a transaction aborts, the server must make sure that
other concurrent transactions do not see...
28
A dirty read when transaction T aborts
U has committed the results, so it cannot be undone
Transaction T:
a.getBalance(...
29
Premature writes - overwriting uncommitted values
Transaction T:
a.setBalance(105)
Transaction U:
a.setBalance(110)
$10...
30
1. Introduction to Transaction
3. RECOVERY FROM ABORTS
Cascading Aborts:
Suppose that U delays committing until after T...
31
2. Nested Transactions
Transactions may be composed of other transactions
– several transactions may be started from wi...
32
2. Nested Transactions
Advantages of Nested transactions over Flat transactions
• Subtransactions may run concurrently ...
33
2. Nested Transactions
Rules for Committing a Nested transactions
• A transaction may commit or abort only after its ch...
34
3. Summary of Transactions
We consider only transactions at a single server, they are:
1. atomic in the presence of con...
35
4. DistributedTransactions
1. Introduction
• Distributed transaction refers to a flat or nested
transaction that access...
36
4. DistributedTransactions
Client
X
Y
Z
X
Y
M
NT
1
T
2
T11
Client
P
T
T
12
T
21
T
22
(a) Flat transaction (b) Nested tr...
37
4. DistributedTransactions
1. Introduction
Nested Distributed Transaction:
a.withdraw(10)
c.deposit(10)
b.withdraw(20)
...
38
4. DistributedTransactions
1. Introduction
A Flat Distributed Transaction
Servers execute requests in a distributed tra...
39
4. DistributedTransactions
1. Introduction
A Flat Distributed Transaction
Scenario: a client’s (flat) banking transacti...
40
4. DistributedTransactions
2. Concurrency Control in Distributed Transactions
Each server manages a set of objects and ...
41
4. DistributedTransactions
2. Concurrency Control in Distributed Transactions
Concurrency control can be achieved using...
42
4. DistributedTransactions
2. Concurrency Control in Distributed Transactions
A. LOCKING
• In a distributed transaction...
43
4. DistributedTransactions
2. Concurrency Control in Distributed Transactions
A. LOCKING
Consider the following interle...
44
4. DistributedTransactions
2. Concurrency Control in Distributed Transactions
B. OPTIMISTIC CONCURRENCY CONTROL
Conside...
45
4. DistributedTransactions
2. Concurrency Control in Distributed Transactions
B. OPTIMISTIC CONCURRENCY CONTROL
• Optim...
46
4. DistributedTransactions
2. Concurrency Control in Distributed Transactions
C. TIMESTAMP ORDERING
• In a single serve...
47
4. DistributedTransactions
2. Concurrency Control in Distributed Transactions
C. TIMESTAMP ORDERING
• The servers of di...
48
4. Distributed Transactions
3. Distributed Deadlocks
• Deadlock is a situation in which two or more competing
actions a...
49
4. DistributedTransactions
3. Distributed Deadlocks
Scenario: Interleaving of transactions U, V, W
• Objects A, B are m...
50
4. Distributed Transactions
3. Distributed Deadlocks
• A deadlock cycle has alternate edges showing wait-for and held-b...
51
4. Distributed Transactions
3. Distributed Deadlocks
Deadlock Detection
• Local wait-for graphs can be built, e.g.
– se...
52
4. DistributedTransactions
3. Distributed Deadlocks
Phantom Deadlock
– a ‘deadlock’ that is detected, but is not really...
53
4. DistributedTransactions
3. Distributed Deadlocks
EDGE CHASING-A distributed approach for Deadlock detection
• a glob...
54
4. DistributedTransactions
3. Distributed Deadlocks
EDGE CHASING-A distributed approach for Deadlock detection
• 3 Step...
3. Distributed Deadlocks
EDGE CHASING-A distributed approach for Deadlock detection
Ex: Edge chasing starts with X sending...
4. Transaction Recovery
• Atomicity property of transactions
– durability and failure atomicity
– durability requires that...
4. Transaction Recovery
• The task of the Recovery Manager (RM) is:
– to save objects in permanent storage (in a recovery ...
4. Transaction Recovery
• Each server records an intentions list for each of its
currently active transactions
– an intent...
4. Transaction Recovery
* For distributed transactions we need information relating to the
2PhaseCommit as well as object ...
4. Transaction Recovery
A. Logging:
The recovery file represents a log of the history of all the
transactions at a server
...
4. Transaction Recovery
A. Logging:
Logging mechanism for example in Slide #23
– initial balances of A, B and C $100, $200...
4. Transaction Recovery
A. Logging:
RECOVERY WITH LOGS
• When a server is replaced after a crash
– it first sets default i...
4. Transaction Recovery
A. Logging:
REORGANIZING THE RECOVERY FILE
• Recovery Manager is responsible for reorganizing its ...
4. Transaction Recovery
B. Shadow Versions
• The shadow versions technique is an alternative way to organize a
recovery fi...
4. Transaction Recovery
B. Shadow Versions
• Shadow versions on their own are not sufficient for a server that
handles dis...
66
1. Introduction
It is the process of maintaining copies of data at multiple computers
Replication can provide the follo...
67
1. Introduction
Common requirement when data is replicated is
• Replication transparency
– Clients see logical objects ...
68
2. System Models & Role of Group Communication
• Each logical object is implemented by a collection of physical
copies ...
69
FE
Requests and
replies
C
ReplicaC
Service
Clients
Front ends
managers
RM
RM
FE
RM
5. Replication
2. System Models & Ro...
70
2. System Models & Role of Group Communication
5 STAGES IN PERFORMING A REQUEST
• Request
– The front end issues the re...
71
3. Role of Group communication
GROUP COMMUNCIATION
• Process groups are useful for managing replicated data
– But repli...
72
Join
Group
address
expansion
Multicast
communication
Group
send
Fail
Group membership
management
Leave
Process group
3....
73
3. Role of Group communication
SERVICES PROVIDED FOR PROCESS GROUPS
• A full membership service maintains group views, ...
74
4. Fault Tolerant Services
Provision of a service that is correct even if f processes
fail
Ex: Specification of bank ac...
75
4. Fault Tolerant Services
Linearizability is the strictest criterion for a replication system
• A replicated object se...
76
4. Fault Tolerant Services
• Sequential Consistency: a replicated shared object service is
sequentially consistent if f...
4. Fault Tolerant Services
A. Passive (primary-backup replication)
• There is at any time a single primary RM and one or m...
4. Fault Tolerant Services
A. Passive (primary-backup replication)
The five phases in performing a client request are as f...
4. Fault Tolerant Services
A. Passive (primary-backup replication)
Discussion
To survive f process crashes, f+1 RMs are re...
4. Fault Tolerant Services
B. Active Replication
• The RMs are state machines all playing the same role and
organised as a...
4. Fault Tolerant Services
B. Active Replication
5 Phases in performing Client Request
Request
• FE attaches a unique id a...
4. Fault Tolerant Services
B. Active Replication
1. A FE multicasts each request to the group of RMs
2. Requires totally o...
83
4. Fault Tolerant Services
B. Active Replication
Discussion
• As RMs are state machines we have sequential consistency
...
84
• Learned the need for a Transaction & its properties
• Understood the problems associated with
concurrency control Und...
85
1. Which of the following is not a property of transaction
a. Atomicity b. Durability
c. Isolation d. Concurrency
2. __...
86
4. A, B have initial balances of $200, $200. Which of
the following problem does the below figure
indicate
a. Lost upda...
87
5. A transaction cannot be recovered if it has committed with
a. Premature Write b. Dirty Read
c. Premature Read d. Dir...
88
8._______concurrency control requires transactions to operate in
a private workspace, so their modifications are not vi...
89
12. Transaction recovery is concerned with
a. Ensuring that a server’s objects are durable
b. Failure atomicity
c. Obje...
90
15. ______ uses a map to locate versions of the server’s objects in a
file called a version store
a. Checkpointing b. S...
91
KEY
1. D
2. A
3. D
4. C
5. B
6. D
7. D
8. C
9. A
10. A
11. B
12. D
13. C
14. B
15. B
16. D
17. B
18. A
Quiz
92
• Transaction: is specified by a client as a set of operations on
objects to be performed as an indivisible unit by the...
93
• Inconsistent Retrievals: It occurs when a retieval transaction
observes values that are involved in an ongoing updati...
94
• Flat Transaction: A flat client transaction completes each of its
requests before going on to the next one. Therefore...
95
• Wait-for graph: is a directed graph in which nodes represent
transactions and objects, and edges represent either an ...
96
• Replication Transparency makes a set of replicas appear as a single logical
object to a client.
• Replica: Each logic...
Upcoming SlideShare
Loading in …5
×

Transactions and Replications

57 views

Published on

Transactions and Replications

Published in: Education
  • Be the first to comment

  • Be the first to like this

Transactions and Replications

  1. 1. DISTRIBUTED SYSTEMS B. Tech IV/IV, II Semester COMPUTER SCIENCE & ENGINEERING 2/6/2017 1@Copyright Material
  2. 2. UNIT VII 2/6/2017 2 Distributed Systems Foundation Middleware Support Systems Algorithms & Shared Data Characterization of Distributed Systems System Models Inter Process Communication Distributed Objects Remote Invocation Operating System Distributed File System Coordination & Agreement Transactions & Replications @Copyright Material
  3. 3. What we learnt till NOW 1. Characterization of a Distributed System 1. Need for a Distributed System 2. Evolution of a Distributed System. 3. Examples of it & details on Resource Sharing 4. Challenges of it 2. How to build a Distributed System Model 1. Introduction to a System Model 2. Types of System models 3. Functionalities of each of these models 3. How to achieve Inter process communication 1. Learned sockets for establishing communication 2. Fundamentals of Protocols required for communication 3. Learnt about XDR for handling heterogeneity 4. Handled Multicast communication 4. Various programming models 1. Object model & Distributed Object Model 2. RMI, CORBA & XML 3. Role of Events & Notification 4. Case study of RMI 2/6/2017 3@Copyright Material
  4. 4. What we learnt till NOW 5. Operating System, Types & Layers 1. Types of Operating System 2. Various Layers of OS 3. Protection 4. Processes & Threads 5. Communication & Invocation 6. Operating System Architecture 6. Distributed File System 1. Introduction to a File System 2. Architecture of a File Service 3. Implementation of a Distributed File System 4. Peer-to-peer Systems 5. Need for pee-to-peer systems 6. Napster & its legacy 7. Routing Overlays 2/6/2017 4@Copyright Material
  5. 5. What we have learnt in Unit VII 1. Introduction 1. Failure Assumption 2. Failure Detectors 2. Distributed Mutual Exclusion 1. Requirements for Mutual Exclusion 2. Algorithms for Mutual Exclusion 1. Central Server Algorithm 2. Ring Based Algorithm 3. Algorithm using Multicast & Logical Clock 4. Voting Algorithm 3. Fault tolerance of these algorithms 4. Considerations 3. Elections 1. Ring Based Election 2. Bully Algorithm 3. Performance 4. Coordination & Agreement in group Communication 1. Basic Multicast 2. Reliable Multicast 3. Ordered Multicast 2/6/2017 5@Copyright Material
  6. 6. Topic of Contents #1 1. Introduction to Transactions 1. ACID Properties 2. Concurrency Control 3. Recovery from Aborts 2. Nested Transactions 3. Summary of Transactions 4. Distributed Transactions 1. Introduction 2. Concurrency Control in Distributed Transactions A. Locking B. Timestamp ordering C. Optimistic Concurrency Control 3. Distributed Deadlocks 4. Transaction Recovery A. Logging B. Shadow Versions 2/6/2017 6@Copyright Material
  7. 7. Topic of Contents #2 2/6/2017 7@Copyright Material 5. Replication 1. Introduction 2. System Model 3. The role of Group Communication 4. Fault Tolerant Services A. Passive Replication B. Active Replication
  8. 8. Learning Objective Upon completion of this unit, students will be able to: • Learn the need for a Transaction & the properties of it • Study problems associated to concurrency control • Understand distributed transactions and the extension of the concurrency control methods to distributed services • Extend the three methods of concurrency control for use with multiple servers. • Study how distributed deadlock may be detected. • Understand system models & the role of group communication • Understand the passive and active architectures for fault- tolerant services. 2/6/2017 8@Copyright Material
  9. 9. 1. Introduction to Transaction Transaction is specified by a client as a set of operations on objects to be performed as an indivisible unit by the servers managing those objects. The servers must guarantee that either the entire transaction is carried out and the results recorded in permanent storage or, in the case that one or more of them crashes, its effects are completely erased • The goal of transactions – the objects managed by a server must remain in a consistent state when they are accessed by multiple transactions and in the presence of server crashes – Objects must be Recoverable • Objects that can be recovered after their server crashes – Transactions must be Resilient • Transactions must also deal with crash failures of processes and omission failures of communication 2/6/2017 9@Copyright Material
  10. 10. 1. Introduction to Transaction An example to understand more. Its terminology • Each account is represented by a remote object whose interface, Account, provides operations for making deposits and withdrawals and for enquiring about and setting the balance • Each branch of the bank is represented by a remote object whose interface, Branch, provides operations for creating a new account, for looking up an account by name and for enquiring about the total funds at that branch. 2/6/2017 10@Copyright Material Operations of the Account interface deposit(amount) deposit amount in the account withdraw(amount) withdraw amount from the account getBalance() amount return the balance of the account setBalance(amount) set the balance of the account to amount Operations of the Branch interface create(name) account create a new account with a given name lookUp(name) account return a reference to the account with the given name branchTotal() amount return the total of all the balances at the branch
  11. 11. 1. Introduction to Transaction A basic Atomic Operation at the Server • If client operations were synchronous, without transactions: • When a server uses multiple threads it can perform several client operations concurrently • But, if we had allowed deposit and withdraw to run concurrently we could get inconsistent results • Objects should be designed for safe concurrent access e.g. in Java use synchronized methods, e.g. – public synchronized void deposit(int amount) throws RemoteException • Atomic operations are those whose operations are free from interference from concurrent operations in other threads. • Usage of any available mutual exclusion mechanism (e.g. mutex) would provide consistent results • Client cooperation by means of synchronizing server operations can be done using wait() & notify() methods in java 2/6/2017 11@Copyright Material
  12. 12. 1. Introduction to Transaction Failure Model for Transactions • Lampson’s failure model for distributed transactions deals with failures of disks, servers and communication. – Algorithms work correctly in the presence of predictable faults but no claims are made about their behavior when a disaster occurs – Although errors may occur, they can be detected and dealt with before any incorrect behavior results • This model states that: – Writes to permanent storage may fail: • Either by writing nothing or writing wrong values • Reads from permanent storage can detect (by a checksum) when a block of data is bad – Servers may crash occasionally. • When a processor is faulty, it is made to crash so that it is prevented from sending erroneous messages and from writing wrong values to permanent storage • Crashes can occur at any time; in particular, they may occur during recovery – There may be an arbitrary delay before a message arrives. • A message may be lost, duplicated or corrupted. 2/6/2017 12@Copyright Material
  13. 13. 1. Introduction to Transaction Failure Model for Transactions – There may be an arbitrary delay before a message arrives. • A message may be lost, duplicated or corrupted. • The fault model for permanent storage, processors and communications was used to design a stable system whose components can survive any single fault and present a simple failure model • In particular, stable storage provided an atomic write operation in the presence of a single fault of the write operation or a crash failure of the process • This was achieved by replicating each block on two disk blocks • Transactions apply to recover objects & are intended to be atomic 2/6/2017 13@Copyright Material
  14. 14. 1. Introduction to Transaction Failure Model for Transactions • The fault model for permanent storage, processors and communications was used to design a stable system whose components can survive any single fault and present a simple failure model • Stable storage provided an atomic write operation in the presence of a single fault of the write operation or a crash failure of the process achieved by replicating each block on two disk blocks – A write operation was applied to the pair of disk blocks, and in the case of a single fault, one good block was always available • A stable processor used stable storage to enable it to recover its objects after a crash • Communication errors were masked by using a reliable remote procedure calling mechanism 2/6/2017 14@Copyright Material
  15. 15. 1. Introduction to Transaction Transactions: In some situations, clients require a sequence of separate requests to a server to be atomic in the sense that 1. They are free from interference by operations being performed on behalf of other concurrent clients. 2. Either all of the operations must be completed successfully or they must have no effect at all in the presence of server crashes. The aim for any server that supports transactions is to maximize concurrency Therefore transactions are allowed to execute concurrently if this would have the same effect as a serial execution – that is, if they are serially equivalent or serializable. 2/6/2017 15@Copyright Material
  16. 16. 1. Introduction to Transaction ACID Properties • Atomicity: a transaction must be all or nothing; • Consistency: a transaction takes the system from one consistent state to another consistent state; • Isolation: Each transaction must be performed without interference from other transactions • Durability: After a transaction has completed successfully, all its effects are saved in permanent storage • Transaction capabilities can be added to servers of recoverable objects. • Each transaction is created and managed by a coordinator, which implements the Coordinator interface, shown in the next slide 2/6/2017 16@Copyright Material
  17. 17. 1. Introduction to Transaction Operations in Coordinator interface • openTransaction() trans; Starts a new transaction and delivers a unique TID trans. This identifier will be used in the other operations in the transaction. • closeTransaction(trans) (commit, abort); Ends a transaction: a commit return value indicates that the transaction has committed; an abort return value indicates that it has aborted. • abortTransaction(trans); Aborts the transaction. • A transaction is either successful (it commits) – the coordinator sees that all objects are saved in permanent storage • or it is aborted by the client or the server – make all temporary effects invisible to other transactions • How will the client know when the server has aborted its transaction?2/6/2017 17@Copyright Material The client finds out next time it tries to access an object at the server.
  18. 18. 1. Introduction to Transaction 2. CONCURRENCY CONTROL • For the understanding, lets learn 2 situations, which arise with improper Concurrency – LOST UPDATE: It occurs when two transactions both read the old value of a variable and use it to calculate a new value – INCONSISTENT RETRIEVALS: It occurs when a retieval transaction observes values that are involved in an ongoing updating transaction • Serial equivalent executions of transactions can avoid these problems • It is assumed that the operations deposit, withdraw, getBalance and setBalance are synchronized operations - that is, their effect on the account balance is atomic. 2/6/2017 18@Copyright Material
  19. 19. 1. Introduction to Transaction 2. CONCURRENCY CONTROL LOST UPDATE PROBLEM A, B, C have initial balances of $100, $200, $300  As shown in the figure:  Transaction T transfers 10% of account B from account A to account B.  Transaction U transfers 10% of account B from account C to account B. 2/6/2017 19@Copyright Material
  20. 20. 1. Introduction to Transaction 2. CONCURRENCY CONTROL LOST UPDATE PROBLEM If both the transactions have executed like shown below The balances are as follows: A = $80, B = $220, C = $280 . IS THIS WHAT WE WANTED? NOPE, BECAUSE A+B+C != $600   $20 LOST How to deal with this problem?? 2/6/2017 20@Copyright Material Transaction T Transaction U Balance = b.getBalance(); Balance = b.getBalance(); b.setBalance(balance *1.1); b.setBalance(balance *1.1); a.withdrawal(balance/10); c.withdrawal(balance/10); $200 $200 $220 $220 $80 $280
  21. 21. 1. Introduction to Transaction 2. CONCURRENCY CONTROL INCONSISTENT RETRIEVAL PROBLEM A, B have initial balances of $200, $200  As shown in the figure:  Transaction V transfers $100 from account A to account B.  Transaction U calculates the total balance of all accounts in the branch.  The following operations are performed 2/6/2017 21@Copyright Material
  22. 22. 1. Introduction to Transaction 2. CONCURRENCY CONTROL INCONSISTENT RETRIEVAL PROBLEM If both the transactions were allowed to execute like this, the result looks like A = $100, B = $300, BranchTotal = $300. IS THIS CORRECT??   A + B = $400   $100 Missing How to solve this problem?? 2/6/2017 22@Copyright Material Transaction V Transaction W a.withdraw(100); total = a.getBalance( ) total = total + b.getBalance() b.deposit(100) • • $100 $300 $100 $300
  23. 23. 1. Introduction to Transaction 2. CONCURRENCY CONTROL • In the Lost Update Problem, the U’s update is lost because T overwrites it without seeing it. • Since the transactions were concurrent, they were getting executed without any order • It should have been • A serially equivalent interleaving is one in which the combined effect is the same as if the transactions had been done one at a time in some order the same effect means – the read operations return the same values – the instance variables of the objects have the same values at the end 2/6/2017 23@Copyright Material A+B+C = $600   Transaction T Transaction U Balance = b.getBalance(); b.setBalance(balance *1.1); Balance = b.getBalance(); b.setBalance(balance *1.1); a.withdrawal(balance/10); c.withdrawal(balance/10); $200 $220 $242 $80 $278 $220
  24. 24. 1. Introduction to Transaction 2. CONCURRENCY CONTROL • In the Inconsistent Retrieval Problem, W’s retrievals are inconsistent because V has performed only the withdrawal part of a transfer at the time the sum is calculated. • Since the transactions were concurrent, they were getting executed without any order • A serially equivalent interleaving is one in which the combined effect is the same as if the transactions had been done one at a time in some order the same effect means – the read operations return the same values – the instance variables of the objects have the same values at the end 2/6/2017 24@Copyright Material Transaction V Transaction W a.withdraw(100); b.deposit(100) total = a.getBalance( ) total = total + b.getBalance() • • $100 $400 $100 $400
  25. 25. 1. Introduction to Transaction 2. CONCURRENCY CONTROL Conflicting Operations • When we say that a pair of operations conflicts we mean that their combined effect depends on the order in which they are executed • For two transactions to be serially equivalent, it is necessary and sufficient that all pairs of conflicting operations of the two transactions be executed in the same order at all of the objects they both access. 2/6/2017 25@Copyright Material
  26. 26. 2. CONCURRENCY CONTROL Conflicting Operations Are these transactions Serially Equivalent?? NO • the ordering is not serially equivalent, because the pairs of conflicting operations are not done in the same order at both objects • Serially equivalent orderings require one of the following two conditions: 1. T accesses i before U and T accesses j before U. 2. U accesses i before T and U accesses j before T. 1. Introduction to Transaction 2/6/2017 26@Copyright Material
  27. 27. 3. RECOVERY FROM ABORTS • If a transaction aborts, the server must make sure that other concurrent transactions do not see any of its effects • We study two problems: • ‘dirty reads’ – an interaction between a read operation in one transaction and an earlier write operation on the same object (by a transaction that then aborts) – a transaction that committed with a ‘dirty read’ is not recoverable • ‘premature writes’ – interactions between write operations on the same object by different transactions, one of which aborts (getBalance() is a read operation and setBalance(bal) a write operation) 1. Introduction to Transaction 2/6/2017 27@Copyright Material
  28. 28. 28 A dirty read when transaction T aborts U has committed the results, so it cannot be undone Transaction T: a.getBalance() a.setBalance(balance + 10) Transaction U: a.getBalance() a.setBalance(balance + 20) balance = a.getBalance() $100 a.setBalance(balance + 10)$110 balance = a.getBalance() $110 a.setBalance(balance + 20)$130 commit transaction abort transaction U reads A’s balance (which was set by T) and then commits T subsequently aborts. U has performed a dirty read What is the problem? These executions are serially equivalent 1. Introduction to Transaction 3. RECOVERY FROM ABORTS
  29. 29. 29 Premature writes - overwriting uncommitted values Transaction T: a.setBalance(105) Transaction U: a.setBalance(110) $100 a.setBalance(105) $105 a.setBalance(110) $110 some database systems keep ‘before images’ and restore them after aborts. –e.g. $100 is before image of T’s write, $105 is before image of U’s write –if U aborts we get the correct balance of $105, –But if U commits and then T aborts, we get $100 instead of $110 interaction between write operations when a transaction aborts serially equivalent executions of T and U before T and U the balance of A was $100 Solving premature writes: Write Operations Must Be Delayed until earlier transactions that updated the same objects have either committed or aborted 1. Introduction to Transaction 3. RECOVERY FROM ABORTS
  30. 30. 30 1. Introduction to Transaction 3. RECOVERY FROM ABORTS Cascading Aborts: Suppose that U delays committing until after T aborts. – then, U must abort as well. – if any other transactions have seen the effects due to U, they too must be aborted. – the aborting of these latter transactions may cause still further transactions to be aborted. Such situations are called cascading aborts. To avoid cascading aborts – transactions are only allowed to read objects written by committed transactions. – to ensure this, any read operation must be delayed until other transactions that applied a write operation to the same object have committed or aborted. – e.g. U waits to perform getBalance() until T commits or aborts Avoidance of cascading aborts is a stronger condition than recoverability
  31. 31. 31 2. Nested Transactions Transactions may be composed of other transactions – several transactions may be started from within a transaction – we have a top-level transaction and subtransactions which may have their own subtransactions – To a parent, a subtransaction is atomic with respect to failures and concurrent access – transactions at the same level (e.g. T1 and T2) can run concurrently but access to common objects is serialised – a subtransaction can fail independently of its parent and other subtransactions – when it aborts, its parent decides what to do, e.g. start another subtransaction or give up – The CORBA transaction service supports both flat and nested transactions T : top-level transaction T1 = openSubTransaction T2 = openSubTransaction openSubTransaction openSubTransactionopenSubTransaction openSubTransaction T1 : T2: T11 : T12 : T211 : T21 : prov.commit prov. commit abort prov. commitprov. commit prov. commit commit
  32. 32. 32 2. Nested Transactions Advantages of Nested transactions over Flat transactions • Subtransactions may run concurrently with other subtransactions at the same level. – this allows additional concurrency in a transaction. – when subtransactions run in different servers, they can work in parallel. • e.g. consider the branchTotal () operation • it can be implemented by invoking getBalance () at every account in the branch. – these can be done in parallel when the branches have different servers • Subtransactions can commit or abort independently. – this is potentially more robust – a parent can decide on different actions according to whether a subtransaction has aborted or not
  33. 33. 33 2. Nested Transactions Rules for Committing a Nested transactions • A transaction may commit or abort only after its child transactions have completed. • A subtransaction decides independently to commit provisionally or to abort. Its decision to abort is final. • When a parent aborts, all of its subtransactions are aborted. • When a subtransaction aborts, the parent can decide whether to abort or not. • If the top-level transaction commits, then all of the subtransactions that have provisionally committed can commit too, provided that none of their ancestors has aborted.
  34. 34. 34 3. Summary of Transactions We consider only transactions at a single server, they are: 1. atomic in the presence of concurrent transactions 1. which can be achieved by serially equivalent executions 2. atomic in the presence of server crashes 1. they save committed state in permanent storage 2. they use strict executions to allow for aborts 3. they use tentative versions to allow for commit/abort 3. nested transactions are structured from sub- transactions 1. they allow concurrent execution of sub-transactions 2. they allow independent recovery of sub-transactions
  35. 35. 35 4. DistributedTransactions 1. Introduction • Distributed transaction refers to a flat or nested transaction that accesses objects managed by multiple servers • When a distributed transaction comes to an end – Either all of the servers commit the transaction – or all of them abort the transaction. • one of the servers act as a coordinator and it must ensure the same outcome at all of the servers. • the ‘two-phase commit protocol’ is the most commonly used protocol for achieving this
  36. 36. 36 4. DistributedTransactions Client X Y Z X Y M NT 1 T 2 T11 Client P T T 12 T 21 T 22 (a) Flat transaction (b) Nested transactions T T 1. Introduction
  37. 37. 37 4. DistributedTransactions 1. Introduction Nested Distributed Transaction: a.withdraw(10) c.deposit(10) b.withdraw(20) d.deposit(20) Client A B C T 1 T 2 T 3 T 4 T D X Y Z T = openTransaction openSubTransaction a.withdraw(10); closeTransaction openSubTransaction b.withdraw(20); openSubTransaction c.deposit(10); openSubTransaction d.deposit(20);
  38. 38. 38 4. DistributedTransactions 1. Introduction A Flat Distributed Transaction Servers execute requests in a distributed transaction – when it commits they must communicate with one another to coordinate their actions – a client starts a transaction by sending an openTransaction () request to a coordinator in any server(next slide) • it returns a TID unique in the distributed system(e.g. server ID + local transaction number) • at the end, it will be responsible for committing or aborting it – each server managing an object accessed by the transaction is a participant - it joins the transaction (next slide) • a participant keeps track of objects involved in the transaction • at the end it cooperates with the coordinator in carrying out the commit protocol – note that a participant can call abortTransaction() in coordinator * Participants aborts it if it crashes and then restarts or if it has a concurrency control problem, e.g. deadlock or failure of validation in optimistic cc or failure of an operation in timestamps.
  39. 39. 39 4. DistributedTransactions 1. Introduction A Flat Distributed Transaction Scenario: a client’s (flat) banking transaction involves accounts A, B, C and D at servers BranchX, BranchY and BranchZ .. BranchZ BranchX participant participant C D Client BranchY B A participantjoin join join T a.withdraw(4); c.deposit(4); b.withdraw(3); d.deposit(3); openTransaction b.withdraw(T, 3); closeTransaction T =openTransaction a.withdraw(4); c.deposit(4); b.withdraw(3); d.deposit(3); closeTransaction Note: the coordinator is in one of the servers, e.g. BranchX
  40. 40. 40 4. DistributedTransactions 2. Concurrency Control in Distributed Transactions Each server manages a set of objects and is responsible for ensuring that they remain consistent when accessed by concurrent transactions – therefore, each server is responsible for applying concurrency control to its own objects. – the members of a collection of servers of distributed transactions are jointly responsible for ensuring that they are performed in a serially equivalent manner – therefore if transaction T is before transaction U in their conflicting access to objects at one of the servers then they must be in that order at all of the servers whose objects are accessed in a conflicting manner by both T and U
  41. 41. 41 4. DistributedTransactions 2. Concurrency Control in Distributed Transactions Concurrency control can be achieved using one of the 3 ways 1. Locking: Each transaction reserves access to the data it uses & this reservation is called a lock. 2. Optimistic Concurrency Control: The alternative approach for “ Lock” is ‘optimistic’. Transactions are allowed to proceed as though there were no possibility of conflict with other transactions until the client completes its task and issues a closeTransaction request 3. Timestamp Ordering: The timestamp defines its position in the time sequence of transactions. Requests from transactions can be totally ordered according to their timestamps.
  42. 42. 42 4. DistributedTransactions 2. Concurrency Control in Distributed Transactions A. LOCKING • In a distributed transaction, the locks on an object are held by the server that manages it. – The local lock manager decides whether to grant a lock or make the requesting transaction wait. – it cannot release any locks until it knows that the transaction has been committed or aborted at all the servers involved in the transaction. – the objects remain locked and are unavailable for other transactions during the atomic commit protocol • an aborted transaction releases its locks after phase 1 of the protocol.
  43. 43. 43 4. DistributedTransactions 2. Concurrency Control in Distributed Transactions A. LOCKING Consider the following interleaving of transactions T and U at servers X and Y: z 1. The transaction T locks object A at server X, and then transaction U locks object B at server Y. 2. T tries to access B at server Y and waits for U’s lock 3. Similarly, transaction U tries to access A at server X and has to wait for T’s lock • Therefore, we have T before U in one server and U before T in the other • These different orderings can lead to cyclic dependencies between transactions, giving rise to a distributed deadlock (will be discussed later)
  44. 44. 44 4. DistributedTransactions 2. Concurrency Control in Distributed Transactions B. OPTIMISTIC CONCURRENCY CONTROL Consider the following interleavings of transactions T and U, which access objects A and B at servers X and Y • The transactions access the objects in the order T before U at server X and in the order U before T at server Y. • Suppose that T and U start validation at about the same time, but server X validates T first and server Y validates U first, • The validation rules assume that validation is fast, which is true for single- server transactions • In distributed optimistic transactions, each server applies a parallel validation protocol
  45. 45. 45 4. DistributedTransactions 2. Concurrency Control in Distributed Transactions B. OPTIMISTIC CONCURRENCY CONTROL • Optimistic concurrency control requires transactions to operate in a private workspace, so their modifications are not visible to other until they commit • When a transaction is ready to commit, a validation is performed on all the data items to see whether the data conflicts with opera tions of other transactions • If the validation fails, then the transaction will have to be aborte d and restarted later (again, optimistically hoping for no conflicts) • Optimistic control is clearly deadlock free (no locking or waiting on resources) and allows for maximum parallelism (since no proces s has to wait for a lock, all can execute in parallel).
  46. 46. 46 4. DistributedTransactions 2. Concurrency Control in Distributed Transactions C. TIMESTAMP ORDERING • In a single server transaction, the coordinator issues a unique timestamp to each transaction when it starts • Serial equivalence is enforced by committing the versions of objects in the order of the timestamps of transactions that accessed them. • In distributed transactions, we require that each coordinator issue globally unique timestamps • A globally unique transaction timestamp is issued to the client by the first coordinator accessed by a transaction • The transaction timestamp is passed to the coordinator at each server whose objects perform an operation in the transaction
  47. 47. 47 4. DistributedTransactions 2. Concurrency Control in Distributed Transactions C. TIMESTAMP ORDERING • The servers of distributed transactions are jointly responsible for ensuring that they are performed in a serially equivalent manner • A timestamp consists of a <local timestamp, server-id> pair • The agreed ordering of pairs of timestamps is based on a comparison in which the server-id part is less significant • The same ordering of transactions can be achieved at all the servers even if their local clocks are not synchronized • However, for reasons of efficiency it is required that the timestamps issued by one coordinator be roughly synchronized with those issued by the other coordinators
  48. 48. 48 4. Distributed Transactions 3. Distributed Deadlocks • Deadlock is a situation in which two or more competing actions are each waiting for the other to finish, and thus neither ever does • Single server transactions can experience deadlocks – prevent or detect and resolve – use of timeouts is clumsy, detection is preferable. • it uses wait-for graphs. • Distributed transactions lead to distributed deadlocks – in theory can construct global wait-for graph from local ones – a cycle in a global wait-for graph that is not in local ones is a distributed deadlock
  49. 49. 49 4. DistributedTransactions 3. Distributed Deadlocks Scenario: Interleaving of transactions U, V, W • Objects A, B are managed by X and Y • Objects C , D are managed by Z A global wait state is detected. Next slides shows wait-for-graph U V W d.deposit(10) lock D b.deposit(10) lock B at Y a.deposit(20) c.deposit(30) lock C at Z b.withdraw(30) c.withdraw(20) wait at Z a.withdraw(20) wait at X lock A at X wait at Y U  V at Y V  W at Z W  U at X
  50. 50. 50 4. Distributed Transactions 3. Distributed Deadlocks • A deadlock cycle has alternate edges showing wait-for and held-by • wait-for added in order: U V at Y; V W at Z and W  U at X D Held by B Waits for Held by Waits for Held byWaits for X Y Z Held by W UV AC W V U DISTRIBUTED DEADLOCK
  51. 51. 51 4. Distributed Transactions 3. Distributed Deadlocks Deadlock Detection • Local wait-for graphs can be built, e.g. – server Y: U  V added when U requests b.withdraw(30) – server Z: V  W added when V requests c.withdraw(20) – server X: W  U added when W requests a.withdraw(20) • To find a global cycle, communication between the servers is needed • centralized deadlock detection – one server takes on role of global deadlock detector – the other servers send it their local graphs from time to time – it detects deadlocks, makes decisions about which transactions to abort and informs the other servers – usual problems of a centralized service - poor availability, lack of fault tolerance and no ability to scale
  52. 52. 52 4. DistributedTransactions 3. Distributed Deadlocks Phantom Deadlock – a ‘deadlock’ that is detected, but is not really one – happens when there appears to be a cycle, but one of the transactions has released a lock, due to time lags in distributing graphs – in the figure suppose U releases the object at X then waits for V at Y • and the global detector gets Y’s graph before X’s (T  U  V  T) X T U Y V T T U V local wait-for graph local wait-for graph global deadlock detector
  53. 53. 53 4. DistributedTransactions 3. Distributed Deadlocks EDGE CHASING-A distributed approach for Deadlock detection • a global graph is not constructed, but each server knows about some of the edges – servers try to find cycles by sending probes which follow the edges of the graph through the distributed system – When should a server send a probe? ? ? From the diagram in slide 50, – Edges were added in order U  V at Y; V  W at Z and W  U at X • when W  U at X was added, U was waiting, but • when V  W at Z, W was not waiting – Send a probe on an edge T1  T2 when T2 is waiting – Each coordinator records whether its transactions are active or waiting • The local lock manager tells coordinators if transactions start/stop waiting • When a transaction is aborted to break a deadlock, the coordinator tells the participants, locks are removed and edges taken from wait-for graphs
  54. 54. 54 4. DistributedTransactions 3. Distributed Deadlocks EDGE CHASING-A distributed approach for Deadlock detection • 3 Steps – Initiation: • When a server notes that T starts waiting for U, where U is waiting at another server, it initiates detection by sending a probe containing the edge < T  U > to the server where U is blocked. • If U is sharing a lock, probes are sent to all the holders of the lock. – Detection: • Detection consists of receiving probes and deciding whether deadlock has occurred and whether to forward the probes. – e.g. when server receives probe < T  U > it checks if U is waiting, e.g. U  V, if so it forwards < T  U  V > to server where V waits – when a server adds a new edge, it checks whether a cycle is there – Resolution: • When a cycle is detected, a transaction in the cycle is aborted to break the deadlock.
  55. 55. 3. Distributed Deadlocks EDGE CHASING-A distributed approach for Deadlock detection Ex: Edge chasing starts with X sending <W  U>, then Y sends <W  U  V >, then Z sends <W  U  V  W> Resolution • One of the transactions in the cycle must be aborted to break the deadlock • The transaction to be aborted can be chosen according to transaction priorities Ex: Timestamps maybe used as priorities • Even if several different servers detect the same cycle, they will all reach the same decision as to which transaction is to be aborted 55 4. DistributedTransactions V WDEADLOCK DETECTED   U C A B Waits forHeld by Waits for Held by Waits for Z Y XInitiation W  U W U  V Held by IT’S A DEADLOCK W U  V  W
  56. 56. 4. Transaction Recovery • Atomicity property of transactions – durability and failure atomicity – durability requires that objects are saved in permanent storage and will be available indefinitely – failure atomicity requires that effects of transactions are atomic even when the server crashes • Recovery is concerned with – ensuring that a server’s objects are durable and – that the service provides failure atomicity. – for simplicity we assume that when a server is running, all of its objects are in volatile memory – and all of its committed objects are in a recovery file in permanent storage – recovery consists of restoring the server with the latest committed versions of all of its objects from its recovery file 56 4. DistributedTransactions
  57. 57. 4. Transaction Recovery • The task of the Recovery Manager (RM) is: – to save objects in permanent storage (in a recovery file) for committed transactions; – to restore the server’s objects after a crash; – to reorganize the recovery file to improve the performance of recovery; – to reclaim storage space (in the recovery file). • media failures – i.e. disk failures affecting the recovery file – need another copy of the recovery file on an independent disk. e.g. implemented as stable storage or using mirrored disks • we deal with recovery of 2PC separately (at the end) – we study logging but not shadow versions 57 4. DistributedTransactions
  58. 58. 4. Transaction Recovery • Each server records an intentions list for each of its currently active transactions – an intentions list contains a list of the object references and the values of all the objects that are altered by a transaction – when a transaction commits, the intentions list is used to identify the objects affected • the committed version of each object is replaced by the tentative one • the new value is written to the server’s recovery file – in 2 Phase Commit, when a participant says it is ready to commit, its RM must record its intentions list and its objects in the recovery file • it will be able to commit later on even if it crashes • when a client has been told a transaction has committed, the recovery files of all participating servers must show that the transaction is committed, – even if they crash between prepare to commit and commit 58 4. DistributedTransactions
  59. 59. 4. Transaction Recovery * For distributed transactions we need information relating to the 2PhaseCommit as well as object values, that is: – transaction status (committed, prepared or aborted) – intentions list 59 4. DistributedTransactions
  60. 60. 4. Transaction Recovery A. Logging: The recovery file represents a log of the history of all the transactions at a server – it includes objects, intentions lists and transaction status – in the order that transactions prepared, committed and aborted – a recent snapshot + a history of transactions after the snapshot – during normal operation the RM is called whenever a transaction prepares, commits or aborts • prepare - RM appends to recovery file all the objects in the intentions list followed by status (prepared) and the intentions list • commit/abort - RM appends to recovery file the corresponding status • assume append operation is atomic, if server fails only the last write will be incomplete • to make efficient use of disk, buffer writes. Note: sequential writes are more efficient than those to random locations • committed status is forced to the log - in case server crashes 60 4. DistributedTransactions
  61. 61. 4. Transaction Recovery A. Logging: Logging mechanism for example in Slide #23 – initial balances of A, B and C $100, $200, $300 – Entries to left of line represent a snapshot (checkpoint) of values of A, B and C before T started. – T sets A and B to $80 and $220 & points to the previous consistent state. – T has committed – U sets B and C to $242 and $278 & U is prepared & points to previous consistent state – the RM gives each object a unique identifier (A, B, C in diagram) – each status entry contains a pointer to the previous status entry, then the checkpoint can follow transactions backwards through the file 61 4. DistributedTransactions P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7 Object: A Object: B Object: C Object: A Object: B Trans: T Trans: T Object: C Object: B Trans: U 100 200 300 80 220 prepared committed 278 242 prepared <A , P 1> < C , P 5> <B , P 2> < B , P 6> P 0 P 3 P 4 End of log Checkpoint
  62. 62. 4. Transaction Recovery A. Logging: RECOVERY WITH LOGS • When a server is replaced after a crash – it first sets default initial values for its objects – and then hands over to its recovery manager. • The RM restores the server’s objects to include – all the effects of all the committed transactions in the correct order and – none of the effects of incomplete or aborted transactions – it ‘reads the recovery file backwards’ (by following the pointers) • restores values of objects with values from committed transactions • continuing until all of the objects have been restored – if it started at the beginning, there would generally be more work to do – to recover the effects of a transaction use the intentions list to find the value of the objects • e.g. look at previous slide (assuming the server crashed before T committed) – the recovery procedure must be idempotent 62 4. DistributedTransactions
  63. 63. 4. Transaction Recovery A. Logging: REORGANIZING THE RECOVERY FILE • Recovery Manager is responsible for reorganizing its recovery file – so as to make the process of recovery faster and – to reduce its use of space • Checkpointing – the process of writing the following to a new recovery file • the current committed values of a server’s objects, • transaction status entries and intentions lists of transactions that have not yet been fully resolved • including information related to the two-phase commit protocol (see later) – checkpointing makes recovery faster and saves disk space • done after recovery and from time to time • can use old recovery file until new one is ready, add a ‘mark’ to old file • do as above and then copy items after the mark to new recovery file • replace old recovery file by new recovery file 63 4. DistributedTransactions
  64. 64. 4. Transaction Recovery B. Shadow Versions • The shadow versions technique is an alternative way to organize a recovery file • It uses a map to locate versions of the server’s objects in a file called a version store • The map associates the identifiers of the server’s objects with the positions of their current versions in the version store • The versions written by each transaction are ‘shadows’ of the previous committed versions • When a transaction commits, a new map is made by copying the old map and entering the positions of the shadow versions • To complete the commit process, the new map replaces the old map • The same example described for logs can be used to understand Shadow Versions 64 4. DistributedTransactions
  65. 65. 4. Transaction Recovery B. Shadow Versions • Shadow versions on their own are not sufficient for a server that handles distributed transactions • Transaction status entries and intentions lists are saved in a file called the transaction status file • Each intentions list represents the part of the map that will be altered by a transaction when it commits • The transaction status file may, for example, be organized as a log • The figure shows the map and the transaction status file for the same example when T has committed and U is prepared to commit: • A server may crash between the Transaction status file updation & the map updation. • The Recovery Manager checks in the replaced server, whether the map includes the effects of the last committed transaction in the transaction status file • If it does not, then the latter should be marked as aborted. 65 4. DistributedTransactions
  66. 66. 66 1. Introduction It is the process of maintaining copies of data at multiple computers Replication can provide the following • Performance enhancement – Ex: Caching is a form of replication (e.g. web servers and browsers) DNS uses replication extensively – Replication of read-only data is easy, but replication of changing data has overheads • Fault-tolerant service – guarantees correct behaviour in spite of certain faults (can include timeliness) – if f of f+1 servers crash then 1 remains to supply the service – if f of 2f+1 servers have byzantine faults then they can supply a correct service • Availability is hindered by – server failures • replicate data at failure- independent servers and when one fails, client may use another. Note that caches do not help with availability(they are incomplete). – network partitions and disconnected operation • Users of mobile computers deliberately disconnect, and then on re- connection, resolve conflicts 5. Replication
  67. 67. 67 1. Introduction Common requirement when data is replicated is • Replication transparency – Clients see logical objects (not several physical copies) • they access one logical item and receive a single result Consistency – Specified to suit the application, • Ex: When a user of a diary disconnects, their local copy may be inconsistent with the others and will need to be reconciled when they connect again. But connected clients using different copies should get consistent results. 5. Replication
  68. 68. 68 2. System Models & Role of Group Communication • Each logical object is implemented by a collection of physical copies called replicas – the replicas are not necessarily consistent all the time (some may have received updates, not yet conveyed to the others) • Replica managers (RM) – a RM contains replicas on a computer and access them directly – RMs apply operations to replicas recoverably • i.e. they do not leave inconsistent results if they crash – Objects are copied at all RMs unless we state otherwise – Static systems are based on a fixed set of RMs – In a dynamic system: RMs may join or leave (e.g. when they crash) – A RM can be a state machine, which has the following properties:  applies operations atomically  its state is a deterministic function of its initial state and the operations applied  all replicas start identical and carry out the same operations  Its operations must not be affected by clock readings etc. 5. Replication
  69. 69. 69 FE Requests and replies C ReplicaC Service Clients Front ends managers RM RM FE RM 5. Replication 2. System Models & Role of Group Communication Basic architectural model for the management of replicated data • A collection of RMs provides a service to clients • Clients see a service that gives them access to logical objects, which are in fact replicated at the RMs • Clients request operations: those without updates are called read-only requests the others are called update requests (they may include reads) • Clients request are handled by front ends. A front end makes replication transparent • The front ends may be in client's address space or in a separate process
  70. 70. 70 2. System Models & Role of Group Communication 5 STAGES IN PERFORMING A REQUEST • Request – The front end issues the request to one or more replica managers • Sends the request to a single RM that passes it on to the others • or Multicasts the request to all of the RMs (in state machine approach) • Coordination – The RMs decide whether to apply the request; and decide on its ordering relative to other requests (according to FIFO, causal or total ordering) • Execution – The RMs execute the request (sometimes tentatively) • Agreement – RMs agree on the effect of the request Ex: perform 'lazily' or immediately • Response – one or more RMs reply to FE. e.g. • For high availability give first response to client. • To tolerate byzantine faults, take a vote 5. Replication
  71. 71. 71 3. Role of Group communication GROUP COMMUNCIATION • Process groups are useful for managing replicated data – But replication systems need to be able to add/remove RMs • Group membership service provides: – Interface for adding/removing members • Create, destroy process groups, add/remove members. A process can generally belong to several groups. – Implements a failure detector • Which monitors members for failures (crashes/communication), • and excludes them when unreachable – Notifies members of changes in membership – Expands group addresses • Multicasts addressed to group identifiers, • Coordinates delivery when membership is changing • Ex: IP multicast allows members to join/leave and performs address expansion, but not the other features 5. Replication
  72. 72. 72 Join Group address expansion Multicast communication Group send Fail Group membership management Leave Process group 3. Role of Group communication ROLE OF GROUP MEMBERSHIP SERVICES Group membership service has 4 main tasks 1. Providing an Interface for group membership changes 2. Implementing a failure Detector 3. Notifying members of group membership changes 4. Performing group address expansion 5. Replication
  73. 73. 73 3. Role of Group communication SERVICES PROVIDED FOR PROCESS GROUPS • A full membership service maintains group views, which are lists of group members, ordered e.g. as members join group. • A new group view is generated each time a process joins or leaves the group. View delivery: The idea is that processes can 'deliver views' (like delivering multicast messages). – ideally we would like all processes to get the same information in the same order relative to the messages. View synchronous group communication – All processes agree on the ordering of messages and membership changes, – A joining process can safely get state from another member. – If one crashes, another will know which operations it had already performed 5. Replication
  74. 74. 74 4. Fault Tolerant Services Provision of a service that is correct even if f processes fail Ex: Specification of bank account does deposits. withdrawals properly, does not lose money – by replicating data and functionality at RMs – assume communication reliable and no partitions – RMs are assumed to behave according to specification or to crash – intuitively, a service is correct if it responds despite failures and clients can’t tell the difference between replicated data and a single copy – but care is needed to ensure that a set of replicas produce the same result as a single one would. 5. Replication
  75. 75. 75 4. Fault Tolerant Services Linearizability is the strictest criterion for a replication system • A replicated object service is linearizable if for any execution there is some interleaving of clients’ operations such that: – the interleaved sequence of operations meets the specification of a (single) correct copy of the objects – the order of operations in the interleaving is consistent with the real time at which they occurred • The correctness criteria for replicated objects are defined by referring to a virtual interleaving which would be correct • For any set of client operations there is a virtual interleaving (which would be correct for a set of single objects). • Each client sees a view of the objects that is consistent with this, that is, the results of clients operations make sense within the interleaving – the bank example did not make sense: if the second update is observed, the first update should be observed too. 5. Replication
  76. 76. 76 4. Fault Tolerant Services • Sequential Consistency: a replicated shared object service is sequentially consistent if for any execution there is some interleaving of clients’ operations such that: – the interleaved sequence of operations meets the specification of a (single) correct copy of the objects – the order of operations in the interleaving is consistent with the program order in which each client executed them • Every linearizable service is also sequentially consistent, since real-time order reflects each client’s program order • The real-time criterion for linearizability is not satisfied, since getBalanceA(x)  0 occurs later than setBalanceB(x  1); • But the following interleaving satisfies both criteria for sequential consistency: getBalanceA(y)  0 , getBalanceA(x)  0 , setBalanceB(x  1), setBalanceB(y  2) 5. Replication
  77. 77. 4. Fault Tolerant Services A. Passive (primary-backup replication) • There is at any time a single primary RM and one or more secondary (backup, slave) RMs • FEs communicate with the primary which executes the operation and sends copies of the updated data to the result to backups • if the primary fails, one of the backups is promoted to act as the primary 77 FEC FEC RM Primary Backup Backup RM RM 5. Replication
  78. 78. 4. Fault Tolerant Services A. Passive (primary-backup replication) The five phases in performing a client request are as follows: 1. Request: a FE issues the request, containing a unique identifier, to the primary RM 2. Coordination: the primary performs each request atomically, in the order in which it receives it relative to other requests it checks the unique id; if it has already done the request it re-sends the response. 3. Execution: The primary executes the request and stores the response. 4. Agreement: If the request is an update the primary sends the updated state, the response and the unique identifier to all the backups. The backups send an acknowledgement. 5. Response: The primary responds to the FE, which hands the response back to the client. 78 5. Replication
  79. 79. 4. Fault Tolerant Services A. Passive (primary-backup replication) Discussion To survive f process crashes, f+1 RMs are required • it cannot deal with byzantine failures because the client can't get replies from the backup RMs To design passive replication that is linearizable • View synchronous communication has relatively large overheads • Several rounds of messages per multicast • After failure of primary, there is latency due to delivery of group view Variant in which clients can read from backups • which reduces the work for the primary • get sequential consistency but not linearizability Sun NIS uses passive replication with weaker guarantees • Weaker than sequential consistency, but adequate to the type of data stored • achieves high availability and good performance • Master receives updates and propagates them to slaves using 1-1 communication. Clients can uses either master or slave • updates are not done via RMs - they are made on the files at the master 79 5. Replication
  80. 80. 4. Fault Tolerant Services B. Active Replication • The RMs are state machines all playing the same role and organised as a group. • all start in the same state and perform the same operations in the same order so that their state remains identical • If an RM crashes it has no effect on performance of the service because the others continue as normal • It can tolerate byzantine failures because the FE can collect and compare the replies it receives 5 Phases in performing Client Request in Active Replication 1. Request 2. Coordination 3. Execution 4. Agreement 5. Response 80 5. Replication 80
  81. 81. 4. Fault Tolerant Services B. Active Replication 5 Phases in performing Client Request Request • FE attaches a unique id and uses totally ordered reliable multicast to send request to RMs. FE can at worst, crash. It does not issue requests in parallel Coordination • the multicast delivers requests to all the RMs in the same (total) order. Execution • every RM executes the request. They are state machines and receive requests in the same order, so the effects are identical. The id is put in the response Agreement • no agreement is required because all RMs execute the same operations in the same order, due to the properties of the totally ordered multicast. Response • FEs collect responses from RMs. FE may just use one or more responses. If it is only trying to tolerate crash failures, it gives the client the first response. 81 5. Replication 81
  82. 82. 4. Fault Tolerant Services B. Active Replication 1. A FE multicasts each request to the group of RMs 2. Requires totally ordered reliable multicast so that all RMs perform the same operations in the same order 3. The RMs process each request identically and reply 82 FE CFEC RM RM RM 5. Replication
  83. 83. 83 4. Fault Tolerant Services B. Active Replication Discussion • As RMs are state machines we have sequential consistency – due to reliable totally ordered multicast, the RMs collectively do the same as a single copy would do – it works in a synchronous system – in an asynchronous system reliable totally ordered multicast is impossible – but failure detectors can be used to work around this problem. How to do that is beyond the scope of this course. • this replication scheme is not linearizable – because total order is not necessarily the same as real-time order • To deal with byzantine failures – For up to f byzantine failures, use 2f+1 RMs – FE collects f+1 identical responses • To improve performance, – FEs send read-only requests to just one RM 5. Replication
  84. 84. 84 • Learned the need for a Transaction & its properties • Understood the problems associated with concurrency control Understand distributed transactions and the extension of the concurrency control methods to distributed services • Learnt the three methods of concurrency control for use in a Distributed Environment. • Applied the learning of detecting a distributed deadlock • Illustrated system models for group communication and its role in Distributed transactions • Learnt the passive and active architectures for fault-tolerant services. Summary
  85. 85. 85 1. Which of the following is not a property of transaction a. Atomicity b. Durability c. Isolation d. Concurrency 2. _____ occurs when two transactions both read the old value of a variable and use it to calculate a new value a. Lost Update b. Concurrency control c. Deadlock d. Inconsistent Retrieval 3. Out of the listed operations, which of them doesn’t lead to a conflicting operation a. Read-Write b. Write-Read c. Read-Read d. Write-Write Quiz
  86. 86. 86 4. A, B have initial balances of $200, $200. Which of the following problem does the below figure indicate a. Lost update b. Concurrency Control c. Deadlock d. Inconsistent Retrieval Quiz
  87. 87. 87 5. A transaction cannot be recovered if it has committed with a. Premature Write b. Dirty Read c. Premature Read d. Dirty Write 6. Which one of the below is in commiting a Nested transaction a. A transaction may commit or abort only after its child transactions have completed b. A subtransaction decides independently to commit provisionally or to abort c. When a subtransaction aborts, the parent must abort d. When a parent aborts, all of its subtransaction’s are aborted 7. Concurrency control can be achieved using ____ a. Locking b. Timestamp Ordering c. Optimistic Concurrency Control d. All Quiz
  88. 88. 88 8._______concurrency control requires transactions to operate in a private workspace, so their modifications are not visible to other until they commit a. Locking b. Timestamp Ordering c. Optimistic Concurrency Control d. All 9. Distributed deadlocks are a situation to handle with one of the concurrency control techniques a. Locking b. Timestamp Ordering c. Optimistic Concurrency Control d. All 10. Which of the following is a ‘deadlock’ that is detected, but is not really one a. Phantom b. Edge-chasing c. Circular d. Probe 11. ______ is not a part of Edge-chasing approach a. Initiation b. Analysis c. Detection d. Resolution Quiz
  89. 89. 89 12. Transaction recovery is concerned with a. Ensuring that a server’s objects are durable b. Failure atomicity c. Objects are in volatile memory d. All of the above 13._____ represents the history of all the transactions at a server a. Replication b. Recovery c. Logging d. Shadow Version 14.The process of writing the current committed values is called as a. Shadow Version b. Checkpointing c. Logging d. Intention List Quiz
  90. 90. 90 15. ______ uses a map to locate versions of the server’s objects in a file called a version store a. Checkpointing b. Shadow Version c. Logging d. Recovery Manager 16. Which one of the following is not a feature of Replication a. Performance enhancement b. Fault-tolerance c. Availability d. None 17. __________makes a set of replicas appear as a single logical object to a client a. Strict Transparency b. Replication Transparency c. Linearizability d. None 18.____ Replication has a single primary RM and one or more secondary (backup, slave) RMs, at any time a. Passive b. Active c. Both d. None Quiz
  91. 91. 91 KEY 1. D 2. A 3. D 4. C 5. B 6. D 7. D 8. C 9. A 10. A 11. B 12. D 13. C 14. B 15. B 16. D 17. B 18. A Quiz
  92. 92. 92 • Transaction: is specified by a client as a set of operations on objects to be performed as an indivisible unit by the servers managing those objects. • Recoverable Objects: Objects that can be recovered after their server crashes • Serially Equivalent: Transactions are allowed to execute concurrently if this would have the same effect as a serial execution • Atomicity: a property of a transaction where the total transaction must be executed or nothing at all • Consistency: A transaction takes the system from one consistent state to another consistent state, without the loss of data • Isolation: Each transaction must be performed without interference from other transactions • Durability: Effects of a transaction are saved in permanent storage, after every partial/successful completion of the transaction • Lost Update: It occurs when two transactions both read the old value of a variable and use it to calculate a new value GLOSSARY
  93. 93. 93 • Inconsistent Retrievals: It occurs when a retieval transaction observes values that are involved in an ongoing updating transaction • Conflicting Operations: If the effect of two transactions depends on the order in which they are executed, then they are conflicting • Dirty reads: When an interaction between a read operation in one transaction and an earlier write operation on the same object by a transaction, which later aborts • Premature Writes: When an interaction between write operations on the same object are done by different transactions and later one of it aborts • Cascading Abort: A situation which arises due to a transaction delaying in committing the values, leading to aborts in many other transactions later. • Distributed transaction: Refers to a flat or nested transaction that accesses objects managed by multiple servers GLOSSARY
  94. 94. 94 • Flat Transaction: A flat client transaction completes each of its requests before going on to the next one. Therefore, each transaction accesses servers’ objects sequentially • Nested Transaction: In a nested transaction, the top-level transaction can open subtransactions, and each subtransaction can open further subtransactions down to any depth of nesting • Concurrency Control: A Set of actions needed for each server who manages a set of objects responsible for ensuring that they remain consistent when accessed by concurrent transactions • Locking: Each transaction reserves access to the data it uses & this reservation is called a lock. • Optimistic Concurrency Control: The alternative approach for “ Lock” is ‘optimistic’. Transactions are allowed to proceed as though there were no possibility of conflict with other transactions until the client completes its task and issues a closeTransaction request • Timestamp Ordering: The timestamp defines its position in the time sequence of transactions. Requests from transactions can be totally ordered according to their timestamps. • Deadlock is a situation in which two or more competing actions are each waiting for the other to finish, and thus neither ever does GLOSSARY
  95. 95. 95 • Wait-for graph: is a directed graph in which nodes represent transactions and objects, and edges represent either an object held by a transaction or a transaction waiting for an object. There is a deadlock if and only if there is a cycle in the wait-for graph. • Distributed Deadlock: A cycle in a global wait-for graph that is not in local ones • Phantom Deadlock: It is a deadlock that is detected, but is not really one • Intentions List contains a list of the object references and the values of all the objects that are altered by a transaction • Log: The recovery file represents the history of all the transactions at a server which includes objects, intentions lists and transaction status • Recovery Manager: The mechanism of maintaining the requirements for durability and failure atomicity in a distributed transaction • Checkpointing : is the process of writing the current committed values of a server’s objects, transaction status entries and intentions lists of transactions that have not yet been fully resolved to a new recovery file • Shadow Version: It uses a map to locate versions of the server’s objects in a file called a version store • Replication: It is the process of maintaining copies of data at multiple computers GLOSSARY
  96. 96. 96 • Replication Transparency makes a set of replicas appear as a single logical object to a client. • Replica: Each logical object is implemented by a collection of physical copies • Replica Manager: Maintains replicas on a computer and access them directly • FIFO ordering: If a Front End issues r then r’, then any correct Replica Manager handles r before r’ • Causal ordering: If r  r’, then any correct Replica Manager handles r before r’ • Total ordering: If a correct Replica Manager handles r before r’, then any correct Replica Manager handles r before r’ • Replica Managers agree – Process of reaching a consensus as to effect of the request • Fault Tolerance: Provision of a service that is correct even if f processes fail • Remote Replication: Front Ends communicate with the primary which executes the operation and sends copies of the updated data to the result to backups • Active Replication: Achieved when a Front End multicasts each request to the group of Replica Managers GLOSSARY

×