2. DISTRIBUTED FILE SYSTEMS (DFS)
A Distributed File System (DFS) is a file system that is distributed on multiple file
servers or multiple locations.
It allows program to access or store isolated files as they do with the local ones,
allowing programmers to access files from any network or computer.
The main purpose of the DBS is to allow users of physically distributed systems to
share their data and resources by using a Common File System.
DFS has two components:
1. Location Transparency - which is achieved through the namespace component.
2. Redundancy – done through a file replication component.
3. Cont….
In the case of failure and heavy load, these components together improve data
availability by allowing the sharing of data in different locations to be logically
grouped under one folder, which is known as DFS root.
It is not necessary to use both the 2 components of DFS together, it is possible to
use the namespace component without using the file replication component and it
is perfectly possible to use the file replication component without using the
namespace component between servers.
FEATURES OF DFS:
TRANSPARENCY
• Structure transparency – There is no need for the client to know about the number
or locations of file servers and the storage devices. Multiple file servers should be
provided for performance, adaptability and dependability.
4. Cont….
• Access transparency – Both local and remote files should be accessible in the same
manner. The file system should be automatically located on the accessed file and
send it to the client’s side.
• Naming transparency – There should not be any hint in the name of the file to the
location of the file. Once a name is given to the file, it should not be changed
during transferring from one node to another.
• Replication transparency – If a file is copied on multiple nodes, both the copies of
the file and their locatioons should be hidden from one node to another.
USER MOBILITY
It will automatically bring the user’s home directory to the node where the user logs
in.
5. Cont….
PERFORMANCE
Performance is based on the average amount of time needed to convince the client
requests. This time covers the CPU time, time taken to access secondary storage and
network access time
SIMPLICITY AND EASE OF USE:
The user interface of a file system shoul be simple and the number of commands in
the file should be small.
HIGH AVAILABILITY
A DFS should be able to continues in case of any partial failires like a link failure, a
node failure, or a storage drive crash. A high authentic and adaptable distributed file
system should have different and independent file servers for controlling different and
independent storage devices.
SCALABILITY
A good DFS should be built to scale quickly as the number of nodes and users in the
system grows.Service should not be substantially disrupted as the number of nodes
and users grows.
6. Cont....
HIGH RELIABILITY
A file system should create backup copies of key files that can be used if the originals
are lost
DATA INTEGRITY
The integrity of data saved in a shared file must be guaranteed by the file system. That
is, concurrent access requests from many users who are competing for access to the
same file muste be correctly synchronised using a concurrency control method.
SECURITY
A DFS should be secure so tha its users may trust that their data will be kept private.
7. Applications:
Network File System (NFS) – A client-server architecture that allows a computer
user to view, store and update remotely.
Common Internet File System (CIFS) – An application of SMB protocol designed by
Microsoft.
Server Message Block (SMB) – Protocol for sharing a file. It was created to allow
computers to perform read and write operations on files to a remote host over a
LAN. The directories present in the remote host can be accessed via SMB and are
called as “shares”.
Hadoop – A group of open-source software services. It gives a software framework
for distributed storage and operating of big data using the MapReduce
programming model.
Netware – An abandon computer network operating system developed by Novell,
inc. It primarily uses combined multitasking to run different services on a personal
computer, using the IPX network protocol.
8. Working of DFS:
There are 2 ways in which DFS can be implemented:
STANDALONE DFS NAMESPACE – It can only be acquired on those computers on
which it is created. It also does not provide any fault liberation and cannot be
linked to any other DFS.
DOMAIN - BASED DFS NAMESPACE – It stores the configuration of DFS in Active
Directory, creating the DFS namespace root accessible at
<domainname><dfsroot> or <FQDN><dfsroot>
9. Advantages of DFS
DFS allows multiple user to access or store the data.
Allows the data to be shared remotely.
It improves the availability of file, access time, and network efficiency.
Improves the capacity to change the size of the data and also improves the ability
to exchange the data.
Provides transparency of data even if server or disk fails.
Disadvantages of DFS
Nodes and connections needs to be secured therefore we can say that security is at
stake.
There is a possibility of lose of messages and data in the network while movement
from one node to another.
Database connection in case of DFS as compared to a single user system.
There are chances that overloading will take place if all nodes tries to send data at
once.
11. EVENT ORDERING
In a centralized system, we can always determine the order in which two events
occurred, since the system has a single common memory and clock.
A distributed system has no common memory and no common clock. Therefore, it
is sometimes impossible to say which of the two events occurred first.
The happened-before relation, discussed next, is only a partial ordering of the
events in distributed systems. Since the ability to define a total ordering is crucial in
many applications, we present a distributed algorithm for extending the happened-
before relation to a consistent total ordering of all the events in the system.
THE HAPPENED BEFORE RELATION
We can define the happened-before relation (denoted by ) on a set of events as
follows(assuming that sending and receiving a message constitutes an event):
1. If A and B are events in the same process, and A was executed before B, then A B.
12. Cont…
2. If A is the event of sending a message by one process and B is the event of receiving that message by
another process, then A B.
3. If A B and B C, then A C.
Since an event cannot happen before itself, the relation is an irreflexive partial ordering.
If two events, A and B are not related by the relation (that is, A did not happen before B, and B did
not happen before A), then we say that these two events were executed concurrently.
these are some of the events related by the happened-before relation
p1 → q2
r0 → q4
q3 → r4
p1 → q4 (since p1 → q2 and q2 → q4)
These are some of the concurrent events in the system:
q0 and p2
r0 and q3
r0 and p3
q3 and p3
13. MUTUAL EXCLUSION
Consists of 3 approaches which are centralized, fully distributed and token-passing
approach.
1.CENTRALISED APPROACH
In the centralised approach to providing mutual exclusion, one of the processes in the
system is chosen to coordinate the entry to the critical section.
Each process that wants to invoke mutual exclusion sends a request message to the
coordinator.
When the process receives a reply message from the coordinator, it can enter its critical
section. After exiting its critical section, the process sends a release message to the
coordinator and proceeds with its execution.
On receiving a request message, the coordinator checks to see whether some other process
is in its critical section. If no process is in its critical section, the coordinator immediately
sends back a reply message, otherwise the request is queued.
When the coordinator receives a release message, it removes one of the request messages
from the queue and sends a reply message to the requesting process. It should be clear
that this algorithm ensures mutual exclusion.
14. Cont…
2. FULLY DISTRIBUTED APPROACH
When a process Pi wants to enter its critical section, it generates a new timestamp, TS and
sends the message request (Pi ,TS) to all processes in the system.
On receiving a request message, a process may reply immediately (that is, send a reply
message back to Pi ), or it may defer sending a reply back( because it is already in its critical
section, for example).
A process that has received a reply message from all other processes in the system can
enter its critical section, queueing incoming requests and deferring them. After exiting its
critical section, the process sends reply messages to all its deferred requests.
The decision whether process Pi replies immediately to a request (Pi ,TS) message or defers
its reply based on three factors:
1. If process Pi is in its critical section, then it defers its reply to Pj.
2. If process Pi does not want to enter its critical section, then it sends a reply
immediately to Pj .
3. If process Pi wants to enter its critical section but has not yet entered it, then it
compares its own request timestamp with the timestamp of the incoming request made
by process Pj .
15. Cont….
This algorithm exhibits the following desirable behaviour:
• Mutual exclusion is obtained.
• Freedom from deadlock is ensured.
• Freedom from starvation is ensured, since entry to the critical section is scheduled
according to the timestamp ordering. The timestamp ordering ensures that processes are
served in FCFS order.
• The number of messages per critical-section entry is 2 ×(n-1). This number represents the
minimum number of required messages per critical-section entry when processes act
independently and concurrently.
TOKEN-PASSING APPROACH
Another method of providing mutual exclusion is to circulate a token among the
processes in the system. A token is a special type of message that is passed from
process to process. Possession of the token entitles the holder to enter the critical
section. Since there is only a single token, only one process can be in its critical section
at a time.
We assume that the processes in the system are logically organized in a ring structure.
The physical communication network need not be a ring. As long as the processes are
connected to one another, it is possible to implement a logical ring. To implement
mutual exclusion, we pass the token around the ring. When a process receives the
token, it may enter its critical section, keeping the token.
16. Cont….
After the process exits its critical section, the token is passed around again. If the
process receiving the token does not want to enter its critical section, it passes the
token to its neighbour.
ATOMICITY
It is the concept of an atomic transaction, which is a program unit that must be
executed automatically. That is, either all the operations associated with it are
executed to completion, or none are performed.
THE TWO-PHASE COMMIT PROTOCOL
All the sites in which a transaction T has executed must agree on the final outcome of
the execution. T must either commit at all sites, or it must abort at all sites.
17. Cont…
Assume that T is a transaction initiated at site Si and that the transaction coordinator
at Si is Ci. When T completes its execution—that is, when all the sites at which T has
executed inform Ci that T has completed—then Ci starts the 2PC protocol.
• Phase 1. Ci adds the record <prepare T> to the log and forces the record onto stable
storage. It then sends a prepare (T) message to all the sites at which T has executed.
On receiving the message, the transaction manager at each of these sites determines
whether it is willing to commit its portion of T. If the answer is no, it adds a
record <no T> to the log, and then it responds by sending an abort (T) message to Ci .
If the answer is yes, it adds a record <ready T> to the log and forces all the log records
corresponding to T onto stable storage. The transaction manager then replies with
a ready (T) message to Ci .
18. Cont….
• Phase 2. When Ci has received responses to the prepare (T) message from all the sites, or when a
pre-specified interval of time has elapsed since the prepare (T) message was sent out, Ci can
determine whether the transaction T can be committed or aborted. Transaction T can be committed
if Ci has received a ready (T) message from all the participating sites. Otherwise, transaction T must be
aborted. Depending on the verdict, either a record <commit T> or a record <abort T> is added to the
log and forced onto stable storage. At this point, the fate of the transaction has been sealed.
Following this, the coordinator sends either a commit (T) or an abort (T) message to all participating
sites. When a site receives that message, it records the message in the log.
A site at which T has executed can unconditionally abort T at any time prior to its sending the
message ready (T) to the coordinator. The ready (T) message is, in effect, a promise by a site to follow
the coordinator’s order to commit T or to abort T. A site can make such a promise only when the
needed information is stored in stable storage. Otherwise, if the site crashes after sending ready (T), it
may be unable to make good on its promise.
Since unanimity is required to commit a transaction, the fate of T is sealed as soon as at least one site
responds with abort (T). Note that the coordinator site Si can decide unilaterally to abort T, as it is one
of the sites at which T has executed. The final verdict regarding T is determined at the time the
coordinator writes that verdict (commit or abort) to the log and forces it to stable storage.
In some implementations of the 2PC protocol, a site sends an acknowledge (T) message to the
coordinator at the end of the second phase of the protocol. When the coordinator has received
the acknowledge (T) message from all the sites, it adds the record <complete T> to the log.
19. CONCURRENCY CONTROL
Modify the centralized concurrency schemes to accommodate the distribution of
transactions
Transaction manager coordinates execution of transactions that access data at local
sites
Local transaction only executes at that site
Global transaction executes at several sites
LOCKING PROTOCOLS
Can use the two-phase locking protocol in a distributed environment by changing hoe
the lock manager is implemented.
Nonreplicated scheme-each site maintains a local lock manager which administers
lock and unlock requests for those data items that are stored in that site.
-Simple implementation involves two message transfers for handling lock requests,
and one message transfer for handling unlock requests.
- Deadlock handling is more complex.
20. Cont….
SINGLE-COORDINATOR APPROACH
-A single lock manager resides in a single chosen site, all lock and unlock requests are
made at that site.
-Simple impkementation
-Simple deadlock handling
-Possibility of bottleneck
Vulnerable to loss of concurrency controller if single site fails
MAJORITY PROTOCOL
-Avoids drawbacks of central control by dealing with replicated data in a decentralized
manner
More complicated to implement
Deadlock-handling algorithms must be modified; possible for deadlock to occur in locking
only one data item.
21. Cont….
BIASED PROTOCOL
-Similar to majority protocol, but requests for shared locks prioritized over requests for exclusive
locks
-Less overhead on read operations than in majority protocol; but has additional overhead on writes
-Like majority protocol, deadlock handling is complex
PRIMARY COPY
-One of the sites at which a replica resides is designated as the primary site.Request to lock a data
item is made at the primary site of that data item
-Concurrency control for replicated data handled in a manner simila to that of unreplicated data
-Simple implementation, but if primary site fails, the data item is unavailable, even though other
sites may have a replica.
22. Cont….
TIMESTAMPING
-Generate unique timestamps in distributed scheme:
Each site generates a unique local timestamp
The global unique timestamp is obtained by concatenation of the unique local
timestamp with the unique site identifier
Use a logical clock defined within each site to ensure the fair generation of
timestamps
-Timestamp-Ordering scheme- combine the centralized concurrency control
timestamp scheme with the 2PC protocol to obtain a protocol that ensures
serializability with no cascading rollbacks.
23. DEADLOCK HANDLING
The deadlock-prevention, deadlock-avoidance, and deadlock-detection algorithms
can be extended so that they can be used in a distributed system.
1. The wait-die scheme. This approach is based on a nonpreemptive technique. When
process Pi requests a resource currently held by Pj , Pi is allowed to wait only if it has a
smaller timestamp than does Pj (that is, Pi is older than Pj ). Otherwise, Pi is rolled back
(dies). For example, suppose that processes P1, P2, and P3 have timestamps 5, 10, and
15, respectively. If P1 requests a resource held by P2, P1 will wait. If P3 requests a
resource held by P2, P3 will be rolled back.
2. The wound-wait scheme. This approach is based on a preemptive technique and is
a counterpart to the wait-die approach. When process Pi requests a resource currently
held by Pj , Pi is allowed to wait only if it has a larger timestamp than does Pj (that
is, Pi is younger than Pj ). Otherwise, Pj is rolled back (Pj is wounded by Pi). Returning
to our previous example, with processes P1, P2, and P3, if P1 requests a resource held
by P2, then the resource will be preempted from P2, and P2 will be rolled back.
If P3 requests a resource held by P2, then P3 will wait.
24. Cont….
CETRALIZED APPROACH
Each site keeps a local wait-for graph
A global wait-for graph is maintained in a single coordination process
There are three different options (points in time) when the wait-for graph may be
constructed:
1. Whenever a new edge is inserted or removed in one of the local wait-for graphs
2. Periodically, when a number of changes have occurred in a wait-for graph
3. Whenever the coordinator needs to invoke the cycle detection algorithm
Unnecessary rollbacks may occur as a result of false cycles.
25. ELECTION ALGORITHM
Determine where a new copy of the coordinator should be restarted
Assume that a unique priority number is associated with each active process in the system, and
assume that the priority number of process Pi is i.
Assume the one-to-one correspondence between processes and sites
The coordinator is always the process with the largest priority number.When a coordinator fails, the
algorithm must elect that active process with the largest priority number.
Two algorithms, the bully algorithm and a ring algorithm, can be used to elect a new coordinator in
case of failures.
BULLY ALGORITHM
- Applicable to systems where every process can send a message to every other process in the
system
- If the process Pi sends a request that is not answered by the coordinator within a time interval T,
assume that the coordinator has failed; Pi tries to elect itself as the new coordinator
- Pi sends an election message to every process with a higher priority number, Pi then waits for any
of these processes to answer within T
- If no response within T, assume that all processes with numbers greater than I have failed; Pi elects
itself the new coordinator.
26. Cont….
- If answer is received, Pi begins time interval T, waiting to receive a message that a process with a
higher priority number has been elected
- If no message is sent within T, assume the process with a higher number has failed; Pi should restart
the algorithm
- After a failed process recovers, it immediately begins execution of the same algorithm
- If there are no active processes with higher numbers, the recovered process forces all processes with
lower number to let it become the coordinator process, even if there is a currently active coordinator
with a lower number.
RING ALGORITHM
- Applicable to systems organized as a ring (logically or physically)
- Assumes that the links are unidirectional, and that processes send their messages to their right
neibhbors
- Each process maintains an active list, consisting of all priority numbers of all active processes in the
system when the algorithm ends.
- If process detects a coordinator failure, I creates a new active list that is initially empty and sends a
message elect(i) to its right neighbour, and adds the number I to its active list
27. Cont….
- If Pi receives a message elect(j) from the process on the left, it must respond in one of
three ways:a. If this is the first elect message it has seen or sent, Pi creates a new active list
with the numbers i and j. It then sends the message elect(i), followed by the
message elect(j).
b. If i ≠ j—that is, if the message received does not contain Pi ’s number —then Pi adds j to its
active list and forwards the message to its neighbor on the right.
c. If i = j—that is, if Pi receives the message elect(i)—then the active list for Pi now contains the
numbers of all the active processes in the system. Process Pi can now determine the largest
number in the active list to identify the new coordinator process.
REACHING AGREEMENT
- For a system to be reliable, we need a mechanism that allows a set of processes to agree
on a common value.
- Such agreement may not take place due to:
- Faulty communication medium
- Faulty processes
