The document discusses the complexities of programming applications in distributed memory systems using the message-passing paradigm and introduces distributed shared memory (DSM) as a solution that facilitates a shared-memory programming approach. It covers various aspects of DSM such as memory coherence, data structuring, consistency models, and issues like thrashing and replacement strategies. Additionally, it touches upon challenges in heterogeneous environments, synchronization mechanisms, mutual exclusion, deadlocks, and election algorithms within distributed systems.

2.  Programming of of applications for loosely coupled
distributed memory machines with the message-
passing paradigm is a difficult and error-prone task.
 The Distributed Shared Memory (DSM) facility
simplifies this programming task by providing a
higher level abstraction that allows programmers to
write programs with the shared-memory
paradigm,which is consistent with the way sequential
applications access data.

4.  The DSM system normally have an architecture of
the form.
 Each node of the system consists of one or more
CPUs and a memory unit.
 The nodes are connected by a high-speed
communication network.
 A simple message-passing system allows processes
on different nodes to exchange messages with each
other

5.  The DSM abstraction presents a large shared-
memory space to the processors of all nodes.
 In contrast to the shared physical memory in
tightly coupled parallel architecture, the shared
memory of DSM exists only virtually.
 A software memory- mapping manager routine in
each node maps the local memory onto the
shared virtual memory.
 To facilitate the mapping operation, the shared-
memory space is partitioned into blocks.

6.  Granularity
 Structure of shared-memory space
 Memory coherence and access
synchronization
 Data location and access
 Replacement strategy
 Thrashing
 Heterogenity

7.  One of the most visible parameters to be chosen in
the design of a DSM system is the block size.
 Just as with paged main memory, there are a number
of trade-offs and no single criterion dominates.

8.  Paging Overhead- Because shared memory
programs provide locality of reference, a process is
likely to access a large region of its shared address
space in a small amount of time.
 Directory Size- Another factor affecting the choice of
block size is the need keep directory information
about the blocks in the system.

9.  Thrashing- The problem of thrashing may occur
when data items in the same data block are being
updated by multiple nodes at the same time, causing
large numbers of data block transfers among the
nodes without much progress in the execution of the
application.
 False Sharing- False sharing occurs when two
different processes access two unrelated variables
that reside in the same data block.

10. 1) No sharing- Most DSM systems do not structure
their shared-memory space. In these systems, the
shared-memory space is simply a linear array of
words. It is convenient to choose any suitable page
size as the unit of sharing and a fixed grain size may
be used for all applications.
2) Structuring by data type- In this method, the shared-
memory space is either as a collection of objects
and Orca or as a collection of variables in the source
language.

11. 3) Structuring as a database- Another method is to
structure the shared memory like a database. A set of
primitives that can be added to any base language are
provided to place tuples in the tuple space and to
read or extract them from tuple space.

12.  Strict Consistency model
 Sequential consistency model
 Causal Consistency model
 Pipelined Random-Access Memory Consistency
Model
 Processor Consistent Model
 Weak Consistency Model
 Release Consistency Model

13.  In DSM systems that allow shared-memory blocks to
be dynamically migrated/replicated, the following
issues must be addressed when the available space
for caching shared data fill up at a node:
1) Which block should be replaced to make
space for a newly required block?
2) Which should the replaced block be placed?

14.  The problem of replacement has been studied
extensively for paged main memories and shared-
memory multiprocessor systems.
 The usual classification of replacement algorithm
group them into the following categories:

15. 1) Usage based versus non-usage based- Usage based
algorithms keep track of the history of usage of a
cache line and use this information to make
replacement decisions.
2) Fixed space versus variable space- Fixed-space
algorithms assume that the cache size fixed while
variable-space algorithms are based on the
assumption that the cache size can be changed
dynamically depending on the need.

16.  Unused: A free memory block that is not currently
being used.
 Nil: A block that has been invalidated.
 Read-only: A block for which the node has only read
right.
 Read-owned: A block for which the node has only
read access right but is also the owner of the block.
 Writable: A block for which the node has write
access permission. Obviously, the node is the owner
of the block because IVY uses the write-invalidate
protocol.

17.  Once a memory block has been selected for
replacement, it should not be lost.
1)Using Secondary storage
2) Using the memory space of other nodes.

18.  Using Secondary store: In this method, the block is
simply transferred on to a local disk. The advantage
of this method is that it does not waste any memory
space, and if the node want to access the same block
again.
 Using the memory space of thee other nodes:
Sometimes it may be faster to transfer a block over
the network than to transfer it to a block disk.

19.  Thrashing is said to occur when the system spends a
large amount of time transferring shared data blocks
from one node to another, compared to the time
spent doing the useful work of executing application
process.

20. 1) When interleaved data accesses made by process on
two or more nodes causes a data block to move
back and forth from one node to another in quick
succession.
2) When blocks with read-only permissions are
repeatedly invalidated soon after they are replicated.

21.  The following methods may be used to solve the thrashing
problem in DSM systems:
1) Providing Application-controlled locks.
2) Nailing a block to a node for a minimum amount of
time.
3) Tailoring the coherence algorithm to the shared-data
usage patterns.

22.  Heterogeneous DSM is a mechanism that provides
the shared-memory paradigm in a heterogeneous
environment and allows memory sharing among
machines of different architecture.

23.  The main issues in building a DSM system on a
network of heterogeneous machines are :
1) Data conversion
2) Block size selection

24.  Simpler Abstraction
 Better Portability of Distributed Applications
Programs
 Better Performance of some Applications
 Flexible Communication Environment
 Ease of Process Migration

25.  A distributed system consists of a collection of
distinct processes that are spatially separated and run
concurrently.
 The rules for enforcing correct interaction are
implemented in the form of synchronization
mechanisms.

26.  Clock synchronization
 Event ordering
 Mutual exclusion
 Deadlock
 Election algorithms

27.  In a distributed system, an application may have
process that concurrently run on multiple nodes of
the system.
 For correct results, several such distributed
applications require that the clocks of the nodes are
synchronized with each other.

28.  In a distributed system, synchronized clocks also
enable one to measure the duration of distributed
activities that start on one node and terminate on
another node, for instance, calculating the time taken
to transmit a message from one node to another at
any arbitrary time.
 It is difficult to get the correct result in this case if
the clocks of the sender and receiver nodes are not
synchronized.

29.  Keeping the clocks in a distributed system
synchronize to within 5 or 10 msec is an expensive
and nontrivial task.
 Lamport observed that for most application it is not
necessary to keep the clocks in a distributed system
synchronized.
 Rather, it is sufficient to ensure that all events that
occur in a distributed system be totally ordered in a
manner that is consistent with an observed
behaviour.

30.  There are several resources in a system that must not
be used simultaneously by multiple processes if
program operation is to be correct.
 For example, a file must not be simultaneously
updated by multiple processes.
 Similarly, use of unit record peripherals such as tape
drives or printers must be restricted to a single
process at a time.

31.  Therefore, exclusive acces to such a shared resource
by a process must be ensured.
 This exclusiveness of access is called mutual
exclusion between process.
 The section of a program that need exclusive access
to shared resources are referred to as critical
sections.

32.  There are three approaches in the mutual exclusion:
1) Centralized approach
2) Distributed approach
3) Token-pausing approach

33.  In concurrent computing, a deadlock is a state in
which each member of a group is waiting for another
member, including itself, to take action, such as
sending a message or more commonly releasing a
lock.

34.  Request: The process first makes a request for the
resource. If the requested resource is not available.
 Allocate: The system allocates the resources to the
requesting process as soon as possible. It maintains a
table in which it records whether each resource is
free or allocated and, if it is allocated.
 Release: After the process has finished using the
allocated resource, it releases the resource to the
system.

35.  Asking for operator intervention
 Termination
 Rollback of process

36.  Asking for operator Intervention: The simplest way
is to inform the operator that a deadlock has occurred
and to let the operator deal with it manually.
 Termination of process: The simplest way to
automatically recover from a deadlock is to
terminate one or more processes and to reclaim the
resources held by them, which can then be
reallocated.
 Rollback of process: Killing a process requires its
restart from the very expensive, particularly when
the process has already run for a substantially long
time.

37.  Election algorithms are meant for electing a
coordinator process from among the currently
running processes in such a manner that any instance
of time there is a single coordinator for all processes
in the system.

38. 1) Each process in the system has a unique priority
number.
2) Whenever an election is held, the process having the
highest priority number among the currently active
processes is elected as the coordinator.
3) On recovery, a failed process can take appropriate
actions to rejoin the set of active processes.