Distributed systems

1. Introduction to
Distributed
Systems

2. Today’s Session…
• What is a distributed system?
• Why do we need distributed system?
• Examples of distributed system
• Advantages
• Challenges
• Architectural categories
• Points to ponder

3. What do we have now which is not before??
Processor Memory
Storage Protocols
Networking

4. What do we have now which is not before??
Processor Memory
Storage Protocols
Networking
Computers got-
• Smaller
• Cheaper
• Power efficient
• Faster

5. What do we have now which is not before??
Processor Memory
Storage Protocols
Networking
• WHY?
- Microprocessor Tech

6. What do we have now which is not before??
Processor Memory
Storage Protocols
Networking
Avg song size = 4.4MB
Then @44kbps, download time was 15 mins
Now @12.9 mbps, download time is 3.5 sec

7. What do we have now which is not before??
Processor Memory
Storage Protocols
Networking
Large amount of Data
can be stored

8. What do we have now which is not before??
Processor Memory
Storage Protocols
Networking
• World is connected with
network.
• Live Streaming is possible
in secs.
• Faster and clear.

9. • A distributed system is a collection of
independent computers that appear to the
users of the system as a single coherent
system.
• Components located at networked
computers communicate and coordinate
their actions only by passing messages.
What is a distributed system??

10. What is a distributed system?? (Contd…)
• Consider large companies like Google or
Facebook with,
• Large number of Services, many users,
many requests at a time.
Can a single computer handle all
this??

11. What is a distributed system?? (Contd…)
• Consider large companies like Google or
Facebook with,
• Large number of Services, many users,
many requests at a time.
NO

12. What is a distributed system?? (Contd…)
• Consider large companies like Google or
Facebook with,
• Large number of Services, many users,
many requests at a time.
Do it with multiple computers which talk
to each other

13. Formal definition of distributed system
• A distributed system is a collection of
independent entities that cooperate to solve
a problem that cannot be individually solved.

15. Three major technologies in distributed system
• Main Frame Systems
• Bulk data processing
• Clusters
• Group of server and other resources connected
together
• Replacement of main frame to reduce cost
• Grid
• Combines resources over different geographical
location

16. Characteristics of distributed system
• No common physical clock
• No shared memory
• Geographical separation
• Autonomy and heterogeneity

17. Motivation for using a Distributed system
• Inherently distributed computations
• Resource sharing
• Access to geographically remote data and
resources
• Enhanced reliability
• Increased performance/cost ratio
• Scalability
• Modularity and incremental expandability

18. Relation to computer system components
• Each computer has a memory-processing unit and the
computers are connected by a communication
network.

19. Relation to computer system components
• The distributed software is also termed as middleware.
• A distributed execution is the execution of processes across the
distributed system to collaboratively achieve a common goal.
• An execution is also sometimes termed a computation or a run.

20. Characteristics of parallel systems
A parallel system may be broadly classified as belonging to one of
three types:
• A multiprocessor system
• A multicomputer parallel system
• Array processors

21. Characteristics of parallel systems
A parallel system may be broadly classified as belonging to one of
three types:
• A multiprocessor system
• A multicomputer parallel system
• Array processors
• A multiprocessor system is a parallel system in which the
multiple processors have direct access to shared memory
which forms a common address space.
• Such processors usually do not have a common clock.

22. Characteristics of parallel systems
A parallel system may be broadly classified as belonging to one of
three types:
• A multiprocessor system
• A multicomputer parallel system
• Array processors
Two standard architectures for parallel systems.
(a) Uniform memory access (UMA) multiprocessor system.
(b) Non-uniform memory access (NUMA) multiprocessor.

23. Characteristics of parallel systems
A parallel system may be broadly classified as belonging to one of
three types:
• A multiprocessor system
• A multicomputer parallel system
• Array processors
(a) Uniform memory access (UMA) multiprocessor system.
(b) Non-uniform memory access (NUMA) multiprocessor.

24. Characteristics of parallel systems
A parallel system may be broadly classified as belonging to one of
three types:
• A multiprocessor system
• A multicomputer parallel system
• Array processors
Uniform Memory Access (UMA)
• All the processors share the physical
memory in a centralized manner with
equal access time to all the memory.
• Each processor may have a private
cache memory.
• When all the processors have equal
access to all the peripheral devices, the
system is called a symmetric
multiprocessor.
• When only one or a few processors can
access the peripheral devices, the system
is called an asymmetric multiprocessor.
• When a CPU wants to access a memory
location, it checks if the bus is free, then
it sends the request to the memory
interface module and waits for the
requested data to be available on the
bus.

25. Characteristics of parallel systems
A parallel system may be broadly classified as belonging to one of
three types:
• A multiprocessor system
• A multicomputer parallel system
• Array processors
Non-uniform Memory Access
(NUMA)
• Shared memory is physically
distributed among all the
processors, called local memories.
• The collection of all local
memories forms a global address
space which can be accessed by all
the processors.
• NUMA systems also share CPUs
and the address space, but each
processor has a local memory,
visible to all other processors.
• In NUMA systems access to local
memory blocks is quicker than
access to remote memory blocks.

26. Characteristics of parallel systems
A parallel system may be broadly classified as belonging to one of
three types:
• A multiprocessor system
• A multicomputer parallel system
• Array processors
Non-uniform Memory Access
(NUMA)
• Shared memory is physically
distributed among all the
processors, called local memories.
• The collection of all local
memories forms a global address
space which can be accessed by all
the processors.
• NUMA systems also share CPUs
and the address space, but each
processor has a local memory,
visible to all other processors.
• In NUMA systems access to local
memory blocks is quicker than
access to remote memory blocks.

27. Characteristics of parallel systems
A parallel system may be broadly classified as belonging to one of
three types:
• A multiprocessor system
• A multicomputer parallel system
• Array processors
Multiple processors do not have direct access to shared memory.
The memory of the multiple processors may or may not form a
common address space.
Such computers usually do not have a common clock.

28. Shared Memory Message Passing
It is one of the region for data
communication
Mainly the message passing is used for
communication.
It is used for communication between
single processor and multiprocessor
systems where the processes that are
to be communicated present on the
same machine and they are sharing
common address space.
It is used in distributed environments
where the communicating processes
are present on remote machines which
are connected with the help of a
network.
The shared memory code that has to be
read or write the data that should be
written explicitly by the application
programmer.
Here no code is required because the
message passing facility provides a
mechanism for communication and
synchronization of actions that are
performed by the communicating
processes.
It is going to provide a maximum speed
of computations because the
communication is done with the help of
shared memory so system calls are
used to establish the shared memory.
Message passing is a time consuming
process because it is implemented
through kernel (system calls).

29. Shared Memory Message Passing
In shared memory make sure that the
processes are not writing to the same
location simultaneously.
Message passing is useful for sharing
small amounts of data so that conflicts
need not occur.
It follows a faster communication
strategy when compared to message
passing technique.
In message passing the communication
is slower when compared to shared
memory technique.
Given below is the structure of shared
memory system −
Given below is the structure of message
passing system −

30. Executing Tasks: Sync versus
Async

31. CS8603 – SYLLABUS
UNIT I INTRODUCTION
Introduction: Definition –Relation to computer system components –Motivation –Relation to parallel
systems — Message-passing systems versus shared memory systems –Primitives for distributed
communication –Synchronous versus asynchronous executions –Design issues and challenges. A model of
distributed computations: A distributed program –A model of distributed executions –Models of
communication networks –Global state — Cuts –Past and future cones of an event –Models of process
communications. Logical Time: A framework for a system of logical clocks –Scalar time –Vector time —
Physical clock synchronization: NTP.
UNIT II MESSAGE ORDERING & SNAPSHOTS
Message ordering and group communication: Message ordering paradigms –Asynchronous execution
with synchronous communication –Synchronous program order on an asynchronous system –Group
communication — Causal order (CO) — Total order. Global state and snapshot recording algorithms:
Introduction –System model and definitions –Snapshot algorithms for FIFO channels
UNIT III DISTRIBUTED MUTEX & DEADLOCK
Distributed mutual exclusion algorithms: Introduction — Preliminaries — Lamport?s algorithm — Ricart-
Agrawala algorithm — Maekawa?s algorithm — Suzuki–Kasami?s broadcast algorithm. Deadlock
detection in distributed systems: Introduction — System model — Preliminaries — Models of deadlocks
— Knapp?s classification — Algorithms for the single resource model, the AND model and the OR model.
UNIT IV RECOVERY & CONSENSUS
Checkpointing and rollback recovery: Introduction — Background and definitions — Issues in failure
recovery — Checkpoint-based recovery — Log-based rollback recovery — Coordinated checkpointing
algorithm — Algorithm for asynchronous checkpointing and recovery. Consensus and agreement
algorithms: Problem definition — Overview of results — Agreement in a failure —
free system — Agreement in synchronous systems with failures.
UNIT V P2P & DISTRIBUTED SHARED MEMORY
Peer-to-peer computing and overlay graphs: Introduction — Data indexing and overlays — Chord —
Content addressable networks — Tapestry. Distributed shared memory: Abstraction and advantages —
Memory consistency models –Shared memory Mutual Exclusion.