Parallel and Distributed System

1. Distributed
and
Parallel System
1

2. Contents
1. Definition of Distributed System
2. Examples/Application of Distributed System
3. Advantage and Disadvantages of Distributed System
4. Definition Of Parallel System
5. Examples/Applications of Parallel System
6. Advantages and Disadvantages of Parallel System
7. References/Sources

3. DEFINITION
 A distributed system is a collection of independent
computers, interconnected via a network, capable of
collaborating on a task.
 A distributed system can be characterized as
collection of multiple autonomous computers that
communicate over a communication network and
having following features:
 No common Physical clock
 Enhanced Reliability
 Increased performance/cost ratio
 Access to geographically remote data and resources
 Scalability
3

4. A collection of independent computers that
appear to the users of the system as a
single computer
A collection of autonomous computers,
connected through a network and
distribution middleware which enables
computers to coordinate their activities
and to share the resources of the system,
so that users perceive the system as a
single, integrated computing facility
Distributed Systems

5. A Distributed System
4

6. Examples of Distributed System
 Telephone Networks and Cellular
Networks
 Computer Networks Such as internet or
intranet
 ATM(bank) Machines
 Distributed database and distributed
database management system
 Network of Workstations
 Mobile Computing etc.
5

7. 6

8. Why Distributed Systems?

9. Design Requirements
 Performance issues
 Responsiveness
 Throughput
 Quality of service
 Correctness
 Reliability, availability, fault tolerance
 Security
 Performance
 Adaptability

10. Advantages Of Distributed System
 Information Sharing among Distributed Users
 Resource Sharing
 Extensibility and Incremental growth
 Shorter Response Time and Higher Output
 Higher Reliability
 Better Flexibility’s in meeting User’s needs
 Better price/performance ratio
 Scalability
 Transparency
7

11. Disadvantages of Distributed System
 Difficulties of developing distributed
software
 Networking Problem
 Security Problems
 Performance
 Openness
 Reliability and Fault Tolerance
8

12. Basic problems and
challenges
 Transparency
 Scalability
 Fault tolerance
 Concurrency
 Openness
 These challenges can also be seen as the goals or desired properties
of a distributed system

13. Transparency
 Concealment from the user and the application programmer
of the separation of the components of a distributed system
 Access Transparency - Local and remote resources are accessed in same
way
 Location Transparency - Users are unaware of the location of resources
 Migration Transparency - Resources can migrate without name change
 Replication Transparency - Users are unaware of the existence of
multiple copies of resources
 Failure Transparency - Users are unaware of the failure of individual
components
 Concurrency Transparency - Users are unaware of sharing resources with
others

14. Scalability
 Addition of users and resources without suffering a noticeable
loss of performance or increase in administrative complexity
 Adding users and resources causes a system to grow:
 Size - growth with regards to the number of users or resources
 System may become overloaded
 May increase administration cost
 Geography - growth with regards to geography or the distance between
nodes
 Greater communication delays
 Administration – increase in administrative cost

15. Openness
 Whether the system can be extended in various ways without
troublesome existing system and services
 Hardware extensions
 adding peripherals, memory, communication interfaces
 Software extensions
 Operating System features
 Communication protocols
 Openness is supported by:
 Public interfaces
 Standardized communication protocols

16. Concurrency
 In a single system several processes are interleaved
 In distributed systems - there are many systems with one or more
processors
 Many users simultaneously invoke commands or applications, access
and update shared data
 Mutual exclusion
 Synchronization
 No global clock

17. Fault tolerance
 Hardware, software and networks fail
 Distributed systems must maintain availability even at low levels of
hardware, software, network reliability
 Fault tolerance is achieved by
 Recovery
 Redundancy
 Issues
 Detecting failures
 Recovery from failures
 Redundancy

18. Fault tolerance
Omission and Arbitrary Failures

19. Parallel System
 A system is said to be a Parallel System in which
multiple processor have direct access to shared
memory which forms a common address space.
 Usually tightly-coupled system are referred to as
Parallel System. In these systems, there is a single
system wide primary memory (address space) that is
shared by all the processors. On the other hand
Distributed System are loosely-coupled system.
 Parallel computing is the use of two or more
processors (cores, computers) in combination to solve
a single problem.
9

20. Examples

21. Loosely-Coupled Systems
 Most distributed systems are “loosely-coupled:
 Each CPU runs an independent autonomous OS.
 Hosts communicate through message passing.
 Computer don’t really trust each other.
 Some resources are shared, but most are not.
 The system may look differently from different
hosts.
 Typically, communication times are long.

22. Tightly-Coupled Systems
 A “tightly-coupled” system usually refers to a
multiprocessor.
 Runs a single copy of the OS with a single job queue
 has a single address space
 usually has a single bus or backplane to which all
processors and memories are connected
 processors communicate through shared memory

23. A Parallel System
10

24. Applications of Parallel System
 An example of Parallel computing
would be two servers that share the
workload of routing mail, managing
connections to an accounting system
or database, solving a mathematical
problem etc
 Supercomputers are usually placed in
parallel system architecture
 Terminals connected to single server
11

25. EXAMPLE: The Earth Simulator Supercomputer
from (2002-2004)
12

26. Advantages of Parallel System
 Provide Concurrency(do multiple things at
the same time)
 Taking advantage of non-local resources
 Cost Savings
 Overcoming memory constraints
 Save time and money
 Global address space provides a user-
friendly programming perspective to
memory
13

27. Disadvantages of Parallel System
 Primary disadvantage is the lack of
scalability between memory and CPUs.
 Programmer responsibility for
synchronization constructs that ensure
"correct" access of global memory.
 It becomes increasingly difficult and
expensive to design and produce shared
memory machines with ever increasing
numbers of processors.
14

28. Parallel vs. Distributed System
15
Parallel Systems Distributed Systems
Memory Tightly coupled system
shared memory
Weakly coupled system
Distributed memory
Control Global clock control No global clock control
Processor
interconnection
Order of Tbps Order of Gbps
Main focus Performance
Scientific computing
Performance(cost and scalability)
Reliability/availability
Information/resource sharing

29. Sources / References
Websites
Books
 en.wikipedia.org
 books.google.com
 www.seminarprojects.com
 http://publib.boulder.ibm.com
 www.webopedia.com
 https://computing.llnl.gov/tutorials
 www.cis.upenn.edu
 Distributed System by Coulouris
 Distributed Computing by D. Kshemkalyani,Mukesh
Singhal
16

30. Thank
You