Message Passing Architecture
Introduction• The architecture is used to communicate  data among a set of processors without the  need for a global memor...
Message Passing Systems104     MESSAGE PASSING ARCHITECTURE            P1        M1           P2          M2             P...
Message Passing Systems104     MESSAGE PASSING ARCHITECTURE            P1        M1           P2          M2             P...
MP Network•   Normally, a static-type network    •   hypercubes    •   nearest-neighbor 2D & 3D mesh•   Two important fact...
Executing Program•   The program is divided into concurrent    processes; each is executed on a separate    processor.•   ...
Process Communication• Processes running on a given processor  use what is called internal channels to  exchange messages ...
Data Exchanged• Data exchanged among processors cannot  be shared; it is rather copied (using send/  receive messages).• A...
More advantages• A message passing scheme offers flexibility  in accommodating a large number of  processors in addition to...
5.2    ROUTING IN MESSAGE PASSING NETWORKS    105                                     Outside World        P1             ...
Process Granularity• It’s a parameter which describes the size of  a process in a message passing system.                 ...
Granularity Types•   There are 3 types of granularity:    •   Coarse - each process holds a large number        of sequent...
Routing in MP Networks• Involves the identification of a set of  permissible paths that may be used by a  message to reach ...
Routing Techniques•   Adaptive    •   the path taken by the message depends on        network conditions for a given sourc...
Routing for Broadcasting     & Multicasting• Communication operations: unicast and  collective • Unicast - communicate to ...
Broadcast                     Multicastone-to-all operation        one-to-many operationdistribute data during            ...
of these resources in a strict monotonic order. This restricted way for using networkresources prevents the occurrence of ...
Routing Potential Problems•   Deadlock    •   When two messages each hold the resources        required by the other in or...
Message Passing vs      Shared Memory•   Shared memory systems may be easier to    program, but the difficult to scale up t...
Upcoming SlideShare
Loading in...5
×

Slide05 Message Passing Architecture

1,782
-1

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,782
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
58
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Slide05 Message Passing Architecture

  1. 1. Message Passing Architecture
  2. 2. Introduction• The architecture is used to communicate data among a set of processors without the need for a global memory.• Each PE has its own local memory and communicates with other PEs using message.
  3. 3. Message Passing Systems104 MESSAGE PASSING ARCHITECTURE P1 M1 P2 M2 Pn Mn Link 1 Link 2 Link n Interconnection Network Figure 5.1 Message passing systems.networks have received considerable attention over the years. As shown in Chapter 2,two important factors must be considered in designing message passing interconnec-tion networks: link bandwidth and the network latency. The link bandwidth is definedas the number of bits that can be transmitted per unit of time (bits/s). Network latencyis defined as the time to complete a message transfer through the network.
  4. 4. Message Passing Systems104 MESSAGE PASSING ARCHITECTURE P1 M1 P2 M2 Pn Mn Link 1 Link 2 Link n Interconnection Network Figure 5.1 Message passing systems.networks have received considerable attention over the years. As shown in Chapter 2,two important factors must be considered in designing message passing interconnec- external channeltion networks: link bandwidth and the network latency. The link bandwidth is definedas the number of bits that can be transmitted per unit of time (bits/s). Network latencyis defined as the time to complete a message transfer through the network.
  5. 5. MP Network• Normally, a static-type network • hypercubes • nearest-neighbor 2D & 3D mesh• Two important factors must be considered: • link bandwidth - the number of bits that can be transmitted per unit of times (bits/s) • network latency - the time to complete a message transfer through the network
  6. 6. Executing Program• The program is divided into concurrent processes; each is executed on a separate processor.• If the no. of processes is larger than the number of processors, then more than one process will have to be executed on a processor in a time-shared fashion. 104 MESSAGE PASSING ARCHITECTURE P1 M1 P2 M2 Pn Mn Link 1 Link 2 Link n Interconnection Network Figure 5.1 Message passing systems. networks have received considerable attention over the years. As shown in Chapter 2, two important factors must be considered in designing message passing interconnec- tion networks: link bandwidth and the network latency. The link bandwidth is defined as the number of bits that can be transmitted per unit of time (bits/s). Network latency
  7. 7. Process Communication• Processes running on a given processor use what is called internal channels to exchange messages among themselves.• Processes running on different processors use the external channels to exchange messages.
  8. 8. Data Exchanged• Data exchanged among processors cannot be shared; it is rather copied (using send/ receive messages).• An important advantage of this form of data exchange is the elimination of the need for synchronization constructs, such as semaphores, which results in performance improvement.
  9. 9. More advantages• A message passing scheme offers flexibility in accommodating a large number of processors in addition to being readily scalable.• A given node can execute more than one process, each at a given time.
  10. 10. 5.2 ROUTING IN MESSAGE PASSING NETWORKS 105 Outside World P1 m1 P2 m4 P3 m2 m3 P4 Figure 5.2 An example of a message passing system.Three types of granularity can be distinguished. These are: 1. Coarse granularity: Each process holds a large number of sequential instruc-
  11. 11. Process Granularity• It’s a parameter which describes the size of a process in a message passing system. computation time Process Granularity = communication time
  12. 12. Granularity Types• There are 3 types of granularity: • Coarse - each process holds a large number of sequential instructions and takes a substantial amount of time to execute. • Medium - a middle ground where communication overhead is reduced • Fine - each process contains a few sequential instructions• Use mostly medium or coarse granularity
  13. 13. Routing in MP Networks• Involves the identification of a set of permissible paths that may be used by a message to reach its destination, and a function (η) that selects one path from the set of permissible paths.
  14. 14. Routing Techniques• Adaptive • the path taken by the message depends on network conditions for a given source and destination pair• Deterministic (oblivious) • determines the path using only the source and destination regardless of the network conditions - simple but bandwidth inefficient
  15. 15. Routing for Broadcasting & Multicasting• Communication operations: unicast and collective • Unicast - communicate to only a single destination • Collective - several routing operations are defined, broadcast & multicast are the most widely used.
  16. 16. Broadcast Multicastone-to-all operation one-to-many operationdistribute data during uses in large-scale computation of a multiprocessors, inc. distributed memory parallel search algorithm program routing algorithms must be deadlock-free little time and short path is desirable
  17. 17. of these resources in a strict monotonic order. This restricted way for using networkresources prevents the occurrence of circular wait, and hence prevents the occur-rence of deadlock. The channel dependency graph (CDG) is a technique usedto develop a deadlock-free routing algorithm. A CDG is a directed graphD ¼ G(C, E ), where the vertex set C consists of all the unidirectional channels inthe network and the set of edges E includes all the pairs of connected channels, 000 110 111 001 010 100 011 101 110 011 101 110 100 101 111 010 100 111 100 111 001 010 011 001 110 101 011 010 000 001 Figure 5.3 Hypercube broadcast tree-based communication.
  18. 18. Routing Potential Problems• Deadlock • When two messages each hold the resources required by the other in order to move, both message will be blocked.• Livelock • A situation in which a message keeps going around the network and never reaches its destination.• Starvation • A node is said to suffer from starvation if it has a message to inject into the network but is never allowed to do so.
  19. 19. Message Passing vs Shared Memory• Shared memory systems may be easier to program, but the difficult to scale up to a large number of processors.• If scalability to larger and larger systems was to continue, systems had to use message passing techniques.• However, a number of problems associated with message passing systems: communication overhead and difficulty of programming.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×