This chapter discusses asynchronous parallelism in MIMD systems. In asynchronous parallelism, problems are partitioned into independent subproblems that are distributed to autonomous processors. The subproblems may require communication and synchronization. MIMD systems allow for multiple instruction streams and asynchronous execution. Processors can have local memory in a loosely coupled system or shared memory in a tightly coupled system. Processes move between ready, running, and blocked states.

1. Chapter 8
Asynchronous parallelism

2. Chapter 8
Asynchronous Parallelism
• 8.1 MIMD Systems
• 8.2 Asynchronous parallelism
• 8.3 Process States

3. Introduction
• A distinction made between two large classes of parallel
processing; asynchronous and synchronous parallelism.
• In the classical asynchronous parallelism, the problem to
be solved is partitioned into sub-problems in the form of
processes that can be distributed among a group of
autonomous processors.
• The sub-problems ma not be totally independent, so the
processes must exchange information among themselves
• And therefore must be mutually synchronized (this would
lead to large synchronization cost).
• Because of the cost asynchronous parallelism is often
characterized as coarse grain parallelism.

4. 8.1 MIMD Systems
• Two interesting class are SIMD (synchronous) and MIMD
(asynchronous).

5. 8.2 Asynchronous parallelism
• means that there are multiple threads of control. (data
exchange, each processor executes individual programs).
• MIMD and SIMD according to their inter connection
topology.
• (From page 8, fig 2.4 and 2.5 and 2.2 senkron)
• (Alan page 9, fig 5)
• This class (MIMD) is more general structure and always
work asynchronously)
• MIMD computers with shared memory are known as
tightly coupled and,

9. • Synchronization and information exchange occur via
memory areas which can be addressed by different
processor in a coordinated manner.
• Accesses to the same portion of shared memory at the
same time requires an arbitration mechanism which must
be used to ensure only one processor accesses that memory
portion at a time.
• This problem of memory contention may restrict the
number of processors that can be interconnected using
shared memory model.
• MIMD computers without shared memory are known as
loosely coupled.
• Each PE has its own local memory.
• Synchronization and communication are much more
costly without share memory, because
• Messages must be exchanged over the network.

10. • PE wishes to access another PE’s private memory, it can
only do so by sending a message to the appropriate PE
along the interconnection network.
• (Kitaptan devam (Brunnel))

11. Structure of a MIMD system
• The most general model of a MIMD computer is shown in
figure 8.x (6.1).
• The processors (PE) are autonomous computer system that
can carry out different programs independently of one
other.
• Depending configuration, the processors may have their
own local memory (loosely coupled) or may use a shared
memory (tightly coupled, shared one is generally with bus
system).
• The concept of virtually shared memory can be applied to
simulate a shared memory area by data exchange protocols
between the PEs.

13. • The processors function asynchronously independently one
another.
• In order to exchange information, they have to be
synchronized.
• A process is divided into individual processes (run
independently, synchronized for data exchange).
• The ideal mapping would be 1 process: 1 processor, which
is normally not possible in practice, due to the limitted
number of available processor.
• The general mapping is n process: 1 processor (time
sharing, multiple process run on a PE, scheduler and extra
control cost).

14. MIMD computer system
• In this section, the MIMD computer systems Sequent
symmetry, Intel hypercube, and Intel Paragon will be
briefly discussed.

15. Sequent symmetry
• The block diagram of sequent symmetry is illustrated in
figure 8.x (6.2).
• The SS MIMD computer is a good example of the tightly
coupling of processors and shared global memory accessed
over a central bus.
• Up to 300 PE of type 80486 with high clock rate may be
connected.
• Bus allows two device communicate simultaneously (then
bus restricts the scalability).

17. Intel iPSC Hypercube
• These systems has several generations and these are based
on 80286, 80386, and i860.
•
• Up to 128 powerful processors can be put together in a
iPSC/860 Hypercube (no share memory, loosely coupled,
message passing procedures exchanges the data between
PEs).
•
• Semaphore and Monitor can only be implemented locally,
but not between PEs.

18. Intel Paragon
• The system contains i860 xp PEs.
• Up to 512 nodes (each node containing two i860 xp PEs).
•
• One processors on each node is assigned to
arithmetic/logic tasks, while other PE is dedicated to
handling data exchange.
•
• The nodes connected by two-dimensional grid.

19. 8.3 Process states
• A process is and individual program segment that executes
asynchronously in parallel other processes.
• They run concurrently and parallel On several PEs.
• Many processes on a PE run concurrently (a time-sliced
fashion).
• At the end the running process changes its state from
running to ready.
• Then all process control data (program counter, registers,
data addresses, etc.) of a process must be stored in its
process control block (PCB), so it can be retrieved lather.

20. • Newly arriving processes are placed immediately in queue
ready, while terminating process are removed from
running state.
• The process state model is illustrated in figure 8.x (6.3).
• The part of the OS for doing these tasks is the scheduler.
• More complex scheduler for multi-processors system will
be given later.
• The scheduler capable of moving processes from a heavily
loaded processor to a more lightly loaded processor.