A parallel computer with a
logically shared memory system provides a single global address space to all
processors, and hence a shared programming paradigm to the users. Such systems
are referred as distributed shared memory (DSM) machines.
Creating parallel programs involves first decomposing the overall computation into
tasks and then assigning the tasks to the processors, this step is also called as
partitioning. The optimization objective for partitioning is to balance the work-load
among processors and to minimize the inter process communication needs. The
number of processes generated by the partitioning step may not be equal to the
processors, thus a processor may be idle or loaded with multiple processes. The
primary optimization objective of mapping is to balance the workload of processors
and to minimize the inter-processor communication cost. Collectively, the problem
of load balancing is to develop partitioning and mapping algorithm for the purpose
of achieving their respective optimization objectives.