This document discusses key topics in parallel and distributed computing including scheduling, network consistency models, load balancing, and memory hierarchies. It describes issues in scheduling parallel tasks and different consistency models like sequential consistency. It also outlines types of load balancing and discusses memory hierarchies in parallel systems.

2. Parallel & Distributed computing
 Presented By:
Group# 03
Shahab Ahmad M. Aashan
Shoaib Hussain Sheraz Yar
SCHEDULING
LOAD
BALANCING
NETWORK
CONSISTENCY
MODEL
MEMORY
HIERARCHIES

3. 1) SCHEDULING

5. Issues in Scheduling in Parallel Computing

7. 2) NERWORK CONSISTENCY MODEL
 ordering of externally visible events (i.e., reads and writes to the
memory system
 Consistency models are not about memory operations from
different processors or not about dependent memory operations
in a single processor’s instruction stream
 Consistency models are all about ordering constraints on
independent memory operations in a single processor’s
instruction stream that have some high-level dependence (such
as locks guarding data) that should be respected to obtain
intuitively reasonable results

8. MODELS
 Sequential Consistency
• SC constrains all memory operations:
Write Read Write Write Read Read
• Write Simple model for reasoning about parallel programs
• You can verify that the examples considered earlier work
correctly under sequential consistency
• This simplicity comes at the cost of uniprocessor
performance

9. Sequential consistency (SC) [Lamport]result of
execution is as if memory operations of each
process are executed in program
orderMEMORYP1 P3 P2 Pn

10. Further relaxation of weak consistency
Synchronization accesses are divided into Acquires: operations like lock
Release: operations like unlock Semantics of acquire: Acquire must
complete before all following memory accesses Semantics of release:
all memory operations before release are complete However, accesses
after release in program order do not have to wait for release
operations which follow release and which need to wait must be
protected by an acquire acquire does not wait for accesses preceding it
Release Consistency

11. 3) LOAD BALANCING

13. Types of Load Balancing

15. 4) MEMORY HIERARCHIES