The document presents a series of multiple-choice questions (MCQs) focused on concepts of parallel and distributed computing, addressing topics such as synchronization models, MPI functions, and programming approaches. It covers various models including shared memory, message passing, and hybrid models, highlighting their advantages, disadvantages, and specific functions related to parallel programming. Additionally, it discusses important concepts such as granularity, load balancing, and embarrassingly parallel problems.

1. Parallel & Distributed Computing MCQS
1. What does the shared memory model use for synchronization?
a) Pointers
b) Semaphores/Locks
c) Arrays
d) Queues
Answer: b
2. What is the main advantage of the shared memory model?
a) Data ownership is explicitly defined
b) No explicit data communication required
c) Easy to manage data locality
d) Always faster than other models
Answer: b
3. Which model involves multiple threads executing concurrently?
a) Data Parallel Model
b) Shared Memory Model
c) Threads Model
d) Hybrid Model
Answer: c
4. What is the standard for thread-based programming on UNIX systems?
a) OpenMP
b) MPI
c) POSIX Threads

2. d) HPF
Answer: c
5. Which programming model divides data into partitions for tasks?
a) Data Parallel Model
b) Message Passing Model
c) Threads Model
d) Functional Decomposition
Answer: a
6. What is the primary use of MPI?
a) Synchronizing shared memory
b) Communicating between processes with separate address spaces
c) Simplifying parallel programming
d) Automatically parallelizing serial code
Answer: b
7. What does MPI_Send() do?
a) Initializes MPI environment
b) Sends messages between processes
c) Receives messages from processes
d) Finalizes MPI environment
Answer: b
8. What function is used to initialize MPI?
a) MPI_Start
b) MPI_Comm_size
c) MPI_Init

3. d) MPI_Finalize
Answer: c
9. What does MPI_Comm_rank provide?
a) Total number of processes
b) Rank of a process
c) Type of communicator
d) Process group details
Answer: b
10. Which MPI function is used to finalize parallel code?
a) MPI_Stop
b) MPI_Close
c) MPI_Finalize
d) MPI_Comm_free
Answer: c
11. What is the first step in designing a parallel program?
a) Code optimization
b) Understanding the problem
c) Writing test cases
d) Selecting a programming language
Answer: b
12. Which of these is an example of an embarrassingly parallel problem?
a) Fibonacci sequence
b) Image processing
c) Climate modeling

4. d) Matrix multiplication
Answer: b
13. Which synchronization type requires all tasks to stop at the same point?
a) Barrier
b) Semaphore
c) Lock
d) Asynchronous communication
Answer: a
14. What is the major inhibitor to parallelism in loops?
a) Data independence
b) Data dependencies
c) Small granularity
d) High I/O
Answer: b
15. Which term describes dividing tasks by functionality?
a) Domain decomposition
b) Data parallelism
c) Functional decomposition
d) Hybrid decomposition
Answer: c
16. What does granularity refer to in parallel programming?
a) Size of parallel tasks
b) Communication overhead
c) Memory allocation

5. d) Processor speed
Answer: a
17. Which factor measures communication efficiency?
a) Latency
b) Granularity
c) Synchronization
d) Load balancing
Answer: a
18. What is the goal of load balancing?
a) Equal distribution of tasks
b) Reducing memory usage
c) Increasing task dependencies
d) Simplifying I/O operations
Answer: a
19. What does the hybrid model combine?
a) Threads and loops
b) Data and memory
c) Different parallel programming models
d) Serial and parallel programming
Answer: c
20. What is the common data type used in MPI communications?
a) MPI_FLOAT
b) MPI_INT
c) MPI_CHAR

6. d) All of the above
Answer: d
21. What is the shared memory model’s main advantage?
a) Explicit data communication
b) Simplified development
c) Easy data ownership
d) High performance always
Answer: b
22. Which mechanism is used for synchronization in shared memory?
a) Loops
b) Locks/Semaphores
c) Threads
d) Buffers
Answer: b
23. Which model is commonly used with global address space?
a) Message Passing Model
b) Shared Memory Model
c) Data Parallel Model
d) Hybrid Model
Answer: b
24. What defines the threads model?
a) Independent processes
b) Tasks sharing memory and resources
c) Communication via MPI

7. d) Explicit ownership of data
Answer: b
25. Which standard provides thread implementations?
a) OpenMP
b) POSIX Threads
c) MPI
d) HPF
Answer: b
26. Which model works best for SIMD architecture?
a) Threads Model
b) Data Parallel Model
c) Shared Memory Model
d) Message Passing Model
Answer: b
27. Which model divides tasks based on computation?
a) Domain Decomposition
b) Functional Decomposition
c) Hybrid Model
d) Data Partitioning
Answer: b
28. What does functional decomposition focus on?
a) Data distribution
b) Computation tasks
c) Message passing

8. d) Synchronization
Answer: b
29. Which hybrid model uses both MPI and GPU programming?
a) CUDA+MPI
b) OpenMP+MPI
c) POSIX Threads+MPI
d) MPI+HPF
Answer: a
30. Which shared memory architecture uses CC-NUMA?
a) SGI Origin
b) IBM Blue Gene
c) Cray T3E
d) Intel Xeon
Answer: a
31. What does MPI_Send() do?
a) Synchronizes processes
b) Sends messages between processes
c) Receives data
d) Finalizes communication
Answer: b
32. What does MPI_Comm_rank() return?
a) Process rank
b) Total processes
c) Message tag

9. d) Data type
Answer: a
33. Which MPI function terminates parallel execution?
a) MPI_Stop
b) MPI_Finalize
c) MPI_Close
d) MPI_End
Answer: b
34. Which function gives the total number of processes?
a) MPI_Comm_total
b) MPI_Comm_size
c) MPI_Init
d) MPI_Rank
Answer: b
35. What does MPI_Recv() require to receive messages?
a) MPI_Init status
b) Process ID, data type, tag
c) Rank and memory address
d) Synchronization locks
Answer: b
36. Which term is used to define all processes in MPI?
a) MPI_GROUP_WORLD
b) MPI_COMM_WORLD
c) MPI_COMM_ALL

10. d) MPI_PROCESS_GROUP
Answer: b
37. What does MPI_ANY_TAG allow?
a) Ignoring rank during sends
b) Receiving messages with any tag
c) Sending without specifying data type
d) Sending from any source
Answer: b
38. What does MPI_Finalize() not do?
a) Clean up resources
b) Free communicator objects
c) Allocate processes
d) End parallel code
Answer: c
39. Which operation in MPI ensures synchronization?
a) Scatter
b) Gather
c) Barrier
d) Rank update
Answer: c
40. What is MPI_Recv() used for?
a) Synchronizing processes
b) Broadcasting messages
c) Receiving data from other processes

11. d) Allocating buffers
Answer: c
41. What is the primary goal of parallel programming?
a) Reducing memory usage
b) Decreasing execution time
c) Increasing code complexity
d) Improving data dependencies
Answer: b
42. Which operation distributes data from one task to all others in MPI?
a) MPI_Scatter
b) MPI_Bcast
c) MPI_Reduce
d) MPI_Gather
Answer: b
43. Which of these is a hybrid communication model example?
a) MPI + OpenMP
b) MPI + CUDA
c) Both a and b
d) Pthreads only
Answer: c
44. Which type of communication requires explicit synchronization in parallel
programs?
a) Synchronous communication
b) Asynchronous communication

12. c) Collective communication
d) None of the above
Answer: a
45. What is an example of an embarrassingly parallel task?
a) Matrix multiplication
b) Image pixel color inversion
c) Recursive Fibonacci calculation
d) Weather modeling
Answer: b
46. What is the primary disadvantage of the message passing model?
a) Scalability issues
b) High memory requirements
c) High communication overhead
d) Poor granularity
Answer: c
47. What is the primary benefit of using asynchronous communication in
MPI?
a) Simplifies debugging
b) Improves execution speed by overlapping computation and communication
c) Reduces synchronization complexity
d) Ensures complete communication reliability
Answer: b
48. Which synchronization construct is used to protect shared resources?
a) Barrier

13. b) Semaphore
c) Broadcast
d) MPI_Finalize
Answer: b
49. What type of dependency arises when one task requires data generated
by another?
a) Data dependency
b) Functional dependency
c) Resource dependency
d) None of the above
Answer: a
50. What is the role of profilers in parallel programming?
a) To debug memory issues
b) To identify code hotspots
c) To partition tasks
d) To optimize communication costs
Answer: b
61. What is the first step in parallel program design?
a) Understanding the problem
b) Writing test cases
c) Optimizing serial code
d) Selecting a language
Answer: a
62. Which problem is embarrassingly parallel?

14. a) Fibonacci sequence
b) Image pixel processing
c) Sorting algorithms
d) Climate modeling
Answer: b
63. Which synchronization method stops all tasks at one point?
a) Semaphore
b) Lock
c) Barrier
d) Deadlock
Answer: c
64. What does domain decomposition focus on?
a) Tasks based on data
b) Tasks based on computation
c) Synchronization methods
d) Communication strategies
Answer: a
65. Which issue arises from data dependency in parallel programming?
a) Faster execution
b) Reduced granularity
c) Inhibited parallelism
d) Simplified communication
Answer: c
66. What is load balancing in parallel programming?

15. a) Distributing tasks evenly
b) Minimizing memory use
c) Ensuring task dependencies
d) Reducing I/O
Answer: a
67. Which communication is non-blocking in nature?
a) Synchronous communication
b) Asynchronous communication
c) Barrier-based methods
d) Data dependencies
Answer: b
68. What does granularity refer to?
a) Task size in parallel programming
b) Memory usage
c) Code complexity
d) Synchronization delays
Answer: a
69. Which type of communication involves only two tasks?
a) Collective communication
b) Point-to-point communication
c) Broadcast communication
d) Scatter-gather communication
Answer: b
70. Which operation divides work into functional chunks?

16. a) Domain decomposition
b) Functional decomposition
c) Hybrid decomposition
d) Barrier decomposition
Answer: b
71. What is the primary disadvantage of the shared memory model?
a) Easy data communication
b) Complexity of data locality management
c) High communication costs
d) Poor scalability
Answer: b
72. Which type of parallelism does the message-passing model support?
a) SIMD
b) MIMD/SPMD
c) Vector Processing
d) None of the above
Answer: b
73. Which is the key feature of the hybrid model?
a) Combines multiple programming models
b) Fully automatic synchronization
c) Only works on shared memory systems
d) Eliminates communication overhead
Answer: a
74. What is the common drawback of functional decomposition?

17. a) Difficult to scale
b) Limited task flexibility
c) Requires specialized hardware
d) Not efficient for independent tasks
Answer: b
75. In domain decomposition, what is divided?
a) Functionality
b) Data
c) Process ranks
d) Memory allocation
Answer: b
76. What is a typical issue with threads in shared memory models?
a) High latency
b) Race conditions
c) Insufficient granularity
d) Limited scalability
Answer: b
77. Which parallel programming model uses directives for parallelism?
a) POSIX Threads
b) OpenMP
c) MPI
d) CUDA
Answer: b
78. Which type of hardware is best suited for the data-parallel model?

18. a) Distributed memory systems
b) SIMD architectures
c) CC-NUMA systems
d) Asynchronous networks
Answer: b
79. What does CUDA mainly focus on?
a) Distributed memory systems
b) GPU computing
c) Shared memory models
d) MPI optimizations
Answer: b
80. In parallel programming, granularity refers to:
a) The size of tasks
b) Communication type
c) Scalability of processors
d) Number of dependencies
Answer: a
81. What does MPI_Init() initialize?
a) MPI processes
b) Communication buffers
c) Parallel regions in code
d) All of the above
Answer: a
82. What is the role of MPI_Comm_world?

19. a) Defines point-to-point communication
b) Contains all MPI processes in the communicator
c) Allocates data buffers
d) Synchronizes processes
Answer: b
83. What does the "rank" in MPI refer to?
a) The priority of the process
b) The unique identifier for a process
c) The speed of a process
d) The number of tasks in a process
Answer: b
84. Which function performs a broadcast operation in MPI?
a) MPI_Scatter
b) MPI_Gather
c) MPI_Bcast
d) MPI_Comm_rank
Answer: c
85. MPI_Send() is an example of:
a) Blocking communication
b) Non-blocking communication
c) Synchronization
d) Data collection
Answer: a
86. What is latency in MPI?

20. a) The maximum bandwidth available
b) Time taken to send a zero-byte message
c) Number of processes involved in communication
d) Size of the message being transferred
Answer: b
87. Which wildcard allows receiving messages from any source?
a) MPI_ANY_SOURCE
b) MPI_ALL_SOURCE
c) MPI_ANY_TAG
d) MPI_ALL_TAGS
Answer: a
88. What is a "blocking communication" in MPI?
a) A communication that completes without delay
b) A communication where tasks wait until the operation finishes
c) Communication performed without synchronization
d) None of the above
Answer: b
89. Which operation combines data from all processes into one?
a) MPI_Scatter
b) MPI_Gather
c) MPI_Reduce
d) MPI_Finalize
Answer: c
90. What happens if MPI_Finalize() is not called?

21. a) Parallel execution stops automatically
b) MPI processes may remain active
c) Processes synchronize automatically
d) It triggers garbage collection
Answer: b
91. What is the key step in identifying program hotspots?
a) Testing the program
b) Using profilers
c) Writing optimized code
d) Measuring I/O latency
Answer: b
92. Which is an example of a non-parallelizable problem?
a) Calculating Fibonacci series
b) Image processing
c) Particle simulation
d) Sorting numbers
Answer: a
93. Why are data dependencies inhibitors to parallelism?
a) They require shared memory
b) They cause race conditions
c) They prevent tasks from executing independently
d) They increase task granularity
Answer: c
94. What is the primary goal of load balancing?

22. a) Equal distribution of tasks across processors
b) Reducing communication overhead
c) Minimizing memory allocation
d) Synchronizing tasks
Answer: a
95. Which synchronization type ensures tasks reach a common point?
a) Semaphore
b) Lock
c) Barrier
d) Non-blocking communication
Answer: c
96. What is "functional decomposition"?
a) Splitting tasks by data dependencies
b) Dividing work based on functionality
c) Partitioning tasks based on communication
d) Synchronizing task execution
Answer: b
97. Which factor does not affect communication efficiency?
a) Bandwidth
b) Task granularity
c) Latency
d) Network traffic
Answer: b
98. Which communication is asynchronous in nature?

23. a) Barrier synchronization
b) Non-blocking communication
c) Blocking message passing
d) Point-to-point communication
Answer: b
99. What does granularity measure?
a) Time required for task execution
b) Size of tasks in parallel programs
c) Total memory allocated
d) Level of task synchronization
Answer: b
100. What is the final step in designing parallel programs?
a) Identifying hotspots
b) Partitioning tasks
c) Performance tuning
d) Selecting synchronization methods
Answer: c