MPI collective communication routines like MPI_Allreduce, MPI_Gather, and MPI_Scatter allow processes in parallel programs to share data. MPI_Allreduce combines data from all processes using a reduction operation like sum or max. MPI_Gather collects portions of a send buffer from each process into a receive buffer. MPI_Scatter distributes different chunks of an array from a root process to other processes. Exercises demonstrate changing data and testing these routines work correctly across processes.

1. MPI Collective
Communication
Parallel Computing for Heterogeneous Platforms

2. MPI_Allreduce • MPI_Allreduce is the equivalent of
doing MPI_Reduce followed by an MPI_Bcast
MPI_Reduce vs MPI_AllReduce
[Source]: https://mpitutorial.com/tutorials/mpi-reduce-and-allreduce/

3. MPI All Reduce (Syntax)

5. Exercise (ex1.c)
• Execute the code given in previous slide
• Make some changes to initialize the data in a different manner and
then compare the results (print receive buffer at different MPI
Processes)
• Ans: lab2.1/ex1.c

6. MPI_Gather

7. MPI_Gather

9. Exercise (ex2.c)
• Execute the code given in previous slide
• Make changes in the send buffer and test the code if it works
correctly to gather elements of the send buffer in different MPI
processes.

10. MPI_Scatter
• MPI_Scatter is a collective routine that is very similar to MPI_Bcast
• MPI_Scatter involves a designated root process sending data to all
processes in a communicator.
• The primary difference between MPI_Bcast and MPI_Scatter is small
but important.
• MPI_Bcast sends the same piece of data to all processes while
MPI_Scatter sends chunks of an array to different processes.

11. MPI_Scatter

13. Reference
• https://mpitutorial.com/tutorials/