MPI Collective Communication Operations, Parallel Computing for Heterogeneous Platforms

1. MPI Collective
Communication
Parallel Computing for Heterogeneous Platforms

2. MPI_Allreduce • MPI_Allreduce is the equivalent of
doing MPI_Reduce followed by an MPI_Bcast
MPI_Reduce vs MPI_AllReduce
[Source]: https://mpitutorial.com/tutorials/mpi-reduce-and-allreduce/

3. MPI All Reduce (Syntax)

5. Exercise (ex1.c)
• Execute the code given in previous slide
• Make some changes to initialize the data in a different manner and
then compare the results (print receive buffer at different MPI
Processes)
• Ans: lab2.1/ex1.c

6. MPI_Gather

7. MPI_Gather

9. Exercise (ex2.c)
• Execute the code given in previous slide
• Make changes in the send buffer and test the code if it works
correctly to gather elements of the send buffer in different MPI
processes.

10. MPI_Scatter
• MPI_Scatter is a collective routine that is very similar to MPI_Bcast
• MPI_Scatter involves a designated root process sending data to all
processes in a communicator.
• The primary difference between MPI_Bcast and MPI_Scatter is small
but important.
• MPI_Bcast sends the same piece of data to all processes while
MPI_Scatter sends chunks of an array to different processes.

11. MPI_Scatter

13. Reference
• https://mpitutorial.com/tutorials/