Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MPI Presentation

4,401 views

Published on

Published in: Technology, Education
  • Be the first to comment

MPI Presentation

  1. 1. Parallel Programming using Message Passing Interface (MPI) metu-ceng ts@TayfunSen.com 11/05/08 Parallel Programming Using MPI 1 25 April 2008
  2. 2. Outline • What is MPI? • MPI Implementations • OpenMPI • MPI • References • Q&A 11/05/08 Parallel Programming Using MPI 2/26
  3. 3. What is MPI? • A standard with many implementations (lam-mpi and mpich, evolving into OpenMPI and MVAPICH). • message passing API • Library for programming clusters • Needs to be high performing, scalable, portable ... 11/05/08 Parallel Programming Using MPI 3/26
  4. 4. MPI Implementations • Is it up for the challenge? MPI does not have many alternatives (what about OpenMP, map-reduce etc?). • Many implementations out there. • The programming interface is all the same. But underlying implementations and what they support in terms of connectivity, fault tolerance etc. differ. • On ceng-hpc, both MVAPICH and OpenMPI is installed. 11/05/08 Parallel Programming Using MPI 4/26
  5. 5. OpenMPI • We'll use OpenMPI for this presentation • It's open source, MPI2 compliant, portable, has fault tolerance, combines best practices of number of other MPI implementations. • To install it, for example on Debian/Ubuntu type: # apt-get install openmpi-bin libopenmpi-dev openmpi-doc 11/05/08 Parallel Programming Using MPI 5/26
  6. 6. MPI – General Information • Functions start with MPI_* to differ from application • MPI has defined its own data types to abstract machine dependent implementations (MPI_CHAR, MPI_INT, MPI_BYTE etc.) 11/05/08 Parallel Programming Using MPI 6/26
  7. 7. MPI - API and other stuff • Housekeeping (initialization, termination, header file) • Two types of communication: Point- to-point and collective communication • Communicators 11/05/08 Parallel Programming Using MPI 7/26
  8. 8. Housekeeping • You include the header mpi.h • Initialize using MPI_Init(&argc, &argv) and end MPI using MPI_Finalize() • Demo time, “hello world!” using MPI 11/05/08 Parallel Programming Using MPI 8/26
  9. 9. Point-to-point communication • Related definitions – source, destination, communicator, tag, buffer, data type, count • man MPI_Send, MPI_Recv int MPI_Send(void *buf, int count, MPI_Datatype datatype, int dest,int tag, MPI_Comm comm) • Blocking send, that is the processor doesn't do anything until the message is sent 11/05/08 Parallel Programming Using MPI 9/26
  10. 10. P2P Communication (cont.) • int MPI_Recv(void *buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status *status) • Source, tag, communicator has to be correct for the message to be received • Demo time – simple send • One last thing, you can use wildcards in place of source and tag. MPI_ANY_SOURCE and MPI_ANY_TAG 11/05/08 Parallel Programming Using MPI 10/26
  11. 11. P2P Communication (cont.) • The receiver actually does not know how much data it received. He takes a guess and tries to get the most. • To be sure of how much received, one can use: • int MPI_Get_count(MPI_Status *status, MPI_Datatype dtype, int *count); • Demo time – change simple send to check the received message size. 11/05/08 Parallel Programming Using MPI 11/26
  12. 12. P2P Communication (cont.) • For a receive operation, communication ends when the message is copied to the local variables. • For a send operation, communication is completed when the message is transferred to MPI for sending. (so that the buffer can be recycled) • Blocked operations continue when the communication has been completed • Beware – There are some intricacies Check [2] for more information. 11/05/08 Parallel Programming Using MPI 12/26
  13. 13. P2P Communication (cont.) • For blocking communications, deadlock is a possibility: if( myrank == 0 ) { /* Receive, then send a message */ MPI_Recv( b, 100, MPI_DOUBLE, 1, 19, MPI_COMM_WORLD, &status ); MPI_Send( a, 100, MPI_DOUBLE, 1, 17, MPI_COMM_WORLD ); } else if( myrank == 1 ) { /* Receive, then send a message */ MPI_Recv( b, 100, MPI_DOUBLE, 0, 17, MPI_COMM_WORLD, &status ); MPI_Send( a, 100, MPI_DOUBLE, 0, 19, MPI_COMM_WORLD ); } • How to remove the deadlock? 11/05/08 Parallel Programming Using MPI 13/26
  14. 14. P2P Communication (cont.) • When non-blocking communication is used, program continues its execution • A program can send a blocking send and the receiver may use non-blocking receive or vice versa. • Very similar function calls int MPI_Isend(void *buf, int count, MPI_Datatype dtype, int dest, int tag, MPI_Comm comm, MPI_Request *request); • Request handle can be used later eg. MPI_Wait, MPI_Test ... 11/05/08 Parallel Programming Using MPI 14/26
  15. 15. P2P Communication (cont.) • Demo time – non_blocking • There are other modes of sending (but not receiving!) check out the documentation for synchronous, buffered and ready mode send in addition to standard one we have seen here. 11/05/08 Parallel Programming Using MPI 15/26
  16. 16. P2P Communication (cont.) • Keep in mind that each send/receive is costly – try to piggyback • You can send different data types at the same time – eg. Integers, floats, characters, doubles... using MPI_Pack. This function gives you an intermediate buffer which you will send. • int MPI_Pack(void *inbuf, int incount, MPI_Datatype datatype, void *outbuf, int outsize, int *position, MPI_Comm comm) • MPI_Send(buffer, count, MPI_PACKED, dest, tag, MPI_COMM_WORLD); 11/05/08 Parallel Programming Using MPI 16/26
  17. 17. P2P Communication (cont.) • You can also send your own structs (user defined types). See the documentation 11/05/08 Parallel Programming Using MPI 17/26
  18. 18. Collective Communication • Works like point to point except you send to all other processors • MPI_Barrier(comm), blocks until each processor calls this. Synchronizes everyone. • Broadcast operation MPI_Bcast copies the data value in one processor to others. • Demo time - bcast_example 11/05/08 Parallel Programming Using MPI 18/26
  19. 19. Collective Communication • MPI_Reduce collects data from other processors, operates on them and returns a single value • reduction operation is performed • Demo time – reduce_op example • There are MPI defined reduce operations but you can define your own 11/05/08 Parallel Programming Using MPI 19/26
  20. 20. Collective Communication - MPI_Gather • gather and scatter operations • Like what their name implies • Gather – like every process sending their send buffer and root process receiving • Demo time - gather_example 11/05/08 Parallel Programming Using MPI 20/26
  21. 21. Collective Communication - MPI_Scatter • Similar to MPI_Gather but here data is sent from root to other processors • Like gather, you can accomplish it by having root calling MPI_Send repeatedly and others calling MPI_Recv • Demo time – scatter_example 11/05/08 Parallel Programming Using MPI 21/26
  22. 22. Collective Communication – More functionality • Many more functions to lift hard work from you. • MPI_Allreduce, MPI_Gatherv, MPI_Scan, MPI_Reduce_Scatter ... • Check out the API documentation • Manual files are your best friend. 11/05/08 Parallel Programming Using MPI 22/26
  23. 23. Communicators • Communicators group processors • Basic communicator MPI_COMM_WORLD defined for all processors • You can create your own communicators to group processors. Thus you can send messages to only a subset of all processors. 11/05/08 Parallel Programming Using MPI 23/26
  24. 24. More Advanced Stuff • Parallel I/O – when one node is used for reading from disk it is slow. You can have each node use its local disk. • One sided communications – Remote memory access • Both are MPI-2 capabilities. Check your MPI implementation to see how much it is implemented. 11/05/08 Parallel Programming Using MPI 24/26
  25. 25. References [1] Wikipedia articles in general, including but not limited to: http://en.wikipedia.org/wiki/Message_Passing_Interface [2] An excellent guide at NCSA (National Center for Supercomputing Applications): http://webct.ncsa.uiuc.edu:8900/public/MPI/ [3] OpenMPI Official Web site: http://www.open-mpi.org/ 11/05/08 Parallel Programming Using MPI 25/26
  26. 26. The End Thanks For Your Time. Any Questions ? 11/05/08 Parallel Programming Using MPI 26/26

×