0
Slidecast: What is [Open] MPI? Jeff Squyres May 2008
What is MPI? <ul><li>Message Passing Interface </li></ul><ul><ul><li>“De facto” standard </li></ul></ul><ul><ul><li>Not an...
MPI Forum <ul><li>Published MPI-1 spec in 1994 </li></ul><ul><li>Published MPI-2 spec in 1996 </li></ul><ul><ul><li>Additi...
What is MPI? <ul><li>Software implementations of spec </li></ul><ul><ul><li>Mostly host-side software </li></ul></ul><ul><...
MPI High-Level View User application MPI API Operating System
What is MPI? <ul><li>Intended to deliver very high performance </li></ul><ul><ul><li>Low latency, high bandwidth </li></ul...
MPI Implementations <ul><li>Many exist / are available for customers </li></ul><ul><ul><li>Vendors: HP MPI, Intel MPI, Sca...
Why So Many MPI’s? <ul><li>A complicated question… </li></ul><ul><ul><li>Some aim to make money (closed source) </li></ul>...
Target Audience <ul><li>Scientists and engineers </li></ul><ul><ul><li>Don’t know or care how network works </li></ul></ul...
Parallel Computing <ul><li>Use 10’s, 100’s, 1000’s of processors </li></ul><ul><ul><li>When the computation is too big for...
MPI Abstracts the Network <ul><li>Sockets?  Shared memory?  Ethernet?  InfiniBand?  …something else? </li></ul><ul><ul><li...
MPI High-Level View User application MPI API Operating System
Example: 4 Servers Server User application MPI API Operating System Server User application MPI API Operating System Serve...
Example: 2 Servers Server User application MPI API Operating System User application MPI API Server User application MPI A...
Example: 1 Server Server User application MPI API Operating System User application MPI API User application MPI API User ...
Runtime <ul><li>MPI implementations also include a runtime environment </li></ul><ul><ul><li>Need to start processes on mu...
Trivial MPI Application <ul><li>int rank, size, message = -1, tag = 11111; </li></ul><ul><li>MPI_Init(NULL, NULL);  /* Sta...
Trivial MPI Application Process 0 to=1 from=3 Process 2 to=3 from=1 Process 1 to=2 from=0 Process 3 to=0 from=2 MPI_Send(…...
Open MPI <ul><li>YAMPI (yet another MPI) </li></ul><ul><ul><li>… but not really </li></ul></ul><ul><ul><li>Replaces severa...
Open MPI <ul><li>It’s two words! </li></ul><ul><ul><li>Open MPI </li></ul></ul><ul><ul><li>NOT  “OpenMPI” </li></ul></ul><...
Open MPI <ul><li>Fundamentally based on plugins </li></ul><ul><ul><li>A.k.a. “components” or “modules” </li></ul></ul><ul>...
Plugin High-Level View User application MPI API Modular Component Architecture (MCA) Framework Comp. Comp. Comp. … Framewo...
Resources <ul><li>MPI Forum </li></ul><ul><ul><li>http://www.mpi-forum.org/   </li></ul></ul><ul><li>Open MPI </li></ul><u...
 
Upcoming SlideShare
Loading in...5
×

What is [Open] MPI?

3,090

Published on

Description of the Message Passing Interface (MPI) and a brief overview of of the Open MPI open source software project.

Published in: Education, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,090
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
81
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "What is [Open] MPI?"

  1. 1. Slidecast: What is [Open] MPI? Jeff Squyres May 2008
  2. 2. What is MPI? <ul><li>Message Passing Interface </li></ul><ul><ul><li>“De facto” standard </li></ul></ul><ul><ul><li>Not an “official” standard (IEEE, IETF, …) </li></ul></ul><ul><li>Written and ratified by the MPI Forum </li></ul><ul><ul><li>Body of academic, research, and industry representatives </li></ul></ul><ul><li>MPI is two spec documents: </li></ul><ul><ul><li>MPI-1 and MPI-2 </li></ul></ul><ul><ul><li>Specified interfaces in C, C++, Fortran 77/90 </li></ul></ul>
  3. 3. MPI Forum <ul><li>Published MPI-1 spec in 1994 </li></ul><ul><li>Published MPI-2 spec in 1996 </li></ul><ul><ul><li>Additions to MPI-1 </li></ul></ul><ul><li>Recently reconvened (Jan 2008) </li></ul><ul><ul><li>Working on MPI-2.1 (small bug fixes) </li></ul></ul><ul><ul><ul><li>Will issue a single document for MPI 1+2 </li></ul></ul></ul><ul><ul><li>Also working on MPI-2.2 (bigger bug fixes) </li></ul></ul><ul><ul><li>Also working on MPI-3 (entirely new stuff) </li></ul></ul>
  4. 4. What is MPI? <ul><li>Software implementations of spec </li></ul><ul><ul><li>Mostly host-side software </li></ul></ul><ul><li>“ Middleware” </li></ul><ul><ul><li>Sits between the application and network </li></ul></ul><ul><ul><li>Simplifies network activity to the application </li></ul></ul><ul><li>Source code portability </li></ul><ul><ul><li>Run apps on commodity clusters and “big iron” supercomputers </li></ul></ul>
  5. 5. MPI High-Level View User application MPI API Operating System
  6. 6. What is MPI? <ul><li>Intended to deliver very high performance </li></ul><ul><ul><li>Low latency, high bandwidth </li></ul></ul><ul><li>Examples </li></ul><ul><ul><li>2 servers + switch, user-level processes </li></ul></ul><ul><ul><li>DDR InfiniBand </li></ul></ul><ul><ul><ul><li>~1-2  s half-round trip 0-byte ping pong </li></ul></ul></ul><ul><ul><ul><li>~14Gbps bandwidth for large messages </li></ul></ul></ul><ul><ul><li>10Gbps Ethernet </li></ul></ul><ul><ul><ul><li>~5-7  s half-round trip 0-byte ping pong </li></ul></ul></ul><ul><ul><ul><li>~10Gbps bandwidth for large messages </li></ul></ul></ul>
  7. 7. MPI Implementations <ul><li>Many exist / are available for customers </li></ul><ul><ul><li>Vendors: HP MPI, Intel MPI, Scali MPI </li></ul></ul><ul><ul><ul><li>Have their own support channels </li></ul></ul></ul><ul><ul><li>Open source: Open MPI, MPICH[2], … </li></ul></ul><ul><ul><ul><li>Rely on open source community for support </li></ul></ul></ul><ul><ul><ul><li>But also have some vendor support </li></ul></ul></ul><ul><li>Various research-quality implementations </li></ul><ul><ul><li>Proof-of-concept </li></ul></ul><ul><ul><li>Not usually intended for production usage </li></ul></ul>
  8. 8. Why So Many MPI’s? <ul><li>A complicated question… </li></ul><ul><ul><li>Some aim to make money (closed source) </li></ul></ul><ul><ul><li>Some targeted at specific platforms </li></ul></ul><ul><ul><li>Others aimed at research (open source) </li></ul></ul><ul><ul><li>History and politics also involved (yuck) </li></ul></ul><ul><li>Open MPI is a fascinating blend of research and industry </li></ul>
  9. 9. Target Audience <ul><li>Scientists and engineers </li></ul><ul><ul><li>Don’t know or care how network works </li></ul></ul><ul><ul><li>Not computer scientists </li></ul></ul><ul><ul><li>Sometimes not even [very good] programmers </li></ul></ul><ul><li>Parallel computing </li></ul><ul><ul><li>Using tens, hundreds, or thousands of servers in a single computational program </li></ul></ul><ul><ul><li>Intended for high-performance computing </li></ul></ul>
  10. 10. Parallel Computing <ul><li>Use 10’s, 100’s, 1000’s of processors </li></ul><ul><ul><li>When the computation is too big for one server </li></ul></ul><ul><li>Spread the job across multiple servers </li></ul><ul><ul><li>Individual user processes running in concert </li></ul></ul><ul><ul><li>Acting together as a single application </li></ul></ul><ul><li>More RAM </li></ul><ul><li>More processing power </li></ul><ul><li>Divide and conquer </li></ul>
  11. 11. MPI Abstracts the Network <ul><li>Sockets? Shared memory? Ethernet? InfiniBand? …something else? </li></ul><ul><ul><li>Doesn’t matter </li></ul></ul><ul><li>Application calls MPI_SEND / MPI_RECV </li></ul><ul><ul><li>The Right magic happens </li></ul></ul><ul><li>Connections are made automatically </li></ul><ul><ul><li>Sockets (IP address/port) </li></ul></ul><ul><ul><li>Shared memory (e.g., mmap file) </li></ul></ul><ul><ul><li>InfiniBand (queue pair setup) </li></ul></ul>
  12. 12. MPI High-Level View User application MPI API Operating System
  13. 13. Example: 4 Servers Server User application MPI API Operating System Server User application MPI API Operating System Server User application MPI API Operating System Server User application MPI API Operating System Network
  14. 14. Example: 2 Servers Server User application MPI API Operating System User application MPI API Server User application MPI API User application MPI API Operating System Network
  15. 15. Example: 1 Server Server User application MPI API Operating System User application MPI API User application MPI API User application MPI API
  16. 16. Runtime <ul><li>MPI implementations also include a runtime environment </li></ul><ul><ul><li>Need to start processes on multiple servers simultaneously </li></ul></ul><ul><ul><li>Typically requires some user-level setup </li></ul></ul><ul><ul><li>Common source of errors </li></ul></ul>
  17. 17. Trivial MPI Application <ul><li>int rank, size, message = -1, tag = 11111; </li></ul><ul><li>MPI_Init(NULL, NULL); /* Startup */ </li></ul><ul><li>MPI_Comm_rank(…, &rank); /* Who am I? */ </li></ul><ul><li>MPI_Comm_size(…, &size); /* How many peers do I have? */ </li></ul><ul><li>to = (rank + 1) % size; </li></ul><ul><li>from = (rank + size - 1) % size; </li></ul><ul><li>/* Send a trivial message around in a ring */ </li></ul><ul><li>if (0 == rank) { </li></ul><ul><li>message = 42; </li></ul><ul><li>MPI_Send(&message, 1, MPI_INT, to, tag, …); </li></ul><ul><li>MPI_Recv(&message, 1, MPI_INT, from, tag, …); </li></ul><ul><li>} else { </li></ul><ul><li>MPI_Recv(&message, 1, MPI_INT, from, tag, …); </li></ul><ul><li>MPI_Send(&message, 1, MPI_INT, to, tag, …); </li></ul><ul><li>} </li></ul><ul><li>MPI_Finalize(); </li></ul>
  18. 18. Trivial MPI Application Process 0 to=1 from=3 Process 2 to=3 from=1 Process 1 to=2 from=0 Process 3 to=0 from=2 MPI_Send(…) MPI_Send(…) MPI_Send(…) MPI_Send(…) MPI_Recv(…) MPI_Recv(…) MPI_Recv(…) MPI_Recv(…)
  19. 19. Open MPI <ul><li>YAMPI (yet another MPI) </li></ul><ul><ul><li>… but not really </li></ul></ul><ul><ul><li>Replaces several prior MPI’s </li></ul></ul><ul><li>Collaborate = great MPI implementation </li></ul><ul><ul><li>What a concept! </li></ul></ul><ul><ul><li>Lots of “MPI-smart” people out there </li></ul></ul><ul><li>Open source project </li></ul><ul><ul><li>Influenced by both research and industry </li></ul></ul>Research / academia Industry
  20. 20. Open MPI <ul><li>It’s two words! </li></ul><ul><ul><li>Open MPI </li></ul></ul><ul><ul><li>NOT “OpenMPI” </li></ul></ul><ul><li>Frequently abbreviated “OMPI” </li></ul><ul><ul><li>Pronounced “oom-pee” </li></ul></ul>
  21. 21. Open MPI <ul><li>Fundamentally based on plugins </li></ul><ul><ul><li>A.k.a. “components” or “modules” </li></ul></ul><ul><li>Plugins for everything </li></ul><ul><ul><li>Back-end resource manager </li></ul></ul><ul><ul><li>Back-end network </li></ul></ul><ul><ul><li>Back-end checkpointer </li></ul></ul><ul><ul><li>… etc. </li></ul></ul><ul><ul><li>Currently ~30 types of plugins in Open MPI </li></ul></ul><ul><li>Recurring theme: run-time decisions </li></ul>
  22. 22. Plugin High-Level View User application MPI API Modular Component Architecture (MCA) Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … Framework Comp. Comp. Comp. … …
  23. 23. Resources <ul><li>MPI Forum </li></ul><ul><ul><li>http://www.mpi-forum.org/ </li></ul></ul><ul><li>Open MPI </li></ul><ul><ul><li>General web site: http://www.open-mpi.org/ </li></ul></ul><ul><ul><li>FAQ: http://www.open-mpi.org/faq/ </li></ul></ul><ul><li>Magazine columns about MPI </li></ul><ul><ul><li>http://cw.squyres.com/ </li></ul></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×