A simple help document on "How to start Parallel programming using C++ using MSMPI and Visual Studio Community in Windows 10"

1. PARALLEL PROGRAMMING
in Win10 using
MS-MPI & VS
Praveen Kumar Ch
prav

2. We do this in Two Stages
• Stage 1: Setup Requirements
1. Understanding the Software Requirements
2. IDE & Compiler : Microsoft Visual Studio
3. Microsoft MPI
• Stage 2: MPI Programming
1. Creating Project in Visual Studio
2. Linking MS-MPI with Project for Parallel programming
3. Writing, Building and Launching “Hello World” Parallel Program
prav

3. prav
Create
Project
Configure MS
MPI in VS2019
Coding in
Parallel
Understand
Requirements
Install IDE
VS2019
Install
MS MPI
Stage 1
Stage 2

4. Stage 1: Setup Requirements
1. Understand the What is Needed…
2. Install Visual Studio..
3. Install MS-MPI…
If you are not interested in understanding the requirements
&&
If you want to directly start installing
Go to Slide 12
prav

5. Stage 1. Step 1:
To understand Software setup requirements
Lets see a typical Compilation Process…
( in Next Slide)
prav

6. Source file
(filename.c)
Intermediate code
(filename.i)
Assembly file
(filename.s)
Object file
(filename.o)
Executable
(filename.exe)
Pre-processor
Compiler
Assembler
Linker
The Code is
written in this
file
The Executable
File that we
run
MS Compiler
The build Process
Header files
(.h)
Library
(.o, .lib)
Include
Link
prav
Don’t bother
about this for
now
This is the
Compiler

7. prav
Now lets see what happens with the Executable file
I mean when we run the executable file…

8. OS
Loads the Instructions
into the processor
stack
prav
Executable File
(filename.exe)
Loader
CPU
RAM
CHIP
CORE1 CORE2
CORE3 CORE4
CORE5 CORE6
CORE7 CORE8
L3 CACHE
L2 CACHE
I/O DEVICES
DISPLAY
KEYBOARD
MOUSE
HARD DISK
The Hardware
Hard Disk
The .exe is stored here
OS
Run time
When we run / click
on the executable file

9. So to run a c program, We need…
• An editor to write the code
• A compiler to Build an Executable
• A Launcher to Load and Run the Executable
prav
• A Multicore Machine
• Operating System
An Integrated Development
Environment (IDE) is like
one stop shop that makes
this stuff easy

10. But we are trying to do some
Parallel Programming using MPI
So…
We also need an MPI implementation
For better understanding, See next slide
prav

11. TH
CPU
prav
IDE : VISUAL STUDIO
WINDOWS
SDK
MS VC++
Compiler
OS
Loader
MS MPI
(INC ,
LIB)
BUILDER
(Compiles and
Builds an
executable file
.exe)
LAUNCH
(Runs the
executable
file)
EDITOR
(The code is
written and
edited here)
.exe
file
.cpp
Source file
RAM
CHIP
CORE1 CORE2
CORE3 CORE4
CORE5 CORE6
CORE7 CORE8
L3 CACHE
L2 CACHE
I/O DEVICES
DISPLAY
KEYBOARD
MOUSE
HARD DISK
The Software – OS & Apps The Hardware

12. TH
CPU
prav
OS
Loader
RAM
CHIP
CORE1 CORE2
CORE3 CORE4
CORE5 CORE6
CORE7 CORE8
L3 CACHE
L2 CACHE
I/O DEVICES
DISPLAY
KEYBOARD
MOUSE
HARD DISK
The Software – OS & Apps The Hardware
We have a PC with a multi
core processor
We have
windows 10 OS
What We already
Have ????

13. Finally the requirements for Parallel
Programming…
The Hardware : A Multicore Machine
The OS : Windows 10
prav
oIDE is required : Visual Studio
oMPI : MS MPI
We should
be having
these
We need to
install and
set up this

14. Stage 1: Step 2 : Download and Install
the Visual Studio Community
Its free… GO here… To this Link…
https://visualstudio.microsoft.com/thank-you-downloading-visual-
studio/?sku=community&rel=16
Download starts automatically…
Download and install … Instructions available there…
prav

15. prav

16. If link doesn't work…
Just Google “Visual Studio”
Find and open Official website : https://visualstudio.microsoft.com/
Help Your self to download and install Latest Community edition
There are many YouTube videos that show
“How to install Visual Studio”
prav

17. https://visualstudio.microsoft.com/
prav
Download
Community
from here

18. After Download and installation
The resources we got now, will be
like… See next slide…
prav

19. TH
CPU
prav
IDE : VISUAL STUDIO
WINDOWS
SDK
MS VC++
Compiler
OS
Loader
BUILDER
(Compiles and
Builds an
executable file
.exe)
LAUNCH
(Runs the
executable
file)
EDITOR
(The code is
written and
edited here)
.exe
file
.cpp
Source file
RAM
CHIP
CORE1 CORE2
CORE3 CORE4
CORE5 CORE6
CORE7 CORE8
L3 CACHE
L2 CACHE
I/O DEVICES
DISPLAY
KEYBOARD
MOUSE
HARD DISK
The Software – OS & Apps The Hardware
We
added
this
now

20. Downloading and Installing MS MPI
Its free… GO here… To this Link…
https://www.microsoft.com/en-us/download/details.aspx?id=100593
prav
Check System Requirements
Follow Installation Instruction

21. prav

22. prav
Download both
msmpisetup.exe
&
msmpisdk.msi

23. If link doesn’t work…
Google “Microsoft MPI”
There is some YouTube content on
“How to Download and Install Microsoft – MPI”
Find and Open Official web page :
https://docs.microsoft.com/en-us/message-passing-
interface/microsoft-mpi
prav

24. prav
Download
from here

25. TH
CPU
prav
IDE : VISUAL STUDIO
WINDOWS
SDK
MS VC++
Compiler
OS
Loader
MS MPI
(INC ,
LIB)
BUILDER
(Compiles and
Builds an
executable file
.exe)
LAUNCH
(Runs the
executable
file)
EDITOR
(The code is
written and
edited here)
.exe
file
.cpp
Source file
RAM
CHIP
CORE1 CORE2
CORE3 CORE4
CORE5 CORE6
CORE7 CORE8
L3 CACHE
L2 CACHE
I/O DEVICES
DISPLAY
KEYBOARD
MOUSE
HARD DISK
The Software – OS & Apps The Hardware
We added this now
This Connection is for Including
header files (mpi.h) and linking
Library (msmpi.inc, msmpi.lib).
We will establish this link when
we are making a new MPI Project
We added this now
This Link is for running a
program in parallel using
msmpi.exe
It is established while installing
MS MPI

26. Ok Now we got Everything Installed …
What Next… Stage 2…
1. Open a new Visual Studio Project
2. Set up MS-MPI with VS-Project: Include files, Libraries and
dependencies
3. Write the First parallel MPI Program : To Print Hello world in parallel
using multiple processes
prav
https://www.youtube.com/watch?time_continue=518&
v=IW3SKDM_yEs&feature=emb_title
You can also learn
how to do that stuff
from this YouTube
Video and PDF file… https://www.math.ucla.edu/~mputhawala/PPT.pdf

27. Stage2 . Step 1 : Open Visual Studio and Create a new Project
prav
Select
Console App
Check these
Options

28. prav
Give a Name For
the project
Choose a
Location to save
the Project

29. prav
New Project Opens Like this…

30. Stage 2 : Step 2
Setup MS- MPI with Visual Studio Compiler
How??
By changing the Project Properties
But first check How good is MSMPI Installation …
By Using Command Prompt…
prav

31. prav
To check if MS –MPI is
properly installed and
default path is set
Open Command
Prompt
Give the command
“mpiexec”
If,
the command is
recognized and its
usage is displayed
Then,
MS-MPI is properly
installed and all is good

32. Check for MS-MPI Environment Variables
Open Command Prompt and type command “set MSMPI” and hit
return key
prav
These Variables with the respective
path strings should be displayed
These Variables contain location path for MPI,
bin, include directories and library directories.
They will be used to set up MPI in VS Project

33. If it looks all good then…,
Lets Go back to Visual Studio to set up MS MPI….
prav

34. prav
Click Here to
Change Project
Properties

35. prav
Add
“$(MSMPI_INC)”
in this blank
Expand and
Open these
Properties
C/C++ :General
We are actually
setting
Additional Include
Directories to
the Path of MS-MPI
Include files
directory.
Where “mpi.h” and
other related header
files are located.Note,
$(MSMPI_INC) = C:Program
Files (x86)Microsoft
SDKsMPIInclude
This window
Opens up

36. prav
Add
“msmpi.lib;”
to the line
• Add it before “%”
• Don’t forget “;”
Expand and
Open these
Properties
Linker: All
Options
We are actually
adding the msmpi
library to Link

37. prav
Expand and
Open these
Properties
Linker: All
Options
For x64 Platform,
Add
“$(MSMPI_LIB64)”
in the blank
For x32 Platform,
Add
“$(MSMPI_LIB32)”
in the blank
We are adding
the path of
MSMPI library
Directory
Where
msmpi.lib for
corresponding
platform exist

38. prav
Replace the command with
“$(MSMPI_BIN)mpiexec”
Replace the command arguments
with
“-n 8 $(TargetPath)”
We are actually setting the debugger to run the project in
parallel with 8 processes,
by using the following command
$(MSMPI_BIN)mpiexec.exe –n 8 $(TargetPath)
Replace ‘8’ with
‘16’ if you want to
run with 16
processes
Expand and
Open these
Properties
Debugging
Click Ok button
And close the
property
window

39. prav
Click Ok button
And close the
property
window

40. Stage 2. Step 3 : Code to Run in Parallel
prav
MPI Parallel Programming key fund
• The complete code is distributed to all Processors.
• We design the program so that it identifies the processor in which it is running
and does tasks which are relevant to it.
• Rank is a number used to identify processor.
• MPI functions are used
• MPI_Comm_rank() is used to identify the rank of current running processor.
• MPI_Comm_size() is used to identify the total number of processors.
• MPI Functions are used for communication / message passing between
processors during runtime
• MPI_Send() & MPI_Recv() to send and receive msgs between processes

41. prav
1. Replace the content in
source file
“Hello_Parallel.cpp” with
the code shown here
and save
2. Click this Green Arrow head to Build and Launch after editing and saving the code

42. prav
Note :
The code run
by processer 0
prints an
additional line
because of if
conditional
statement
Every
processor
prints Hello
message by
identifying its
number (rank)
Output after Running the code

43. prav
ANOTHER PROGRAM
Try this code,
It takes two numbers and
perform various operations
based on the processor in
which this code is running

44. prav
#include <mpi.h>
#include <stdio.h>
int main(int argc, char** argv[]) {
// Defining two integer numbers
int a = 10;
int b = 20;
// Initialize the MPI environment
MPI_Init(NULL, NULL);
// Get the total number of processes running this code
int nProc;
MPI_Comm_size(MPI_COMM_WORLD, &nProc);
// Get the rank of this process in which this code is running
int iProc;
MPI_Comm_rank(MPI_COMM_WORLD, &iProc);
// Print a text corresponding to this process
printf("nHello Parallel world from processor %d out of %d processors", iProc, nProc);
The Code contd..,

45. prav
// Calculating third number C as a result of various operations with a and b
int c;
if (iProc == 0) { // if Processor rank is 0 then c=a+b
c = a + b;
printf("nc=(a+b)=%d", c);
}
if (iProc == 1) { // if Processor rank is 1 then c=a-b
c = a - b;
printf("nc=(a-b)=%d", c);
}
if (iProc == 2) { // if Processor rank is 2 then c=a*b
c = a * b;
printf("nc=(a*b)=%d", c);
}
if (iProc == 3) { // if Processor rank is 3 then c=a/b
c = a / b;
printf("nc=(a/b)=%d", c);
}
if (iProc>3){
printf("nThis processor is not performing any operation on a and b");
}
// Finalize the MPI environment.
MPI_Finalize();
}
The Code Ends

46. prav
The output after running
the code

47. Additional references
https://mpitutorial.com/tutorials/
https://docs.microsoft.com/en-us/archive/blogs/windowshpc/how-to-
compile-and-run-a-simple-ms-mpi-program
prav
The End…