This document discusses the progression of multi-core processors from dual-core to octa-core CPUs over recent years. It then provides an overview of parallel programming with OpenMP, including its advantages, execution model, data sharing rules, and examples of parallelizing a for loop and calculating Pi in parallel.

1. Multi-Processor
computing with
OpenMP
T: 051 401 9700 coetzeesj@ufs.ac.za http://www.ufs.ac.za

2. Progress on Multi-Core Processors
Dual Core
●April 16, 2005 - Intel releases Pentium Extreme Edition 840
●June 5, 2005 - AMD releases Athlon 64 X2
Quad Core
●November 19, 2007 - AMD releases Phenom X4
●November 17, 2008 - Intel releases Core i7
Hex Core
●April 27, 2010 - AMD releases Phenom II X6
Octa Core
●Yesterday (October 12, 2011) - AMD releases FX-8150
Bulldozer

3. Why?
●Previously multiple processors was only available to high
end servers
●Difficult to scale to high clock speeds with current transistor
technologies
●Better manufacturing processes creates smaller transistors

4. Limitations
●Developing software that scales across multiple processors
is difficult to develop.
●Memory access should be governed to protect data that is
being accessed by different parts of the program at the same
point in time.

5. What is Parallelization?
●"Something" is parallel if there is a certain level of
independence in the order of operations
●Parallelization is an optimization technique to reduce the
execution time of an application or part thereof.

6. Scalability
The more independent the parts of the application the more
scalable the application become. Applications that scale almost
linearly is called "embarrassingly parallel" applications.
Amdahl's Law
Assume our program has a parallel fraction "f"
This implies the execution time T(1) = f*T(1) + (1-f)*T(1)
On P processors T(P) = (f/P)*T(1) + (1-f)*T(1)
Amdahl's Law: S(P) = 1/(f/P + 1-f)

7. Amdahl's Law

8. Parallel Programming
Distributed Memory:
●Sockets
●PVM - Parallel Virtual
Machine (obsolete)
●MPI - Message Passing
Interface
Shared Memory:
●Posix Threads
●OpenMP
●Automatic Parallelization
(Compiler optimizations)

9. OpenMP
●De-facto standard Application Programming Interface to write
shared memory parallel applications in C, C++, and Fortran
●Consists of:
○Compiler directives
○Run time routines
○Environment variables
●Specification maintained by the OpenMP Architecture
Review Board
●Release dates:
○Version 1.0 - October 1997
○Version 2.0 - November 2000
○Version 3.0 - May 2008

10. Advantages of OpenMP
●Good performance
●Mature standard
●Supported by all major compilers
○GNU Compiler Collection (GCC)
○Intel Compiler (ICC)
○Microsoft Visual C++ (2005 and up)
○Portland Group Compiler
●Requires little programming effort and change to code.
●Allows the program to be parallelized incrementally.

11. OpenMP Execution Model
OpenMP uses fork and join model. Application runs in serial
until execution hits an area of application that can run parallel.
In the parallel region, openMP creates worker threads to
execute concurrently with master thread. At the end of the
parallel section, openMP synchronise the data of the threads,
and execution continues on master thread.

12. Data-sharing
●In OpenMP data needs to be "labeled"
○Shared
■All threads can read and write the data, unless
protected through a specific OpenMP construct
■Changes made a visible to all threads
■Not necessarily immediately, unless forced through a
specific OpenMP construct
○Private
■Data only available to thread
■Changes only visible to thread owning the data

13. OpenMP example
For-loop with independent
iterations
For-loop parallelized using
OpenMP
for (int i=0; i < n;
i++)
c[i] = a[i] + b[i];
#pragma omp parallel
for
for (int i=0; i < n;
i++)
c[i] = a[i] + b[i];

14. OpenMP computing Pi
Currently the more
preferred solutions for
calculation pi is
numerical integration of

15. Monte Carlo Approach
By using a pseudo random number
generator you can calculate pi by
determining the percentage of darts
that are inside the circle. To make
calculations simpler, we only use
the top right quadrant, and multiply
our findings by 4.

16. T: 051 401 9700 coetzeesj@ufs.ac.za http://www.ufs.ac.za