This document provides an overview of multiprocessing programming and OpenMP. It discusses symmetric multiprocessing using shared memory, as well as message passing interface programming. It then covers OpenMP programming models including parallel sections, clauses like private and shared to control data sharing, and synchronization using critical sections. The document aims to eliminate synchronization and dependency through OpenMP directives to generate multi-threaded code supported by compilers.

2. AgendaBackgroundMultiprocessing ProgrammingPreferenceProgramming ModelParallel SectionClausesSynchronization

3. BackgroundParallel has no semantic impactConcurrency has a substantial semantic impact

4. BackgroundSymmetric Multi Processor(SMP)Shared Memory Multiprocessing ProgrammingOpenMPMassively Parallel Processor(MPP)Message Passing Interface ProgrammingMPI

5. Background

6. Multiprocessing Programming

7. Multiprocessing ProgrammingGeneral Threading ModelMain thread executes the program’s main functionMain thread creates subthreads, witch execute other functions in the program.Subthreads can also create additional threads.

8. Multiprocessing ProgrammingBenefitsIncreased performance and better resource utilizationEven on single processor systems – for hiding latency and increasing throughputIPC thread shared memory is more efficientRiskIncreases complexity of the applicationDifficult to debug (data races, deadlocks, etc.)

9. OpenMP Preference (later VS2005)

10. OpenMP Preference (later GCC 4.2)

11. Programming ModelGenerate multi-thread code by referencing directive code Compiler is required to support openMPTo eliminate synchronization and dependency

12. Programming Model

13. Parallel Section (for)#pragmaomp for [clause [clause …] ] { for loop}

14. Parallel Section (sections)#pragmaomp sections [clause [clause …] ] { [#progmaomp section] structured code block}

15. Clausesprivate(var1, …) to block sharing selected values with threadsshared(var1, …)to share selected values with threads schedule(type [, chunk_size])to divide work with threads for equalitystaticdynamic

16. Synchronization (critical section)Declaring critical section in parallel sectionRunning only single thread at moment#pragmaomp critical [(name)]{ structured code block}

17. https://computing.llnl.gov/tutorials/openMP/