Successfully reported this slideshow.

Patterns of parallel programming

0

Share

Loading in …3
×
1 of 19
1 of 19

More Related Content

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Patterns of parallel programming

  1. 1. Patterns of Parallel Programming Prepared by Yan Drugalya ydrugalya@gmail.com @ydrugalya
  2. 2. Agenda • Why parallel? • Terms and measures • Building Blocks • Patterns overview – Pipeline and data flow – Producer-Consumer – Map-Reduce – Other
  3. 3. Why Moore's law is not working anymore • Power consumption • Wire delays • DRAM access latency • Diminishing returns of more instruction-level parallelism
  4. 4. Power consumption Sun’s Surface 10,000 1,000 Rocket Nozzle Power Density (W/cm2) 100 Nuclear Reactor 10 Pentium® processors Hot Plate 1 8080 ‘70 ‘80 ’90 ’00 ‘10
  5. 5. Wire delays
  6. 6. Diminishing returns • 80’s – 10 CPI  1 CPI • 90 – 1 CPI  0.5CPI • 00’s: multicore
  7. 7. No matter how fast processors get, software consistently finds new ways to eat up the extra speed. Herb Sutter
  8. 8. Survival  To scale performance, put many processing cores on the microprocessor chip  New Moore’s law edition is about doubling of cores.
  9. 9. Terms & Measures • Work = T1 • Span = T∞ • Work Law: Tp>=T1/P • Span Law: Tp>=T∞ • Speedup: Tp/T1 – Linear: θ(P) – Perfect: P • Parallelism: T1/T∞ • Tp<=(T1-T∞)/P + T∞
  10. 10. Definitions • Concurrent - Several things happenings at the same time • Multithreaded – Multiple execution contexts • Parallel – Multiple simultaneous computations • Asynchronous – Not having to wait
  11. 11. Dangers • Race Conditions • Starvations • Deadlocks • Livelock • Optimizing compilers • …
  12. 12. Data parallelism Parallel.ForEach(letters, ch => Capitalize(ch));
  13. 13. Task parallelism Parallel.Invoke(() => Average(), () => Minimum() …);
  14. 14. Fork-Join • Additional work may be started only when specific subsets of the original elements have completed processing • All elements should be given the chance to run even if one invocation fails (Ping) Parallel.Invoke( () => ComputeMean(), Fork () => ComputeMedian(), () => ComputeMode()); Compute Compute Compute static void MyParallelInvoke(params Action[] actions) Median Mean Mode { var tasks = new Task[actions.Length]; for (int i = 0; i < actions.Length; i++) tasks[i] = Task.Factory.StartNew(actions[i]); Join Task.WaitAll(tasks); }
  15. 15. Pipeline pattern Task<int> T1 = Task.Factory.StartNew(() => Task 1 { return result1(); }); Task<double> T2 = T1.ContinueWith((antecedent) => Task 2 { return result2(antecedent.Result); }); Task<double> T3 = T2.ContinueWith((antecedent) => Task 3 { return result3(antecedent.Result); });
  16. 16. Producer/Consumer Disk/Net Read 1 Read 2 Read 3 BlockingCollection<T> Process Process Process
  17. 17. Other patterns • Speculative Execution • APM (IAsyncResult, Begin/end pairs) • EAP(Operation/Callback pairs)
  18. 18. References • Patterns for Parallel Programming: Understanding and Applying Parallel Patterns with the .NET Framework 4 • Pluralsight: – Introduction to Async and Parallel Programming in .NET 4 – Async and Parallel Programming: Application Design • The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software • Chapter 27 Multithreaded Algorithms from Introduction to algorithms 3rd edition

×