SlideShare a Scribd company logo
Patterns of Parallel
  Programming
Prepared by Yan Drugalya
 ydrugalya@gmail.com
      @ydrugalya
Agenda
•   Why parallel?
•   Terms and measures
•   Building Blocks
•   Patterns overview
    – Pipeline and data flow
    – Producer-Consumer
    – Map-Reduce
    – Other
Why Moore's law is not working
               anymore
•   Power consumption
•   Wire delays
•   DRAM access latency
•   Diminishing returns of more instruction-level
    parallelism
Power consumption
                                                                                      Sun’s Surface
                        10,000


                         1,000                                          Rocket Nozzle
Power Density (W/cm2)




                          100                               Nuclear Reactor



                           10                            Pentium® processors

                                                                               Hot Plate
                            1
                                   8080




                             ‘70             ‘80   ’90               ’00                        ‘10
Wire delays
Diminishing returns
• 80’s
  – 10 CPI  1 CPI
• 90
  – 1 CPI  0.5CPI
• 00’s: multicore
No matter how fast processors get, software
consistently finds new ways to eat up the extra
speed.
                                 Herb Sutter
Survival




 To scale performance, put many processing cores on the
  microprocessor chip
 New Moore’s law edition is about doubling of cores.
Terms & Measures
•   Work = T1
•   Span = T∞
•   Work Law: Tp>=T1/P
•   Span Law: Tp>=T∞
•   Speedup: Tp/T1
    – Linear: θ(P)
    – Perfect: P
• Parallelism: T1/T∞
• Tp<=(T1-T∞)/P + T∞
Definitions
• Concurrent
  - Several things happenings at the same time
• Multithreaded
  – Multiple execution contexts
• Parallel
  – Multiple simultaneous computations
• Asynchronous
  – Not having to wait
Dangers
•   Race Conditions
•   Starvations
•   Deadlocks
•   Livelock
•   Optimizing compilers
•   …
Data parallelism




Parallel.ForEach(letters, ch => Capitalize(ch));
Task parallelism




Parallel.Invoke(() => Average(), () => Minimum() …);
Fork-Join
 • Additional work may be started only when specific subsets of
   the original elements have completed processing
 • All elements should be given the chance to run even if one
   invocation fails (Ping)


                               Parallel.Invoke(
                                   () => ComputeMean(),
            Fork                   () => ComputeMedian(),
                                   () => ComputeMode());

Compute   Compute   Compute    static void MyParallelInvoke(params Action[] actions)
 Median    Mean      Mode      {
                                    var tasks = new Task[actions.Length];
                                    for (int i = 0; i < actions.Length; i++)
                                            tasks[i] = Task.Factory.StartNew(actions[i]);
            Join                    Task.WaitAll(tasks);
                               }
Pipeline pattern
             Task<int> T1 = Task.Factory.StartNew(() =>
Task 1           { return result1(); });




             Task<double> T2 = T1.ContinueWith((antecedent) =>
Task 2           { return result2(antecedent.Result); });




             Task<double> T3 = T2.ContinueWith((antecedent) =>
Task 3           { return result3(antecedent.Result); });
Producer/Consumer
    Disk/Net
                   Read 1   Read 2    Read 3
BlockingCollection<T>       Process   Process   Process
Patterns of parallel programming
Other patterns
• Speculative Execution
• APM (IAsyncResult, Begin/end pairs)
• EAP(Operation/Callback pairs)
References
• Patterns for Parallel Programming: Understanding and Applying
  Parallel Patterns with the .NET Framework 4
• Pluralsight:
   – Introduction to Async and Parallel Programming in .NET 4
   – Async and Parallel Programming: Application Design
• The Free Lunch Is Over: A Fundamental Turn Toward
  Concurrency in Software
• Chapter 27 Multithreaded Algorithms from Introduction to
  algorithms 3rd edition

More Related Content

What's hot

MapR Tutorial Series
MapR Tutorial SeriesMapR Tutorial Series
MapR Tutorial Series
selvaraaju
 
Storm: Distributed and fault tolerant realtime computation
Storm: Distributed and fault tolerant realtime computationStorm: Distributed and fault tolerant realtime computation
Storm: Distributed and fault tolerant realtime computation
Ferran Galí Reniu
 
Apache Storm
Apache StormApache Storm
Apache Storm
Nguyen Quang
 
GoodFit: Multi-Resource Packing of Tasks with Dependencies
GoodFit: Multi-Resource Packing of Tasks with DependenciesGoodFit: Multi-Resource Packing of Tasks with Dependencies
GoodFit: Multi-Resource Packing of Tasks with Dependencies
DataWorks Summit/Hadoop Summit
 
Prometheus and Thanos
Prometheus and ThanosPrometheus and Thanos
Prometheus and Thanos
CloudOps2005
 
Storm
StormStorm
Multi-Tenant Storm Service on Hadoop Grid
Multi-Tenant Storm Service on Hadoop GridMulti-Tenant Storm Service on Hadoop Grid
Multi-Tenant Storm Service on Hadoop Grid
DataWorks Summit
 
Resource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache StormResource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache Storm
DataWorks Summit/Hadoop Summit
 
Distributed real time stream processing- why and how
Distributed real time stream processing- why and howDistributed real time stream processing- why and how
Distributed real time stream processing- why and how
Petr Zapletal
 
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
GeeksLab Odessa
 
Apache Storm Internals
Apache Storm InternalsApache Storm Internals
Apache Storm Internals
Humoyun Ahmedov
 
Apache Storm Tutorial
Apache Storm TutorialApache Storm Tutorial
Apache Storm Tutorial
Farzad Nozarian
 
Real-Time Streaming with Apache Spark Streaming and Apache Storm
Real-Time Streaming with Apache Spark Streaming and Apache StormReal-Time Streaming with Apache Spark Streaming and Apache Storm
Real-Time Streaming with Apache Spark Streaming and Apache Storm
Davorin Vukelic
 
Spark vs storm
Spark vs stormSpark vs storm
Spark vs storm
Trong Ton
 
One Grid to rule them all: Building a Multi-tenant Data Cloud with YARN
One Grid to rule them all: Building a Multi-tenant Data Cloud with YARNOne Grid to rule them all: Building a Multi-tenant Data Cloud with YARN
One Grid to rule them all: Building a Multi-tenant Data Cloud with YARN
DataWorks Summit
 
NYC Hadoop Meetup - MapR, Architecture, Philosophy and Applications
NYC Hadoop Meetup - MapR, Architecture, Philosophy and ApplicationsNYC Hadoop Meetup - MapR, Architecture, Philosophy and Applications
NYC Hadoop Meetup - MapR, Architecture, Philosophy and Applications
Jason Shao
 
Cassandra and Storm at Health Market Sceince
Cassandra and Storm at Health Market SceinceCassandra and Storm at Health Market Sceince
Cassandra and Storm at Health Market Sceince
P. Taylor Goetz
 
Kubernetes Observability with Prometheus by Example
Kubernetes Observability with Prometheus by ExampleKubernetes Observability with Prometheus by Example
Kubernetes Observability with Prometheus by Example
Thomas Riley
 
Hadoop performance optimization tips
Hadoop performance optimization tipsHadoop performance optimization tips
Hadoop performance optimization tips
Subhas Kumar Ghosh
 
Real-time Big Data Processing with Storm
Real-time Big Data Processing with StormReal-time Big Data Processing with Storm
Real-time Big Data Processing with Storm
viirya
 

What's hot (20)

MapR Tutorial Series
MapR Tutorial SeriesMapR Tutorial Series
MapR Tutorial Series
 
Storm: Distributed and fault tolerant realtime computation
Storm: Distributed and fault tolerant realtime computationStorm: Distributed and fault tolerant realtime computation
Storm: Distributed and fault tolerant realtime computation
 
Apache Storm
Apache StormApache Storm
Apache Storm
 
GoodFit: Multi-Resource Packing of Tasks with Dependencies
GoodFit: Multi-Resource Packing of Tasks with DependenciesGoodFit: Multi-Resource Packing of Tasks with Dependencies
GoodFit: Multi-Resource Packing of Tasks with Dependencies
 
Prometheus and Thanos
Prometheus and ThanosPrometheus and Thanos
Prometheus and Thanos
 
Storm
StormStorm
Storm
 
Multi-Tenant Storm Service on Hadoop Grid
Multi-Tenant Storm Service on Hadoop GridMulti-Tenant Storm Service on Hadoop Grid
Multi-Tenant Storm Service on Hadoop Grid
 
Resource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache StormResource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache Storm
 
Distributed real time stream processing- why and how
Distributed real time stream processing- why and howDistributed real time stream processing- why and how
Distributed real time stream processing- why and how
 
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
AI&BigData Lab 2016. Сарапин Виктор: Размер имеет значение: анализ по требова...
 
Apache Storm Internals
Apache Storm InternalsApache Storm Internals
Apache Storm Internals
 
Apache Storm Tutorial
Apache Storm TutorialApache Storm Tutorial
Apache Storm Tutorial
 
Real-Time Streaming with Apache Spark Streaming and Apache Storm
Real-Time Streaming with Apache Spark Streaming and Apache StormReal-Time Streaming with Apache Spark Streaming and Apache Storm
Real-Time Streaming with Apache Spark Streaming and Apache Storm
 
Spark vs storm
Spark vs stormSpark vs storm
Spark vs storm
 
One Grid to rule them all: Building a Multi-tenant Data Cloud with YARN
One Grid to rule them all: Building a Multi-tenant Data Cloud with YARNOne Grid to rule them all: Building a Multi-tenant Data Cloud with YARN
One Grid to rule them all: Building a Multi-tenant Data Cloud with YARN
 
NYC Hadoop Meetup - MapR, Architecture, Philosophy and Applications
NYC Hadoop Meetup - MapR, Architecture, Philosophy and ApplicationsNYC Hadoop Meetup - MapR, Architecture, Philosophy and Applications
NYC Hadoop Meetup - MapR, Architecture, Philosophy and Applications
 
Cassandra and Storm at Health Market Sceince
Cassandra and Storm at Health Market SceinceCassandra and Storm at Health Market Sceince
Cassandra and Storm at Health Market Sceince
 
Kubernetes Observability with Prometheus by Example
Kubernetes Observability with Prometheus by ExampleKubernetes Observability with Prometheus by Example
Kubernetes Observability with Prometheus by Example
 
Hadoop performance optimization tips
Hadoop performance optimization tipsHadoop performance optimization tips
Hadoop performance optimization tips
 
Real-time Big Data Processing with Storm
Real-time Big Data Processing with StormReal-time Big Data Processing with Storm
Real-time Big Data Processing with Storm
 

Viewers also liked

Object-2-Object mapping, как приправа к вашему проекту
Object-2-Object mapping, как приправа к вашему проектуObject-2-Object mapping, как приправа к вашему проекту
Object-2-Object mapping, как приправа к вашему проекту
Alex Tumanoff
 
Bdd by Dmitri Aizenberg
Bdd by Dmitri AizenbergBdd by Dmitri Aizenberg
Bdd by Dmitri Aizenberg
Alex Tumanoff
 
Navigation map factory by Alexey Klimenko
Navigation map factory by Alexey KlimenkoNavigation map factory by Alexey Klimenko
Navigation map factory by Alexey Klimenko
Alex Tumanoff
 
Microsoft Office 2013 новая модель разработки приложений
Microsoft Office 2013 новая модель разработки приложенийMicrosoft Office 2013 новая модель разработки приложений
Microsoft Office 2013 новая модель разработки приложений
Alex Tumanoff
 
Deep Dive C# by Sergey Teplyakov
Deep Dive  C# by Sergey TeplyakovDeep Dive  C# by Sergey Teplyakov
Deep Dive C# by Sergey Teplyakov
Alex Tumanoff
 
Anti patterns
Anti patternsAnti patterns
Anti patterns
Alex Tumanoff
 
"Drools: декларативная бизнес-логика в Java-приложениях" by Дмитрий Контрерас...
"Drools: декларативная бизнес-логика в Java-приложениях" by Дмитрий Контрерас..."Drools: декларативная бизнес-логика в Java-приложениях" by Дмитрий Контрерас...
"Drools: декларативная бизнес-логика в Java-приложениях" by Дмитрий Контрерас...
Alex Tumanoff
 
Async clinic by by Sergey Teplyakov
Async clinic by by Sergey TeplyakovAsync clinic by by Sergey Teplyakov
Async clinic by by Sergey Teplyakov
Alex Tumanoff
 
Mono
MonoMono

Viewers also liked (10)

Design principles
Design principles Design principles
Design principles
 
Object-2-Object mapping, как приправа к вашему проекту
Object-2-Object mapping, как приправа к вашему проектуObject-2-Object mapping, как приправа к вашему проекту
Object-2-Object mapping, как приправа к вашему проекту
 
Bdd by Dmitri Aizenberg
Bdd by Dmitri AizenbergBdd by Dmitri Aizenberg
Bdd by Dmitri Aizenberg
 
Navigation map factory by Alexey Klimenko
Navigation map factory by Alexey KlimenkoNavigation map factory by Alexey Klimenko
Navigation map factory by Alexey Klimenko
 
Microsoft Office 2013 новая модель разработки приложений
Microsoft Office 2013 новая модель разработки приложенийMicrosoft Office 2013 новая модель разработки приложений
Microsoft Office 2013 новая модель разработки приложений
 
Deep Dive C# by Sergey Teplyakov
Deep Dive  C# by Sergey TeplyakovDeep Dive  C# by Sergey Teplyakov
Deep Dive C# by Sergey Teplyakov
 
Anti patterns
Anti patternsAnti patterns
Anti patterns
 
"Drools: декларативная бизнес-логика в Java-приложениях" by Дмитрий Контрерас...
"Drools: декларативная бизнес-логика в Java-приложениях" by Дмитрий Контрерас..."Drools: декларативная бизнес-логика в Java-приложениях" by Дмитрий Контрерас...
"Drools: декларативная бизнес-логика в Java-приложениях" by Дмитрий Контрерас...
 
Async clinic by by Sergey Teplyakov
Async clinic by by Sergey TeplyakovAsync clinic by by Sergey Teplyakov
Async clinic by by Sergey Teplyakov
 
Mono
MonoMono
Mono
 

Similar to Patterns of parallel programming

Multicore programmingandtpl(.net day)
Multicore programmingandtpl(.net day)Multicore programmingandtpl(.net day)
Multicore programmingandtpl(.net day)
Yan Drugalya
 
Multicore programmingandtpl
Multicore programmingandtplMulticore programmingandtpl
Multicore programmingandtpl
Yan Drugalya
 
04 performance
04 performance04 performance
04 performance
marangburu42
 
Full introduction to_parallel_computing
Full introduction to_parallel_computingFull introduction to_parallel_computing
Full introduction to_parallel_computing
Supasit Kajkamhaeng
 
High Performance & High Throughput Computing - EUDAT Summer School (Giuseppe ...
High Performance & High Throughput Computing - EUDAT Summer School (Giuseppe ...High Performance & High Throughput Computing - EUDAT Summer School (Giuseppe ...
High Performance & High Throughput Computing - EUDAT Summer School (Giuseppe ...
EUDAT
 
03 performance
03 performance03 performance
03 performance
marangburu42
 
Hanborq optimizations on hadoop map reduce 20120221a
Hanborq optimizations on hadoop map reduce 20120221aHanborq optimizations on hadoop map reduce 20120221a
Hanborq optimizations on hadoop map reduce 20120221a
Schubert Zhang
 
Hanborq Optimizations on Hadoop MapReduce
Hanborq Optimizations on Hadoop MapReduceHanborq Optimizations on Hadoop MapReduce
Hanborq Optimizations on Hadoop MapReduce
Hanborq Inc.
 
Quantifying Overheads in Charm++ and HPX using Task Bench
Quantifying Overheads in Charm++ and HPX using Task BenchQuantifying Overheads in Charm++ and HPX using Task Bench
Quantifying Overheads in Charm++ and HPX using Task Bench
Patrick Diehl
 
Adam Sitnik "State of the .NET Performance"
Adam Sitnik "State of the .NET Performance"Adam Sitnik "State of the .NET Performance"
Adam Sitnik "State of the .NET Performance"
Yulia Tsisyk
 
State of the .Net Performance
State of the .Net PerformanceState of the .Net Performance
State of the .Net Performance
CUSTIS
 
Tutotial 2 answer
Tutotial 2 answerTutotial 2 answer
Tutotial 2 answer
Udaya Kumar
 
Distributed systems scheduling
Distributed systems schedulingDistributed systems scheduling
Distributed systems scheduling
Pragati Startup Presentation Designer firm
 
Fast Optimization Intevac
Fast Optimization IntevacFast Optimization Intevac
Fast Optimization Intevac
vvk0
 
Parallel Algorithms
Parallel AlgorithmsParallel Algorithms
Parallel Algorithms
Dr Sandeep Kumar Poonia
 
Lecture1
Lecture1Lecture1
Lecture1
tt_aljobory
 
Load balancing theory and practice
Load balancing theory and practiceLoad balancing theory and practice
Load balancing theory and practice
FoundationDB
 
Dasia 2022
Dasia 2022Dasia 2022
Where the wild things are - Benchmarking and Micro-Optimisations
Where the wild things are - Benchmarking and Micro-OptimisationsWhere the wild things are - Benchmarking and Micro-Optimisations
Where the wild things are - Benchmarking and Micro-Optimisations
Matt Warren
 
02 performance
02 performance02 performance
02 performance
marangburu42
 

Similar to Patterns of parallel programming (20)

Multicore programmingandtpl(.net day)
Multicore programmingandtpl(.net day)Multicore programmingandtpl(.net day)
Multicore programmingandtpl(.net day)
 
Multicore programmingandtpl
Multicore programmingandtplMulticore programmingandtpl
Multicore programmingandtpl
 
04 performance
04 performance04 performance
04 performance
 
Full introduction to_parallel_computing
Full introduction to_parallel_computingFull introduction to_parallel_computing
Full introduction to_parallel_computing
 
High Performance & High Throughput Computing - EUDAT Summer School (Giuseppe ...
High Performance & High Throughput Computing - EUDAT Summer School (Giuseppe ...High Performance & High Throughput Computing - EUDAT Summer School (Giuseppe ...
High Performance & High Throughput Computing - EUDAT Summer School (Giuseppe ...
 
03 performance
03 performance03 performance
03 performance
 
Hanborq optimizations on hadoop map reduce 20120221a
Hanborq optimizations on hadoop map reduce 20120221aHanborq optimizations on hadoop map reduce 20120221a
Hanborq optimizations on hadoop map reduce 20120221a
 
Hanborq Optimizations on Hadoop MapReduce
Hanborq Optimizations on Hadoop MapReduceHanborq Optimizations on Hadoop MapReduce
Hanborq Optimizations on Hadoop MapReduce
 
Quantifying Overheads in Charm++ and HPX using Task Bench
Quantifying Overheads in Charm++ and HPX using Task BenchQuantifying Overheads in Charm++ and HPX using Task Bench
Quantifying Overheads in Charm++ and HPX using Task Bench
 
Adam Sitnik "State of the .NET Performance"
Adam Sitnik "State of the .NET Performance"Adam Sitnik "State of the .NET Performance"
Adam Sitnik "State of the .NET Performance"
 
State of the .Net Performance
State of the .Net PerformanceState of the .Net Performance
State of the .Net Performance
 
Tutotial 2 answer
Tutotial 2 answerTutotial 2 answer
Tutotial 2 answer
 
Distributed systems scheduling
Distributed systems schedulingDistributed systems scheduling
Distributed systems scheduling
 
Fast Optimization Intevac
Fast Optimization IntevacFast Optimization Intevac
Fast Optimization Intevac
 
Parallel Algorithms
Parallel AlgorithmsParallel Algorithms
Parallel Algorithms
 
Lecture1
Lecture1Lecture1
Lecture1
 
Load balancing theory and practice
Load balancing theory and practiceLoad balancing theory and practice
Load balancing theory and practice
 
Dasia 2022
Dasia 2022Dasia 2022
Dasia 2022
 
Where the wild things are - Benchmarking and Micro-Optimisations
Where the wild things are - Benchmarking and Micro-OptimisationsWhere the wild things are - Benchmarking and Micro-Optimisations
Where the wild things are - Benchmarking and Micro-Optimisations
 
02 performance
02 performance02 performance
02 performance
 

More from Alex Tumanoff

Sql server 2019 New Features by Yevhen Nedaskivskyi
Sql server 2019 New Features by Yevhen NedaskivskyiSql server 2019 New Features by Yevhen Nedaskivskyi
Sql server 2019 New Features by Yevhen Nedaskivskyi
Alex Tumanoff
 
Odessa .net-user-group-sql-server-2019-hidden-gems by Denis Reznik
Odessa .net-user-group-sql-server-2019-hidden-gems by Denis ReznikOdessa .net-user-group-sql-server-2019-hidden-gems by Denis Reznik
Odessa .net-user-group-sql-server-2019-hidden-gems by Denis Reznik
Alex Tumanoff
 
Azure data bricks by Eugene Polonichko
Azure data bricks by Eugene PolonichkoAzure data bricks by Eugene Polonichko
Azure data bricks by Eugene Polonichko
Alex Tumanoff
 
Sdlc by Anatoliy Anthony Cox
Sdlc by  Anatoliy Anthony CoxSdlc by  Anatoliy Anthony Cox
Sdlc by Anatoliy Anthony Cox
Alex Tumanoff
 
Kostenko ux november-2014_1
Kostenko ux november-2014_1Kostenko ux november-2014_1
Kostenko ux november-2014_1
Alex Tumanoff
 
Java 8 in action.jinq.v.1.3
Java 8 in action.jinq.v.1.3Java 8 in action.jinq.v.1.3
Java 8 in action.jinq.v.1.3
Alex Tumanoff
 
Spring.new hope.1.3
Spring.new hope.1.3Spring.new hope.1.3
Spring.new hope.1.3
Alex Tumanoff
 
Serialization and performance by Sergey Morenets
Serialization and performance by Sergey MorenetsSerialization and performance by Sergey Morenets
Serialization and performance by Sergey Morenets
Alex Tumanoff
 
Игры для мобильных платформ by Алексей Рыбаков
Игры для мобильных платформ by Алексей РыбаковИгры для мобильных платформ by Алексей Рыбаков
Игры для мобильных платформ by Алексей Рыбаков
Alex Tumanoff
 
Android sync adapter
Android sync adapterAndroid sync adapter
Android sync adapter
Alex Tumanoff
 
Неформальные размышления о сертификации в IT
Неформальные размышления о сертификации в ITНеформальные размышления о сертификации в IT
Неформальные размышления о сертификации в IT
Alex Tumanoff
 
Разработка расширений Firefox
Разработка расширений FirefoxРазработка расширений Firefox
Разработка расширений Firefox
Alex Tumanoff
 
"AnnotatedSQL - провайдер с плюшками за 5 минут" - Геннадий Дубина, Senior So...
"AnnotatedSQL - провайдер с плюшками за 5 минут" - Геннадий Дубина, Senior So..."AnnotatedSQL - провайдер с плюшками за 5 минут" - Геннадий Дубина, Senior So...
"AnnotatedSQL - провайдер с плюшками за 5 минут" - Геннадий Дубина, Senior So...
Alex Tumanoff
 
Lambda выражения и Java 8
Lambda выражения и Java 8Lambda выражения и Java 8
Lambda выражения и Java 8
Alex Tumanoff
 
XP практики в проектах с тяжелой наследственностью
XP практики в проектах с тяжелой наследственностьюXP практики в проектах с тяжелой наследственностью
XP практики в проектах с тяжелой наследственностью
Alex Tumanoff
 
Первые шаги во фрилансе
Первые шаги во фрилансеПервые шаги во фрилансе
Первые шаги во фрилансе
Alex Tumanoff
 
Spring Web Flow. A little flow of happiness.
Spring Web Flow. A little flow of happiness.Spring Web Flow. A little flow of happiness.
Spring Web Flow. A little flow of happiness.
Alex Tumanoff
 
Какой фреймворк нам нужен для Web? Денис Цыплаков
Какой фреймворк нам нужен для Web? Денис ЦыплаковКакой фреймворк нам нужен для Web? Денис Цыплаков
Какой фреймворк нам нужен для Web? Денис Цыплаков
Alex Tumanoff
 
Distilled mongo db by Boris Trofimov
Distilled mongo db by Boris TrofimovDistilled mongo db by Boris Trofimov
Distilled mongo db by Boris Trofimov
Alex Tumanoff
 
Enterprise or not to enterprise
Enterprise or not to enterpriseEnterprise or not to enterprise
Enterprise or not to enterprise
Alex Tumanoff
 

More from Alex Tumanoff (20)

Sql server 2019 New Features by Yevhen Nedaskivskyi
Sql server 2019 New Features by Yevhen NedaskivskyiSql server 2019 New Features by Yevhen Nedaskivskyi
Sql server 2019 New Features by Yevhen Nedaskivskyi
 
Odessa .net-user-group-sql-server-2019-hidden-gems by Denis Reznik
Odessa .net-user-group-sql-server-2019-hidden-gems by Denis ReznikOdessa .net-user-group-sql-server-2019-hidden-gems by Denis Reznik
Odessa .net-user-group-sql-server-2019-hidden-gems by Denis Reznik
 
Azure data bricks by Eugene Polonichko
Azure data bricks by Eugene PolonichkoAzure data bricks by Eugene Polonichko
Azure data bricks by Eugene Polonichko
 
Sdlc by Anatoliy Anthony Cox
Sdlc by  Anatoliy Anthony CoxSdlc by  Anatoliy Anthony Cox
Sdlc by Anatoliy Anthony Cox
 
Kostenko ux november-2014_1
Kostenko ux november-2014_1Kostenko ux november-2014_1
Kostenko ux november-2014_1
 
Java 8 in action.jinq.v.1.3
Java 8 in action.jinq.v.1.3Java 8 in action.jinq.v.1.3
Java 8 in action.jinq.v.1.3
 
Spring.new hope.1.3
Spring.new hope.1.3Spring.new hope.1.3
Spring.new hope.1.3
 
Serialization and performance by Sergey Morenets
Serialization and performance by Sergey MorenetsSerialization and performance by Sergey Morenets
Serialization and performance by Sergey Morenets
 
Игры для мобильных платформ by Алексей Рыбаков
Игры для мобильных платформ by Алексей РыбаковИгры для мобильных платформ by Алексей Рыбаков
Игры для мобильных платформ by Алексей Рыбаков
 
Android sync adapter
Android sync adapterAndroid sync adapter
Android sync adapter
 
Неформальные размышления о сертификации в IT
Неформальные размышления о сертификации в ITНеформальные размышления о сертификации в IT
Неформальные размышления о сертификации в IT
 
Разработка расширений Firefox
Разработка расширений FirefoxРазработка расширений Firefox
Разработка расширений Firefox
 
"AnnotatedSQL - провайдер с плюшками за 5 минут" - Геннадий Дубина, Senior So...
"AnnotatedSQL - провайдер с плюшками за 5 минут" - Геннадий Дубина, Senior So..."AnnotatedSQL - провайдер с плюшками за 5 минут" - Геннадий Дубина, Senior So...
"AnnotatedSQL - провайдер с плюшками за 5 минут" - Геннадий Дубина, Senior So...
 
Lambda выражения и Java 8
Lambda выражения и Java 8Lambda выражения и Java 8
Lambda выражения и Java 8
 
XP практики в проектах с тяжелой наследственностью
XP практики в проектах с тяжелой наследственностьюXP практики в проектах с тяжелой наследственностью
XP практики в проектах с тяжелой наследственностью
 
Первые шаги во фрилансе
Первые шаги во фрилансеПервые шаги во фрилансе
Первые шаги во фрилансе
 
Spring Web Flow. A little flow of happiness.
Spring Web Flow. A little flow of happiness.Spring Web Flow. A little flow of happiness.
Spring Web Flow. A little flow of happiness.
 
Какой фреймворк нам нужен для Web? Денис Цыплаков
Какой фреймворк нам нужен для Web? Денис ЦыплаковКакой фреймворк нам нужен для Web? Денис Цыплаков
Какой фреймворк нам нужен для Web? Денис Цыплаков
 
Distilled mongo db by Boris Trofimov
Distilled mongo db by Boris TrofimovDistilled mongo db by Boris Trofimov
Distilled mongo db by Boris Trofimov
 
Enterprise or not to enterprise
Enterprise or not to enterpriseEnterprise or not to enterprise
Enterprise or not to enterprise
 

Patterns of parallel programming

  • 1. Patterns of Parallel Programming Prepared by Yan Drugalya ydrugalya@gmail.com @ydrugalya
  • 2. Agenda • Why parallel? • Terms and measures • Building Blocks • Patterns overview – Pipeline and data flow – Producer-Consumer – Map-Reduce – Other
  • 3. Why Moore's law is not working anymore • Power consumption • Wire delays • DRAM access latency • Diminishing returns of more instruction-level parallelism
  • 4. Power consumption Sun’s Surface 10,000 1,000 Rocket Nozzle Power Density (W/cm2) 100 Nuclear Reactor 10 Pentium® processors Hot Plate 1 8080 ‘70 ‘80 ’90 ’00 ‘10
  • 6. Diminishing returns • 80’s – 10 CPI  1 CPI • 90 – 1 CPI  0.5CPI • 00’s: multicore
  • 7. No matter how fast processors get, software consistently finds new ways to eat up the extra speed. Herb Sutter
  • 8. Survival  To scale performance, put many processing cores on the microprocessor chip  New Moore’s law edition is about doubling of cores.
  • 9. Terms & Measures • Work = T1 • Span = T∞ • Work Law: Tp>=T1/P • Span Law: Tp>=T∞ • Speedup: Tp/T1 – Linear: θ(P) – Perfect: P • Parallelism: T1/T∞ • Tp<=(T1-T∞)/P + T∞
  • 10. Definitions • Concurrent - Several things happenings at the same time • Multithreaded – Multiple execution contexts • Parallel – Multiple simultaneous computations • Asynchronous – Not having to wait
  • 11. Dangers • Race Conditions • Starvations • Deadlocks • Livelock • Optimizing compilers • …
  • 13. Task parallelism Parallel.Invoke(() => Average(), () => Minimum() …);
  • 14. Fork-Join • Additional work may be started only when specific subsets of the original elements have completed processing • All elements should be given the chance to run even if one invocation fails (Ping) Parallel.Invoke( () => ComputeMean(), Fork () => ComputeMedian(), () => ComputeMode()); Compute Compute Compute static void MyParallelInvoke(params Action[] actions) Median Mean Mode { var tasks = new Task[actions.Length]; for (int i = 0; i < actions.Length; i++) tasks[i] = Task.Factory.StartNew(actions[i]); Join Task.WaitAll(tasks); }
  • 15. Pipeline pattern Task<int> T1 = Task.Factory.StartNew(() => Task 1 { return result1(); }); Task<double> T2 = T1.ContinueWith((antecedent) => Task 2 { return result2(antecedent.Result); }); Task<double> T3 = T2.ContinueWith((antecedent) => Task 3 { return result3(antecedent.Result); });
  • 16. Producer/Consumer Disk/Net Read 1 Read 2 Read 3 BlockingCollection<T> Process Process Process
  • 18. Other patterns • Speculative Execution • APM (IAsyncResult, Begin/end pairs) • EAP(Operation/Callback pairs)
  • 19. References • Patterns for Parallel Programming: Understanding and Applying Parallel Patterns with the .NET Framework 4 • Pluralsight: – Introduction to Async and Parallel Programming in .NET 4 – Async and Parallel Programming: Application Design • The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software • Chapter 27 Multithreaded Algorithms from Introduction to algorithms 3rd edition