Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
PARALLEL & ASYNC
PROCESSING USING TPL
DATAFLOW
Petru Rebeja
AGENDA
• What is Dataflow?
• When to use it?
• How to use it?
• Q&A
THE BIG PICTURE
CLR Thread Pool
Tasks
PLINQ Parallel Loops
Concurrent Collections
Dataflow
DATAFLOW BENEFITS
• Effortless use of multi-threading
• Performance boost via painless optimization
• Development focus is...
DATAFLOW USAGES
High throughput, low-latency scenarios
Robotics
Manufacturing
Imaging Biology
Oil & Gas
Finance
PROGRAMMING MODEL
• Actor-based programming
• In-process message passing
• Components (blocks) for creating data processin...
ARCHITECTURE
IDataflowBlock
ISourceBlock<TOutput> ITargetBlock<TInput>
IPropagatorBlock<Tinput,Toutput>
COMPOSITION
Source
Target
Propagator
Optional
Transform
BUFFERING BLOCKS
BufferBlock<T>
BroadcastBlock<T>
WriteOnceBlock<T>
EXECUTION BLOCKS
ActionBlock<T>
TransformBlock<T,V>
TransformManyBlock<T,V>
GROUPING BLOCKS
BatchBlock<T>
JoinBlock<T1,T2,…>
BatchedJoinBlock<T1,T2>
BEHAVIOR CONFIGURATION OPTIONS
• BufferBlock<T>
• BroadcastBlock<T>
• WriteOnceBlock<T>
DataflowBlockOptions
• ActionBlock...
COMPLETION & CANCELLATION
• To know when a block completes await block.Completion
or add a continuation task to it
• To pr...
ERROR HANDLING
• If the exception does not affect the integrity of the
pipeline – use a try/catch inside the block
• Other...
DEALING WITH CONCURRENCY
• Rule of thumb: avoid shared state whenever possible.
• Use ConcurrentExclusiveSchedulerPair to ...
CREATING CUSTOM BLOCKS
The easy way:
DataflowBlock.Encapsulate<TInput, TOutput>(
target, source)
CREATING CUSTOM BLOCKS
The hard(core) way:
class CustomBlock:
IPropagatorBlock<TInput, TOutput>
{
}
CREATING CUSTOM BLOCKS
Either way you choose, don’t forget to:
• Propagate completion
• Pool for cancellation
REFERENCES & FURTHER READING
Dataflow (Task Parallel Library) http://msdn.microsoft.com/en-us/library/hh228603(v=vs.110).a...
QUESTIONS?
THANK YOU!
Petru.Rebeja@gmail.com
Parallel & Async Processing using TPL
Dataflow
Parallel & async processing using tpl dataflow
Upcoming SlideShare
Loading in …5
×

Parallel & async processing using tpl dataflow

801 views

Published on

Parallel & async processing using tpl dataflow

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Parallel & async processing using tpl dataflow

  1. 1. PARALLEL & ASYNC PROCESSING USING TPL DATAFLOW Petru Rebeja
  2. 2. AGENDA • What is Dataflow? • When to use it? • How to use it? • Q&A
  3. 3. THE BIG PICTURE CLR Thread Pool Tasks PLINQ Parallel Loops Concurrent Collections Dataflow
  4. 4. DATAFLOW BENEFITS • Effortless use of multi-threading • Performance boost via painless optimization • Development focus is on the ‘what’ rather than ‘how’
  5. 5. DATAFLOW USAGES High throughput, low-latency scenarios Robotics Manufacturing Imaging Biology Oil & Gas Finance
  6. 6. PROGRAMMING MODEL • Actor-based programming • In-process message passing • Components (blocks) for creating data processing pipelines
  7. 7. ARCHITECTURE IDataflowBlock ISourceBlock<TOutput> ITargetBlock<TInput> IPropagatorBlock<Tinput,Toutput>
  8. 8. COMPOSITION Source Target Propagator Optional Transform
  9. 9. BUFFERING BLOCKS BufferBlock<T> BroadcastBlock<T> WriteOnceBlock<T>
  10. 10. EXECUTION BLOCKS ActionBlock<T> TransformBlock<T,V> TransformManyBlock<T,V>
  11. 11. GROUPING BLOCKS BatchBlock<T> JoinBlock<T1,T2,…> BatchedJoinBlock<T1,T2>
  12. 12. BEHAVIOR CONFIGURATION OPTIONS • BufferBlock<T> • BroadcastBlock<T> • WriteOnceBlock<T> DataflowBlockOptions • ActionBlock<T> • TransformBlock<TIn, TOut> • TransformManyBlock<TIn, TOut> ExecutionDataflowBlockOptions • BatchBlock<T> • JoinBlock<T1, T2[, T3]> • BatchedJoinBlock<T1, T2> GroupingDataflowBlockOptions
  13. 13. COMPLETION & CANCELLATION • To know when a block completes await block.Completion or add a continuation task to it • To propagate completion from source to target, set DataflowLinkOptions.PropagateCompletion when linking • Set DataflowBlockOptions.CancellationToken to enable cancellation
  14. 14. ERROR HANDLING • If the exception does not affect the integrity of the pipeline – use a try/catch inside the block • Otherwise, handle errors outside of the pipeline by • Adding a continuation to block.Completion • Propagating errors through the pipeline
  15. 15. DEALING WITH CONCURRENCY • Rule of thumb: avoid shared state whenever possible. • Use ConcurrentExclusiveSchedulerPair to perform updates on shared state • Be aware of the caveats with ConcurrentExclusiveSchedulerPair
  16. 16. CREATING CUSTOM BLOCKS The easy way: DataflowBlock.Encapsulate<TInput, TOutput>( target, source)
  17. 17. CREATING CUSTOM BLOCKS The hard(core) way: class CustomBlock: IPropagatorBlock<TInput, TOutput> { }
  18. 18. CREATING CUSTOM BLOCKS Either way you choose, don’t forget to: • Propagate completion • Pool for cancellation
  19. 19. REFERENCES & FURTHER READING Dataflow (Task Parallel Library) http://msdn.microsoft.com/en-us/library/hh228603(v=vs.110).aspx Stephen Toub TPL Dataflow Tour http://channel9.msdn.com/posts/TPL-Dataflow-Tour Joseph Albahari The Future of .NET Parallel Programming http://channel9.msdn.com/events/TechEd/Australia/Tech-Ed-Australia- 2011/DEV308 Stephen Toub Inside TPL Dataflow http://channel9.msdn.com/Shows/Going+Deep/Stephen-Toub-Inside-TPL- Dataflow Alexey Kursov Pipeline TPL Dataflow Usage examples https://www.youtube.com/watch?v=AI9KxgDF43k https://www.youtube.com/watch?v=AI9KxgDF43k Richard Blewett, Andrew Clymer Pro Asynchronous Programming with .NET APRESS 2013 ISBN: 978-1430259206 AKKA.NET http://getakka.net/
  20. 20. QUESTIONS?
  21. 21. THANK YOU! Petru.Rebeja@gmail.com Parallel & Async Processing using TPL Dataflow

×