GParscoz concurrency is GroovyVáclav Pech
Asynchronous calculationsFork/JoinParallel collectionsActorsAgents, StmDataflowCSPWhy this talk?
StructureParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalS...
StructureParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalS...
ParallelismTask parallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierar...
Asynchronous invocationFuture f = threadPool.submit(calculation);…System.out.println(“Result: “ + f.get());
Async the Groovy waytask {calculation.process()}
Async the Groovy waydef group = new NonDaemonPGroup(10)group.task {calculation.process()}
group.task {->…}group.task new Runnable() {…}group.task new Callable<V>() {...}Async the Groovy way
Independent tasksdef group = new NonDaemonPGroup(10)submissions.each {form →group.task {form.process()}}
Dependent tasksParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarc...
State sharingParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchi...
State sharingList registrations = []submissions.each {form →group.task {if (form.process().valid) {registrations << form}}}
State sharingList registrations = []submissions.each {form →group.task {if (form.process().valid) {registrations << form}}...
Agent Lock Shared Mutable State in a SafeAltertheStateAltertheState
Agent insideDouble IncrementAdd 25Message Queue36 thread
Sharing through agentsAgent registrations = group.agent( [] )submissions.each {form →group.task {if (form.process().valid)...
Random task dependencyParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctional...
Dataflow ConcurrencyNo race-conditionsNo live-locksDeterministic deadlocks
Dataflow Variables / Promisesmain task2 task3xyztask1
Dataflow Variables / Promisestask2xtask1
Dataflow Variables / Promisesmain task2 task3xyztask1
Chaining promisesdef h1 = download(url) then {text → text.trim()} then hash
Error handlingurl.then(download).then(calculateHash).then(formatResult).then(printResult, printError).then(sendNotificatio...
List registrations = []List<Promise> forms=submissions.collect {form →group.task {if (form.process().valid) return form}}f...
Dataflow Variables / Promisestask2xtask1
Dataflow Channelstask2x|y|ztask1
Synchronous Channelstask2x|y|ztask1
Tasks with progress indicationList<Promise> forms=submissions.collect {form →group.task {def result = form.process()progre...
Channel SelectionSelect alt = group.select(validForms, invalidForms)SelectResult selectResult = alt.select() //alt.priorit...
ParallelismData parallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierar...
ParallelismData parallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierar...
Geometric decompositionimages.eachParallel {it.process()}documents.sumParallel()candidates.maxParallel {it.salary}.marry()
Geometric decompositionregistrations = submissions.collectParallel { form -> form.process()}.findAllParallel { it.valid }r...
Frequent confusion
Improper use 1def accumulator = 0myCollection.eachParallel {accumulator += calculate(it)}
Improper use 2new File("/file.txt").withReader{reader ->reader.eachParallel {def r1 = step1(r)def r2 = step2(r1)def r3 = s...
Unroll iterationdef pipeline = data | step1 | step2 | step3new File("/file.txt").withReader{reader ->reader.each {data << ...
Unroll iteration
ParallelismStreamed dataTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchi...
Pipeline DSLdef toProcess = new DataflowQueue()def validated = new DataflowQueue()toProcess | {form -> process(form)} |{pr...
Url resolverUrl resolverUrl resolverprocessvalidateinProcessvalidated
Url resolverUrl resolverDownloaderGroovy scanner Scala scannerUrl resolverUrl resolverReporterSpeculatorEvaluatorSplitterC...
Dataflow Operatorsoperator(inputs: [headers, bodies, footers],outputs: [articles, summaries]){header, body, footer ->def a...
ConclusionParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchical...
SummaryParallelism is not hard, multi-threading isJon Kerridge, Napier University
Questions?Find more at:http://gpars.codehaus.orghttp://www.jroller.com/vaclavhttp://twitter.com/vaclav_pech
GPars howto - when to use which concurrency abstraction
Upcoming SlideShare
Loading in...5
×

GPars howto - when to use which concurrency abstraction

1,295

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,295
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
20
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

GPars howto - when to use which concurrency abstraction

  1. 1. GParscoz concurrency is GroovyVáclav Pech
  2. 2. Asynchronous calculationsFork/JoinParallel collectionsActorsAgents, StmDataflowCSPWhy this talk?
  3. 3. StructureParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  4. 4. StructureParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  5. 5. ParallelismTask parallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  6. 6. Asynchronous invocationFuture f = threadPool.submit(calculation);…System.out.println(“Result: “ + f.get());
  7. 7. Async the Groovy waytask {calculation.process()}
  8. 8. Async the Groovy waydef group = new NonDaemonPGroup(10)group.task {calculation.process()}
  9. 9. group.task {->…}group.task new Runnable() {…}group.task new Callable<V>() {...}Async the Groovy way
  10. 10. Independent tasksdef group = new NonDaemonPGroup(10)submissions.each {form →group.task {form.process()}}
  11. 11. Dependent tasksParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  12. 12. State sharingParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  13. 13. State sharingList registrations = []submissions.each {form →group.task {if (form.process().valid) {registrations << form}}}
  14. 14. State sharingList registrations = []submissions.each {form →group.task {if (form.process().valid) {registrations << form}}}Needs protection
  15. 15. Agent Lock Shared Mutable State in a SafeAltertheStateAltertheState
  16. 16. Agent insideDouble IncrementAdd 25Message Queue36 thread
  17. 17. Sharing through agentsAgent registrations = group.agent( [] )submissions.each {form →group.task {if (form.process().valid) {registrations.send {it << form}}}}
  18. 18. Random task dependencyParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  19. 19. Dataflow ConcurrencyNo race-conditionsNo live-locksDeterministic deadlocks
  20. 20. Dataflow Variables / Promisesmain task2 task3xyztask1
  21. 21. Dataflow Variables / Promisestask2xtask1
  22. 22. Dataflow Variables / Promisesmain task2 task3xyztask1
  23. 23. Chaining promisesdef h1 = download(url) then {text → text.trim()} then hash
  24. 24. Error handlingurl.then(download).then(calculateHash).then(formatResult).then(printResult, printError).then(sendNotificationEmail);
  25. 25. List registrations = []List<Promise> forms=submissions.collect {form →group.task {if (form.process().valid) return form}}forms.each{registrations << it.get()}Pass results around
  26. 26. Dataflow Variables / Promisestask2xtask1
  27. 27. Dataflow Channelstask2x|y|ztask1
  28. 28. Synchronous Channelstask2x|y|ztask1
  29. 29. Tasks with progress indicationList<Promise> forms=submissions.collect {form →group.task {def result = form.process()progressQueue << 1if (result.valid) {return form}}}
  30. 30. Channel SelectionSelect alt = group.select(validForms, invalidForms)SelectResult selectResult = alt.select() //alt.prioritySelect()switch (selectResult.index) {case 0: registrations << selectResult.value; breakcase 1: ...}
  31. 31. ParallelismData parallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  32. 32. ParallelismData parallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  33. 33. Geometric decompositionimages.eachParallel {it.process()}documents.sumParallel()candidates.maxParallel {it.salary}.marry()
  34. 34. Geometric decompositionregistrations = submissions.collectParallel { form -> form.process()}.findAllParallel { it.valid }registrations = submissions.parallel.map { form -> form.process()}.filter { it.valid }.collection
  35. 35. Frequent confusion
  36. 36. Improper use 1def accumulator = 0myCollection.eachParallel {accumulator += calculate(it)}
  37. 37. Improper use 2new File("/file.txt").withReader{reader ->reader.eachParallel {def r1 = step1(r)def r2 = step2(r1)def r3 = step3(r2)}}
  38. 38. Unroll iterationdef pipeline = data | step1 | step2 | step3new File("/file.txt").withReader{reader ->reader.each {data << it}}
  39. 39. Unroll iteration
  40. 40. ParallelismStreamed dataTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  41. 41. Pipeline DSLdef toProcess = new DataflowQueue()def validated = new DataflowQueue()toProcess | {form -> process(form)} |{processedForm -> validate(processedForm)} | validatedsubmissions.each {toProcess << it}
  42. 42. Url resolverUrl resolverUrl resolverprocessvalidateinProcessvalidated
  43. 43. Url resolverUrl resolverDownloaderGroovy scanner Scala scannerUrl resolverUrl resolverReporterSpeculatorEvaluatorSplitterConfirmCacheupdatesApprovals
  44. 44. Dataflow Operatorsoperator(inputs: [headers, bodies, footers],outputs: [articles, summaries]){header, body, footer ->def article = buildArticle(header, body, footer)bindOutput(0, article)bindOutput(1, buildSummary(article))}
  45. 45. ConclusionParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  46. 46. SummaryParallelism is not hard, multi-threading isJon Kerridge, Napier University
  47. 47. Questions?Find more at:http://gpars.codehaus.orghttp://www.jroller.com/vaclavhttp://twitter.com/vaclav_pech
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×