Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

GPars howto - when to use which concurrency abstraction

1,891 views

Published on

Published in: Technology
  • Be the first to comment

GPars howto - when to use which concurrency abstraction

  1. 1. GParscoz concurrency is GroovyVáclav Pech
  2. 2. Asynchronous calculationsFork/JoinParallel collectionsActorsAgents, StmDataflowCSPWhy this talk?
  3. 3. StructureParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  4. 4. StructureParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  5. 5. ParallelismTask parallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  6. 6. Asynchronous invocationFuture f = threadPool.submit(calculation);…System.out.println(“Result: “ + f.get());
  7. 7. Async the Groovy waytask {calculation.process()}
  8. 8. Async the Groovy waydef group = new NonDaemonPGroup(10)group.task {calculation.process()}
  9. 9. group.task {->…}group.task new Runnable() {…}group.task new Callable<V>() {...}Async the Groovy way
  10. 10. Independent tasksdef group = new NonDaemonPGroup(10)submissions.each {form →group.task {form.process()}}
  11. 11. Dependent tasksParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  12. 12. State sharingParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  13. 13. State sharingList registrations = []submissions.each {form →group.task {if (form.process().valid) {registrations << form}}}
  14. 14. State sharingList registrations = []submissions.each {form →group.task {if (form.process().valid) {registrations << form}}}Needs protection
  15. 15. Agent Lock Shared Mutable State in a SafeAltertheStateAltertheState
  16. 16. Agent insideDouble IncrementAdd 25Message Queue36 thread
  17. 17. Sharing through agentsAgent registrations = group.agent( [] )submissions.each {form →group.task {if (form.process().valid) {registrations.send {it << form}}}}
  18. 18. Random task dependencyParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  19. 19. Dataflow ConcurrencyNo race-conditionsNo live-locksDeterministic deadlocks
  20. 20. Dataflow Variables / Promisesmain task2 task3xyztask1
  21. 21. Dataflow Variables / Promisestask2xtask1
  22. 22. Dataflow Variables / Promisesmain task2 task3xyztask1
  23. 23. Chaining promisesdef h1 = download(url) then {text → text.trim()} then hash
  24. 24. Error handlingurl.then(download).then(calculateHash).then(formatResult).then(printResult, printError).then(sendNotificationEmail);
  25. 25. List registrations = []List<Promise> forms=submissions.collect {form →group.task {if (form.process().valid) return form}}forms.each{registrations << it.get()}Pass results around
  26. 26. Dataflow Variables / Promisestask2xtask1
  27. 27. Dataflow Channelstask2x|y|ztask1
  28. 28. Synchronous Channelstask2x|y|ztask1
  29. 29. Tasks with progress indicationList<Promise> forms=submissions.collect {form →group.task {def result = form.process()progressQueue << 1if (result.valid) {return form}}}
  30. 30. Channel SelectionSelect alt = group.select(validForms, invalidForms)SelectResult selectResult = alt.select() //alt.prioritySelect()switch (selectResult.index) {case 0: registrations << selectResult.value; breakcase 1: ...}
  31. 31. ParallelismData parallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  32. 32. ParallelismData parallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  33. 33. Geometric decompositionimages.eachParallel {it.process()}documents.sumParallel()candidates.maxParallel {it.salary}.marry()
  34. 34. Geometric decompositionregistrations = submissions.collectParallel { form -> form.process()}.findAllParallel { it.valid }registrations = submissions.parallel.map { form -> form.process()}.filter { it.valid }.collection
  35. 35. Frequent confusion
  36. 36. Improper use 1def accumulator = 0myCollection.eachParallel {accumulator += calculate(it)}
  37. 37. Improper use 2new File("/file.txt").withReader{reader ->reader.eachParallel {def r1 = step1(r)def r2 = step2(r1)def r3 = step3(r2)}}
  38. 38. Unroll iterationdef pipeline = data | step1 | step2 | step3new File("/file.txt").withReader{reader ->reader.each {data << it}}
  39. 39. Unroll iteration
  40. 40. ParallelismStreamed dataTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  41. 41. Pipeline DSLdef toProcess = new DataflowQueue()def validated = new DataflowQueue()toProcess | {form -> process(form)} |{processedForm -> validate(processedForm)} | validatedsubmissions.each {toProcess << it}
  42. 42. Url resolverUrl resolverUrl resolverprocessvalidateinProcessvalidated
  43. 43. Url resolverUrl resolverDownloaderGroovy scanner Scala scannerUrl resolverUrl resolverReporterSpeculatorEvaluatorSplitterConfirmCacheupdatesApprovals
  44. 44. Dataflow Operatorsoperator(inputs: [headers, bodies, footers],outputs: [articles, summaries]){header, body, footer ->def article = buildArticle(header, body, footer)bindOutput(0, article)bindOutput(1, buildSummary(article))}
  45. 45. ConclusionParallelismTask DataRandomIndependent DependentHierarchicalGeometricalStreamedEvent-drivenFunctionalHierarchicalShared state
  46. 46. SummaryParallelism is not hard, multi-threading isJon Kerridge, Napier University
  47. 47. Questions?Find more at:http://gpars.codehaus.orghttp://www.jroller.com/vaclavhttp://twitter.com/vaclav_pech

×