Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
GPars workshopVáclav Pechhttp://jroller.com/vaclavRussel Winderhttp://www.russel.org.uk
Agenda● Threads, thread pools, tasks● Agents● Fork/Join● Parallel collections● Dataflow● Actorshttp://gpars.org/GPars_work...
We’re all in the parallel computing business!
Multithreaded programs today work mostly by accident!
Map/ReduceMap/ReduceFork/JoinFork/JoinActorsActorsSTMSTMDataflowDataflowAgentAgentCan we do better?
Part 1Thread management
Asynchronous invocationFuture f = threadPool.submit(calculation);…System.out.println(“Result: “ + f.get());
Async the Groovy waydef group = new NonDaemonPGroup(10)group.task {calculation.process()}
Part 2Agents
Shared Mutable StateMisused most of the timeWhen really needed, use Agents Software Transactional Memory Locks
Agent insideDouble IncrementAdd 25Message Queue36 thread
Agent Listdef jugMembers = new Agent([Me]) //add Metask {jugMembers.send {it.add Joe} //add Joe}task {jugMembers << {it.ad...
Part 3Fork/Join
Thread PoolTasksWorker threadsQueue
Thread PoolContention!
Fork/Join Thread Pool
Fork/Join Thread PoolWork stealing
Fork/Join Solve hierarchicalproblems Divide and conquer Merge sort, Quick sort Tree traversal File scan / search …[a...
Fork/Join (GPars)runForkJoin(new File(“./src”)) {currentDir ->long count = 0;currentDir.eachFile {if (it.isDirectory()) {f...
Part 4Parallel collections
Parallel collectionsimages.eachParallel {it.process()}documents.sumParallel()candidates.maxParallel {it.salary}.marry()
Parallel collectionsregistrations = submissions.collectParallel { form -> form.process()}.findAllParallel { it.valid }regi...
GPUJava watch list:https://github.com/pcpratts/rootbeer1/http://openjdk.java.net/projects/sumatra/
Part 5Dataflow
Composing async functionsint hash1 = hash(download(http://www.gpars.org))int hash2 = hash(loadFile(/gpars/website/index.ht...
Composing async functions@AsyncFun hash = oldHash@AsyncFun compare = oldCompare@AsyncFun download = oldDownload@AsyncFun l...
Composing async functions@AsyncFun hash = oldHash@AsyncFun(blocking = true) compare = oldCompare@AsyncFun download = oldDo...
int hash(String text) {…}Promise<int> hash(Promise<String> | String text)
int hash(String text) {…}Promise<int> hash(Promise<String> | String text)compare(hash( download() ),hash( loadFile() ))
int hash(String text) {…}Promise<int> hash(Promise<String> | String text) {1.Return a Promise for the result2.Wait (non-bl...
Composing async functionsCombine functions as usualParallelism is detected automatically
Composing async functions@AsyncFun hash = oldHash@AsyncFun compare = oldCompare@AsyncFun download = oldDownload@AsyncFun l...
Promise chaining@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFiledef h1 = download(http://www.gpars.org) th...
Promise chaining@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFilewhenAllBound ([download(http://www.gpars.o...
Dataflow Concurrency No race-conditions No live-locks Deterministic deadlocksCompletely deterministic programsBEAUTIFUL...
Dataflow Variables / Promisesmain task2 task3xyztask1
Dataflow Variables / Promisestask2xtask1
Promises to exchange datatask { z << x.val + y.val }task { x << 10 }task {println ”I am task 3”y << 5}assert 15 == z.val
Dataflow Variables / Promisestask2xtask1
Dataflow Channelstask2x|y|ztask1
Synchronous Channelstask2x|y|ztask1
operator(inputs: [headers, bodies, footers],outputs: [articles, summaries]){header, body, footer ->def article = buildArti...
Url resolverUrl resolverDownloaderGroovy scanner Scala scannerUrl resolverUrl resolverReporterSpeculatorEvaluatorSplitterC...
Part 6Actors
ActorsProcesses with a mail-boxShare no dataCommunicate by sending messagesUse a thread-pool
Actors Isolated CommunicatingImmutable messages ActivePooled shared threads ActivitiesCreate a new actorSend a mes...
Actors useGateKeeperFormHTTPFingerPrintsAddressCheckEmailCheckProcessFraudDetectResponseSOAPSMTP
Sending messagesbuddy.send 10.eurbuddy << new Book(title:’Groovy Recipes’,author:’Scott Davis’)def canChat = buddy.sendAnd...
Stateless Actors (pure Java)class MyActor extends DynamicDispatchActor {Account account = ...public void onMessage(String ...
Stateful Actorsclass MyActor extends DefaultActor {void act() {def buddy = new YourActor()buddy << ‘Hi man, how’re things?...
Implicit State in Actorsval me = actor {react {msg1 ->switch (msg1) {case Work: reply “I dont work so early” ; stop();case...
Continuation Styleloop {…react {…react {/*schedule the block; throw CONTINUE*/…}//Never reached}//Never reached}//Never re...
Active Objects@ActiveObjectclass MyCounter {private int counter = 0@ActiveMethoddef incrementBy(int value) {println "Recei...
coz concurrency is Groovy
Upcoming SlideShare
Loading in …5
×

Gpars workshop

1,057 views

Published on

  • Be the first to comment

  • Be the first to like this

Gpars workshop

  1. 1. GPars workshopVáclav Pechhttp://jroller.com/vaclavRussel Winderhttp://www.russel.org.uk
  2. 2. Agenda● Threads, thread pools, tasks● Agents● Fork/Join● Parallel collections● Dataflow● Actorshttp://gpars.org/GPars_workshop.zip
  3. 3. We’re all in the parallel computing business!
  4. 4. Multithreaded programs today work mostly by accident!
  5. 5. Map/ReduceMap/ReduceFork/JoinFork/JoinActorsActorsSTMSTMDataflowDataflowAgentAgentCan we do better?
  6. 6. Part 1Thread management
  7. 7. Asynchronous invocationFuture f = threadPool.submit(calculation);…System.out.println(“Result: “ + f.get());
  8. 8. Async the Groovy waydef group = new NonDaemonPGroup(10)group.task {calculation.process()}
  9. 9. Part 2Agents
  10. 10. Shared Mutable StateMisused most of the timeWhen really needed, use Agents Software Transactional Memory Locks
  11. 11. Agent insideDouble IncrementAdd 25Message Queue36 thread
  12. 12. Agent Listdef jugMembers = new Agent([Me]) //add Metask {jugMembers.send {it.add Joe} //add Joe}task {jugMembers << {it.add Dave} //add DavejugMembers << {it.add Alice} //add Alice}...println jugMembers.val
  13. 13. Part 3Fork/Join
  14. 14. Thread PoolTasksWorker threadsQueue
  15. 15. Thread PoolContention!
  16. 16. Fork/Join Thread Pool
  17. 17. Fork/Join Thread PoolWork stealing
  18. 18. Fork/Join Solve hierarchicalproblems Divide and conquer Merge sort, Quick sort Tree traversal File scan / search …[a, b, c, d, e, f, g, h][a, b, c, d] [e, f, g, h][a, b] [c, d] [e, f] [g, h]
  19. 19. Fork/Join (GPars)runForkJoin(new File(“./src”)) {currentDir ->long count = 0;currentDir.eachFile {if (it.isDirectory()) {forkOffChild it} else {count++}}return count + childrenResults.sum(0)}Waits for childrenwithout blocking thethread!
  20. 20. Part 4Parallel collections
  21. 21. Parallel collectionsimages.eachParallel {it.process()}documents.sumParallel()candidates.maxParallel {it.salary}.marry()
  22. 22. Parallel collectionsregistrations = submissions.collectParallel { form -> form.process()}.findAllParallel { it.valid }registrations = submissions.parallel.map { form -> form.process()}.filter { it.valid }.collection
  23. 23. GPUJava watch list:https://github.com/pcpratts/rootbeer1/http://openjdk.java.net/projects/sumatra/
  24. 24. Part 5Dataflow
  25. 25. Composing async functionsint hash1 = hash(download(http://www.gpars.org))int hash2 = hash(loadFile(/gpars/website/index.html))boolean result = compare(hash1, hash2)println result
  26. 26. Composing async functions@AsyncFun hash = oldHash@AsyncFun compare = oldCompare@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFiledef hash1 = hash(download(http://www.gpars.org))def hash2 = hash(loadFile(/gpars/website/index.html))def result = compare(hash1, hash2)println result.get()
  27. 27. Composing async functions@AsyncFun hash = oldHash@AsyncFun(blocking = true) compare = oldCompare@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFiledef hash1 = hash(download(http://www.gpars.org))def hash2 = hash(loadFile(/gpars/website/index.html))boolean result = compare(hash1, hash2)println result
  28. 28. int hash(String text) {…}Promise<int> hash(Promise<String> | String text)
  29. 29. int hash(String text) {…}Promise<int> hash(Promise<String> | String text)compare(hash( download() ),hash( loadFile() ))
  30. 30. int hash(String text) {…}Promise<int> hash(Promise<String> | String text) {1.Return a Promise for the result2.Wait (non-blocking) for the text param3.Call the original hash()4.Bind the result}
  31. 31. Composing async functionsCombine functions as usualParallelism is detected automatically
  32. 32. Composing async functions@AsyncFun hash = oldHash@AsyncFun compare = oldCompare@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFiledef hash1 = hash(download(http://www.gpars.org))def hash2 = hash(loadFile(/gpars/website/index.html))def result = compare(hash1, hash2)println result.get()
  33. 33. Promise chaining@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFiledef h1 = download(http://www.gpars.org) then hashdef h2 = loadFile(/gpars/website/index.html) then hashwhenAllBound([h1, h2], compare) then {println it}
  34. 34. Promise chaining@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFilewhenAllBound ([download(http://www.gpars.org) >> hash,loadFile(/gpars/website/index.html) >> hash], compare) >> {println it}
  35. 35. Dataflow Concurrency No race-conditions No live-locks Deterministic deadlocksCompletely deterministic programsBEAUTIFUL code(Jonas Bonér)
  36. 36. Dataflow Variables / Promisesmain task2 task3xyztask1
  37. 37. Dataflow Variables / Promisestask2xtask1
  38. 38. Promises to exchange datatask { z << x.val + y.val }task { x << 10 }task {println ”I am task 3”y << 5}assert 15 == z.val
  39. 39. Dataflow Variables / Promisestask2xtask1
  40. 40. Dataflow Channelstask2x|y|ztask1
  41. 41. Synchronous Channelstask2x|y|ztask1
  42. 42. operator(inputs: [headers, bodies, footers],outputs: [articles, summaries]){header, body, footer ->def article = buildArticle(header, body, footer)bindOutput(0, article)bindOutput(1, buildSummary(article))} *+<>Dataflow Operators
  43. 43. Url resolverUrl resolverDownloaderGroovy scanner Scala scannerUrl resolverUrl resolverReporterSpeculatorEvaluatorSplitterConfirmCacheupdatesApprovalsDataflow Operators
  44. 44. Part 6Actors
  45. 45. ActorsProcesses with a mail-boxShare no dataCommunicate by sending messagesUse a thread-pool
  46. 46. Actors Isolated CommunicatingImmutable messages ActivePooled shared threads ActivitiesCreate a new actorSend a messageReceive a messageActorActorActorActorActorActorActorTTTThread pool
  47. 47. Actors useGateKeeperFormHTTPFingerPrintsAddressCheckEmailCheckProcessFraudDetectResponseSOAPSMTP
  48. 48. Sending messagesbuddy.send 10.eurbuddy << new Book(title:’Groovy Recipes’,author:’Scott Davis’)def canChat = buddy.sendAndWait ‘Got time?’buddy.sendAndContinue ‘Need money!’, {cash->pocket.add cash}
  49. 49. Stateless Actors (pure Java)class MyActor extends DynamicDispatchActor {Account account = ...public void onMessage(String msg) {String encripted = encrypt(msg);reply(encripted);}public void onMessage(Integer number) {reply(2 * number);}public void onMessage(Money cash) {System.out.println("Received a donation " + cash);account.deposit(cash);}}
  50. 50. Stateful Actorsclass MyActor extends DefaultActor {void act() {def buddy = new YourActor()buddy << ‘Hi man, how’re things?’def response = receive()}}
  51. 51. Implicit State in Actorsval me = actor {react {msg1 ->switch (msg1) {case Work: reply “I dont work so early” ; stop();case Breakfast:msg1.eat()react {msg2 →switch (msg2) {case Work: reply “OK, time to work”; msg2.do()case Lunch: ...} } } }
  52. 52. Continuation Styleloop {…react {…react {/*schedule the block; throw CONTINUE*/…}//Never reached}//Never reached}//Never reached
  53. 53. Active Objects@ActiveObjectclass MyCounter {private int counter = 0@ActiveMethoddef incrementBy(int value) {println "Received an integer: $value"this.counter += value}}
  54. 54. coz concurrency is Groovy

×