Gpars workshop

942 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
942
On SlideShare
0
From Embeds
0
Number of Embeds
151
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Gpars workshop

  1. 1. GPars workshopVáclav Pechhttp://jroller.com/vaclavRussel Winderhttp://www.russel.org.uk
  2. 2. Agenda● Threads, thread pools, tasks● Agents● Fork/Join● Parallel collections● Dataflow● Actorshttp://gpars.org/GPars_workshop.zip
  3. 3. We’re all in the parallel computing business!
  4. 4. Multithreaded programs today work mostly by accident!
  5. 5. Map/ReduceMap/ReduceFork/JoinFork/JoinActorsActorsSTMSTMDataflowDataflowAgentAgentCan we do better?
  6. 6. Part 1Thread management
  7. 7. Asynchronous invocationFuture f = threadPool.submit(calculation);…System.out.println(“Result: “ + f.get());
  8. 8. Async the Groovy waydef group = new NonDaemonPGroup(10)group.task {calculation.process()}
  9. 9. Part 2Agents
  10. 10. Shared Mutable StateMisused most of the timeWhen really needed, use Agents Software Transactional Memory Locks
  11. 11. Agent insideDouble IncrementAdd 25Message Queue36 thread
  12. 12. Agent Listdef jugMembers = new Agent([Me]) //add Metask {jugMembers.send {it.add Joe} //add Joe}task {jugMembers << {it.add Dave} //add DavejugMembers << {it.add Alice} //add Alice}...println jugMembers.val
  13. 13. Part 3Fork/Join
  14. 14. Thread PoolTasksWorker threadsQueue
  15. 15. Thread PoolContention!
  16. 16. Fork/Join Thread Pool
  17. 17. Fork/Join Thread PoolWork stealing
  18. 18. Fork/Join Solve hierarchicalproblems Divide and conquer Merge sort, Quick sort Tree traversal File scan / search …[a, b, c, d, e, f, g, h][a, b, c, d] [e, f, g, h][a, b] [c, d] [e, f] [g, h]
  19. 19. Fork/Join (GPars)runForkJoin(new File(“./src”)) {currentDir ->long count = 0;currentDir.eachFile {if (it.isDirectory()) {forkOffChild it} else {count++}}return count + childrenResults.sum(0)}Waits for childrenwithout blocking thethread!
  20. 20. Part 4Parallel collections
  21. 21. Parallel collectionsimages.eachParallel {it.process()}documents.sumParallel()candidates.maxParallel {it.salary}.marry()
  22. 22. Parallel collectionsregistrations = submissions.collectParallel { form -> form.process()}.findAllParallel { it.valid }registrations = submissions.parallel.map { form -> form.process()}.filter { it.valid }.collection
  23. 23. GPUJava watch list:https://github.com/pcpratts/rootbeer1/http://openjdk.java.net/projects/sumatra/
  24. 24. Part 5Dataflow
  25. 25. Composing async functionsint hash1 = hash(download(http://www.gpars.org))int hash2 = hash(loadFile(/gpars/website/index.html))boolean result = compare(hash1, hash2)println result
  26. 26. Composing async functions@AsyncFun hash = oldHash@AsyncFun compare = oldCompare@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFiledef hash1 = hash(download(http://www.gpars.org))def hash2 = hash(loadFile(/gpars/website/index.html))def result = compare(hash1, hash2)println result.get()
  27. 27. Composing async functions@AsyncFun hash = oldHash@AsyncFun(blocking = true) compare = oldCompare@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFiledef hash1 = hash(download(http://www.gpars.org))def hash2 = hash(loadFile(/gpars/website/index.html))boolean result = compare(hash1, hash2)println result
  28. 28. int hash(String text) {…}Promise<int> hash(Promise<String> | String text)
  29. 29. int hash(String text) {…}Promise<int> hash(Promise<String> | String text)compare(hash( download() ),hash( loadFile() ))
  30. 30. int hash(String text) {…}Promise<int> hash(Promise<String> | String text) {1.Return a Promise for the result2.Wait (non-blocking) for the text param3.Call the original hash()4.Bind the result}
  31. 31. Composing async functionsCombine functions as usualParallelism is detected automatically
  32. 32. Composing async functions@AsyncFun hash = oldHash@AsyncFun compare = oldCompare@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFiledef hash1 = hash(download(http://www.gpars.org))def hash2 = hash(loadFile(/gpars/website/index.html))def result = compare(hash1, hash2)println result.get()
  33. 33. Promise chaining@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFiledef h1 = download(http://www.gpars.org) then hashdef h2 = loadFile(/gpars/website/index.html) then hashwhenAllBound([h1, h2], compare) then {println it}
  34. 34. Promise chaining@AsyncFun download = oldDownload@AsyncFun loadFile = oldLoadFilewhenAllBound ([download(http://www.gpars.org) >> hash,loadFile(/gpars/website/index.html) >> hash], compare) >> {println it}
  35. 35. Dataflow Concurrency No race-conditions No live-locks Deterministic deadlocksCompletely deterministic programsBEAUTIFUL code(Jonas Bonér)
  36. 36. Dataflow Variables / Promisesmain task2 task3xyztask1
  37. 37. Dataflow Variables / Promisestask2xtask1
  38. 38. Promises to exchange datatask { z << x.val + y.val }task { x << 10 }task {println ”I am task 3”y << 5}assert 15 == z.val
  39. 39. Dataflow Variables / Promisestask2xtask1
  40. 40. Dataflow Channelstask2x|y|ztask1
  41. 41. Synchronous Channelstask2x|y|ztask1
  42. 42. operator(inputs: [headers, bodies, footers],outputs: [articles, summaries]){header, body, footer ->def article = buildArticle(header, body, footer)bindOutput(0, article)bindOutput(1, buildSummary(article))} *+<>Dataflow Operators
  43. 43. Url resolverUrl resolverDownloaderGroovy scanner Scala scannerUrl resolverUrl resolverReporterSpeculatorEvaluatorSplitterConfirmCacheupdatesApprovalsDataflow Operators
  44. 44. Part 6Actors
  45. 45. ActorsProcesses with a mail-boxShare no dataCommunicate by sending messagesUse a thread-pool
  46. 46. Actors Isolated CommunicatingImmutable messages ActivePooled shared threads ActivitiesCreate a new actorSend a messageReceive a messageActorActorActorActorActorActorActorTTTThread pool
  47. 47. Actors useGateKeeperFormHTTPFingerPrintsAddressCheckEmailCheckProcessFraudDetectResponseSOAPSMTP
  48. 48. Sending messagesbuddy.send 10.eurbuddy << new Book(title:’Groovy Recipes’,author:’Scott Davis’)def canChat = buddy.sendAndWait ‘Got time?’buddy.sendAndContinue ‘Need money!’, {cash->pocket.add cash}
  49. 49. Stateless Actors (pure Java)class MyActor extends DynamicDispatchActor {Account account = ...public void onMessage(String msg) {String encripted = encrypt(msg);reply(encripted);}public void onMessage(Integer number) {reply(2 * number);}public void onMessage(Money cash) {System.out.println("Received a donation " + cash);account.deposit(cash);}}
  50. 50. Stateful Actorsclass MyActor extends DefaultActor {void act() {def buddy = new YourActor()buddy << ‘Hi man, how’re things?’def response = receive()}}
  51. 51. Implicit State in Actorsval me = actor {react {msg1 ->switch (msg1) {case Work: reply “I dont work so early” ; stop();case Breakfast:msg1.eat()react {msg2 →switch (msg2) {case Work: reply “OK, time to work”; msg2.do()case Lunch: ...} } } }
  52. 52. Continuation Styleloop {…react {…react {/*schedule the block; throw CONTINUE*/…}//Never reached}//Never reached}//Never reached
  53. 53. Active Objects@ActiveObjectclass MyCounter {private int counter = 0@ActiveMethoddef incrementBy(int value) {println "Received an integer: $value"this.counter += value}}
  54. 54. coz concurrency is Groovy

×