Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Gpars concepts explained

8,239 views

Published on

Shows and explains the basic concepts and features of the GPars Groovy concurrency library - http://gpars.codehaus.org .

Published in: Education, Technology

Gpars concepts explained

  1. 1. <ul>GPars </ul><ul>Groovy Parallel Systems Václav Pech </ul>
  2. 2. <ul>About me </ul><ul><li>Passionate programmer
  3. 3. Concurrency enthusiast
  4. 4. GPars @ Codehaus lead
  5. 5. Technology evangelist @ JetBrains
  6. 6. JetBrains Academy member </li></ul><ul>http://www.jroller.com/vaclav </ul><ul><ul><li>http://twitter.com/vaclav_pech </li></ul></ul>
  7. 7. <ul>GPars </ul><ul>Apache 2 license Hosted @ Codehaus 7 commiters Tight relation with the Groovy team </ul><ul><ul><li>Originally named GParallelizer </li></ul></ul>
  8. 8. <ul>Pantha rei </ul><ul>Entering a new era! </ul>
  9. 9. <ul>We’re quad core already </ul><ul>Beware: More cores to come shortly! </ul>
  10. 10. <ul>SW is being late to the game </ul>Dead-locks Live-locks Race conditions Starvation Shared Mutable State
  11. 11. <ul>Lock and synchronize </ul><ul>Multi-threaded programs today work mostly by accident! </ul>
  12. 12. <ul>Toolset </ul><ul>Fork/Join </ul><ul>Actors </ul><ul>Collections </ul><ul>Dataflow </ul><ul>Safe </ul>
  13. 13. <ul>Parallel Collection Processing </ul><ul>withPool { images. findAllParallel {it.contains me} . collectParallel {convert it} . groupByParallel {it.size()} } </ul>Important: <ul><ul><li>Side-effect-free functions only! </li></ul></ul>
  14. 14. <ul>Transparently parallel </ul><ul>withPool ( 4 ) { images. makeTransparent() . findAll {it.contains me} . collect {convert it} . groupBy {it.size()} } </ul>
  15. 15. <ul>Parallelize existing code </ul><ul>buyBargain stocks. makeTransparent() </ul><ul>def buyBargain(stocks) { buy stocks. findAll { it.download().price < 50 } } Use with CAUTION! </ul>
  16. 16. <ul>Functional flavor </ul><ul>Map / Reduce </ul><ul><ul><li>map, reduce, filter, min, max, sum, … </li></ul></ul><ul>(1..n).parallel.filter {it%2==0} .map {it ** 2} .reduce {a, b -> a + b} </ul>
  17. 17. <ul>Call closures asynchronously </ul><ul>def isSelfPortrait = {image -> image.contains me} </ul>SYNC: <ul>def flag = isSelfPortrait. call (img1) </ul>ASYNC: <ul>def future = isSelfPortrait. callAsync (img1) … def flag = future.get() </ul>
  18. 18. <ul>Asynchronous closures </ul><ul>def resize = {img -> img.resize(64, 64)} def fastResize = resize. async() def resized = images.collect fastResize … createAlbum resized*.get() </ul>
  19. 19. <ul>Fork/Join </ul><ul><li>Solve hierarchical problems </li></ul><ul><ul><ul><li>Divide and conquer </li></ul><li>Merge sort, Quick sort
  20. 20. Tree traversal
  21. 21. File scan / search
  22. 22. … </li></ul></ul><ul>[a, b, c, d, e, f, g, h] </ul><ul>[a, b, c, d] </ul><ul>[e, f, g, h] </ul><ul>[a, b] </ul><ul>[c, d] </ul><ul>[e, f] </ul><ul>[g, h] </ul>
  23. 23. <ul>Fork/Join Orchestration </ul><ul>runForkJoin(new File( “./src” )) {currentDir -> long count = 0; currentDir .eachFile { if (it.isDirectory()) { println &quot;Forking a thread for $it&quot; forkOffChild it } else { count++ } } return count + childrenResults .sum(0) } </ul><ul>Waits for children without blocking the thread! </ul>
  24. 24. <ul>GroovyDSL – IntelliJ IDEA CE </ul>
  25. 25. <ul>Dataflow Concurrency </ul><ul><li>No race-conditions
  26. 26. No live-locks
  27. 27. Deterministic deadlocks </li></ul><ul><ul><li>Completely deterministic programs </li></ul></ul><ul>BEAUTIFUL code </ul><ul><ul><li>(Jonas Bonér) </li></ul></ul>
  28. 28. <ul>Dataflow Variables / Promises </ul><ul>main </ul><ul>task2 </ul><ul>task3 </ul><ul>x </ul><ul>y </ul><ul>z </ul><ul>task1 </ul>
  29. 29. <ul>Dataflow Variables </ul><ul>task { z << x.val + y.val } task { x << 40 } task { y << 2 } assert 42 == z.val </ul><ul><li>Single-assignment variables </li></ul><ul><ul><li>with blocking read </li></ul></ul>
  30. 30. <ul>DataFlows </ul><ul>def df = new DataFlows () task { df .z = df .x + df .y } task { df .x = 10 } task { df .y = 5 } assert 15 == df .z </ul>
  31. 31. <ul>DataFlows Making Money </ul><ul>def stocks = ['AAPL', 'GOOG', 'IBM', 'JAVA'] def price = new DataFlows() stocks.each( {stock -> price [stock] = getClosing(stock, 2008) }.async()) def topStock = stocks.max { price [it] } </ul>
  32. 32. <ul>Dataflow Operators </ul><ul>operator(inputs: [stocksStream], outputs: [pricedStocks]) { stock -> def price = getClosing( stock , 2008) bindOutput(0, [stock: stock , price: price ]) } </ul><ul>* </ul><ul>+ </ul><ul><> </ul>
  33. 33. <ul>Actors </ul><ul><li>Isolated
  34. 34. Communicating </li></ul><ul><ul><li>Immutable messages </li></ul></ul><ul><li>Active </li></ul><ul><ul><li>Pooled shared threads </li></ul></ul><ul><li>Activities </li></ul><ul><ul><li>Create a new actor
  35. 35. Send a message
  36. 36. Receive a message </li></ul></ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>T </ul><ul>T </ul><ul>T </ul><ul>Thread pool </ul>
  37. 37. <ul>Actors use </ul><ul>Gate Keeper </ul><ul>Form </ul><ul>HTTP </ul><ul>Finger Prints </ul><ul>Address Check </ul><ul>Email Check </ul><ul>Process </ul><ul>Fraud Detect </ul><ul>Response </ul><ul>SOAP </ul><ul>SMTP </ul>
  38. 38. <ul>Actors patterns </ul><ul>Enricher Router Translator Endpoint Splitter Agregator Filter Resequencer Checker </ul>
  39. 39. <ul>Creating Actors </ul><ul>class MyActor extends AbstractPooledActor { void act() { def buddy = new YourActor() buddy << ‘Hi man, how’re things?’ def response = receive () } } </ul>
  40. 40. <ul>Creating Actors </ul><ul>def decryptor = actor { loop { react {msg -> reply msg.reverse() } } } decryptor << ‘noitcA nI yvoorG’ </ul>
  41. 41. <ul>Sending messages </ul><ul>buddy. send 10.eur buddy << new Book(title: ’Groovy Recipes’ , author: ’Scott Davis’ ) def canChat = buddy. sendAndWait ‘Got time?’ buddy. sendAndContinue ‘Need money!’ , {cash-> pocket.add cash } </ul>
  42. 42. <ul>Reacting to messages </ul><ul>react {gift1 -> reply ”Thank you for $gift1” react {gift2 -> reply ”Wow, one more $gift2” [gift1, gift2].max{it.price}. reply ‘ Will you marry me?’ } //Never reached } </ul>
  43. 43. <ul>Choosing the Reaction </ul><ul>react / receive {gift -> switch (gift) { case Money: reply ‘Thanks’ ;pocket gift; break case [iPhone, iPod]:child << gift; break case (BigFlat..SmallHouse):moveIn(gift); break case Clothes: putOn(gift) fits(gift)?reply ‘Thanks’ : reply gift break case EXIT: stop() } } </ul>
  44. 44. <ul>Continuation Style </ul><ul>loop { … react { … react { /*schedule the block; throw CONTINUE*/ … } //Never reached } //Never reached } //Never reached </ul>
  45. 45. <ul>Receiving messages </ul><ul>def gift = receive () reply ”Thank you for $gift” gift = receive () reply ”Wow, one more $gift” </ul><ul><li>Blocks the calling thread </li></ul>
  46. 46. <ul>Dynamic Dispatch Actor </ul><ul>def actor = new DynamicDispatchActor ({ when {BigDecimal num -> println 'Received BigDecimal' } when {String code -> compileAndRun code } when {Book book -> read book } }) </ul>
  47. 47. <ul>Safe (Agent) </ul><ul><li>Lock Shared Mutable State in a Safe </li></ul><ul>Alter the State </ul><ul>Alter the State </ul>
  48. 48. <ul>Safe inside </ul><ul>Double </ul><ul>Increment </ul><ul>Add 25 </ul><ul>Message Queue </ul><ul>36 </ul><ul>thread </ul>
  49. 49. <ul>Safe List </ul><ul>def jugMembers = new Safe( ['Me'] ) //add Me task { jugMembers.send {it.add 'Joe'} //add Joe } task { jugMembers << {it.add 'Dave'} //add Dave jugMembers << {it.add 'Alice'} //add Alice } ... println jugMembers. val </ul>
  50. 50. <ul>Safe-based Shopping Cart </ul><ul><li>Cart cart = new Cart()
  51. 51. cart.addItem 'Pilsner', 10
  52. 52. cart.addItem 'Budweisser', 5
  53. 53. cart.removeItem 'Budweisser‘
  54. 54. cart.increaseQuantity 'Budweisser', 3 </li></ul>
  55. 55. <ul>Cart implementation </ul><ul>class ShoppingCart { private def cartState = new Safe( [:] ) public void addItem(String product, int quantity) { cartState << {it[product] = quantity} } public void removeItem(String product) { cartState << {it.remove(product)} } public Object listContent() { return cartState. val } </ul>
  56. 56. <ul>Integration </ul><ul><li>Bundled with Groovy dists
  57. 57. Maven
  58. 58. Gradle
  59. 59. Grape (@Grab)
  60. 60. Griffon plugin
  61. 61. Grails plugin </li></ul>
  62. 62. <ul>Roadmap </ul>Actor performance Actor and DataFlow remoting CSP implementation STM?, ActiveObjects?
  63. 63. <ul>No more threads and locks </ul><ul>images.parallel. map { //concurrency agnostic code here } def myActor = actor { //concurrency agnostic code here } Identically with agents, STM, dataflow, … </ul>
  64. 64. <ul>Summary </ul><ul>Tasty concurrency menu </ul><ul><ul><li>Actors, Collections, Dataflow, Safe, … </li></ul></ul><ul>Enjoyable parallelism http://gpars.codehaus.org </ul>
  65. 65. <ul>Questions? </ul>

×