Your SlideShare is downloading. ×
0
<ul>GPars </ul><ul>Groovy Parallel Systems Václav Pech </ul>
<ul>About me </ul><ul><li>Passionate programmer
Concurrency enthusiast
GPars @ Codehaus lead
Technology evangelist @ JetBrains
JetBrains Academy member </li></ul><ul>http://www.jroller.com/vaclav   </ul><ul><ul><li>http://twitter.com/vaclav_pech </l...
<ul>GPars </ul><ul>Apache 2 license Hosted @ Codehaus 7 commiters Tight relation with the Groovy team </ul><ul><ul><li>Ori...
<ul>Pantha rei </ul><ul>Entering a new era! </ul>
<ul>We’re quad core already </ul><ul>Beware: More cores to come shortly! </ul>
<ul>SW is being late to the game </ul>Dead-locks Live-locks Race conditions Starvation Shared Mutable State
<ul>Lock and synchronize </ul><ul>Multi-threaded programs today work mostly  by accident! </ul>
<ul>Toolset </ul><ul>Fork/Join </ul><ul>Actors </ul><ul>Collections </ul><ul>Dataflow </ul><ul>Safe </ul>
<ul>Parallel Collection Processing </ul><ul>withPool  { images. findAllParallel {it.contains me} . collectParallel {conver...
<ul>Transparently parallel </ul><ul>withPool ( 4 ) { images. makeTransparent() . findAll  {it.contains me}   . collect  {c...
<ul>Parallelize existing code </ul><ul>buyBargain stocks. makeTransparent() </ul><ul>def buyBargain(stocks) { buy stocks. ...
<ul>Functional flavor </ul><ul>Map / Reduce </ul><ul><ul><li>map, reduce, filter, min, max, sum, … </li></ul></ul><ul>(1.....
<ul>Call closures asynchronously </ul><ul>def isSelfPortrait = {image -> image.contains me} </ul>SYNC: <ul>def flag = isSe...
<ul>Asynchronous closures </ul><ul>def  resize  = {img -> img.resize(64, 64)} def  fastResize  = resize. async() def resiz...
<ul>Fork/Join </ul><ul><li>Solve hierarchical problems </li></ul><ul><ul><ul><li>Divide and conquer </li></ul><li>Merge so...
Tree traversal
File scan / search
… </li></ul></ul><ul>[a, b, c, d, e, f, g, h] </ul><ul>[a, b, c, d] </ul><ul>[e, f, g, h] </ul><ul>[a, b] </ul><ul>[c, d] ...
<ul>Fork/Join Orchestration </ul><ul>runForkJoin(new File( “./src” )) {currentDir -> long count = 0; currentDir .eachFile ...
<ul>GroovyDSL – IntelliJ IDEA CE </ul>
<ul>Dataflow Concurrency </ul><ul><li>No race-conditions
No live-locks
Deterministic deadlocks  </li></ul><ul><ul><li>Completely deterministic programs  </li></ul></ul><ul>BEAUTIFUL code </ul><...
<ul>Dataflow Variables / Promises </ul><ul>main </ul><ul>task2 </ul><ul>task3 </ul><ul>x </ul><ul>y </ul><ul>z </ul><ul>ta...
<ul>Dataflow Variables </ul><ul>task { z << x.val + y.val } task { x << 40 } task { y << 2 } assert 42 == z.val </ul><ul><...
<ul>DataFlows </ul><ul>def  df  = new  DataFlows () task {  df .z =  df .x +  df .y } task {  df .x = 10 } task {  df .y =...
<ul>DataFlows Making Money </ul><ul>def stocks = ['AAPL', 'GOOG', 'IBM', 'JAVA'] def  price  = new DataFlows() stocks.each...
<ul>Dataflow Operators </ul><ul>operator(inputs: [stocksStream], outputs: [pricedStocks])  { stock  -> def  price  = getCl...
<ul>Actors </ul><ul><li>Isolated
Communicating </li></ul><ul><ul><li>Immutable messages </li></ul></ul><ul><li>Active </li></ul><ul><ul><li>Pooled shared t...
Send a message
Receive a message </li></ul></ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul...
<ul>Actors use </ul><ul>Gate Keeper </ul><ul>Form </ul><ul>HTTP </ul><ul>Finger Prints </ul><ul>Address Check </ul><ul>Ema...
<ul>Actors patterns </ul><ul>Enricher Router Translator Endpoint Splitter Agregator Filter Resequencer Checker </ul>
<ul>Creating Actors </ul><ul>class MyActor extends  AbstractPooledActor  { void  act()  { def buddy =  new  YourActor() bu...
<ul>Creating Actors </ul><ul>def decryptor =  actor  { loop  { react  {msg -> reply  msg.reverse() } } } decryptor  <<   ‘...
<ul>Sending messages </ul><ul>buddy. send  10.eur buddy  <<  new Book(title: ’Groovy Recipes’ ,  author: ’Scott Davis’ ) d...
<ul>Reacting to messages </ul><ul>react  {gift1 ->  reply   ”Thank you for $gift1” react  {gift2 -> reply   ”Wow, one more...
<ul>Choosing the Reaction </ul><ul>react / receive  {gift -> switch (gift) { case Money: reply   ‘Thanks’ ;pocket gift; br...
<ul>Continuation Style </ul><ul>loop  { … react  { … react  { /*schedule the block; throw CONTINUE*/ …  } //Never reached ...
Upcoming SlideShare
Loading in...5
×

Gpars concepts explained

7,514

Published on

Shows and explains the basic concepts and features of the GPars Groovy concurrency library - http://gpars.codehaus.org .

Published in: Education, Technology

Transcript of "Gpars concepts explained"

  1. 1. <ul>GPars </ul><ul>Groovy Parallel Systems Václav Pech </ul>
  2. 2. <ul>About me </ul><ul><li>Passionate programmer
  3. 3. Concurrency enthusiast
  4. 4. GPars @ Codehaus lead
  5. 5. Technology evangelist @ JetBrains
  6. 6. JetBrains Academy member </li></ul><ul>http://www.jroller.com/vaclav </ul><ul><ul><li>http://twitter.com/vaclav_pech </li></ul></ul>
  7. 7. <ul>GPars </ul><ul>Apache 2 license Hosted @ Codehaus 7 commiters Tight relation with the Groovy team </ul><ul><ul><li>Originally named GParallelizer </li></ul></ul>
  8. 8. <ul>Pantha rei </ul><ul>Entering a new era! </ul>
  9. 9. <ul>We’re quad core already </ul><ul>Beware: More cores to come shortly! </ul>
  10. 10. <ul>SW is being late to the game </ul>Dead-locks Live-locks Race conditions Starvation Shared Mutable State
  11. 11. <ul>Lock and synchronize </ul><ul>Multi-threaded programs today work mostly by accident! </ul>
  12. 12. <ul>Toolset </ul><ul>Fork/Join </ul><ul>Actors </ul><ul>Collections </ul><ul>Dataflow </ul><ul>Safe </ul>
  13. 13. <ul>Parallel Collection Processing </ul><ul>withPool { images. findAllParallel {it.contains me} . collectParallel {convert it} . groupByParallel {it.size()} } </ul>Important: <ul><ul><li>Side-effect-free functions only! </li></ul></ul>
  14. 14. <ul>Transparently parallel </ul><ul>withPool ( 4 ) { images. makeTransparent() . findAll {it.contains me} . collect {convert it} . groupBy {it.size()} } </ul>
  15. 15. <ul>Parallelize existing code </ul><ul>buyBargain stocks. makeTransparent() </ul><ul>def buyBargain(stocks) { buy stocks. findAll { it.download().price < 50 } } Use with CAUTION! </ul>
  16. 16. <ul>Functional flavor </ul><ul>Map / Reduce </ul><ul><ul><li>map, reduce, filter, min, max, sum, … </li></ul></ul><ul>(1..n).parallel.filter {it%2==0} .map {it ** 2} .reduce {a, b -> a + b} </ul>
  17. 17. <ul>Call closures asynchronously </ul><ul>def isSelfPortrait = {image -> image.contains me} </ul>SYNC: <ul>def flag = isSelfPortrait. call (img1) </ul>ASYNC: <ul>def future = isSelfPortrait. callAsync (img1) … def flag = future.get() </ul>
  18. 18. <ul>Asynchronous closures </ul><ul>def resize = {img -> img.resize(64, 64)} def fastResize = resize. async() def resized = images.collect fastResize … createAlbum resized*.get() </ul>
  19. 19. <ul>Fork/Join </ul><ul><li>Solve hierarchical problems </li></ul><ul><ul><ul><li>Divide and conquer </li></ul><li>Merge sort, Quick sort
  20. 20. Tree traversal
  21. 21. File scan / search
  22. 22. … </li></ul></ul><ul>[a, b, c, d, e, f, g, h] </ul><ul>[a, b, c, d] </ul><ul>[e, f, g, h] </ul><ul>[a, b] </ul><ul>[c, d] </ul><ul>[e, f] </ul><ul>[g, h] </ul>
  23. 23. <ul>Fork/Join Orchestration </ul><ul>runForkJoin(new File( “./src” )) {currentDir -> long count = 0; currentDir .eachFile { if (it.isDirectory()) { println &quot;Forking a thread for $it&quot; forkOffChild it } else { count++ } } return count + childrenResults .sum(0) } </ul><ul>Waits for children without blocking the thread! </ul>
  24. 24. <ul>GroovyDSL – IntelliJ IDEA CE </ul>
  25. 25. <ul>Dataflow Concurrency </ul><ul><li>No race-conditions
  26. 26. No live-locks
  27. 27. Deterministic deadlocks </li></ul><ul><ul><li>Completely deterministic programs </li></ul></ul><ul>BEAUTIFUL code </ul><ul><ul><li>(Jonas Bonér) </li></ul></ul>
  28. 28. <ul>Dataflow Variables / Promises </ul><ul>main </ul><ul>task2 </ul><ul>task3 </ul><ul>x </ul><ul>y </ul><ul>z </ul><ul>task1 </ul>
  29. 29. <ul>Dataflow Variables </ul><ul>task { z << x.val + y.val } task { x << 40 } task { y << 2 } assert 42 == z.val </ul><ul><li>Single-assignment variables </li></ul><ul><ul><li>with blocking read </li></ul></ul>
  30. 30. <ul>DataFlows </ul><ul>def df = new DataFlows () task { df .z = df .x + df .y } task { df .x = 10 } task { df .y = 5 } assert 15 == df .z </ul>
  31. 31. <ul>DataFlows Making Money </ul><ul>def stocks = ['AAPL', 'GOOG', 'IBM', 'JAVA'] def price = new DataFlows() stocks.each( {stock -> price [stock] = getClosing(stock, 2008) }.async()) def topStock = stocks.max { price [it] } </ul>
  32. 32. <ul>Dataflow Operators </ul><ul>operator(inputs: [stocksStream], outputs: [pricedStocks]) { stock -> def price = getClosing( stock , 2008) bindOutput(0, [stock: stock , price: price ]) } </ul><ul>* </ul><ul>+ </ul><ul><> </ul>
  33. 33. <ul>Actors </ul><ul><li>Isolated
  34. 34. Communicating </li></ul><ul><ul><li>Immutable messages </li></ul></ul><ul><li>Active </li></ul><ul><ul><li>Pooled shared threads </li></ul></ul><ul><li>Activities </li></ul><ul><ul><li>Create a new actor
  35. 35. Send a message
  36. 36. Receive a message </li></ul></ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>Actor </ul><ul>T </ul><ul>T </ul><ul>T </ul><ul>Thread pool </ul>
  37. 37. <ul>Actors use </ul><ul>Gate Keeper </ul><ul>Form </ul><ul>HTTP </ul><ul>Finger Prints </ul><ul>Address Check </ul><ul>Email Check </ul><ul>Process </ul><ul>Fraud Detect </ul><ul>Response </ul><ul>SOAP </ul><ul>SMTP </ul>
  38. 38. <ul>Actors patterns </ul><ul>Enricher Router Translator Endpoint Splitter Agregator Filter Resequencer Checker </ul>
  39. 39. <ul>Creating Actors </ul><ul>class MyActor extends AbstractPooledActor { void act() { def buddy = new YourActor() buddy << ‘Hi man, how’re things?’ def response = receive () } } </ul>
  40. 40. <ul>Creating Actors </ul><ul>def decryptor = actor { loop { react {msg -> reply msg.reverse() } } } decryptor << ‘noitcA nI yvoorG’ </ul>
  41. 41. <ul>Sending messages </ul><ul>buddy. send 10.eur buddy << new Book(title: ’Groovy Recipes’ , author: ’Scott Davis’ ) def canChat = buddy. sendAndWait ‘Got time?’ buddy. sendAndContinue ‘Need money!’ , {cash-> pocket.add cash } </ul>
  42. 42. <ul>Reacting to messages </ul><ul>react {gift1 -> reply ”Thank you for $gift1” react {gift2 -> reply ”Wow, one more $gift2” [gift1, gift2].max{it.price}. reply ‘ Will you marry me?’ } //Never reached } </ul>
  43. 43. <ul>Choosing the Reaction </ul><ul>react / receive {gift -> switch (gift) { case Money: reply ‘Thanks’ ;pocket gift; break case [iPhone, iPod]:child << gift; break case (BigFlat..SmallHouse):moveIn(gift); break case Clothes: putOn(gift) fits(gift)?reply ‘Thanks’ : reply gift break case EXIT: stop() } } </ul>
  44. 44. <ul>Continuation Style </ul><ul>loop { … react { … react { /*schedule the block; throw CONTINUE*/ … } //Never reached } //Never reached } //Never reached </ul>
  45. 45. <ul>Receiving messages </ul><ul>def gift = receive () reply ”Thank you for $gift” gift = receive () reply ”Wow, one more $gift” </ul><ul><li>Blocks the calling thread </li></ul>
  46. 46. <ul>Dynamic Dispatch Actor </ul><ul>def actor = new DynamicDispatchActor ({ when {BigDecimal num -> println 'Received BigDecimal' } when {String code -> compileAndRun code } when {Book book -> read book } }) </ul>
  47. 47. <ul>Safe (Agent) </ul><ul><li>Lock Shared Mutable State in a Safe </li></ul><ul>Alter the State </ul><ul>Alter the State </ul>
  48. 48. <ul>Safe inside </ul><ul>Double </ul><ul>Increment </ul><ul>Add 25 </ul><ul>Message Queue </ul><ul>36 </ul><ul>thread </ul>
  49. 49. <ul>Safe List </ul><ul>def jugMembers = new Safe( ['Me'] ) //add Me task { jugMembers.send {it.add 'Joe'} //add Joe } task { jugMembers << {it.add 'Dave'} //add Dave jugMembers << {it.add 'Alice'} //add Alice } ... println jugMembers. val </ul>
  50. 50. <ul>Safe-based Shopping Cart </ul><ul><li>Cart cart = new Cart()
  51. 51. cart.addItem 'Pilsner', 10
  52. 52. cart.addItem 'Budweisser', 5
  53. 53. cart.removeItem 'Budweisser‘
  54. 54. cart.increaseQuantity 'Budweisser', 3 </li></ul>
  55. 55. <ul>Cart implementation </ul><ul>class ShoppingCart { private def cartState = new Safe( [:] ) public void addItem(String product, int quantity) { cartState << {it[product] = quantity} } public void removeItem(String product) { cartState << {it.remove(product)} } public Object listContent() { return cartState. val } </ul>
  56. 56. <ul>Integration </ul><ul><li>Bundled with Groovy dists
  57. 57. Maven
  58. 58. Gradle
  59. 59. Grape (@Grab)
  60. 60. Griffon plugin
  61. 61. Grails plugin </li></ul>
  62. 62. <ul>Roadmap </ul>Actor performance Actor and DataFlow remoting CSP implementation STM?, ActiveObjects?
  63. 63. <ul>No more threads and locks </ul><ul>images.parallel. map { //concurrency agnostic code here } def myActor = actor { //concurrency agnostic code here } Identically with agents, STM, dataflow, … </ul>
  64. 64. <ul>Summary </ul><ul>Tasty concurrency menu </ul><ul><ul><li>Actors, Collections, Dataflow, Safe, … </li></ul></ul><ul>Enjoyable parallelism http://gpars.codehaus.org </ul>
  65. 65. <ul>Questions? </ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×