Gpars concepts explained

  • 7,347 views
Uploaded on

Shows and explains the basic concepts and features of the GPars Groovy concurrency library - http://gpars.codehaus.org .

Shows and explains the basic concepts and features of the GPars Groovy concurrency library - http://gpars.codehaus.org .

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,347
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
129
Comments
0
Likes
13

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1.
      GPars
      Groovy Parallel Systems Václav Pech
  • 2.
      About me
    • Passionate programmer
    • 3. Concurrency enthusiast
    • 4. GPars @ Codehaus lead
    • 5. Technology evangelist @ JetBrains
    • 6. JetBrains Academy member
      http://www.jroller.com/vaclav
      • http://twitter.com/vaclav_pech
  • 7.
      GPars
      Apache 2 license Hosted @ Codehaus 7 commiters Tight relation with the Groovy team
      • Originally named GParallelizer
  • 8.
      Pantha rei
      Entering a new era!
  • 9.
      We’re quad core already
      Beware: More cores to come shortly!
  • 10.
      SW is being late to the game
    Dead-locks Live-locks Race conditions Starvation Shared Mutable State
  • 11.
      Lock and synchronize
      Multi-threaded programs today work mostly by accident!
  • 12.
      Toolset
      Fork/Join
      Actors
      Collections
      Dataflow
      Safe
  • 13.
      Parallel Collection Processing
      withPool { images. findAllParallel {it.contains me} . collectParallel {convert it} . groupByParallel {it.size()} }
    Important:
      • Side-effect-free functions only!
  • 14.
      Transparently parallel
      withPool ( 4 ) { images. makeTransparent() . findAll {it.contains me} . collect {convert it} . groupBy {it.size()} }
  • 15.
      Parallelize existing code
      buyBargain stocks. makeTransparent()
      def buyBargain(stocks) { buy stocks. findAll { it.download().price < 50 } } Use with CAUTION!
  • 16.
      Functional flavor
      Map / Reduce
      • map, reduce, filter, min, max, sum, …
      (1..n).parallel.filter {it%2==0} .map {it ** 2} .reduce {a, b -> a + b}
  • 17.
      Call closures asynchronously
      def isSelfPortrait = {image -> image.contains me}
    SYNC:
      def flag = isSelfPortrait. call (img1)
    ASYNC:
      def future = isSelfPortrait. callAsync (img1) … def flag = future.get()
  • 18.
      Asynchronous closures
      def resize = {img -> img.resize(64, 64)} def fastResize = resize. async() def resized = images.collect fastResize … createAlbum resized*.get()
  • 19.
      Fork/Join
    • Solve hierarchical problems
        • Divide and conquer
      • Merge sort, Quick sort
      • 20. Tree traversal
      • 21. File scan / search
      • 22.
      [a, b, c, d, e, f, g, h]
      [a, b, c, d]
      [e, f, g, h]
      [a, b]
      [c, d]
      [e, f]
      [g, h]
  • 23.
      Fork/Join Orchestration
      runForkJoin(new File( “./src” )) {currentDir -> long count = 0; currentDir .eachFile { if (it.isDirectory()) { println &quot;Forking a thread for $it&quot; forkOffChild it } else { count++ } } return count + childrenResults .sum(0) }
      Waits for children without blocking the thread!
  • 24.
      GroovyDSL – IntelliJ IDEA CE
  • 25.
      Dataflow Concurrency
    • No race-conditions
    • 26. No live-locks
    • 27. Deterministic deadlocks
      • Completely deterministic programs
      BEAUTIFUL code
      • (Jonas Bonér)
  • 28.
      Dataflow Variables / Promises
      main
      task2
      task3
      x
      y
      z
      task1
  • 29.
      Dataflow Variables
      task { z << x.val + y.val } task { x << 40 } task { y << 2 } assert 42 == z.val
    • Single-assignment variables
      • with blocking read
  • 30.
      DataFlows
      def df = new DataFlows () task { df .z = df .x + df .y } task { df .x = 10 } task { df .y = 5 } assert 15 == df .z
  • 31.
      DataFlows Making Money
      def stocks = ['AAPL', 'GOOG', 'IBM', 'JAVA'] def price = new DataFlows() stocks.each( {stock -> price [stock] = getClosing(stock, 2008) }.async()) def topStock = stocks.max { price [it] }
  • 32.
      Dataflow Operators
      operator(inputs: [stocksStream], outputs: [pricedStocks]) { stock -> def price = getClosing( stock , 2008) bindOutput(0, [stock: stock , price: price ]) }
      *
      +
      <>
  • 33.
      Actors
    • Isolated
    • 34. Communicating
      • Immutable messages
    • Active
      • Pooled shared threads
    • Activities
      • Create a new actor
      • 35. Send a message
      • 36. Receive a message
      Actor
      Actor
      Actor
      Actor
      Actor
      Actor
      Actor
      T
      T
      T
      Thread pool
  • 37.
      Actors use
      Gate Keeper
      Form
      HTTP
      Finger Prints
      Address Check
      Email Check
      Process
      Fraud Detect
      Response
      SOAP
      SMTP
  • 38.
      Actors patterns
      Enricher Router Translator Endpoint Splitter Agregator Filter Resequencer Checker
  • 39.
      Creating Actors
      class MyActor extends AbstractPooledActor { void act() { def buddy = new YourActor() buddy << ‘Hi man, how’re things?’ def response = receive () } }
  • 40.
      Creating Actors
      def decryptor = actor { loop { react {msg -> reply msg.reverse() } } } decryptor << ‘noitcA nI yvoorG’
  • 41.
      Sending messages
      buddy. send 10.eur buddy << new Book(title: ’Groovy Recipes’ , author: ’Scott Davis’ ) def canChat = buddy. sendAndWait ‘Got time?’ buddy. sendAndContinue ‘Need money!’ , {cash-> pocket.add cash }
  • 42.
      Reacting to messages
      react {gift1 -> reply ”Thank you for $gift1” react {gift2 -> reply ”Wow, one more $gift2” [gift1, gift2].max{it.price}. reply ‘ Will you marry me?’ } //Never reached }
  • 43.
      Choosing the Reaction
      react / receive {gift -> switch (gift) { case Money: reply ‘Thanks’ ;pocket gift; break case [iPhone, iPod]:child << gift; break case (BigFlat..SmallHouse):moveIn(gift); break case Clothes: putOn(gift) fits(gift)?reply ‘Thanks’ : reply gift break case EXIT: stop() } }
  • 44.
      Continuation Style
      loop { … react { … react { /*schedule the block; throw CONTINUE*/ … } //Never reached } //Never reached } //Never reached
  • 45.
      Receiving messages
      def gift = receive () reply ”Thank you for $gift” gift = receive () reply ”Wow, one more $gift”
    • Blocks the calling thread
  • 46.
      Dynamic Dispatch Actor
      def actor = new DynamicDispatchActor ({ when {BigDecimal num -> println 'Received BigDecimal' } when {String code -> compileAndRun code } when {Book book -> read book } })
  • 47.
      Safe (Agent)
    • Lock Shared Mutable State in a Safe
      Alter the State
      Alter the State
  • 48.
      Safe inside
      Double
      Increment
      Add 25
      Message Queue
      36
      thread
  • 49.
      Safe List
      def jugMembers = new Safe( ['Me'] ) //add Me task { jugMembers.send {it.add 'Joe'} //add Joe } task { jugMembers << {it.add 'Dave'} //add Dave jugMembers << {it.add 'Alice'} //add Alice } ... println jugMembers. val
  • 50.
      Safe-based Shopping Cart
    • Cart cart = new Cart()
    • 51. cart.addItem 'Pilsner', 10
    • 52. cart.addItem 'Budweisser', 5
    • 53. cart.removeItem 'Budweisser‘
    • 54. cart.increaseQuantity 'Budweisser', 3
  • 55.
      Cart implementation
      class ShoppingCart { private def cartState = new Safe( [:] ) public void addItem(String product, int quantity) { cartState << {it[product] = quantity} } public void removeItem(String product) { cartState << {it.remove(product)} } public Object listContent() { return cartState. val }
  • 56.
      Integration
  • 62.
      Roadmap
    Actor performance Actor and DataFlow remoting CSP implementation STM?, ActiveObjects?
  • 63.
      No more threads and locks
      images.parallel. map { //concurrency agnostic code here } def myActor = actor { //concurrency agnostic code here } Identically with agents, STM, dataflow, …
  • 64.
      Summary
      Tasty concurrency menu
      • Actors, Collections, Dataflow, Safe, …
      Enjoyable parallelism http://gpars.codehaus.org
  • 65.
      Questions?