Gpars concepts explained

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    3 Favorites

    Gpars concepts explained - Presentation Transcript

    1. GPars Groovy Parallel Systems Václav Pech
    2. About
      • Passionate programmer
      • Open source software author
      • Concurrency enthusiast
      • Technology evangelist @ JetBrains
      • JetBrains Academy member
      • http://www.jroller.com/vaclav
        • http://twitter.com/vaclav_pech
    3. GPars
      • Nov 2008 – Sep 2009 as GParallelizer
      • Since Sep 2009
        • GPars
        • @Codehaus
        • 5 commiters
        • Tight relation with the Groovy team
    4. Agenda
      • Why?
      • What?
      • How?
    5. Pantha rei
      • Entering a new era!
    6. We’re quad core already Beware: More cores to come shortly!
    7. SW is being late to the game
      • Dead-locks
      • Live-locks
      • Missed writes
      • Race conditions
      • The Root Cause
        • Shared Mutable State
    8. Lock and synchronize
      • Multithreaded programs today work mostly
      • by accident!
    9. Can we do better?
      • STM
      • Dataflow
      • CSP
      • Actors (message passing)
      • Agents
      • Fork/Join
      • Parallel Arrays
    10. GPars Aims
      • Easy concurrency through
        • Promising concurrency concepts
        • Natural and elegant DSLs
        • Flexible Groovy meta-programming
    11. Toolset Fork/Join Actors Collections Dataflow Safe
    12. Parallelizer
      • doParallel { forkJoinPool ->
      • images. findAllParallel {it.contains me}
      • . collectParallel {convert it}
      • . groupByParallel {it.size()}
      • }
      • Important:
        • Side-effect-free functions only!
    13. Transparently parallel
      • doParallel {
      • images. makeTransparent()
      • . findAll {it.contains me}
      • . collect {convert it}
      • . groupBy {it.size()}
      • }
    14. Parallelize existing code
      • buyBargain stocks. makeTransparent()
      • def buyBargain(stocks) {
      • buy stocks. findAll {
      • it.download().price < 50
      • }
      • }
      • Use with CAUTION!
    15. Functional flavor
      • Map / Reduce
        • map, reduce, filter, min, max, sum, …
      • (1..n).parallel.filter{it%2==0}
      • .map{it ** 2}
      • .reduce{a, b -> a + b}
    16. Call closures asynchronously
      • def isSelfPortrait = {image -> image.contains me}
      • def flag = isSelfPortrait. call (img1)
      • def future = isSelfPortrait. callAsync (img1)
      • def flag = future.get()
    17. Asynchronous closures
      • def resize = {img -> img.resize(64, 64)}
      • def fastResize = resize. async()
      • def resized = images.collect fastResize
      • createAlbum resized*.get()
    18. Fork/Join Orchestration
      • protected void compute() {
      • long count = 0;
      • file.eachFile {
      • if (it.isDirectory()) {
      • println &quot;Forking a thread for $it&quot;
      • forkOffChild (new FileCounter(it))
      • } else {
      • count++
      • }
      • }
      • setResult (count + ( childrenResults ?.sum() ?: 0))
      • }
      Waits for children without blocking the thread!
    19. GroovyDSL – IntelliJ IDEA CE
    20. Dataflow Concurrency
      • No race-conditions
      • No live-locks
      • Deterministic deadlocks
        • Completely deterministic programs
      • BEAUTIFUL code
        • (Jonas Bonér)
    21. Dataflow Variables
      • start { z << x.val + y.val }
      • start { x << 40 }
      • start { y << 2 }
      • assert 42 == z.val
      • Single-assignment variables
        • with blocking read
    22. DataFlows
      • def df = new DataFlows ()
      • start { df .z = df .x + df .y }
      • start { df .x = 10 }
      • start { df .y = 5 }
      • assert 15 == df .z
    23. DataFlows Making Money
      • def stocks = ['AAPL', 'GOOG', 'IBM', 'JAVA']
      • def price = new DataFlows()
      • stocks.each( {stock ->
      • price [stock] = getClosing(stock, 2008)
      • }.async())
      • def topStock = stocks.max { price [it] }
    24. Dataflow Streams
      • def words = ['Groovy', 'fantastic', ‘parallel’, ‘enjoyable’, …]
      • final def buffer = new DataFlowStream()
      • start { //producer
      • for (word in words) {
      • buffer << word.toUpperCase()
      • }
      • }
      • start { //consumer
      • while (true) println buffer .val
      • }
    25. Dataflow Operators
      • operator(inputs: [stocksStream],
      • outputs: [pricedStocks], group)
      • { stock ->
      • def price = getClosing( stock , 2008)
      • bindOutput(0, [stock: stock , price: price ])
      • }
      * + <>
    26. Actors
      • Isolated
      • Communicating
        • Immutable messages
      • Active
        • Pooled shared threads
      • Activities
        • Create a new actor
        • Send a message
        • Receive a message
      Actor Actor Actor Actor Actor Actor Actor T T T Thread pool
    27. Actors use Gate Keeper Form HTTP Finger Prints Address Check Email Check Process Fraud Detect Response SOAP SMTP
    28. Actors patterns Enricher Router Translator Endpoint Splitter Agregator Filter Resequencer Checker
    29. Creating Actors
      • class MyActor extends AbstractPooledActor {
      • void act() {
      • def buddy = new YourActor()
      • buddy << ‘Hi man, how’re things?’
      • def response = receive ()
      • }
      • }
    30. Creating Actors
      • def decryptor = actor {
      • loop {
      • react {msg ->
      • reply msg.reverse()
      • }
      • }
      • }
      • decryptor << ‘noitcA nI yvoorG’
    31. Sending messages
      • buddy. send 10.eur
      • buddy << new Book(title: ’Groovy Recipes’ ,
      • author: ’Scott Davis’ )
      • def canChat = buddy. sendAndWait ‘Got time?’
      • buddy. sendAndContinue ‘Need money!’ , {cash->
      • pocket.add cash
      • }
    32. Reacting to messages
      • react {gift1 ->
      • reply ”Thank you for $gift1”
      • react {gift2 ->
      • reply ”Wow, one more $gift2”
      • [gift1, gift2].max{it.price}. reply
      • ‘ Will you marry me?’
      • }
      • //Never reached
      • }
    33. Choosing the Reaction
      • react / receive {gift ->
      • switch (gift) {
      • case Money: reply ‘Thanks’ ;pocket gift; break
      • case [iPhone, iPod]:child << gift; break
      • case (BigFlat..SmallHouse):moveIn(gift); break
      • case Clothes:
      • putOn(gift)
      • fits(gift)?reply ‘Thanks’ : reply gift
      • break
      • case EXIT: stop()
      • }
      • }
    34. Continuation Style
      • loop {
      • react {
      • react { /*schedule the block; throw CONTINUE*/
      • }
      • //Never reached
      • }
      • //Never reached
      • }
      • //Never reached
    35. Receiving messages
      • def gift = receive ()
      • reply ”Thank you for $gift”
      • gift = receive ()
      • reply ”Wow, one more $gift”
      • Blocks the calling thread
    36. Dynamic Dispatch Actor
      • def actor = new DynamicDispatchActor ({
      • when {BigDecimal num ->
      • println 'Received BigDecimal' }
      • when {String code ->
      • compileAndRun code }
      • when {Book book ->
      • read book }
      • })
    37. Reactor
      • def decryptor = reactor {
      • msg.reverse()
      • }
      • decryptor << ‘rotca na si rotcaeR’
    38. Safe (Agent)
      • Lock Shared Mutable State in a Safe
      Alter the State Alter the State
    39. Safe inside Double Increment Add 25 Message Queue 36 thread
    40. Agent principles
      • Builds on actors
        • Accepts functions / commands
      • Runs the functions against the state
      • Output becomes the new state
      • Functions don’t care about other functions
        • No locking or sync code in them
    41. Safe List
      • def jugMembers = new Safe<List>( ['Me'] ) //add Me
      • final Thread t1 = Thread.start {
      • jugMembers.send {it.add 'Joe'} //add Joe
      • }
      • final Thread t2 = Thread.start {
      • jugMembers << {it.add 'Dave'} //add Dave
      • jugMembers << {it.add 'Alice'} //add Alice
      • }
      • [t1, t2]*.join()
      • println jugMembers. val
    42. Safe-based Shopping Cart
      • Cart cart = new Cart()
      • cart.addItem 'Pilsner', 10
      • cart.addItem 'Budweisser', 5
      • cart.removeItem 'Budweisser‘
      • cart.increaseQuantity 'Budweisser', 3
    43. Cart implementation
      • class ShoppingCart {
      • private def cartState = new Safe( [:] )
      • public void addItem(String product, int quantity) {
      • cartState << {it[product] = quantity}
      • }
      • public void removeItem(String product) {
      • cartState << {it.remove(product)}
      • }
      • public Object listContent() {
      • return cartState. val
      • }
    44. Integration
      • Maven
      • Gradle
      • Grape (@Grab)
      • Griffon plugin
      • Grails plugin
    45. Roadmap
      • Map/Reduce, Fork/Join
      • Dataflow enhancements
      • API evolution
      • Actor and DataFlow remoting
      • CSP?, STM?
    46. Summary
      • Tasty concurrency menu
        • Actors, Collections, Dataflow, Safe, …
      • Enjoyable parallelism
      • http://gpars.codehaus.org
      • Questions?
    SlideShare Zeitgeist 2009

    + Václav PechVáclav Pech Nominate

    custom

    690 views, 3 favs, 3 embeds more stats

    Shows and explains the basic concepts and features more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 690
      • 686 on SlideShare
      • 4 from embeds
    • Comments 0
    • Favorites 3
    • Downloads 14
    Most viewed embeds
    • 2 views on http://soup.robert42.com
    • 1 views on http://www.soup.io
    • 1 views on http://www.twittertim.es

    more

    All embeds
    • 2 views on http://soup.robert42.com
    • 1 views on http://www.soup.io
    • 1 views on http://www.twittertim.es

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories