GR8Conf 2011: GPars

GPars – coz concurrency is GroovyVáclav Pech, JetBrains

Whos this geek?Passionate programmerConcurrency enthusiastGPars leadGroovy contributorTechnology evangelist / developer @ JetBrainsJetBrains Academy member http://www.jroller.com/vaclav http://twitter.com/vaclav_pech

We can do better! Agent Map/Reduce Dataflow Fork/Join AsyncFun Actors

Intel I7 quad-core

Collectionsimages.eachParallel {it.process()}documents.sumParallel()candidates.maxParallel {it.salary}.marry()

Transparently parallelwithPool(4) { images.makeConcurrent() .findAll {it.contains me} .collect {convert it} .groupBy {it.size()}}

Parallelize existing codebuyBargain ( stocks.makeConcurrent() )def buyBargain(stocks) { buy stocks.findAll { it.download().price < 50 }} Use with CAUTION!

Functional flavorMap / Reduce map, reduce, filter, min, max, sum, …(1..n).parallel.filter {it%2==0} .map {it ** 2} .reduce {a, b -> a + b}

Asynchronous invocationFuture f = threadPool.submit(calculation);…System.out.println(“Result: “ + f.get());main calculation get()

Call closures asynchronouslydef isSelfPortrait = {image -> image.contains me}SYNC:def flag = isSelfPortrait.call(img1)ASYNC:def future = isSelfPortrait.callAsync(img1)…def flag = future.get()

Asynchronous closuresdef resize = {img -> img.resize(64, 64)}def fastResize = resize.asyncFun()def resized = images.collect fastResize…createAlbum resized*.get()

Composable Asynchronous Functionsint hash1 = hash(download(http://www.gpars.org))int hash2 = hash(loadFile(/gpars/website/index.html))boolean result = compare(hash1, hash2)println result

Composable Asynchronous Functionshash = hash.asyncFun(); compare = compare.asyncFun()download = download.asyncFun()loadFile = loadFile.asyncFun()def hash1 = hash(download(http://www.gpars.org))def hash2 = hash(loadFile(/gpars/website/index.html))def result = compare(hash1, hash2)println result.get()

Composable Asynchronous Functions int hash(String text) {…}Promise<int> hash(Promise<String> | String text) { 1. Return a Promise for the result 2. Wait (non-blocking) for the text param 3. Call the original hash() 4. Bind the result}

Composable Asynchronous FunctionsValid combinations of functions are also valid combinations oftheir asynchronous variantsParallelism is detected automatically without programmerintervention

Fork/Join [a, b, c, d, e, f, g, h]● Solve hierarchical problems  Divide and conquer  Merge sort, Quick sort [a, b, c, d] [e, f, g, h]  Tree traversal  File scan / search … [a, b] [c, d] [e, f] [g, h]

Hierarchical / Recursive Functionsdef fib = {n -> if (n <=2) return 1 forkOffChild(n - 1) forkOffChild(n - 2) return childrenResults.sum()}

Dataflow Concurrency● No race-conditions● No live-locks● Deterministic deadlocks Completely deterministic programs BEAUTIFUL code (Jonas Bonér)

Dataflow Variables / Promisesmain task1 task2 task3 x z y

DataFlow Variablesdef df = new DataFlows()task { df.z = df.x + df.y }task { df.x = 10 }task { println ”I am task 3” df.y = 5}assert 15 == df.z

Dataflow Operatorsoperator(inputs: [headers, bodies, footers], outputs: [articles, summaries]) {header, body, footer -> def article = buildArticle(header, body, footer) bindOutput(0, article) bindOutput(1, buildSummary(article)) } * <> +

Dataflow Operators Url resolver Url resolver Downloader Groovy scanner Scala scanner Reporter

Dataflow Operators Url resolver Speculator Cache Downloader updates Evaluator Approvals Splitter Groovy scanner Scala scanner Reporter Confirm

A CSP flavourCommunicating Sequential Processes Focus on composable processes more than on data

Dataflow Processes Out a Prefix - 0 Copy b Successorc

Dataflow Processes Out a Prefix - 0 Copy b Successorc Numbers Out

Dataflow ProcessesIn Out a Plus Copy b Prefix - 0 c

Dataflow ProcessesIn Out a Plus Copy b Prefix - 0 c In Integrate Out

Dataflow Processes Numbers Out In Integrate Out

Dataflow Processes Numbers Integrate Out

Actors Actor Isolated Actor● Actor Actor Communicating Actor● Actor Actor  Immutable messages● Active  Pooled shared threads● Activities  Create a new actor  Send a message TT  Receive a message T Thread pool

Stateless Actorsclass MyActor extends DynamicDispatchActor { Account account = ... public void onMessage(String msg) { String encripted = encrypt(msg); reply(encripted); } public void onMessage(Integer number) { reply(2 * number); } public void onMessage(Money cash) { System.out.println("Received a donation " + cash); account.deposit(cash); }}

Stateful Actorsclass MyActor extends AbstractPooledActor { void act() { def buddy = new YourActor() buddy << ‘Hi man, how’re things?’ def response = receive() }}

Active Objects@ActiveObjectclass MyCounter { private int counter = 0@ActiveMethod def incrementBy(int value) { println "Received an integer: $value" this.counter += value }}

IntegrationBundled with Groovy distsAvailable the maven central repository(Maven, Ivy, Gradle, Grape)@Grab(group=org.codehaus.gpars, module=gpars, version=0.11)

RoadmapSTMActor and DataFlow remoting

GPars – coz concurrency is Groovy!Find more at: http://gpars.codehaus.org http://www.jroller.com/vaclav http://twitter.com/vaclav_pech

GR8Conf 2011: GPars

by GR8Conf

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 10

Statistics

GR8Conf 2011: GPars Presentation Transcript

http://universegroovygrails.blogspot.com	8
http://paper.li	2