groovy and concurrency

Concurrent Programming in Groovy • Java concurrent programming enhancements – Normal OO methods – Ability to have immutable types – Some concurrency building blocks – Annotations with baked in goodness • Process/Thread ease of use – AntBuilder and GDK methods • Closures for greater flexibility – Enabler for concurrency – Closure is Runnable and Callable • Third-party libraries – GPars, Functional Java (Actors), Multiverse, JCSP – Cascading.groovy subproject for Hadoop clusters – Jetlang, JPPF, GridGain, Google Collections, Gruple – Groovy actors: http://www.groovyactors.org Concurrency - 2

Java Concurrency Features • The early years – Threads, synchronised and non-synchronised collections, synchronisation at the language level, Monitors (wait/notify), Locks, ThreadLocal, final, ... • More recent enhancements – java.util.concurrent: Executors, Thread Pools, Optimistic updates, Blocking queues, Synchronizers, Callables, Futures, Atomic operations, Deques, ... • Emerging – Fork/Join & others, Kilim, Phasers, PicoThreads ... • Leverage related APIs/technologies – Networking, real-time, GUIs, simulation, database, multimedia, operating systems, parallel processing, distribution, mobile agents, nio, ... Concurrency - 3

Topics Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • Fibonacci Case Study • GPars © ASERT 2006-2010 • More Info Concurrency - 4

What is Groovy? • “Groovy is like a super version of Java. It can leverage Java's enterprise capabilities but also has cool productivity features like closures, DSL support, builders and dynamic typing.” © ASERT 2006-2010 Groovy = Java – boiler plate code + optional dynamic typing + closures + domain specific languages + builders + metaprogramming Concurrency - 5

Groovy Goodies Overview • Fully object oriented • Closures: reusable and assignable pieces of code • Operators can be • GPath: efficient overloaded © ASERT 2006-2010 object navigation • Multimethods • GroovyBeans • Literal declaration • grep and switch for lists (arrays), maps, ranges and • Templates, builder, regular expressions swing, Ant, markup, XML, SQL, XML-RPC, Scriptom, Grails, tests, Mocks Concurrency - 6

Growing Acceptance … A slow and steady start but now gaining in momentum, maturity and mindshare Now free

… Growing Acceptance … What alternative JVM language are you using or intending to use © ASERT 2006-2010 http://www.leonardoborges.com/writings http://it-republik.de/jaxenter/quickvote/results/1/poll/44 (translated using http://babelfish.yahoo.com) Concurrency - 8

… Growing Acceptance … © ASERT 2006-2010 Source: http://www.micropoll.com/akira/mpresult/501697-116746 Source: http://www.grailspodcast.com/ Concurrency - 9

… Growing Acceptance … © ASERT 2006-2010 http://www.jroller.com/scolebourne/entry/devoxx_2008_whiteboard_votes http://www.java.net Concurrency - 10

… Growing Acceptance … © ASERT 2006-2010 Groovy and Grails downloads: 70-90K per month and growing Frequent topic at the popular conferences Concurrency - 11

… Growing Acceptance © ASERT 2006-2010 Concurrency - 12

The Landscape of JVM Languages optional static types © ASERT 2006-2010 Dynamic features call for dynamic types Java bytecode calls for static types The terms “Java Virtual Machine” and “JVM” mean a Virtual Machine for the Java™ platform. Concurrency - 13

Groovy Starter System.out.println("Hello, World!"); // supports Java syntax println 'Hello, World!' // but can remove some syntax String name = 'Guillaume' // Explicit typing/awareness println "$name, I'll get the car." // Gstring (interpolation) def longer = """${name}, the car is in the next row.""" // multi-line, implicit type © ASERT 2006-2010 assert 0.5 == 1/2 // BigDecimal equals() assert 0.1 + 0.2 == 0.3 // and arithmetic def printSize(obj) { // implicit/duck typing print obj?.size() // safe dereferencing } def pets = ['ant', 'bee', 'cat'] // native list syntax pets.each { pet -> // closure support assert pet < 'dog' // overloading '<' on String } // or: for (pet in pets)... Concurrency - 14

A Better Java... import java.util.List; import java.util.ArrayList; class Erase { private List removeLongerThan(List strings, int length) { This code List result = new ArrayList(); for (int i = 0; i < strings.size(); i++) { is valid String s = (String) strings.get(i); Java and if (s.length() <= length) { result.add(s); valid Groovy } } © ASERT 2006-2010 return result; } public static void main(String[] args) { List names = new ArrayList(); names.add("Ted"); names.add("Fred"); names.add("Jed"); names.add("Ned"); System.out.println(names); Based on an Erase e = new Erase(); example by List shortNames = e.removeLongerThan(names, 3); Jim Weirich System.out.println(shortNames.size()); for (int i = 0; i < shortNames.size(); i++) { & Ted Leung String s = (String) shortNames.get(i); System.out.println(s); } } } Concurrency - 15

...A Better Java... import java.util.List; import java.util.ArrayList; class Erase { private List removeLongerThan(List strings, int length) { Do the List result = new ArrayList(); for (int i = 0; i < strings.size(); i++) { semicolons String s = (String) strings.get(i); add anything? if (s.length() <= length) { result.add(s); And shouldn‟t } } we us more © ASERT 2006-2010 } return result; modern list public static void main(String[] args) { notation? List names = new ArrayList(); names.add("Ted"); names.add("Fred"); Why not names.add("Jed"); names.add("Ned"); System.out.println(names); import common Erase e = new Erase(); libraries? List shortNames = e.removeLongerThan(names, 3); System.out.println(shortNames.size()); for (int i = 0; i < shortNames.size(); i++) { String s = (String) shortNames.get(i); System.out.println(s); } } } Concurrency - 16

...A Better Java... class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList() for (String s in strings) { if (s.length() <= length) { result.add(s) } } return result } public static void main(String[] args) { © ASERT 2006-2010 List names = new ArrayList() names.add("Ted"); names.add("Fred") names.add("Jed"); names.add("Ned") System.out.println(names) Erase e = new Erase() List shortNames = e.removeLongerThan(names, 3) System.out.println(shortNames.size()) for (String s in shortNames) { System.out.println(s) } } } Concurrency - 17

...A Better Java... class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList() for (String s in strings) { if (s.length() <= length) { result.add(s) Do we need } the static types? } return result Must we always } have a main public static void main(String[] args) { method and © ASERT 2006-2010 List names = new ArrayList() names.add("Ted"); names.add("Fred") class definition? names.add("Jed"); names.add("Ned") System.out.println(names) How about Erase e = new Erase() List shortNames = e.removeLongerThan(names, 3) improved System.out.println(shortNames.size()) consistency? for (String s in shortNames) { System.out.println(s) } } } Concurrency - 18

...A Better Java... def removeLongerThan(strings, length) { def result = new ArrayList() for (s in strings) { if (s.size() <= length) { result.add(s) } } return result } names = new ArrayList() names.add("Ted") names.add("Fred") © ASERT 2006-2010 names.add("Jed") names.add("Ned") System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) for (s in shortNames) { System.out.println(s) } Concurrency - 19

...A Better Java... def removeLongerThan(strings, length) { def result = new ArrayList() for (s in strings) { if (s.size() <= length) { result.add(s) } } Shouldn‟t we return result } have special names = new ArrayList() notation for lists? names.add("Ted") names.add("Fred") And special © ASERT 2006-2010 names.add("Jed") names.add("Ned") facilities for System.out.println(names) shortNames = removeLongerThan(names, 3) list processing? System.out.println(shortNames.size()) for (s in shortNames) { Is „return‟ System.out.println(s) } needed at end? Concurrency - 20

...A Better Java... def removeLongerThan(strings, length) { strings.findAll{ it.size() <= length } } names = ["Ted", "Fred", "Jed", "Ned"] System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) shortNames.each{ System.out.println(s) } © ASERT 2006-2010 Concurrency - 21

...A Better Java... def removeLongerThan(strings, length) { strings.findAll{ it.size() <= length } } Is the method names = ["Ted", "Fred", "Jed", "Ned"] System.out.println(names) now needed? shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) Easier ways to shortNames.each{ System.out.println(s) } use common methods? © ASERT 2006-2010 Are brackets required here? Concurrency - 22

...A Better Java names = ["Ted", "Fred", "Jed", "Ned"] println names shortNames = names.findAll{ it.size() <= 3 } println shortNames.size() shortNames.each{ println it } © ASERT 2006-2010 Output: ["Ted", "Fred", "Jed", "Ned"] 3 Ted Jed Ned Concurrency - 23

Grapes / Grab: Google collections @Grab('com.google.collections:google-collections:1.0') import com.google.common.collect.HashBiMap HashBiMap fruit = [grape:'purple', lemon:'yellow', lime:'green'] assert fruit.lemon == 'yellow' © ASERT 2006-2010 assert fruit.inverse().yellow == 'lemon' Concurrency - 24

Better Design Patterns: Delegate… public Date getWhen() { import java.util.Date; return when; } public class Event { private String title; public void setWhen(Date when) { private String url; this.when = when; private Date when; } public String getUrl() { public boolean before(Date other) { return url; return when.before(other); } © ASERT 2006-2010 } public void setUrl(String url) { public void setTime(long time) { this.url = url; when.setTime(time); } } public String getTitle() { public long getTime() { return title; return when.getTime(); } } public void setTitle(String title) { public boolean after(Date other) { this.title = title; return when.after(other); } } // ... // ... ESDC 2010 - 25

…Better Design Patterns: Delegate… public Date getWhen() { import java.util.Date; return when; boilerplate } public class Event { private String title; public void setWhen(Date when) { private String url; this.when = when; private Date when; } public String getUrl() { public boolean before(Date other) { return url; return when.before(other); } © ASERT 2006-2010 } public void setUrl(String url) { public void setTime(long time) { this.url = url; when.setTime(time); } } public String getTitle() { public long getTime() { return title; return when.getTime(); } } public void setTitle(String title) { public boolean after(Date other) { this.title = title; return when.after(other); } } // ... // ... ESDC 2010 - 26

…Better Design Patterns: Delegate class Event { String title, url @Delegate Date when } © ASERT 2006-2010 def gr8conf = new Event(title: "GR8 Conference", url: "http://www.gr8conf.org", when: Date.parse("yyyy/MM/dd", "2009/05/18")) def javaOne = new Event(title: "JavaOne", url: "http://java.sun.com/javaone/", when: Date.parse("yyyy/MM/dd", "2009/06/02")) assert gr8conf.before(javaOne.when) ESDC 2010 - 27

Why Groovy? Technical Answer • Minimal learning curve • Compiles to bytecode • Java object model & integration • Annotations • "Optional" static typing • Both run-time and compile-time metaprogramming Concurrency - 28

Why Groovy? Adoption Assessment • Innovators/Thought leaders – Ideas, power, flexibility, novelty, thinking community • Early adopters – Productivity benefits and collegiate community – Leverage JVM and potential for mainstream • Mainstream – Leverage existing Java skills, low learning curve – Leverage JVM and production infrastructure – Professional community – Tools, tools, tools

Source: Herb Sutter: http://www.gotw.ca/publications/concurrency-ddj.htm "Andy giveth and Bill taketh away" Concurrency - 30

Why is it hard? • Many issues to deal with: – Doing things in parallel, concurrently, asynchronously • Processes, Threads, Co-routines, Events, Scheduling – Sharing/Synchronization Mechanisms • shared memory, locks, transactions, wait/notify, STM, message passing, actors, serializability, persistence, immutability – Abstractions • Shared memory on top of messaging passing • Message passing on top of shared memory • Dataflow, Selective Communication, Continuations – Data Structures and Algorithms • Queues, Heaps, Trees • Sorting, Graph Algorithms Concurrency - 31

Topics • Groovy Intro Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • Fibonacci Case Study • GPars © ASERT 2006-2010 • More Info Concurrency - 32

Thread Enhancements… • Thread improvements Thread.start{ sleep 1001; println 'one' } Thread.start{ sleep 1000; println 'two' } println 'one' def t = Thread.start{ sleep 100; println 'three' } println 'two' t.join() println 'four' Concurrency - 33

…Thread Enhancements… • Thread improvements (Cont’d) class Storage { List stack = [] synchronized void leftShift(value){ stack << value println "push: $value" notifyAll() } synchronized Object pop() { while (!stack) { try{ wait() } catch(InterruptedException e){} } def value = stack.pop() println "pop : $value" return value } } ... Concurrency - 34

…Thread Enhancements… • Thread improvements (Cont’d) push: 0 push: 1 ... push: 2 storage = new Storage() pop : 2 push: 3 Thread.start { push: 4 for (i in 0..9) { pop : 4 storage << i push: 5 sleep 100 push: 6 } pop : 6 } push: 7 push: 8 Thread.start { pop : 8 10.times { push: 9 sleep 200 pop : 9 storage.pop() pop : 7 } pop : 5 } pop : 3 pop : 1 pop : 0 Concurrency - 35

…Thread Enhancements… • Thread improvements (Cont’d) ROCK! . 25 import java.util.concurrent.locks.ReentrantLock . 33 import static System.currentTimeMillis as now . 34 def startTime = now() . 35 ReentrantLock.metaClass.withLock = { critical -> . 36 lock() .. 131 try { critical() } .. 134 finally { unlock() } } .. 137 def lock = new ReentrantLock() .. 138 def worker = { threadNum -> .. 139 4.times { count -> ... 232 lock.withLock { ... 234 print " " * threadNum ... 237 print "." * (count + 1) ... 238 println " ${now() - startTime}" ... 239 } Thread.sleep 100 .... 334 } .... 336 } .... 337 5.times { Thread.start worker.curry(it) } .... 338 println "ROCK!" .... 339 Source: http://chrisbroadfoot.id.au/articles/2008/08/06/groovy-threads Concurrency - 36

… Thread Enhancements • Thread Management meets AtomicInteger import java.util.concurrent.atomic.AtomicInteger def counter = new AtomicInteger() synchronized out(message) { println(message) thread loop 1 } main loop 1 def th = Thread.start { thread loop 2 for( i in 1..8 ) { thread loop 3 sleep 30 main loop 2 out "thread loop $i" thread loop 4 counter.incrementAndGet() } thread loop 5 } main loop 3 for( j in 1..4 ) { thread loop 6 sleep 50 main loop 4 out "main loop $j" thread loop 7 counter.incrementAndGet() thread loop 8 } th.join() assert counter.get() == 12 Concurrency - 37

Process Enhancements… • Using AntBuilder [echo] Message 6 [echo] Message 9 [echo] Message 8 def ant = new AntBuilder() [echo] Message 0 ant.parallel { [echo] Message 2 10.times { echo "Message $it" } [echo] Message 4 } [echo] Message 1 [echo] Message 7 [echo] Message 5 [echo] Message 3 Concurrency - 38

…Process Enhancements… • Using AntBuilder (Cont’d) def ant = new AntBuilder() ant.taskdef(name:'groovy', classname:'org.codehaus.groovy.ant.Groovy') ant.parallel { 10.times { echo "Ant message $it" groovy "println 'Groovy via ant message $it'" println "Groovy message $it" // or fork java, command line, thread ... } } def ant = new AntBuilder() ant.exec(outputproperty:"cmdOut", errorproperty: "cmdErr", resultproperty:"cmdExit", failonerror: "true", executable: /opt/myExecutable') { arg(line:'*"first with space"* second') } Concurrency - 39

…Process Enhancements • Process Management proc1 = 'ls'.execute() def process = "ls -l".execute() proc2 = 'tr -d o'.execute() println "Found text ${process.text}" proc3 = 'tr -d e'.execute() proc4 = 'tr -d i'.execute() proc1 | proc2 | proc3 | proc4 def process = "ls -l".execute() proc4.waitFor() process.in.eachLine { line -> if (proc4.exitValue()) println line } print proc4.err.text def sout = new StringBuffer() else def serr = new StringBuffer() print proc4.text proc1 = 'gzip -c'.execute() proc2 = 'gunzip -c'.execute() proc2.consumeProcessOutput(sout, serr) proc1 | proc2 proc1.consumeProcessErrorStream(serr) proc1.withWriter { writer -> writer << 'test text' } proc2.waitForOrKill(1000) println 'sout: ' + sout // => test text println 'serr: ' + serr Concurrency - 40

Parallelize your arrays with JSR 166y //Create a pool with size close to the number of processor cores def pool = new ForkJoinPool(2) def createParallelArray(pool, collection) { return ParallelArray.createFromCopy( collection.toArray(new Object[collection.size()]), pool) } // Enhance ArrayLists to find matching objects in parallel ArrayList.metaClass.findAll = {Closure cl -> createParallelArray(pool, delegate). withFilter({cl(it)} as Predicate).all().asList() } def sites=['http://www.jroller.com', 'http://www.infoq.com', 'http://java.dzone.com'] def groovySites = sites.findAll { new URL(it).text.toLowerCase().contains('groovy')} println "These sites talk about Groovy today: ${groovySites}" Source: http://www.jroller.com/vaclav/date/20080923 Concurrency - 41

Java Concurrency Best Practice? • Java Concurrency in Practice: – “If mutable threads access the same mutable state variable without appropriate synchronization, your program is broken” – “When designing thread-safe classes, good object-oriented techniques – encapsulation, immutability, and clear specification of invariants – are your best friends” Concurrency - 42

Immutability options • Built-in def animals = ['cat', 'dog', 'horse'].asImmutable() animals << 'fish' // => java.lang.UnsupportedOperationException • Google Collections – Numerous improved immutable collection types import com.google.common.collect.* List<String> animals = ImmutableList.of("cat", "dog", "horse") animals << 'fish' // => java.lang.UnsupportedOperationException • Groovy run-time metaprogramming def animals = ['cat', 'dog', 'horse'] ArrayList.metaClass.leftShift = { throw new UnsupportedOperationException() } animals << 'fish' // => java.lang.UnsupportedOperationException • Groovy 1.6+ compile-time metaprogramming – @Immutable can help us create such classes Concurrency - 43

@Immutable... • Java Immutable Class – As per Joshua Bloch // ... @Override Effective Java public boolean equals(Object obj) { if (this == obj) public final class Punter { return true; private final String first; if (obj == null) private final String last; return false; if (getClass() != obj.getClass()) public String getFirst() { return false; return first; Punter other = (Punter) obj; } if (first == null) { © ASERT 2006-2010 if (other.first != null) public String getLast() { return false; return last; } else if (!first.equals(other.first)) } return false; if (last == null) { @Override if (other.last != null) public int hashCode() { return false; final int prime = 31; } else if (!last.equals(other.last)) int result = 1; return false; result = prime * result + ((first == null) return true; ? 0 : first.hashCode()); } result = prime * result + ((last == null) ? 0 : last.hashCode()); @Override return result; public String toString() { } return "Punter(first:" + first + ", last:" + last + ")"; public Punter(String first, String last) { } this.first = first; this.last = last; } } // ... Concurrency - 44

...@Immutable... • Java Immutable Class boilerplate – As per Joshua Bloch // ... @Override Effective Java public boolean equals(Object obj) { if (this == obj) public final class Punter { return true; private final String first; if (obj == null) private final String last; return false; if (getClass() != obj.getClass()) public String getFirst() { return false; return first; Punter other = (Punter) obj; } if (first == null) { © ASERT 2006-2010 if (other.first != null) public String getLast() { return false; return last; } else if (!first.equals(other.first)) } return false; if (last == null) { @Override if (other.last != null) public int hashCode() { return false; final int prime = 31; } else if (!last.equals(other.last)) int result = 1; return false; result = prime * result + ((first == null) return true; ? 0 : first.hashCode()); } result = prime * result + ((last == null) ? 0 : last.hashCode()); @Override return result; public String toString() { } return "Punter(first:" + first + ", last:" + last + ")"; public Punter(String first, String last) { } this.first = first; this.last = last; } } // ... Concurrency - 45

...@Immutable @Immutable class Punter { String first, last © ASERT 2006-2010 } Concurrency - 46

@Synchronized – Before Transform class SynchronizedExample { private final myLock = new Object() @Synchronized static void greet() { println "world" } © ASERT 2006-2010 @Synchronized int answerToEverything() { return 42 } @Synchronized("myLock") void foo() { println "bar" } } Concurrency - 47

@Synchronized – After Transform class SynchronizedExample { Inspired by private static final $LOCK = new Object[0] Project private final $lock = new Object[0] Lombok private final myLock = new Object() static void greet() { synchronized ($LOCK) { println "world" } } © ASERT 2006-2010 int answerToEverything() { synchronized ($lock) { return 42 } } void foo() { synchronized (myLock) { println "bar" } } } Concurrency - 48

@Lazy... • Safe initialization idioms – Eager or fully synchronized import net.jcip.annotations.ThreadSafe @ThreadSafe class EagerInitialization { static final resource = new Resource() © ASERT 2006-2010 } @ThreadSafe class SafeLazyInitialization { private static ExpensiveResource resource synchronized static ExpensiveResource getInstance() { if (!resource) resource = new ExpensiveResource () resource } } Concurrency - 49

...@Lazy... • Java Concurrency in Practice – Race condition with lazy initialization import net.jcip.annotations.NotThreadSafe @NotThreadSafe class LazyInitRace { © ASERT 2006-2010 private ExpensiveResource instance = null ExpensiveResource getInstance() { if (!instance) instance = new ExpensiveResource() instance } } – No problems: just apply Double-checked locking pattern Concurrency - 50

...@Lazy... • Java Concurrency in Practice – Double checked locking anti-pattern @NotThreadSafe class DoubleCheckedLocking { private static ExpensiveResource instance = null © ASERT 2006-2010 static ExpensiveResource getInstance() { if (!instance) { synchronized (DoubleCheckedLocking) { if (!instance) instance = new ExpensiveResource() } } instance } } Concurrency - 51

...@Lazy... • Java Concurrency in Practice – Double checked locking less broken-pattern class DoubleCheckedLocking { private static volatile ExpensiveResource instance = null static ExpensiveResource getInstance() { © ASERT 2006-2010 if (!instance) { synchronized (DoubleCheckedLocking) { if (!instance) instance = new ExpensiveResource() } } instance } } Concurrency - 52

...@Lazy... • Java Concurrency in Practice – Lazy initialization holder class idiom import net.jcip.annotations.ThreadSafe @ThreadSafe class ResourceFactory { © ASERT 2006-2010 private static class ResourceHolder { static Resource resource = new Resource() } static Resource getResource() { ResourceHolder.resource } } Concurrency - 53

...@Lazy @Lazy Resource first @Lazy volatile Resource second © ASERT 2006-2010 @Lazy volatile Resource third = { new Resource(args) }() @Lazy(soft = true) volatile Resource fourth Concurrency - 54

Topics • Groovy Intro • Useful Groovy features for Concurrency Related Concurrency Libraries & Tools • Fibonacci Case Study • GPars © ASERT 2006-2010 • More Info Concurrency - 55

Lightweight threads: Jetlang • Jetlang – A high performance java threading library Fiber receiver = new ThreadFiber(); // JAVA receiver.start(); // create java.util.concurrent.CountDownLatch to notify when message arrives final CountDownLatch latch = new CountDownLatch(1); // create channel to message between threads Channel<String> channel = new MemoryChannel<String>(); Callback<String> onMsg = new Callback<String>() { public void onMessage(String message) { //open latch latch.countDown(); } }; //add subscription for message on receiver thread channel.subscribe(receiver, onMsg); //publish message to receive thread. the publish method is thread safe. channel.publish("Hello"); //wait for receiving thread to receive message latch.await(10, TimeUnit.SECONDS); //shutdown thread receiver.dispose(); Concurrency - 56

Other High-Level Libraries: JPPF – Open source Grid Computing platform – http://www.jppf.org/ import org.jppf.client.* import java.util.concurrent.Callable class Task implements Callable, Serializable { private static final long serialVersionUID = 1162L; public Object call() { println 'Executing Groovy' "Hello JPPF from Groovy" } } def client = new JPPFClient() def job = new JPPFJob() def task = new Task() job.addTask task def results = client.submit(job) for (t in results) { if (t.exception) throw t.exception println "Result: " + t.result } Concurrency - 57

Other High-Level Libraries: Gruple... – http://code.google.com/p/gruple – Gruple aims to provide a simple abstraction to allow programmers to coordinate and synchronize threads with ease – based on Tuplespaces • Tuplespaces provide the illusion of a shared memory on top of a message passing system, along with a small set of operations to greatly simplify parallel programming – Example Tuple: [fname:"Vanessa", lname:"Williams", project:"Gruple"] – Basic operations within a Tuplespace are: • put - insert a tuple into the space • get - read a tuple from the space (non-destructively) • take - take a tuple from the space (a destructive read) – Further reading: Eric Freeman, Susanne Hupfer, and Ken Arnold. JavaSpaces Principles, Patterns, and Practice, Addison Wesley, 1999 Concurrency - 58

Other High-Level Libraries: ...Gruple... – Mandelbrot example (included in Gruple download) ... Space space = SpaceService.getSpace("mandelbrot") Map template = createTaskTemplate() Map task String threadName = Thread.currentThread().name while(true) { ArrayList points task = space.take(template) println "Worker $threadName got task ${task['start']} for job ${task['jobId'] points = calculateMandelbrot(task) Map result = createResult(task['jobId'], task['start'], points) println "Worker $threadName writing result for task ${result['start']} for jo space.put(result) } ... Concurrency - 59

Other High-Level Libraries: ...Gruple Concurrency - 60

Other High-Level Libraries: Cascading.groovy – API/DSL for executing tasks on a Hadoop cluster – http://www.cascading.org/ def assembly = builder.assembly(name: "wordcount") { eachTuple(args: ["line"], results: ["word"]) { regexSplitGenerator(declared: ["word"], pattern: /[.,]*s+/) } group(["word"]) everyGroup(args: ["word"], results: ["word", "count"]) { count() } group(["count"], reverse: true) } def map = builder.map() { source(name: "wordcount") { hfs(input) { text(["line"]) } } sink(name: "wordcount") { hfs(output) { text() } } } def flow = builder.flow(name: "wordcount", map: map, assembly: assembly) Concurrency - 61

Other High-Level Libraries: GridGain – Simple & productive to use grid computing platform – http://www.gridgain.com/ class GridHelloWorldGroovyTask extends GridTaskSplitAdapter<String, Integer> { Collection split(int gridSize, Object phrase) throws GridException { // ... } Object reduce(List results) throws GridException { // ... } } import static GridFactory.* start() def grid = getGrid() def future = grid.execute(GridHelloWorldGroovyTask, "Hello World") def phraseLen = future.get() stop(true) Concurrency - 62

Multiverse STM… – http://www.multiverse.org/ import org.multiverse.api.GlobalStmInstance import org.multiverse.api.Transaction import org.multiverse.templates.TransactionTemplate import org.multiverse.transactional.refs.LongRef def from = new Account(10) def to = new Account(10) atomic { from.balance -= 5 to.balance += 5 } println "from $from.balance" println "to $to.balance" ... Concurrency - 63

…Multiverse STM… ... void atomic(Closure block) { atomic([:], block) } void atomic(Map args, Closure block) { boolean readonly = args['readonly'] ?: false boolean trackreads = args['trackreads'] ?: true def txFactory = GlobalStmInstance.globalStmInstance. transactionFactoryBuilder. setReadonly(readonly). setReadTrackingEnabled(trackreads).build() new TransactionTemplate(txFactory) { Object execute(Transaction transaction) { block.call() return null } }.execute() } ... Concurrency - 64

…Multiverse STM class Account { private final balance = new LongRef() Account(long initial) { balance.set initial } void setBalance(long newBalance) { if (newBalance < 0) throw new RuntimeException("not enough money") balance.set newBalance } long getBalance() { balance.get() } } Concurrency - 65

Testing multi-threaded applications: ConTest... • Advanced Testing for Multi-Threaded Applications – Tool for testing, debugging, and coverage-measuring of concurrent programs (collects runtime statistics) – Systematically and transparently (using a java agent) schedules the execution of program threads in ways likely to reveal race conditions, deadlocks, and other intermittent bugs (collectively called synchronization problems) with higher than normal frequency – The ConTest run-time engine adds heuristically controlled conditional instructions (adjustable by a preferences file) that force thread switches, thus helping to reveal concurrent bugs. You can use existing tests and run ConTest multiple times – by default different heuristics used each time it is run • http://www.alphaworks.ibm.com/tech/contest Concurrency - 66

...Testing multi-threaded applications: ConTest NUM = 5 ParalInc.groovy count = 0 def incThread = { n -> Thread.start{ sleep n*10 targetClasses = ParalInc //synchronized(ParalInc) { timeoutTampering = true count++ noiseFrequency = 500 //} strength = 10000 } } def threads = (1..NUM).collect(incThread) threads.each{ it.join() } assert count == NUM > groovyc ParalInc.groovy > java -javaagent:../../Lib/ConTest.jar -cp %GROOVY_JAR%;. ParalInc Exception in thread "main" Assertion failed: assert count == NUM | | | 4 | 5 false Concurrency - 67

GContracts @Grab('org.gcontracts:gcontracts:1.0.2') import org.gcontracts.annotations.* 1.8+ @Invariant({ first != null && last != null }) class Person { String first, last @Requires({ delimiter in ['.', ',', ' '] }) @Ensures({ result == first + delimiter + last }) String getName(String delimiter) { first + delimiter + last } } new Person(first: 'John', last: 'Smith').getName('.') Concurrency - 68

Testing: Spock class HelloSpock extends spock.lang.Specification { def "length of Spock's and his friends' names"() { expect: name.size() == length where: name | length "Spock" | 5 "Kirk" | 4 "Scotty" | 6 } } Concurrency - 69

Topics • Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools Fibonacci Case Study • GPars © ASERT 2006-2010 • More Info Concurrency - 70

Fibonacci Case Study Concurrency - 71

Fibonacci… START = 8 Serial END = 16 version fib = {n -> n < 2 ? n : fib(n - 1) + fib(n - 2) } (START..END).each {num -> println "n:$num => ${fib(num)}" } Concurrency - 72

…Fibonacci… import java.util.concurrent.* ConcurrentHashMap THREADS = 4 version START = 8 END = 16 QUIT = -1 class Fibonacci { def values = new ConcurrentHashMap() int calc(x) { x < 2 ? x : calc(x-1) + calc(x-2) } int calcWithCache(x) { def result = values[x] if (!result) { result = calc(x) values.putIfAbsent(x, result) } result } } println "Calculating Fibonacci sequence in parallel..." def queue = new ArrayBlockingQueue(10) Concurrency - 73

…Fibonacci… Thread.start('Producer') { int x = START while (x <= END) { sleep 200 queue << x++ } sleep 1000 THREADS.times { queue << QUIT } } (1..THREADS).each { def name = "Consumer$it" Thread.start(name) { def done = false def fib = new Fibonacci() while (!done) { def n = queue.take() if (n == QUIT) done = true else println "$name n:$n => ${fib.calcWithCache(n)}" } } } Concurrency - 74

…Fibonacci… Executor import java.util.concurrent.* version CUTOFF = 12 // not worth parallelizing for small n THREADS = 100 println "Calculating Fibonacci sequence in parallel..." serialFib = {n -> (n < 2) ? n : serialFib(n - 1) + serialFib(n - 2) } pool = Executors.newFixedThreadPool(THREADS) defer = {c -> pool.submit(c as Callable) } fib = {n -> if (n < CUTOFF) return serialFib(n) def left = defer { fib(n - 1) } def right = defer { fib(n - 2) } left.get() + right.get() } (8..16).each {n -> println "n=$n => ${fib(n)}" } pool.shutdown() Concurrency - 75

…Fibonacci… import EDU.oswego.cs.dl.util.concurrent.FJTask import EDU.oswego.cs.dl.util.concurrent.FJTaskRunnerGroup class Fib extends FJTask { static final CUTOFF = 12 Fork/Join volatile int number version int getAnswer() { if (!isDone()) throw new IllegalStateException() number } void run() { int n = number if (n <= CUTOFF) number = seqFib(n) else { def f1 = new Fib(number: n - 1) def f2 = new Fib(number: n - 2) coInvoke(f1, f2) number = f1.number + f2.number } } int seqFib(int n) { n < 2 ? n : seqFib(n - 1) + seqFib(n - 2) } } Concurrency - 76

…Fibonacci… def THREADS = 2 def group = new FJTaskRunnerGroup(THREADS) def START = 8 def END = 16 (START..END).each {num -> def f = new Fib(number: num) group.invoke(f) println "n:$num => $f.answer" } Concurrency - 77

…Fibonacci… import fj.* import fj.control.parallel.Strategy FunctionalJava import static fj.Function.curry as fcurry version import static fj.P1.curry as pcurry import static fj.P1.fmap import static fj.control.parallel.Actor.actor import static fj.control.parallel.Promise.* import static fj.data.List.range import static java.util.concurrent.Executors.* CUTOFF = 12 // not worth parallelizing for small n START = 8 END = 16 THREADS = 4 pool = newFixedThreadPool(THREADS) su = Strategy.executorStrategy(pool) spi = Strategy.executorStrategy(pool) add = fcurry({a, b -> a + b } as F2) nums = range(START, END + 1) println "Calculating Fibonacci sequence in parallel..." Concurrency - 78

…Fibonacci… serialFib = {n -> n < 2 ? n : serialFib(n - 1) + serialFib(n - 2) } print = {results -> def n = START results.each { println "n=${n++} => $it" } pool.shutdown() } as Effect calc = {n -> n < CUTOFF ? promise(su, P.p(serialFib(n))) : calc.f(n - 1).bind(join(su, pcurry(calc).f(n - 2)), add) } as F out = actor(su, print) join(su, fmap(sequence(su)).f(spi.parMapList(calc).f(nums))).to(out) Concurrency - 79

…Fibonacci… import org.jetlang.core.Callback import org.jetlang.channels.MemoryChannel Jetlang import org.jetlang.fibers.ThreadFiber version import java.util.concurrent.* println "Calculating Fibonacci sequence with two cooperating fibers..." class FibonacciCalc implements Callback { private channel, receiver def limit, name, latch void onMessage(inpair) { def next = inpair[0] + inpair[1] def outpair = [inpair[1], next] println "$name next:$next" channel.publish(outpair) if (next > limit) { latch.countDown() sleep 200 receiver.dispose() } } // ... Concurrency - 80

…Fibonacci // ... void subscribe(other) { channel.subscribe(receiver, other) } FibonacciCalc() { channel = new MemoryChannel() receiver = new ThreadFiber() receiver.start() } } def seed = [0, 1] def latch = new CountDownLatch(2) def calcA = new FibonacciCalc(limit: 500, name: 'CalcA', latch: latch) def calcB = new FibonacciCalc(limit: 500, name: 'CalcB', latch: latch) calcA.subscribe calcB calcB.subscribe calcA calcA.onMessage seed latch.await(10, TimeUnit.SECONDS) Concurrency - 81

Topics • Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • Fibonacci Case Study GPars © ASERT 2006-2010 • More Info Concurrency - 82

GPars • http://gpars.codehaus.org/ • Library classes and DSL sugar providing intuitive ways for Groovy developers to handle tasks concurrently. Logical parts: – Actors provide a Groovy implementation of Scala-like actors including "remote" actors on other machines – Dataflow Concurrency supports natural shared-memory © ASERT 2006-2010 concurrency model, using single-assignment variables – Asynchronizer extends the Java 1.5 built-in support for executor services to enable multi-threaded collection and closure processing – Parallelizer uses JSR-166y Parallel Arrays to enable multi-threaded collection processing – Safe a non-blocking mt-safe reference to mutable state that is inspired by "agents" in Clojure Concurrency - 83

GPars: Parallel Collection Functions def nums = 1..100000 def squares = nums .collect{ it ** 2 } .grep{ it % 7 == it % 5 } .grep{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] © ASERT 2006-2010 @Grab('org.codehaus.gpars:gpars:0.10') import static groovyx.gpars.GParsPool.withPool def nums = 1..100000 withPool(5) { def squares = nums. collectParallel{ it ** 2 }. grepParallel{ it % 7 == it % 5 }. grepParallel{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] } Concurrency - 85

GPars: Transparent Parallel Collections • Applies some Groovy metaprogramming import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 nums.makeTransparent() © ASERT 2006-2010 def squares = nums. collect{ it ** 2 }. grep{ it % 7 == it % 5 }. grep{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] } Concurrency - 86

Gpars concurrency-aware methods Transparent Transitive? Parallel any { ... } anyParallel { ... } collect { ... } yes collectParallel { ... } count(filter) countParallel(filter) each { ... } eachParallel { ... } eachWithIndex { ... } eachWithIndexParallel { ... } every { ... } everyParallel { ... } find { ... } findParallel { ... } findAll { ... } yes findAllParallel { ... } findAny { ... } findAnyParallel { ... } fold { ... } foldParallel { ... } fold(seed) { ... } foldParallel(seed) { ... } grep(filter) yes grepParallel(filter) groupBy { ... } groupByParallel { ... } max { ... } maxParallel { ... } max() maxParallel() min { ... } minParallel { ... } min() minParallel() split { ... } yes splitParallel { ... } Source: ReGina Concurrency - 87

GPars: Map-Reduce... import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 println nums.parallel. © ASERT 2006-2010 map{ it ** 2 }. filter{ it % 7 == it % 5 }. filter{ it % 3 == 0 }. collection } Concurrency - 88

...GPars: Map-Reduce import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 println nums.parallel. © ASERT 2006-2010 map{ it ** 2 }. filter{ it % 7 == it % 5 }. filter{ it % 3 == 0 }. reduce{ a, b -> a + b } } Concurrency - 89

Parallel Collections vs Map-Reduce Source: ReGina Concurrency - 90

GPars: Dataflows... import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.result = flow.x + flow.y } task { flow.x = 10 } task { flow.y = 5 } © ASERT 2006-2010 assert 15 == flow.result new DataFlows().with { task { result = x * y } task { x = 10 } task { y = 5 } assert 50 == result } Concurrency - 91

...GPars: Dataflows... • Evaluating: result = (a – b) * (a + b) x y import groovyx.gpars.dataflow.DataFlows © ASERT 2006-2010 import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() 10 5 task { flow.a = 10 } task { flow.b = 5 } task { flow.x = flow.a - flow.b } a b task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } - + assert flow.result == 75 * Concurrency - 92

...GPars: Dataflows... • Naive attempt for loops import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() ... [10, 20].each { thisA -> task { flow.a = 10 } [4, 5].each { thisB -> ... © ASERT 2006-2010 task { flow.a = thisA } task { flow.a = 20 } task { flow.b = thisB } task { flow.x = flow.a - flow.b } task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } println flow.result } } // => java.lang.IllegalStateException: A DataFlowVariable can only be assigned once. Concurrency - 93

...GPars: Dataflows... import groovyx.gpars.dataflow.DataFlowStream 10 4 import static groovyx.gpars.dataflow.DataFlow.* 10 5 20 4 final streamA = new DataFlowStream() 20 5 final streamB = new DataFlowStream() final streamX = new DataFlowStream() a b final streamY = new DataFlowStream() final results = new DataFlowStream() - + © ASERT 2006-2010 operator(inputs: [streamA, streamB], outputs: [streamX, streamY]) { * a, b -> streamX << a - b; streamY << a + b } operator(inputs: [streamX, streamY], outputs: [results]) { x, y -> results << x * y } 84 75 [[10, 20], [4, 5]].combinations().each{ thisA, thisB -> 384 task { streamA << thisA } 375 task { streamB << thisB } } 4.times { println results.val } Concurrency - 94

...GPars: Dataflows • Amenable to static analysis • Race conditions avoided • Deadlocks “typically” become repeatable import groovyx.gpars.dataflow.DataFlows © ASERT 2006-2010 import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.x = flow.y } task { flow.y = flow.x } Concurrency - 95

GPars: Dataflow Sieve final int requestedPrimeNumberCount = 1000 final DataFlowStream initialChannel = new DataFlowStream() task { (2..10000).each { initialChannel << it } } def filter(inChannel, int prime) { def outChannel = new DataFlowStream() © ASERT 2006-2010 operator([inputs: [inChannel], outputs: [outChannel]]) { if (it % prime != 0) { bindOutput it } } return outChannel } def currentOutput = initialChannel requestedPrimeNumberCount.times { int prime = currentOutput.val println "Found: $prime" currentOutput = filter(currentOutput, prime) } Source: http://groovyconsole.appspot.com/script/235002 Concurrency - 96

GPars: Actors... • Predefined coordination • Class with the with fork/join & following lifecycle & map/filter/reduce methods • Implicit coordination – But also DSL sugar & with dataflow augmentation • Actors provide explicit start() © ASERT 2006-2010 stop() coordination and act() enforce* no sharing of send(msg) state, process a single sendAndWait(msg) activity/message at a loop { } time react { msg -> } msg.reply(replyMsg) receive() join() * mostly Concurrency - 97

…GPars: Actors... import static groovyx.gpars.actor.Actors.* def decrypt = reactor { code -> code.reverse() } def audit = reactor { println it } def main = actor { decrypt 'terces pot' © ASERT 2006-2010 react { plainText -> audit plainText } } main.join() audit.stop() audit.join() Source: ReGina Concurrency - 98

…GPars: Actors... final class FilterActor extends DynamicDispatchActor { private final int myPrime private def follower def FilterActor(final myPrime) { this.myPrime = myPrime; } def onMessage(int value) { if (value % myPrime != 0) { if (follower) follower value else { println "Found $value" © ASERT 2006-2010 follower = new FilterActor(value).start() } } } def onMessage(def poisson) { if (follower) { def sender = poisson.sender follower.sendAndContinue(poisson, {this.stop(); sender?.send('Done } else { //I am the last in the chain stop() reply 'Done' } } } Source: http://groovyconsole.appspot.com/script/242001 Concurrency - 99

…GPars: Actors (2..requestedPrimeNumberBoundary).each { firstFilter it } firstFilter.sendAndWait 'Poisson' © ASERT 2006-2010 Source: http://groovyconsole.appspot.com/script/242001 Concurrency - 100

GPars: Agents... • Agents safeguard non-thread safe objects • Only the agent can update the underlying object • “Code” to update the protected object is sent to the agent © ASERT 2006-2010 • Can be used with other approaches Concurrency - 101

…GPars: Agents @Grab('org.codehaus.gpars:gpars:0.10') import groovyx.gpars.agent.Agent def speakers = new Agent<List>(['Alex'], {it?.clone()}) // add Alex speakers.send {updateValue it << 'Hilary'} // add Hilary final Thread t1 = Thread.start { speakers.send {updateValue it << 'Ken'} // add Ken } © ASERT 2006-2010 final Thread t2 = Thread.start { speakers << {updateValue it << 'Guy'} // add Guy speakers << {updateValue it << 'Ralph'} // add Ralph } [t1, t2]*.join() assert new HashSet(speakers.val) == new HashSet(['Alex', 'Hilary', 'Ken', 'Guy', 'Ralph']) Source: Gpars examples Concurrency - 102

GPars for testing @Grab('net.sourceforge.htmlunit:htmlunit:2.6') import com.gargoylesoftware.htmlunit.WebClient @Grab('org.codehaus.gpars:gpars:0.10') import static groovyx.gpars.GParsPool.* def testCases = [ ['Home', 'Bart', 'Content 1'], ['Work', 'Homer', 'Content 2'], ['Travel', 'Marge', 'Content 3'], © ASERT 2006-2010 ['Food', 'Lisa', 'Content 4'] ] withPool(3) { testCases.eachParallel{ category, author, content -> postAndCheck category, author, content } } private postAndCheck(category, author, content) { ... Concurrency - 103

Guy Steele example in Groovy… Sequential version def words = { s -> Guy Steele‟s example from keynote (from slide 52 onwards for several slides): http://strangeloop2010.com/talk/presentation_file/14299/GuySteele-parallel.pdf def result = [] def word = '' s.each{ ch -> if (ch == ' ') { if (word) result += word word = '' } else word += ch © ASERT 2006-2010 } if (word) result += word result } assert words("This is a sample") == ['This', 'is', 'a', 'sample'] assert words(" Here is another sample ") == ['Here', 'is', 'another', 'sample'] assert words("JustOneWord") == ['JustOneWord'] assert words("Here is a sesquipedalian string of words") == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words'] assert words(" ") == [] && words("") == [] Concurrency - 104

…Guy Steele example in Groovy… @Immutable class Chunk { Refactored sequential version String s def plus(Chunk other) { new Chunk(s + other.s) } def plus(Segment other) { new Segment(s + other.l, other.m, other.r) } } @Immutable class Segment { String l; List m; String r © ASERT 2006-2010 def plus(Chunk other) { new Segment(l, m, r + other.s) } def plus(Segment other) { new Segment(l, m + maybeWord(r + other.l) + other.m, other.r) } } class Util { static processChar(ch) { ch == ' ' ? new Segment('', [], '') : new Chunk(ch) } static maybeWord(s) { s ? [s] : [] } } import static Util.* ... Concurrency - 105

…Guy Steele example in Groovy… def words = { s -> Refactored sequential version def result s.each{ ch -> if (!result) result = processChar(ch) else result += processChar(ch) } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } case null: return [] © ASERT 2006-2010 } } assert words("This is a sample") == ['This', 'is', 'a', 'sample'] assert words(" Here is another sample ") == ['Here', 'is', 'another', 'sample'] assert words("JustOneWord") == ['JustOneWord'] assert words("Here is a sesquipedalian string of words") == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words'] assert words(" ") == [] && words("") == [] Concurrency - 106

…Guy Steele example in Groovy… Roll your own threading with ConcurrentHashMap version def swords = { s -> def result s.each{ ch -> if (!result) result = processChar(ch) else result += processChar(ch) } result ?: new Chunk('') } THREADS = 4 © ASERT 2006-2010 def words = { s -> int n = (s.size() + THREADS - 1) / THREADS def map = new java.util.concurrent.ConcurrentHashMap() (0..<THREADS).collect { i -> Thread.start { def (min, max) = [[s.size(),i*n].min(), [s.size(),(i+1)*n].min()] map[i] = swords(s[min..<max]) }}*.join() def result = map.entrySet().sort{ it.key }.sum{ it.value } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } } Concurrency - 107

…Guy Steele example in Groovy… DataFlow version: partially hard-coded to 4 partitions for easier reading def words = { s -> int n = (s.size() + THREADS - 1) / THREADS def min = (0..<THREADS).collectEntries{ [it, [s.size(),it*n].min()] } def max = (0..<THREADS).collectEntries{ [it, [s.size(),(it+1)*n].min()] } def result = new DataFlows().with { task { a = swords(s[min[0]..<max[0]]) } task { b = swords(s[min[1]..<max[1]]) } task { c = swords(s[min[2]..<max[2]]) } task { d = swords(s[min[3]..<max[3]]) } © ASERT 2006-2010 task { sum1 = a + b } task { sum2 = c + d } task { sum = sum1 + sum2 } println 'Tasks ahoy!' sum } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } } Concurrency - 108

…Guy Steele example in Groovy… Fork/Join version GRANULARITY_THRESHHOLD = 10 THREADS = 4 println GParsPool.withPool(THREADS) { def result = runForkJoin(0, input.size(), input){ first, last, s -> def size = last - first if (size <= GRANULARITY_THRESHHOLD) { swords(s[first..<last]) } else { // divide and conquer © ASERT 2006-2010 def mid = first + ((last - first) >> 1) forkOffChild(first, mid, s) forkOffChild(mid, last, s) childrenResults.sum() } } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } } Concurrency - 109

…Guy Steele example in Groovy… Map/Reduce version THRESHHOLD = 10 def split(raw) { raw.size() <= THRESHHOLD ? raw : [raw[0..<THRESHHOLD]] + split(raw.substring(THRESHHOLD)) } © ASERT 2006-2010 println GParsPool.withPool(THREADS) { def ans = split(input).parallel.map(swords).reduce{ a,b -> a + b } switch(ans) { case Chunk: return maybeWord(ans.s) case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) } } } Concurrency - 110

…Guy Steele example in Groovy Just leveraging the algorithm‟s parallel nature println GParsPool.withPool(THREADS) { def ans = input.collectParallel{ processChar(it) }.sum() switch(ans) { case Chunk: return maybeWord(ans.s) case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) } © ASERT 2006-2010 } } Concurrency - 111

Topics • Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • Fibonacci Case Study • GPars © ASERT 2006-2010 More Info Concurrency - 112

More Information about Concurrency • Web sites – http://gpars.codehaus.org/ – http://g.oswego.edu/ Doug Lea's home page – http://gee.cs.oswego.edu/dl/concurrency-interest/ – http://jcip.net/ Companion site for Java Concurrency in Practice – http://www.eecs.usma.edu/webs/people/okasaki/pubs.html#cup98 Purely Functional Data Structures – http://delicious.com/kragen/concurrency Concurrency bookmark list – http://www.gotw.ca/publications/concurrency-ddj.htm The Free Lunch is Over, Herb Sutter – http://manticore.cs.uchicago.edu/papers/damp07.pdf – http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=10142 Concepts, Techniques, and Models of Computer Programming Concurrency - 113

More Information about Groovy • Web sites – http://groovy.codehaus.org – http://grails.codehaus.org – http://pleac.sourceforge.net/pleac_groovy (many examples) – http://www.asert.com.au/training/java/GV110.htm (workshop) • Mailing list for users – user@groovy.codehaus.org • Information portals – http://www.aboutgroovy.org – http://www.groovyblogs.org • Documentation (1000+ pages) – Getting Started Guide, User Guide, Developer Guide, Testing Guide, Cookbook Examples, Advanced Usage Guide • Books – Several to choose from ... Concurrency - 114

More Information: Groovy in Action Concurrency - 115

groovy and concurrency

by Paul King

Accessibility

Categories

Upload Details

Usage Rights

Statistics

groovy and concurrency Presentation Transcript