• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
groovy and concurrency
 

groovy and concurrency

on

  • 6,386 views

Dr Paul King presentation slides on "Groovy and Concurrency" presented at StrangeLoop 2010 in St Loius

Dr Paul King presentation slides on "Groovy and Concurrency" presented at StrangeLoop 2010 in St Loius

Statistics

Views

Total Views
6,386
Views on SlideShare
6,385
Embed Views
1

Actions

Likes
9
Downloads
146
Comments
0

1 Embed 1

http://de.pinterest.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

groovy and concurrency groovy and concurrency Presentation Transcript

  • © ASERT 2006-2010 Groovy and Concurrency Dr Paul King, @paulk_asert paulk at asert.com.au
  • Concurrent Programming in Groovy • Java concurrent programming enhancements – Normal OO methods – Ability to have immutable types – Some concurrency building blocks – Annotations with baked in goodness • Process/Thread ease of use – AntBuilder and GDK methods • Closures for greater flexibility – Enabler for concurrency – Closure is Runnable and Callable • Third-party libraries – GPars, Functional Java (Actors), Multiverse, JCSP – Cascading.groovy subproject for Hadoop clusters – Jetlang, JPPF, GridGain, Google Collections, Gruple – Groovy actors: http://www.groovyactors.org Concurrency - 2
  • Java Concurrency Features • The early years – Threads, synchronised and non-synchronised collections, synchronisation at the language level, Monitors (wait/notify), Locks, ThreadLocal, final, ... • More recent enhancements – java.util.concurrent: Executors, Thread Pools, Optimistic updates, Blocking queues, Synchronizers, Callables, Futures, Atomic operations, Deques, ... • Emerging – Fork/Join & others, Kilim, Phasers, PicoThreads ... • Leverage related APIs/technologies – Networking, real-time, GUIs, simulation, database, multimedia, operating systems, parallel processing, distribution, mobile agents, nio, ... Concurrency - 3
  • Topics Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • Fibonacci Case Study • GPars © ASERT 2006-2010 • More Info Concurrency - 4
  • What is Groovy? • “Groovy is like a super version of Java. It can leverage Java's enterprise capabilities but also has cool productivity features like closures, DSL support, builders and dynamic typing.” © ASERT 2006-2010 Groovy = Java – boiler plate code + optional dynamic typing + closures + domain specific languages + builders + metaprogramming Concurrency - 5
  • Groovy Goodies Overview • Fully object oriented • Closures: reusable and assignable pieces of code • Operators can be • GPath: efficient overloaded © ASERT 2006-2010 object navigation • Multimethods • GroovyBeans • Literal declaration • grep and switch for lists (arrays), maps, ranges and • Templates, builder, regular expressions swing, Ant, markup, XML, SQL, XML-RPC, Scriptom, Grails, tests, Mocks Concurrency - 6
  • Growing Acceptance … A slow and steady start but now gaining in momentum, maturity and mindshare Now free
  • … Growing Acceptance … What alternative JVM language are you using or intending to use © ASERT 2006-2010 http://www.leonardoborges.com/writings http://it-republik.de/jaxenter/quickvote/results/1/poll/44 (translated using http://babelfish.yahoo.com) Concurrency - 8
  • … Growing Acceptance … © ASERT 2006-2010 Source: http://www.micropoll.com/akira/mpresult/501697-116746 Source: http://www.grailspodcast.com/ Concurrency - 9
  • … Growing Acceptance … © ASERT 2006-2010 http://www.jroller.com/scolebourne/entry/devoxx_2008_whiteboard_votes http://www.java.net Concurrency - 10
  • … Growing Acceptance … © ASERT 2006-2010 Groovy and Grails downloads: 70-90K per month and growing Frequent topic at the popular conferences Concurrency - 11
  • … Growing Acceptance © ASERT 2006-2010 Concurrency - 12
  • The Landscape of JVM Languages optional static types © ASERT 2006-2010 Dynamic features call for dynamic types Java bytecode calls for static types The terms “Java Virtual Machine” and “JVM” mean a Virtual Machine for the Java™ platform. Concurrency - 13
  • Groovy Starter System.out.println("Hello, World!"); // supports Java syntax println 'Hello, World!' // but can remove some syntax String name = 'Guillaume' // Explicit typing/awareness println "$name, I'll get the car." // Gstring (interpolation) def longer = """${name}, the car is in the next row.""" // multi-line, implicit type © ASERT 2006-2010 assert 0.5 == 1/2 // BigDecimal equals() assert 0.1 + 0.2 == 0.3 // and arithmetic def printSize(obj) { // implicit/duck typing print obj?.size() // safe dereferencing } def pets = ['ant', 'bee', 'cat'] // native list syntax pets.each { pet -> // closure support assert pet < 'dog' // overloading '<' on String } // or: for (pet in pets)... Concurrency - 14
  • A Better Java... import java.util.List; import java.util.ArrayList; class Erase { private List removeLongerThan(List strings, int length) { This code List result = new ArrayList(); for (int i = 0; i < strings.size(); i++) { is valid String s = (String) strings.get(i); Java and if (s.length() <= length) { result.add(s); valid Groovy } } © ASERT 2006-2010 return result; } public static void main(String[] args) { List names = new ArrayList(); names.add("Ted"); names.add("Fred"); names.add("Jed"); names.add("Ned"); System.out.println(names); Based on an Erase e = new Erase(); example by List shortNames = e.removeLongerThan(names, 3); Jim Weirich System.out.println(shortNames.size()); for (int i = 0; i < shortNames.size(); i++) { & Ted Leung String s = (String) shortNames.get(i); System.out.println(s); } } } Concurrency - 15
  • ...A Better Java... import java.util.List; import java.util.ArrayList; class Erase { private List removeLongerThan(List strings, int length) { Do the List result = new ArrayList(); for (int i = 0; i < strings.size(); i++) { semicolons String s = (String) strings.get(i); add anything? if (s.length() <= length) { result.add(s); And shouldn‟t } } we us more © ASERT 2006-2010 } return result; modern list public static void main(String[] args) { notation? List names = new ArrayList(); names.add("Ted"); names.add("Fred"); Why not names.add("Jed"); names.add("Ned"); System.out.println(names); import common Erase e = new Erase(); libraries? List shortNames = e.removeLongerThan(names, 3); System.out.println(shortNames.size()); for (int i = 0; i < shortNames.size(); i++) { String s = (String) shortNames.get(i); System.out.println(s); } } } Concurrency - 16
  • ...A Better Java... class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList() for (String s in strings) { if (s.length() <= length) { result.add(s) } } return result } public static void main(String[] args) { © ASERT 2006-2010 List names = new ArrayList() names.add("Ted"); names.add("Fred") names.add("Jed"); names.add("Ned") System.out.println(names) Erase e = new Erase() List shortNames = e.removeLongerThan(names, 3) System.out.println(shortNames.size()) for (String s in shortNames) { System.out.println(s) } } } Concurrency - 17
  • ...A Better Java... class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList() for (String s in strings) { if (s.length() <= length) { result.add(s) Do we need } the static types? } return result Must we always } have a main public static void main(String[] args) { method and © ASERT 2006-2010 List names = new ArrayList() names.add("Ted"); names.add("Fred") class definition? names.add("Jed"); names.add("Ned") System.out.println(names) How about Erase e = new Erase() List shortNames = e.removeLongerThan(names, 3) improved System.out.println(shortNames.size()) consistency? for (String s in shortNames) { System.out.println(s) } } } Concurrency - 18
  • ...A Better Java... def removeLongerThan(strings, length) { def result = new ArrayList() for (s in strings) { if (s.size() <= length) { result.add(s) } } return result } names = new ArrayList() names.add("Ted") names.add("Fred") © ASERT 2006-2010 names.add("Jed") names.add("Ned") System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) for (s in shortNames) { System.out.println(s) } Concurrency - 19
  • ...A Better Java... def removeLongerThan(strings, length) { def result = new ArrayList() for (s in strings) { if (s.size() <= length) { result.add(s) } } Shouldn‟t we return result } have special names = new ArrayList() notation for lists? names.add("Ted") names.add("Fred") And special © ASERT 2006-2010 names.add("Jed") names.add("Ned") facilities for System.out.println(names) shortNames = removeLongerThan(names, 3) list processing? System.out.println(shortNames.size()) for (s in shortNames) { Is „return‟ System.out.println(s) } needed at end? Concurrency - 20
  • ...A Better Java... def removeLongerThan(strings, length) { strings.findAll{ it.size() <= length } } names = ["Ted", "Fred", "Jed", "Ned"] System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) shortNames.each{ System.out.println(s) } © ASERT 2006-2010 Concurrency - 21
  • ...A Better Java... def removeLongerThan(strings, length) { strings.findAll{ it.size() <= length } } Is the method names = ["Ted", "Fred", "Jed", "Ned"] System.out.println(names) now needed? shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) Easier ways to shortNames.each{ System.out.println(s) } use common methods? © ASERT 2006-2010 Are brackets required here? Concurrency - 22
  • ...A Better Java names = ["Ted", "Fred", "Jed", "Ned"] println names shortNames = names.findAll{ it.size() <= 3 } println shortNames.size() shortNames.each{ println it } © ASERT 2006-2010 Output: ["Ted", "Fred", "Jed", "Ned"] 3 Ted Jed Ned Concurrency - 23
  • Grapes / Grab: Google collections @Grab('com.google.collections:google-collections:1.0') import com.google.common.collect.HashBiMap HashBiMap fruit = [grape:'purple', lemon:'yellow', lime:'green'] assert fruit.lemon == 'yellow' © ASERT 2006-2010 assert fruit.inverse().yellow == 'lemon' Concurrency - 24
  • Better Design Patterns: Delegate… public Date getWhen() { import java.util.Date; return when; } public class Event { private String title; public void setWhen(Date when) { private String url; this.when = when; private Date when; } public String getUrl() { public boolean before(Date other) { return url; return when.before(other); } © ASERT 2006-2010 } public void setUrl(String url) { public void setTime(long time) { this.url = url; when.setTime(time); } } public String getTitle() { public long getTime() { return title; return when.getTime(); } } public void setTitle(String title) { public boolean after(Date other) { this.title = title; return when.after(other); } } // ... // ... ESDC 2010 - 25
  • …Better Design Patterns: Delegate… public Date getWhen() { import java.util.Date; return when; boilerplate } public class Event { private String title; public void setWhen(Date when) { private String url; this.when = when; private Date when; } public String getUrl() { public boolean before(Date other) { return url; return when.before(other); } © ASERT 2006-2010 } public void setUrl(String url) { public void setTime(long time) { this.url = url; when.setTime(time); } } public String getTitle() { public long getTime() { return title; return when.getTime(); } } public void setTitle(String title) { public boolean after(Date other) { this.title = title; return when.after(other); } } // ... // ... ESDC 2010 - 26
  • …Better Design Patterns: Delegate class Event { String title, url @Delegate Date when } © ASERT 2006-2010 def gr8conf = new Event(title: "GR8 Conference", url: "http://www.gr8conf.org", when: Date.parse("yyyy/MM/dd", "2009/05/18")) def javaOne = new Event(title: "JavaOne", url: "http://java.sun.com/javaone/", when: Date.parse("yyyy/MM/dd", "2009/06/02")) assert gr8conf.before(javaOne.when) ESDC 2010 - 27
  • Why Groovy? Technical Answer • Minimal learning curve • Compiles to bytecode • Java object model & integration • Annotations • "Optional" static typing • Both run-time and compile-time metaprogramming Concurrency - 28
  • Why Groovy? Adoption Assessment • Innovators/Thought leaders – Ideas, power, flexibility, novelty, thinking community • Early adopters – Productivity benefits and collegiate community – Leverage JVM and potential for mainstream • Mainstream – Leverage existing Java skills, low learning curve – Leverage JVM and production infrastructure – Professional community – Tools, tools, tools
  • Source: Herb Sutter: http://www.gotw.ca/publications/concurrency-ddj.htm "Andy giveth and Bill taketh away" Concurrency - 30
  • Why is it hard? • Many issues to deal with: – Doing things in parallel, concurrently, asynchronously • Processes, Threads, Co-routines, Events, Scheduling – Sharing/Synchronization Mechanisms • shared memory, locks, transactions, wait/notify, STM, message passing, actors, serializability, persistence, immutability – Abstractions • Shared memory on top of messaging passing • Message passing on top of shared memory • Dataflow, Selective Communication, Continuations – Data Structures and Algorithms • Queues, Heaps, Trees • Sorting, Graph Algorithms Concurrency - 31
  • Topics • Groovy Intro Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • Fibonacci Case Study • GPars © ASERT 2006-2010 • More Info Concurrency - 32
  • Thread Enhancements… • Thread improvements Thread.start{ sleep 1001; println 'one' } Thread.start{ sleep 1000; println 'two' } println 'one' def t = Thread.start{ sleep 100; println 'three' } println 'two' t.join() println 'four' Concurrency - 33
  • …Thread Enhancements… • Thread improvements (Cont’d) class Storage { List stack = [] synchronized void leftShift(value){ stack << value println "push: $value" notifyAll() } synchronized Object pop() { while (!stack) { try{ wait() } catch(InterruptedException e){} } def value = stack.pop() println "pop : $value" return value } } ... Concurrency - 34
  • …Thread Enhancements… • Thread improvements (Cont’d) push: 0 push: 1 ... push: 2 storage = new Storage() pop : 2 push: 3 Thread.start { push: 4 for (i in 0..9) { pop : 4 storage << i push: 5 sleep 100 push: 6 } pop : 6 } push: 7 push: 8 Thread.start { pop : 8 10.times { push: 9 sleep 200 pop : 9 storage.pop() pop : 7 } pop : 5 } pop : 3 pop : 1 pop : 0 Concurrency - 35
  • …Thread Enhancements… • Thread improvements (Cont’d) ROCK! . 25 import java.util.concurrent.locks.ReentrantLock . 33 import static System.currentTimeMillis as now . 34 def startTime = now() . 35 ReentrantLock.metaClass.withLock = { critical -> . 36 lock() .. 131 try { critical() } .. 134 finally { unlock() } } .. 137 def lock = new ReentrantLock() .. 138 def worker = { threadNum -> .. 139 4.times { count -> ... 232 lock.withLock { ... 234 print " " * threadNum ... 237 print "." * (count + 1) ... 238 println " ${now() - startTime}" ... 239 } Thread.sleep 100 .... 334 } .... 336 } .... 337 5.times { Thread.start worker.curry(it) } .... 338 println "ROCK!" .... 339 Source: http://chrisbroadfoot.id.au/articles/2008/08/06/groovy-threads Concurrency - 36
  • … Thread Enhancements • Thread Management meets AtomicInteger import java.util.concurrent.atomic.AtomicInteger def counter = new AtomicInteger() synchronized out(message) { println(message) thread loop 1 } main loop 1 def th = Thread.start { thread loop 2 for( i in 1..8 ) { thread loop 3 sleep 30 main loop 2 out "thread loop $i" thread loop 4 counter.incrementAndGet() } thread loop 5 } main loop 3 for( j in 1..4 ) { thread loop 6 sleep 50 main loop 4 out "main loop $j" thread loop 7 counter.incrementAndGet() thread loop 8 } th.join() assert counter.get() == 12 Concurrency - 37
  • Process Enhancements… • Using AntBuilder [echo] Message 6 [echo] Message 9 [echo] Message 8 def ant = new AntBuilder() [echo] Message 0 ant.parallel { [echo] Message 2 10.times { echo "Message $it" } [echo] Message 4 } [echo] Message 1 [echo] Message 7 [echo] Message 5 [echo] Message 3 Concurrency - 38
  • …Process Enhancements… • Using AntBuilder (Cont’d) def ant = new AntBuilder() ant.taskdef(name:'groovy', classname:'org.codehaus.groovy.ant.Groovy') ant.parallel { 10.times { echo "Ant message $it" groovy "println 'Groovy via ant message $it'" println "Groovy message $it" // or fork java, command line, thread ... } } def ant = new AntBuilder() ant.exec(outputproperty:"cmdOut", errorproperty: "cmdErr", resultproperty:"cmdExit", failonerror: "true", executable: /opt/myExecutable') { arg(line:'*"first with space"* second') } Concurrency - 39
  • …Process Enhancements • Process Management proc1 = 'ls'.execute() def process = "ls -l".execute() proc2 = 'tr -d o'.execute() println "Found text ${process.text}" proc3 = 'tr -d e'.execute() proc4 = 'tr -d i'.execute() proc1 | proc2 | proc3 | proc4 def process = "ls -l".execute() proc4.waitFor() process.in.eachLine { line -> if (proc4.exitValue()) println line } print proc4.err.text def sout = new StringBuffer() else def serr = new StringBuffer() print proc4.text proc1 = 'gzip -c'.execute() proc2 = 'gunzip -c'.execute() proc2.consumeProcessOutput(sout, serr) proc1 | proc2 proc1.consumeProcessErrorStream(serr) proc1.withWriter { writer -> writer << 'test text' } proc2.waitForOrKill(1000) println 'sout: ' + sout // => test text println 'serr: ' + serr Concurrency - 40
  • Parallelize your arrays with JSR 166y //Create a pool with size close to the number of processor cores def pool = new ForkJoinPool(2) def createParallelArray(pool, collection) { return ParallelArray.createFromCopy( collection.toArray(new Object[collection.size()]), pool) } // Enhance ArrayLists to find matching objects in parallel ArrayList.metaClass.findAll = {Closure cl -> createParallelArray(pool, delegate). withFilter({cl(it)} as Predicate).all().asList() } def sites=['http://www.jroller.com', 'http://www.infoq.com', 'http://java.dzone.com'] def groovySites = sites.findAll { new URL(it).text.toLowerCase().contains('groovy')} println "These sites talk about Groovy today: ${groovySites}" Source: http://www.jroller.com/vaclav/date/20080923 Concurrency - 41
  • Java Concurrency Best Practice? • Java Concurrency in Practice: – “If mutable threads access the same mutable state variable without appropriate synchronization, your program is broken” – “When designing thread-safe classes, good object-oriented techniques – encapsulation, immutability, and clear specification of invariants – are your best friends” Concurrency - 42
  • Immutability options • Built-in def animals = ['cat', 'dog', 'horse'].asImmutable() animals << 'fish' // => java.lang.UnsupportedOperationException • Google Collections – Numerous improved immutable collection types import com.google.common.collect.* List<String> animals = ImmutableList.of("cat", "dog", "horse") animals << 'fish' // => java.lang.UnsupportedOperationException • Groovy run-time metaprogramming def animals = ['cat', 'dog', 'horse'] ArrayList.metaClass.leftShift = { throw new UnsupportedOperationException() } animals << 'fish' // => java.lang.UnsupportedOperationException • Groovy 1.6+ compile-time metaprogramming – @Immutable can help us create such classes Concurrency - 43
  • @Immutable... • Java Immutable Class – As per Joshua Bloch // ... @Override Effective Java public boolean equals(Object obj) { if (this == obj) public final class Punter { return true; private final String first; if (obj == null) private final String last; return false; if (getClass() != obj.getClass()) public String getFirst() { return false; return first; Punter other = (Punter) obj; } if (first == null) { © ASERT 2006-2010 if (other.first != null) public String getLast() { return false; return last; } else if (!first.equals(other.first)) } return false; if (last == null) { @Override if (other.last != null) public int hashCode() { return false; final int prime = 31; } else if (!last.equals(other.last)) int result = 1; return false; result = prime * result + ((first == null) return true; ? 0 : first.hashCode()); } result = prime * result + ((last == null) ? 0 : last.hashCode()); @Override return result; public String toString() { } return "Punter(first:" + first + ", last:" + last + ")"; public Punter(String first, String last) { } this.first = first; this.last = last; } } // ... Concurrency - 44
  • ...@Immutable... • Java Immutable Class boilerplate – As per Joshua Bloch // ... @Override Effective Java public boolean equals(Object obj) { if (this == obj) public final class Punter { return true; private final String first; if (obj == null) private final String last; return false; if (getClass() != obj.getClass()) public String getFirst() { return false; return first; Punter other = (Punter) obj; } if (first == null) { © ASERT 2006-2010 if (other.first != null) public String getLast() { return false; return last; } else if (!first.equals(other.first)) } return false; if (last == null) { @Override if (other.last != null) public int hashCode() { return false; final int prime = 31; } else if (!last.equals(other.last)) int result = 1; return false; result = prime * result + ((first == null) return true; ? 0 : first.hashCode()); } result = prime * result + ((last == null) ? 0 : last.hashCode()); @Override return result; public String toString() { } return "Punter(first:" + first + ", last:" + last + ")"; public Punter(String first, String last) { } this.first = first; this.last = last; } } // ... Concurrency - 45
  • ...@Immutable @Immutable class Punter { String first, last © ASERT 2006-2010 } Concurrency - 46
  • @Synchronized – Before Transform class SynchronizedExample { private final myLock = new Object() @Synchronized static void greet() { println "world" } © ASERT 2006-2010 @Synchronized int answerToEverything() { return 42 } @Synchronized("myLock") void foo() { println "bar" } } Concurrency - 47
  • @Synchronized – After Transform class SynchronizedExample { Inspired by private static final $LOCK = new Object[0] Project private final $lock = new Object[0] Lombok private final myLock = new Object() static void greet() { synchronized ($LOCK) { println "world" } } © ASERT 2006-2010 int answerToEverything() { synchronized ($lock) { return 42 } } void foo() { synchronized (myLock) { println "bar" } } } Concurrency - 48
  • @Lazy... • Safe initialization idioms – Eager or fully synchronized import net.jcip.annotations.ThreadSafe @ThreadSafe class EagerInitialization { static final resource = new Resource() © ASERT 2006-2010 } @ThreadSafe class SafeLazyInitialization { private static ExpensiveResource resource synchronized static ExpensiveResource getInstance() { if (!resource) resource = new ExpensiveResource () resource } } Concurrency - 49
  • ...@Lazy... • Java Concurrency in Practice – Race condition with lazy initialization import net.jcip.annotations.NotThreadSafe @NotThreadSafe class LazyInitRace { © ASERT 2006-2010 private ExpensiveResource instance = null ExpensiveResource getInstance() { if (!instance) instance = new ExpensiveResource() instance } } – No problems: just apply Double-checked locking pattern Concurrency - 50
  • ...@Lazy... • Java Concurrency in Practice – Double checked locking anti-pattern @NotThreadSafe class DoubleCheckedLocking { private static ExpensiveResource instance = null © ASERT 2006-2010 static ExpensiveResource getInstance() { if (!instance) { synchronized (DoubleCheckedLocking) { if (!instance) instance = new ExpensiveResource() } } instance } } Concurrency - 51
  • ...@Lazy... • Java Concurrency in Practice – Double checked locking less broken-pattern class DoubleCheckedLocking { private static volatile ExpensiveResource instance = null static ExpensiveResource getInstance() { © ASERT 2006-2010 if (!instance) { synchronized (DoubleCheckedLocking) { if (!instance) instance = new ExpensiveResource() } } instance } } Concurrency - 52
  • ...@Lazy... • Java Concurrency in Practice – Lazy initialization holder class idiom import net.jcip.annotations.ThreadSafe @ThreadSafe class ResourceFactory { © ASERT 2006-2010 private static class ResourceHolder { static Resource resource = new Resource() } static Resource getResource() { ResourceHolder.resource } } Concurrency - 53
  • ...@Lazy @Lazy Resource first @Lazy volatile Resource second © ASERT 2006-2010 @Lazy volatile Resource third = { new Resource(args) }() @Lazy(soft = true) volatile Resource fourth Concurrency - 54
  • Topics • Groovy Intro • Useful Groovy features for Concurrency Related Concurrency Libraries & Tools • Fibonacci Case Study • GPars © ASERT 2006-2010 • More Info Concurrency - 55
  • Lightweight threads: Jetlang • Jetlang – A high performance java threading library Fiber receiver = new ThreadFiber(); // JAVA receiver.start(); // create java.util.concurrent.CountDownLatch to notify when message arrives final CountDownLatch latch = new CountDownLatch(1); // create channel to message between threads Channel<String> channel = new MemoryChannel<String>(); Callback<String> onMsg = new Callback<String>() { public void onMessage(String message) { //open latch latch.countDown(); } }; //add subscription for message on receiver thread channel.subscribe(receiver, onMsg); //publish message to receive thread. the publish method is thread safe. channel.publish("Hello"); //wait for receiving thread to receive message latch.await(10, TimeUnit.SECONDS); //shutdown thread receiver.dispose(); Concurrency - 56
  • Other High-Level Libraries: JPPF – Open source Grid Computing platform – http://www.jppf.org/ import org.jppf.client.* import java.util.concurrent.Callable class Task implements Callable, Serializable { private static final long serialVersionUID = 1162L; public Object call() { println 'Executing Groovy' "Hello JPPF from Groovy" } } def client = new JPPFClient() def job = new JPPFJob() def task = new Task() job.addTask task def results = client.submit(job) for (t in results) { if (t.exception) throw t.exception println "Result: " + t.result } Concurrency - 57
  • Other High-Level Libraries: Gruple... – http://code.google.com/p/gruple – Gruple aims to provide a simple abstraction to allow programmers to coordinate and synchronize threads with ease – based on Tuplespaces • Tuplespaces provide the illusion of a shared memory on top of a message passing system, along with a small set of operations to greatly simplify parallel programming – Example Tuple: [fname:"Vanessa", lname:"Williams", project:"Gruple"] – Basic operations within a Tuplespace are: • put - insert a tuple into the space • get - read a tuple from the space (non-destructively) • take - take a tuple from the space (a destructive read) – Further reading: Eric Freeman, Susanne Hupfer, and Ken Arnold. JavaSpaces Principles, Patterns, and Practice, Addison Wesley, 1999 Concurrency - 58
  • Other High-Level Libraries: ...Gruple... – Mandelbrot example (included in Gruple download) ... Space space = SpaceService.getSpace("mandelbrot") Map template = createTaskTemplate() Map task String threadName = Thread.currentThread().name while(true) { ArrayList points task = space.take(template) println "Worker $threadName got task ${task['start']} for job ${task['jobId'] points = calculateMandelbrot(task) Map result = createResult(task['jobId'], task['start'], points) println "Worker $threadName writing result for task ${result['start']} for jo space.put(result) } ... Concurrency - 59
  • Other High-Level Libraries: ...Gruple Concurrency - 60
  • Other High-Level Libraries: Cascading.groovy – API/DSL for executing tasks on a Hadoop cluster – http://www.cascading.org/ def assembly = builder.assembly(name: "wordcount") { eachTuple(args: ["line"], results: ["word"]) { regexSplitGenerator(declared: ["word"], pattern: /[.,]*s+/) } group(["word"]) everyGroup(args: ["word"], results: ["word", "count"]) { count() } group(["count"], reverse: true) } def map = builder.map() { source(name: "wordcount") { hfs(input) { text(["line"]) } } sink(name: "wordcount") { hfs(output) { text() } } } def flow = builder.flow(name: "wordcount", map: map, assembly: assembly) Concurrency - 61
  • Other High-Level Libraries: GridGain – Simple & productive to use grid computing platform – http://www.gridgain.com/ class GridHelloWorldGroovyTask extends GridTaskSplitAdapter<String, Integer> { Collection split(int gridSize, Object phrase) throws GridException { // ... } Object reduce(List results) throws GridException { // ... } } import static GridFactory.* start() def grid = getGrid() def future = grid.execute(GridHelloWorldGroovyTask, "Hello World") def phraseLen = future.get() stop(true) Concurrency - 62
  • Multiverse STM… – http://www.multiverse.org/ import org.multiverse.api.GlobalStmInstance import org.multiverse.api.Transaction import org.multiverse.templates.TransactionTemplate import org.multiverse.transactional.refs.LongRef def from = new Account(10) def to = new Account(10) atomic { from.balance -= 5 to.balance += 5 } println "from $from.balance" println "to $to.balance" ... Concurrency - 63
  • …Multiverse STM… ... void atomic(Closure block) { atomic([:], block) } void atomic(Map args, Closure block) { boolean readonly = args['readonly'] ?: false boolean trackreads = args['trackreads'] ?: true def txFactory = GlobalStmInstance.globalStmInstance. transactionFactoryBuilder. setReadonly(readonly). setReadTrackingEnabled(trackreads).build() new TransactionTemplate(txFactory) { Object execute(Transaction transaction) { block.call() return null } }.execute() } ... Concurrency - 64
  • …Multiverse STM class Account { private final balance = new LongRef() Account(long initial) { balance.set initial } void setBalance(long newBalance) { if (newBalance < 0) throw new RuntimeException("not enough money") balance.set newBalance } long getBalance() { balance.get() } } Concurrency - 65
  • Testing multi-threaded applications: ConTest... • Advanced Testing for Multi-Threaded Applications – Tool for testing, debugging, and coverage-measuring of concurrent programs (collects runtime statistics) – Systematically and transparently (using a java agent) schedules the execution of program threads in ways likely to reveal race conditions, deadlocks, and other intermittent bugs (collectively called synchronization problems) with higher than normal frequency – The ConTest run-time engine adds heuristically controlled conditional instructions (adjustable by a preferences file) that force thread switches, thus helping to reveal concurrent bugs. You can use existing tests and run ConTest multiple times – by default different heuristics used each time it is run • http://www.alphaworks.ibm.com/tech/contest Concurrency - 66
  • ...Testing multi-threaded applications: ConTest NUM = 5 ParalInc.groovy count = 0 def incThread = { n -> Thread.start{ sleep n*10 targetClasses = ParalInc //synchronized(ParalInc) { timeoutTampering = true count++ noiseFrequency = 500 //} strength = 10000 } } def threads = (1..NUM).collect(incThread) threads.each{ it.join() } assert count == NUM > groovyc ParalInc.groovy > java -javaagent:../../Lib/ConTest.jar -cp %GROOVY_JAR%;. ParalInc Exception in thread "main" Assertion failed: assert count == NUM | | | 4 | 5 false Concurrency - 67
  • GContracts @Grab('org.gcontracts:gcontracts:1.0.2') import org.gcontracts.annotations.* 1.8+ @Invariant({ first != null && last != null }) class Person { String first, last @Requires({ delimiter in ['.', ',', ' '] }) @Ensures({ result == first + delimiter + last }) String getName(String delimiter) { first + delimiter + last } } new Person(first: 'John', last: 'Smith').getName('.') Concurrency - 68
  • Testing: Spock class HelloSpock extends spock.lang.Specification { def "length of Spock's and his friends' names"() { expect: name.size() == length where: name | length "Spock" | 5 "Kirk" | 4 "Scotty" | 6 } } Concurrency - 69
  • Topics • Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools Fibonacci Case Study • GPars © ASERT 2006-2010 • More Info Concurrency - 70
  • Fibonacci Case Study Concurrency - 71
  • Fibonacci… START = 8 Serial END = 16 version fib = {n -> n < 2 ? n : fib(n - 1) + fib(n - 2) } (START..END).each {num -> println "n:$num => ${fib(num)}" } Concurrency - 72
  • …Fibonacci… import java.util.concurrent.* ConcurrentHashMap THREADS = 4 version START = 8 END = 16 QUIT = -1 class Fibonacci { def values = new ConcurrentHashMap() int calc(x) { x < 2 ? x : calc(x-1) + calc(x-2) } int calcWithCache(x) { def result = values[x] if (!result) { result = calc(x) values.putIfAbsent(x, result) } result } } println "Calculating Fibonacci sequence in parallel..." def queue = new ArrayBlockingQueue(10) Concurrency - 73
  • …Fibonacci… Thread.start('Producer') { int x = START while (x <= END) { sleep 200 queue << x++ } sleep 1000 THREADS.times { queue << QUIT } } (1..THREADS).each { def name = "Consumer$it" Thread.start(name) { def done = false def fib = new Fibonacci() while (!done) { def n = queue.take() if (n == QUIT) done = true else println "$name n:$n => ${fib.calcWithCache(n)}" } } } Concurrency - 74
  • …Fibonacci… Executor import java.util.concurrent.* version CUTOFF = 12 // not worth parallelizing for small n THREADS = 100 println "Calculating Fibonacci sequence in parallel..." serialFib = {n -> (n < 2) ? n : serialFib(n - 1) + serialFib(n - 2) } pool = Executors.newFixedThreadPool(THREADS) defer = {c -> pool.submit(c as Callable) } fib = {n -> if (n < CUTOFF) return serialFib(n) def left = defer { fib(n - 1) } def right = defer { fib(n - 2) } left.get() + right.get() } (8..16).each {n -> println "n=$n => ${fib(n)}" } pool.shutdown() Concurrency - 75
  • …Fibonacci… import EDU.oswego.cs.dl.util.concurrent.FJTask import EDU.oswego.cs.dl.util.concurrent.FJTaskRunnerGroup class Fib extends FJTask { static final CUTOFF = 12 Fork/Join volatile int number version int getAnswer() { if (!isDone()) throw new IllegalStateException() number } void run() { int n = number if (n <= CUTOFF) number = seqFib(n) else { def f1 = new Fib(number: n - 1) def f2 = new Fib(number: n - 2) coInvoke(f1, f2) number = f1.number + f2.number } } int seqFib(int n) { n < 2 ? n : seqFib(n - 1) + seqFib(n - 2) } } Concurrency - 76
  • …Fibonacci… def THREADS = 2 def group = new FJTaskRunnerGroup(THREADS) def START = 8 def END = 16 (START..END).each {num -> def f = new Fib(number: num) group.invoke(f) println "n:$num => $f.answer" } Concurrency - 77
  • …Fibonacci… import fj.* import fj.control.parallel.Strategy FunctionalJava import static fj.Function.curry as fcurry version import static fj.P1.curry as pcurry import static fj.P1.fmap import static fj.control.parallel.Actor.actor import static fj.control.parallel.Promise.* import static fj.data.List.range import static java.util.concurrent.Executors.* CUTOFF = 12 // not worth parallelizing for small n START = 8 END = 16 THREADS = 4 pool = newFixedThreadPool(THREADS) su = Strategy.executorStrategy(pool) spi = Strategy.executorStrategy(pool) add = fcurry({a, b -> a + b } as F2) nums = range(START, END + 1) println "Calculating Fibonacci sequence in parallel..." Concurrency - 78
  • …Fibonacci… serialFib = {n -> n < 2 ? n : serialFib(n - 1) + serialFib(n - 2) } print = {results -> def n = START results.each { println "n=${n++} => $it" } pool.shutdown() } as Effect calc = {n -> n < CUTOFF ? promise(su, P.p(serialFib(n))) : calc.f(n - 1).bind(join(su, pcurry(calc).f(n - 2)), add) } as F out = actor(su, print) join(su, fmap(sequence(su)).f(spi.parMapList(calc).f(nums))).to(out) Concurrency - 79
  • …Fibonacci… import org.jetlang.core.Callback import org.jetlang.channels.MemoryChannel Jetlang import org.jetlang.fibers.ThreadFiber version import java.util.concurrent.* println "Calculating Fibonacci sequence with two cooperating fibers..." class FibonacciCalc implements Callback { private channel, receiver def limit, name, latch void onMessage(inpair) { def next = inpair[0] + inpair[1] def outpair = [inpair[1], next] println "$name next:$next" channel.publish(outpair) if (next > limit) { latch.countDown() sleep 200 receiver.dispose() } } // ... Concurrency - 80
  • …Fibonacci // ... void subscribe(other) { channel.subscribe(receiver, other) } FibonacciCalc() { channel = new MemoryChannel() receiver = new ThreadFiber() receiver.start() } } def seed = [0, 1] def latch = new CountDownLatch(2) def calcA = new FibonacciCalc(limit: 500, name: 'CalcA', latch: latch) def calcB = new FibonacciCalc(limit: 500, name: 'CalcB', latch: latch) calcA.subscribe calcB calcB.subscribe calcA calcA.onMessage seed latch.await(10, TimeUnit.SECONDS) Concurrency - 81
  • Topics • Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • Fibonacci Case Study GPars © ASERT 2006-2010 • More Info Concurrency - 82
  • GPars • http://gpars.codehaus.org/ • Library classes and DSL sugar providing intuitive ways for Groovy developers to handle tasks concurrently. Logical parts: – Actors provide a Groovy implementation of Scala-like actors including "remote" actors on other machines – Dataflow Concurrency supports natural shared-memory © ASERT 2006-2010 concurrency model, using single-assignment variables – Asynchronizer extends the Java 1.5 built-in support for executor services to enable multi-threaded collection and closure processing – Parallelizer uses JSR-166y Parallel Arrays to enable multi-threaded collection processing – Safe a non-blocking mt-safe reference to mutable state that is inspired by "agents" in Clojure Concurrency - 83
  • GPars: Parallel Collection Functions def nums = 1..100000 def squares = nums .collect{ it ** 2 } .grep{ it % 7 == it % 5 } .grep{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] © ASERT 2006-2010 @Grab('org.codehaus.gpars:gpars:0.10') import static groovyx.gpars.GParsPool.withPool def nums = 1..100000 withPool(5) { def squares = nums. collectParallel{ it ** 2 }. grepParallel{ it % 7 == it % 5 }. grepParallel{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] } Concurrency - 85
  • GPars: Transparent Parallel Collections • Applies some Groovy metaprogramming import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 nums.makeTransparent() © ASERT 2006-2010 def squares = nums. collect{ it ** 2 }. grep{ it % 7 == it % 5 }. grep{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] } Concurrency - 86
  • Gpars concurrency-aware methods Transparent Transitive? Parallel any { ... } anyParallel { ... } collect { ... } yes collectParallel { ... } count(filter) countParallel(filter) each { ... } eachParallel { ... } eachWithIndex { ... } eachWithIndexParallel { ... } every { ... } everyParallel { ... } find { ... } findParallel { ... } findAll { ... } yes findAllParallel { ... } findAny { ... } findAnyParallel { ... } fold { ... } foldParallel { ... } fold(seed) { ... } foldParallel(seed) { ... } grep(filter) yes grepParallel(filter) groupBy { ... } groupByParallel { ... } max { ... } maxParallel { ... } max() maxParallel() min { ... } minParallel { ... } min() minParallel() split { ... } yes splitParallel { ... } Source: ReGina Concurrency - 87
  • GPars: Map-Reduce... import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 println nums.parallel. © ASERT 2006-2010 map{ it ** 2 }. filter{ it % 7 == it % 5 }. filter{ it % 3 == 0 }. collection } Concurrency - 88
  • ...GPars: Map-Reduce import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 println nums.parallel. © ASERT 2006-2010 map{ it ** 2 }. filter{ it % 7 == it % 5 }. filter{ it % 3 == 0 }. reduce{ a, b -> a + b } } Concurrency - 89
  • Parallel Collections vs Map-Reduce Source: ReGina Concurrency - 90
  • GPars: Dataflows... import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.result = flow.x + flow.y } task { flow.x = 10 } task { flow.y = 5 } © ASERT 2006-2010 assert 15 == flow.result new DataFlows().with { task { result = x * y } task { x = 10 } task { y = 5 } assert 50 == result } Concurrency - 91
  • ...GPars: Dataflows... • Evaluating: result = (a – b) * (a + b) x y import groovyx.gpars.dataflow.DataFlows © ASERT 2006-2010 import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() 10 5 task { flow.a = 10 } task { flow.b = 5 } task { flow.x = flow.a - flow.b } a b task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } - + assert flow.result == 75 * Concurrency - 92
  • ...GPars: Dataflows... • Naive attempt for loops import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() ... [10, 20].each { thisA -> task { flow.a = 10 } [4, 5].each { thisB -> ... © ASERT 2006-2010 task { flow.a = thisA } task { flow.a = 20 } task { flow.b = thisB } task { flow.x = flow.a - flow.b } task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } println flow.result } } // => java.lang.IllegalStateException: A DataFlowVariable can only be assigned once. Concurrency - 93
  • ...GPars: Dataflows... import groovyx.gpars.dataflow.DataFlowStream 10 4 import static groovyx.gpars.dataflow.DataFlow.* 10 5 20 4 final streamA = new DataFlowStream() 20 5 final streamB = new DataFlowStream() final streamX = new DataFlowStream() a b final streamY = new DataFlowStream() final results = new DataFlowStream() - + © ASERT 2006-2010 operator(inputs: [streamA, streamB], outputs: [streamX, streamY]) { * a, b -> streamX << a - b; streamY << a + b } operator(inputs: [streamX, streamY], outputs: [results]) { x, y -> results << x * y } 84 75 [[10, 20], [4, 5]].combinations().each{ thisA, thisB -> 384 task { streamA << thisA } 375 task { streamB << thisB } } 4.times { println results.val } Concurrency - 94
  • ...GPars: Dataflows • Amenable to static analysis • Race conditions avoided • Deadlocks “typically” become repeatable import groovyx.gpars.dataflow.DataFlows © ASERT 2006-2010 import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.x = flow.y } task { flow.y = flow.x } Concurrency - 95
  • GPars: Dataflow Sieve final int requestedPrimeNumberCount = 1000 final DataFlowStream initialChannel = new DataFlowStream() task { (2..10000).each { initialChannel << it } } def filter(inChannel, int prime) { def outChannel = new DataFlowStream() © ASERT 2006-2010 operator([inputs: [inChannel], outputs: [outChannel]]) { if (it % prime != 0) { bindOutput it } } return outChannel } def currentOutput = initialChannel requestedPrimeNumberCount.times { int prime = currentOutput.val println "Found: $prime" currentOutput = filter(currentOutput, prime) } Source: http://groovyconsole.appspot.com/script/235002 Concurrency - 96
  • GPars: Actors... • Predefined coordination • Class with the with fork/join & following lifecycle & map/filter/reduce methods • Implicit coordination – But also DSL sugar & with dataflow augmentation • Actors provide explicit start() © ASERT 2006-2010 stop() coordination and act() enforce* no sharing of send(msg) state, process a single sendAndWait(msg) activity/message at a loop { } time react { msg -> } msg.reply(replyMsg) receive() join() * mostly Concurrency - 97
  • …GPars: Actors... import static groovyx.gpars.actor.Actors.* def decrypt = reactor { code -> code.reverse() } def audit = reactor { println it } def main = actor { decrypt 'terces pot' © ASERT 2006-2010 react { plainText -> audit plainText } } main.join() audit.stop() audit.join() Source: ReGina Concurrency - 98
  • …GPars: Actors... final class FilterActor extends DynamicDispatchActor { private final int myPrime private def follower def FilterActor(final myPrime) { this.myPrime = myPrime; } def onMessage(int value) { if (value % myPrime != 0) { if (follower) follower value else { println "Found $value" © ASERT 2006-2010 follower = new FilterActor(value).start() } } } def onMessage(def poisson) { if (follower) { def sender = poisson.sender follower.sendAndContinue(poisson, {this.stop(); sender?.send('Done } else { //I am the last in the chain stop() reply 'Done' } } } Source: http://groovyconsole.appspot.com/script/242001 Concurrency - 99
  • …GPars: Actors (2..requestedPrimeNumberBoundary).each { firstFilter it } firstFilter.sendAndWait 'Poisson' © ASERT 2006-2010 Source: http://groovyconsole.appspot.com/script/242001 Concurrency - 100
  • GPars: Agents... • Agents safeguard non-thread safe objects • Only the agent can update the underlying object • “Code” to update the protected object is sent to the agent © ASERT 2006-2010 • Can be used with other approaches Concurrency - 101
  • …GPars: Agents @Grab('org.codehaus.gpars:gpars:0.10') import groovyx.gpars.agent.Agent def speakers = new Agent<List>(['Alex'], {it?.clone()}) // add Alex speakers.send {updateValue it << 'Hilary'} // add Hilary final Thread t1 = Thread.start { speakers.send {updateValue it << 'Ken'} // add Ken } © ASERT 2006-2010 final Thread t2 = Thread.start { speakers << {updateValue it << 'Guy'} // add Guy speakers << {updateValue it << 'Ralph'} // add Ralph } [t1, t2]*.join() assert new HashSet(speakers.val) == new HashSet(['Alex', 'Hilary', 'Ken', 'Guy', 'Ralph']) Source: Gpars examples Concurrency - 102
  • GPars for testing @Grab('net.sourceforge.htmlunit:htmlunit:2.6') import com.gargoylesoftware.htmlunit.WebClient @Grab('org.codehaus.gpars:gpars:0.10') import static groovyx.gpars.GParsPool.* def testCases = [ ['Home', 'Bart', 'Content 1'], ['Work', 'Homer', 'Content 2'], ['Travel', 'Marge', 'Content 3'], © ASERT 2006-2010 ['Food', 'Lisa', 'Content 4'] ] withPool(3) { testCases.eachParallel{ category, author, content -> postAndCheck category, author, content } } private postAndCheck(category, author, content) { ... Concurrency - 103
  • Guy Steele example in Groovy… Sequential version def words = { s -> Guy Steele‟s example from keynote (from slide 52 onwards for several slides): http://strangeloop2010.com/talk/presentation_file/14299/GuySteele-parallel.pdf def result = [] def word = '' s.each{ ch -> if (ch == ' ') { if (word) result += word word = '' } else word += ch © ASERT 2006-2010 } if (word) result += word result } assert words("This is a sample") == ['This', 'is', 'a', 'sample'] assert words(" Here is another sample ") == ['Here', 'is', 'another', 'sample'] assert words("JustOneWord") == ['JustOneWord'] assert words("Here is a sesquipedalian string of words") == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words'] assert words(" ") == [] && words("") == [] Concurrency - 104
  • …Guy Steele example in Groovy… @Immutable class Chunk { Refactored sequential version String s def plus(Chunk other) { new Chunk(s + other.s) } def plus(Segment other) { new Segment(s + other.l, other.m, other.r) } } @Immutable class Segment { String l; List m; String r © ASERT 2006-2010 def plus(Chunk other) { new Segment(l, m, r + other.s) } def plus(Segment other) { new Segment(l, m + maybeWord(r + other.l) + other.m, other.r) } } class Util { static processChar(ch) { ch == ' ' ? new Segment('', [], '') : new Chunk(ch) } static maybeWord(s) { s ? [s] : [] } } import static Util.* ... Concurrency - 105
  • …Guy Steele example in Groovy… def words = { s -> Refactored sequential version def result s.each{ ch -> if (!result) result = processChar(ch) else result += processChar(ch) } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } case null: return [] © ASERT 2006-2010 } } assert words("This is a sample") == ['This', 'is', 'a', 'sample'] assert words(" Here is another sample ") == ['Here', 'is', 'another', 'sample'] assert words("JustOneWord") == ['JustOneWord'] assert words("Here is a sesquipedalian string of words") == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words'] assert words(" ") == [] && words("") == [] Concurrency - 106
  • …Guy Steele example in Groovy… Roll your own threading with ConcurrentHashMap version def swords = { s -> def result s.each{ ch -> if (!result) result = processChar(ch) else result += processChar(ch) } result ?: new Chunk('') } THREADS = 4 © ASERT 2006-2010 def words = { s -> int n = (s.size() + THREADS - 1) / THREADS def map = new java.util.concurrent.ConcurrentHashMap() (0..<THREADS).collect { i -> Thread.start { def (min, max) = [[s.size(),i*n].min(), [s.size(),(i+1)*n].min()] map[i] = swords(s[min..<max]) }}*.join() def result = map.entrySet().sort{ it.key }.sum{ it.value } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } } Concurrency - 107
  • …Guy Steele example in Groovy… DataFlow version: partially hard-coded to 4 partitions for easier reading def words = { s -> int n = (s.size() + THREADS - 1) / THREADS def min = (0..<THREADS).collectEntries{ [it, [s.size(),it*n].min()] } def max = (0..<THREADS).collectEntries{ [it, [s.size(),(it+1)*n].min()] } def result = new DataFlows().with { task { a = swords(s[min[0]..<max[0]]) } task { b = swords(s[min[1]..<max[1]]) } task { c = swords(s[min[2]..<max[2]]) } task { d = swords(s[min[3]..<max[3]]) } © ASERT 2006-2010 task { sum1 = a + b } task { sum2 = c + d } task { sum = sum1 + sum2 } println 'Tasks ahoy!' sum } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } } Concurrency - 108
  • …Guy Steele example in Groovy… Fork/Join version GRANULARITY_THRESHHOLD = 10 THREADS = 4 println GParsPool.withPool(THREADS) { def result = runForkJoin(0, input.size(), input){ first, last, s -> def size = last - first if (size <= GRANULARITY_THRESHHOLD) { swords(s[first..<last]) } else { // divide and conquer © ASERT 2006-2010 def mid = first + ((last - first) >> 1) forkOffChild(first, mid, s) forkOffChild(mid, last, s) childrenResults.sum() } } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } } Concurrency - 109
  • …Guy Steele example in Groovy… Map/Reduce version THRESHHOLD = 10 def split(raw) { raw.size() <= THRESHHOLD ? raw : [raw[0..<THRESHHOLD]] + split(raw.substring(THRESHHOLD)) } © ASERT 2006-2010 println GParsPool.withPool(THREADS) { def ans = split(input).parallel.map(swords).reduce{ a,b -> a + b } switch(ans) { case Chunk: return maybeWord(ans.s) case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) } } } Concurrency - 110
  • …Guy Steele example in Groovy Just leveraging the algorithm‟s parallel nature println GParsPool.withPool(THREADS) { def ans = input.collectParallel{ processChar(it) }.sum() switch(ans) { case Chunk: return maybeWord(ans.s) case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) } © ASERT 2006-2010 } } Concurrency - 111
  • Topics • Groovy Intro • Useful Groovy features for Concurrency • Related Concurrency Libraries & Tools • Fibonacci Case Study • GPars © ASERT 2006-2010 More Info Concurrency - 112
  • More Information about Concurrency • Web sites – http://gpars.codehaus.org/ – http://g.oswego.edu/ Doug Lea's home page – http://gee.cs.oswego.edu/dl/concurrency-interest/ – http://jcip.net/ Companion site for Java Concurrency in Practice – http://www.eecs.usma.edu/webs/people/okasaki/pubs.html#cup98 Purely Functional Data Structures – http://delicious.com/kragen/concurrency Concurrency bookmark list – http://www.gotw.ca/publications/concurrency-ddj.htm The Free Lunch is Over, Herb Sutter – http://manticore.cs.uchicago.edu/papers/damp07.pdf – http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=10142 Concepts, Techniques, and Models of Computer Programming Concurrency - 113
  • More Information about Groovy • Web sites – http://groovy.codehaus.org – http://grails.codehaus.org – http://pleac.sourceforge.net/pleac_groovy (many examples) – http://www.asert.com.au/training/java/GV110.htm (workshop) • Mailing list for users – user@groovy.codehaus.org • Information portals – http://www.aboutgroovy.org – http://www.groovyblogs.org • Documentation (1000+ pages) – Getting Started Guide, User Guide, Developer Guide, Testing Guide, Cookbook Examples, Advanced Usage Guide • Books – Several to choose from ... Concurrency - 114
  • More Information: Groovy in Action Concurrency - 115