Your SlideShare is downloading. ×
Functional Groovy
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Functional Groovy

374
views

Published on

Speaker: Paul King …

Speaker: Paul King
Groovy doesn't claim to be a fully-fledged functional programming language but it does provide the Java or Groovy developer with a whole toolbox of features for doing functional style programs. This talk looks at the key Groovy features which support a functional style. Topics covered include using closures, currying and partial evaluation, closure composition, useful functional-centric AST macros, useful functional-centric runtime meta-programming tricks, trampolining, using Java functional libraries, immutable data structures, lazy and infinite lists, using Groovy 2's static typing and approaches for moving beyond Java's type system.
There are many advantages to using a functional style in your programs. Learn what can be done to leverage functional style while retaining many of the productivity gains of the Groovy programming language.

Published in: Technology, Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
374
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. © ASERT 2006-2013 Functional Groov Dr Paul King @paulk_asert http:/slideshare.net/paulk_asert/functional-groovy https://github.com/paulk-asert/functional-groovy
  • 2. Topics © ASERT 2006-2013 Intro • Functional Style • Design Patterns • Immutability • Laziness • GPars • Word Split (bonus material) • More Info
  • 3. Introduction • What is functional programming? – Favour evaluation of composable expressions over execution of commands – Encourage particular idioms such as sideeffect free functions & immutability © ASERT 2006-2013 • And why should I care? – – – – – Declarative understandable code Reduction of errors Better patterns and approaches to design Improved reusability Leverage concurrency
  • 4. What makes up functional style? • Functions, Closures, Lambda, Blocks as first-class citizens • Higher order functions • Mutable vs Immutable data structures • Recursion • Lazy vs Eager evaluation • Declarative vs Imperative style • Advanced Techniques – Memoization, Trampolines, Composition and Curry • Compile-time safety • Concurrency
  • 5. What makes up functional style? • Functions, Closures, Lambda, Blocks as first-class citizens • Higher order functions • Mutable vs Immutable data structures • Recursion • Lazy vs Eager evaluation • Declarative vs Imperative style • Advanced Techniques – Memoization, Trampolines, Composition and Curry • Compile-time safety • Concurrency
  • 6. Using Closures... • Code deserves to be free public class Main { public static void main(String[] args) { print(reverse("Hello")); } © ASERT 2006-2013 public static String reverse(String arg) { return arg.reverse(); } public void static print(String arg) { System.out.println(arg); } } Main.main();
  • 7. ...Using Closures... • Code deserves to be free def code = { arg -> println "Hello $arg" } code.call('Washington') code('Washington') // => Hello Washington © ASERT 2006-2013 def codeListInG = [ { println it }, { it.reverse() } ] def main(first, second, arg) { def third = second >> first third(arg) } // def main = { Closure first, Closure second, String arg -> // def mainClosure = this.&main main(codeListInG[0], codeListInG[1], "Hello") // => olleH
  • 8. ...Using Closures... • Used for many things in Groovy: © ASERT 2006-2013 • • • • • • • • new File('/x.txt').eachLine Iterators println it Callbacks } Higher-order functions Specialized control structures Dynamic method definition Resource allocation 3.times { println 'Hi' } Threads [0, 1, 2].each { number -> Continuation-like coding println number def houston(Closure doit) { (10..1).each { count -> doit(count) } } houston { println it } { } [0, 1, 2].each { println it} def printit = { println it } [0, 1, 2].each printit
  • 9. ...Using Closures import static Math.* piA piB piC piD piE piF = = = = = = { { { { { { Algorithm Algorithm Algorithm Algorithm Algorithm Algorithm 22 / 7 } 333/106 } 355/113 } 0.6 * (3 + sqrt(5)) } 22/17 + 37/47 + 88/83 } sqrt(sqrt(2143/22)) } piE piF piC piD piB piA differs differs differs differs differs differs by by by by by by 1.0206946399193839E-11 1.0070735356748628E-9 2.668102068170697E-7 4.813291008076703E-5 8.321958979307098E-5 0.001264489310206951 © ASERT 2006-2013 howCloseToPI = { abs(it.value() - PI) } algorithms = [piA:piA, piB:piB, piC:piC, piD:piD, piE:piE, piF:piF] findBestPI(algorithms) def findBestPI(map) { map.entrySet().sort(howCloseToPI).each { entry -> def diff = howCloseToPI(entry) println "Algorithm $entry.key differs by $diff" } }
  • 10. Better Design Patterns: Builder • Markup Builder © ASERT 2006-2013 import groovy.xml.* def page = new MarkupBuilder() page.html { head { title 'Hello' } body { ul { for (count in 1..5) { li "world $count" } } } } <html> <head> <title>Hello</title> </head> <body> <ul> <li>world 1</li> <li>world 2</li> <li>world 3</li> <li>world 4</li> <li>world 5</li> </ul> </body> </html>
  • 11. SwingBuilder import java.awt.FlowLayout builder = new groovy.swing.SwingBuilder() langs = ["Groovy", "Ruby", "Python", "Pnuts"] © ASERT 2006-2013 gui = builder.frame(size: [290, 100], title: 'Swinging with Groovy!') { panel(layout: new FlowLayout()) { panel(layout: new FlowLayout()) { for (lang in langs) { checkBox(text: lang) } } button(text: 'Groovy Button', actionPerformed: { builder.optionPane(message: 'Indubitably Groovy!'). createDialog(null, 'Zen Message').show() }) button(text: 'Groovy Quit', actionPerformed: {System.exit(0)}) } } gui.show() Source: http://www.ibm.com/developerworks/java/library/j-pg04125/
  • 12. Better File Manipulation... import import import import import import import import import import import java.util.List; java.util.ArrayList; java.util.Arrays; java.io.File; java.io.FileOutputStream; java.io.PrintWriter; java.io.FileNotFoundException; java.io.IOException; java.io.BufferedReader; java.io.InputStreamReader; java.io.FileInputStream; © ASERT 2006-2013 public class PrintJavaSourceFileLinesThatContainTheWordJava { public static void main(String[] args) { File outfile = new File("result.txt"); outfile.delete(); File basedir = new File(".."); List<File> files = new ArrayList<File>(); files.add(basedir); FileOutputStream fos = null; PrintWriter out = null; try { fos = new FileOutputStream(outfile); out = new PrintWriter(fos); while (!files.isEmpty()) { File file = files.remove(0); if (file.isDirectory()) { files.addAll(Arrays.asList(file.listFiles())); } else { if (file.getName().endsWith(".java")) { processFile(file, out); } } } } catch (FileNotFoundException e) { e.printStackTrace(); } finally { if (out != null) { out.close(); } if (fos != null) { try { fos.close(); } catch (IOException e) { e.printStackTrace(); } } } } // ... // ... private static void processFile(File file, PrintWriter out) { FileInputStream fis = null; InputStreamReader isr = null; BufferedReader reader = null; try { fis = new FileInputStream(file); isr = new InputStreamReader(fis); reader = new BufferedReader(isr); String nextline; int count = 0; while ((nextline = reader.readLine()) != null) { count++; if (nextline.toLowerCase().contains("java")) { out.println("File '" + file + "' on line " + count); out.println(nextline); out.println(); } } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (reader != null) { try { reader.close(); } catch (IOException e) { e.printStackTrace(); } } if (isr != null) { try { isr.close(); } catch (IOException e) { e.printStackTrace(); } } if (fis != null) { try { fis.close(); } catch (IOException e) { e.printStackTrace(); } } } } }
  • 13. ...Better File Manipulation... import import import import import import import import import import import java.util.List; java.util.ArrayList; java.util.Arrays; java.io.File; java.io.FileOutputStream; java.io.PrintWriter; java.io.FileNotFoundException; java.io.IOException; java.io.BufferedReader; java.io.InputStreamReader; java.io.FileInputStream; © ASERT 2006-2013 public class PrintJavaSourceFileLinesThatContainTheWordJava { public static void main(String[] args) { File outfile = new File("result.txt"); outfile.delete(); File basedir = new File(".."); List<File> files = new ArrayList<File>(); files.add(basedir); FileOutputStream fos = null; PrintWriter out = null; try { fos = new FileOutputStream(outfile); out = new PrintWriter(fos); while (!files.isEmpty()) { File file = files.remove(0); if (file.isDirectory()) { files.addAll(Arrays.asList(file.listFiles())); } else { if (file.getName().endsWith(".java")) { processFile(file, out); } } } } catch (FileNotFoundException e) { e.printStackTrace(); } finally { if (out != null) { out.close(); } if (fos != null) { try { fos.close(); } catch (IOException e) { e.printStackTrace(); } } } } // ... boilerplate // ... private static void processFile(File file, PrintWriter out) { FileInputStream fis = null; InputStreamReader isr = null; BufferedReader reader = null; try { fis = new FileInputStream(file); isr = new InputStreamReader(fis); reader = new BufferedReader(isr); String nextline; int count = 0; while ((nextline = reader.readLine()) != null) { count++; if (nextline.toLowerCase().contains("java")) { out.println("File '" + file + "' on line " + count); out.println(nextline); out.println(); } } } catch (FileNotFoundException e) { e.printStackTrace(); } catch (IOException e) { e.printStackTrace(); } finally { if (reader != null) { try { reader.close(); } catch (IOException e) { e.printStackTrace(); } } if (isr != null) { try { isr.close(); } catch (IOException e) { e.printStackTrace(); } } if (fis != null) { try { fis.close(); } catch (IOException e) { e.printStackTrace(); } } } } }
  • 14. ...Better File Manipulation © ASERT 2006-2013 def out = new File('result.txt') out.delete() new File('..').eachFileRecurse { file -> if (file.name.endsWith('.groovy')) { file.eachLine { line, num -> if (line.toLowerCase().contains('groovy')) out << "File '$file' on line $numn$linenn" } } } File '..filessrcPrintGroovySourceLines.groovy' on line 4 if (file.name.endsWith('.groovy')) { File '..filessrcPrintGroovySourceLines.groovy' on line 6 if (line.toLowerCase().contains('groovy')) File '..jdbcsrcJdbcGroovy.groovy' on line 1 import groovy.sql.Sql …
  • 15. DSL example... Object.metaClass.please = { clos -> clos(delegate) } Object.metaClass.the = { clos -> delegate[1](clos(delegate[0])) } © ASERT 2006-2013 show = { thing -> [thing, { println it }] } square_root = { Math.sqrt(it) } given = { it } given 100 please show the square_root // ==> 10.0
  • 16. ...DSL example... Object.metaClass.of = { delegate[0](delegate[1](it)) } Object.metaClass.the = { clos -> [delegate[0], clos] } © ASERT 2006-2013 show = [{ println it }] square_root = { Math.sqrt(it) } please = { it } please show the square_root of 100 // ==> 10.0
  • 17. ...DSL example... show = { println it } square_root = { Math.sqrt(it) } © ASERT 2006-2013 def please(action) { [the: { what -> [of: { n -> action(what(n)) }] }] } please show the square_root of 100 // ==> 10.0 Inspiration for this example came from …
  • 18. ...DSL example // Japanese DSL using GEP3 rules Object.metaClass.を = Object.metaClass.の = { clos -> clos(delegate) } © ASERT 2006-2013 まず = { it } 表示する = { println it } 平方根 = { Math.sqrt(it) } まず 100 の 平方根 を 表示する // First, show the square root of 100 // => 10.0 // source: http://d.hatena.ne.jp/uehaj/20100919/1284906117 // http://groovyconsole.appspot.com/edit/241001
  • 19. Topics © ASERT 2006-2013 • Intro Functional Style • Design Patterns • Immutability • Laziness • GPars • Word Split (bonus material) • More Info
  • 20. What makes up functional style? • Functions, Closures, Lambda, Blocks as first-class citizens • Higher order functions • Mutable vs Immutable data structures • Recursion • Lazy vs Eager evaluation • Declarative vs Imperative style • Advanced Techniques – Memoization, Trampolines, Composition and Curry • Compile-time safety • Concurrency
  • 21. Show me the code Example1.groovy
  • 22. What makes up functional style? • Functions, Closures, Lambda, Blocks as first-class citizens • Higher order functions • Mutable vs Immutable data structures • Recursion • Lazy vs Eager evaluation • Declarative vs Imperative style • Advanced Techniques – Memoization, Trampolines, Composition and Curry • Compile-time safety • Concurrency
  • 23. Show me the code Example2.groovy
  • 24. Topics © ASERT 2006-2013 • Intro • Functional Style Design Patterns • Immutability • Laziness • GPars • Word Split (bonus material) • More Info
  • 25. Language features instead of Patterns interface Calc { def execute(n, m) } class CalcByMult implements Calc { def execute(n, m) { n * m } } (c) ASERT 2006-2013 class CalcByManyAdds implements Calc { def execute(n, m) { def result = 0 n.times { result += m } return result } } def sampleData = [ [3, 4, 12], [5, -5, -25] ] Calc[] multiplicationStrategies = [ new CalcByMult(), new CalcByManyAdds() ] Strategy Pattern with interfaces with closures def multiplicationStrategies = [ { n, m -> n * m }, { n, m -> def total = 0; n.times{ total += m }; total }, { n, m -> ([m] * n).sum() } ] def sampleData = [ [3, 4, 12], [5, -5, -25] ] sampleData.each{ data -> multiplicationStrategies.each{ calc -> assert data[2] == calc(data[0], data[1]) } } sampleData.each {data -> multiplicationStrategies.each {calc -> assert data[2] == calc.execute(data[0], data[1]) } }
  • 26. Adapter Pattern… class RoundPeg { def radius String toString() { "RoundPeg with radius $radius" } } (c) ASERT 2006-2013 class RoundHole { def radius def pegFits(peg) { peg.radius <= radius } String toString() { "RoundHole with radius $radius" } } def pretty(hole, peg) { if (hole.pegFits(peg)) println "$peg fits in $hole" else println "$peg does not fit in $hole" } def hole = new RoundHole(radius:4.0) (3..6).each { w -> pretty(hole, new RoundPeg(radius:w)) } RoundPeg with radius RoundPeg with radius RoundPeg with radius RoundPeg with radius 3 fits in RoundHole with radius 4.0 4 fits in RoundHole with radius 4.0 5 does not fit in RoundHole with radius 4.0 6 does not fit in RoundHole with radius 4.0
  • 27. …Adapter Pattern… class SquarePeg { def width String toString() { "SquarePeg with width $width" } } (c) ASERT 2006-2013 class SquarePegAdapter { def peg def getRadius() { Math.sqrt(((peg.width/2) ** 2)*2) } String toString() { "SquarePegAdapter with width $peg.width (and notional radius $radius)" } } def hole = new RoundHole(radius:4.0) (4..7).each { w -> pretty(hole, new SquarePegAdapter(peg: new SquarePeg(width: w))) } SquarePegAdapter with width 4 (and notional radius 2.8284271247461903) fits in RoundHole with radius 4.0 SquarePegAdapter with width 5 (and notional radius 3.5355339059327378) fits in RoundHole with radius 4.0 SquarePegAdapter with width 6 (and notional radius 4.242640687119285) does not fit in RoundHole with radius 4.0 SquarePegAdapter with width 7 (and notional radius 4.949747468305833) does not fit in RoundHole with radius 4.0
  • 28. …Adapter Pattern SquarePeg.metaClass.getRadius = { Math.sqrt(((delegate.width/2)**2)*2) } (4..7).each { w -> pretty(hole, new SquarePeg(width:w)) } (c) ASERT 2006-2013 Adapter Pattern Do I create a whole new class or just add the method I need on the fly? Consider the Pros and Cons! SquarePeg with width SquarePeg with width SquarePeg with width SquarePeg with width 4 fits in RoundHole with radius 4.0 5 fits in RoundHole with radius 4.0 6 does not fit in RoundHole with radius 4.0 7 does not fit in RoundHole with radius 4.0 Further reading: James Lyndsay, Agile is Groovy, Testing is Square
  • 29. Topics © ASERT 2006-2013 • Intro • Functional Style • Design Patterns Immutability • Laziness • GPars • Word Split (bonus material) • More Info
  • 30. What makes up functional style? • Functions, Closures, Lambda, Blocks as first-class citizens • Higher order functions • Mutable vs Immutable data structures • Recursion • Lazy vs Eager evaluation • Declarative vs Imperative style • Advanced Techniques – Memoization, Trampolines, Composition and Curry • Compile-time safety • Concurrency
  • 31. Immutability options • Built-in def animals = ['cat', 'dog', 'horse'].asImmutable() animals << 'fish' // => java.lang.UnsupportedOperationException • Google Collections import com.google.common.collect.* – Numerous improved immutable collection types List<String> animals = ImmutableList.of("cat", "dog", "horse") animals << 'fish' // => java.lang.UnsupportedOperationException • Groovy run-time metaprogramming def animals = ['cat', 'dog', 'horse'] ArrayList.metaClass.leftShift = { throw new UnsupportedOperationException() } animals << 'fish' // => java.lang.UnsupportedOperationException • Groovy compile-time metaprogramming – @Immutable can help us create such classes – Also gives us @Synchronized and @Lazy
  • 32. @Immutable... • Java Immutable Class – As per Joshua Bloch Effective Java // ... @Override public boolean equals(Object obj) { if (this == obj) return true; if (obj == null) return false; if (getClass() != obj.getClass()) return false; Person other = (Person) obj; if (first == null) { if (other.first != null) return false; } else if (!first.equals(other.first)) return false; if (last == null) { if (other.last != null) return false; } else if (!last.equals(other.last)) return false; return true; } public final class Person { private final String first; private final String last; © ASERT 2006-2013 public String getFirst() { return first; } public String getLast() { return last; } @Override public int hashCode() { final int prime = 31; int result = 1; result = prime * result + ((first == null) ? 0 : first.hashCode()); result = prime * result + ((last == null) ? 0 : last.hashCode()); return result; } public Person(String first, String last) { this.first = first; this.last = last; } // ... @Override public String toString() { return "Person(first:" + first + ", last:" + last + ")"; } }
  • 33. ...@Immutable... • Java Immutable Class boilerplate – As per Joshua Bloch Effective Java // ... @Override public boolean equals(Object obj) { if (this == obj) return true; if (obj == null) return false; if (getClass() != obj.getClass()) return false; Person other = (Person) obj; if (first == null) { if (other.first != null) return false; } else if (!first.equals(other.first)) return false; if (last == null) { if (other.last != null) return false; } else if (!last.equals(other.last)) return false; return true; } public final class Person { private final String first; private final String last; © ASERT 2006-2013 public String getFirst() { return first; } public String getLast() { return last; } @Override public int hashCode() { final int prime = 31; int result = 1; result = prime * result + ((first == null) ? 0 : first.hashCode()); result = prime * result + ((last == null) ? 0 : last.hashCode()); return result; } public Person(String first, String last) { this.first = first; this.last = last; } // ... @Override public String toString() { return "Person(first:" + first + ", last:" + last + ")"; } }
  • 34. ...@Immutable © ASERT 2006-2013 @Immutable class Person { String first, last }
  • 35. Approaches to managing collection storage • Mutable ‘c’ ‘a’ © ASERT 2006-2013 Add ‘t’ ‘c’ ‘a’ ‘t’ • Immutable ‘c’ ‘a’ Add ‘t’ X ‘c’ ‘a’ ‘c’ ‘a’ ‘t’ • Persistent ‘c’ ‘a’ Add ‘t’ ‘c’ ‘a’ ‘t’
  • 36. Persistent collections @Grab('org.pcollections:pcollections:2.1.2') import org.pcollections.* © ASERT 2006-2013 PSet<String> set = HashTreePSet.empty() set += "something" println set println set + "something else" assert set.size() == 1 // => [something] // => [something, something else] • See also – Clojure (next) – TotallyLazy (soon) – Functional Java (tomorrow)
  • 37. Clojure Libraries @Grab('org.clojure:clojure:1.0.0') import clojure.lang.* © ASERT 2006-2013 def ss = StringSeq.create('The quick brown fox') def done = false while (!done) { println ss.first() ss = ss.next() done = !ss } 'The quick brown fox'.each{ println it } <= Plain Groovy equivalent
  • 38. Topics © ASERT 2006-2013 • Intro • Functional Style • Design Patterns • Immutability Laziness • GPars • Word Split (bonus material) • More Info
  • 39. What makes up functional style? • Functions, Closures, Lambda, Blocks as first-class citizens • Higher order functions • Mutable vs Immutable data structures • Recursion • Lazy vs Eager evaluation • Declarative vs Imperative style • Advanced Techniques – Memoization, Trampolines, Composition and Curry • Compile-time safety • Concurrency
  • 40. Show me the code TotallyLazyScript.groovy
  • 41. GPars and TotallyLazy library @GrabResolver('http://repo.bodar.com') @Grab('com.googlecode.totallylazy:totallylazy:808') import static groovyx.gpars.GParsExecutorsPool.withPool import static com.googlecode.totallylazy.Callables.asString import static com.googlecode.totallylazy.Sequences.sequence © ASERT 2006-2013 withPool { pool -> assert ['5', '6'] == sequence(4, 5, 6) .drop(1) .mapConcurrently(asString(), pool) .toList() } withPool { assert ['5', '6'] == [4, 5, 6] .drop(1) .collectParallel{ it.toString() } } <= Plain GPars equivalent
  • 42. What makes up functional style? • Functions, Closures, Lambda, Blocks as first-class citizens • Higher order functions • Mutable vs Immutable data structures • Recursion • Lazy vs Eager evaluation • Declarative vs Imperative style • Advanced Techniques – Memoization, Trampolines, Composition and Curry • Compile-time safety • Concurrency
  • 43. Memoization def plus = { a, b assert plus(1, 2) assert plus(1, 2) assert plus(2, 2) assert plus(2, 2) -> == == == == sleep 1000; a + b }.memoize() 3 // after 1000ms 3 // return immediately 4 // after 1000ms 4 // return immediately © ASERT 2006-2013 // other forms: // at least 10 invocations cached def plusAtLeast = { ... }.memoizeAtLeast(10) // at most 10 invocations cached def plusAtMost = { ... }.memoizeAtMost(10) // between 10 and 20 invocations cached def plusAtLeast = { ... }.memoizeBetween(10, 20)
  • 44. Show me the code Memoize.groovy, Factorial.groovy
  • 45. What makes up functional style? • Functions, Closures, Lambda, Blocks as first-class citizens • Higher order functions • Mutable vs Immutable data structures • Recursion • Lazy vs Eager evaluation • Declarative vs Imperative style • Advanced Techniques – Memoization, Trampolines, Composition and Curry • Compile-time safety • Concurrency
  • 46. Show me the code JScience, SPrintfChecker, GenericStackTest
  • 47. Topics © ASERT 2006-2013 Intro • Functional Style • Design Patterns • Immutability • Laziness GPars • Word Split (bonus material) • More Info
  • 48. What makes up functional style? • Functions, Closures, Lambda, Blocks as first-class citizens • Higher order functions • Mutable vs Immutable data structures • Recursion • Lazy vs Eager evaluation • Declarative vs Imperative style • Advanced Techniques – Memoization, Trampolines, Composition and Curry • Compile-time safety • Concurrency
  • 49. Ralph Johnson: Parallel Programming • Styles of parallel programming – Threads and locks • Nondeterministic, low-level, rumored humans can do this – Asynchronous messages e.g. Actors – no or limited shared memory © ASERT 2006-2013 • Nondeterministic, ok for I/O but be careful with side-effects – Sharing with deterministic restrictions e.g. Fork-join • Hopefully deterministic semantics, not designed for I/O – Data parallelism • Deterministic semantics, easy, efficient, not designed for I/O Each approach has some caveats http://strangeloop2010.com/talk/presentation_file/14485/Johnson-DataParallelism.pdf
  • 50. GPars © ASERT 2006-2013 • http://gpars.codehaus.org/ • Library classes and DSL sugar providing intuitive ways for Groovy developers to handle tasks concurrently. Logical parts: – Data Parallelism features use JSR-166y Parallel Arrays to enable multi-threaded collection processing – Asynchronous functions extend the Java 1.5 built-in support for executor services to enable multi-threaded closure processing – Dataflow Concurrency supports natural shared-memory concurrency model, using single-assignment variables – Actors provide an implementation of Erlang/Scala-like actors including "remote" actors on other machines – Safe Agents provide a non-blocking mt-safe reference to mutable state; inspired by "agents" in Clojure
  • 51. Coordination approaches Fixed coordination (for collections) Actors Source: ReGinA – Groovy in Action, 2nd edition Data Parallelism: Fork/Join Map/Reduce Explicit coordination Safe Agents Delegated coordination Dataflow Implicit coordination
  • 52. GPars: Choosing approaches Streamed Data Parallelism For more details see: http://gpars.codehaus.org/Concepts+compared Data Parallelism Task Parallelism Linear Parallel Collections Irregular Recursive Regular Fork/ Join Shared Data Linear Dataflow operators CSP Recursive Dataflow tasks Actors Asynch fun’s CSP Fork/ Join Immutable Stm, Agents Special collections Synchronization Actors
  • 53. Groovy Sequential Collection def oneStarters = (1..30) .collect { it ** 2 } .findAll { it ==~ '1.*' } assert oneStarters == [1, 16, 100, 121, 144, 169, 196] © ASERT 2006-2013 assert oneStarters.max() == 196 assert oneStarters.sum() == 747
  • 54. GPars Parallel Collections… import static groovyx.gpars.GParsPool.withPool withPool { def oneStarters = (1..30) .collectParallel { it ** 2 } .findAllParallel { it ==~ '1.*' } assert oneStarters == [1, 16, 100, 121, 144, 169, 196] © ASERT 2006-2013 assert oneStarters.maxParallel() == 196 assert oneStarters.sumParallel() == 747 }
  • 55. …GPars Parallel Collections import static groovyx.gpars.GParsPool.withPool withPool { def oneStarters = (1..30) .collectParallel { it ** 2 } .findAllParallel { it ==~ '1.*' } assert oneStarters == [1, 16, 100, 121, 144, 169, 196] © ASERT 2006-2013 assert oneStarters.maxParallel() == 196 assert oneStarters.sumParallel() == 747 } • Suitable when – Each iteration is independent, i.e. not: fact[index] = index * fact[index - 1] – Iteration logic doesn’t use non-thread safe code – Size and indexing of iteration are important
  • 56. Parallel Collection Variations • Apply some Groovy metaprogramming import static groovyx.gpars.GParsPool.withPool © ASERT 2006-2013 withPool { def oneStarters = (1..30).makeConcurrent() .collect { it ** 2 } .findAll { it ==~ '1.*' } .findAll { it ==~ '...' } assert oneStarters == [100, 121, 144, 169, 196] } import groovyx.gpars.ParallelEnhancer def nums = 1..5 ParallelEnhancer.enhanceInstance(nums) assert [1, 4, 9, 16, 25] == nums.collectParallel{ it * it }
  • 57. GPars parallel methods for collections For more details see ReGinA or the GPars documentation Transparent any { ... } collect { ... } count(filter) each { ... } eachWithIndex { ... } every { ... } find { ... } findAll { ... } findAny { ... } fold { ... } fold(seed) { ... } grep(filter) groupBy { ... } max { ... } max() min { ... } min() split { ... } sum Transitive? yes yes yes yes Parallel anyParallel { ... } collectParallel { ... } countParallel(filter) eachParallel { ... } eachWithIndexParallel { ... } everyParallel { ... } findParallel { ... } findAllParallel { ... } findAnyParallel { ... } foldParallel { ... } foldParallel(seed) { ... } grepParallel(filter) groupByParallel { ... } maxParallel { ... } maxParallel() minParallel { ... } minParallel() splitParallel { ... } sumParallel // foldParallel + Transitive means result is automatically transparent; Lazy means fails fast Lazy? yes yes
  • 58. GPars: Map-Reduce import static groovyx.gpars.GParsPool.withPool © ASERT 2006-2013 withPool { def oneStarters = (1..30).parallel .map { it ** 2 } .filter { it ==~ '1.*' } assert oneStarters.collection == [1, 16, 100, 121, 144, 169, 196] // aggregations/reductions assert oneStarters.max() == 196 assert oneStarters.reduce { a, b -> a + b } == 747 assert oneStarters.sum() == 747 }
  • 59. GPars parallel array methods For more details see ReGinA or the GPars documentation Method combine(initValue) { ... } filter { ... } collection groupBy { ... } map { ... } max() max { ... } min() min { ... } reduce { ... } reduce(seed) { ... } size() sort { ... } sum() parallel // on a Collection Return Type Map Parallel array Collection Map Parallel array T T T T T T int Parallel array T Parallel array
  • 60. Parallel Collections vs Map-Reduce Fork Fork Join Join Filter Map Map Filter Reduce Map Reduce Map Map Reduce Map
  • 61. Concurrency challenge… • Suppose we have the following calculation involving several functions: // example adapted from Parallel Programming with .Net def (f1, f2, f3, f4) = [{ sleep 1000; it }] * 3 + [{ x, y -> x + y }] © ASERT 2006-2013 def a = 5 def def def def b c d f = = = = f1(a) f2(a) f3(c) f4(b, d) assert f == 10 • And we want to use our available cores …
  • 62. …Concurrency challenge… • We can analyse the example’s task graph: // example adapted from Parallel Programming with .Net def (f1, f2, f3, f4) = [{ sleep 1000; it }] * 3 + [{ x, y -> x + y }] © ASERT 2006-2013 def a = 5 def def def def b c d f = = = = f1(a) f2(a) f3(c) f4(b, d) a a f2 f1 c b f3 assert f == 10 f4 f d
  • 63. …Concurrency challenge… • Manually using asynchronous functions: // example adapted from Parallel Programming with .Net def (f1, f2, f3, f4) = [{ sleep 1000; it }] * 3 + [{ x, y -> x + y }] import static groovyx.gpars.GParsPool.withPool © ASERT 2006-2013 withPool(2) { def a = 5 def def def def futureB = f1.callAsync(a) c = f2(a) d = f3(c) f = f4(futureB.get(), d) a a f2 f1 c b f3 f4 assert f == 10 } f d
  • 64. …Concurrency challenge • And with GPars Dataflows: def (f1, f2, f3, f4) = [{ sleep 1000; it }] * 3 + [{ x, y -> x + y }] import groovyx.gpars.dataflow.Dataflows import static groovyx.gpars.dataflow.Dataflow.task © ASERT 2006-2013 new Dataflows().with task { a = 5 } task { b = f1(a) task { c = f2(a) task { d = f3(c) task { f = f4(b, assert f == 10 } { a a } } } d) } f2 f1 c b f3 f4 f d
  • 65. …Concurrency challenge • And with GPars Dataflows: def (f1, f2, f3, f4) = [{ sleep 1000; it }] * 3 + [{ x, y -> x + y }] import groovyx.gpars.dataflow.Dataflows import static groovyx.gpars.dataflow.Dataflow.task © ASERT 2006-2013 new Dataflows().with task { f = f4(b, task { d = f3(c) task { c = f2(a) task { b = f1(a) task { a = 5 } assert f == 10 } { d) } } } } a a f2 f1 c b f3 f4 f d
  • 66. GPars: Dataflows... import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task © ASERT 2006-2013 final flow = new DataFlows() task { flow.result = flow.x + flow.y } task { flow.x = 10 } task { flow.y = 5 } assert 15 == flow.result new DataFlows().with { task { result = x * y } task { x = 10 } task { y = 5 } assert 50 == result } 10 5 x y *
  • 67. ...GPars: Dataflows... • Evaluating: result = (a – b) * (a + b) x y © ASERT 2006-2013 import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.a = 10 } task { flow.b = 5 } task { flow.x = flow.a - flow.b } task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } assert flow.result == 75 10 5 a b - + * Question: what happens if I change the order of the task statements here?
  • 68. ...GPars: Dataflows... • Naive attempt for loops Don’t do this! import groovyx.gpars.dataflow.Dataflows import static groovyx.gpars.dataflow.Dataflow.task © ASERT 2006-2013 final flow = new Dataflows() ... [10, 20].each { thisA -> task { flow.a = 10 } [4, 5].each { thisB -> ... task { flow.a = thisA } task { flow.a = 20 } task { flow.b = thisB } task { flow.x = flow.a - flow.b } task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } println flow.result } } // => java.lang.IllegalStateException: A DataflowVariable can only be assigned once. X
  • 69. ...GPars: Dataflows... import groovyx.gpars.dataflow.DataflowStream import static groovyx.gpars.dataflow.Dataflow.* streamA streamB streamX streamY results = = = = = new new new new new DataflowStream() DataflowStream() DataflowStream() DataflowStream() DataflowStream() © ASERT 2006-2013 4 5 4 5 a b - final final final final final 10 10 20 20 + operator(inputs: [streamA, streamB], outputs: [streamX, streamY]) { * a, b -> streamX << a - b; streamY << a + b } operator(inputs: [streamX, streamY], outputs: [results]) { x, y -> results << x * y } [[10, 20], [4, 5]].combinations().each{ thisA, thisB -> task { streamA << thisA } task { streamB << thisB } } 4.times { println results.val } 84 75 384 375
  • 70. ...GPars: Dataflows • Suitable when: – Your algorithms can be expressed as mutuallyindependent logical tasks • Properties: © ASERT 2006-2013 – Inherently safe and robust (no race conditions or livelocks) – Amenable to static analysis – Deadlocks “typically” become repeatable – “Beautiful” (declarative) code import groovyx.gpars.dataflow.Dataflows import static groovyx.gpars.dataflow.Dataflow.task final flow = new Dataflows() task { flow.x = flow.y } task { flow.y = flow.x }
  • 71. …GPars: Actors... import static groovyx.gpars.actor.Actors.* def votes = reactor { it.endsWith('y') ? "You voted for $it" : "Sorry, please try again" } © ASERT 2006-2013 println votes.sendAndWait('Groovy') println votes.sendAndWait('JRuby') println votes.sendAndWait('Go') def languages = ['Groovy', 'Dart', 'C++'] def booth = actor { languages.each{ votes << it } loop { languages.size().times { react { println it } } stop() } } booth.join(); votes.stop(); votes.join() You voted for Groovy You voted for JRuby Sorry, please try again You voted for Groovy Sorry, please try again Sorry, please try again
  • 72. Software Transactional Memory… @Grab('org.multiverse:multiverse-beta:0.7-RC-1') import org.multiverse.api.references.LongRef import static groovyx.gpars.stm.GParsStm.atomic import static org.multiverse.api.StmUtils.newLongRef class Account { private final LongRef balance © ASERT 2006-2013 Account(long initial) { balance = newLongRef(initial) } void setBalance(long newBalance) { if (newBalance < 0) throw new RuntimeException("not enough money") balance.set newBalance } long getBalance() { balance.get() } } // ...
  • 73. …Software Transactional Memory // ... def from = new Account(20) def to = new Account(20) def amount = 10 © ASERT 2006-2013 def watcher = Thread.start { 15.times { atomic { println "from: ${from.balance}, to: ${to.balance}" } sleep 100 } } sleep 150 try { atomic { from.balance -= amount to.balance += amount sleep 500 } println 'transfer success' } catch(all) { println all.message } atomic { println "from: $from.balance, to: $to.balance" } watcher.join()
  • 74. Topics © ASERT 2006-2013 • Intro • Functional Style • Design Patterns • Immutability • Laziness • GPars Word Split (bonus material) • More Info
  • 75. Word Split with Fortress Guy Steele’s StrangeLoop keynote (from slide 52 onwards for several slides): http://strangeloop2010.com/talk/presentation_file/14299/GuySteele-parallel.pdf © ASERT 2006-2013
  • 76. Word Split… © ASERT 2006-2013 def swords = { s -> def result = [] def word = '' s.each{ ch -> if (ch == ' ') { if (word) result += word word = '' } else word += ch } if (word) result += word result } assert swords("This is a sample") == ['This', 'is', 'a', 'sample'] assert swords("Here is a sesquipedalian string of words") == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words']
  • 77. Word Split… © ASERT 2006-2013 def swords = { s -> def result = [] def word = '' s.each{ ch -> if (ch == ' ') { if (word) result += word word = '' } else word += ch } if (word) result += word result }
  • 78. Word Split… © ASERT 2006-2013 def swords = { s -> def result = [] def word = '' s.each{ ch -> if (ch == ' ') { if (word) result += word word = '' } else word += ch } if (word) result += word result }
  • 79. …Word Split… © ASERT 2006-2013
  • 80. …Word Split… © ASERT 2006-2013
  • 81. …Word Split… © ASERT 2006-2013 Segment(left1, m1, right1) Segment(left2, m2, right2) Segment(left1, m1 + [ ? ] + m2, right2)
  • 82. …Word Split… class Util { static maybeWord(s) { s ? [s] : [] } } import static Util.* © ASERT 2006-2013 @Immutable class Chunk { String s public static final ZERO = new Chunk('') def plus(Chunk other) { new Chunk(s + other.s) } def plus(Segment other) { new Segment(s + other.l, other.m, other.r) } def flatten() { maybeWord(s) } } @Immutable class Segment { String l; List m; String r public static final ZERO = new Segment('', [], '') def plus(Chunk other) { new Segment(l, m, r + other.s) } def plus(Segment other) { new Segment(l, m + maybeWord(r + other.l) + other.m, other.r) } def flatten() { maybeWord(l) + m + maybeWord(r) } }
  • 83. …Word Split… def processChar(ch) { ch == ' ' ? new Segment('', [], '') : new Chunk(ch) } © ASERT 2006-2013 def swords(s) { s.inject(Chunk.ZERO) { result, ch -> result + processChar(ch) } } assert swords("Here is a sesquipedalian string of words").flatten() == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words']
  • 84. …Word Split… © ASERT 2006-2013
  • 85. …Word Split… © ASERT 2006-2013
  • 86. …Word Split… © ASERT 2006-2013 THREADS = 4 def pwords(s) { int n = (s.size() + THREADS - 1) / THREADS def map = new ConcurrentHashMap() (0..<THREADS).collect { i -> Thread.start { def (min, max) = [ [s.size(), i * n].min(), [s.size(), (i + 1) * n].min() ] map[i] = swords(s[min..<max]) } }*.join() (0..<THREADS).collect { i -> map[i] }.sum().flatten() }
  • 87. …Word Split… import static groovyx.gpars.GParsPool.withPool THRESHHOLD = 10 def partition(piece) { piece.size() <= THRESHHOLD ? piece : [piece[0..<THRESHHOLD]] + partition(piece.substring(THRESHHOLD)) } © ASERT 2006-2013 def pwords = { input -> withPool(THREADS) { partition(input).parallel.map(swords).reduce{ a, b -> a + b }.flatten() } }
  • 88. …Guy Steele example in Groovy… DataFlow version: partially hard-coded to 4 partitions for easier reading © ASERT 2006-2013 def words = { s -> int n = (s.size() + THREADS - 1) / THREADS def min = (0..<THREADS).collectEntries{ [it, [s.size(),it*n].min()] } def max = (0..<THREADS).collectEntries{ [it, [s.size(),(it+1)*n].min()] } def result = new DataFlows().with { task { a = swords(s[min[0]..<max[0]]) } task { b = swords(s[min[1]..<max[1]]) } task { c = swords(s[min[2]..<max[2]]) } task { d = swords(s[min[3]..<max[3]]) } task { sum1 = a + b } task { sum2 = c + d } task { sum = sum1 + sum2 } println 'Tasks ahoy!' sum } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } }
  • 89. …Guy Steele example in Groovy… Fork/Join version GRANULARITY_THRESHHOLD = 10 THREADS = 4 © ASERT 2006-2013 println GParsPool.withPool(THREADS) { def result = runForkJoin(0, input.size(), input){ first, last, s -> def size = last - first if (size <= GRANULARITY_THRESHHOLD) { swords(s[first..<last]) } else { // divide and conquer def mid = first + ((last - first) >> 1) forkOffChild(first, mid, s) forkOffChild(mid, last, s) childrenResults.sum() } } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } }
  • 90. …Guy Steele example in Groovy Just leveraging the algorithm’s parallel nature © ASERT 2006-2013 println GParsPool.withPool(THREADS) { def ans = input.collectParallel{ processChar(it) }.sum() switch(ans) { case Chunk: return maybeWord(ans.s) case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) } } }
  • 91. Topics © ASERT 2006-2013 • Intro • Functional Style • Design Patterns • Immutability • Laziness • GPars • Word Split (bonus material) More Info
  • 92. More Information: Groovy in Action