Groovy On The Trading Desk       Jonathan Felch      Quantitative Strategies       Volant Trading LLC
Overview●   Motivations for Polyglot Software with Groovy●   Introducing Groovy: Scripting on the VM●   Skills To Take Wit...
Motivations●   High Speed OO Tools + Dynamic / Functional Tools●   Re-use existing assets●   Lazy Evaluation●   Externaliz...
Why Groovy ?●   Meet Requirements       –   Lazy & Dynamic Evaluation,       –   Concise Synatx, Rapid Development, Iterat...
Scripting on the VMEvolution of the VM: Towards Dynamic Programming     –   Features Popular in Most Used Frameworks Explo...
Introducing Groovy●   The Blue Pill: Syntactically compatible with Java       –   Easy Adoption for Java Programmers      ...
Job Trends / Adoption Hints for new JVM Languages
Groovy Use Cases:●   Super Glue: I/O and Integration●   Code as Data: Configuration, Process Scripting,    Business Rules●...
Learning GroovyIm prov in g o n O b je c t    def abstract class OptionPxModel {       def vol, carry, strike, expiry, pc ...
Learning Groovy: CollectionsL is tsdef list = [5, 6, 7, 8]assert list.size == list.size()assert list[2] == list.getAt(2) =...
Groovy One (or 2, 3, 4) Liners AppsExecuting A Process:”processMyFile ./output.txt $Destination”.execute()Adding A She-Ban...
Learning Groovy: OperatorsA s O pe ra to r    "3.14159" as Double    def readable = { it.put("12 34".reverse()); 5 } as Re...
One Slide Apps: RSS Readerdef url =http://www.groovyblogs.org/feed/rssdef items = new XmlParser().parse(url).channel.itemd...
One Slide Apps: Monte Carlodef px = 100, rate = 0.05, vol = 0.15, time = 1.0def strikes = [ 8..12 ].collect { it * 10 }def...
One Slide Apps: RESTful Web Serverimport com.sun.net.httpserver.*;def isPrime = { Long N ->    if (N < 2) return false;   ...
Unlearning Java: Taking the Red Pill●   Meta-Programming●   Design Patterns●   Compile Time Meta-Programming●   Power of L...
Dynamic Language:                Evaluation or Syntax“In Lisp, you don’t just write your program down towardthe language, ...
Meta-Programming●   Understanding The Type System       –   Groovy has an optionally dynamic strongly typed type system   ...
GORM as a DSL// These Class Definitions Generate SQL and Database Tablesclass Book {   String title   Date releaseDate   A...
GORM as a DSLclass GORM {   def dynamicMethods = [ ]    def methodMissing(String name, args) {        def method = dynamic...
DSL: Tool Kit●   Meta-Methods       –   String.metaClass.swapCase= {               delegate.collect{ c →                  ...
Function Eval / Visualizationnew SwingBuilder().frame(title:Quant Modeller, size : [1100,600], // ..parameters here not sh...
New AST Transforms• @Singleton — okay, not really a pattern :-)• @Immutable, @Lazy, @Delegate• @Newify• More DSL and Meta-...
The Evil Java Singleton  public class Evil {     public static final Evil instance = new Evil();      private Evil() {}   ...
DSLs → They Start Out As APIs// closure maddness// Before refactoringdef phrase = "The quick brown fox jumps over the lazy...
DSL: APIs That Get Refactored// Refactored helper closuresdef   lowercaseLetters= phrase.toLowerCase()def   vowels = { it ...
Putting All Together I●   Drop Parenthesis●   Named Parameters●   Objects Graphs → Tree of Nouns and Verbs●   Human Readab...
Putting All Together IIorder to buy 200.shares of GOOG {   limitPrice 500   allOrNone false   at the value of { qty * unit...
Config SlurperLog4j {    appender.stdout = "org.apache.log4j.ConsoleAppender"    appender."stdout.layout"="org.apache.log4...
Performance●   Dynamic Typing is Expensive (But Worth It)       –   Hot Spot Opportunity●   Dynamic Method Invocation is e...
Performance II●   Getting Too Groovy ???       –   Abusing .each { tuple →} // for loops versus anon closures●   Simplicit...
ActorsScaling the VM: Immutable Data Message + Servicesclass Player extends AbstractPooledActor {   String name   def rand...
Are Java & C# Becoming Groovy ?●   Google Guava (formerly Collections)●   Lots of new Libraries for FP, Enhanced    Collec...
A few bad months for Java●    JavaTM By Oracle ,●   The Java Community Process, Doug Lea●   The Java Community Process, Ap...
Where is Goovy Going ?●   Some Important JDK Improvements will be in    JDK 7 (as planned): invokeDynamic●   More explicit...
Where is Goovy Going ?●   Closures with Memory      def cl = {a, b →         sleep(3000) // create a noticeable pause     ...
Groovy Gotchas!●   Subclassing Map, Unexpected Results            ●   Map.class            ●   the dot operator,          ...
Best Practices●   Explicitly Signatures and Types Where You Can              ●   Better GroovyDoc documentation           ...
Acknowledgments & Suggestions●   Derik Kognig, Author Groovy In Action●   Peter Bell, Systems Forge●   Paul King, Asset Au...
Contact Info●   jonathan.felch@gmail.com●   jfelch@volanttrading.com●   www.twitter.com/Jonathan.Felch       –   Rare but ...
Upcoming SlideShare
Loading in …5
×

Groovy On Trading Desk (2010)

2,414 views

Published on

Introductionary talk and tutorial on Groovy, discussion of its success and popularity, issues with the future of Java with examples

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,414
On SlideShare
0
From Embeds
0
Number of Embeds
20
Actions
Shares
0
Downloads
22
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Groovy On Trading Desk (2010)

  1. 1. Groovy On The Trading Desk Jonathan Felch Quantitative Strategies Volant Trading LLC
  2. 2. Overview● Motivations for Polyglot Software with Groovy● Introducing Groovy: Scripting on the VM● Skills To Take With You: Learning Groovy● Groovy DSLs: Creating the syntax you need● Unlearning Java, C++, C#● Groovy Performance and Concurrency● Relearning Java: The Future of the VM● Questions
  3. 3. Motivations● High Speed OO Tools + Dynamic / Functional Tools● Re-use existing assets● Lazy Evaluation● Externalize Logic and Business Rules● Dynamic Evaluation
  4. 4. Why Groovy ?● Meet Requirements – Lazy & Dynamic Evaluation, – Concise Synatx, Rapid Development, Iterative Prototyping – Supported But Did Not Require Functional Style, Support for Closures / Lamda Expressions – Tool Support, Attractive Adoption Pattern – Human Readable / Machine Executable Data Interchange – Natural Data Integration Platform● Strong and Focused Community● Familiar Syntax / Ease of Adoption● Expensible Syntax : DSLs
  5. 5. Scripting on the VMEvolution of the VM: Towards Dynamic Programming – Features Popular in Most Used Frameworks Exploited Specific Benefits and Uses of Dynamic Programming for a general-purpose or cross-cutting concern ● Hibernate, Spring Framework, Jboss AOP, AspectJ – Introduction of Dynamic Programming for Testing, Configuration and Business Rules ● BeanShell, Jython, JRuby – Next Generation Scripting: Groovy, Scala, Clojure ● Specific to VM, not a straight port of another language ● Emphasis on Functional Programming ● Tight Integration of FP / DP with a high performance OO Language
  6. 6. Introducing Groovy● The Blue Pill: Syntactically compatible with Java – Easy Adoption for Java Programmers – Virtually seamless / virtually flawless integration with Java – Natural Support for Multiple Styles – Compatibility allows for remarkable performance tuning● The Red Pill: A Language On Its Own – Type-system: Dynamically / Optionally Strong Typing – Closures: Lambda Expressions / Functions as Objects – GDK: Object and Collection Enhancements – Dynam Eval: Code is Data, Data is Code – Dynam Syntax: Write your own syntax – Enhancements: New operators, data literals, new basic types
  7. 7. Job Trends / Adoption Hints for new JVM Languages
  8. 8. Groovy Use Cases:● Super Glue: I/O and Integration● Code as Data: Configuration, Process Scripting, Business Rules● Live Analysis: Ad-Hoc Queries & Expressions● Language Building: New Syntax and DSLs● Half-Baked Ideas● House Elf: Cleanup and Maintenance
  9. 9. Learning GroovyIm prov in g o n O b je c t def abstract class OptionPxModel { def vol, carry, strike, expiry, pc def abstract px (def spot) } OptionPxModel model = new BlackScholesModel(strike : 100, pc : C ) model.vol = 0.15 model.expiry = Date.parse(MM/dd/yyyy, 11/18/2010) println “The Strike 100 Nov 2010 Call is worth ${model.px(102.0)}”C lo s u re s : def r = new Random() def normRand = { r.nextGaussian() } def stockPrice = { px, vol, rate, time → px * exp((rate – vol**2) * time + normRand() * vol * time) }
  10. 10. Learning Groovy: CollectionsL is tsdef list = [5, 6, 7, 8]assert list.size == list.size()assert list[2] == list.getAt(2) == list.get(2) == 7assert [1,2,3,4,5][-1] == 5 && [1,2,3,4,5][-2] == 4assert list.find { it > 6 } == 7 && list.findAll { it > 6 } == [7, 8]list.each { println “$it ${it * 2}” }[a, b, c].eachWithIndex{ it, i -> println "$i: $it" }Mu ltiple A s s ig n m e n tdef (int a, int b) = [1, 2](a, b) = [b, a]Ra ng e sassert (a..d) == [a,b,c,d]assert [ 1, *3..5, 7, *9..<12 ] == [1,3,4,5,7,9,10,11]def oneWeekForward = (new Date() .. (new Date() + 7))Ma ps[ id: FX-17, name: Turnip, 99: 123, (-97): 987, "tails": true].each { key, value → println “$key => $value” }
  11. 11. Groovy One (or 2, 3, 4) Liners AppsExecuting A Process:”processMyFile ./output.txt $Destination”.execute()Adding A She-Bang To All Groovy Files:groovy -i .bak -pe "if (count == 1) println #!/usr/bin/groovy" *.groovyPrint the 1st column and 2nd-to-last column from filegroovy -a -pe "split[0] split[-2]" some.filename // or set of filesCreate and Shuffle a Deck of Cardsdef deck = [ [ *2..10, *(J Q K A.split()) ], C D H S.split() ] .combinations().collect { it.join() };Collections.shuffle(deck)Generate 21-Character OSI Symbol for Optiondef osi = { symbol, year, month, day, cp, strike -> sprintf(%1$s%2$2d%3$2d%4$2d%5$s%6$08.3f, [ symbol.padLeft(6), year as Integer, month as Integer, day as Integer, cp as String, strike as Double ] as Object[] )}
  12. 12. Learning Groovy: OperatorsA s O pe ra to r "3.14159" as Double def readable = { it.put("12 34".reverse()); 5 } as ReadableA s O pe ra to r O n C lo s u re s + Ma ps → O b je c ts ? def map; map = [ i: 10, hasNext: { map.i > 0 }, next: { map.i-- } ] as Iterator while ( map.hasNext() ) println map.next()E lv is , S a fe Re fe re n c e , S pre a d & S pa c e s h ip def lang = data?.favoriteLanguage ?: Groovy assert [cat, elephant]*.size() == [3, 8] assert (3 <=> 4) == -1 && (3 <=> 3) == 0 && (5 <=> 3) == 1Re g e x def list = ["foo", "bar", "moo"] def pattern = ~/foo/ assert "foo" =~ list.find { it =~ pattern } assert ["foo", "moo"] == list.findAll { it ==~ /.*oo/ }
  13. 13. One Slide Apps: RSS Readerdef url =http://www.groovyblogs.org/feed/rssdef items = new XmlParser().parse(url).channel.itemdef cols = pubDate title description.tokenize()groovy.swing.SwingBuilder.build { frame(id:f, title: Groovy RSS, visible :true) { scrollPane { table { tableModel(list: items) { cols.each { col → closureColumn header: col, read: { it[col].text() } } } } } } f.pack()}.show() Credit: Dierk König, GR8 Conference 2009, Copenhagen
  14. 14. One Slide Apps: Monte Carlodef px = 100, rate = 0.05, vol = 0.15, time = 1.0def strikes = [ 8..12 ].collect { it * 10 }def w = RandomNumbers.getNormDist(1000,1000)def S = px * exp((rate-vol ** 2)* time + sqrt(time)* vol * w)strikes.each { K → def optionValue = max(0, S – K) def df = exp(-rate * time) println “${K} : ${df * optionValue as Number}”})
  15. 15. One Slide Apps: RESTful Web Serverimport com.sun.net.httpserver.*;def isPrime = { Long N -> if (N < 2) return false; for (long i = 2; i*i < N; i++) if (N % i == 0) return false return true}def pws = { HttpExchange exch -> exch.sendResponseHeaders(200,0); def pe = exch.requestURI.path.split(/) if (pe.size() == 3 && pe[1] == prime) { exch.responseBody.write("${isPrime(pe[2] as Long)}".bytes) } else exch.responseBody.write("${exch.requestURI.path}".bytes) exch.responseBody.close();} as HttpHandlerdef run = { handler, minutes -> def server = HttpServer.create(new InetSocketAddress(8888),0) server.createContext(/, handler) server.start()}run(pws,1)
  16. 16. Unlearning Java: Taking the Red Pill● Meta-Programming● Design Patterns● Compile Time Meta-Programming● Power of Lazy Evaluation
  17. 17. Dynamic Language: Evaluation or Syntax“In Lisp, you don’t just write your program down towardthe language, you also build the language up toward yourprogram”- Paul Graham“When [Smalltalk] is used to describe an applicationsystem, the developer extends Smalltalk, creating adomain-specific language by adding a new vocabulary oflanguage elements ...”- Adele Goldberg DSLs
  18. 18. Meta-Programming● Understanding The Type System – Groovy has an optionally dynamic strongly typed type system – What does that mean ?● Dynamic Method Invocation and GroovyObject – invokeMethod, get/setProperty – What is a MetaClass ?● Meta Object Protocol – def methodMissing(String name, args) { } – def propertyMissing(String name) – def propertyMissing(String name, value)● Overloading Operators
  19. 19. GORM as a DSL// These Class Definitions Generate SQL and Database Tablesclass Book { String title Date releaseDate Author author}class Author { String name}// These Operation Generate SQL and Database Queriesdef book = Book.findByTitle("The Stand")book = Book.findByTitleLike("Harry Pot%")book = Book.findByReleaseDateBetween( firstDate, secondDate )book = Book.findByReleaseDateGreaterThan( someDate )book = Book.findByTitleLikeOrReleaseDateLessThan( "%Something%", someDate )
  20. 20. GORM as a DSLclass GORM { def dynamicMethods = [ ] def methodMissing(String name, args) { def method = dynamicMethods.find { it.match(name) } if(method) { GORM.metaClass."$name" = { Object[] varArgs → method.invoke(delegate, name, varArgs) } return method.invoke(delegate,name, args) } // GORM actually writes or creates a new method here else throw new MissingMethodException( name, delegate, args ) }}
  21. 21. DSL: Tool Kit● Meta-Methods – String.metaClass.swapCase= { delegate.collect{ c → c in A..Z? c.toLowerCase() :c.toUpperCase() }.join()} assert "Foo".swapCase()== "fOO" Number.metaClass.multiply= { Grid g → g.multiple(delegate) }● MOP – propertyMissing, methodMissing, respondsTo, hasProperty, listMethods, selectMethod● Categories, ObjectBuilder, MetaBuilder
  22. 22. Function Eval / Visualizationnew SwingBuilder().frame(title:Quant Modeller, size : [1100,600], // ..parameters here not shown { tableLayout() { tr { td { widget(gui) } td { widget(editor, border : titledBorder(Model Script, preferredSize :[900,600])) } td { panel(border : titledBorder(Model Calibration), preferredSize : [200,600]) { tableLayout(cellpadding :15) { tr { td { label(text : Params:) } td { button(text : Add, * Edited Out * } tr { td(colspan : 2, align : center) { button(text : Execute, actionPerformed : { shell.setVariable f, [:] shell.eval1uate(editor.text) def f = shell.getVariable(f) def obj = [ f : f] as Model3d gui.model = obj gui.redraw() } ) } } }
  23. 23. New AST Transforms• @Singleton — okay, not really a pattern :-)• @Immutable, @Lazy, @Delegate• @Newify• More DSL and Meta-Magic! @Category and @Mixin• @PackageScope• Swing’s @Bindable and @Vetoable• Grape’s @Grab
  24. 24. The Evil Java Singleton public class Evil { public static final Evil instance = new Evil(); private Evil() {} static Evil getInstance() { return instance; } } • In Groovy now: –@Singleton class Evil {} • A lazy version also: –@Singleton(lazy = true) class Evil {}Credit: Guillaume Laforge,“To Infinity and Beyond” & GDK 1.7 Release notes
  25. 25. DSLs → They Start Out As APIs// closure maddness// Before refactoringdef phrase = "The quick brown fox jumps over the lazy dog"def result = phrase.toLowerCase().toList(). findAll{ it in "aeiou".toList() }. // like WHERE … groupBy{ it }. // like GROUP BY … findAll{ it.value.size() > 1 }. // like HAVING … sort{ it.key}.reverse(). // like ORDER BY … collect{ "$it.key:${it.value.size()}" }.join(", ")println resultCredit: Paul King, “Groovy DSL”
  26. 26. DSL: APIs That Get Refactored// Refactored helper closuresdef lowercaseLetters= phrase.toLowerCase()def vowels = { it in "aeiou".toList() }def occursMoreThanOnce= { it.value.size() > 1 }def byReverseKey= { a, b -> b.key<=> a.key}def self = { it }def entriesAsPrettyString= { "$it.key:${it.value.size()}" }def withCommaDelimiter= ", "// Refactored main closureprintln lowercaseLetters. findAll(vowels). groupBy(self). findAll(occursMoreThanOnce). sort(byReverseKey). collect(entriesAsPrettyString). join(withCommaDelimiter) Credit: Paul King, “Groovy DSL”
  27. 27. Putting All Together I● Drop Parenthesis● Named Parameters● Objects Graphs → Tree of Nouns and Verbs● Human Readable / Machine Executable● Specific Syntax
  28. 28. Putting All Together IIorder to buy 200.shares of GOOG { limitPrice 500 allOrNone false at the value of { qty * unitPrice - 100 }}take 2.pills of chloroquinine after 6.hoursdef "length of Spocks & his friends names"() { expect: name.size() == length where: name | length "Spock" | 5 "Kirk" | 4 "Scotty" | 6} Credit: Paul King, “Groovy DSL”
  29. 29. Config SlurperLog4j { appender.stdout = "org.apache.log4j.ConsoleAppender" appender."stdout.layout"="org.apache.log4j.PatternLayout" rootLogger="error,stdout" logger { org.springframework="info,stdout" } additivity { org.springframework=false }}databaseURL : //development { databaseURL : //}testing { databaseURL : //}
  30. 30. Performance● Dynamic Typing is Expensive (But Worth It) – Hot Spot Opportunity● Dynamic Method Invocation is expensive (But Getting Better) – Da Vinci Project and invokeDynamic bytecode – Hot Spot Opportunity● BigDecimal & Collections → Insanely Expensive For Serious Math – People need to better learn to use IEEE floating point types –
  31. 31. Performance II● Getting Too Groovy ??? – Abusing .each { tuple →} // for loops versus anon closures● Simplicity --> faster – Groovy reduces your dependency on Java Enterprise – Gpath vs Xpath (Gpath almost always wins) – Use Visual VM: Groovy Memory Use Can Be Mysterious● Too Many Collections ? – Is The Idea to Cache The World As A Map● Use Groovy-Aware Java for Faster Code – People Need to learn IEEE Floating Point Types
  32. 32. ActorsScaling the VM: Immutable Data Message + Servicesclass Player extends AbstractPooledActor { String name def random = new Random() void act() { loop { react { // player replies with a random move reply Move.values()[ random.nextInt(Move.values().length) ] } } }}
  33. 33. Are Java & C# Becoming Groovy ?● Google Guava (formerly Collections)● Lots of new Libraries for FP, Enhanced Collections, Tuples, Monads, etc...● Doug Leas Fork-Join + Predicate Libraries● Project Lambda : Closures for Java (JDK 8 ?)● Da Vinci Project : invokeDynamic (JDK 7 ?)● Is Gpath LINQ for Java ? (Now ?)● Universal Low Level VM● Will Groovy or Groovy++ Continue ?
  34. 34. A few bad months for Java● JavaTM By Oracle ,● The Java Community Process, Doug Lea● The Java Community Process, Apache● JavaOne, The Conference● Oracle vs Google● Oracle vs Apache● IBM and Apache● OpenJDK
  35. 35. Where is Goovy Going ?● Some Important JDK Improvements will be in JDK 7 (as planned): invokeDynamic● More explicitly FP approach to concurrency and state – AST MP / GoF Patterns – @Immutable / @Singleton ( lazy : true) – GPars Actor-like syntax● Groovy Server for Fast Scripting – Difficult on a cluster
  36. 36. Where is Goovy Going ?● Closures with Memory def cl = {a, b → sleep(3000) // create a noticeable pause a + b } def mem = cl.memoize() // memorize() variants are // memorizeAtMost(n), memorizeAtLeast(n), // memorizeBetween(m, n)● Groovy++ – Not a fully compatible fork – An interesting argument – But no.
  37. 37. Groovy Gotchas!● Subclassing Map, Unexpected Results ● Map.class ● the dot operator, getAt(...), getXXX(), [ name ] literals● Private isnt Private ● Inconvenient when you really want to hide implementation data● Unexpected Weird Effects – Combining Static and Dynamic Numeric Types Across Java Code and Groovy Code ● Be careful and deliberate with typing – Method selection surprises ● identical static closures and static functions do not respond to the same way Invoked dynamically
  38. 38. Best Practices● Explicitly Signatures and Types Where You Can ● Better GroovyDoc documentation ● Faster method selection ● More shareable (with Java / Scala / Ioke / etc..) Libraries● Write Groovy Aware (GroovySupport) Java ● This is where to build your high-speed / high-performance code● Use ConfigSlurper to externalize environment / context data● Use CliBuilder and Groovy Server if it fits your production model● Add Performance Tests To Your Suite / Learn to use Visual VM – Testing is more important with dynamic and functional programming – Small tests – small in terms of code coverage – on big data sets ● Best place to find surprising performance enhancements – Automating Just Got Lucky Into Your Development Cycle
  39. 39. Acknowledgments & Suggestions● Derik Kognig, Author Groovy In Action● Peter Bell, Systems Forge● Paul King, Asset Australia● Guillaume Laforge, Groovy Project Manager● PLEAC, Project Euler, Groovy Zone
  40. 40. Contact Info● jonathan.felch@gmail.com● jfelch@volanttrading.com● www.twitter.com/Jonathan.Felch – Rare but regular contributions via Blog, GitHub, SlideShare ● 3 – 5 contributions a year, typically unfinished half-baked ideas ● Really targeted at a dozen or so people that I have worked with in the past or occasionally collaborate

×