Metaprogramming Techniques In Groovy And Grails

23,522 views
23,343 views

Published on

Published in: Technology

Metaprogramming Techniques In Groovy And Grails

  1. 1. Metaprogramming techniques in Groovy and Grails Numan Salati numan.salati@gmail.com NY Groovy/Grails Meetup,
  2. 2. What makes a language dynamic? • Dynamic type system • Mutable types • Flexible method dispatch • Evaluate code at runtime (access to the interpreter) Key Idea: late binding !
  3. 3. Dynamic features in Java • dynamic class loading • dynamic binding – subclass Polymorphism • runtime annotations • dynamic proxies and reflection API – mostly read only – dynamic implementation of interfaces
  4. 4. Dynamic Groovy • Groovy has all this and much more – Intercept methods/properties – Create new methods/properties/constructors – Create classes at runtime – Runtime mixins (mutable types) – Evaluate any valid code string
  5. 5. method invocation example obj.method(arg1, arg2, arg3) • In Java – single dispatch – invokedynamic bytecode instruction • In Groovy – multiple dispatch – much more complicated logic but very flexible
  6. 6. method invocation example class Foo { def print (Object o) { println “println objectquot; } def print (String s) { println “println stringquot; } } Object arg = quot;stringquot; new Foo().print(arg) What gets called in Java vs. Groovy ?
  7. 7. Metaprogramming • Wikipedia definitions: – Programs that write or manipulate other programs – Expose internals of runtime engine to programming code through API” – Dynamic execution of string expression • Meta object protocol: Make program semantics – Explicit – Extensible How much of runtime and compile time structures are exposed?
  8. 8. Groovy MOP Excellent support for metaprogramming Compile time • Hook into the Groovy AST during compilation Runtime • Hook into method dispatching • Dynamically create methods/properties • Mutable types • Execution of code strings
  9. 9. Example class Person { def name def sleep() { println quot;sleepingquot;} } >> groovyc Person.groovy >> javap –public Person Compiled from quot;Person.groovyquot; public class Test extends java.lang.Object implements groovy.lang.GroovyObject { ….. public groovy.lang.MetaClass getMetaClass(); public void setMetaClass(groovy.lang.MetaClass); public java.lang.Object invokeMethod(java.lang.String, java.lang.Object); GroovyObject public java.lang.Object getProperty(java.lang.String); public void setProperty(java.lang.String, java.lang.Object); …. }
  10. 10. GroovyObject • All Groovy classes implement this interface • Open the class file in jad decompiler: public Object invokeMethod(String s, Object obj) { return getMetaClass().invokeMethod(this, s, obj); Default implementation } delegates to metaClass public Object getProperty(String s) { return getMetaClass().getProperty(this, s); } • Compiler assigns a metaClass to every POJO and POGO
  11. 11. GroovyObject • These methods are the hooks into method dispatch and property access/assignment • Overriding getProperty() and setProperty we can dynamically add properties and methods – This is exactly what Expando does • Dynamically create classes • Add methods by creating properties that are closures
  12. 12. Expando in Groovy class SimpleExpando { def propertyMap = [:] def getProperty(String name) { propertyMap[name]?: null } void setProperty(String name, Object value) { propertyMap[name] = value; } def invokeMethod(String name, Object args) { try { metaClass.invokeMethod(name, args); } catch (GroovyRuntimeException e) { def value = propertyMap[name]; if (value instanceof Closure) { Why set delegate value.setDelegate(this) value.call(args); before invoking? } else { throw e } } } } s = new SimpleExpando() s.add = {x, y -> x + y} println s.add(19,1)
  13. 13. InvokeMethod • Overriding invokeMethod in the class – intercepts all non existing method calls • What if we want to intercept all method calls? – Implement GroovyInterceptable marker interface – Override invokeMethod(String name, args) – Careful about stack overflow! • use metaClass.invokeMethod inside invokeMethod • Only non existing methods? – implement methodMissing(String name, args) – higher precedence than invokeMethod if both present
  14. 14. MetaClass • invokeMethod and methodMissing can be implemented in the class or on the metaClass • Metaclass defines the dynamic behavior of the object • Query runtime structure of the class/object – respondsTo – hasProperty – getMethods vs. getMetaMethods – getProperties vs. etMetaProperties
  15. 15. MetaClass • Define new methods and constructors on class using ExpandoMetaClass Person.metaClass.play = { println quot;playquot;} Person.metaClass.eat = { pritnln quot;eatquot; } Person.metaClass.code = { println quot;codequot;} Person.metaClass.static.read = { println quot;readingquot; } Person.metaClass.constructor = {name -> new Person(quot;Sir: quot; + name) } Heap Overflow! - use BeanUtils.instantiateClass to instantiate outside of groovy or using EMC DSL Person.metaClass { play { println quot;playquot;} eat { pritnln quot;eatquot; } code { println quot;codequot;} 'static' { read { println quot;readingquot; } } constructor { name -> BeanUtils.instantiateClass(Person, quot;Sir: quot; + name) } }
  16. 16. EMC • Injection works for both POJOs and POGOs – Integer.randomNum = { … } – String.metaClass = <Roll your own super cool metaClass> • Add to instance only? p1 = new Person() p2 = new Person() p2.metaClass.party = { println quot;partyingquot;} p2.party() p1.party() MissingMethodException • Works for POJOs too
  17. 17. EMC • new methods are reflected in the subclass hierarchy • Adding methods to interfaces? – set enableGlobally on EMC to affect all implementing classes
  18. 18. Summary of method dispatch so far.. • If a method is defined in the metaClass invoke that • Or else look for hooks in the class or metaClass: – invokeMethod – methodMissing – getProperty – setProperty
  19. 19. Method dispatch flow diagram
  20. 20. Categories • Injection of methods within a scope import org.codehaus.groovy.runtime.TimeCategory use(TimeCategory) { println 2.days.from.now println 3.years.from.now println 10.minutes.from.now println 3.weeks.from.now } • Injects getDays() and getYears() method defined in TimeCategory into the meta class of Integer objects in the “use” block and for the current thread
  21. 21. Categories public class TimeCategory { .... • public static DatumDependentDuration getMonths(final Integer self) { • return new DatumDependentDuration(0, self.intValue(), 0, 0, 0, 0, 0); • } • public static DatumDependentDuration getYears(final Integer self) { • return new DatumDependentDuration(self.intValue(), 0, 0, 0, 0, 0, 0); • } • public static Duration getWeeks(final Integer self) { • return new Duration(self.intValue() * 7, 0, 0, 0, 0); • } • All methods are static • public static TimeDuration getHours(final Integer self) { • return new TimeDuration(0, self.intValue(), 0, 0, 0); • } • First argument is the class .... getting injected }
  22. 22. Categories • Can nest categories – in case of method clash, last one takes precedence • Can use multiple categories in the “use” clause – same precedence rule as above • Other built in categories – DomCategory – SerlvetCategory
  23. 23. Categories • How it work internally: 1. Creates new scope 2. Adds static methods from category class into thread local stack 3. Call closure 4. Remove methods from stack – check out “use” method in “GroovyCategoryStack.java” • Slower than metaClass injection – scanning static methods – cleanup
  24. 24. Runtime Mixins • Java mixins vs. Groovy mixins • Inject methods from other types • Works on classes and interfaces class Superman { def fly() { println quot;flyingquot; } } • Doesn’t not work on instances class Ninja { def fight() { println quot;fightingquot; } } • Global change Person.mixin Superman, Ninja • Easier to use than Categories p = new Person() p.sleep() • For method conflict last mixin p.fly() p.fight() takes precedence
  25. 25. Applications • Dynamic finders • Builders • Custom DSL • Dependency Injection • Method injection • Interceptors
  26. 26. Dynamic finders in Grails private static addDynamicFinderSupport(GrailsDomainClass dc, ….) { def mc = dc.metaClass findAllBy, CountBy, ListOrderBy def dynamicMethods = [ … ] patterns… mc.static.methodMissing = {String methodName, args -> def result = null StaticMethodInvocation method = dynamicMethods.find {it.isMethodMatch(methodName)} if (method) { synchronized(this) { mc.static.quot;$methodNamequot; = {List varArgs -> Register method on metaclass for method.invoke(dc.clazz, methodName, varArgs) faster lookup on subsequent } invocations } result = method.invoke(dc.clazz, methodName, args) } else { throw new MissingMethodException(methodName, delegate, args) } result } HibernatePluginSupport.groovy }
  27. 27. Builders • Easy way to hierarchical/recursive structures like XML, GUI components builder = new NodeBuilder() root = builder.persons { • person (name: 'obama') { • address(zip: 10016, street: 36) profession 'president' • } • person (name: 'joe') { • address(zip: 30312, street: 12) profession 'vice-president' • } } GPath expression println root.'**'.profession
  28. 28. Main Concepts • Method interception through invokeMethod or methodMissing – intercept method calls and dynamically create a node • Closure delegates – make sure all closures are relaying method calls to the builder
  29. 29. Anatomy of a builder builder = new NodeBuilder() root = builder.persons { 1. create node person by intercepting g • • person (name: 'obama') { address(zip: 10016, street: 36) the method call ( e.g builder.persons {…} ) profession 'president' • } 2. Execute the closure but first set the • person (name: 'joe') { delegate to the builder • address(zip: 30312, street: 12) profession 'vice-president' • } 3. Recursively do this until all nodes are } created println root.'**'.profession
  30. 30. How to write a builder • Extends BuilderSupport public class NodeBuilder extends BuilderSupport { • public static NodeBuilder newInstance() { • return new NodeBuilder(); BuilderSupport takes care of method • } interception and setting closure • protected void setParent(Object parent, Object child) { delegates • } • protected Object createNode(Object name) { • return new Node(getCurrentNode(), name, new ArrayList()); • } • protected Object createNode(Object name, Object value) { • return new Node(getCurrentNode(), name, value); • } • protected Object createNode(Object name, Map attributes) { • return new Node(getCurrentNode(), name, attributes, new ArrayList()); • } • protected Object createNode(Object name, Map attributes, Object value) { • return new Node(getCurrentNode(), name, attributes, value); • } • protected Node getCurrentNode() { • return (Node) getCurrent(); • } }
  31. 31. Some common builders • Grails - MarkupBuilder - SwingBuilder • Groovy - ConstrainedPropertyBuilder - BeanBuilder - HibernateCriteriaBuilder
  32. 32. AST Transformation • Compile time metaprogramming technique • Example from languages – C Macros (preprocessing) – C++ Templates (compile time instantiation) – Lisp Macros (very powerful) – Java Annotations (mostly code gen)
  33. 33. Basic Idea • You can manipulate code at many representations – Source (templating), AST, Bytecode (e.g AspectJ), runtime • Hook into compilation process and manipulate the AST – Higher level abstraction that bytecode – Different from traditional java annotations where transformations happen outside of the compiler
  34. 34. Groovy compilation process: 100,000 ft view Source Lexical analysis with a scanner Tokens Parsing with ANTLR Antlr AST Transform ANTL AST To Groovy AST Compile phases on the AST - Semantic analysis Groovy AST - Canonicalization - Instruction selection - Class generation - Output - Finalization Bytecode
  35. 35. Example AST 1 def sum (List lst){ 2 def total = lst.inject(0) { s, i -> s = s + i } 3 println total 4 return total 5 } println total ExpressionStatement MethodCallStatement VariableExpressions ConstantExpression ArgumentListExpression “println” VariableExpression “this” “total”
  36. 36. Example AST def total = lst.inject(0) { s, i -> s = s + i } ExpressionStatement DeclarationExpression VariableExpression “=” MethodCallExpression “total” VariableExpression ConstantExpression ArgumentListExpression “lst” “inject” ConstantExpression ClosureExpression “0” Parameter Parameter BlockStatement ExpressionStatement “s” “i” BinaryExpression VariableExpression “=” BinaryExpression VariableExpression VariableExpression “s” “=” “s” “i”
  37. 37. Types of transformations • Local – Applied to tagged (annotated) elements – Annotation driven – Can only be applied to Semantic Analysis phase or later • Global – applied to all classes that are compiled
  38. 38. Local AST Transformation Steps: 1. Create your transformation class by implementing ASTTransformation interface. • specify compile phase @GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS) 2. Create annotation and link to your transformation class • @GroovyASTTransformationClass(“full path to your transformation class”) 3. Write client code and annotate your elements (methods, fields, classes etc)
  39. 39. Step 1 @GroovyASTTransformation (phase = CompilePhase.SEMANTIC_ANALYSIS) Compile Phase class LoggingASTTransformation implements ASTTransformation { def void visit(ASTNode[] nodes, SourceUnit sourceUnit) Visitor pattern • { def methodList = sourceUnit.ast?.methods.findAll {MethodNode method -> method.getAnnotations(new ClassNode(WithLogging)) • } AST through sourceUnit methodList.each {MethodNode method -> • Statement startMessage = createPrintlnAst(quot;Starting $method.namequot;) • Statement endMessage = createPrintlnAst(quot;Ending $method.namequot;) • Statement code = method.getCode() • List existingStatements = code.getStatements() existingStatements.add(0, startMessage) Expression and statements existingStatements.add(endMessage) within the method body • } • } • private Statement createPrintlnAst(String message) • { • return new ExpressionStatement( • new MethodCallExpression( • new VariableExpression(quot;thisquot;), • new ConstantExpression(quot;printlnquot;), Creating AST for simple statement. YUCK! • new ArgumentListExpression( • new ConstantExpression(message) • ) • ) • ) • } }
  40. 40. Step 2 @Retention(RetentionPolicy.SOURCE) @Target([ElementType.METHOD]) @GroovyASTTransformationClass([“full path to your transformation class” quot;]) public @interface WithLogging { }
  41. 41. Step 3 @full.path.to.WithLogging def sum(List lst) { def total = lst.inject(0) { s, i -> s = s + i } println total • return total }
  42. 42. Examples • @Immutable – No mutators – All fields must be private and final – All fields must be included in equals, hashCode and toString computation – class must be final • @Singleton – lazy flavor – static instance • Grails – @EntityASTTransformation: • Injects Id, Version, toString and Associations to grails domain classes
  43. 43. Final Thoughts on AST Transformations • Cumbersome to write transformation currently • Future tooling (Groovy 1.7): – AST Browser – AST Builder
  44. 44. Summary of techniques • evaluate(“def add = {x, y -> x + y”) – Evaluate string as code • invokeMethod – Intercept all method call (Existing and non existing methods) R • methodMissing Can be defined on the class u – Intercept only non existing methods itself or on the metaClass n • getProperty/setProperty – Intercept property access and assignments t • ExpandoMetaClass i – Dynamically add methods, constructors, properties m • Categories – scoped injection e • Runtime Mixins – add methods from other types • AST Transformations – Transformations on groovy AST Compile time
  45. 45. References 1. What’s new in Groovy 1.6: http://www.infoq.com/articles/groovy-1-6 2. Hamlet D’Arcy blog: http://hamletdarcy.blogspot.com 3. Book: “Groovy in Action” by Dierk Koenig with Andrew Glover, Paul King, Guillaume Laforge and Jon Skeetsdsd 4. Various examples on http://groovy.codehaus.org

×