Metaprogramming Techniques In Groovy And Grails

Metaprogramming techniques in Groovy and Grails Numan Salati numan.salati@gmail.com NY Groovy/Grails Meetup,

What makes a language dynamic? • Dynamic type system • Mutable types • Flexible method dispatch • Evaluate code at runtime (access to the interpreter) Key Idea: late binding !

Dynamic features in Java • dynamic class loading • dynamic binding – subclass Polymorphism • runtime annotations • dynamic proxies and reflection API – mostly read only – dynamic implementation of interfaces

Dynamic Groovy • Groovy has all this and much more – Intercept methods/properties – Create new methods/properties/constructors – Create classes at runtime – Runtime mixins (mutable types) – Evaluate any valid code string

method invocation example obj.method(arg1, arg2, arg3) • In Java – single dispatch – invokedynamic bytecode instruction • In Groovy – multiple dispatch – much more complicated logic but very flexible

method invocation example class Foo { def print (Object o) { println “println objectquot; } def print (String s) { println “println stringquot; } } Object arg = quot;stringquot; new Foo().print(arg) What gets called in Java vs. Groovy ?

Metaprogramming • Wikipedia definitions: – Programs that write or manipulate other programs – Expose internals of runtime engine to programming code through API” – Dynamic execution of string expression • Meta object protocol: Make program semantics – Explicit – Extensible How much of runtime and compile time structures are exposed?

Groovy MOP Excellent support for metaprogramming Compile time • Hook into the Groovy AST during compilation Runtime • Hook into method dispatching • Dynamically create methods/properties • Mutable types • Execution of code strings

Example class Person { def name def sleep() { println quot;sleepingquot;} } >> groovyc Person.groovy >> javap –public Person Compiled from quot;Person.groovyquot; public class Test extends java.lang.Object implements groovy.lang.GroovyObject { ….. public groovy.lang.MetaClass getMetaClass(); public void setMetaClass(groovy.lang.MetaClass); public java.lang.Object invokeMethod(java.lang.String, java.lang.Object); GroovyObject public java.lang.Object getProperty(java.lang.String); public void setProperty(java.lang.String, java.lang.Object); …. }

GroovyObject • All Groovy classes implement this interface • Open the class file in jad decompiler: public Object invokeMethod(String s, Object obj) { return getMetaClass().invokeMethod(this, s, obj); Default implementation } delegates to metaClass public Object getProperty(String s) { return getMetaClass().getProperty(this, s); } • Compiler assigns a metaClass to every POJO and POGO

GroovyObject • These methods are the hooks into method dispatch and property access/assignment • Overriding getProperty() and setProperty we can dynamically add properties and methods – This is exactly what Expando does • Dynamically create classes • Add methods by creating properties that are closures

Expando in Groovy class SimpleExpando { def propertyMap = [:] def getProperty(String name) { propertyMap[name]?: null } void setProperty(String name, Object value) { propertyMap[name] = value; } def invokeMethod(String name, Object args) { try { metaClass.invokeMethod(name, args); } catch (GroovyRuntimeException e) { def value = propertyMap[name]; if (value instanceof Closure) { Why set delegate value.setDelegate(this) value.call(args); before invoking? } else { throw e } } } } s = new SimpleExpando() s.add = {x, y -> x + y} println s.add(19,1)

InvokeMethod • Overriding invokeMethod in the class – intercepts all non existing method calls • What if we want to intercept all method calls? – Implement GroovyInterceptable marker interface – Override invokeMethod(String name, args) – Careful about stack overflow! • use metaClass.invokeMethod inside invokeMethod • Only non existing methods? – implement methodMissing(String name, args) – higher precedence than invokeMethod if both present

MetaClass • invokeMethod and methodMissing can be implemented in the class or on the metaClass • Metaclass defines the dynamic behavior of the object • Query runtime structure of the class/object – respondsTo – hasProperty – getMethods vs. getMetaMethods – getProperties vs. etMetaProperties

MetaClass • Define new methods and constructors on class using ExpandoMetaClass Person.metaClass.play = { println quot;playquot;} Person.metaClass.eat = { pritnln quot;eatquot; } Person.metaClass.code = { println quot;codequot;} Person.metaClass.static.read = { println quot;readingquot; } Person.metaClass.constructor = {name -> new Person(quot;Sir: quot; + name) } Heap Overflow! - use BeanUtils.instantiateClass to instantiate outside of groovy or using EMC DSL Person.metaClass { play { println quot;playquot;} eat { pritnln quot;eatquot; } code { println quot;codequot;} 'static' { read { println quot;readingquot; } } constructor { name -> BeanUtils.instantiateClass(Person, quot;Sir: quot; + name) } }

EMC • Injection works for both POJOs and POGOs – Integer.randomNum = { … } – String.metaClass = <Roll your own super cool metaClass> • Add to instance only? p1 = new Person() p2 = new Person() p2.metaClass.party = { println quot;partyingquot;} p2.party() p1.party() MissingMethodException • Works for POJOs too

EMC • new methods are reflected in the subclass hierarchy • Adding methods to interfaces? – set enableGlobally on EMC to affect all implementing classes

Summary of method dispatch so far.. • If a method is defined in the metaClass invoke that • Or else look for hooks in the class or metaClass: – invokeMethod – methodMissing – getProperty – setProperty

Method dispatch flow diagram

Categories • Injection of methods within a scope import org.codehaus.groovy.runtime.TimeCategory use(TimeCategory) { println 2.days.from.now println 3.years.from.now println 10.minutes.from.now println 3.weeks.from.now } • Injects getDays() and getYears() method defined in TimeCategory into the meta class of Integer objects in the “use” block and for the current thread

Categories public class TimeCategory { .... • public static DatumDependentDuration getMonths(final Integer self) { • return new DatumDependentDuration(0, self.intValue(), 0, 0, 0, 0, 0); • } • public static DatumDependentDuration getYears(final Integer self) { • return new DatumDependentDuration(self.intValue(), 0, 0, 0, 0, 0, 0); • } • public static Duration getWeeks(final Integer self) { • return new Duration(self.intValue() * 7, 0, 0, 0, 0); • } • All methods are static • public static TimeDuration getHours(final Integer self) { • return new TimeDuration(0, self.intValue(), 0, 0, 0); • } • First argument is the class .... getting injected }

Categories • Can nest categories – in case of method clash, last one takes precedence • Can use multiple categories in the “use” clause – same precedence rule as above • Other built in categories – DomCategory – SerlvetCategory

Categories • How it work internally: 1. Creates new scope 2. Adds static methods from category class into thread local stack 3. Call closure 4. Remove methods from stack – check out “use” method in “GroovyCategoryStack.java” • Slower than metaClass injection – scanning static methods – cleanup

Runtime Mixins • Java mixins vs. Groovy mixins • Inject methods from other types • Works on classes and interfaces class Superman { def fly() { println quot;flyingquot; } } • Doesn’t not work on instances class Ninja { def fight() { println quot;fightingquot; } } • Global change Person.mixin Superman, Ninja • Easier to use than Categories p = new Person() p.sleep() • For method conflict last mixin p.fly() p.fight() takes precedence

Applications • Dynamic finders • Builders • Custom DSL • Dependency Injection • Method injection • Interceptors

Dynamic finders in Grails private static addDynamicFinderSupport(GrailsDomainClass dc, ….) { def mc = dc.metaClass findAllBy, CountBy, ListOrderBy def dynamicMethods = [ … ] patterns… mc.static.methodMissing = {String methodName, args -> def result = null StaticMethodInvocation method = dynamicMethods.find {it.isMethodMatch(methodName)} if (method) { synchronized(this) { mc.static.quot;$methodNamequot; = {List varArgs -> Register method on metaclass for method.invoke(dc.clazz, methodName, varArgs) faster lookup on subsequent } invocations } result = method.invoke(dc.clazz, methodName, args) } else { throw new MissingMethodException(methodName, delegate, args) } result } HibernatePluginSupport.groovy }

Builders • Easy way to hierarchical/recursive structures like XML, GUI components builder = new NodeBuilder() root = builder.persons { • person (name: 'obama') { • address(zip: 10016, street: 36) profession 'president' • } • person (name: 'joe') { • address(zip: 30312, street: 12) profession 'vice-president' • } } GPath expression println root.'**'.profession

Main Concepts • Method interception through invokeMethod or methodMissing – intercept method calls and dynamically create a node • Closure delegates – make sure all closures are relaying method calls to the builder

Anatomy of a builder builder = new NodeBuilder() root = builder.persons { 1. create node person by intercepting g • • person (name: 'obama') { address(zip: 10016, street: 36) the method call ( e.g builder.persons {…} ) profession 'president' • } 2. Execute the closure but first set the • person (name: 'joe') { delegate to the builder • address(zip: 30312, street: 12) profession 'vice-president' • } 3. Recursively do this until all nodes are } created println root.'**'.profession

How to write a builder • Extends BuilderSupport public class NodeBuilder extends BuilderSupport { • public static NodeBuilder newInstance() { • return new NodeBuilder(); BuilderSupport takes care of method • } interception and setting closure • protected void setParent(Object parent, Object child) { delegates • } • protected Object createNode(Object name) { • return new Node(getCurrentNode(), name, new ArrayList()); • } • protected Object createNode(Object name, Object value) { • return new Node(getCurrentNode(), name, value); • } • protected Object createNode(Object name, Map attributes) { • return new Node(getCurrentNode(), name, attributes, new ArrayList()); • } • protected Object createNode(Object name, Map attributes, Object value) { • return new Node(getCurrentNode(), name, attributes, value); • } • protected Node getCurrentNode() { • return (Node) getCurrent(); • } }

Some common builders • Grails - MarkupBuilder - SwingBuilder • Groovy - ConstrainedPropertyBuilder - BeanBuilder - HibernateCriteriaBuilder

AST Transformation • Compile time metaprogramming technique • Example from languages – C Macros (preprocessing) – C++ Templates (compile time instantiation) – Lisp Macros (very powerful) – Java Annotations (mostly code gen)

Basic Idea • You can manipulate code at many representations – Source (templating), AST, Bytecode (e.g AspectJ), runtime • Hook into compilation process and manipulate the AST – Higher level abstraction that bytecode – Different from traditional java annotations where transformations happen outside of the compiler

Groovy compilation process: 100,000 ft view Source Lexical analysis with a scanner Tokens Parsing with ANTLR Antlr AST Transform ANTL AST To Groovy AST Compile phases on the AST - Semantic analysis Groovy AST - Canonicalization - Instruction selection - Class generation - Output - Finalization Bytecode

Example AST 1 def sum (List lst){ 2 def total = lst.inject(0) { s, i -> s = s + i } 3 println total 4 return total 5 } println total ExpressionStatement MethodCallStatement VariableExpressions ConstantExpression ArgumentListExpression “println” VariableExpression “this” “total”

Example AST def total = lst.inject(0) { s, i -> s = s + i } ExpressionStatement DeclarationExpression VariableExpression “=” MethodCallExpression “total” VariableExpression ConstantExpression ArgumentListExpression “lst” “inject” ConstantExpression ClosureExpression “0” Parameter Parameter BlockStatement ExpressionStatement “s” “i” BinaryExpression VariableExpression “=” BinaryExpression VariableExpression VariableExpression “s” “=” “s” “i”

Types of transformations • Local – Applied to tagged (annotated) elements – Annotation driven – Can only be applied to Semantic Analysis phase or later • Global – applied to all classes that are compiled

Local AST Transformation Steps: 1. Create your transformation class by implementing ASTTransformation interface. • specify compile phase @GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS) 2. Create annotation and link to your transformation class • @GroovyASTTransformationClass(“full path to your transformation class”) 3. Write client code and annotate your elements (methods, fields, classes etc)

Step 1 @GroovyASTTransformation (phase = CompilePhase.SEMANTIC_ANALYSIS) Compile Phase class LoggingASTTransformation implements ASTTransformation { def void visit(ASTNode[] nodes, SourceUnit sourceUnit) Visitor pattern • { def methodList = sourceUnit.ast?.methods.findAll {MethodNode method -> method.getAnnotations(new ClassNode(WithLogging)) • } AST through sourceUnit methodList.each {MethodNode method -> • Statement startMessage = createPrintlnAst(quot;Starting $method.namequot;) • Statement endMessage = createPrintlnAst(quot;Ending $method.namequot;) • Statement code = method.getCode() • List existingStatements = code.getStatements() existingStatements.add(0, startMessage) Expression and statements existingStatements.add(endMessage) within the method body • } • } • private Statement createPrintlnAst(String message) • { • return new ExpressionStatement( • new MethodCallExpression( • new VariableExpression(quot;thisquot;), • new ConstantExpression(quot;printlnquot;), Creating AST for simple statement. YUCK! • new ArgumentListExpression( • new ConstantExpression(message) • ) • ) • ) • } }

Step 2 @Retention(RetentionPolicy.SOURCE) @Target([ElementType.METHOD]) @GroovyASTTransformationClass([“full path to your transformation class” quot;]) public @interface WithLogging { }

Step 3 @full.path.to.WithLogging def sum(List lst) { def total = lst.inject(0) { s, i -> s = s + i } println total • return total }

Examples • @Immutable – No mutators – All fields must be private and final – All fields must be included in equals, hashCode and toString computation – class must be final • @Singleton – lazy flavor – static instance • Grails – @EntityASTTransformation: • Injects Id, Version, toString and Associations to grails domain classes

Final Thoughts on AST Transformations • Cumbersome to write transformation currently • Future tooling (Groovy 1.7): – AST Browser – AST Builder

Summary of techniques • evaluate(“def add = {x, y -> x + y”) – Evaluate string as code • invokeMethod – Intercept all method call (Existing and non existing methods) R • methodMissing Can be defined on the class u – Intercept only non existing methods itself or on the metaClass n • getProperty/setProperty – Intercept property access and assignments t • ExpandoMetaClass i – Dynamically add methods, constructors, properties m • Categories – scoped injection e • Runtime Mixins – add methods from other types • AST Transformations – Transformations on groovy AST Compile time

References 1. What’s new in Groovy 1.6: http://www.infoq.com/articles/groovy-1-6 2. Hamlet D’Arcy blog: http://hamletdarcy.blogspot.com 3. Book: “Groovy in Action” by Dierk Koenig with Andrew Glover, Paul King, Guillaume Laforge and Jon Skeetsdsd 4. Various examples on http://groovy.codehaus.org

http://paraggajbhiye.wordpress.com	165
http://www.webdesigner.la	127
http://www.slideshare.net	116
http://blog.ultrainno.com	31
https://twitter.com	3

Metaprogramming Techniques In Groovy And Grails

by zenMonkey

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 442

Statistics

Metaprogramming Techniques In Groovy And Grails Presentation Transcript