Metaprogramming techniques in
Groovy and Grails
Numan Salati
numan.salati@gmail.com
NY Groovy/Grails Meetup,

What makes a language dynamic?
• Dynamic type system
• Mutable types
• Flexible method dispatch
• Evaluate code at runtime (access to the
interpreter)
Key Idea: late binding !

Dynamic features in Java
• dynamic class loading
• dynamic binding
– subclass Polymorphism
• runtime annotations
• dynamic proxies and reflection API
– mostly read only
– dynamic implementation of interfaces

Dynamic Groovy
• Groovy has all this and much more
– Intercept methods/properties
– Create new methods/properties/constructors
– Create classes at runtime
– Runtime mixins (mutable types)
– Evaluate any valid code string

method invocation example
obj.method(arg1, arg2, arg3)
• In Java
– single dispatch
– invokedynamic bytecode instruction
• In Groovy
– multiple dispatch
– much more complicated logic but very flexible

method invocation example
class Foo
{
def print (Object o) { println “println object\" }
def print (String s) { println “println string\" }
}
Object arg = \"string\"
new Foo().print(arg)
What gets called in Java vs. Groovy ?

Metaprogramming
• Wikipedia definitions:
– Programs that write or manipulate other programs
– Expose internals of runtime engine to programming code
through API”
– Dynamic execution of string expression
• Meta object protocol: Make program semantics
– Explicit
– Extensible
How much of runtime and compile time structures are
exposed?

Groovy MOP
Excellent support for metaprogramming
Compile time
• Hook into the Groovy AST during compilation
Runtime
• Hook into method dispatching
• Dynamically create methods/properties
• Mutable types
• Execution of code strings

Example
class Person
{
def name
def sleep() { println \"sleeping\"}
}
>> groovyc Person.groovy
>> javap –public Person
Compiled from \"Person.groovy\"
public class Test extends java.lang.Object implements groovy.lang.GroovyObject
{
…..
public groovy.lang.MetaClass getMetaClass();
public void setMetaClass(groovy.lang.MetaClass);
public java.lang.Object invokeMethod(java.lang.String, java.lang.Object); GroovyObject
public java.lang.Object getProperty(java.lang.String);
public void setProperty(java.lang.String, java.lang.Object);
….
}

GroovyObject
• All Groovy classes implement this interface
• Open the class file in jad decompiler:
public Object invokeMethod(String s, Object obj)
{
return getMetaClass().invokeMethod(this, s, obj); Default implementation
} delegates to metaClass
public Object getProperty(String s)
{
return getMetaClass().getProperty(this, s);
}
• Compiler assigns a metaClass to every POJO and POGO

GroovyObject
• These methods are the hooks into method
dispatch and property access/assignment
• Overriding getProperty() and setProperty we
can dynamically add properties and methods
– This is exactly what Expando does
• Dynamically create classes
• Add methods by creating properties that are closures

Expando in Groovy
class SimpleExpando {
def propertyMap = [:]
def getProperty(String name) {
propertyMap[name]?: null
}
void setProperty(String name, Object value) {
propertyMap[name] = value;
}
def invokeMethod(String name, Object args) {
try {
metaClass.invokeMethod(name, args);
}
catch (GroovyRuntimeException e) {
def value = propertyMap[name];
if (value instanceof Closure) { Why set delegate
value.setDelegate(this)
value.call(args); before invoking?
}
else { throw e }
}
}
}
s = new SimpleExpando()
s.add = {x, y -> x + y}
println s.add(19,1)

InvokeMethod
• Overriding invokeMethod in the class
– intercepts all non existing method calls
• What if we want to intercept all method calls?
– Implement GroovyInterceptable marker interface
– Override invokeMethod(String name, args)
– Careful about stack overflow!
• use metaClass.invokeMethod inside invokeMethod
• Only non existing methods?
– implement methodMissing(String name, args)
– higher precedence than invokeMethod if both present

MetaClass
• invokeMethod and methodMissing can be
implemented in the class or on the metaClass
• Metaclass defines the dynamic behavior of the
object
• Query runtime structure of the class/object
– respondsTo
– hasProperty
– getMethods vs. getMetaMethods
– getProperties vs. etMetaProperties

MetaClass
• Define new methods and constructors on class using ExpandoMetaClass
Person.metaClass.play = { println \"play\"}
Person.metaClass.eat = { pritnln \"eat\" }
Person.metaClass.code = { println \"code\"}
Person.metaClass.static.read = { println \"reading\" }
Person.metaClass.constructor = {name -> new Person(\"Sir: \" + name) }
Heap Overflow! - use BeanUtils.instantiateClass to instantiate outside of groovy
or using EMC DSL
Person.metaClass {
play { println \"play\"}
eat { pritnln \"eat\" }
code { println \"code\"}
'static' { read { println \"reading\" } }
constructor { name -> BeanUtils.instantiateClass(Person, \"Sir: \" + name) }
}

EMC
• Injection works for both POJOs and POGOs
– Integer.randomNum = { … }
– String.metaClass = <Roll your own super cool metaClass>
• Add to instance only?
p1 = new Person()
p2 = new Person()
p2.metaClass.party = { println \"partying\"}
p2.party()
p1.party() MissingMethodException
• Works for POJOs too

EMC
• new methods are reflected in the subclass
hierarchy
• Adding methods to interfaces?
– set enableGlobally on EMC to affect all
implementing classes

Summary of method dispatch so far..
• If a method is defined in the metaClass invoke
that
• Or else look for hooks in the class or
metaClass:
– invokeMethod
– methodMissing
– getProperty
– setProperty

Method dispatch flow diagram

Categories
• Injection of methods within a scope
import org.codehaus.groovy.runtime.TimeCategory
use(TimeCategory)
{
println 2.days.from.now
println 3.years.from.now
println 10.minutes.from.now
println 3.weeks.from.now
}
• Injects getDays() and getYears() method defined in
TimeCategory into the meta class of Integer objects in
the “use” block and for the current thread

Categories
public class TimeCategory
{
....
• public static DatumDependentDuration getMonths(final Integer self) {
• return new DatumDependentDuration(0, self.intValue(), 0, 0, 0, 0, 0);
• }
• public static DatumDependentDuration getYears(final Integer self) {
• return new DatumDependentDuration(self.intValue(), 0, 0, 0, 0, 0, 0);
• }
• public static Duration getWeeks(final Integer self) {
• return new Duration(self.intValue() * 7, 0, 0, 0, 0);
• }
• All methods are static
• public static TimeDuration getHours(final Integer self) {
• return new TimeDuration(0, self.intValue(), 0, 0, 0);
• } • First argument is the class
.... getting injected
}

Categories
• Can nest categories
– in case of method clash, last one takes precedence
• Can use multiple categories in the “use” clause
– same precedence rule as above
• Other built in categories
– DomCategory
– SerlvetCategory

Categories
• How it work internally:
1. Creates new scope
2. Adds static methods from category class into thread local
stack
3. Call closure
4. Remove methods from stack
– check out “use” method in “GroovyCategoryStack.java”
• Slower than metaClass injection
– scanning static methods
– cleanup

Runtime Mixins
• Java mixins vs. Groovy mixins
• Inject methods from other types
• Works on classes and interfaces
class Superman {
def fly() { println \"flying\" }
}
• Doesn’t not work on instances
class Ninja {
def fight() { println \"fighting\" }
} • Global change
Person.mixin Superman, Ninja • Easier to use than Categories
p = new Person()
p.sleep() • For method conflict last mixin
p.fly()
p.fight() takes precedence

Applications
• Dynamic finders
• Builders
• Custom DSL
• Dependency Injection
• Method injection
• Interceptors

Dynamic finders in Grails
private static addDynamicFinderSupport(GrailsDomainClass dc, ….) {
def mc = dc.metaClass
findAllBy, CountBy, ListOrderBy
def dynamicMethods = [ … ] patterns…
mc.static.methodMissing = {String methodName, args ->
def result = null
StaticMethodInvocation method = dynamicMethods.find {it.isMethodMatch(methodName)}
if (method) {
synchronized(this) {
mc.static.\"$methodName\" = {List varArgs -> Register method on metaclass for
method.invoke(dc.clazz, methodName, varArgs) faster lookup on subsequent
} invocations
}
result = method.invoke(dc.clazz, methodName, args)
}
else {
throw new MissingMethodException(methodName, delegate, args)
}
result
}
HibernatePluginSupport.groovy
}

Builders
• Easy way to hierarchical/recursive structures like XML, GUI components
builder = new NodeBuilder()
root = builder.persons {
• person (name: 'obama') {
• address(zip: 10016, street: 36)
profession 'president'
• }
• person (name: 'joe') {
• address(zip: 30312, street: 12)
profession 'vice-president'
• }
}
GPath expression
println root.'**'.profession

Main Concepts
• Method interception through invokeMethod
or methodMissing
– intercept method calls and dynamically create a
node
• Closure delegates
– make sure all closures are relaying method calls to
the builder

Anatomy of a builder
builder = new NodeBuilder()
root = builder.persons { 1. create node person by intercepting g
•
•
person (name: 'obama') {
address(zip: 10016, street: 36)
the method call ( e.g builder.persons {…} )
profession 'president'
• }
2. Execute the closure but first set the
• person (name: 'joe') { delegate to the builder
• address(zip: 30312, street: 12)
profession 'vice-president'
• } 3. Recursively do this until all nodes are
}
created
println root.'**'.profession

How to write a builder
• Extends BuilderSupport
public class NodeBuilder extends BuilderSupport {
• public static NodeBuilder newInstance() {
• return new NodeBuilder(); BuilderSupport takes care of method
• } interception and setting closure
• protected void setParent(Object parent, Object child) {
delegates
• }
• protected Object createNode(Object name) {
• return new Node(getCurrentNode(), name, new ArrayList());
• }
• protected Object createNode(Object name, Object value) {
• return new Node(getCurrentNode(), name, value);
• }
• protected Object createNode(Object name, Map attributes) {
• return new Node(getCurrentNode(), name, attributes, new ArrayList());
• }
• protected Object createNode(Object name, Map attributes, Object value) {
• return new Node(getCurrentNode(), name, attributes, value);
• }
• protected Node getCurrentNode() {
• return (Node) getCurrent();
• }
}

Some common builders
• Grails
- MarkupBuilder
- SwingBuilder
• Groovy
- ConstrainedPropertyBuilder
- BeanBuilder
- HibernateCriteriaBuilder

AST Transformation
• Compile time metaprogramming technique
• Example from languages
– C Macros (preprocessing)
– C++ Templates (compile time instantiation)
– Lisp Macros (very powerful)
– Java Annotations (mostly code gen)

Basic Idea
• You can manipulate code at many representations
– Source (templating), AST, Bytecode (e.g AspectJ), runtime
• Hook into compilation process and manipulate the AST
– Higher level abstraction that bytecode
– Different from traditional java annotations where
transformations happen outside of the compiler

Groovy compilation process: 100,000 ft view
Source
Lexical analysis with a scanner
Tokens
Parsing with ANTLR
Antlr AST
Transform ANTL AST To Groovy AST
Compile phases on the AST
- Semantic analysis Groovy AST
- Canonicalization
- Instruction selection
- Class generation
- Output
- Finalization
Bytecode

Example AST
1 def sum (List lst){
2 def total = lst.inject(0) { s, i -> s = s + i }
3 println total
4 return total
5 }
println total
ExpressionStatement
MethodCallStatement
VariableExpressions ConstantExpression ArgumentListExpression
“println” VariableExpression
“this”
“total”

Example AST
def total = lst.inject(0) { s, i -> s = s + i }
ExpressionStatement
DeclarationExpression
VariableExpression “=” MethodCallExpression
“total” VariableExpression ConstantExpression ArgumentListExpression
“lst” “inject” ConstantExpression ClosureExpression
“0” Parameter Parameter BlockStatement
ExpressionStatement
“s” “i”
BinaryExpression
VariableExpression “=” BinaryExpression
VariableExpression VariableExpression
“s” “=”
“s” “i”

Types of transformations
• Local
– Applied to tagged (annotated) elements
– Annotation driven
– Can only be applied to Semantic Analysis phase or
later
• Global
– applied to all classes that are compiled

Local AST Transformation
Steps:
1. Create your transformation class by
implementing ASTTransformation interface.
• specify compile phase
@GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS)
2. Create annotation and link to your
transformation class
• @GroovyASTTransformationClass(“full path to your transformation class”)
3. Write client code and annotate your elements
(methods, fields, classes etc)

Step 1
@GroovyASTTransformation (phase = CompilePhase.SEMANTIC_ANALYSIS) Compile Phase
class LoggingASTTransformation implements ASTTransformation
{
def void visit(ASTNode[] nodes, SourceUnit sourceUnit) Visitor pattern
• {
def methodList = sourceUnit.ast?.methods.findAll {MethodNode method ->
method.getAnnotations(new ClassNode(WithLogging))
• } AST through sourceUnit
methodList.each {MethodNode method ->
• Statement startMessage = createPrintlnAst(\"Starting $method.name\")
• Statement endMessage = createPrintlnAst(\"Ending $method.name\")
• Statement code = method.getCode()
• List existingStatements = code.getStatements()
existingStatements.add(0, startMessage)
Expression and statements
existingStatements.add(endMessage) within the method body
• }
• }
• private Statement createPrintlnAst(String message)
• {
• return new ExpressionStatement(
• new MethodCallExpression(
• new VariableExpression(\"this\"),
• new ConstantExpression(\"println\"),
Creating AST for simple statement. YUCK!
• new ArgumentListExpression(
• new ConstantExpression(message)
• )
• )
• )
• }
}

Step 2
@Retention(RetentionPolicy.SOURCE)
@Target([ElementType.METHOD])
@GroovyASTTransformationClass([“full path to your transformation
class” \"])
public @interface WithLogging
{
}

Step 3
@full.path.to.WithLogging
def sum(List lst)
{
def total = lst.inject(0) { s, i -> s = s + i }
println total
• return total
}

Examples
• @Immutable
– No mutators
– All fields must be private and final
– All fields must be included in equals, hashCode and toString
computation
– class must be final
• @Singleton
– lazy flavor
– static instance
• Grails
– @EntityASTTransformation:
• Injects Id, Version, toString and Associations to grails domain classes

Final Thoughts on AST Transformations
• Cumbersome to write transformation
currently
• Future tooling (Groovy 1.7):
– AST Browser
– AST Builder

Summary of techniques
• evaluate(“def add = {x, y -> x + y”)
– Evaluate string as code
• invokeMethod
– Intercept all method call (Existing and non existing methods) R
• methodMissing Can be defined on the class u
– Intercept only non existing methods itself or on the metaClass
n
• getProperty/setProperty
– Intercept property access and assignments t
• ExpandoMetaClass i
– Dynamically add methods, constructors, properties
m
• Categories
– scoped injection e
• Runtime Mixins
– add methods from other types
• AST Transformations
– Transformations on groovy AST
Compile time

References
1. What’s new in Groovy 1.6: http://www.infoq.com/articles/groovy-1-6
2. Hamlet D’Arcy blog: http://hamletdarcy.blogspot.com
3. Book: “Groovy in Action” by Dierk Koenig with Andrew Glover, Paul
King, Guillaume Laforge and Jon Skeetsdsd
4. Various examples on http://groovy.codehaus.org