Groovy DSLs (JavaOne Presentation)

Experience • ADFm – Part of Oracle’s ADF Framework • Makes heavy use of Groovy as a scripting language – (But almost all the extensions are written in Java) • Extensions include – Accessing Field (Attribute) names as read-only variables – Associating functions with tables (Views) at runtime – Creating new top-level functions at runtime – A security overlay to existing classes – Extending existing classes with new functions Copyright © 2011, Oracle and/or its affiliates. All rights reserved.5

What’s a DSL • In general terms... – A purpose built language to solve a specific task • For example, LOGO, SQL, YACC, HTML • Some (i.e. Fowler) advocate it be minimal – But that’s not what we’re talking about today • For our purposes today: – Creating a custom language via adding and removing language features until you have what your users need – We’ll mostly focus on adding features today – (But I’ll show you how to take stuff out too.) Copyright © 2011, Oracle and/or its affiliates. All rights reserved.6

Why Groovy? • Groovy is very extensible – Language features (via AST Transforms, Customizers) – Object behavior (via MOP, invokeMethod, getProperty) • Groovy runs in a JVM – And most Java source will run as Groovy without modification • Groovy is very dynamic – Optional (duck) typing – Easy inline compilation • Groovy’s Open Source – With an incredibly helpful community Copyright © 2011, Oracle and/or its affiliates. All rights reserved.7

Some Caveats • I’m not an expert in Groovy – So many of my examples are in Java • There are many different ways to do almost everything – There are at least 7 different ways use the String “System.exit(0)” to shut down the VM. • Groovy 1.8 adds even more functionality – I haven’t used it in production yet. – But I’ll still talk about CompilationCustomizers today • This is a BOF - I’d love any comments realtime Copyright © 2011, Oracle and/or its affiliates. All rights reserved.8

Executing Programs • You’ll want to run user provided scripts at runtime • Simplest way is with GroovyShell.evaluate(String) – GroovyShell.evaluate(“println ‘hello world’”); – but this is expensive... • Separate parsing from execution: – Script script = GroovyShell.parse(String) • Time measured in seconds (you’re running a compiler) • Offers an intercept point for caching – Script.run() Copyright © 2011, Oracle and/or its affiliates. All rights reserved.10

Executing Scripts import groovy.lang.GroovyShell; import groovy.lang.Script; public class JavaOne01 { public static void main(String[] args) { GroovyShell shell = new GroovyShell(); Script script = shell.parse("println hello world"); script.run(); } } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.11

Under the hood • Groovy will take your unqualified text, and place it in the Script’s run method. • run has a return of type Object • If you don’t have a return value, the final statement executed will be turned into one – thus “x == y” is turned into “return x == y”, which is a Boolean • You can provide inline methods... – “int test() {return 1} n test()+2” is valid, returns 3 – but you can also run the test() method separately via reflection Copyright © 2011, Oracle and/or its affiliates. All rights reserved.12

Adding Objects via Bindings • A Binding is a way to pass values in/out of a script – Used as unqualified variables (“println name”) – declared variables (def, or a type) go in their own memory space • Groovy provides a default Binding, and you can write your own • Use it to provide: – Constant values – read only variables – default values Copyright © 2011, Oracle and/or its affiliates. All rights reserved.14

Sample Binding public class MyBinding extends groovy.lang.Binding { public Object getVariable(String name) { if ("name".equals(name)) { return "Jim"; } else if ("date".equals(name)) { return new Date(); } return super.getVariable(name); } public void setVariable(String name, Object value) { if ("date".equals(name) || "name".equals(name)) { throw new RuntimeException("variable "+name+" is read only"); } super.setVariable(name, value); } } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.15

Using the binding package javaone02; The script: import groovy.lang.GroovyShell; import groovy.lang.Script; println ‘hello ‘ + name println ‘it is now: ‘ + date public class JavaOne02 { public static void main(String[] args) { The output is: String runMe = "println hello + name n"+ hello Jim "println it is now: +date"; GroovyShell shell = new GroovyShell(); it is now: Wed Sep 28 18:35:41 PDT 2011 Script script = shell.parse(runMe); script.setBinding(new MyBinding()); script.run(); } } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.18

The ScriptBaseClass • Allows you to declare top level methods • Inserted as part of the CompilerConfiguration – An optional parameter to GroovyShell • Combine this with Groovy’s optional parenthesis... – (And optional chaining, introduced in 1.8...) – And simple constructs like LOGO’s turtle manipulation language start to come within reach Copyright © 2011, Oracle and/or its affiliates. All rights reserved.20

Turtle.groovy class Turtle { int distance = 0; def forward(int i) { distance += i println "moved forward "+i return this } def right(int i) { println "steering wheel stuck" return this } def traveled() { println "total distance traveled " + distance return this } } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.21

Turtle.groovy class Turtle { int distance = 0; def forward(int i) { distance += i println "moved forward "+i return this } def right(int i) { println "steering wheel stuck" return this } def traveled() { println "total distance traveled " + distance return this } } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.22

BaseScript.groovy abstract class BaseScript extends Script { Turtle t = new Turtle(); def forward(int i) { t.forward(i) } def right(int i) { t.right(i) } def traveled() { t.traveled() } } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.23

Main Class import groovy.lang.GroovyShell; import groovy.lang.Script; import org.codehaus.groovy.control.CompilerConfiguration; public class JavaOne03BaseScript { public static void main(String[] args) { CompilerConfiguration config = new CompilerConfiguration(); config.setScriptBaseClass("BaseScript"); GroovyShell shell = new GroovyShell(config); Script script = shell.parse("forward 10 right 3 forward 5 traveled()"); script.run(); } } Outputs: moved forward 10 steering wheel stuck moved forward 5 total distance traveled 15 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.24

invokeMethod • invokeMethod is a standard method, inherited from GroovyObject • Allows dynamic execution of methods • Used with BaseScript, it allows using a context object • One of many ways to do the same thing in Groovy • Can test before you run with – object.metaClass.respondsTo(object,name,*args) Copyright © 2011, Oracle and/or its affiliates. All rights reserved.28

Binding class MyBinding extends Binding { def a = new Dog(); def getVariable(String name) { if (name == "animal") { return a } return super.getVariable(name); } void setVariable(String name, Object value) { if ("animal".equals(name)) { throw new RuntimeException("variable "+name+" is read only"); } super.setVariable(name, value); } void changeAnimal(def animal) { a = animal; } } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.30

BaseScript.groovy abstract class BaseScript extends Script { def invokeMethod(String name, args) { animal."$name"(*args) } def change(String animal) { if (animal == "dog") { binding.changeAnimal(new Dog()) } else if (animal == "cat") { binding.changeAnimal(new Cat()) } } } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.33

get/setProperty • Properties are accessed via “object.propertyname” • Properties can be either fields or accessors/mutators – foo.bar can either be get/setBar, or the variable bar in foo • Like invokeMethod, this behavior can be overridden • Beware the “Java Field” operator - .@, which bypasses the get/setProperty methods Copyright © 2011, Oracle and/or its affiliates. All rights reserved.39

Compilation Customizers • New in 1.8 • Replace and/or augment ASTTransformations • Modify code as part of the compilation process • For example: – adding imports (via ImportCompilationCustomizer) – disabling language features (via SecureASTCustomizer) – automatically applying ASTTransforms (either local or global) (via ASTTranformationCustomizer) Copyright © 2011, Oracle and/or its affiliates. All rights reserved.41

Adding Imports import groovy.lang.GroovyShell; import groovy.lang.Script; import org.codehaus.groovy.control.CompilerConfiguration; import org.codehaus.groovy.control.customizers.ImportCustomizer; public class JavaOne05Customizer { public static void main(String[] args) { CompilerConfiguration config = new CompilerConfiguration(); ImportCustomizer customizer = new ImportCustomizer(); customizer.addStaticStars("java.lang.Math"); config.addCompilationCustomizers(customizer); GroovyShell shell = new GroovyShell(config); Output: Script script = shell.parse("println PI; println cos(1)"); 3.141592653589793 script.run(); 0.5403023058681398 } } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.42

Disallowing ++ Operations Script script; CompilerConfiguration config = new CompilerConfiguration(); SecureASTCustomizer customizer = new SecureASTCustomizer(); List<Integer> tokens = new ArrayList<Integer>(); tokens.add(Types.PLUS_PLUS); customizer.setTokensBlacklist(tokens); config.addCompilationCustomizers(customizer); GroovyShell shell = new GroovyShell(config); try { script = shell.parse("def i = 1 ; println i++"); } catch (MultipleCompilationErrorsException mcee) { Throwable t = mcee.getErrorCollector().getException(0); System.out.println(t.getClass().getName()); System.out.println(t.getMessage()); return; Output: } java.lang.SecurityException script.run(); Token ("++" at 1:22: "++" ) is not allowed Copyright © 2011, Oracle and/or its affiliates. All rights reserved.45

Forbidding for loopsScript script;CompilerConfiguration config = new CompilerConfiguration();SecureASTCustomizer customizer = new SecureASTCustomizer();List<Class<? extends Statement>> statements = new ArrayList<Class<? extends Statement>>();statements.add(ForStatement.class);customizer.setStatementsBlacklist(statements);config.addCompilationCustomizers(customizer);GroovyShell shell = new GroovyShell(config);try { script = shell.parse("for (i in [1,2,3]) {}");} catch (MultipleCompilationErrorsException mcee) { Throwable t = mcee.getErrorCollector().getException(0); System.out.println(t.getClass().getName()); System.out.println(t.getMessage()); return; Output:} java.lang.SecurityExceptionscript.run(); ForStatements are not allowed Copyright © 2011, Oracle and/or its affiliates. All rights reserved.49

ASTTransformations • Used to modify generated code from the compiler • Either global or local – Globals are triggered by compile time configuration – Locals are triggered by Annotations • Groovy compiler generates an Abstract Syntax Tree – breaks code into tree of Statements • which in turn are combinations of Statements and/or Expressions • You can examine code with groovyConsole to see the tree via Script -> Inspect AST Copyright © 2011, Oracle and/or its affiliates. All rights reserved.54

A Simple Use of AST - Checking Bind Variables • An early problem we ran into • Need to determine if binding variables are valid – Most useful for read-only bindings – Check for things like misspelling (so, “usrename” vs. “username”) • Create a new CompilationCustomizer • Visitor pattern means we visit every VariableExpression • Test if it’s going against the supplied Binding Copyright © 2011, Oracle and/or its affiliates. All rights reserved.56

Scanning for Binding variables, part 1 class ScriptVisitingCustomizer extends CompilationCustomizer { def visitor ScriptVisitingCustomizer() {super(CompilePhase.SEMANTIC_ANALYSIS)} void call(SourceUnit source, GeneratorContext context, ClassNode classNode) throws CompilationFailedException { for (method in classNode.methods) { if (method.name == run) method.code.visit(visitor) } } } class MyVisitor extends ClassCodeVisitorSupport { void visitVariableExpression(VariableExpression expression) { if (expression.accessedVariable instanceof DynamicVariable) println expression.name } protected SourceUnit getSourceUnit() {return source} } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.57

Scanning for Binding variables, part 2 def scriptText = """ println baz println foo def bar = 2 println bar """ CompilerConfiguration config = new CompilerConfiguration(); def customizer = new ScriptVisitingCustomizer(visitor: new MyVisitor()); Output: config.addCompilationCustomizers(customizer); baz foo GroovyShell shell = new GroovyShell(config); Script script = shell.parse(scriptText); Copyright © 2011, Oracle and/or its affiliates. All rights reserved.60

Scanning for Binding variables, part 2 def scriptText = """ println baz println foo def bar = 2 println bar """ CompilerConfiguration config = new CompilerConfiguration(); def customizer = new ScriptVisitingCustomizer(visitor: new MyVisitor()); Output: config.addCompilationCustomizers(customizer); baz foo GroovyShell shell = new GroovyShell(config); Script script = shell.parse(scriptText); Copyright © 2011, Oracle and/or its affiliates. All rights reserved.61

ASTTransformations • Numerous standard ASTTransforms are available – ThreadInterrupt - add Thread.currentThread().isInterrupted checks on loops, closures, first Statement in a Block – TimedInterrupt - as above, with a timeout check – ConditionalInterrupt - as above, with any closure – EqualsAndHashCode - create equals() and hashCode() methods – AutoClone - create clone() method – Immutable - create constructors, accessors, make final Copyright © 2011, Oracle and/or its affiliates. All rights reserved.62

Creating an ASTTransformation • Create an Annotation • Create an ASTTransformation class • (Optional, but likely) Create a ClassCodeVisitor • To use: – Decorate relevant object (method, class, etc) with annotation or – Use ASTTransformationCustomizer in GroovyShell Copyright © 2011, Oracle and/or its affiliates. All rights reserved.63

NameChange.groovy @Retention(RetentionPolicy.SOURCE) @Target(ElementType.TYPE) @GroovyASTTransformationClass("NameChangeASTTransformation") public @interface NameChange { } Copyright © 2011, Oracle and/or its affiliates. All rights reserved.64

NameChangeASTTransformation@GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS)public class NameChangeASTTransformation implements ASTTransformation { public void visit(ASTNode[] nodes, SourceUnit source) { def visitor = new CustomClassVisitor(); if (nodes.length != 2) throw new RuntimeException("exception"); nodes[1].visitContents(visitor); }}class CustomClassVisitor extends ClassCodeVisitorSupport { protected SourceUnit getSourceUnit() { throw new UnsupportedOperationException("Not supported yet."); } public void visitConstantExpression(ConstantExpression expression) { if (expression.value == "Bob") { expression.value = "Alice" } super.visitConstantExpression(expression); }} Copyright © 2011, Oracle and/or its affiliates. All rights reserved.65

Run with Annotation String runMe = "@NameChange n"+ "class Name {n"+ " static def name = Bobn"+ "} n" + "println Name.name"; GroovyShell shell = new GroovyShell(); Script script = shell.parse(runMe); script.run(); Output: Alice Copyright © 2011, Oracle and/or its affiliates. All rights reserved.69

Run with Annotation String runMe = "class Name {n"+ "static def name = Bobn"+ "} n" + "println Name.name"; CompilerConfiguration config = new CompilerConfiguration(); ASTTransformationCustomizer customizer = new ASTTransformationCustomizer(NameChange.class); config.addCompilationCustomizers(customizer); GroovyShell shell = new GroovyShell(config); Script script = shell.parse(runMe); Output: script.run(); Alice Copyright © 2011, Oracle and/or its affiliates. All rights reserved.70

Securing Your Code • GroovyShell(ClassLoader,Binding,CompilerConfiguration) • GroovyShell.parse(GroovyCodeSource) – GroovyCodeSource(String script, String name, String codeBase) • SecureASTCustomizer – ArithmeticShell – static analysis only! • method/property wrapping Copyright © 2011, Oracle and/or its affiliates. All rights reserved.71

Last Note: Naming Matters! This Code: Can take this as input: class Book { void name(String name) { Book.create { println name name "DSLs in Action" } loanedTo "Jim","Sint Si" void quantity(int i) { quantity 1 println i } } void loanedTo(String[] names) { println names } static create(closure) { def book = new Book() book.with closure return book } } *example inspired by “DSLs in Action”* Copyright © 2011, Oracle and/or its affiliates. All rights reserved.72

Resources • General Groovy Books – Groovy Programming: An Introduction for Java Programmers • (K&R style simple book) – Programming Groovy – Groovy in Action • (2nd Ed coming soon, with DSL chapters) • Domain Specific Languages – Groovy for Domain-Specific Languages • (great discussion of MOP, Builders) – DSLs in Action • (contains multiple language examples) Copyright © 2011, Oracle and/or its affiliates. All rights reserved.73

Groovy DSLs (JavaOne Presentation)

by Jim Driscoll

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Groovy DSLs (JavaOne Presentation) Presentation Transcript