Groovy DSLs (JavaOne Presentation)

Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 8
1

DSLs with Groovy
Jim Driscoll
2

Talk Agenda

• Introduction
• Executing Programs
• The Nouns - Objects
• The Verbs - Functions
• The AST

Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
3

Introductions

• Experiences
• What is a DSL
• Why Groovy?

4

Experience

• ADFm
– Part of Oracle’s ADF Framework
• Makes heavy use of Groovy as a scripting language
– (But almost all the extensions are written in Java)
• Extensions include
– Accessing Field (Attribute) names as read-only variables
– Associating functions with tables (Views) at runtime
– Creating new top-level functions at runtime
– A security overlay to existing classes
– Extending existing classes with new functions
5

What’s a DSL

• In general terms...
– A purpose built language to solve a specific task
• For example, LOGO, SQL, YACC, HTML
• Some (i.e. Fowler) advocate it be minimal
– But that’s not what we’re talking about today
• For our purposes today:
– Creating a custom language via adding and removing language
features until you have what your users need
– We’ll mostly focus on adding features today
– (But I’ll show you how to take stuff out too.)
6

Why Groovy?

• Groovy is very extensible
– Language features (via AST Transforms, Customizers)
– Object behavior (via MOP, invokeMethod, getProperty)
• Groovy runs in a JVM
– And most Java source will run as Groovy without modification
• Groovy is very dynamic
– Optional (duck) typing
– Easy inline compilation
• Groovy’s Open Source
– With an incredibly helpful community
7

Some Caveats

• I’m not an expert in Groovy
– So many of my examples are in Java
• There are many different ways to do almost everything
– There are at least 7 different ways use the String “System.exit(0)”
to shut down the VM.
• Groovy 1.8 adds even more functionality
– I haven’t used it in production yet.
– But I’ll still talk about CompilationCustomizers today
• This is a BOF - I’d love any comments realtime

8

Talk Agenda

• Introduction
• The AST

9

Executing Programs

• You’ll want to run user provided scripts at runtime
• Simplest way is with GroovyShell.evaluate(String)
– GroovyShell.evaluate(“println ‘hello world’”);
– but this is expensive...
• Separate parsing from execution:
– Script script = GroovyShell.parse(String)
• Time measured in seconds (you’re running a compiler)
• Offers an intercept point for caching
– Script.run()

10

Executing Scripts

import groovy.lang.GroovyShell;
import groovy.lang.Script;

public class JavaOne01 {

public static void main(String[] args) {
GroovyShell shell = new GroovyShell();
Script script = shell.parse("println 'hello world'");
script.run();
}
}

11

Under the hood

• Groovy will take your unqualified text, and place it in the
Script’s run method.
• run has a return of type Object
• If you don’t have a return value, the final statement
executed will be turned into one
– thus “x == y” is turned into “return x == y”, which is a Boolean
• You can provide inline methods...
– “int test() {return 1} n test()+2” is valid, returns 3
– but you can also run the test() method separately via reflection

12

Talk Agenda

• Introduction
• The AST

13

Adding Objects via Bindings

• A Binding is a way to pass values in/out of a script
– Used as unqualified variables (“println name”)
– declared variables (def, or a type) go in their own memory space
• Groovy provides a default Binding, and you can write
your own
• Use it to provide:
– Constant values
– read only variables
– default values

14

Sample Binding

public class MyBinding extends groovy.lang.Binding {
public Object getVariable(String name) {
if ("name".equals(name)) {
return "Jim";
} else if ("date".equals(name)) {
return new Date();
}
return super.getVariable(name);
}
public void setVariable(String name, Object value) {
if ("date".equals(name) || "name".equals(name)) {
throw new RuntimeException("variable "+name+" is read only");
}
super.setVariable(name, value);
}
}

15

Sample Binding

return "Jim";
return new Date();
}
}
}
}
}

16

Sample Binding

return "Jim";
return new Date();
}
}
}
}
}

17

Using the binding
package javaone02; The script:
import groovy.lang.Script; println ‘hello ‘ + name
println ‘it is now: ‘ + date
public class JavaOne02 {

public static void main(String[] args) { The output is:
String runMe =
"println 'hello '+ name n"+ hello Jim
"println 'it is now: ' +date";
GroovyShell shell = new GroovyShell(); it is now: Wed Sep 28 18:35:41 PDT 2011
Script script = shell.parse(runMe);
script.setBinding(new MyBinding());
script.run();
}
}

18

Talk Agenda

• Introduction
• The AST

19

The ScriptBaseClass

• Allows you to declare top level methods
• Inserted as part of the CompilerConfiguration
– An optional parameter to GroovyShell
• Combine this with Groovy’s optional parenthesis...
– (And optional chaining, introduced in 1.8...)
– And simple constructs like LOGO’s turtle manipulation language
start to come within reach

20

Turtle.groovy
class Turtle {
int distance = 0;
def forward(int i) {
distance += i
println "moved forward "+i
return this
}
def right(int i) {
println "steering wheel stuck"
return this
}
def traveled() {
println "total distance traveled " + distance
return this
}
}

21

Turtle.groovy
class Turtle {
int distance = 0;
distance += i
println "moved forward "+i
return this
}
def right(int i) {
println "steering wheel stuck"
return this
}
def traveled() {
println "total distance traveled " + distance
return this
}
}

22

BaseScript.groovy
abstract class BaseScript extends Script {
Turtle t = new Turtle();
t.forward(i)
}
def right(int i) {
t.right(i)
}
def traveled() {
t.traveled()
}
}

23

Main Class
import org.codehaus.groovy.control.CompilerConfiguration;

public class JavaOne03BaseScript {
CompilerConfiguration config = new CompilerConfiguration();
config.setScriptBaseClass("BaseScript");
GroovyShell shell = new GroovyShell(config);
Script script = shell.parse("forward 10 right 3 forward 5 traveled()");
script.run();
}
}

Outputs:
moved forward 10
steering wheel stuck
moved forward 5
total distance traveled 15
24

Main Class

script.run();
}
}

Outputs:
moved forward 10
moved forward 5
25

Main Class

script.run();
}
}

Outputs:
moved forward 10
moved forward 5
26

Main Class

script.run();
}
}

Outputs:
moved forward 10
moved forward 5
27

invokeMethod

• invokeMethod is a standard method, inherited from
GroovyObject
• Allows dynamic execution of methods
• Used with BaseScript, it allows using a context object
• One of many ways to do the same thing in Groovy
• Can test before you run with
– object.metaClass.respondsTo(object,name,*args)

28

Objects - Cat and Dog

class Dog {
def speak() {
println "woof"
}
}
class Cat {
def speak() {
println "meow"
}
}

29

Binding
class MyBinding extends Binding {
def a = new Dog();
def getVariable(String name) {
if (name == "animal") {
return a
}
}
void setVariable(String name, Object value) {
if ("animal".equals(name)) {
}
}
void changeAnimal(def animal) {
a = animal;
}
}

30

Binding
def a = new Dog();
return a
}
}
}
}
a = animal;
}
}

31

Binding
def a = new Dog();
return a
}
}
}
}
a = animal;
}
}

32

BaseScript.groovy
def invokeMethod(String name, args) {
animal."$name"(*args)
}
def change(String animal) {
if (animal == "dog") {
binding.changeAnimal(new Dog())
} else if (animal == "cat") {
binding.changeAnimal(new Cat())
}
}
}

33

BaseScript.groovy
}
}
}
}

34

BaseScript.groovy
}
}
}
}

35

BaseScript.groovy
}
}
}
}

36

BaseScript.groovy
}
}
}
}

37

Results

The Script:
speak()
change(‘cat’)
speak()
change(‘dog’)
animal.speak()

Produces the Output:
woof
meow
woof

38

get/setProperty

• Properties are accessed via “object.propertyname”
• Properties can be either fields or accessors/mutators
– foo.bar can either be get/setBar, or the variable bar in foo
• Like invokeMethod, this behavior can be overridden
• Beware the “Java Field” operator - .@, which bypasses
the get/setProperty methods

39

Talk Agenda

• Introduction
• The AST

40

Compilation Customizers

• New in 1.8
• Replace and/or augment ASTTransformations
• Modify code as part of the compilation process
• For example:
– adding imports (via ImportCompilationCustomizer)
– disabling language features (via SecureASTCustomizer)
– automatically applying ASTTransforms (either local or global) (via
ASTTranformationCustomizer)

41

Adding Imports
import org.codehaus.groovy.control.customizers.ImportCustomizer;

public class JavaOne05Customizer {

ImportCustomizer customizer = new ImportCustomizer();
customizer.addStaticStars("java.lang.Math");
config.addCompilationCustomizers(customizer);
GroovyShell shell = new GroovyShell(config); Output:
Script script = shell.parse("println PI; println cos(1)"); 3.141592653589793
script.run(); 0.5403023058681398
}
}

42

Adding Imports


script.run(); 0.5403023058681398
}
}

43

Adding Imports


script.run(); 0.5403023058681398
}
}

44

Disallowing ++ Operations
Script script;
SecureASTCustomizer customizer = new SecureASTCustomizer();
List<Integer> tokens = new ArrayList<Integer>();
tokens.add(Types.PLUS_PLUS);
customizer.setTokensBlacklist(tokens);
try {
script = shell.parse("def i = 1 ; println i++");
} catch (MultipleCompilationErrorsException mcee) {
Throwable t = mcee.getErrorCollector().getException(0);
System.out.println(t.getClass().getName());
System.out.println(t.getMessage());
return; Output:
} java.lang.SecurityException
script.run(); Token ("++" at 1:22: "++" ) is not allowed

45

Script script;
try {
return; Output:

46

Script script;
try {
return; Output:

47

Script script;
try {
return; Output:

48

Forbidding for loops
Script script;
List<Class<? extends Statement>> statements = new ArrayList<Class<? extends Statement>>();
statements.add(ForStatement.class);
customizer.setStatementsBlacklist(statements);
try {
script = shell.parse("for (i in [1,2,3]) {}");
return;
Output:
script.run(); ForStatements are not allowed

49

Script script;
try {
return;
Output:

50

Script script;
try {
return;
Output:

51

Script script;
try {
return;
Output:

52

Script script;
try {
return;
Output:

53

ASTTransformations

• Used to modify generated code from the compiler
• Either global or local
– Globals are triggered by compile time configuration
– Locals are triggered by Annotations
• Groovy compiler generates an Abstract Syntax Tree
– breaks code into tree of Statements
• which in turn are combinations of Statements and/or Expressions
• You can examine code with groovyConsole to see the
tree via Script -> Inspect AST

54

Sample AST
Sample code:
def i = 1
println i

55

A Simple Use of AST - Checking Bind Variables

• An early problem we ran into
• Need to determine if binding variables are valid
– Most useful for read-only bindings
– Check for things like misspelling (so, “usrename” vs. “username”)
• Create a new CompilationCustomizer
• Visitor pattern means we visit every VariableExpression
• Test if it’s going against the supplied Binding

56

Scanning for Binding variables, part 1

class ScriptVisitingCustomizer extends CompilationCustomizer {
def visitor
ScriptVisitingCustomizer() {super(CompilePhase.SEMANTIC_ANALYSIS)}
void call(SourceUnit source, GeneratorContext context, ClassNode classNode) throws CompilationFailedException {
for (method in classNode.methods) {
if (method.name == 'run') method.code.visit(visitor)
}
}
}

class MyVisitor extends ClassCodeVisitorSupport {
void visitVariableExpression(VariableExpression expression) {
if (expression.accessedVariable instanceof DynamicVariable)
println expression.name
}
protected SourceUnit getSourceUnit() {return source}
}

57


def visitor
}
}
}

}
}

58


def visitor
}
}
}

}
}

59


def scriptText = """
println baz
println foo
def bar = 2
println bar
"""

def customizer = new ScriptVisitingCustomizer(visitor: new MyVisitor()); Output:
config.addCompilationCustomizers(customizer); baz
foo
Script script = shell.parse(scriptText);

60


def scriptText = """
println baz
println foo
def bar = 2
println bar
"""

def customizer = new ScriptVisitingCustomizer(visitor: new MyVisitor()); Output:
config.addCompilationCustomizers(customizer); baz
foo
Script script = shell.parse(scriptText);

61

ASTTransformations

• Numerous standard ASTTransforms are available
– ThreadInterrupt - add Thread.currentThread().isInterrupted
checks on loops, closures, first Statement in a Block
– TimedInterrupt - as above, with a timeout check
– ConditionalInterrupt - as above, with any closure
– EqualsAndHashCode - create equals() and hashCode() methods
– AutoClone - create clone() method
– Immutable - create constructors, accessors, make final

62

Creating an ASTTransformation

• Create an Annotation
• Create an ASTTransformation class
• (Optional, but likely) Create a ClassCodeVisitor
• To use:
– Decorate relevant object (method, class, etc) with annotation or
– Use ASTTransformationCustomizer in GroovyShell

63

NameChange.groovy

@Retention(RetentionPolicy.SOURCE)
@Target(ElementType.TYPE)
@GroovyASTTransformationClass("NameChangeASTTransformation")
public @interface NameChange {
}

64

NameChangeASTTransformation
@GroovyASTTransformation(phase = CompilePhase.SEMANTIC_ANALYSIS)
public class NameChangeASTTransformation implements ASTTransformation {
public void visit(ASTNode[] nodes, SourceUnit source) {
def visitor = new CustomClassVisitor();
if (nodes.length != 2) throw new RuntimeException("exception");
nodes[1].visitContents(visitor);
}
}
class CustomClassVisitor extends ClassCodeVisitorSupport {
protected SourceUnit getSourceUnit() {
throw new UnsupportedOperationException("Not supported yet.");
}
public void visitConstantExpression(ConstantExpression expression) {
if (expression.value == "Bob") {
expression.value = "Alice"
}
super.visitConstantExpression(expression);
}
}

65

}
}
}
}
}
}

66

}
}
}
}
}
}

67

}
}
}
}
}
}

68

Run with Annotation

String runMe = "@NameChange n"+
"class Name {n"+
" static def name = 'Bob'n"+
"} n" +
"println Name.name";

GroovyShell shell = new GroovyShell();
script.run();
Output:
Alice

69

Run with Annotation

String runMe = "class Name {n"+
"static def name = 'Bob'n"+
"} n" +
"println Name.name";

ASTTransformationCustomizer customizer = new ASTTransformationCustomizer(NameChange.class);

Output:
script.run(); Alice

70

Securing Your Code

• GroovyShell(ClassLoader,Binding,CompilerConfiguration)
• GroovyShell.parse(GroovyCodeSource)
– GroovyCodeSource(String script, String name, String codeBase)
• SecureASTCustomizer
– ArithmeticShell
– static analysis only!
• method/property wrapping

71

Last Note: Naming Matters!

This Code: Can take this as input:
class Book {
void name(String name) { Book.create {
println name name "DSLs in Action"
} loanedTo "Jim","Sint Si"
void quantity(int i) { quantity 1
println i }
}
void loanedTo(String[] names) {
println names
}
static create(closure) {
def book = new Book()
book.with closure
return book
}
} *example inspired by “DSLs in Action”*

72

Resources

• General Groovy Books
– Groovy Programming: An Introduction for Java Programmers
• (K&R style simple book)
– Programming Groovy
– Groovy in Action
• (2nd Ed coming soon, with DSL chapters)
• Domain Specific Languages
– Groovy for Domain-Specific Languages
• (great discussion of MOP, Builders)
– DSLs in Action
• (contains multiple language examples)
73

Q&A

74

75

Groovy DSLs (JavaOne Presentation)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Groovy DSLs (JavaOne Presentation)

Similar to Groovy DSLs (JavaOne Presentation) (20)

Recently uploaded

Recently uploaded (20)

Groovy DSLs (JavaOne Presentation)

Editor's Notes