3. Guillaume Laforge
• Groovy Project Manager at VMware
•Initiator of the Grails framework
•Creator of the Gaelyk
and Caelyf toolkits
• Co-author of Groovy in Action
• Follow me on...
•My blog: http://glaforge.appspot.com
•Twitter: @glaforge
•Google+: http://gplus.to/glaforge
3
5. Domain-Specific Languages
• Wikipedia definition
{ }
A Domain-Specific Language is a programming
language or executable specification language that
offers, through appropriate notations and
abstractions, expressive power focused on, and usually
restricted to, a particular problem domain.
• In contrast to General Purprose Languages
• Also known as: fluent / human interfaces, language oriented programming,
little or mini languages, macros, business natural languages...
5
6. Technical examples
Glade XSLT
<?xml version="1.0"?> <?xml version="1.0"?>
<GTK-Interface> <xsl:stylesheetversion="1.0"
<widget> xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<class>GtkWindow</class> <xsl:output method="xml"/>
<name>HelloWindow</name> <xsl:template match="*">
<border_width>5</border_width>
<Signal>
<xsl:element name="{name()}"> Regex
<xsl:for-each select="@*">
<name>destroy</name>
<handler>gtk_main_quit</handler>
<xsl:element name="{name()}">
<xsl:value-of select="."/>
"x.z?z{1,3}y"
</Signal>
<title>Hello</title> </xsl:element>
<type>GTK_WINDOW_TOPLEVEL</type> </xsl:for-each>
<position>GTK_WIN_POS_NONE</position> <xsl:apply-templates select="*|text()"/>
<allow_shrink>True</allow_shrink> </xsl:element>
<allow_grow>True</allow_grow> </xsl:template>
<auto_shrink>False</auto_shrink> </xsl:stylesheet>
<widget>
<class>GtkButton</class>
<name>Hello World</name>
<can_focus>True</can_focus>
fetchmail
<label>Hello World</label>
# Poll this site first each cycle.
</widget> poll pop.provider.net proto pop3
</widget> user "jsmith" with pass "secret1" is "smith" here
</GTK-Interface> user jones with pass "secret2" is "jjones" here with options keep
SQL # Poll this site second, unless Lord Voldemort zaps us first.
poll billywig.hogwarts.com with proto imap:
user harry_potter with pass "floo" is harry_potter here
SELECT * FROM TABLE
# Poll this site third in the cycle.
WHERE NAME LIKE '%SMI' # Password will be fetched from ~/.netrc
ORDER BY NAME poll mailhost.net with proto imap:
user esr is esr here
8. Goals of DSLs
• Use a more expressive language than a general-purpose one
• Share a common metaphor of understanding
between developers and subject matter experts
• Have domain experts help with the design
of the business logic of an application
• Avoid cluttering business code with too much boilerplate technical code
thanks to a clean separation
• Let business rules have their own lifecycle
8
9. Pros and cons
Pros Cons
• Domain experts can help, validate, • Learning cost vs. limited applicability
modify, and often develop DSL
programs
• Cost of designing, implementing &
maintaining DSLs as well as tools/IDEs
• Somewhat self-documenting
• Attaining proper scope
• Enhance quality, productivity,
• Trade-offs between domain specificity
reliability, maintainability, portability, and general purpose language
reusability constructs
• Safety; as long as the language
• Efficiency cost
constructs are safe, any DSL
sentence can be considered safe • Proliferation of similar
non-standard DSLs
9
10. Groovy provides...
• A flexible and malleable syntax
•scripts, native syntax constructs (list, map, ranges), closures, less punctuation...
• Compile-time and runtime meta-programming
•metaclasses, AST transformations
•also operator overloading
• The ability to easily integrate into Java / Spring apps
•also security and safety
10
17. More explicit direction
class!Robot!{
!!!!void)move(Direction!dir)!{}
}
enum!Direction!{
!!!!left,!right,!forward,!backward
}
17
18. Now how can we control it?
import)static)mars.Direction.*;
import)mars.Robot;
public)class)Command!{
))))public)static)void)main(String[]!args)!{
!!!!!!!!Robot!robot!=)new!Robot();
!!!!!!!!robot.move(left);
!!!!}
}
18
19. Now how can we control it?
import)static)mars.Direction.*;
import)mars.Robot;
public)class)Command!{
))))public)static)void)main(String[]!args)!{
!!!!!!!!Robot!robot!=)new!Robot();
!!!!!!!!robot.move(left);
!!!!}
} Syntactical
noise!
18
20. Now how can we control it?
))))))))))))))))))))))))))))))—
import)static)mars.Direction.*;
)))))))))))))))))—
import)mars.Robot;
——————————————————————
public)class)Command!{
))))————————————————————————————————————————
))))public)static)void)main(String[]!args)!{
))))))))—————))))))))))))))))))))—
!!!!!!!!Robot!robot!=)new!Robot();
))))))))))))))))))—))))——
!!!!!!!!robot.move(left);
))))—
!!!!}
—
} Syntactical
noise!
18
23. Scripts vs Classes
Optional semicolons & parens
import)static)mars.Direction.*
import)mars.Robot
Optional typing
!!!!!!!!def))!robot!=)new!Robot()
!!!!!!!!robot.move!left
But I don’t want to
compile a script for
every command!
19
26. GroovyShell to the rescue
def$shell%=$new%GroovyShell()
shell.evaluate(
%%%%new%File("command.groovy")
)
21
27. GroovyShell to the rescue
def$shell%=$new%GroovyShell()
shell.evaluate(
integration.groovy %%%%new%File("command.groovy")
)
21
28. GroovyShell to the rescue
def$shell%=$new%GroovyShell()
shell.evaluate(
integration.groovy %%%%new%File("command.groovy")
)
import$static$mars.Direction.*
import$mars.Robot
def$robot%=$new%Robot()
robot.move%left
21
29. GroovyShell to the rescue
def$shell%=$new%GroovyShell()
shell.evaluate(
integration.groovy %%%%new%File("command.groovy")
)
import$static$mars.Direction.*
import$mars.Robot
command.groovy
def$robot%=$new%Robot()
robot.move%left
21
30. Integration mechanisms
• Different solutions available:
•Groovy’s own mechanisms
• GroovyScriptEngine, Eval,
GroovyClassLoader, GroovyShell
•Java 6: javax.script.* / JSR-223
•
Groovy provides a JSR-223 implementation
•Spring’s lang namespace
• Groovy provides the highest level of flexibility
and customization, but JSR-223 is a standard...
22
31. Integration mechanisms
• Different solutions available:
•Groovy’s own mechanisms
• GroovyScriptEngine, Eval,
GroovyClassLoader, GroovyShell
•Java 6: javax.script.* / JSR-223
•
Groovy provides a JSR-223 implementation
•Spring’s lang namespace
• Groovy provides the highest level of flexibility
and customization, but JSR-223 is a standard...
22
33. What’s wrong with our DSL?
Can’t we hide
those imports?
import)static)mars.Direction.*
import)mars.Robot
def)robot!=)new!Robot()
robot.move!left
23
34. What’s wrong with our DSL?
Can’t we hide
those imports?
import)static)mars.Direction.*
import)mars.Robot
def)robot!=)new!Robot()
robot.move!left
Can’t we inject
the robot?
23
35. What’s wrong with our DSL?
Can’t we hide
those imports?
import)static)mars.Direction.*
import)mars.Robot
def)robot!=)new!Robot()
robot.move!left
Can’t we inject
Do we really need to the robot?
repeat ‘robot’?
23
39. Let’s inject a robot!
• We can pass data in / out of scripts through the Binding
•it’s basically just like a map of variable name keys and their associated values
26
40. Let’s inject a robot!
• We can pass data in / out of scripts through the Binding
•it’s basically just like a map of variable name keys and their associated values
def)binding!=)new!Binding([
!!!!robot:!new!Robot()
])
def)shell!=)new!GroovyShell(binding)
shell.evaluate(
!!!!new!File("command.groovy")
)
26
41. Let’s inject a robot!
• We can pass data in / out of scripts through the Binding
•it’s basically just like a map of variable name keys and their associated values
integration.groovy
def)binding!=)new!Binding([
!!!!robot:!new!Robot()
])
def)shell!=)new!GroovyShell(binding)
shell.evaluate(
!!!!new!File("command.groovy")
)
26
45. How to inject the direction?
• Using the binding...
def)binding!=)new!Binding([
!!!!robot:!new!Robot(),
!!!!left:!!!!!Direction.left,
!!!!right:!!!!Direction.right,
!!!!backward:!Direction.backward,
!!!!forward:!!Direction.forward
])
def)shell!=)new!GroovyShell(binding)
shell.evaluate(
!!!!new!File("command.groovy")
)
28
46. How to inject the direction?
Fragile in case of
new directions!
• Using the binding...
def)binding!=)new!Binding([
!!!!robot:!new!Robot(),
!!!!left:!!!!!Direction.left,
!!!!right:!!!!Direction.right,
!!!!backward:!Direction.backward,
!!!!forward:!!Direction.forward
])
def)shell!=)new!GroovyShell(binding)
shell.evaluate(
!!!!new!File("command.groovy")
)
28
47. How to inject the direction?
• Using the binding...
def)binding!=)new!Binding([
!!!!robot:!new!Robot(),
!!!!*:!Direction.values()
!!!!!!!!!!!!.collectEntries!{
!!!!!!!!!!!!!!!![(it.name()):!it]
!!!!!!!!!!!!}
])
def)shell!=)new!GroovyShell(binding)
shell.evaluate(
!!!!new!File("command.groovy")
)
29
48. How to inject the direction?
• Using compiler customizers...
• Let’s have a look at them
30
49. Compilation customizers
• Ability to apply some customization
to the Groovy compilation process
• Three available customizers
•ImportCustomizer: add transparent imports
•ASTTransformationCustomizer: injects an AST transform
•SecureASTCustomizer: restrict the groovy language to an allowed subset
• But you can implement your own
31
57. Secure AST customizer
Idea: Secure the rocket’s onboard trajectory
calculation system by allowing only math
expressions to be evaluated by the calculator
• Let’s setup our environment
•an import customizer to import java.lang.Math.*
•prepare a secure AST customizer
def%imports%=%new%ImportCustomizer().addStaticStars('java.lang.Math')
def%secure%=%new%SecureASTCustomizer()
36
62. Secure AST customizer You can build a subset of
the Groovy syntax!
...
//%language%tokens%allowed
tokensWhitelist%=%[
PLUS,%MINUS,%MULTIPLY,%DIVIDE,%MOD,%POWER,%PLUS_PLUS,%MINUS_MINUS,%
COMPARE_EQUAL,%COMPARE_NOT_EQUAL,%COMPARE_LESS_THAN,%COMPARE_LESS_THAN_EQUAL,%
COMPARE_GREATER_THAN,%COMPARE_GREATER_THAN_EQUAL
]
%
//%types%allowed%to%be%used%(including%primitive%types)
constantTypesClassesWhiteList%=%[
Integer,%Float,%Long,%Double,%BigDecimal,%
Integer.TYPE,%Long.TYPE,%Float.TYPE,%Double.TYPE
]
%
//%classes%who%are%allowed%to%be%receivers%of%method%calls
receiversClassesWhiteList%=%[%
Math,%Integer,%Float,%Double,%Long,%BigDecimal%]
}
...
38
63. Secure AST customizer You can build a subset of
the Groovy syntax!
...
//%language%tokens%allowed
tokensWhitelist%=%[
PLUS,%MINUS,%MULTIPLY,%DIVIDE,%MOD,%POWER,%PLUS_PLUS,%MINUS_MINUS,%
COMPARE_EQUAL,%COMPARE_NOT_EQUAL,%COMPARE_LESS_THAN,%COMPARE_LESS_THAN_EQUAL,%
COMPARE_GREATER_THAN,%COMPARE_GREATER_THAN_EQUAL
]
%
//%types%allowed%to%be%used%(including%primitive%types)
Black / white list
constantTypesClassesWhiteList%=%[ usage of classes
Integer,%Float,%Long,%Double,%BigDecimal,%
Integer.TYPE,%Long.TYPE,%Float.TYPE,%Double.TYPE
]
%
//%classes%who%are%allowed%to%be%receivers%of%method%calls
receiversClassesWhiteList%=%[%
Math,%Integer,%Float,%Double,%Long,%BigDecimal%]
}
...
38
64. Secure AST customizer
• Ready to evaluate our flight equations!
def%config%=%new%CompilerConfiguration()
config.addCompilationCustomizers(imports,%secure)
def%shell%=%new%GroovyShell(config)
%
shell.evaluate%'cos%PI/3'
• But the following would have failed:
shell.evaluate%'System.exit(0)'
39
69. How to remove the ‘robot’?
• Instead of calling the move() method on the robot
instance, we should be able to call the move() method
directly from within the script
• Two approaches:
• Inject a ‘move’ closure in the • Use a base script class with a
‘move’ method delegating to the
binding with a method pointer
robot
42
70. Inject a closure in the binding
def$robot%=%new%Robot()
binding%=$new%Binding([
%%%%robot:%robot,
%%%%*:%Direction.values()
%%%%%%%%%%%%.collectEntries%{
%%%%%%%%%%%%%%%%[(it.name()):%it]
%%%%%%%%%%%%},
%%%%move:%robot.&move
])
43
71. Inject a closure in the binding
def$robot%=%new%Robot()
binding%=$new%Binding([
%%%%robot:%robot,
%%%%*:%Direction.values()
%%%%%%%%%%%%.collectEntries%{
%%%%%%%%%%%%%%%%[(it.name()):%it]
%%%%%%%%%%%%},
%%%%move:%robot.&move
])
Method pointer (a closure) on
robot’s move instance method
43
72. Define a base script class
abstract)class!RobotBaseScriptClass!!!
!!!!!!!extends!Script!{
!!!!void!move(Direction!dir)!{
!!!!!!!!def)robot!=!this.binding.robot
!!!!!!!!robot.move!dir
!!!!}
}
44
73. Define a base script class
abstract)class!RobotBaseScriptClass!!!
!!!!!!!extends!Script!{
!!!!void!move(Direction!dir)!{
!!!!!!!!def)robot!=!this.binding.robot
!!!!!!!!robot.move!dir
!!!!}
}
The move() method is
now at the script level
44
74. Define a base script class
abstract)class!RobotBaseScriptClass!!!
!!!!!!!extends!Script!{
!!!!void!move(Direction!dir)!{
!!!!!!!!def)robot!=!this.binding.robot
!!!!!!!!robot.move!dir
!!!!}
}
The move() method is Access the robot through
now at the script level the script’s binding
44
75. Configure the base script class
def)conf)=)new)CompilerConfiguration()
conf.scriptBaseClass)=)RobotBaseScriptClass
45
76. Configure the base script class
def)conf)=)new)CompilerConfiguration()
conf.scriptBaseClass)=)RobotBaseScriptClass
Scripts evaluated with
this configuration will
inherit from that class
45
81. What we could do now is...
Mix of named and
normal parameters
move!left,!at:!3.km/h
49
82. What we could do now is...
Mix of named and
normal parameters
move!left,!at:!3.km/h
How to add a km
property to numbers?
49
83. Adding properties to numbers
• We need to:
• define units, distance and speed
• have a nice notation for them
• that’s where we add properties to numbers!
50
84. Unit enum and Distance class
enum$DistanceUnit%{
%%%%centimeter%('cm',%%%%0.01),
%%%%meter%%%%%%(%'m',%%%%1),%
%%%%kilometer%%('km',%1000)%
%%%%
%%%%String%abbreviation
%%%%double%multiplier
%%%%
%%%%Unit(String%abbr,%double%mult)%{
%%%%%%%%this.abbreviation%=%abbr
%%%%%%%%this.multiplier%=%mult%
%%%%}
%%%%String%toString()%{%abbreviation%}%
}
51
85. Unit enum and Distance class
@TupleConstructor%
enum$DistanceUnit%{ class%Distance%{
%%%%centimeter%('cm',%%%%0.01), %%%%double%amount%
%%%%meter%%%%%%(%'m',%%%%1),% %%%%DistanceUnit%unit
%%%%kilometer%%('km',%1000)%
%%%% %%%%String%toString()%{%
%%%%String%abbreviation %%%%%%%%"$amount%$unit"%
%%%%double%multiplier %%%%}%
%%%% }
%%%%Unit(String%abbr,%double%mult)%{
%%%%%%%%this.abbreviation%=%abbr
%%%%%%%%this.multiplier%=%mult%
%%%%}
%%%%String%toString()%{%abbreviation%}%
}
51
86. Different techniques
• To add dynamic methods or properties,
there are several approaches at your disposal:
• ExpandoMetaClass
• custom MetaClass
• Categories
• Let’s have a look at the ExpandoMetaClass
52
88. Using ExpandoMetaClass
Add that to
integration.groovy
Number.metaClass.getCm%=%{%f>%
%%%%new%Distance(delegate,%Unit.centimeter)%
}
Number.metaClass.getM%=%{%f>%
%%%%new%Distance(delegate,%Unit.meter)%
}
Number.metaClass.getKm%=%{%f>%
%%%%new%Distance(delegate,%Unit.kilometer)%
}
53
89. Using ExpandoMetaClass
Add that to
integration.groovy
Number.metaClass.getCm%=%{%f>%
%%%%new%Distance(delegate,%Unit.centimeter)%
}
Number.metaClass.getM%=%{%f>%
%%%%new%Distance(delegate,%Unit.meter)%
}
Number.metaClass.getKm%=%{%f>%
%%%%new%Distance(delegate,%Unit.kilometer)%
}
‘delegate’ is the
current number
53
90. Using ExpandoMetaClass
Add that to
integration.groovy Usage in
your DSLs
Number.metaClass.getCm%=%{%f>%
%%%%new%Distance(delegate,%Unit.centimeter)%
} 40.cm!
Number.metaClass.getM%=%{%f>%
%%%%new%Distance(delegate,%Unit.meter)%
3.5.m
} 4.km
Number.metaClass.getKm%=%{%f>%
%%%%new%Distance(delegate,%Unit.kilometer)%
}
‘delegate’ is the
current number
53
91. Distance okay, but speed?
• For distance, we just added a property access after the number,
but we now need to divide (‘div’) by the time
2.km/h
54
92. Distance okay, but speed?
• For distance, we just added a property access after the number,
but we now need to divide (‘div’) by the time
The div() method
on Distance
2.km/h
54
93. Distance okay, but speed?
• For distance, we just added a property access after the number,
but we now need to divide (‘div’) by the time
The div() method
on Distance
2.km/h
An h duration
instance in the binding
54
99. Operator overloading
• Update the Distance class with a div() method
following the naming convetion for operators
class%Distance%{
%%%%...
%%%%Speed%div(Duration%t)%{
%%%%%%%%new%Speed(this,%t)
%%%%}
%%%%...
}
59
100. Operator overloading
• Update the Distance class with a div() method
following the naming convetion for operators
class%Distance%{
%%%%...
%%%%Speed%div(Duration%t)%{
%%%%%%%%new%Speed(this,%t)
%%%%}
%%%%...
} Optional return
59
106. Named parameters usage
move!left,!at:!3.km/h
Normal Named
parameter parameter
Will call:
def!take(Map!m,!Direction!q)
61
107. Named parameters usage
move!left,!at:!3.km/h
Normal Named
parameter parameter
Will call:
def!take(Map!m,!Direction!q)
All named parameters go
into the map argument
61
108. Named parameters usage
move!left,!at:!3.km/h
Normal Named
parameter parameter
Will call:
def!take(Map!m,!Direction!q)
All named parameters go Positional parameters
into the map argument come afterwards
61
111. Named parameters usage
move!left,!at:!3.km/h
Can we get rid of What about the
the comma? colon too?
62
112. Command chains
• A grammar improvement in Groovy 1.8 allowing you
to drop dots & parens when chaining method calls
• an extended version of top-level statements like println
• Less dots, less parens allow you to
•write more readable business rules
•in almost plain English sentences
•
(or any language, of course)
63
146. Playing it safe...
• You have to think carefully about
what DSL users are allowed to do with your DSL
• Forbid things which are not allowed
•leverage the JVM’s Security Managers
•
this might have an impact on performance
•use a Secure AST compilation customizer
•
not so easy to think about all possible cases
•avoid long running scripts with *Interrupt transformations
74
147. Security Managers
• Groovy is just a language leaving on the JVM,
so you have access to the usual Security Managers mechanism
• Nothing Groovy specific here
• Please check the documentation on Security Managers
and how to design policy files
75
149. Controlling code execution
• Your application may run user’s code
•what if the code runs in infinite loops or for too long?
•what if the code consumes too many resources?
• 3 new transforms at your rescue
• @ThreadInterrupt: adds Thread#isInterrupted checks
so your executing thread stops when interrupted
• @TimedInterrupt: adds checks in method and closure bodies
to verify it’s run longer than expected
• @ConditionalInterrupt: adds checks with your own
conditional logic to break out from the user code
77
152. @TimedInterrupt
@TimedInterrupt(10)
import!groovy.transform.TimedInterrupt!
!
while!(true)!{
!!!!move!left
!!!!//!circle!forever
}
• InterruptedException thrown
when checks indicate code ran longer than desired
79
153. @ConditionalInterrupt
• Specify your own condition to be inserted
at the start of method and closure bodies
• check for available resources, number of times run, etc.
• Leverages closure annotation parameters from Groovy 1.8
@ConditionalInterrupt({$battery.level$<$O.1$})
import%groovy.transform.ConditionalInterrupt
100.times%{%%%%
%%%%move%forward%at%10.km/h
}
80
154. Using compilation customizers
• In our previous three examples, the usage of the interrupts were explicit,
and users had to type them
• if they want to deplete the battery of your robot,
they won’t use interrupts, so you have to impose interrupts yourself
• With compilation customizers you can inject those interrupts
thanks to the ASTTransformationCustomizer
81
156. Groovy Power!™
• A flexible and malleable syntax
• scripts vs classes, optional typing, colons and parens
• Groovy offers useful dynamic features for DSLs
• operator overloading, ExpandoMetaClass
• Can write almost plain natural language sentences
• for readable, concise and expressive DSLs
• Groovy DSLs are easy to integrate,
and can be secured to run safely in your own sandbox
83
157. Groovy Power!™ Groovy is a
great fit for
DSLs!
• A flexible and malleable syntax
• scripts vs classes, optional typing, colons and parens
• Groovy offers useful dynamic features for DSLs
• operator overloading, ExpandoMetaClass
• Can write almost plain natural language sentences
• for readable, concise and expressive DSLs
• Groovy DSLs are easy to integrate,
and can be secured to run safely in your own sandbox
83
158. And there’s more!
• We haven’t dived into...
•How to implement your own control structures with the help of closures
•How to create Groovy « builders »
•How to hijack the Groovy syntax to develop our own language extensions with
AST Transformations
•Source preprocessing for custom syntax
•How to use the other dynamic metaprogramming techniques available
•How to improve error reporting with customizers
•IDE support with DSL descriptors (GDSL and DSLD)
84
159. Thank you!
ge
e Lafor lopment
Gui llaum ovy Deve
Head of Gro
om
@ gmail.c
laforge rge
Email: g @glafo o/glaforg
e
Twitter : http://gplus.t spot.com
:
G oogle+ //glaforge.app
p:
B log: htt
85