Vert.x using Groovy - Simplifying non-blocking code

Alexander (Sascha) Klein
codecentric AG
vert.x with Groovy
Simplifying non-blocking code

codecentric AG
Alexander Klein, 2014-06-03
vert.x with Groovy – Simpliyfing non-blocking code

codecentric AG
Why using vert.x ?
CC BY 2.0 > http://www.flickr.com/photos/girliemac/6509400997

codecentric AG
Principal Consultant
codecentric AG in Stuttgart
Germany
Groovy, JavaFX, UI / UX
Griffon committer
alexander.klein@codecentric.de
@saschaklein
http://gplus.to/karfunkel

codecentric AG
vert.x
Framework to write polyglot, highly concurrent applications
Similar to Node.js
Asynchronous, non-blocking API
Polyglot (Java, JavaScript, Groovy, Ruby, Python and others)

codecentric AG
Architecture
Client Background
Threadpool
Worker-Verticle
Worker-Verticle
Worker-Verticle
Event Loop
Verticle
Verticle
Verticle
Event Bus
Request
Response
delegating
long-running tasks
non-blocking blocking

codecentric AG
Yoke
Middleware framework for vert.x
Currently only Java, JavaScript and Groovy supported
Many helpful implementations
Request body and Cookie parser
Static file server
Request Router
Virtual host support
Templateengines
and more ...

codecentric AG
Calculating CRC32's for a directory
Read directory entries
Read file properties for each entry
Determine if entry is a directory
Handle directories recursively
Read file
Calculate CRC32 via worker verticle

codecentric AG
Classic vert.x/yoke code
container.deployWorkerVerticle 'CRC.groovy', [:]
GRouter router = new GRouter()
router.get("/crc") { GYokeRequest request ->
request.response.chunked = true
request.response.contentType = 'text/plain'
this.crc('/home/aklein/tmp/ConfigParser', request)
}
router.get("/") { GYokeRequest request, Handler next ->
request.response.render 'web/index.gsp', next
}

codecentric AG
def yoke = new GYoke(vertx, container)
yoke.engine new GroovyTemplateEngine()
yoke.use(router)
yoke.use new Static("web", 24 * 60 * 60 * 1000, true, false)
yoke.use { request ->
request.response.statusCode = 404
request.response.statusMessage = 'Not Found'
request.response.contentType = 'text/plain'
request.response.end('404 - Not Found')
}
yoke.listen(8080)

codecentric AG
def crc(String baseDir, GYokeRequest request) {
EventBus bus = vertx.eventBus
FileSystem fs = vertx.fileSystem
fs.readDir(baseDir) { AsyncResult<String[]> rs ->
if (rs.succeeded) {
String[] paths = rs.result
paths.each { String path ->
fs.props(path) { AsyncResult<FileProps> rs1 ->
if (rs1.succeeded) {
FileProps props = rs1.result
if (props.directory) {
crc(path, request)
} else {
fs.readFile(path) { AsyncResult<Buffer> rs2 ->
Buffer content = rs2.result
bus.send("create.crc", content) { Message result ->
if (result.body().status == 'error') {
request.response.statusMessage = "Error processing file " +
"$path: ${result.body().message}: ${result.body().error} n" +
"${result.body().stacktrace}"
request.response.end()
} else {
request.response.write "$path = ${result.body().message}n"
}
}
} else {
request.response.statusMessage = "Failed to read file $path"
}
}
}
} else {
request.response.statusMessage = "Failed to read properties for $path"
}
}
}
} else {
request.response.statusMessage = "Failed to read $baseDir"
}
}
}

codecentric AG
Preparing gradle build
Download from: http://github.com/vert-x/vertx-gradle-template
build.gradle
provided "com.jetdrone:yoke:$yokeVersion@jar" // (optional for using yoke)
gradle.properties
groovyVersion=2.2.1
yokeVersion=1.0.13 // (optional for using yoke)

codecentric AG
Preparing gradle build
gradle/vertx.gradle
task startMod(dependsOn: copyMod, description: 'Run the module', type: JavaExec) {
classpath = sourceSets.main.compileClasspath + sourceSets.main.runtimeClasspath
main = 'org.vertx.java.platform.impl.cli.Starter'
args(['runmod', moduleName])
args runModArgs.split("s+")
// jvmArgs "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
systemProperties([
"vertx.clusterManagerFactory": "org.vertx.java.spi.cluster.impl.hazelcast.HazelcastClusterManagerFactory",
"vertx.mods" : "$projectDir/build/mods"
])
}

codecentric AG
Compress Errorhandling - Method
request.response.statusMessage = "Failed to read file $path"
------------------------------------------------------------------------------------------------------------------------------------------------------------------
def end(YokeResponse response, int statusCode, String statusMessage = null) {
response.statusCode = statusCode
if(statusMessage)
response.statusMessage = statusMessage
response.end()
}
------------------------------------------------------------------------------------------------------------------------------------------------------------------
end request.response, 500, "Failed to read file $path"

codecentric AG
Compress Errorhandling - Dynamic Mixins
class YokeExtension {
static String end(YokeResponse self, Integer statusCode, String statusMessage = null) {
self.statusCode = statusCode
if (statusMessage)
self.statusMessage = statusMessage
self.end()
}
}
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
YokeResponse.mixin(YokeExtension)
request.response.end 500, "Failed to read file $path"

codecentric AG
Compress Errorhandling - Static Mixins (vert.x 2.1)
...
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
compilerConfiguration.groovy:
customizer = { org.codehaus.groovy.control.CompilerConfiguration config ->
config.addCompilationCustomizers(
new ASTTransformationCustomizer(Mixin, value: YokeExtension) )
return config
}
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

codecentric AG
Compress Errorhandling - Module Extension (vert.x 2.1)
...
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
META-INF/services/org.codehaus.groovy.runtime.ExtensionModule:
moduleName = vertx-module
moduleVersion = 1.0
extensionClasses = de.codecentric.vertx.YokeExtension
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
build.gradle:
repositories {
mavenLocal()
}
dependencies {
compile "de.codecentric:vertx-extension:1.0.0-SNAPSHOT@jar"
}

codecentric AG
After YokeResponse enhancement
if (rs.succeeded) {
crc(path, request)
} else {
request.response.end 500, "Error processing file " +
} else
}
} else
}
}
} else
request.response.end 500, "Failed to read properties for $path"
}
}
} else
request.response.end 500, "Failed to read $baseDir“
}
}

codecentric AG
Bus communication
request.response.end 500, "Error processing file " +
} else
}
} else

codecentric AG
Worker Module
bus.registerHandler('create.crc') { Message msg ->
try {
Buffer buffer = new Buffer(msg.body())
CRC32 crc = new CRC32()
int start = 0, end, length = buffer.length
while (start < length) {
end = Math.min(start + 1024, length)
crc.update(buffer.getBytes(start, end))
start = end
}
msg.reply([status: 'ok', message: crc.value ])
} catch (e) {
StringWriter sw = new StringWriter()
e.printStackTrace(sw.newPrintWriter())
msg.reply([status: 'error', message: 'Failure creating crc', error: e.message, stacktrace: sw.toString()])
}
}

codecentric AG
Standardizing bus communication – Worker
try { ...
msg.reply([status: 'ok', message: crc.value ])
} catch (e) {
msg.reply([status: 'error', message: 'Failure creating crc', error: e.message, stacktrace: sw.toString()])
}
}
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
try { ...
msg.replySuccess(crc.value)
} catch (e) {
msg.replyFailure('Failure creating crc', e)
}
}

codecentric AG
Standardizing bus communication - Module
class MessageExtension {
static final String OK = 'ok'
static final String ERROR = 'error'
static void replySuccess(Message self, message) {
self.reply([status: OK, message: message])
}
static void replyFailure(Message self, Throwable e) {
replyFailure(self, null, e)
}
static void replyFailure(Message self, String msg,
Throwable e = null) {
def message = [status: ERROR]
if (msg)
message.message = msg
if (e) {
message.error = e.message
message.stacktrace = sw.toString()
}
self.reply(message)
}

codecentric AG
static String getStacktrace(Message self) {
self.body().stacktrace
}
static String getError(Message self) {
self.body().error
}
static def getMessage(Message self) {
return self.body().message
}
...

codecentric AG
Standardizing bus communication – Caller
if (result.body().status == 'error')
request.response.end 500,
"Error processing file $path: ${result.body().message}: ${result.body().error} n $result.body().stacktrace}"
else
}
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
if (result)
request.response.write "$path = ${result.message}n"
else
request.response.end 500, "Error processing file $path: $result.logMessage"
}

codecentric AG
static boolean isSucceeded(Message self) {
def result = self.body()
if (result instanceof Map) {
return result.status == OK
} else
return false
}
static boolean asBoolean(Message self) {
return self.isSucceeded()
}
static String getLogMessage(Message self) {
return self.getError() ? "${self.getMessage()}: ${self.getError()} n${self.getStacktrace()}" : self.getMessage()
}
...

codecentric AG
Streamlining AsyncResult API
static boolean asBoolean(AsyncResult self) {
return self.isSucceeded()
}
static String getStacktrace(AsyncResult self) {
if (!self.cause)
return ''
PrintWriter pw = sw.newPrintWriter()
self.cause.printStackTrace(pw)
return sw.toString()
}
static String getError(AsyncResult self) {
return self.cause ? self.cause.message : ''
}
static def getMessage(AsyncResult self) {
return self.result
}
static String getLogMessage(AsyncResult self) {
return self.getError() ? self.getMessage() +
": ${self.getError()} n${self.getStacktrace()}" :
self.getMessage()
}

codecentric AG
With standardized bus communication
if (rs) {
if (rs1) {
crc(path, request)
} else {
if (rs2) {
if (result) {
} else
request.response.end 500, "Error processing file $path:" +
result.logMessage
} else
}
}
} else
request.response.end 500, "Failed to read properties for $path"
}
}
} else
request.response.end 500, "Failed to read $baseDir“
}
}

codecentric AG
Handler chains
Event-based programming often results in multiple, stacked Handlers / Closures
Difficult to read
Order of commands from left to right / outside to inside
Horizontal scrolling because of indentation
Hard to find the begining of a logical part
Difficult to test
Loops are difficult or impossible to implement
When is the for loop finished to send the .end()?

codecentric AG
Closure Chaining - Syntax
chain { next -> next() }, { next -> next(10) }, { input, next -> println input }
chain ( 10, { input, next -> next(input) }, { input, next -> println input } )
chain 10, { input, next -> next(input) }, { input, next -> println input }
------------------------------------------------------------------------------------------------------------------------------------------------------------------
chain { next -> next() } { next -> next(10) } { input, next -> println input }
chain (10) { input, next -> next(input) } { input, next -> println input }
chain (10) { input, next ->
next(input)
} {
input, next -> println input
}

codecentric AG
Closure Chaining – Module
class StructureExtension {
static void chain(final Object self, def arguments, Closure... actions) {
if (arguments instanceof Closure) {
actions = [arguments, *actions] as Closure[]
arguments = null
}
if (!actions)
throw new IllegalArgumentException("One or more arguments of type groovy.lang.Closure required")
_chain(arguments, actions.iterator())
}
...
chain{} …
chain(arg)
chain(arg) {} ...

codecentric AG
static void chain(final Object self, Object... arguments) {
if (!arguments.any { it instanceof Closure })
throw new IllegalArgumentException("One or more arguments of type groovy.lang.Closure required")
int i; def actions = []
for (i = arguments.size() - 1; i >= 0; i--) {
if (arguments[i] instanceof Closure)
actions.add(0, arguments[i])
else
break
}
_chain(arguments[0..i], actions.iterator())
}
...
chain()
chain(arg1, arg2, ...)
chain(arg1, arg2, ...) {} …

codecentric AG
private static void _chain(final Object arguments, final Iterator<Closure> actions) {
if (actions) {
def action = actions.next()
if (arguments != null) {
action = action.curry(arguments as Object[])
}
action.call { Object[] args ->
_chain(args, actions)
}
}
}
...

codecentric AG
Looping - Syntax
[1,2,3].loop { element, next -> next() }
[a:1, b:2, c:3].loop { key, value, next -> next() }
[1,2,3].loop { element, next ->
next()
} {
// called after the last iteration
}

codecentric AG
Looping – Module
static void loop(final Object[] array, final Closure action) { loop(array, action, {} }
static void loop(final Object[] array, final Closure action, final Closure next) { _loop(array?.iterator(), action, next) }
static void loop(final Collection collection, final Closure action) { loop(collection, action, {} }
static void loop(final Collection collection, final Closure action, final Closure next) {
_loop(collection.iterator(), action, next)
}
static void loop(final Map map, final Closure action) { loop(map, action, {} }
static void loop(final Map map, final Closure action, final Closure next) { _loop(map.iterator(), action, next) }
...

codecentric AG
Looping – Module
private static void _loop(final Iterator<?> iterator, final Closure action, Closure next = {}) {
if(iterator) {
def element = iterator.next()
def nextAction
if (iterator)
nextAction = StructureExtension.&_loop.curry(iterator, action, next)
else
nextAction = next
if (element instanceof Map.Entry)
action.call(element.key, element.value, nextAction)
else
action.call(element, nextAction)
} else next.call()
}

codecentric AG
With chaining and looping
def crc(String baseDir, GYokeRequest request, Closure nextCrc = null) {
chain { nextChain -> // Read directory
if (rs) nextChain(rs.result as List)
else request.response.end 500, "Failed to read $baseDir"
}
} { List paths, nextChain -> // Loop over files
paths.loop { String path, nextLoop ->
chain { next -> // Read file properties
fs.props(path) { AsyncResult<FileProps> rs ->
if (rs) next(rs.result)
else request.response.end 500, "Failed to read properties for $path"
}
} { FileProps props, next -> // Check for directory
if (props.directory) crc(path, request, nextLoop)
else next()
}
{ next -> // Read file
fs.readFile(path) { AsyncResult<Buffer> rs ->
if (rs) next(rs.result)
else request.response.end 500, "Failed to read file $path"
}
} { Buffer content, next -> // Call module to calculate crc
if (result) {
nextLoop()
} else request.response.end 500, "Error processing file $path"
}
}
}
}
}

codecentric AG
Adding .end() after the loop
{ Buffer content, next -> // Call module to calculate crc
Vertx.eventBus.send("create.crc", content) { Message result ->
if (result) {
nextLoop()
}
}
} { // finish everything up after loop
if (nextCrc) nextCrc()
else request.response.end()
}
}
}

codecentric AG
Using a template engine
router.get("/crc") { GYokeRequest request ->
request.context.files = [:]
...
def crc(String baseDir, GYokeRequest request, Closure nextCrc = null) {
request.context.files[baseDir] = null
...
{ Buffer content, next -> // Call module to calculate crc
Vertx.eventBus.send("create.crc", content) { Message result ->
if (result) {
request.context.files[path] = result.message
nextLoop()
}
}
}
...
} { // finish everything up after loop
if (nextCrc) nextCrc()
else
request.response.render('web/crc.gsp',
files: request.context.files)
}
}
}

codecentric AG
Accessing the context - !!! Hack Alert !!!
...
static Context getContext(YokeRequest self) {
// Reflection because context is a private field of the super class for GYokeRequest
Field field = YokeRequest.getDeclaredField('context')
field.accessible = true
return (Context) field.get(self)
}
static Context getContext(YokeResponse self) {
// Reflection because context is a private field of the super class for GYokeResponse
Field field = YokeResponse.getDeclaredField('context')
field.accessible = true
return (Context) field.get(self)
}

codecentric AG
Adding custom context for rendering
static void render(GYokeResponse self, Map<String, Object> context, String template) { render(self, context, template, null, null) }
static void render(GYokeResponse self, Map<String, Object> context, String template, Closure next) {
render(self, context, template, null, next)
}
static void render(GYokeResponse self, Map<String, Object> context, String template, String layoutTemplate) {
render(self, context, template, layoutTemplate, null)
}
static void render(GYokeResponse self, Map<String, Object> context, String template, String layoutTemplate, Closure next) {
Map<String, Object> oldContext = getContext(self).clone()
getContext(self).clear()
getContext(self).putAll(context)
if (next)
self.render(template, layoutTemplate, next)
else
self.render(template, layoutTemplate)
getContext(self).clear()
getContext(self).putAll(oldContext)
}

codecentric AG
The template
<html>
<head>
<title>CRC</title>
</head>
<body>
<ul>
<% def data = files.sort { a, b -> a.key <=> b.key }
data.each { k, v ->
if (v != null) { %>
<li>${k} = ${v}</li>
<% } else { %>
<li>${k}</li>
<% } %>
<% } %>
</ul>
</body>
</html>

codecentric AG
Smoothen all up with a custom BaseScriptClass
abstract class VerticleScript extends Script {
Vertx getVertx() {
return binding.vertx
}
void setVertx(Vertx vertx) {
binding.vertx = vertx
}
Container getContainer() {
return binding.container
}
void setContainer(Container container) {
binding.container = container
}
EventBus getBus() {
vertx.eventBus
}
SharedData getSharedData() {
vertx.sharedData
}
Logger getLog() {
container.logger
}
Map<String, Object> getConfig() {
container.config
}
Map<String, String> getEnv() {
container.env
}

codecentric AG
Using the BaseScriptClass (vert.x 2.1)
Global usage:
compilerConfiguration.groovy:
customizer = { org.codehaus.groovy.control.CompilerConfiguration config ->
config.scriptBaseClass = 'de.codecentric.vertx.VerticleScript'
return config
}
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Local usage per Script:
@groovy.transform.BaseScript de.codecentric.vertx.VerticleScript verticleScript

codecentric AG
API to smoothen MongoDB usage
def db(String address, Map message, Closure success = null, Closure failure = null) {
bus.send(address, message) { Message result ->
Map reply = result.body()
if (reply.status == 'ok') {
if (success) {
if (success.maximumNumberOfParameters == 2) success(reply, result)
else success(reply)
}
} else {
if (failure) {
if (failure.maximumNumberOfParameters == 2) failure(reply, result)
else failure(result)
}
}
}
}

codecentric AG
def save(String address, String collection, Map document, Closure success = null, Closure failure = null) {
db(address,
[action: 'save', collection: collection, document: document, write_concern: 'SAFE'],
success, failure)
}
def update(String address, String collection, Map criteria, Map update, Closure success=null, Closure failure=null) {
db(address, [action: 'update', collection: collection, criteria: criteria, objNew: update, write_concern: 'SAFE'],
success, failure)
}
def delete(String address, String collection, Map matcher, Closure success = null, Closure failure = null) {
db(address, [action: 'delete', collection: collection, matcher: matcher], success, failure)
}
def read(String address, String collection, Map matcher, Closure success = null, Closure failure = null) {
db(address, [action: 'findone', collection: collection, matcher: matcher,], success, failure)
}

codecentric AG
def exists(String address, String collection, Map matcher, Closure success = null, Closure failure = null) {
def command = [action: 'find', collection: collection, matcher: matcher, batch_size: 100]
db(address, command, success) { Map reply, Message result ->
if (reply.status == 'more-exist') {
if (success.maximumNumberOfParameters == 2)
success(reply, result)
else
success(result)
} else {
if (failure.maximumNumberOfParameters == 2)
failure(reply, result)
else
failure(result)
}
}
}

codecentric AG
def query(String address, String collection, Map matcher,
Map options, Closure success, Closure failure) {
int max = options.max ?: -1
int offset = options.offset ?: -1
Map orderby = options.orderby ?: null
Map keys = options.keys ?: null
def data = []
def queryHandler
queryHandler = { Map reply, Message result ->
if (reply.status == 'more-exist') {
data.addAll reply.results
result.reply([:], queryHandler)
} else if (reply.status == 'ok') {
success(data)
} else if (reply.status == 'ok') {
success(data)
} else if (failure.maximumNumberOfParameters == 2) {
failure(reply, result)
} else failure(result)
}
def command = [ action: 'find', collection: collection,
matcher : matcher, batch_size: 100]
if (max >= 0) command.max = max
if (offset >= 0) command.offset = offset
if (orderby) command.orderby = orderby
if (keys) command.keys = keys
db(address, command, queryHandler, queryHandler)
}

codecentric AG
def query(String address, String collection, Map matcher, Closure success) {
query(address, collection, matcher, [:], success, null)
}
def query(String address, String collection, Map matcher, Closure success, Closure failure) {
query(address, collection, matcher, [:], success, failure)
}
def query(String address, String collection, Map matcher, Map options, Closure success) {
query(address, collection, matcher, options, success, null)
}

codecentric AG
Questions?
codecentric AG
Curiestr. 2
70563 Stuttgart
tel +49 (0) 711.674 00 - 328
fax +49 (0) 172.529 40 20
alexander.klein@codecentric.de
@saschaklein
www.codecentric.de
blog.codecentric.de
03.06.14 50

Vert.x using Groovy - Simplifying non-blocking code

More Related Content

What's hot

Viewers also liked

Similar to Vert.x using Groovy - Simplifying non-blocking code

More from sascha_klein

Recently uploaded

Vert.x using Groovy - Simplifying non-blocking code