1. Devoxx Antwerpen 2015
Concurrency Models on the JVM
Lutz Hühnken
@lutzhuehnken
https://github.com/lutzh/concurrency_talk

2. #devoxx 2
Before we start:
Why Concurrency is hard
(demonstrated with printed books)

3. #devoxx 3

4. #devoxx 4

5. #devoxx 5

6. #devoxx 6

7. #devoxx 7
Provocative claim
(for you to discuss over drinks later):
We need a new standard concurrency
model for Java!

8. #devoxx 8
Thread Problems
Problem 1: Efficiency (resp. the lack thereof)

9. #devoxx 9
CPU
Threads
Work

10. #devoxx 10
1 2 3 … 10.000
4-core CPU
Thousands of Threads
Work (a lot)

11. #devoxx 11
We want to move towards this:

12. #devoxx 12
From
ThreadProcess
Process Worker Task
To
Also see https://en.wikipedia.org/wiki/Reactor_pattern

13. #devoxx
Thinking about blocking I/O
When one thread corresponds to one task
13
Note: This is a snapshot in time of the system, not a sequence of events

14. #devoxx
Never block!
When one thread is a worker, working on multiple tasks
14
Note: This is a snapshot in time of the system, not a sequence of events

15. #devoxx
Solution for the efficiency problem:
• Sub-thread level concurrency
• Asynchronous I/O
• This is something the approaches we will look at have in common
• Note: This is also something that all Reactive Systems* have in
common
* http://www.reactivemanifesto.org
15

16. #devoxx 16
Thread Problems
Problem 1: Efficiency (resp. the lack thereof)
Problem 2: Programming Model

17. #devoxx 17
They discard the most essential and
appealing properties of sequential
computation: understandability,
predictability, and determinism.
Threads, as a model of computation,
are wildly nondeterministic, and the
job of the programmer becomes one
of pruning that nondeterminism.

18. #devoxx 18
Source: John Rose, Java VM Architect, JFokus, Stockholm, February 2015

19. #devoxx
Digression: Callback Hell
fs.readdir(source, function(err, files) {
if (err) {
console.log('Error finding files: ' + err)
} else {
files.forEach(function(filename, fileIndex) {
console.log(filename)
gm(source + filename).size(function(err, values) {
if (err) {
console.log('Error identifying file size: ' + err)
} else {
console.log(filename + ' : ' + values)
aspect = (values.width / values.height)
widths.forEach(function(width, widthIndex) {
height = Math.round(width / aspect)
console.log('resizing ' + filename + 'to ' + height + 'x' + height)
this.resize(width, height).write(destination + 'w' + width + '_' + filename,
function(err) {
if (err) console.log('Error writing file: ' + err)
})
}.bind(this))
}
})
})
}
})
19

20. #devoxx
I admit: It could be written in a more readable way
• Scala Futures / for-expressions
• async / await (also Scala)
• JavaScript Promises (then.. then.. then)
• All this is syntactic sugar for callbacks - which is
good
• But let’s explore other (better) approaches for
composition
20

21. #devoxx 21
What are we interested in?
Composition
State Management
Integration, I/O
(both async and sync)
Predictability

22. #devoxx 22
Coroutines (Green Threads, User Mode Threads,
IOC Threads, Fibers)
Quasar Fibers
http://www.paralleluniverse.co/quasar/
Channels Clojure core.async
https://github.com/clojure/core.async/
Event Loop vert.x
http://vertx.io
Actor Model Akka
http://akka.io
Concurrency Models

23. Fibers

24. #devoxx 24
new Fiber<V>() {
@Override
protected V run() throws SuspendExecution,
InterruptedException {
// code here
}
}.start();
looks like a thread,
walks like a thread..

25. #devoxx 25
class RetrieveInfo extends Fiber<Void> {
...
@Override
public Void run() {
final CloseableHttpClient client =
FiberHttpClientBuilder.create().build();
try {
String response =
client.execute(new HttpGet(MessageFormat.format(url, searchTerm)), new
BasicResponseHandler());
addResult(key, StringUtils.abbreviate(response, 1000));
} catch (IOException iox) {
...
Use like blocking
Need to aggregate results..
„fiber blocking“
async http client

26. #devoxx 26
private final Map<String, String> store = new ConcurrentHashMap<>();
public void define(final String searchTerm) {
for (String currentKey : sources.keySet()) {
new RetrieveInfo(currentKey, sources.get(currentKey),
searchTerm).start();
}
}
/**
* This method is working on shared mutual state - this is what we look to
avoid!
*/
void addResult(final String key, final String result) {
store.put(key, result);
...This method is working on shared mutual state - this is what we look to
shared mutable state
Start retrieval of definitions in background

27. #devoxx
Fibers
• Major advantage: Imperative Programming,
just like with threads
• Major disadvantage: Imperative
Programming, just like with threads
27

28. #devoxx 28
class FooAsync extends FiberAsync<String, FooException> implements FooCompletion {
@Override
public void success(String result) {
asyncCompleted(result);
}
@Override
public void failure(FooException exception) {
asyncFailed(exception);
}
}
String op() {
new FooAsync() {
protected void requestAsync() {
Foo.asyncOp(this);
}
}.run();
}
Turn callback APIs into fiber blocking APIs!

29. #devoxx
Fibers Summary
• Thread-like programming model
• Nice „tricks“: Instrumentation,
Suspendable, Thread Interop
• Fibers alone are a bit low-level
• (Almost) drop-in replacement for threads
29

30. Channels

31. #devoxx
Channels
• Theoretical Foundation:
Communicating Sequential Processes
(Tony Hoare 1978, https://
en.wikipedia.org/wiki/
Communicating_sequential_processes)
• Very popular in the „Go“ community
• Our example: Clojure core.async
31
Channels

32. #devoxx 32
(def echo-chan (chan))
(go (println (<! echo-chan)))
(>!! echo-chan „ketchup")
; => true
; => ketchup
go block - like a fiber
>!! blocks thread
<! blocks go block

33. #devoxx 33
(def echo-buffer (chan 2))
(>!! echo-buffer "ketchup")
; => true
(>!! echo-buffer "ketchup")
; => true
(>!! echo-buffer "ketchup")
; blocks
buffered channel
- async

34. #devoxx 34
(def responses-channel (chan 10))
(go
(println
(loop [values []]
(if (= 3 (count values))
values
(recur (conj values (<! responses-channel)))))))
(defn async-get [url result]
(http/get url #(go (>! result (:body %)))))
(async-get "http:m-w.com..." responses-channel)
(async-get "http:wiktionary.." responses-channel)
(async-get "http:urbandictionary.." responses-channel)
callback puts result in channel
channel for
responses
Aggregate results recursively

35. #devoxx
Channels
• Allows flow-oriented programming, very
flexible composition (e.g. alts!)
• Sanity through sequencing
• go block / thread duality allows integration
with both async and sync libraries
• Channels as keepers of state?
35
Channels Summary

36. Event Loop

37. #devoxx
Channels
• Event loop model - think node.js
• „multi event loop“, on the JVM
• Polyglot - Java, Scala, Clojure, JRuby, JS..
• Deploy small pieces (Verticles) and have
them send JSON to each other
37
vert.x

38. #devoxx 38
public class Receiver extends AbstractVerticle {
@Override
public void start() throws Exception {
EventBus eb = vertx.eventBus();
eb.consumer("ping-address", message -> {
System.out.println("Received message: " + message.body());
// Now send back reply
message.reply("pong!");
});
System.out.println("Receiver ready!");
}
}
register on event bus
extendVerticle
connect

39. #devoxx 39
MessageConsumer<JsonObject> consumer = eventBus.consumer("definer.task");
consumer.handler(message -> {
httpClient.getNow(message.body().getString("host"),
message.body().getString("path"), httpClientResponse ->
{
httpClientResponse.bodyHandler(buffer -> {
eventBus.send("collector",
new JsonObject()
.put(...)
callback based async http
register
send result as message

40. #devoxx 40
Vertx vertx = Vertx.vertx();
EventBus eventBus = vertx.eventBus();
DeploymentOptions options = new
DeploymentOptions().setInstances(sources.size());
vertx.deployVerticle("de.huehnken.concurrency.Task", options,
msg -> {
for (String currentKey : sources.keySet()) {
eventBus.send("definer.task",
new JsonObject()
.put(..)
which one gets the message?
deploy multiple

41. #devoxx 41
private final Map<String, String> store = new HashMap<>();
@Override
public void start(Future<Void> startFuture) {
EventBus eventBus = vertx.eventBus();
MessageConsumer<JsonObject> consumer = eventBus.consumer("collector");
consumer.handler(message -> {
store.put(message.body().getString(„key"),
message.body().getString("definition"));
wait for results..
encapsultate state inVerticle
messages are usually JSON

42. #devoxx
Event Bus (vert.x)
42
Image from Jonas Bandi @jbandi

43. #devoxx
Channels
• Loose coupling - very
• Sanity through events („single thread illusion“)
• Hybrid thread model allows integration of
synchronous APIs, off-loading work from event
loop
• State handling needs some care
• Utility APIs are callback heavy, node inspired
• Distributed! Unified model for local and
distributed concurrency
43
vert.x Summary

44. Actors

45. #devoxx
Channels
• The actor model [..] is a [model] that treats
"actors" as the universal primitives of
concurrent computation: in response to a
message that it receives, an actor can make
local decisions, create more actors, send more
messages, and determine how to respond to the
next message received. https://en.wikipedia.org/wiki/Actor_model
• Described by Carl Hewitt 1973
• Popular in the Erlang community
45
Akka

46. #devoxx
Actors (Akka)
46

47. #devoxx
Actors (Akka)
47

48. #devoxx
Actors (Akka)
48

49. #devoxx 49
class Coordinator(searchTerm: String) extends Actor {
var results = Map[String, String]()
def receive = {
case StartCommand =>
sources foreach {
case (key, url) =>
context.actorOf(Task.props(key, self)) ! GetDefinition(url)
}
context.system.scheduler.scheduleOnce(5 seconds, self, Done)
case Result(key, definition) =>
results += key -> definition
if (results.size == sources.size) {
self ! Done
}
case Done =>
spawn child actors, send command
encapsulate state
aggregate results

50. #devoxx 50
class Task(key: String, coordinator: ActorRef) extends Actor {
def receive = {
case GetDefinition(url) =>
http.singleRequest(HttpRequest(uri = url)) pipeTo self
case HttpResponse(StatusCodes.OK, headers, entity, _) =>
coordinator ! Result(key, entity.dataBytes.utf8String)
case HttpResponse(code, _, _, _) =>
// handle error
send result to aggregator
async call, translated to message

51. #devoxx
Actors (Akka)
51

52. #devoxx
Channels
• Sanity through messages („single thread illusion“)
• Messages (as opposed to events)
• Dispatchers allows integration of synchronous
APIs, off-loading work from event loop
• pipeTo & ask to integrate with async APIs
• Distributed! Unified model for local and
distributed concurrency.
• Supervision! Fault handling enforced.
52
Actors Summary

53. #devoxx 53
Programming model chart
Tasks (sub-
thread level)
Asynchronous
Messaging
Distribution
(unified model)
Supervision
Fibers
✔
Channels
(core.async) ✔ ✔
Event Bus
(vert.x) ✔ ✔ ✔
Aktoren
(Akka) ✔ ✔ ✔ ✔

54. #devoxx
• Concurrency is interesting
• Threads are passé, we want a better abstraction on top of it
• Alternatives are available today
• If you only have time to look at one model, I recommend Akka
54
Summary

55. #devoxx 55
Closing the loop.. another book..

56. Thank You
Lutz Hühnken
@lutzhuehnken
https://github.com/lutzh/concurrency_talk

57. Agents

58. #devoxx 58
(def x (agent 0))
(defn increment [c n] (+ c n))
(send x increment 5) ; @x -> 5
(send x increment 10) ; @x -> 15
„wrap“ a value in an agent
send the code to the agent to
modify value!
a function

59. #devoxx
Agents
• The agent encapsulates state
• Send code to the agent to change value
• Interesting feature especially for functional
programming
• Async execution, but not really a
concurrency model
59

60. #devoxx
Beware of the benchmark!
• Unfit: CPU bound
• Also unfit: Anything using blocking I/O
• It’s not about single request performance,
it’s about scalability!
60

61. #devoxx
But I just want to build a little REST service..
• If you don’t program on this level yourself:
know you frameworks!
• Look out for „reactive“ or „async“
• Play!
• Vert.x
• Akka HTTP
• (Pivotal Reactor ? Rat pack ?)
61

62. #devoxx
For completeness
• Quasar also offers Channels, and even
Actors
• Akka also offers a Bus, and Agents.
• A lot of this comes from outside of the JVM
(Go Channels, node.js, Erlang Actors)
62

63. Thanks Again!
Lutz Hühnken
@lutzhuehnken
https://github.com/lutzh/concurrency_talk

64. #devoxx
Image Credits
Cat on hot tin roof - from "McPhillamyActorBlog" - http://mcphillamy.com
Agents - from "Matrix Wiki" - http://matrix.wikia.com/wiki/Agent
Event Loop - http://www.freeimages.com/photo/big-looping-1522456
Channel - http://www.freeimages.com/photo/white-cliffs-of-dover-1256973
Fibers - http://www.freeimages.com/photo/fiber-optics-1503081
64