1. Reactive Slick
The principles behind Slick 3
Lutz Hühnken (@lutzhuehnken)

2. JDBC
try {
// make the connection
Class.forName(driver)
connection = DriverManager.getConnection(url, username,
password)
// create the statement, and run the select query
val statement = connection.createStatement()
val resultSet = statement.executeQuery("SELECT host, user
FROM user")
while ( resultSet.next() ) {
val host = resultSet.getString("host")
val user = resultSet.getString("user")
println("host, user = " + host + ", " + user)
}
} catch {
case e => e.printStackTrace
}
connection.close()
}
2

3. Reactive Slick
JDBC
The problem with that code
• well there are probably many, style etc.
But the problem:
• Synchronous I/O
• Will block the thread it’s running on.
• This is true not only for JDBC, but any DB lib running on top of
JDBC, Slick included.
We like to avoid that. Why?
3

4. Reactive Slick
Digression: Concurrency model
Java EE: Threads. Servlet API: Thread per Request
4
Note: This is a snapshot at one point in time, not a sequence of events.

5. Reactive Slick
Problems with Threads
• Context Switching is expensive
• Memory overhead per thread
• Lock contention when communicating between threads (if
you have shared mutable state, which is likely..)
5

6. Reactive Slick
Digression: Concurrency model
6
Source: John Rose, Java VM Architect, JFokus, Stockholm, February 2015

7. Reactive Slick
Task level concurrency
This does not mean: „no threads“.
It means: Threads should not be the finest level of
concurrency. Not be the level the application developer
works on.
7

8. Reactive Slick
Digression: Concurrency model
Reactive: Sub-Thread. Play: n threads per m requests
8
Note: This is a snapshot at one point in time, not a sequence of events.

9. Reactive Slick
Solution: Use Future / blocking combo
• Put blocking database calls in
Future(blocking( ... ))
• Contention for Connections (but may be limited by the
ExecutionContext)
• A saturated thread pool blocks everything.
9

10. Reactive Slick
Better solution: Isolation
10
Note: This is a snapshot at one point in time, not a sequence of events.

11. Reactive Slick
New Question: Size
Idea: number of threads <= number of connections in pool, so
getConnection never blocks.
Given you want to handle 10.000 concurrent clients.. what size
should your connection pool be?
• 10
• 100
• 1.000
• 10.000
11

12. Reactive Slick
It was a trick question…
A formula which has held up pretty well across a
lot of benchmarks for years is that for optimal
throughput the number of active connections should
be somewhere near ((core_count * 2) +
effective_spindle_count). Core count should not
include HT threads, even if hyperthreading is
enabled. Effective spindle count is zero if
the active data set is fully cached, and approaches
the actual number of spindles as the cache hit rate
falls. ... There hasn't been any analysis so far
regarding how well the formula works with SSDs.
12
(From PostgreSQL)

13. Reactive Slick
Example: Quad-core server
13
Image by Stefan Zeiger

14. Reactive Slick
The difference
14
Image by Stefan Zeiger

15. Reactive Slick
Slick
Every Database contains an AsyncExecutor that manages the
thread pool for asynchronous execution of Database I/O
Actions.
15
mydb = {
dataSourceClass = "org.postgresql.ds.PGSimpleDataSource"
properties = {
databaseName = "mydb"
user = "myuser"
password = "secret"
}
numThreads = 10
}

16. Switch to code

17. Reactive Slick
JDBC (again, it’s the same slide)
try {
// make the connection
Class.forName(driver)
connection = DriverManager.getConnection(url, username,
password)
// create the statement, and run the select query
val statement = connection.createStatement()
val resultSet = statement.executeQuery("SELECT host, user FROM
user")
while ( resultSet.next() ) {
val host = resultSet.getString("host")
val user = resultSet.getString("user")
println("host, user = " + host + ", " + user)
}
} catch {
case e => e.printStackTrace
}
connection.close()
}
17

18. Reactive Streams

19. Reactive Slick
Reactive Streams Overview
• How do we:
• Handle potentially infinite streams of data?
• Handle data in a reactive manner?
• Achieve asynchronous non-blocking data flow?
• Avoid out of memory errors?
19

20. Reactive Slick
Supply and Demand
• Data Items Flow Downstream
• Demand Flows Upstream
• Data Items flow only when there is demand.
20

21. Reactive Slick
Reactive Streams Specification
• Interface Specification
• Java interfaces for implementations
• TCK
• Test Harness to validate implementations
• Specification Website
• http://www.reactive-streams.org/
21

22. Reactive Slick
Publisher
package org.reactivestreams;
public interface Publisher<T> {
public void subscribe<T>(
Subscriber<T> subscriber);
}
22

23. Reactive Slick
Subscriber
public interface Subscriber<T> {
public void onSubscribe(
Subscription subscription);
public void onNext(T element);
public void onComplete();
public void onError(Throwable cause);
}
23

24. Reactive Slick
Subscription
public interface Subscription {
public void cancel();
public void request(long elements);
}
24

25. Reactive Slick
Dynamic Push-Pull
• “Push” behavior when consumer is faster
• “Pull” behavior when producer is faster
• Switches automatically between these
• Batching demand allows batching data
25

26. Reactive Slick
Linear Transformations
26
Demand
Source Sinkmap …

27. Reactive Slick
Linear Transformations
27
Source
Deman
Sink
drop
map

28. Reactive Slick
Linear Stream Transformations
• Deterministic (like for collections)
• map, filter, collect, grouped, drop, take, groupBy, ...
• Time-Based
• takeWithin, dropWithin, groupedWithin, ...
• Rate-Detached
• expand, conflate, buffer, ...
• asynchronous
• mapAsync, mapAsyncUnordered, ...
28

29. Reactive Slick
Non-linear Stream Transformations
• Fan-In
• merge, concat, zip
• Fan-Out
• broadcast, route, unzip
29

30. Reactive Slick
Fan In
30

31. Reactive Slick
Fan Out
31

32. Reactive Slick
Materialization
• Akka Streams separate the what from the how
• declarative Source/Flow/Sink DSL to create blueprint
• FlowMaterializer turns this into running Actors
32

33. Switch to code

34. Reactive Slick
Try it yourself!
34

35. Reactive Slick
Thank You
Further information:
http://slick.typesafe.com
Contact:
lutz.huehnken@typesafe.com
Twitter:
@lutzhuehnken
35

36. ©Typesafe 2015 – All Rights Reserved