The document proposes designing a streaming API that aggregates data from wristbands worn by runners. It defines endpoints to create a stream, add filters to the stream to calculate metrics like maximum heart rate per second within groups, ingest data, and retrieve the filtered output. The API is built on Akka actors for asynchronous and reactive processing and on Spray for building HTTP services on top of Akka actors.

1. Streaming Api Design
with

4. By Gruban / Patrick Gruban from Munich, Germany (originally posted to Flickr as IMG_9038) [CC-BY-
SA-2.0 (http://creativecommons.org/licenses/by-sa/2.0)], via Wikimedia Commons

5. Define an API that ...
given a collection of wristband data from
a bunch of runners
{
"heart": 108,
"name" : "Judy"
... ...
}
...

6. Define an API that ...
given a collection of wristband data from
a bunch of runners
{
"Fred": 125,
"Judy": 110
}
... ...
Aggregates the data,
calculate max heart rate each second
grouped by name ...
{
"heart": 108,
"name" : "Judy"
}
...

7. Let’s design this!
Let’s create a stream ...
POST /api/streams

8. Let’s design this!
Let’s create a stream ...
GET /api/streams/1
{
"id": 1,
"links": {
"uri": "/api/streams/1",
"input": "/api/streams/1/in",
"filters": "/api/streams/1/in/filters"
}
}

9. Let’s design this!
Given a stream id, lets create a filter,
that gives the highest heart rate of the group,
in a window of 1 seconds ...
POST /api/streams/1/in/filters
{
"resolution": 1,
"field" : "heart",
"transform" : "max"
"group_by" : "name"
}'

10. Let’s design this!
Given a stream id, lets create a filter,
that gives the highest heart rate of the group,
in a window of 1 seconds ...
GET /api/streams/1/in/filters/1

11. Let’s design this!
Now I can feed running data in ...
POST /api/streams/1/in
{
"heart": 115,
"name" : "Judy"
}

12. Let’s design this!
And get the filtered data out ...
GET /api/streams/1/in/filtered_by/1/out
{
"Fred": 121,
"Judy": 110
}

13. This API is fluent … nice.
/api/streams/1/in
/api/streams/1/in/filtered_by/1/out

14. Akka: why?
●
●
●
●
●

15. Akka: asynchronous
…

17. Akka: asynchronous
●
●
●

18. Akka: actors
●
●
●
●

19. Akka: actors
class HeartRateActor extends Actor {
var heart_rate = 0.0
def receive = {
case value: Double =>
heart_rate = max(heart_rate, value)
case "get" =>
sender ! heart_rate
}
}

20. Actors as Resources
Streams Actor
/api/streams
POST
def receive = {
case CreateStream =>
val stream = system.actorOf(StreamActor(id), s"stream-$id")

21. Actors as Resources
Streams Actor
Stream-1 Actor
/api/streams/1

22. Actors as Resources
Streams Actor
Stream-1 Actor
Filter-1 Actor
/api/streams/1/in/filters/1

23. Actors as Resources
/api/streams/1/in/filters/2

24. Actors as Resources
/api/streams/2/in/filters/1

25. "New York marathon Verrazano bridge" by Martineric from Lille, France - Marathon de New York : Verrazano
Bridge. Licensed under Creative Commons Attribution-Share Alike 2.0 via Wikimedia Commons -

26. Actor as Scalable Resources
jvm: This is where my actors run

27. Actor as Scalable Resources
jvm: This is where my actors run

28. Actor as Scalable Resources
jvm: This is where my actors run
Many jvm’s, cores, nodes, racks

29. Actor as Scalable Resources

30. Actors as Scalable Resources
●
●
●

31. Where is my http port?

32. Spray: HTTP toolkit based on Akka

33. Spray: HTTP toolkit based on Akka

34. Spray: Reactive and Scalable
●
●
●

35. Spray: Reactive and Scalable
●
●
●
●
●

36. Spray: Routing
val route = {
pathPrefix("api/streams" / IntNumber) {
id => {
get {
ctx => (coreActor ? Get(id) ).mapTo[Int]
.onSuccess { resource => complete(resource) }
}
} ~ someOtherRoute
}
}

37. Spray: Web API get the output
●
●
●
●

38. APIs: back to back connections
…

39. "Pedalboard (995939579)-2" by Pedalboard_(995939579).jpg: Michael
Morel from Barcelona, Spainderivative work: Atlantictire (talk) -
Pedalboard_(995939579).jpg. Licensed under Creative Commons
Attribution 2.0 via Wikimedia Commons -