Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Streaming Api Design with Akka, Scala and Spray

3,643 views

Published on

An introduction on how to design APIs, followed by a recipe using scala, and a REST server implementation in Akka and Spray.

This presentation looks at API design from two different angles. Firstly, from a API developer/user point of view, we define the API developer journey, we introduce and craft meaningful REST url paths and RESTful resources. Secondly, from an API implementation point of view, we dive into the mechanics of akka, and spray.

During the first part we highlight the process of value creation from idea to API design, which translate into the definition of fluent API paths where resources are concatenated, rendering in facts the elements of a streaming analytical pipeline into a set of concatenated resources accessible via the api. We highlight the importance of a meaningful semantic for the API design process and a good match between the value proposition and the resources exposed by the api itself. To render the concept clearer we take the example of a streaming api which collects and gathers readings from a set of jogging wrist bands connected to internet.

During the second part of the presentation, we focus on scalability, resiliency and availability of the solution. We introduce the concept of actor as basic processing element. We introduce the concept of location transparencies and hierarchical supervision when mapping actors to a cluster of computing resources. Finally we describe how to translate a url path into a set of routes, and how to dispatch and process the incoming json readings to the right analytical actors. This short overview gives an idea on the mechanics behind a real time, streaming api. And on how to implement api resources as asynchronous communicating actors, in order to provide a rudimental but effective streaming analytical engine.

Published in: Software
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Streaming Api Design with Akka, Scala and Spray

  1. 1. Streaming Api Design with
  2. 2. By Gruban / Patrick Gruban from Munich, Germany (originally posted to Flickr as IMG_9038) [CC-BY- SA-2.0 (http://creativecommons.org/licenses/by-sa/2.0)], via Wikimedia Commons
  3. 3. Define an API that ... given a collection of wristband data from a bunch of runners { "heart": 108, "name" : "Judy" ... ... } ...
  4. 4. Define an API that ... given a collection of wristband data from a bunch of runners { "Fred": 125, "Judy": 110 } ... ... Aggregates the data, calculate max heart rate each second grouped by name ... { "heart": 108, "name" : "Judy" } ...
  5. 5. Let’s design this! Let’s create a stream ... POST /api/streams
  6. 6. Let’s design this! Let’s create a stream ... GET /api/streams/1 { "id": 1, "links": { "uri": "/api/streams/1", "input": "/api/streams/1/in", "filters": "/api/streams/1/in/filters" } }
  7. 7. Let’s design this! Given a stream id, lets create a filter, that gives the highest heart rate of the group, in a window of 1 seconds ... POST /api/streams/1/in/filters { "resolution": 1, "field" : "heart", "transform" : "max" "group_by" : "name" }'
  8. 8. Let’s design this! Given a stream id, lets create a filter, that gives the highest heart rate of the group, in a window of 1 seconds ... GET /api/streams/1/in/filters/1
  9. 9. Let’s design this! Now I can feed running data in ... POST /api/streams/1/in { "heart": 115, "name" : "Judy" }
  10. 10. Let’s design this! And get the filtered data out ... GET /api/streams/1/in/filtered_by/1/out { "Fred": 121, "Judy": 110 }
  11. 11. This API is fluent … nice. /api/streams/1/in /api/streams/1/in/filtered_by/1/out
  12. 12. Akka: why? ● ● ● ● ●
  13. 13. Akka: asynchronous …
  14. 14. Akka: asynchronous ● ● ●
  15. 15. Akka: actors ● ● ● ●
  16. 16. Akka: actors class HeartRateActor extends Actor { var heart_rate = 0.0 def receive = { case value: Double => heart_rate = max(heart_rate, value) case "get" => sender ! heart_rate } }
  17. 17. Actors as Resources Streams Actor /api/streams POST def receive = { case CreateStream => val stream = system.actorOf(StreamActor(id), s"stream-$id")
  18. 18. Actors as Resources Streams Actor Stream-1 Actor /api/streams/1
  19. 19. Actors as Resources Streams Actor Stream-1 Actor Filter-1 Actor /api/streams/1/in/filters/1
  20. 20. Actors as Resources /api/streams/1/in/filters/2
  21. 21. Actors as Resources /api/streams/2/in/filters/1
  22. 22. "New York marathon Verrazano bridge" by Martineric from Lille, France - Marathon de New York : Verrazano Bridge. Licensed under Creative Commons Attribution-Share Alike 2.0 via Wikimedia Commons -
  23. 23. Actor as Scalable Resources jvm: This is where my actors run
  24. 24. Actor as Scalable Resources jvm: This is where my actors run
  25. 25. Actor as Scalable Resources jvm: This is where my actors run Many jvm’s, cores, nodes, racks
  26. 26. Actor as Scalable Resources
  27. 27. Actors as Scalable Resources ● ● ●
  28. 28. Where is my http port?
  29. 29. Spray: HTTP toolkit based on Akka
  30. 30. Spray: HTTP toolkit based on Akka
  31. 31. Spray: Reactive and Scalable ● ● ●
  32. 32. Spray: Reactive and Scalable ● ● ● ● ●
  33. 33. Spray: Routing val route = { pathPrefix("api/streams" / IntNumber) { id => { get { ctx => (coreActor ? Get(id) ).mapTo[Int] .onSuccess { resource => complete(resource) } } } ~ someOtherRoute } }
  34. 34. Spray: Web API get the output ● ● ● ●
  35. 35. APIs: back to back connections …
  36. 36. "Pedalboard (995939579)-2" by Pedalboard_(995939579).jpg: Michael Morel from Barcelona, Spainderivative work: Atlantictire (talk) - Pedalboard_(995939579).jpg. Licensed under Creative Commons Attribution 2.0 via Wikimedia Commons -

×