Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Akka Streams & HTTP
Dr. Roland Kuhn
@rolandkuhn — Akka Tech Lead
Streams Occur Naturally
• traditionally:
• file input/output
• network connections
• more modern:
• processing big data wi...
3
What is a Stream?
• ephemeral, time-dependent sequence of elements
• possibly unbounded in length
• therefore focusing o...
Reactive Traits
4
Reactive means Message Flow
5
Reactive means Message Flow
6
37% discount
with code
kuhnco
Distributed vs. Static Typing
• in general a very tough problem
• independent entities evolving concurrently
• promising p...
Possible Solutions
• the Traditional way: blocking calls
8
Possible Solutions
• the Push way: buffering and/or dropping

(optimized for fast consumer)
9
Possible Solutions
• the Pull way: poor performance

(optimized for fast producer)
10
Possible Solutions
• the Reactive way:

non-blocking & non-dropping & bounded
11
Reactive Streams
Origin and motivation
• all participants face the same basic problem
• all are building tools for their community
• a comm...
Goals
• minimal interfaces—essentials only
• rigorous specification of semantics
• TCK for verification of implementation
...
Reactive Streams
• asynchronous & non-blocking
• flow of data
• flow of demand
• minimal coordination and contention
• mes...
A Data Market using Supply & Demand
• data elements flow downstream
• demand flows upstream
• data elements flow only when...
Reactive Push–Pull
• “push”—when consumer is faster
• “pull”—when producer is faster
• switches automatically between thes...
Explicit Demand: One-to-many
18
demand
data
Splitting the data means merging the demand
Explicit Demand: Many-to-one
19
Merging the data means splitting the demand
Back-Pressure is Contagious
• C is slow
• B must slow down
• A must slow down
20
CA B
• TCP for example has it built-in
Back-Pressure can be Propagated
21
CA B
networkhosts
The Meat
22
trait Publisher[T] {
def subscribe(sub: Subscriber[T]): Unit
}
trait Subscription {
def request(n: Long): Unit...
Akka Streams
Declaring a Stream Topology
24
Declaring a Stream Topology
25
Declaring a Stream Topology
26
Declaring a Stream Topology
27
Declaring a Stream Topology
28
Declaring a Stream Topology
29
Declaring a Stream Topology
30
API Design
• goals:
• no magic
• supreme compositionality
• exhaustive model of bounded stream processing
• consequences:
...
Declaring and Running a Stream
32
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () =...
Declaring and Running a Stream
33
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () =...
Declaring and Running a Stream
34
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () =...
Materialization
• Akka Streams separate the what from the how
• declarative Source/Flow/Sink DSL to create blueprint
• Flo...
Stream Sources
• org.reactivestreams.Publisher[T]
• org.reactivestreams.Subscriber[T]
• Iterator[T] / Iterable[T]
• Code b...
Stream Sinks
• org.reactivestreams.Publisher[T]
• org.reactivestreams.Subscriber[T]
• ActorSubscriber
• scala.concurrent.F...
Linear Stream Transformations
• Deterministic (like for collections)
• map, filter, collect, grouped, drop, take, groupBy,...
Nonlinear Stream Transformations
• Fan-In
• merge, concat, zip, …
• Fan-Out
• broadcast, route, balance, unzip, …
39
Akka HTTP
Why do we add an HTTP module?
• Akka is about building distributed applications
• distribution implies integration
• betwe...
Akka HTTP—The Origins: Spray.IO
• Fully embeddable HTTP stack based on Actors
• focused on HTTP integration (toolkit)
• se...
Spray.IO Features
• immutable, case class-based HTTP model
• fast, lightweight HTTP client and server
• powerful DSL for s...
Spray.IO Weaknesses
• handling of chunked requests is clunky, incomplete
• dealing with large message entities can be diff...
Proxying Large Responses
45
Akka HTTP is Spray 2.0
• addressing the weaknesses, polishing the features
• Java API
• simplified module structure
• core...
HTTP Stream Topology
47
The Application Stack
48
O/S-level network stack
Java NIO (JDK)
Akka IO
Akka HTTP Core
Akka HTTP
user-
level
Akka IO
• bridges Java NIO with Akka Actors or streams
• supports TCP, UDP and SSL
49
Akka HTTP Core
• implements HTTP/1.1 according to the RFCs
• based upon the TCP facilities of Akka IO
• exposed as freely ...
A Quick View of the HTTP Model
51
case class HttpRequest(
method: HttpMethod = HttpMethods.GET,
uri: Uri = Uri./,
headers:...
Akka HTTP
• builds upon Akka HTTP Core
• adds (un)marshaling for HttpEntities
• adds (de)compression (gzip / deflate)
• hi...
Stream Pipelines
53
TCP SSL HTTP App
Requests
Responses
optional
(not yet)
A Simple Server Example
54
import Directives._
val binding = Http().bind("localhost", 8080)
binding startHandlingWith {
pa...
Summary
When can we have it?
• currently pre-release versions:
• reactive-streams 1.0.0-RC1
• Akka Streams & HTTP 1.0-M2
• still m...
Resources
• https://github.com/akka/akka/tree/release-2.3-dev
• http://doc.akka.io/docs/akka-stream-and-http-
experimental...
©Typesafe 2014 – All Rights Reserved
Upcoming SlideShare
Loading in …5
×

Akka Streams and HTTP

39,470 views

Published on

Akka Streams are an implementation of the Reactive Streams specification (http://reactive-streams.org/), a joint effort that aims at standardizing the exchange of streams of data across asynchronous boundaries in a fully non-blocking way while providing flow control and mediating back pressure. In this presentation we go into the details of what this new abstraction can be used for and what the guiding principles are behind its development. We then focus on one prominent use-case which is the upcoming Akka HTTP module: a fully stream-enabled, reactive HTTP server and client implementation.

Published in: Software
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Akka Streams and HTTP

  1. 1. Akka Streams & HTTP Dr. Roland Kuhn @rolandkuhn — Akka Tech Lead
  2. 2. Streams Occur Naturally • traditionally: • file input/output • network connections • more modern: • processing big data with finite memory • real-time data processing (CEP) • serving numerous clients simultaneously with bounded resources (IoT, streaming HTTP APIs) 2
  3. 3. 3 What is a Stream? • ephemeral, time-dependent sequence of elements • possibly unbounded in length • therefore focusing on transformations «You cannot step twice into the same stream. For as you are stepping in, other waters are ever flowing on to you.» — Heraclitus
  4. 4. Reactive Traits 4
  5. 5. Reactive means Message Flow 5
  6. 6. Reactive means Message Flow 6 37% discount with code kuhnco
  7. 7. Distributed vs. Static Typing • in general a very tough problem • independent entities evolving concurrently • promising progress on Project Gålbma • in many specific cases no type evolution necessary • letting uniform messages flow in safe conduits • making streams even more interesting 7
  8. 8. Possible Solutions • the Traditional way: blocking calls 8
  9. 9. Possible Solutions • the Push way: buffering and/or dropping
 (optimized for fast consumer) 9
  10. 10. Possible Solutions • the Pull way: poor performance
 (optimized for fast producer) 10
  11. 11. Possible Solutions • the Reactive way:
 non-blocking & non-dropping & bounded 11
  12. 12. Reactive Streams
  13. 13. Origin and motivation • all participants face the same basic problem • all are building tools for their community • a common solution benefits everybody • interoperability to make best use of efforts • propose to include in future JDK 13
  14. 14. Goals • minimal interfaces—essentials only • rigorous specification of semantics • TCK for verification of implementation • complete freedom for many idiomatic APIs • specification should be efficiently implementable 14
  15. 15. Reactive Streams • asynchronous & non-blocking • flow of data • flow of demand • minimal coordination and contention • message passing allows for distribution across • applications, nodes, CPUs, threads, actors 15
  16. 16. A Data Market using Supply & Demand • data elements flow downstream • demand flows upstream • data elements flow only when there is demand • data in flight is bounded by signaled demand • recipient is in control of maximal incoming data rate 16 Publisher Subscriber data demand
  17. 17. Reactive Push–Pull • “push”—when consumer is faster • “pull”—when producer is faster • switches automatically between these • batching demand allows batching data 17 Publisher Subscriber data demand
  18. 18. Explicit Demand: One-to-many 18 demand data Splitting the data means merging the demand
  19. 19. Explicit Demand: Many-to-one 19 Merging the data means splitting the demand
  20. 20. Back-Pressure is Contagious • C is slow • B must slow down • A must slow down 20 CA B
  21. 21. • TCP for example has it built-in Back-Pressure can be Propagated 21 CA B networkhosts
  22. 22. The Meat 22 trait Publisher[T] { def subscribe(sub: Subscriber[T]): Unit } trait Subscription { def request(n: Long): Unit def cancel(): Unit } trait Subscriber[T] { def onSubscribe(s: Subscription): Unit def onNext(e: T): Unit def onError(t: Throwable): Unit def onComplete(): Unit } Not user-level API Intended as SPI for interop
  23. 23. Akka Streams
  24. 24. Declaring a Stream Topology 24
  25. 25. Declaring a Stream Topology 25
  26. 26. Declaring a Stream Topology 26
  27. 27. Declaring a Stream Topology 27
  28. 28. Declaring a Stream Topology 28
  29. 29. Declaring a Stream Topology 29
  30. 30. Declaring a Stream Topology 30
  31. 31. API Design • goals: • no magic • supreme compositionality • exhaustive model of bounded stream processing • consequences: • immutable and reusable stream blueprints • explicit materialization step 31 http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M2/stream-design.html
  32. 32. Declaring and Running a Stream 32 val upper = Source(Iterator from 0).take(10) val lower = Source(1.second, 1.second, () => Tick) val source = Source[(Int, Tick)]() { implicit b => val zip = Zip[Int, Tick] val out = UndefinedSink[(Int, Tick)] upper ~> zip.left ~> out lower ~> zip.right out } val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" } val sink = Sink.foreach(println) val future = source.connect(flow).runWith(sink)
  33. 33. Declaring and Running a Stream 33 val upper = Source(Iterator from 0).take(10) val lower = Source(1.second, 1.second, () => Tick) val source = Source[(Int, Tick)]() { implicit b => val zip = Zip[Int, Tick] val out = UndefinedSink[(Int, Tick)] upper ~> zip.left ~> out lower ~> zip.right out } val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" } val sink = Sink.foreach(println) val future = source.connect(flow).runWith(sink)
  34. 34. Declaring and Running a Stream 34 val upper = Source(Iterator from 0).take(10) val lower = Source(1.second, 1.second, () => Tick) val source = Source[(Int, Tick)]() { implicit b => val zip = Zip[Int, Tick] val out = UndefinedSink[(Int, Tick)] upper ~> zip.left ~> out lower ~> zip.right out } val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" } val sink = Sink.foreach(println) val future = source.connect(flow).runWith(sink)
  35. 35. Materialization • Akka Streams separate the what from the how • declarative Source/Flow/Sink DSL to create blueprint • FlowMaterializer turns this into running Actors • this allows alternative materialization strategies • optimization • verification / validation • cluster deployment • only Akka Actors for now, but more to come! 35
  36. 36. Stream Sources • org.reactivestreams.Publisher[T] • org.reactivestreams.Subscriber[T] • Iterator[T] / Iterable[T] • Code block (function that produces Option[T]) • scala.concurrent.Future[T] • TickSource • ActorPublisher • singleton / empty / failed • … plus write your own (fully extensible) 36
  37. 37. Stream Sinks • org.reactivestreams.Publisher[T] • org.reactivestreams.Subscriber[T] • ActorSubscriber • scala.concurrent.Future[T] • blackhole / foreach / fold / onComplete • … or create your own 37
  38. 38. Linear Stream Transformations • Deterministic (like for collections) • map, filter, collect, grouped, drop, take, groupBy, … • Time-Based • takeWithin, dropWithin, groupedWithin, … • Rate-Detached • expand, conflate, buffer, … • asynchronous • mapAsync, mapAsyncUnordered, flatten, … 38
  39. 39. Nonlinear Stream Transformations • Fan-In • merge, concat, zip, … • Fan-Out • broadcast, route, balance, unzip, … 39
  40. 40. Akka HTTP
  41. 41. Why do we add an HTTP module? • Akka is about building distributed applications • distribution implies integration • between internal (sub)systems
 ➜ akka-clusterbasedonakka-remote • with external systems
 ➜ akka-http(linguafrancaoftheinternet) 41
  42. 42. Akka HTTP—The Origins: Spray.IO • Fully embeddable HTTP stack based on Actors • focused on HTTP integration (toolkit) • server- and client-side • seamlessly integrated with Akka 42
  43. 43. Spray.IO Features • immutable, case class-based HTTP model • fast, lightweight HTTP client and server • powerful DSL for server-side API definition • fully async & non-blocking, actor-friendly,
 modular, testable • based entirely on Scala & Akka Actors 43
  44. 44. Spray.IO Weaknesses • handling of chunked requests is clunky, incomplete • dealing with large message entities can be difficult • some unintuitive corner-cases in routing DSL • deep implicit argument chains in some cases hard to debug • missing features, foremost websocket support 44
  45. 45. Proxying Large Responses 45
  46. 46. Akka HTTP is Spray 2.0 • addressing the weaknesses, polishing the features • Java API • simplified module structure • core improvement: fully stream-based 46
  47. 47. HTTP Stream Topology 47
  48. 48. The Application Stack 48 O/S-level network stack Java NIO (JDK) Akka IO Akka HTTP Core Akka HTTP user- level
  49. 49. Akka IO • bridges Java NIO with Akka Actors or streams • supports TCP, UDP and SSL 49
  50. 50. Akka HTTP Core • implements HTTP/1.1 according to the RFCs • based upon the TCP facilities of Akka IO • exposed as freely reusable stream transformations • low-level and extensible implementation of
 HTTP model and spec 50
  51. 51. A Quick View of the HTTP Model 51 case class HttpRequest( method: HttpMethod = HttpMethods.GET, uri: Uri = Uri./, headers: immutable.Seq[HttpHeader] = Nil, entity: RequestEntity = HttpEntity.Empty, protocol: HttpProtocol = HttpProtocols.`HTTP/1.1` ) extends HttpMessage case class HttpResponse( status: StatusCode = StatusCodes.OK, headers: immutable.Seq[HttpHeader] = Nil, entity: ResponseEntity = HttpEntity.Empty, protocol: HttpProtocol = HttpProtocols.`HTTP/1.1` ) extends HttpMessage
  52. 52. Akka HTTP • builds upon Akka HTTP Core • adds (un)marshaling for HttpEntities • adds (de)compression (gzip / deflate) • high-level server-side routing DSL • type-safe and flexible • highly compositional building blocks (Directives) • includes common behaviors pre-canned 52
  53. 53. Stream Pipelines 53 TCP SSL HTTP App Requests Responses optional (not yet)
  54. 54. A Simple Server Example 54 import Directives._ val binding = Http().bind("localhost", 8080) binding startHandlingWith { path("order" / HexIntNumber) { id => get { complete(s"Received GET for order $id") } ~ put { complete((actor ? Put(id)).mapTo[String]) } } }
  55. 55. Summary
  56. 56. When can we have it? • currently pre-release versions: • reactive-streams 1.0.0-RC1 • Akka Streams & HTTP 1.0-M2 • still missing: • Akka HTTP client features • SSL integration • websockets • Akka Streams & HTTP 1.0 expected end of Feb’15 56
  57. 57. Resources • https://github.com/akka/akka/tree/release-2.3-dev • http://doc.akka.io/docs/akka-stream-and-http- experimental/1.0-M2/ • akka-user mailing list 57
  58. 58. ©Typesafe 2014 – All Rights Reserved

×