Akka Streams & HTTP provides reactive, asynchronous, and non-blocking streams for processing data and HTTP requests and responses. It builds upon Akka IO and the Reactive Streams initiative to allow stream-based topologies to be declared and run for tasks like processing big data, serving clients simultaneously with limited resources, and building distributed applications that integrate with external systems over HTTP. Key features include stream sources, sinks, and transformations along with a routing DSL for building HTTP servers and clients on top of Akka IO and HTTP Core.

1. Akka Streams & HTTP
Dr. Roland Kuhn
@rolandkuhn — Akka Tech Lead

2. Streams Occur Naturally
• traditionally:
• file input/output
• network connections
• more modern:
• processing big data with finite memory
• real-time data processing (CEP)
• serving numerous clients simultaneously with bounded
resources (IoT, streaming HTTP APIs)
2

3. 3
What is a Stream?
• ephemeral, time-dependent sequence of elements
• possibly unbounded in length
• therefore focusing on transformations
«You cannot step twice into the same stream.
For as you are stepping in, other waters are ever
flowing on to you.» — Heraclitus

4. Reactive Traits
4

5. Reactive means Message Flow
5

6. Reactive means Message Flow
6
37% discount
with code
kuhnco

7. Distributed vs. Static Typing
• in general a very tough problem
• independent entities evolving concurrently
• promising progress on Project Gålbma
• in many specific cases no type evolution necessary
• letting uniform messages flow in safe conduits
• making streams even more interesting
7

8. Possible Solutions
• the Traditional way: blocking calls
8

9. Possible Solutions
• the Push way: buffering and/or dropping
(optimized for fast consumer)
9

10. Possible Solutions
• the Pull way: poor performance
(optimized for fast producer)
10

11. Possible Solutions
• the Reactive way:
non-blocking & non-dropping & bounded
11

12. Reactive Streams

13. Origin and motivation
• all participants face the same basic problem
• all are building tools for their community
• a common solution benefits everybody
• interoperability to make best use of efforts
• propose to include in future JDK
13

14. Goals
• minimal interfaces—essentials only
• rigorous specification of semantics
• TCK for verification of implementation
• complete freedom for many idiomatic APIs
• specification should be efficiently implementable
14

15. Reactive Streams
• asynchronous & non-blocking
• flow of data
• flow of demand
• minimal coordination and contention
• message passing allows for distribution across
• applications, nodes, CPUs, threads, actors
15

16. A Data Market using Supply & Demand
• data elements flow downstream
• demand flows upstream
• data elements flow only when there is demand
• data in flight is bounded by signaled demand
• recipient is in control of maximal incoming data rate
16
Publisher Subscriber
data
demand

17. Reactive Push–Pull
• “push”—when consumer is faster
• “pull”—when producer is faster
• switches automatically between these
• batching demand allows batching data
17
Publisher Subscriber
data
demand

18. Explicit Demand: One-to-many
18
demand
data
Splitting the data means merging the demand

19. Explicit Demand: Many-to-one
19
Merging the data means splitting the demand

20. Back-Pressure is Contagious
• C is slow
• B must slow down
• A must slow down
20
CA B

21. • TCP for example has it built-in
Back-Pressure can be Propagated
21
CA B
networkhosts

22. The Meat
22
trait Publisher[T] {
def subscribe(sub: Subscriber[T]): Unit
}
trait Subscription {
def request(n: Long): Unit
def cancel(): Unit
}
trait Subscriber[T] {
def onSubscribe(s: Subscription): Unit
def onNext(e: T): Unit
def onError(t: Throwable): Unit
def onComplete(): Unit
}
Not user-level API
Intended as SPI for interop

23. Akka Streams

24. Declaring a Stream Topology
24

25. Declaring a Stream Topology
25

26. Declaring a Stream Topology
26

27. Declaring a Stream Topology
27

28. Declaring a Stream Topology
28

29. Declaring a Stream Topology
29

30. Declaring a Stream Topology
30

31. API Design
• goals:
• no magic
• supreme compositionality
• exhaustive model of bounded stream processing
• consequences:
• immutable and reusable stream blueprints
• explicit materialization step
31
http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M2/stream-design.html

32. Declaring and Running a Stream
32
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () => Tick)
val source = Source[(Int, Tick)]() { implicit b =>
val zip = Zip[Int, Tick]
val out = UndefinedSink[(Int, Tick)]
upper ~> zip.left ~> out
lower ~> zip.right
out
}
val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" }
val sink = Sink.foreach(println)
val future = source.connect(flow).runWith(sink)

33. Declaring and Running a Stream
33
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () => Tick)
val source = Source[(Int, Tick)]() { implicit b =>
val zip = Zip[Int, Tick]
val out = UndefinedSink[(Int, Tick)]
upper ~> zip.left ~> out
lower ~> zip.right
out
}
val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" }
val sink = Sink.foreach(println)
val future = source.connect(flow).runWith(sink)

34. Declaring and Running a Stream
34
val upper = Source(Iterator from 0).take(10)
val lower = Source(1.second, 1.second, () => Tick)
val source = Source[(Int, Tick)]() { implicit b =>
val zip = Zip[Int, Tick]
val out = UndefinedSink[(Int, Tick)]
upper ~> zip.left ~> out
lower ~> zip.right
out
}
val flow = Flow[(Int, Tick)].map{ case (x, _) => s"tick $x" }
val sink = Sink.foreach(println)
val future = source.connect(flow).runWith(sink)

35. Materialization
• Akka Streams separate the what from the how
• declarative Source/Flow/Sink DSL to create blueprint
• FlowMaterializer turns this into running Actors
• this allows alternative materialization strategies
• optimization
• verification / validation
• cluster deployment
• only Akka Actors for now, but more to come!
35

36. Stream Sources
• org.reactivestreams.Publisher[T]
• org.reactivestreams.Subscriber[T]
• Iterator[T] / Iterable[T]
• Code block (function that produces Option[T])
• scala.concurrent.Future[T]
• TickSource
• ActorPublisher
• singleton / empty / failed
• … plus write your own (fully extensible)
36

37. Stream Sinks
• org.reactivestreams.Publisher[T]
• org.reactivestreams.Subscriber[T]
• ActorSubscriber
• scala.concurrent.Future[T]
• blackhole / foreach / fold / onComplete
• … or create your own
37

38. Linear Stream Transformations
• Deterministic (like for collections)
• map, filter, collect, grouped, drop, take, groupBy, …
• Time-Based
• takeWithin, dropWithin, groupedWithin, …
• Rate-Detached
• expand, conflate, buffer, …
• asynchronous
• mapAsync, mapAsyncUnordered, flatten, …
38

39. Nonlinear Stream Transformations
• Fan-In
• merge, concat, zip, …
• Fan-Out
• broadcast, route, balance, unzip, …
39

40. Akka HTTP

41. Why do we add an HTTP module?
• Akka is about building distributed applications
• distribution implies integration
• between internal (sub)systems
➜ akka-clusterbasedonakka-remote
• with external systems
➜ akka-http(linguafrancaoftheinternet)
41

42. Akka HTTP—The Origins: Spray.IO
• Fully embeddable HTTP stack based on Actors
• focused on HTTP integration (toolkit)
• server- and client-side
• seamlessly integrated with Akka
42

43. Spray.IO Features
• immutable, case class-based HTTP model
• fast, lightweight HTTP client and server
• powerful DSL for server-side API definition
• fully async & non-blocking, actor-friendly,
modular, testable
• based entirely on Scala & Akka Actors
43

44. Spray.IO Weaknesses
• handling of chunked requests is clunky, incomplete
• dealing with large message entities can be difficult
• some unintuitive corner-cases in routing DSL
• deep implicit argument chains in some cases hard
to debug
• missing features, foremost websocket support
44

45. Proxying Large Responses
45

46. Akka HTTP is Spray 2.0
• addressing the weaknesses, polishing the features
• Java API
• simplified module structure
• core improvement: fully stream-based
46

47. HTTP Stream Topology
47

48. The Application Stack
48
O/S-level network stack
Java NIO (JDK)
Akka IO
Akka HTTP Core
Akka HTTP
user-
level

49. Akka IO
• bridges Java NIO with Akka Actors or streams
• supports TCP, UDP and SSL
49

50. Akka HTTP Core
• implements HTTP/1.1 according to the RFCs
• based upon the TCP facilities of Akka IO
• exposed as freely reusable stream transformations
• low-level and extensible implementation of
HTTP model and spec
50

51. A Quick View of the HTTP Model
51
case class HttpRequest(
method: HttpMethod = HttpMethods.GET,
uri: Uri = Uri./,
headers: immutable.Seq[HttpHeader] = Nil,
entity: RequestEntity = HttpEntity.Empty,
protocol: HttpProtocol = HttpProtocols.`HTTP/1.1`
) extends HttpMessage
case class HttpResponse(
status: StatusCode = StatusCodes.OK,
headers: immutable.Seq[HttpHeader] = Nil,
entity: ResponseEntity = HttpEntity.Empty,
protocol: HttpProtocol = HttpProtocols.`HTTP/1.1`
) extends HttpMessage

52. Akka HTTP
• builds upon Akka HTTP Core
• adds (un)marshaling for HttpEntities
• adds (de)compression (gzip / deflate)
• high-level server-side routing DSL
• type-safe and flexible
• highly compositional building blocks (Directives)
• includes common behaviors pre-canned
52

53. Stream Pipelines
53
TCP SSL HTTP App
Requests
Responses
optional
(not yet)

54. A Simple Server Example
54
import Directives._
val binding = Http().bind("localhost", 8080)
binding startHandlingWith {
path("order" / HexIntNumber) { id =>
get {
complete(s"Received GET for order $id")
} ~
put {
complete((actor ? Put(id)).mapTo[String])
}
}
}

55. Summary

56. When can we have it?
• currently pre-release versions:
• reactive-streams 1.0.0-RC1
• Akka Streams & HTTP 1.0-M2
• still missing:
• Akka HTTP client features
• SSL integration
• websockets
• Akka Streams & HTTP 1.0 expected end of Feb’15
56

57. Resources
• https://github.com/akka/akka/tree/release-2.3-dev
• http://doc.akka.io/docs/akka-stream-and-http-
experimental/1.0-M2/
• akka-user mailing list
57

58. ©Typesafe 2014 – All Rights Reserved