Akka Streams: Module on top of Akka to make the ingestion and processing of streams easier, using the actor model under the hood.

1. By:
Harmeet Singh (Taara)
Sr. Software Consultant

2. Agenda
1. What are Streams.
2. Challenges using Streams.
3. Goals of Streams Application.
4. Akka Streams.

3. What are Streams ???
Stream of elements that could have not end.
Or
Conceptually, a stream is transient in that it only exist
as long as there producer providing elements to the
stream and a consumer reading elements from the
stream.

4. Challenges using Streams.
✔ We don’t know how much data we will handle because
data may be produce any time.
✔ Speeds of producer and consumer varying in streams.

5. Goals of Stream Application.
✔ Bounded Memory Use: Never read all elements into
memory. Possibly collect elements into temporary
buffer.
✔ Asynchronous Non Blocking: Resources should be
used efficiently and blocking threads should be limited
as much as possible.
✔ Varying Speed: Producer and consumer should be
able to operate on different speed.

6. Akka Stream
✔ Akka Stream is a module on top of Akka to make the
ingestion and processing of streams easier, using the
actor model under the hood.
✔ Akka Stream provide a way to handle unbounded
stream with bounded buffer.

7. Akka Stream: Processing

8. Akka Stream
Akka Stream usually involve in two steps
✔ Define Blueprint: A graph of streaming components.
The graph define how stream need to processed.
✔ Execute The Blueprint: Run the graph on an
ActorSystem. The graph is turned into actors that do
all the work required to actually stream the data.

9. Akka Stream: Example
object StreamingCopyApp extends App {
val spath = Paths.get("/home/harmeet/akka­stream/knolx.log")
val source: Source[ByteString, Future[IOResult]] =
FileIO.fromPath(spath)
val dpath = Paths.get("/home/harmeet/akka­stream­copy")
val sink: Sink[ByteString, Future[IOResult]] = FileIO.toPath(dpath)
val runnableGraph: RunnableGraph[Future[IOResult]] = source.to(sink)
implicit val system = ActorSystem("akka­stream")
implicit val ec = system.dispatcher
implicit val materializer = ActorMaterializer()
runnableGraph.run().foreach { result =>
println(s"${result.status}, ${result.count} bytes read")
system.terminate()
}
}

10. Akka Stream: Materializer
✔ Containstheconfigurableoptionson how to run thestream.
✔ Help usto decidewhat dispatcher isto useand how to sizetheinput and
output buffers.
✔ It hepsusto convertstheRunnableGraph into actors, which execute
thegraph.
When will AkkaStream start
Materializing???
Answer: runnableGraph.run()

11. Akka Stream: Play the Example
✔ ActorMaterializer checksif theSource and Sink in thegraph
areproperly connected, and requeststheSource and Sink internal to
setup resources.
✔ Internally fromPath create aFileSource(akka) and
FileSource(akka) is asked to create its resources and creates
a FilePublisher(akka) actor and open
FileChannel(java.nio).

12. Akka Stream: Play the Example
✔ Internally toPath create a Sink from SinkModule(akka) and
FileSink(akka) creates a FileSubscriber(akka) actor and open
FileChannel(java.nio).
✔ Internally to method is used to combines the module of Source and
Sink together into one module.

13. Akka Stream: Play the Example
✔ Now ActorMaterializer subscribes, subscribers to publishers
according to how modules are connected, like in our case
FileSubscriber to the FilePublisher.
✔ Now, the FilePublisher reads any ByteStrings from the file until
it reaches ends and FileSubscriber writes any ByteStrings it
receives from the FilePublisher to the output file.

14. Akka Stream: Play the Example
✔ The FilePublisher completes the stream once it has read all the
data from the file and The FileSubscriber receives and
OnComplete message when this happens and close the file that was
writter to.

15. Akka Stream: Combining Materialized
val runnableGraph: RunnableGraph[Future[IOResult]] = source.to(sink)

16. Akka Stream: Combining Materialized
val runnableGraph: RunnableGraph[Future[IOResult]] =
source.toMat(sink)(Keep.left)
val runnableGraph: RunnableGraph[Future[IOResult]] =
source.toMat(sink)(Keep.right)
val graphBoth: RunnableGraph[(Future[IOResult], Future[IOResult])] =
source.toMat(sink) (Keep.both)

17. Akka Stream: Back Pressure
Back Pressure is a simple protocol that allows the consumer to
requests, how my data it can consume at a time, effectively
provide a feedback channel to producer.

18. Akka Stream: Back Pressure
When will Back Pressure comes into picture ???
Ans: Stream Pull-Push Model (Slow Subscriber & Fast Producer)
Rules for Publisher:
✔ Stop generating elements. (Not all publisher supports)
✔ Buffer the elements in a bounded manner until more demands
shows up.
✔ Drop elements until more demands shows up.
✔ Stop the stream if it can’t take any of the preceding actions.

19. Akka Stream: Internal Buffers
In AkkaStream, instead of requesting and publishing every single
element, internally batches are requested and published.
Request Strategy
✔ OneByOneRequestStrategy: Requests one more element when
remainingRequested is 0.
✔ ZeroRequestStrategy: When request is only controlled with manual calls.
✔ WatermarkRequestStrategy: Requests up to the highWatermark when
the remainingRequested is below the lowWatermark.

20. Akka Stream: Internal Buffers
What strategy are used by our example ???
Ans: WatermarkRequestStrategy
Requests up to the highWatermark when the remainingRequested is
below the lowWatermark.
Default Values:
✔ Chunk Read Size: 8KB
✔ Maximum Input Buffer Size: 16 Elements
Customize Values:
✔ akka.stream.materializer.max­input­buffer­size
✔ ActorMaterializerSettings

21. References:
✔ AkkainAction by Raymond Roestenburg, Rob Bakker, Rob Williams.
✔ MasteringAkkaby Christian Baxter
✔ https://doc.akka.io/api/akka/current/akka/stream/index.html?_ga=2.25006541