This document discusses Akka Streams including flows, operator fusion, and the graph DSL. It provides examples of using flows to perform transformations on streaming data between a source and sink. Operator fusion is described as optimizing performance by mitigating the cost of passing elements between stages. The graph DSL allows modeling data flows that can't be represented linearly, such as those involving fan-in and fan-out patterns. Partial graphs are discussed as reusable components that can be used to construct graphs according to different requirements.

1. By:
Harmeet Singh (Taara)
Sr. Software Consultant

2. Agenda
1. Flows
2. Operator Fusion.
3. Graph DSL.
4. Leftover.

3. Akka Stream: Flows
The way to perform transformations to the data as it
flows between Source and Sink.

4. Akka Stream: Flows

5. Akka Stream: Example
object StreamingCopyApp extends App {
val spath = Paths.get("/home/harmeet/akka­stream/knolx.log")
val source: Source[ByteString, Future[IOResult]] =
FileIO.fromPath(spath)
val dpath = Paths.get("/home/harmeet/akka­stream­copy")
val sink: Sink[ByteString, Future[IOResult]] = FileIO.toPath(dpath)
val runnableGraph: RunnableGraph[Future[IOResult]] = source.to(sink)
implicit val system = ActorSystem("akka­stream")
implicit val ec = system.dispatcher
implicit val materializer = ActorMaterializer()
runnableGraph.run().foreach { result =>
println(s"${result.status}, ${result.count} bytes read")
system.terminate()
}
}

6. Akka Stream: Flow Example
val frame: Flow[ByteString, String, NotUsed] =
Framing.delimiter(ByteString("n"), 10240).map(_.decodeString("UTF8"))
val parse: Flow[String, Event, NotUsed] = Flow[String].map(Event.parsing)
val filter: Flow[Event, Event, NotUsed] = Flow[Event].filter(_.log ==
"DEBUG")
val serialize: Flow[Event, ByteString, NotUsed] = Flow[Event].map{ event =>
ByteString(event.toJson.compactPrint)
}
val composedFlow: Flow[ByteString, ByteString, NotUsed] = frame.via(parse)
.via(filter)
.via(serialize)
val runnableGraph: RunnableGraph[Future[IOResult]] =
source.via(composedFlow).toMat(sink) (Keep.right)

7. Akka Stream: Flow Example
val composedFlow: Flow[ByteString, ByteString, NotUsed] =
Framing.delimiter(ByteString("n"), 10240)
.map(_.decodeString("UTF8"))
.map(Event.parsing)
.filter(_.log == "DEBUG")
.map { event =>
ByteString(event.toJson.compactPrint)
}
val runnableGraph: RunnableGraph[Future[IOResult]] =
source.via(composedFlow)
.toMat(sink) (Keep.right)

8. Akka Stream: Operator Fusion
Operator Fusion is used to performance optimization to
mitigate the cost of passing elements between the
different stages.
Two main consequences are:
1. Passing elements fused stages faster.
2. Fused stages no longer run in parallel.

9. Akka Stream: Operator Fusion
Q. How can we execute fused stages in parallel???
Ans:
1. By using async method
2. Turn off auto-fusing (akka.stream.materializer.auto­
fusing=off)
Example:
Source(List(1, 2, 3)).map(_ + 1)
.async
.map(_ * 2).to(Sink.ignore)

10. Akka Stream: Operator Fusion

11. Akka Stream: Graph DSL
What if your data flow cannot be modeled as a simple linear process?
What if process is better modeled as computation graph instead?

12. Akka Stream: Graph DSL
The Graph DSL is kind of diagram ASCII art – in many cases
you could translate a white board diagram of a graph into
the DSL.
A graph problem is one that typically involves the concept of
fanning out and fanning in.
●
Fanning out: Single input to multiple output.
●
Fanning in: Multiple inputs single output.

13. Akka Stream: Graph DSL Example
val in = Source(1 to 5)
val f1 = Flow[Int].map(_ * 2)
val f2 = Flow[Int].map(_ * 1)

14. Akka Stream: Graph DSL Example
val f3 = Flow[Int].map(_ * 2)
val f4 = Flow[Int].map(_ + 1)
val out = Sink.foreach[Int](println)

15. Akka Stream: Graph DSL Example
val bcast = builder.add(Broadcast[Int](2))
val merge = builder.add(Merge[Int](2))

16. Akka Stream: Graph DSL Example
in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out

17. Akka Stream: Graph DSL Example
bcast ~> f3 ~> merge

18. Akka Stream: Graph DSL Example
val graph = RunnableGraph.fromGraph(GraphDSL.create() {
implicit builder: GraphDSL.Builder[NotUsed] =>
import GraphDSL.Implicits._
val in = Source(1 to 5)
val out = Sink.foreach[Int](println)
val f1 = Flow[Int].map(_ * 2)
val f2 = Flow[Int].map(_ * 1)
val f3 = Flow[Int].map(_ * 2)
val f4 = Flow[Int].map(_ + 1)
val bcast = builder.add(Broadcast[Int](2))
val merge = builder.add(Merge[Int](2))
in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out
bcast ~> f3 ~> merge
ClosedShape
})

19. Akka Stream: Graph DSL ClosedShape
Thegraph itself wasfully self-contained and complete.
OR
It doesnot haveany open input port and output port like
aclosed circuit.

20. Akka Stream: Partial Graphs
Whenever weneed to createareusablecomponent for
graph, which wecan useaccording to requirementswith
existing or new graph.
https://doc.akka.io/docs/akka/2.5/scala/stream/stream-graphs.html#constructing-and-combinin

21. Leftovers
➔ API’s
➔ Error Handling.
➔ Streaming HTTP
➔ Handling IO Stream
➔ Unit Testing

22. References:
✔ AkkainAction by Raymond Roestenburg, Rob Bakker, Rob Williams.
✔ MasteringAkkaby Christian Baxter
✔ https://doc.akka.io/api/akka/current/akka/stream/index.html?_ga=2.25006541