Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Akka Streams [Part-II]

1,267 views

Published on

Part I - Advanced concepts of Akka Streams.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Introduction to Akka Streams [Part-II]

  1. 1. By: Harmeet Singh (Taara) Sr. Software Consultant
  2. 2. Agenda 1. Flows 2. Operator Fusion. 3. Graph DSL. 4. Leftover.
  3. 3. Akka Stream: Flows The way to perform transformations to the data as it flows between Source and Sink.
  4. 4. Akka Stream: Flows
  5. 5. Akka Stream: Example object StreamingCopyApp extends App {   val spath = Paths.get("/home/harmeet/akka­stream/knolx.log")   val source: Source[ByteString, Future[IOResult]] =  FileIO.fromPath(spath)   val dpath = Paths.get("/home/harmeet/akka­stream­copy")   val sink: Sink[ByteString, Future[IOResult]] = FileIO.toPath(dpath)   val runnableGraph: RunnableGraph[Future[IOResult]] = source.to(sink)   implicit val system = ActorSystem("akka­stream")   implicit val ec = system.dispatcher   implicit  val materializer = ActorMaterializer()   runnableGraph.run().foreach { result =>     println(s"${result.status}, ${result.count} bytes read")     system.terminate()   } }
  6. 6. Akka Stream: Flow Example val frame: Flow[ByteString, String, NotUsed] =  Framing.delimiter(ByteString("n"), 10240).map(_.decodeString("UTF8")) val parse: Flow[String, Event, NotUsed] = Flow[String].map(Event.parsing) val filter: Flow[Event, Event, NotUsed] = Flow[Event].filter(_.log ==  "DEBUG") val serialize: Flow[Event, ByteString, NotUsed] = Flow[Event].map{ event =>     ByteString(event.toJson.compactPrint) } val composedFlow: Flow[ByteString, ByteString, NotUsed] = frame.via(parse)     .via(filter)     .via(serialize) val runnableGraph: RunnableGraph[Future[IOResult]] =  source.via(composedFlow).toMat(sink) (Keep.right)
  7. 7. Akka Stream: Flow Example val composedFlow: Flow[ByteString, ByteString, NotUsed] =  Framing.delimiter(ByteString("n"), 10240)     .map(_.decodeString("UTF8"))     .map(Event.parsing)     .filter(_.log == "DEBUG")     .map { event =>       ByteString(event.toJson.compactPrint)     } val runnableGraph: RunnableGraph[Future[IOResult]] =  source.via(composedFlow)     .toMat(sink) (Keep.right)
  8. 8. Akka Stream: Operator Fusion Operator Fusion is used to performance optimization to mitigate the cost of passing elements between the different stages. Two main consequences are: 1. Passing elements fused stages faster. 2. Fused stages no longer run in parallel.
  9. 9. Akka Stream: Operator Fusion Q. How can we execute fused stages in parallel??? Ans: 1. By using async method 2. Turn off auto-fusing (akka.stream.materializer.auto­ fusing=off) Example: Source(List(1, 2, 3)).map(_ + 1) .async .map(_ * 2).to(Sink.ignore)
  10. 10. Akka Stream: Operator Fusion
  11. 11. Akka Stream: Graph DSL What if your data flow cannot be modeled as a simple linear process? What if process is better modeled as computation graph instead?
  12. 12. Akka Stream: Graph DSL The Graph DSL is kind of diagram ASCII art – in many cases you could translate a white board diagram of a graph into the DSL. A graph problem is one that typically involves the concept of fanning out and fanning in. ● Fanning out: Single input to multiple output. ● Fanning in: Multiple inputs single output.
  13. 13. Akka Stream: Graph DSL Example val in = Source(1 to 5)          val f1 = Flow[Int].map(_ * 2)    val f2 = Flow[Int].map(_ * 1)
  14. 14. Akka Stream: Graph DSL Example val f3 = Flow[Int].map(_ * 2)         val f4 = Flow[Int].map(_ + 1) val out = Sink.foreach[Int](println)
  15. 15. Akka Stream: Graph DSL Example val bcast = builder.add(Broadcast[Int](2)) val merge = builder.add(Merge[Int](2))
  16. 16. Akka Stream: Graph DSL Example  in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out
  17. 17. Akka Stream: Graph DSL Example        bcast ~> f3 ~> merge
  18. 18. Akka Stream: Graph DSL Example val graph = RunnableGraph.fromGraph(GraphDSL.create() {     implicit builder: GraphDSL.Builder[NotUsed] =>       import GraphDSL.Implicits._       val in = Source(1 to 5)       val out = Sink.foreach[Int](println)       val f1 = Flow[Int].map(_ * 2)       val f2 = Flow[Int].map(_ * 1)       val f3 = Flow[Int].map(_ * 2)       val f4 = Flow[Int].map(_ + 1)       val bcast = builder.add(Broadcast[Int](2))       val merge = builder.add(Merge[Int](2))       in ~> f1 ~> bcast ~> f2 ~> merge ~> f4 ~> out       bcast ~> f3 ~> merge       ClosedShape   })
  19. 19. Akka Stream: Graph DSL ClosedShape Thegraph itself wasfully self-contained and complete. OR It doesnot haveany open input port and output port like aclosed circuit.
  20. 20. Akka Stream: Partial Graphs Whenever weneed to createareusablecomponent for graph, which wecan useaccording to requirementswith existing or new graph. https://doc.akka.io/docs/akka/2.5/scala/stream/stream-graphs.html#constructing-and-combinin
  21. 21. Leftovers ➔ API’s ➔ Error Handling. ➔ Streaming HTTP ➔ Handling IO Stream ➔ Unit Testing
  22. 22. References: ✔ AkkainAction by Raymond Roestenburg, Rob Bakker, Rob Williams. ✔ MasteringAkkaby Christian Baxter ✔ https://doc.akka.io/api/akka/current/akka/stream/index.html?_ga=2.25006541

×