Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Akka Streams [Part-I]


Published on

Akka Streams: Module on top of Akka to make the ingestion and processing of streams easier, using the actor model under the hood.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Introduction to Akka Streams [Part-I]

  1. 1. By: Harmeet Singh (Taara) Sr. Software Consultant
  2. 2. Agenda 1. What are Streams. 2. Challenges using Streams. 3. Goals of Streams Application. 4. Akka Streams.
  3. 3. What are Streams ??? Stream of elements that could have not end. Or Conceptually, a stream is transient in that it only exist as long as there producer providing elements to the stream and a consumer reading elements from the stream.
  4. 4. Challenges using Streams. ✔ We don’t know how much data we will handle because data may be produce any time. ✔ Speeds of producer and consumer varying in streams.
  5. 5. Goals of Stream Application. ✔ Bounded Memory Use: Never read all elements into memory. Possibly collect elements into temporary buffer. ✔ Asynchronous Non Blocking: Resources should be used efficiently and blocking threads should be limited as much as possible. ✔ Varying Speed: Producer and consumer should be able to operate on different speed.
  6. 6. Akka Stream ✔ Akka Stream is a module on top of Akka to make the ingestion and processing of streams easier, using the actor model under the hood. ✔ Akka Stream provide a way to handle unbounded stream with bounded buffer.
  7. 7. Akka Stream: Processing
  8. 8. Akka Stream Akka Stream usually involve in two steps ✔ Define Blueprint: A graph of streaming components. The graph define how stream need to processed. ✔ Execute The Blueprint: Run the graph on an ActorSystem. The graph is turned into actors that do all the work required to actually stream the data.
  9. 9. Akka Stream: Example object StreamingCopyApp extends App {   val spath = Paths.get("/home/harmeet/akka­stream/knolx.log")   val source: Source[ByteString, Future[IOResult]] =  FileIO.fromPath(spath)   val dpath = Paths.get("/home/harmeet/akka­stream­copy")   val sink: Sink[ByteString, Future[IOResult]] = FileIO.toPath(dpath)   val runnableGraph: RunnableGraph[Future[IOResult]] =   implicit val system = ActorSystem("akka­stream")   implicit val ec = system.dispatcher   implicit  val materializer = ActorMaterializer() { result =>     println(s"${result.status}, ${result.count} bytes read")     system.terminate()   } }
  10. 10. Akka Stream: Materializer ✔ Containstheconfigurableoptionson how to run thestream. ✔ Help usto decidewhat dispatcher isto useand how to sizetheinput and output buffers. ✔ It hepsusto convertstheRunnableGraph into actors, which execute thegraph. When will AkkaStream start  Materializing??? Answer:
  11. 11. Akka Stream: Play the Example ✔ ActorMaterializer checksif theSource and Sink in thegraph areproperly connected, and requeststheSource and Sink internal to setup resources. ✔ Internally fromPath create aFileSource(akka) and FileSource(akka) is asked to create its resources and creates a FilePublisher(akka) actor and open FileChannel(java.nio).
  12. 12. Akka Stream: Play the Example ✔ Internally toPath create a Sink from SinkModule(akka) and FileSink(akka) creates a FileSubscriber(akka) actor and open FileChannel(java.nio). ✔ Internally to method is used to combines the module of Source and Sink together into one module.
  13. 13. Akka Stream: Play the Example ✔ Now ActorMaterializer subscribes, subscribers to publishers according to how modules are connected, like in our case FileSubscriber to the FilePublisher. ✔ Now, the FilePublisher reads any ByteStrings from the file until it reaches ends and FileSubscriber writes any ByteStrings it receives from the FilePublisher to the output file.
  14. 14. Akka Stream: Play the Example ✔ The FilePublisher completes the stream once it has read all the data from the file and The FileSubscriber receives and OnComplete message when this happens and close the file that was writter to.
  15. 15. Akka Stream: Combining Materialized   val runnableGraph: RunnableGraph[Future[IOResult]] =
  16. 16. Akka Stream: Combining Materialized   val runnableGraph: RunnableGraph[Future[IOResult]] =  source.toMat(sink)(Keep.left)   val runnableGraph: RunnableGraph[Future[IOResult]] =  source.toMat(sink)(Keep.right)   val graphBoth: RunnableGraph[(Future[IOResult], Future[IOResult])] =  source.toMat(sink) (Keep.both)
  17. 17. Akka Stream: Back Pressure Back Pressure is a simple protocol that allows the consumer to requests, how my data it can consume at a time, effectively provide a feedback channel to producer.
  18. 18. Akka Stream: Back Pressure When will Back Pressure comes into picture ??? Ans: Stream Pull-Push Model (Slow Subscriber & Fast Producer) Rules for Publisher: ✔ Stop generating elements. (Not all publisher supports) ✔ Buffer the elements in a bounded manner until more demands shows up. ✔ Drop elements until more demands shows up. ✔ Stop the stream if it can’t take any of the preceding actions.
  19. 19. Akka Stream: Internal Buffers In AkkaStream, instead of requesting and publishing every single element, internally batches are requested and published. Request Strategy ✔ OneByOneRequestStrategy: Requests one more element when remainingRequested is 0. ✔ ZeroRequestStrategy: When request is only controlled with manual calls. ✔ WatermarkRequestStrategy: Requests up to the highWatermark when the remainingRequested is below the lowWatermark.
  20. 20. Akka Stream: Internal Buffers What strategy are used by our example ??? Ans: WatermarkRequestStrategy Requests up to the highWatermark when the remainingRequested is below the lowWatermark. Default Values: ✔ Chunk Read Size: 8KB ✔ Maximum Input Buffer Size: 16 Elements Customize Values: ✔­input­buffer­size ✔ ActorMaterializerSettings
  21. 21. References: ✔ AkkainAction by Raymond Roestenburg, Rob Bakker, Rob Williams. ✔ MasteringAkkaby Christian Baxter ✔