Johan Andrén, profile picture
Uploaded byJohan Andrén
PDF, PPTX1,944 views

Reactive streams processing using Akka Streams

This document provides a summary of a presentation about Akka streams. It discusses Reactive Streams, which is a standard for asynchronous stream processing with non-blocking back pressure. It provides examples of building stream processing graphs with Akka Streams in just a few lines of code to transform and process data from sources to sinks. It also gives a more complex example of processing sensor measurements from a web socket connection and writing the data to a database.

Embed presentation

Download as PDF, PPTX
Akka streams Johan Andrén JavaZone, Oslo, 2017-09-13 Reactive stream processing using
Johan Andrén Akka Team Stockholm Scala User Group @apnylle johan.andren@lightbend.com
Make building powerful concurrent & distributed applications simple. Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM Akka
Actors – simple & high performance concurrency Cluster / Remoting, Cluster tools – location transparency, resilience – and prepackaged tools for building distributed systems Streams – back-pressured stream processing Persistence – CQRS + Event Sourcing for Actors HTTP – fully async streaming HTTP Server Complete Java & Scala APIs for all features What’s in the toolkit?
Reactive Streams reactive-streams.org
Reactive Streams timeline Oct 2013 RxJava, Akka and Twitter- people meeting “Soon thereafter” 2013 Reactive Streams Expert group formed Apr 2015 Reactive Streams Spec 1.0 TCK 5+ impls 2017 Inclusion in JDK9 Akka Streams, RxJava Vert.x, MongoDB, … Jul 2015 Akka Streams 1.0
Reactive Streams Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org “
Reactive Streams Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org “
Stream processing Source Sink Flow
Reactive Streams Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org “
Asynchronous stream processing Source Sink (possible) asynchronous boundaries Flow
Reactive Streams Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org “
No back pressure Source Sink 10 msg/s 1 msg/s Flow asynchronous boundary
No back pressure Source Sink 10 msg/s 1 msg/s Flow asynchronous boundary OOM!!
No back pressure - bounded buffer Source Sink 10 msg/s 1 msg/s Flow buffer size 6 🗑 asynchronous boundary
Async non blocking back pressure Source Sink 1 msg/s 1 msg/s Flow buffer size 6 🗑 asynchronous boundary Hey! give me 2 more
Reactive Streams RS Library A RS library B async boundary
Reactive Streams “Make building powerful concurrent & distributed applications simple.”
Complete and awesome Java and Scala APIs (Just like everything in Akka) Akka Streams
Akka Streams in ~20 seconds: final ActorSystem system = ActorSystem.create();
 final Materializer materializer = ActorMaterializer.create(system);
 
 final Source<Integer, NotUsed> source =
 Source.range(0, 20000000);
 
 final Flow<Integer, String, NotUsed> flow =
 Flow.fromFunction((Integer n) -> n.toString());
 
 final Sink<String, CompletionStage<Done>> sink =
 Sink.foreach(str -> System.out.println(str));
 
 final RunnableGraph<NotUsed> runnable = source.via(flow).to(sink);
 
 runnable.run(materializer); complete sources on github
Akka Streams in ~20 seconds: final ActorSystem system = ActorSystem.create();
 final Materializer materializer = ActorMaterializer.create(system);
 
 final Source<Integer, NotUsed> source =
 Source.range(0, 20000000);
 
 final Flow<Integer, String, NotUsed> flow =
 Flow.fromFunction((Integer n) -> n.toString());
 
 final Sink<String, CompletionStage<Done>> sink =
 Sink.foreach(str -> System.out.println(str));
 
 final RunnableGraph<NotUsed> runnable = source.via(flow).to(sink);
 
 runnable.run(materializer); complete sources on github
final ActorSystem system = ActorSystem.create();
 final Materializer materializer = ActorMaterializer.create(system);
 
 final Source<Integer, NotUsed> source =
 Source.range(0, 20000000);
 
 final Flow<Integer, String, NotUsed> flow =
 Flow.fromFunction((Integer n) -> n.toString());
 
 final Sink<String, CompletionStage<Done>> sink =
 Sink.foreach(str -> System.out.println(str));
 
 final RunnableGraph<NotUsed> runnable = source.via(flow).to(sink);
 
 runnable.run(materializer); Akka Streams in ~20 seconds: complete sources on github Source (range) String Flow (fromFunction) Sink (foreach) String Integer Integer
Akka Streams in ~20 seconds: implicit val system = ActorSystem()
 implicit val mat = ActorMaterializer()
 
 val source = Source(0 to 20000000)
 
 val flow = Flow[Int].map(_.toString())
 
 val sink = Sink.foreach[String](println(_))
 
 val runnable = source.via(flow).to(sink)
 
 runnable.run() complete sources on github
Akka Streams in ~20 seconds: Source.range(0, 20000000)
 .map(Object::toString)
 .runForeach(str -> System.out.println(str), materializer); complete sources on github
Akka Streams in ~20 seconds: Source(0 to 20000000)
 .map(_.toString)
 .runForeach(println) complete sources on github
Numbers as a service final Source<ByteString, NotUsed> numbers = Source.unfold(0L, n -> {
 long next = n + 1;
 return Optional.of(Pair.create(next, next));
 }).map(n -> ByteString.fromString(n.toString() + "n"));
 
 
 final Route route =
 path("numbers", () ->
 get(() ->
 complete(HttpResponse.create()
 .withStatus(StatusCodes.OK)
 .withEntity(HttpEntities.create(
 ContentTypes.TEXT_PLAIN_UTF8,
 numbers
 )))
 )
 );
 
 final CompletionStage<ServerBinding> bindingCompletionStage =
 http.bindAndHandle(route.flow(system, materializer), host, materializer); complete sources on github
final Source<ByteString, NotUsed> numbers = Source.unfold(0L, n -> {
 long next = n + 1;
 return Optional.of(Pair.create(next, next));
 }).map(n -> ByteString.fromString(n.toString() + "n"));
 
 
 final Route route =
 path("numbers", () ->
 get(() ->
 complete(HttpResponse.create()
 .withStatus(StatusCodes.OK)
 .withEntity(HttpEntities.create(
 Numbers as a service complete sources on github
return Optional.of(Pair.create(next, next));
 }).map(n -> ByteString.fromString(n.toString() + "n"));
 
 
 final Route route =
 path("numbers", () ->
 get(() ->
 complete(HttpResponse.create()
 .withStatus(StatusCodes.OK)
 .withEntity(HttpEntities.create(
 ContentTypes.TEXT_PLAIN_UTF8,
 numbers
 )))
 )
 );
 
 final CompletionStage<ServerBinding> bindingCompletionStage =
 http.bindAndHandle(route.flow(system, materializer), host, materializer); Numbers as a service complete sources on github
Numbers as a service val numbers =
 Source.unfold(0L) { (n) =>
 val next = n + 1
 Some((next, next))
 }.map(n => ByteString(n + "n"))
 
 val route =
 path("numbers") {
 get {
 complete( HttpResponse(entity = HttpEntity(`text/plain(UTF-8)`, numbers)) )
 }
 }
 val futureBinding = Http().bindAndHandle(route, "127.0.0.1", 8080) complete sources on github
recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚚 Back pressure over TCP numbers TCP HTTP Server Client
recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚑 Back pressure over TCP numbers TCP HTTP Backpressure Server Client
recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚑 Back pressure over TCP numbers TCP HTTP Backpressure Backpressure Server Client
A more useful example complete sources on github final Flow<Message, Message, NotUsed> measurementsFlow =
 Flow.of(Message.class)
 .flatMapConcat((Message message) ->
 message.asTextMessage()
 .getStreamedText()
 .fold("", (acc, elem) -> acc + elem)
 )
 .groupedWithin(1000, FiniteDuration.create(1, SECONDS))
 .mapAsync(5, database::asyncBulkInsert)
 .map(written ->
 TextMessage.create("wrote up to: " + written.get(written.size() - 1))
 );
 
 final Route route = path("measurements", () ->
 get(() ->
 handleWebSocketMessages(measurementsFlow)
 )
 );
 
 final CompletionStage<ServerBinding> bindingCompletionStage =
 http.bindAndHandle(route.flow(system, materializer), host, materializer); Credit to: Colin Breck
final Flow<Message, Message, NotUsed> measurementsFlow =
 Flow.of(Message.class)
 .flatMapConcat((Message message) ->
 message.asTextMessage()
 .getStreamedText()
 .fold("", (acc, elem) -> acc + elem)
 )
 .groupedWithin(1000, FiniteDuration.create(1, SECONDS))
 .mapAsync(5, database::asyncBulkInsert)
 .map(written ->
 TextMessage.create("wrote up to: " + written.get(written.size() - 1))
 );
 
 final Route route = path("measurements", () ->
 A more useful example complete sources on github Credit to: Colin Breck
)
 .groupedWithin(1000, FiniteDuration.create(1, SECONDS))
 .mapAsync(5, database::asyncBulkInsert)
 .map(written ->
 TextMessage.create("wrote up to: " + written.get(written.size() - 1))
 );
 
 final Route route = path("measurements", () ->
 get(() ->
 handleWebSocketMessages(measurementsFlow)
 )
 );
 
 final CompletionStage<ServerBinding> bindingCompletionStage =
 http.bindAndHandle(route.flow(system, materializer), host, materializer); A more useful example complete sources on github Credit to: Colin Breck
A more useful example complete sources on github val measurementsFlow =
 Flow[Message].flatMapConcat(message =>
 message.asTextMessage.getStreamedText.fold("")(_ + _)
 )
 .groupedWithin(1000, 1.second)
 .mapAsync(5)(Database.asyncBulkInsert)
 .map(written => TextMessage("wrote up to: " + written.last))
 
 val route =
 path("measurements") {
 get {
 handleWebSocketMessages(measurementsFlow)
 }
 }
 
 val futureBinding = Http().bindAndHandle(route, "127.0.0.1", 8080)
The tale of the two pancake chefs HungrySink Frying Pan BatterSource Scoops of batter Pancakes nom nom nom asynchronous boundaries Roland Patrik
Rolands pipelined pancakes HungrySinkPan 2BatterSource Pan 1 nom nom nom
Rolands pipelined pancakes Flow<ScoopOfBatter, HalfCookedPancake, NotUsed> fryingPan1 =
 Flow.of(ScoopOfBatter.class).map(batter -> new HalfCookedPancake());
 
 Flow<HalfCookedPancake, Pancake, NotUsed> fryingPan2 =
 Flow.of(HalfCookedPancake.class).map(halfCooked -> new Pancake()); Flow<ScoopOfBatter, Pancake, NotUsed> pancakeChef =
 fryingPan1.async().via(fryingPan2.async()); section in docs
Rolands pipelined pancakes // Takes a scoop of batter and creates a pancake with one side cooked val fryingPan1: Flow[ScoopOfBatter, HalfCookedPancake, NotUsed] =
 Flow[ScoopOfBatter].map { batter => HalfCookedPancake() }
 
 // Finishes a half-cooked pancake
 val fryingPan2: Flow[HalfCookedPancake, Pancake, NotUsed] =
 Flow[HalfCookedPancake].map { halfCooked => Pancake() } 
 // With the two frying pans we can fully cook pancakes val pancakeChef: Flow[ScoopOfBatter, Pancake, NotUsed] =
 Flow[ScoopOfBatter].via(fryingPan1.async).via(fryingPan2.async) section in docs
Patriks parallel pancakes HungrySink Pan 2 BatterSource Pan 1 Balance Merge nom nom nom
Patriks parallel pancakes Flow<ScoopOfBatter, Pancake, NotUsed> fryingPan =
 Flow.of(ScoopOfBatter.class).map(batter -> new Pancake());
 
 Flow<ScoopOfBatter, Pancake, NotUsed> pancakeChef =
 Flow.fromGraph(GraphDSL.create(builder -> {
 final UniformFanInShape<Pancake, Pancake> mergePancakes =
 builder.add(Merge.create(2));
 final UniformFanOutShape<ScoopOfBatter, ScoopOfBatter> dispatchBatter =
 builder.add(Balance.create(2));
 
 builder.from(dispatchBatter.out(0)) .via(builder.add(fryingPan.async())) .toInlet(mergePancakes.in(0));
 builder.from(dispatchBatter.out(1)) .via(builder.add(fryingPan.async())) .toInlet(mergePancakes.in(1));
 
 return FlowShape.of(dispatchBatter.in(), mergePancakes.out());
 })); section in docs
Patriks parallel pancakes val pancakeChef: Flow[ScoopOfBatter, Pancake, NotUsed] =
 Flow.fromGraph(GraphDSL.create() { implicit builder =>
 import GraphDSL.Implicits._ 
 val dispatchBatter = builder.add(Balance[ScoopOfBatter](2))
 val mergePancakes = builder.add(Merge[Pancake](2))
 
 // Using two pipelines, having two frying pans each, in total using
 // four frying pans
 dispatchBatter.out(0) ~> fryingPan1.async ~> fryingPan2.async ~> mergePancakes.in(0)
 dispatchBatter.out(1) ~> fryingPan1.async ~> fryingPan2.async ~> mergePancakes.in(1)
 
 FlowShape(dispatchBatter.in, mergePancakes.out)
 }) section in docs
Making pancakes together HungrySink Pan 3 BatterSource Pan 1 Balance Merge Pan 2 Pan 4 nom nom nom
Built in stages Flow stages map/fromFunction, mapConcat, statefulMapConcat, filter, filterNot, collect, grouped, sliding, scan, scanAsync, fold, foldAsync, reduce, drop, take, takeWhile, dropWhile, recover, recoverWith, recoverWithRetries, mapError, detach, throttle, intersperse, limit, limitWeighted, log, recoverWithRetries, mapAsync, mapAsyncUnordered, takeWithin, dropWithin, groupedWithin, initialDelay, delay, conflate, conflateWithSeed, batch, batchWeighted, expand, buffer, prefixAndTail, groupBy, splitWhen, splitAfter, flatMapConcat, flatMapMerge, initialTimeout, completionTimeout, idleTimeout, backpressureTimeout, keepAlive, initialDelay, merge, mergeSorted, Source stages fromIterator, apply, single, repeat, cycle, tick, fromFuture, fromCompletionStage, unfold, unfoldAsync, empty, maybe, failed, lazily, actorPublisher, actorRef, combine, unfoldResource, unfoldResourceAsync, queue, asSubscriber, fromPublisher, zipN, zipWithN Sink stages head, headOption, last, lastOption, ignore, cancelled, seq, foreach, foreachParallel, onComplete, lazyInit, queue, fold, reduce, combine, actorRef, actorRefWithAck, actorSubscriber, asPublisher, fromSubscriber Additional Sink and Source converters {from,as}OutputStream, {from,as}InputStream, {as,from} javaCollector, javaCollectorParallelUnordered File IO Sinks and Sources fromPath, toPath mergePreferred, zip, zipWith, zipWithIndex, concat, prepend, orElse, interleave, unzip, unzipWith, broadcast, balance, partition, watchTermination, monitor Even more Framing, JSON framing, killswitch, BroadcastHub, MergeHub
But I want to connect other things!
A community for Akka Streams connectors http://github.com/akka/alpakka Alpakka
Alpakka – a community for Stream connectors Existing Alpakka MQTT AMQP/ RabbitMQ SSE Cassandra FTP/ SFTP JSON, XML, CSV, RecordIO IronMq Files AWS DynamoDB AWS SNS,SQS, S3, Kinesis,Lambda JMS Azure Storage Queue TCP In Akka Actors Reactive Streams Java Streams Basic File IO External Apache Geode Eventuate FS2 Akka Http HBase http://developer.lightbend.com/docs/alpakka/current/index.html Google Cloud Pub/Sub Camel Kafka MongoDB Azure IoT
But my usecase is a unique snowflake! ❄ ❄ ❄
GraphStage API public class Map<A, B> extends GraphStage<FlowShape<A, B>> {
 private final Function<A, B> f;
 public final Inlet<A> in = Inlet.create("Map.in");
 public final Outlet<B> out = Outlet.create("Map.out"); private final FlowShape<A, B> shape = FlowShape.of(in, out); public Map(Function<A, B> f) {
 this.f = f;
 }
 public FlowShape<A,B> shape() {
 return shape;
 }
 public GraphStageLogic createLogic(Attributes inheritedAttributes) {
 return new GraphStageLogic(shape) {
 {
 setHandler(in, new AbstractInHandler() {
 @Override
 public void onPush() throws Exception {
 push(out, f.apply(grab(in)));
 }
 });
 setHandler(out, new AbstractOutHandler() {
 @Override
 public void onPull() throws Exception {
 pull(in);
 }
 });
 }
 };
 }
 } complete sources on github
public class Map<A, B> extends GraphStage<FlowShape<A, B>> {
 private final Function<A, B> f;
 public final Inlet<A> in = Inlet.create("Map.in");
 public final Outlet<B> out = Outlet.create("Map.out"); private final FlowShape<A, B> shape = FlowShape.of(in, out); public Map(Function<A, B> f) {
 this.f = f;
 }
 public FlowShape<A,B> shape() {
 return shape;
 }
 public GraphStageLogic createLogic(Attributes inheritedAttributes) {
 return new GraphStageLogic(shape) {
 {
 setHandler(in, new AbstractInHandler() {
 @Override
 public void onPush() throws Exception {
 GraphStage API complete sources on github
public FlowShape<A,B> shape() {
 return shape;
 }
 public GraphStageLogic createLogic(Attributes inheritedAttributes) {
 return new GraphStageLogic(shape) {
 {
 setHandler(in, new AbstractInHandler() {
 @Override
 public void onPush() throws Exception {
 push(out, f.apply(grab(in)));
 }
 });
 setHandler(out, new AbstractOutHandler() {
 @Override
 public void onPull() throws Exception {
 pull(in);
 }
 });
 }
 };
 }
 } GraphStage API complete sources on github
GraphStage API class Map[A, B](f: A => B) extends GraphStage[FlowShape[A, B]] {
 
 val in = Inlet[A]("Map.in")
 val out = Outlet[B]("Map.out")
 override val shape = FlowShape.of(in, out)
 
 override def createLogic(attr: Attributes): GraphStageLogic =
 new GraphStageLogic(shape) {
 setHandler(in, new InHandler {
 override def onPush(): Unit = {
 push(out, f(grab(in)))
 }
 })
 setHandler(out, new OutHandler {
 override def onPull(): Unit = {
 pull(in)
 }
 })
 }
 } complete sources on github
When to use what abstraction modelling power complexity actors streams (completable) futures java.concurrency
Actors ❤ Streams = True Actors – perfect for managing state, things with lifecycles, restarting, running many separate instances in parallel A running stream – can be seen as a state, has a lifecycle, is - not yet started, running, failed, stopped, can be stopped using special stages blog.colinbreck.com/integrating-akka-streams-and-akka-actors-part-i/ Again, Colin Breck:
But, distributed systems? Stream-in-Actor C-1 Shard C Akka Streams is a local abstraction Combine with Akka Cluster for distributed superpowers! Stream-in-Actor A-1 Shard A Stream-in-Actor B-1 Shard B Stream-in-Actor B-2
The community Mailing list: https://groups.google.com/group/akka-user Public chat rooms: http://gitter.im/akka/dev developing Akka http://gitter.im/akka/akka using Akka Easy to contribute tickets: https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22 ~200 active contributors!
Spørsmål? @apnylle johan.andren@lightbend.com http://akka.io Akka
Takk! @apnylle johan.andren@lightbend.com github.com/johanandren All sample code (Java & Scala) https://github.com/johanandren/akka-stream-samples/tree/javazone-2017 http://akka.io Akka

More Related Content

PDF
Reactive stream processing using Akka streams
byJohan Andrén
 
PDF
Reactive Applications in Java
byAlexander Mrynskyi
 
PDF
Next generation actors with Akka
byJohan Andrén
 
PDF
Buiilding reactive distributed systems with Akka
byJohan Andrén
 
PDF
Next generation message driven systems with Akka
byJohan Andrén
 
PDF
Next generation message driven systems with Akka
byJohan Andrén
 
PDF
Networks and types - the future of Akka
byJohan Andrén
 
PDF
Streaming all the things with akka streams
byJohan Andrén
 
Reactive stream processing using Akka streams
byJohan Andrén
 
Reactive Applications in Java
byAlexander Mrynskyi
 
Next generation actors with Akka
byJohan Andrén
 
Buiilding reactive distributed systems with Akka
byJohan Andrén
 
Next generation message driven systems with Akka
byJohan Andrén
 
Next generation message driven systems with Akka
byJohan Andrén
 
Networks and types - the future of Akka
byJohan Andrén
 
Streaming all the things with akka streams
byJohan Andrén
 

What's hot

PDF
Akka streams - Umeå java usergroup
byJohan Andrén
 
PDF
Asynchronous stream processing with Akka Streams
byJohan Andrén
 
PDF
Scala usergroup stockholm - reactive integrations with akka streams
byJohan Andrén
 
PDF
Async - react, don't wait - PingConf
byJohan Andrén
 
PDF
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
byKonrad Malawski
 
PPTX
Akka Actor presentation
byGene Chang
 
PDF
Reactive Streams / Akka Streams - GeeCON Prague 2014
byKonrad Malawski
 
PDF
A dive into akka streams: from the basics to a real-world scenario
byGioia Ballin
 
PDF
Reactive integrations with Akka Streams
byKonrad Malawski
 
PDF
VJUG24 - Reactive Integrations with Akka Streams
byJohan Andrén
 
PDF
Building reactive distributed systems with Akka
byJohan Andrén
 
PDF
Journey into Reactive Streams and Akka Streams
byKevin Webber
 
PPTX
Introduction to rx java for android
byEsa Firman
 
PDF
Reactive Streams: Handling Data-Flow the Reactive Way
byRoland Kuhn
 
PDF
End to End Akka Streams / Reactive Streams - from Business to Socket
byKonrad Malawski
 
PPTX
Developing distributed applications with Akka and Akka Cluster
byKonstantin Tsykulenko
 
PDF
Reactive programming with RxJava
byJobaer Chowdhury
 
PDF
Resilient Applications with Akka Persistence - Scaladays 2014
byBjörn Antonsson
 
PPTX
Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...
byCodemotion
 
PDF
Using akka streams to access s3 objects
byMikhail Girkin
 
Akka streams - Umeå java usergroup
byJohan Andrén
 
Asynchronous stream processing with Akka Streams
byJohan Andrén
 
Scala usergroup stockholm - reactive integrations with akka streams
byJohan Andrén
 
Async - react, don't wait - PingConf
byJohan Andrén
 
Fresh from the Oven (04.2015): Experimental Akka Typed and Akka Streams
byKonrad Malawski
 
Akka Actor presentation
byGene Chang
 
Reactive Streams / Akka Streams - GeeCON Prague 2014
byKonrad Malawski
 
A dive into akka streams: from the basics to a real-world scenario
byGioia Ballin
 
Reactive integrations with Akka Streams
byKonrad Malawski
 
VJUG24 - Reactive Integrations with Akka Streams
byJohan Andrén
 
Building reactive distributed systems with Akka
byJohan Andrén
 
Journey into Reactive Streams and Akka Streams
byKevin Webber
 
Introduction to rx java for android
byEsa Firman
 
Reactive Streams: Handling Data-Flow the Reactive Way
byRoland Kuhn
 
End to End Akka Streams / Reactive Streams - from Business to Socket
byKonrad Malawski
 
Developing distributed applications with Akka and Akka Cluster
byKonstantin Tsykulenko
 
Reactive programming with RxJava
byJobaer Chowdhury
 
Resilient Applications with Akka Persistence - Scaladays 2014
byBjörn Antonsson
 
Samuele Resca - REACTIVE PROGRAMMING, DAMN. IT IS NOT ABOUT REACTJS - Codemot...
byCodemotion
 
Using akka streams to access s3 objects
byMikhail Girkin
 

Similar to Reactive streams processing using Akka Streams

PDF
Exploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka
byLightbend
 
PDF
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
byKonrad Malawski
 
PDF
Akka Streams and HTTP
byRoland Kuhn
 
PDF
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...
byLightbend
 
PPTX
Reactive Streams - László van den Hoek
byRubiX BV
 
PDF
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
byKonrad Malawski
 
PDF
Akka stream and Akka CQRS
byMilan Das
 
PDF
Understanding Akka Streams, Back Pressure, and Asynchronous Architectures
byLightbend
 
PDF
Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...
byLightbend
 
PDF
Not Only Streams for Akademia JLabs
byKonrad Malawski
 
PPTX
Intro to Akka Streams
byMichael Kendra
 
PDF
Reactive Streams 1.0 and Akka Streams
byDean Wampler
 
PDF
Reactive Stream Processing with Akka Streams
byKonrad Malawski
 
PDF
Functional Programming and Composing Actors
bylegendofklang
 
PDF
[PDF Download] Akka in Action 1st Edition Raymond Roestenburg fulll chapter
bylabliinagai
 
PDF
Akka in Action 1st Edition Raymond Roestenburg download pdf
byhimlalnunys
 
ODP
Introduction to Akka Streams [Part-I]
byKnoldus Inc.
 
PDF
Akka Streams - From Zero to Kafka
byMark Harrison
 
PDF
Writing Asynchronous Programs with Scala & Akka
byYardena Meymann
 
PPTX
Taking Akka Streams & Akka Http to Large Scale Production Applications
byAkara Sucharitakul
 
Exploring Reactive Integrations With Akka Streams, Alpakka And Apache Kafka
byLightbend
 
Building a Reactive System with Akka - Workshop @ O'Reilly SAConf NYC
byKonrad Malawski
 
Akka Streams and HTTP
byRoland Kuhn
 
Build Real-Time Streaming ETL Pipelines With Akka Streams, Alpakka And Apache...
byLightbend
 
Reactive Streams - László van den Hoek
byRubiX BV
 
[Tokyo Scala User Group] Akka Streams & Reactive Streams (0.7)
byKonrad Malawski
 
Akka stream and Akka CQRS
byMilan Das
 
Understanding Akka Streams, Back Pressure, and Asynchronous Architectures
byLightbend
 
Akka A to Z: A Guide To The Industry’s Best Toolkit for Fast Data and Microse...
byLightbend
 
Not Only Streams for Akademia JLabs
byKonrad Malawski
 
Intro to Akka Streams
byMichael Kendra
 
Reactive Streams 1.0 and Akka Streams
byDean Wampler
 
Reactive Stream Processing with Akka Streams
byKonrad Malawski
 
Functional Programming and Composing Actors
bylegendofklang
 
[PDF Download] Akka in Action 1st Edition Raymond Roestenburg fulll chapter
bylabliinagai
 
Akka in Action 1st Edition Raymond Roestenburg download pdf
byhimlalnunys
 
Introduction to Akka Streams [Part-I]
byKnoldus Inc.
 
Akka Streams - From Zero to Kafka
byMark Harrison
 
Writing Asynchronous Programs with Scala & Akka
byYardena Meymann
 
Taking Akka Streams & Akka Http to Large Scale Production Applications
byAkara Sucharitakul
 

More from Johan Andrén

PDF
Introduction to akka actors with java 8
byJohan Andrén
 
PDF
Scala frukostseminarium
byJohan Andrén
 
PDF
Introduction to Akka
byJohan Andrén
 
PDF
Async – react, don't wait
byJohan Andrén
 
PDF
Akka frukostseminarium
byJohan Andrén
 
PDF
Macros and reflection in scala 2.10
byJohan Andrén
 
PDF
Introduction to Scala
byJohan Andrén
 
PDF
Duchess scala-2012
byJohan Andrén
 
Introduction to akka actors with java 8
byJohan Andrén
 
Scala frukostseminarium
byJohan Andrén
 
Introduction to Akka
byJohan Andrén
 
Async – react, don't wait
byJohan Andrén
 
Akka frukostseminarium
byJohan Andrén
 
Macros and reflection in scala 2.10
byJohan Andrén
 
Introduction to Scala
byJohan Andrén
 
Duchess scala-2012
byJohan Andrén
 

Recently uploaded

PDF
Certified AI-Empowered SAFe Release Train Engineer.pdf
byMarc Entsminger SPC6, RTE, SSM, SA, SDP
 
PDF
Transcript: What Thema can do: Leveraging metadata to support the discoverabi...
byBookNet Canada
 
PDF
A Simple Guide to Real Estate Tokenization on XRP Ledger.pdf
byemmajoh2025
 
PDF
Afsar Ahmed - Digital Marketing Portfolio
byAfsar Ahmed
 
PDF
"Painless Major Upgrade: A Strategy for Updating Large Codebases", Andrii Yat...
byFwdays
 
PDF
Parental Control App for Phones_ The Complete 2026 Guide for Safer, Smarter P...
byRyan Cooper
 
PDF
Computer-Based Training (CBT) The Backbone of Modern Technical & Defence Trai...
bycomputerbasedtrainin1
 
PDF
Dev Dives: Build and deploy agentic automations - the unified way
byUiPathCommunity
 
PPTX
Spacecraft Guidance Quick Research Guide by Arthur Morgan
byArthur Morgan
 
PDF
Real-Time AI at Scale Masterclass - Challenges of AI at Scale
byScyllaDB
 
PDF
How PayPal Account Verification Works – Complete Guide for Online Businesses
byjhdhj3989
 
PDF
EU regulations for the North American book supply chain - Tech Forum 2026
byBookNet Canada
 
PPTX
INSY_7213_Theme Sessions 47 & 48 Advanced databases.pptx
bystormzy56
 
PDF
Why AI Girlfriends Are the Future of Digital Companionship
byAi Angels
 
PDF
Certified AI-Native Foundations Professional.pdf
byMarc Entsminger SPC6, RTE, SSM, SA, SDP
 
PDF
TrustArc Webinar - From Zero to Privacy Hero: Launching Your Program Right an...
byTrustArc
 
PDF
When Drones Decide for Themselves_ Inside the Rise of Machine-Led Flight.pdf
byLyra Anderson
 
PPTX
The Lex Wire Precedent: A Technical Standard for Machine-Mediated Authority ...
byJeff Howell
 
PDF
Transcript: EU regulations for the North American book supply chain - Tech Fo...
byBookNet Canada
 
PPTX
Code Like Bro -A guide for better Coding
byMadan Panthi
 
Certified AI-Empowered SAFe Release Train Engineer.pdf
byMarc Entsminger SPC6, RTE, SSM, SA, SDP
 
Transcript: What Thema can do: Leveraging metadata to support the discoverabi...
byBookNet Canada
 
A Simple Guide to Real Estate Tokenization on XRP Ledger.pdf
byemmajoh2025
 
Afsar Ahmed - Digital Marketing Portfolio
byAfsar Ahmed
 
"Painless Major Upgrade: A Strategy for Updating Large Codebases", Andrii Yat...
byFwdays
 
Parental Control App for Phones_ The Complete 2026 Guide for Safer, Smarter P...
byRyan Cooper
 
Computer-Based Training (CBT) The Backbone of Modern Technical & Defence Trai...
bycomputerbasedtrainin1
 
Dev Dives: Build and deploy agentic automations - the unified way
byUiPathCommunity
 
Spacecraft Guidance Quick Research Guide by Arthur Morgan
byArthur Morgan
 
Real-Time AI at Scale Masterclass - Challenges of AI at Scale
byScyllaDB
 
How PayPal Account Verification Works – Complete Guide for Online Businesses
byjhdhj3989
 
EU regulations for the North American book supply chain - Tech Forum 2026
byBookNet Canada
 
INSY_7213_Theme Sessions 47 & 48 Advanced databases.pptx
bystormzy56
 
Why AI Girlfriends Are the Future of Digital Companionship
byAi Angels
 
Certified AI-Native Foundations Professional.pdf
byMarc Entsminger SPC6, RTE, SSM, SA, SDP
 
TrustArc Webinar - From Zero to Privacy Hero: Launching Your Program Right an...
byTrustArc
 
When Drones Decide for Themselves_ Inside the Rise of Machine-Led Flight.pdf
byLyra Anderson
 
The Lex Wire Precedent: A Technical Standard for Machine-Mediated Authority ...
byJeff Howell
 
Transcript: EU regulations for the North American book supply chain - Tech Fo...
byBookNet Canada
 
Code Like Bro -A guide for better Coding
byMadan Panthi
 

Reactive streams processing using Akka Streams

  • 1.
    Akka streams Johan Andrén JavaZone,Oslo, 2017-09-13 Reactive stream processing using
  • 2.
    Johan Andrén Akka Team StockholmScala User Group @apnylle johan.andren@lightbend.com
  • 3.
    Make building powerfulconcurrent & distributed applications simple. Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM Akka
  • 4.
    Actors – simple& high performance concurrency Cluster / Remoting, Cluster tools – location transparency, resilience – and prepackaged tools for building distributed systems Streams – back-pressured stream processing Persistence – CQRS + Event Sourcing for Actors HTTP – fully async streaming HTTP Server Complete Java & Scala APIs for all features What’s in the toolkit?
  • 5.
  • 6.
    Reactive Streams timeline Oct2013 RxJava, Akka and Twitter- people meeting “Soon thereafter” 2013 Reactive Streams Expert group formed Apr 2015 Reactive Streams Spec 1.0 TCK 5+ impls 2017 Inclusion in JDK9 Akka Streams, RxJava Vert.x, MongoDB, … Jul 2015 Akka Streams 1.0
  • 7.
    Reactive Streams Reactive Streamsis an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org “
  • 8.
    Reactive Streams Reactive Streamsis an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org “
  • 9.
  • 10.
    Reactive Streams Reactive Streamsis an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org “
  • 11.
    Asynchronous stream processing SourceSink (possible) asynchronous boundaries Flow
  • 12.
    Reactive Streams Reactive Streamsis an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org “
  • 13.
    No back pressure SourceSink 10 msg/s 1 msg/s Flow asynchronous boundary
  • 14.
    No back pressure SourceSink 10 msg/s 1 msg/s Flow asynchronous boundary OOM!!
  • 15.
    No back pressure- bounded buffer Source Sink 10 msg/s 1 msg/s Flow buffer size 6 🗑 asynchronous boundary
  • 16.
    Async non blockingback pressure Source Sink 1 msg/s 1 msg/s Flow buffer size 6 🗑 asynchronous boundary Hey! give me 2 more
  • 17.
    Reactive Streams RS LibraryA RS library B async boundary
  • 18.
    Reactive Streams “Make buildingpowerful concurrent & distributed applications simple.”
  • 19.
    Complete and awesome Javaand Scala APIs (Just like everything in Akka) Akka Streams
  • 20.
    Akka Streams in~20 seconds: final ActorSystem system = ActorSystem.create();
 final Materializer materializer = ActorMaterializer.create(system);
 
 final Source<Integer, NotUsed> source =
 Source.range(0, 20000000);
 
 final Flow<Integer, String, NotUsed> flow =
 Flow.fromFunction((Integer n) -> n.toString());
 
 final Sink<String, CompletionStage<Done>> sink =
 Sink.foreach(str -> System.out.println(str));
 
 final RunnableGraph<NotUsed> runnable = source.via(flow).to(sink);
 
 runnable.run(materializer); complete sources on github
  • 21.
    Akka Streams in~20 seconds: final ActorSystem system = ActorSystem.create();
 final Materializer materializer = ActorMaterializer.create(system);
 
 final Source<Integer, NotUsed> source =
 Source.range(0, 20000000);
 
 final Flow<Integer, String, NotUsed> flow =
 Flow.fromFunction((Integer n) -> n.toString());
 
 final Sink<String, CompletionStage<Done>> sink =
 Sink.foreach(str -> System.out.println(str));
 
 final RunnableGraph<NotUsed> runnable = source.via(flow).to(sink);
 
 runnable.run(materializer); complete sources on github
  • 22.
    final ActorSystem system= ActorSystem.create();
 final Materializer materializer = ActorMaterializer.create(system);
 
 final Source<Integer, NotUsed> source =
 Source.range(0, 20000000);
 
 final Flow<Integer, String, NotUsed> flow =
 Flow.fromFunction((Integer n) -> n.toString());
 
 final Sink<String, CompletionStage<Done>> sink =
 Sink.foreach(str -> System.out.println(str));
 
 final RunnableGraph<NotUsed> runnable = source.via(flow).to(sink);
 
 runnable.run(materializer); Akka Streams in ~20 seconds: complete sources on github Source (range) String Flow (fromFunction) Sink (foreach) String Integer Integer
  • 23.
    Akka Streams in~20 seconds: implicit val system = ActorSystem()
 implicit val mat = ActorMaterializer()
 
 val source = Source(0 to 20000000)
 
 val flow = Flow[Int].map(_.toString())
 
 val sink = Sink.foreach[String](println(_))
 
 val runnable = source.via(flow).to(sink)
 
 runnable.run() complete sources on github
  • 24.
    Akka Streams in~20 seconds: Source.range(0, 20000000)
 .map(Object::toString)
 .runForeach(str -> System.out.println(str), materializer); complete sources on github
  • 25.
    Akka Streams in~20 seconds: Source(0 to 20000000)
 .map(_.toString)
 .runForeach(println) complete sources on github
  • 26.
    Numbers as aservice final Source<ByteString, NotUsed> numbers = Source.unfold(0L, n -> {
 long next = n + 1;
 return Optional.of(Pair.create(next, next));
 }).map(n -> ByteString.fromString(n.toString() + "n"));
 
 
 final Route route =
 path("numbers", () ->
 get(() ->
 complete(HttpResponse.create()
 .withStatus(StatusCodes.OK)
 .withEntity(HttpEntities.create(
 ContentTypes.TEXT_PLAIN_UTF8,
 numbers
 )))
 )
 );
 
 final CompletionStage<ServerBinding> bindingCompletionStage =
 http.bindAndHandle(route.flow(system, materializer), host, materializer); complete sources on github
  • 27.
    final Source<ByteString, NotUsed>numbers = Source.unfold(0L, n -> {
 long next = n + 1;
 return Optional.of(Pair.create(next, next));
 }).map(n -> ByteString.fromString(n.toString() + "n"));
 
 
 final Route route =
 path("numbers", () ->
 get(() ->
 complete(HttpResponse.create()
 .withStatus(StatusCodes.OK)
 .withEntity(HttpEntities.create(
 Numbers as a service complete sources on github
  • 28.
    return Optional.of(Pair.create(next, next));
 }).map(n-> ByteString.fromString(n.toString() + "n"));
 
 
 final Route route =
 path("numbers", () ->
 get(() ->
 complete(HttpResponse.create()
 .withStatus(StatusCodes.OK)
 .withEntity(HttpEntities.create(
 ContentTypes.TEXT_PLAIN_UTF8,
 numbers
 )))
 )
 );
 
 final CompletionStage<ServerBinding> bindingCompletionStage =
 http.bindAndHandle(route.flow(system, materializer), host, materializer); Numbers as a service complete sources on github
  • 29.
    Numbers as aservice val numbers =
 Source.unfold(0L) { (n) =>
 val next = n + 1
 Some((next, next))
 }.map(n => ByteString(n + "n"))
 
 val route =
 path("numbers") {
 get {
 complete( HttpResponse(entity = HttpEntity(`text/plain(UTF-8)`, numbers)) )
 }
 }
 val futureBinding = Http().bindAndHandle(route, "127.0.0.1", 8080) complete sources on github
  • 30.
    recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚚 Backpressure over TCP numbers TCP HTTP Server Client
  • 31.
    recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚑 Backpressure over TCP numbers TCP HTTP Backpressure Server Client
  • 32.
    recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚑 Backpressure over TCP numbers TCP HTTP Backpressure Backpressure Server Client
  • 33.
    A more usefulexample complete sources on github final Flow<Message, Message, NotUsed> measurementsFlow =
 Flow.of(Message.class)
 .flatMapConcat((Message message) ->
 message.asTextMessage()
 .getStreamedText()
 .fold("", (acc, elem) -> acc + elem)
 )
 .groupedWithin(1000, FiniteDuration.create(1, SECONDS))
 .mapAsync(5, database::asyncBulkInsert)
 .map(written ->
 TextMessage.create("wrote up to: " + written.get(written.size() - 1))
 );
 
 final Route route = path("measurements", () ->
 get(() ->
 handleWebSocketMessages(measurementsFlow)
 )
 );
 
 final CompletionStage<ServerBinding> bindingCompletionStage =
 http.bindAndHandle(route.flow(system, materializer), host, materializer); Credit to: Colin Breck
  • 34.
    final Flow<Message, Message,NotUsed> measurementsFlow =
 Flow.of(Message.class)
 .flatMapConcat((Message message) ->
 message.asTextMessage()
 .getStreamedText()
 .fold("", (acc, elem) -> acc + elem)
 )
 .groupedWithin(1000, FiniteDuration.create(1, SECONDS))
 .mapAsync(5, database::asyncBulkInsert)
 .map(written ->
 TextMessage.create("wrote up to: " + written.get(written.size() - 1))
 );
 
 final Route route = path("measurements", () ->
 A more useful example complete sources on github Credit to: Colin Breck
  • 35.
    )
 .groupedWithin(1000, FiniteDuration.create(1, SECONDS))
 .mapAsync(5,database::asyncBulkInsert)
 .map(written ->
 TextMessage.create("wrote up to: " + written.get(written.size() - 1))
 );
 
 final Route route = path("measurements", () ->
 get(() ->
 handleWebSocketMessages(measurementsFlow)
 )
 );
 
 final CompletionStage<ServerBinding> bindingCompletionStage =
 http.bindAndHandle(route.flow(system, materializer), host, materializer); A more useful example complete sources on github Credit to: Colin Breck
  • 36.
    A more usefulexample complete sources on github val measurementsFlow =
 Flow[Message].flatMapConcat(message =>
 message.asTextMessage.getStreamedText.fold("")(_ + _)
 )
 .groupedWithin(1000, 1.second)
 .mapAsync(5)(Database.asyncBulkInsert)
 .map(written => TextMessage("wrote up to: " + written.last))
 
 val route =
 path("measurements") {
 get {
 handleWebSocketMessages(measurementsFlow)
 }
 }
 
 val futureBinding = Http().bindAndHandle(route, "127.0.0.1", 8080)
  • 37.
    The tale ofthe two pancake chefs HungrySink Frying Pan BatterSource Scoops of batter Pancakes nom nom nom asynchronous boundaries Roland Patrik
  • 38.
    Rolands pipelined pancakes HungrySinkPan2BatterSource Pan 1 nom nom nom
  • 39.
    Rolands pipelined pancakes Flow<ScoopOfBatter,HalfCookedPancake, NotUsed> fryingPan1 =
 Flow.of(ScoopOfBatter.class).map(batter -> new HalfCookedPancake());
 
 Flow<HalfCookedPancake, Pancake, NotUsed> fryingPan2 =
 Flow.of(HalfCookedPancake.class).map(halfCooked -> new Pancake()); Flow<ScoopOfBatter, Pancake, NotUsed> pancakeChef =
 fryingPan1.async().via(fryingPan2.async()); section in docs
  • 40.
    Rolands pipelined pancakes //Takes a scoop of batter and creates a pancake with one side cooked val fryingPan1: Flow[ScoopOfBatter, HalfCookedPancake, NotUsed] =
 Flow[ScoopOfBatter].map { batter => HalfCookedPancake() }
 
 // Finishes a half-cooked pancake
 val fryingPan2: Flow[HalfCookedPancake, Pancake, NotUsed] =
 Flow[HalfCookedPancake].map { halfCooked => Pancake() } 
 // With the two frying pans we can fully cook pancakes val pancakeChef: Flow[ScoopOfBatter, Pancake, NotUsed] =
 Flow[ScoopOfBatter].via(fryingPan1.async).via(fryingPan2.async) section in docs
  • 41.
    Patriks parallel pancakes HungrySink Pan2 BatterSource Pan 1 Balance Merge nom nom nom
  • 42.
    Patriks parallel pancakes Flow<ScoopOfBatter,Pancake, NotUsed> fryingPan =
 Flow.of(ScoopOfBatter.class).map(batter -> new Pancake());
 
 Flow<ScoopOfBatter, Pancake, NotUsed> pancakeChef =
 Flow.fromGraph(GraphDSL.create(builder -> {
 final UniformFanInShape<Pancake, Pancake> mergePancakes =
 builder.add(Merge.create(2));
 final UniformFanOutShape<ScoopOfBatter, ScoopOfBatter> dispatchBatter =
 builder.add(Balance.create(2));
 
 builder.from(dispatchBatter.out(0)) .via(builder.add(fryingPan.async())) .toInlet(mergePancakes.in(0));
 builder.from(dispatchBatter.out(1)) .via(builder.add(fryingPan.async())) .toInlet(mergePancakes.in(1));
 
 return FlowShape.of(dispatchBatter.in(), mergePancakes.out());
 })); section in docs
  • 43.
    Patriks parallel pancakes valpancakeChef: Flow[ScoopOfBatter, Pancake, NotUsed] =
 Flow.fromGraph(GraphDSL.create() { implicit builder =>
 import GraphDSL.Implicits._ 
 val dispatchBatter = builder.add(Balance[ScoopOfBatter](2))
 val mergePancakes = builder.add(Merge[Pancake](2))
 
 // Using two pipelines, having two frying pans each, in total using
 // four frying pans
 dispatchBatter.out(0) ~> fryingPan1.async ~> fryingPan2.async ~> mergePancakes.in(0)
 dispatchBatter.out(1) ~> fryingPan1.async ~> fryingPan2.async ~> mergePancakes.in(1)
 
 FlowShape(dispatchBatter.in, mergePancakes.out)
 }) section in docs
  • 44.
    Making pancakes together HungrySink Pan3 BatterSource Pan 1 Balance Merge Pan 2 Pan 4 nom nom nom
  • 45.
    Built in stagesFlow stages map/fromFunction, mapConcat, statefulMapConcat, filter, filterNot, collect, grouped, sliding, scan, scanAsync, fold, foldAsync, reduce, drop, take, takeWhile, dropWhile, recover, recoverWith, recoverWithRetries, mapError, detach, throttle, intersperse, limit, limitWeighted, log, recoverWithRetries, mapAsync, mapAsyncUnordered, takeWithin, dropWithin, groupedWithin, initialDelay, delay, conflate, conflateWithSeed, batch, batchWeighted, expand, buffer, prefixAndTail, groupBy, splitWhen, splitAfter, flatMapConcat, flatMapMerge, initialTimeout, completionTimeout, idleTimeout, backpressureTimeout, keepAlive, initialDelay, merge, mergeSorted, Source stages fromIterator, apply, single, repeat, cycle, tick, fromFuture, fromCompletionStage, unfold, unfoldAsync, empty, maybe, failed, lazily, actorPublisher, actorRef, combine, unfoldResource, unfoldResourceAsync, queue, asSubscriber, fromPublisher, zipN, zipWithN Sink stages head, headOption, last, lastOption, ignore, cancelled, seq, foreach, foreachParallel, onComplete, lazyInit, queue, fold, reduce, combine, actorRef, actorRefWithAck, actorSubscriber, asPublisher, fromSubscriber Additional Sink and Source converters {from,as}OutputStream, {from,as}InputStream, {as,from} javaCollector, javaCollectorParallelUnordered File IO Sinks and Sources fromPath, toPath mergePreferred, zip, zipWith, zipWithIndex, concat, prepend, orElse, interleave, unzip, unzipWith, broadcast, balance, partition, watchTermination, monitor Even more Framing, JSON framing, killswitch, BroadcastHub, MergeHub
  • 46.
    But I wantto connect other things!
  • 47.
    A community forAkka Streams connectors http://github.com/akka/alpakka Alpakka
  • 48.
    Alpakka – acommunity for Stream connectors Existing Alpakka MQTT AMQP/ RabbitMQ SSE Cassandra FTP/ SFTP JSON, XML, CSV, RecordIO IronMq Files AWS DynamoDB AWS SNS,SQS, S3, Kinesis,Lambda JMS Azure Storage Queue TCP In Akka Actors Reactive Streams Java Streams Basic File IO External Apache Geode Eventuate FS2 Akka Http HBase http://developer.lightbend.com/docs/alpakka/current/index.html Google Cloud Pub/Sub Camel Kafka MongoDB Azure IoT
  • 49.
    But my usecaseis a unique snowflake! ❄ ❄ ❄
  • 50.
    GraphStage API public classMap<A, B> extends GraphStage<FlowShape<A, B>> {
 private final Function<A, B> f;
 public final Inlet<A> in = Inlet.create("Map.in");
 public final Outlet<B> out = Outlet.create("Map.out"); private final FlowShape<A, B> shape = FlowShape.of(in, out); public Map(Function<A, B> f) {
 this.f = f;
 }
 public FlowShape<A,B> shape() {
 return shape;
 }
 public GraphStageLogic createLogic(Attributes inheritedAttributes) {
 return new GraphStageLogic(shape) {
 {
 setHandler(in, new AbstractInHandler() {
 @Override
 public void onPush() throws Exception {
 push(out, f.apply(grab(in)));
 }
 });
 setHandler(out, new AbstractOutHandler() {
 @Override
 public void onPull() throws Exception {
 pull(in);
 }
 });
 }
 };
 }
 } complete sources on github
  • 51.
    public class Map<A,B> extends GraphStage<FlowShape<A, B>> {
 private final Function<A, B> f;
 public final Inlet<A> in = Inlet.create("Map.in");
 public final Outlet<B> out = Outlet.create("Map.out"); private final FlowShape<A, B> shape = FlowShape.of(in, out); public Map(Function<A, B> f) {
 this.f = f;
 }
 public FlowShape<A,B> shape() {
 return shape;
 }
 public GraphStageLogic createLogic(Attributes inheritedAttributes) {
 return new GraphStageLogic(shape) {
 {
 setHandler(in, new AbstractInHandler() {
 @Override
 public void onPush() throws Exception {
 GraphStage API complete sources on github
  • 52.
    public FlowShape<A,B> shape(){
 return shape;
 }
 public GraphStageLogic createLogic(Attributes inheritedAttributes) {
 return new GraphStageLogic(shape) {
 {
 setHandler(in, new AbstractInHandler() {
 @Override
 public void onPush() throws Exception {
 push(out, f.apply(grab(in)));
 }
 });
 setHandler(out, new AbstractOutHandler() {
 @Override
 public void onPull() throws Exception {
 pull(in);
 }
 });
 }
 };
 }
 } GraphStage API complete sources on github
  • 53.
    GraphStage API class Map[A,B](f: A => B) extends GraphStage[FlowShape[A, B]] {
 
 val in = Inlet[A]("Map.in")
 val out = Outlet[B]("Map.out")
 override val shape = FlowShape.of(in, out)
 
 override def createLogic(attr: Attributes): GraphStageLogic =
 new GraphStageLogic(shape) {
 setHandler(in, new InHandler {
 override def onPush(): Unit = {
 push(out, f(grab(in)))
 }
 })
 setHandler(out, new OutHandler {
 override def onPull(): Unit = {
 pull(in)
 }
 })
 }
 } complete sources on github
  • 54.
    When to usewhat abstraction modelling power complexity actors streams (completable) futures java.concurrency
  • 55.
    Actors ❤ Streams= True Actors – perfect for managing state, things with lifecycles, restarting, running many separate instances in parallel A running stream – can be seen as a state, has a lifecycle, is - not yet started, running, failed, stopped, can be stopped using special stages blog.colinbreck.com/integrating-akka-streams-and-akka-actors-part-i/ Again, Colin Breck:
  • 56.
    But, distributed systems? Stream-in-Actor C-1 ShardC Akka Streams is a local abstraction Combine with Akka Cluster for distributed superpowers! Stream-in-Actor A-1 Shard A Stream-in-Actor B-1 Shard B Stream-in-Actor B-2
  • 57.
    The community Mailing list: https://groups.google.com/group/akka-user Publicchat rooms: http://gitter.im/akka/dev developing Akka http://gitter.im/akka/akka using Akka Easy to contribute tickets: https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22 ~200 active contributors!
  • 58.
  • 59.
    Takk! @apnylle johan.andren@lightbend.com github.com/johanandren All sample code(Java & Scala) https://github.com/johanandren/akka-stream-samples/tree/javazone-2017 http://akka.io Akka