Reactive integrations with Akka Streams

205 views

Published on

Talk from Reactive Summit, together with Johan Andren

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
205
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
10
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Reactive integrations with Akka Streams

  1. 1. akka streams Reactive Integrations with that just work™ Johan Andrén Konrad Malawski
  2. 2. Johan Andrén Akka Team Stockholm Scala User Group
  3. 3. Konrad `ktoso` Malawski Akka Team, Reactive Streams TCK, Persistence, HTTP
  4. 4. Make building powerful concurrent & distributed applications simple. Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM
  5. 5. Actors – simple & high performance concurrency Cluster / Remoting – location transparency, resilience Cluster tools – and more prepackaged patterns Streams – back-pressured stream processing Persistence – Event Sourcing HTTP – complete, fully async and reactive HTTP Server Official Kafka, Cassandra, DynamoDB integrations, tons more in the community Complete Java & Scala APIs for all features What’s in the toolkit?
  6. 6. “Stream” has many meanings
  7. 7. akka streams Asynchronous back pressured stream processing Source Sink Flow
  8. 8. akka streams Asynchronous back pressured stream processing Source Sink (possible) asynchronous boundaries Flow
  9. 9. akka streams Asynchronous back pressured stream processing Source Sink 10 msg/s 1 msg/s OutOfMemoryError!! Flow
  10. 10. akka streams Asynchronous back pressured stream processing Source Sink 10 msg/s 1 msg/s hand me 3 morehand me 3 more 1 msg/s Flow
  11. 11. akka streams Not only linear streams Source SinkFlow Source Sink Flow Flow
  12. 12. Reactive Streams Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments as well as network protocols http://www.reactive-streams.org
  13. 13. Part of JDK 9 java.util.concurrent.Flow
  14. 14. Reactive Streams RS Library A RS library B async boundary
  15. 15. Reactive Streams RS Library A RS library B async boundary Make building powerful concurrent & distributed applications simple.
  16. 16. The API Akka Streams Complete and awesome Java and Scala APIs (Just like everything in Akka)
  17. 17. Akka Streams in 20 seconds: Source<Integer, NotUsed> source = null;
 
 Flow<Integer, String, NotUsed> flow =
 Flow.<Integer>create().map((Integer n) -> n.toString());
 
 Sink<String, CompletionStage<Done>> sink =
 Sink.foreach(str -> System.out.println(str));
 
 RunnableGraph<NotUsed> runnable = source.via(flow).to(sink);
 
 runnable.run(materializer);

  18. 18. Akka Streams in 20 seconds: CompletionStage<String> firstString =
 Source.single(1)
 .map(n -> n.toString())
 .runWith(Sink.head(), materializer);

  19. 19. Source.single(1).map(i -> i.toString).runWith(Sink.head()) // types: _ Source<Int, NotUsed> Flow<Int, String, NotUsed> Sink<String, CompletionStage<String>> Akka Streams in 20 seconds:
  20. 20. Source.single(1).map(i -> i.toString).runWith(Sink.head()) // types: _ Source<Int, NotUsed> Flow<Int, String, NotUsed> Sink<String, CompletionStage<String>> Akka Streams in 20 seconds:
  21. 21. Materialization Gears from GeeCON.org,(it’s an awesome conf)
  22. 22. What is “materialization” really?
  23. 23. What is “materialization” really?
  24. 24. What is “materialization” really?
  25. 25. What is “materialization” really?
  26. 26. What is “materialization” really? Check out the “Implementing an akka-streams materializer for big data” talk later today.
  27. 27. AlpakkaA community for Streams connectors http://blog.akka.io/integrations/2016/08/23/intro-alpakka
  28. 28. Alpakka – a community for Stream connectors Threading & Concurrency in Akka Streams Explained (part I) Mastering GraphStages (part I, Introduction) Akka Streams Integration, codename Alpakka A gentle introduction to building Sinks and Sources using GraphStage APIs (Mastering GraphStages, Part II) Writing Akka Streams Connectors for existing APIs Flow control at the boundary of Akka Streams and a data provider Akka Streams Kafka 0.11
  29. 29. Alpakka – a community for Stream connectors Existing examples: MQTT AMQP Streaming HTTP Streaming TCP Streaming FileIO Cassandra Queries “Reactive Kafka” (akka-stream-kafka) S3, SQS & other Amazon APIs Streaming JSON Streaming XML …
  30. 30. Alpakka – a community for Stream connectors Demo
  31. 31. Alpakka – a community for Stream connectors Demo
  32. 32. Akka Streams & HTTP streams & HTTP
  33. 33. A core feature not obvious to the untrained eye…! Akka Streams / HTTP Quiz time! TCP is a ______ protocol?
  34. 34. A core feature not obvious to the untrained eye…! Akka Streams / HTTP Quiz time! TCP is a STREAMING protocol!
  35. 35. Streaming in Akka HTTP http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html HttpServer as a: Flow[HttpRequest, HttpResponse]
  36. 36. Streaming in Akka HTTP HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
  37. 37. Streaming in Akka HTTP HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] Websocket connection as a: Flow[ws.Message, ws.Message] http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
  38. 38. It’s turtles buffers all the way down! xkcd.com
  39. 39. Streaming from Akka HTTP
  40. 40. Streaming from Akka HTTP
  41. 41. Streaming from Akka HTTP No demand from TCP = No demand upstream = Source won’t generate tweets => Bounded memory stream processing! Demo
  42. 42. Streaming from Akka HTTP (Java) public static void main(String[] args) { final ActorSystem system = ActorSystem.create(); final Materializer materializer = ActorMaterializer.create(system); final Http http = Http.get(system); final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world")); final Route tweetsRoute = path("tweets", () -> completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json()) ); final Flow<HttpRequest, HttpResponse, NotUsed> handler = tweetsRoute.flow(system, materializer); http.bindAndHandle(handler, ConnectHttp.toHost("localhost", 8080), materializer ); System.out.println("Running at http://localhost:8080"); }
  43. 43. Streaming from Akka HTTP (Java) public static void main(String[] args) { final ActorSystem system = ActorSystem.create(); final Materializer materializer = ActorMaterializer.create(system); final Http http = Http.get(system); final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world")); final Route tweetsRoute = path("tweets", () -> completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json()) ); final Flow<HttpRequest, HttpResponse, NotUsed> handler = tweetsRoute.flow(system, materializer); http.bindAndHandle(handler, ConnectHttp.toHost("localhost", 8080), materializer ); System.out.println("Running at http://localhost:8080"); }
  44. 44. Streaming from Akka HTTP (Scala) object Example extends App with SprayJsonSupport with DefaultJsonProtocol { import akka.http.scaladsl.server.Directives._ implicit val system = ActorSystem() implicit val mat = ActorMaterializer() implicit val jsonRenderingMode = EntityStreamingSupport.json() implicit val TweetFormat = jsonFormat1(Tweet) def tweetsStreamRoutes = path("tweets") { complete { Source.repeat(Tweet("")) } } Http().bindAndHandle(tweetsStreamRoutes, "127.0.0.1", 8080) System.out.println("Running at http://localhost:8080"); }
  45. 45. Next steps for Akka Completely new Akka Remoting (goal: 700.000+ msg/s (!)), (it is built using Akka Streams, Aeron). More integrations for Akka Streams stages, project Alpakka. Reactive Kafka polishing with SoftwareMill, Krzysiek Ciesielski Akka Typed progressing again, likely towards 3.0. Akka HTTP 2.0 Proof of Concept in progress. Collaboration with Reactive Sockets
  46. 46. Ready to adopt on prod?
  47. 47. Totally, go for it.
  48. 48. Akka <3 contributions Easy to contribute tickets: https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22 Akka Stream Contrib https://github.com/akka/akka-stream-contrib Mailing list: https://groups.google.com/group/akka-user Public chat rooms: http://gitter.im/akka/dev developing Akka http://gitter.im/akka/akka using Akka
  49. 49. Reactive Platform Reactive Platform Reactive Platform
  50. 50. Further reading: Reactive Streams: reactive-streams.org Akka documentation: akka.io/docs Free O’Reilly report – very out soon. Example Sources: ktoso/akka-streams-alpakka-talk-demos-2016 Get involved: sources: github.com/akka/akka mailing list: akka-user @ google groups gitter channel: https://gitter.im/akka/akka Contact: Konrad ktoso@lightbend.com Malawski http://kto.so / @ktosopl
  51. 51. Thanks! Questions? @apnylle johan.andren@lightbend.com @ktosopl konrad.malawski@lightbend.com

×