Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scala usergroup stockholm - reactive integrations with akka streams

480 views

Published on

Akka Streams introduction at the Scala Usergroup Stockholm meetup 2016-10-18

Published in: Data & Analytics
  • Be the first to comment

Scala usergroup stockholm - reactive integrations with akka streams

  1. 1. akka streams Reactive Integrations with that just work™ Johan Andrén Scala Usergroup Stockholm 2016-10-18
  2. 2. Johan Andrén Akka Team Stockholm Scala User Group
  3. 3. Make building powerful concurrent & distributed applications simple. Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM
  4. 4. Actors – simple & high performance concurrency Cluster / Remoting – location transparency, resilience Cluster tools – and more prepackaged patterns Streams – back-pressured stream processing Persistence – Event Sourcing HTTP – complete, fully async and reactive HTTP Server Official Kafka, Cassandra, DynamoDB integrations, tons more in the community Complete Java & Scala APIs for all features What’s in the toolkit?
  5. 5. “Stream”has many meanings…
  6. 6. akka streams Asynchronous back pressured stream processing Source Sink Flow
  7. 7. akka streams Asynchronous back pressured stream processing Source Sink (possible) asynchronous boundaries Flow
  8. 8. akka streams Asynchronous back pressured stream processing Source Sink 10 msg/s 1 msg/s OutOfMemoryError!! Flow
  9. 9. akka streams Asynchronous back pressured stream processing Source Sink 10 msg/s 1 msg/s hand me 3 morehand me 3 more 1 msg/s Flow
  10. 10. akka streams Not only linear streams Source SinkFlow Source Sink Flow Flow
  11. 11. Reactive Streams Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols http://www.reactive-streams.org
  12. 12. Part of JDK 9 java.util.concurrent.Flow
  13. 13. Reactive Streams RS Library A RS library B async boundary
  14. 14. Reactive Streams RS Library A RS library B async boundary Make building powerful concurrent & distributed applications simple.
  15. 15. The API Akka Streams Complete and awesome Java and Scala APIs (Just like everything in Akka)
  16. 16. Akka Streams in 20 seconds: val source: Source[Int, NotUsed] =
 Source(0 to 200000)
 
 val flow: Flow[Int, String, NotUsed] =
 Flow[Int].map(_.toString)
 
 val sink: Sink[String, Future[Done]] =
 Sink.foreach(println)
 
 val runnableGraph = source.via(flow).to(sink)
 
 runnableGraph.run()
  17. 17. Akka Streams in 20 seconds: source.via(flow).to(sink)
 Source[Int, NotUsed] Flow[Int, String, NotUsed] Sink[String, Future[Done]]
  18. 18. Materialization Gears from GeeCON.org,(it’s an awesome conf)
  19. 19. What is “materialization” really?
  20. 20. What is “materialization” really?
  21. 21. What is “materialization” really?
  22. 22. What is “materialization” really?
  23. 23. AlpakkaA community for Streams connectors http://blog.akka.io/integrations/2016/08/23/intro-alpakka
  24. 24. Alpakka – a community for Stream connectors Threading & Concurrency in Akka Streams Explained (part I) Mastering GraphStages (part I, Introduction) Akka Streams Integration, codename Alpakka A gentle introduction to building Sinks and Sources using GraphStage APIs (Mastering GraphStages, Part II) Writing Akka Streams Connectors for existing APIs Flow control at the boundary of Akka Streams and a data provider Akka Streams Kafka 0.11
  25. 25. Alpakka – a community for Stream connectors Existing examples: MQTT AMQP Streaming HTTP Streaming TCP Streaming FileIO Cassandra Queries “Reactive Kafka” (akka-stream-kafka) S3, SQS & other Amazon APIs Streaming JSON Streaming XML …
  26. 26. Alpakka – a community for Stream connectors Demo KafkaSink FileTailSource Kafka Bytes => Lines
  27. 27. Alpakka – a community for Stream connectors Demo Kafka parse line window last 10 Find hacker KafkaSource Sound the alarm!
  28. 28. Alpakka – a community for Stream connectors Kafka Stream Stream Stream Stream cluster
  29. 29. Akka Streams & HTTP streams & HTTP
  30. 30. Akka Streams / HTTP Quiz time!
  31. 31. Akka Streams / HTTP recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚚
  32. 32. Akka Streams / HTTP recv buffer send buffer 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚚 🚑
  33. 33. Streaming in Akka HTTP http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html HttpServer as a: Flow[HttpRequest, HttpResponse]
  34. 34. Streaming in Akka HTTP HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
  35. 35. Streaming in Akka HTTP HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] Websocket connection as a: Flow[ws.Message, ws.Message] http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
  36. 36. Streaming from Akka HTTP send buffer 🚚 🚚 OSApp EntrySource Json => Bytes Entry => Json TcpSink
  37. 37. Streaming from Akka HTTP send buffer 🚚 🚚 🚚 OSApp EntrySource Json => Bytes Entry => Json TcpSink 🚚 🚚
  38. 38. Streaming from Akka HTTP send buffer 🚚 🚚 🚚 OSApp EntrySource Json => Bytes Entry => Json TcpSink 🚚 🚚 No demand from TCP = No demand upstream = Source won’t read log => Bounded memory stream processing! Demo
  39. 39. Akka ❤ contributions Easy to contribute tickets: https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22 Akka Stream Contrib https://github.com/akka/akka-stream-contrib Mailing list: https://groups.google.com/group/akka-user Public chat rooms: http://gitter.im/akka/dev developing Akka http://gitter.im/akka/akka using Akka
  40. 40. Thanks for listening! @apnylle johan.andren@lightbend.com Sample sources https://github.com/johanandren/akka-stream-samples/tree/scala-stockholm-2016-10

×