Akka Streams introduction at the Scala Usergroup Stockholm meetup 2016-10-18

1. akka streams
Reactive Integrations with
that just work™
Johan Andrén
Scala Usergroup Stockholm 2016-10-18

2. Johan Andrén
Akka Team
Stockholm Scala User Group

3. Make building powerful concurrent &
distributed applications simple.
Akka is a toolkit and runtime
for building highly concurrent,
distributed, and resilient
message-driven applications
on the JVM

4. Actors – simple & high performance concurrency
Cluster / Remoting – location transparency, resilience
Cluster tools – and more prepackaged patterns
Streams – back-pressured stream processing
Persistence – Event Sourcing
HTTP – complete, fully async and reactive HTTP Server
Official Kafka, Cassandra, DynamoDB integrations, tons
more in the community
Complete Java & Scala APIs for all features
What’s in the toolkit?

5. “Stream”has many meanings…

6. akka streams
Asynchronous back pressured stream processing
Source Sink
Flow

7. akka streams
Asynchronous back pressured stream processing
Source Sink
(possible)
asynchronous
boundaries
Flow

8. akka streams
Asynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
OutOfMemoryError!!
Flow

9. akka streams
Asynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
hand me 3 morehand me 3 more
1 msg/s Flow

10. akka streams
Not only linear streams
Source
SinkFlow
Source
Sink
Flow
Flow

11. Reactive Streams
Reactive Streams is an initiative to provide a standard for
asynchronous stream processing with non-blocking back
pressure. This encompasses efforts aimed at runtime
environments (JVM and JavaScript) as well as network
protocols
http://www.reactive-streams.org

12. Part of JDK 9
java.util.concurrent.Flow

13. Reactive Streams
RS Library A RS library B
async
boundary

14. Reactive Streams
RS Library A RS library B
async
boundary
Make building powerful concurrent &
distributed applications simple.

15. The API
Akka Streams
Complete and awesome
Java and Scala APIs
(Just like everything in Akka)

16. Akka Streams in 20 seconds:
val source: Source[Int, NotUsed] =
Source(0 to 200000)
val flow: Flow[Int, String, NotUsed] =
Flow[Int].map(_.toString)
val sink: Sink[String, Future[Done]] =
Sink.foreach(println)
val runnableGraph =
source.via(flow).to(sink)
runnableGraph.run()

17. Akka Streams in 20 seconds:
source.via(flow).to(sink)
Source[Int, NotUsed]
Flow[Int, String, NotUsed]
Sink[String, Future[Done]]

18. Materialization
Gears from GeeCON.org,(it’s an awesome conf)

19. What is “materialization” really?

20. What is “materialization” really?

21. What is “materialization” really?

22. What is “materialization” really?

23. AlpakkaA community for Streams connectors
http://blog.akka.io/integrations/2016/08/23/intro-alpakka

24. Alpakka – a community for Stream connectors
Threading & Concurrency in Akka Streams Explained (part I)
Mastering GraphStages (part I, Introduction)
Akka Streams Integration, codename Alpakka
A gentle introduction to building Sinks and Sources using GraphStage APIs
(Mastering GraphStages, Part II)
Writing Akka Streams Connectors for existing APIs
Flow control at the boundary of Akka Streams and a data provider
Akka Streams Kafka 0.11

25. Alpakka – a community for Stream connectors
Existing examples:
MQTT
AMQP
Streaming HTTP
Streaming TCP
Streaming FileIO
Cassandra Queries
“Reactive Kafka” (akka-stream-kafka)
S3, SQS & other Amazon APIs
Streaming JSON
Streaming XML
…

26. Alpakka – a community for Stream connectors
Demo
KafkaSink
FileTailSource
Kafka
Bytes =>
Lines

27. Alpakka – a community for Stream connectors
Demo
Kafka
parse line
window
last 10
Find
hacker
KafkaSource
Sound the
alarm!

28. Alpakka – a community for Stream connectors
Kafka Stream Stream
Stream
Stream
cluster

29. Akka Streams & HTTP
streams
& HTTP

30. Akka Streams / HTTP
Quiz time!

31. Akka Streams / HTTP
recv buffer
send buffer
🚚
🚚
🚚
🚚
🚚
🚚
🚚

32. Akka Streams / HTTP
recv buffer
send buffer
🚚
🚚
🚚
🚚
🚚
🚚
🚚
🚚
🚚
🚚
🚑

33. Streaming in Akka HTTP
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
HttpServer as a:
Flow[HttpRequest, HttpResponse]

34. Streaming in Akka HTTP
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html

35. Streaming in Akka HTTP
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]
Websocket connection as a:
Flow[ws.Message, ws.Message]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html

36. Streaming from Akka HTTP
send buffer
🚚
🚚
OSApp
EntrySource
Json =>
Bytes
Entry =>
Json
TcpSink

37. Streaming from Akka HTTP
send buffer
🚚
🚚
🚚
OSApp
EntrySource
Json =>
Bytes
Entry =>
Json
TcpSink
🚚
🚚

38. Streaming from Akka HTTP
send buffer
🚚
🚚
🚚
OSApp
EntrySource
Json =>
Bytes
Entry =>
Json
TcpSink
🚚
🚚
No demand from TCP
=
No demand upstream
=
Source won’t read log
=>
Bounded memory
stream processing!
Demo

39. Akka ❤ contributions
Easy to contribute tickets:
https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute
https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22
Akka Stream Contrib
https://github.com/akka/akka-stream-contrib
Mailing list:
https://groups.google.com/group/akka-user
Public chat rooms:
http://gitter.im/akka/dev developing Akka
http://gitter.im/akka/akka using Akka

40. Thanks for listening!
@apnylle
johan.andren@lightbend.com
Sample sources
https://github.com/johanandren/akka-stream-samples/tree/scala-stockholm-2016-10