Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
WEBINAR
Understanding Akka Streams, Back Pressure
and Asynchronous Architectures
by Konrad Malawski (@ktosopl)
Konrad `ktoso` Malawski
Akka Team,
Reactive Streams TCK,
Persistence, HTTP
Konrad `@ktosopl` Malawski
akka.io
typesafe.com
geecon.org
Java.pl / KrakowScala.pl
sckrk.com
GDGKrakow.pl
lambdakrk.pl
“Stream”
“Stream”
What does it mean?!
Akka Streams
Akka Streams && Reactive Streams
Why back-pressure?
?
Why back-pressure?
So you’ve built your app and it’s awesome.
Why back-pressure?
Let’s not smash it horribly under load.
What is back-pressure?
?
What is back-pressure?
No no no…!
Not THAT Back-pressure!
No no no…!
Not THAT Back-pressure!
What is back-pressure?
Publisher[T] Subscriber[T]
Back-pressure explained
Fast Publisher Slow Subscriber
Push model
Subscriber usually has some kind of buffer.
Push model
Push model
Push model
What if the buffer overflows?
Push model
Use bounded buffer,
drop messages + require re-sending
Push model
Kernel does this!
Routers do this!
(TCP)
Use bounded buffer,
drop messages + require re-sending
Push model
Increase buffer size…
Well, while you have memory available!
Push model
Push model
D
EM
O
Reactive Streams explained
Reactive Streams
explained in 1 slide
Fast Publisher will send at-most 3 elements.
This is pull-based-backpressure.
Reactive Streams: “dynamic push/pull”
JEP-266 – soon…!
public final class Flow {
private Flow() {} // uninstantiable
@FunctionalInterface
public static interfac...
Reactive Streams: goals
1) Avoiding unbounded buffering across async boundaries
2)Inter-op interfaces between various libr...
Reactive Streams: goals
1) Avoiding unbounded buffering across async boundaries
2)Inter-op interfaces between various libr...
Reactive Streams: goals
1) Avoiding unbounded buffering across async boundaries
2)Inter-op interfaces between various libr...
Reactive Streams: goals
1) Avoiding unbounded buffering across async boundaries
2)Inter-op interfaces between various libr...
Akka Streams
Streams complement Actors,
they do not replace them.
Actors – distribution (location transparency)
Streams – ...
Akka is a Toolkit, pick the right tools for the job.
Runar’s excellent talk @ Scala.World 2015
Asynchronous processing too...
Akka is a Toolkit, pick the right tools for the job.
Asynchronous processing toolbox:
Constraints
Power
Akka is a Toolkit, pick the right tools for the job.
Single value, no streaming by definition.
Local abstraction.

Executio...
Akka is a Toolkit, pick the right tools for the job.
Mostly static processing layouts.
Well typed and Back-pressured!
Cons...
Akka is a Toolkit, pick the right tools for the job.
Plain Actor’s younger brother, experimental.
Location transparent, we...
Akka is a Toolkit, pick the right tools for the job.
Runar’s excellent talk @ Scala.World 2015
Location transparent.
Vario...
Akka Streams
streams
Akka Streams in 20 seconds:
// types:
Source[Out, Mat]
Flow[In, Out, Mat]
Sink[In, Mat]
// generally speaking, it's always...
Akka Streams in 20 seconds:
// types: _
Source[Int, Unit]
Flow[Int, String, Unit]
Sink[String, Future[String]]
Source.sing...
Akka Streams in 20 seconds:
// types: _
Source[Int, Unit]
Flow[Int, String, Unit]
Sink[String, Future[String]]
Source.sing...
Materialization
Gears from GeeCON.org, did I mention it’s an awesome conf?
What is “materialization” really?
What is “materialization” really?
What is “materialization” really?
What is “materialization” really?
Akka Streams & HTTP
streams
& HTTP
Akka HTTP
Joint effort of Spray and Akka teams.
Complete HTTP Server/Client implementation.
Learns from Spray’s 3-4 years ...
Streaming in Akka HTTP
DEMO
http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala
“Framed...
Streaming in Akka HTTP
DEMO
http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala
“Framed...
Streaming in Akka HTTP
DEMO
http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala
“Framed...
Persistence Query (experimental)
“The Query Side”
of Akka Persistence
Persistence Query (experimental)
Persistence Query Journals
akka/akka-persistence-cassandra 0.16
akka/akka @ leveldb-journ...
Akka + Kafka = BFF
Reactive Kafka
+
Started by Krzysiek Ciesielski & Adam Warski @ SofwareMill.com
“ACKnowladged streams”
happy ACKing!
Kafka + Akka = BFF
Akka is Arbitrary processing.
Kafka is somewhat more than a message queue,
but very focused on “the log...
Kafka + Akka = BFF
Kafka + Akka = BFF
Streams talking to Actors
&&
Actors talking to Streams
Streams <=> Actors inter-op
Source.actorRef (no back-pressure)
Source.queue (safe)
Sink.actorRef (no back-pressure)
Sink.a...
Exciting times ahead!
Next steps for Akka
Completely new Akka Remoting (goal: 1M+ msg/s (!)),
(it is built using Akka Streams).
More integration...
Upgrade your grey matter

Two free O’Reilly eBooks by Lightbend
DOWNLOAD	NOWDOWNLOAD	NOW
lightbend.com/pov
Reactive Roundtable

World Tour by Lightbend
lightbend.com/reactive-roundtable
Proof of Value Service

A...
Q/A
ktoso @ lightbend.com
twitter: ktosopl
github: ktoso
team blog: letitcrash.com
home: akka.io
Understanding Akka Streams, Back Pressure, and Asynchronous Architectures
Upcoming SlideShare
Loading in …5
×

Understanding Akka Streams, Back Pressure, and Asynchronous Architectures

20,122 views

Published on

The term 'streams' has been getting pretty overloaded recently–it's hard to know where to best use different technologies with streams in the name. In this talk by noted hAkker Konrad Malawski, we'll disambiguate what streams are and what they aren't, taking a deeper look into Akka Streams (the implementation) and Reactive Streams (the standard).

You'll be introduced to a number of real life scenarios where applying back-pressure helps to keep your systems fast and healthy at the same time. While the focus is mainly on the Akka Streams implementation, the general principles apply to any kind of asynchronous, message-driven architectures.

Published in: Software

Understanding Akka Streams, Back Pressure, and Asynchronous Architectures

  1. 1. WEBINAR Understanding Akka Streams, Back Pressure and Asynchronous Architectures by Konrad Malawski (@ktosopl)
  2. 2. Konrad `ktoso` Malawski Akka Team, Reactive Streams TCK, Persistence, HTTP
  3. 3. Konrad `@ktosopl` Malawski akka.io typesafe.com geecon.org Java.pl / KrakowScala.pl sckrk.com GDGKrakow.pl lambdakrk.pl
  4. 4. “Stream”
  5. 5. “Stream” What does it mean?!
  6. 6. Akka Streams Akka Streams && Reactive Streams
  7. 7. Why back-pressure? ?
  8. 8. Why back-pressure? So you’ve built your app and it’s awesome.
  9. 9. Why back-pressure? Let’s not smash it horribly under load.
  10. 10. What is back-pressure? ?
  11. 11. What is back-pressure?
  12. 12. No no no…! Not THAT Back-pressure! No no no…! Not THAT Back-pressure! What is back-pressure?
  13. 13. Publisher[T] Subscriber[T] Back-pressure explained
  14. 14. Fast Publisher Slow Subscriber Push model
  15. 15. Subscriber usually has some kind of buffer. Push model
  16. 16. Push model
  17. 17. Push model
  18. 18. What if the buffer overflows? Push model
  19. 19. Use bounded buffer, drop messages + require re-sending Push model
  20. 20. Kernel does this! Routers do this! (TCP) Use bounded buffer, drop messages + require re-sending Push model
  21. 21. Increase buffer size… Well, while you have memory available! Push model
  22. 22. Push model D EM O
  23. 23. Reactive Streams explained Reactive Streams explained in 1 slide
  24. 24. Fast Publisher will send at-most 3 elements. This is pull-based-backpressure. Reactive Streams: “dynamic push/pull”
  25. 25. JEP-266 – soon…! public final class Flow { private Flow() {} // uninstantiable @FunctionalInterface public static interface Publisher<T> { public void subscribe(Subscriber<? super T> subscriber); } public static interface Subscriber<T> { public void onSubscribe(Subscription subscription); public void onNext(T item); public void onError(Throwable throwable); public void onComplete(); } public static interface Subscription { public void request(long n); public void cancel(); } public static interface Processor<T,R> extends Subscriber<T>, Publisher<R> { } }
  26. 26. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries
  27. 27. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries Argh, implementing a correct RS Publisher or Subscriber is so hard!
  28. 28. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries Argh, implementing a correct RS Publisher or Subscriber is so hard!
  29. 29. Reactive Streams: goals 1) Avoiding unbounded buffering across async boundaries 2)Inter-op interfaces between various libraries Argh, implementing a correct RS Publisher or Subscriber is so hard! You should be using Akka Streams abstractions instead!
  30. 30. Akka Streams Streams complement Actors, they do not replace them. Actors – distribution (location transparency) Streams – back-pressured + more rigid-blueprint
  31. 31. Akka is a Toolkit, pick the right tools for the job. Runar’s excellent talk @ Scala.World 2015 Asynchronous processing toolbox: Power Constraints
  32. 32. Akka is a Toolkit, pick the right tools for the job. Asynchronous processing toolbox: Constraints Power
  33. 33. Akka is a Toolkit, pick the right tools for the job. Single value, no streaming by definition. Local abstraction.
 Execution contexts. Power Constraints
  34. 34. Akka is a Toolkit, pick the right tools for the job. Mostly static processing layouts. Well typed and Back-pressured! Constraints Power
  35. 35. Akka is a Toolkit, pick the right tools for the job. Plain Actor’s younger brother, experimental. Location transparent, well typed. Technically unconstrained in actions performed Constraints Power
  36. 36. Akka is a Toolkit, pick the right tools for the job. Runar’s excellent talk @ Scala.World 2015 Location transparent. Various resilience mechanisms. (watching, persistent recovering, migration, pools) Untyped and unconstrained in actions performed. Constraints Power
  37. 37. Akka Streams streams
  38. 38. Akka Streams in 20 seconds: // types: Source[Out, Mat] Flow[In, Out, Mat] Sink[In, Mat] // generally speaking, it's always: val ready = Source(???).via(flow).map(_ * 2).to(sink) val mat: Mat = ready.run() // the usual example: val f: Future[String] = Source.single(1).map(_.toString).runWith(Sink.head) Proper static typing!
  39. 39. Akka Streams in 20 seconds: // types: _ Source[Int, Unit] Flow[Int, String, Unit] Sink[String, Future[String]] Source.single(1).map(_.toString).runWith(Sink.head)
  40. 40. Akka Streams in 20 seconds: // types: _ Source[Int, Unit] Flow[Int, String, Unit] Sink[String, Future[String]] Source.single(1).map(_.toString).runWith(Sink.head)
  41. 41. Materialization Gears from GeeCON.org, did I mention it’s an awesome conf?
  42. 42. What is “materialization” really?
  43. 43. What is “materialization” really?
  44. 44. What is “materialization” really?
  45. 45. What is “materialization” really?
  46. 46. Akka Streams & HTTP streams & HTTP
  47. 47. Akka HTTP Joint effort of Spray and Akka teams. Complete HTTP Server/Client implementation. Learns from Spray’s 3-4 years history. Since the beginning with streaming as first class citizen. Side note: Lagom also utilises Akka Streams for streaming.
  48. 48. Streaming in Akka HTTP DEMO http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778 HttpServer as a: Flow[HttpRequest, HttpResponse]
  49. 49. Streaming in Akka HTTP DEMO http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778 HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _]
  50. 50. Streaming in Akka HTTP DEMO http://doc.akka.io/docs/akka/2.4.7/scala/stream/stream-customize.html#graphstage-scala “Framed entity streaming” https://github.com/akka/akka/pull/20778 HttpServer as a: Flow[HttpRequest, HttpResponse] HTTP Entity as a: Source[ByteString, _] Websocket connection as a: Flow[ws.Message, ws.Message]
  51. 51. Persistence Query (experimental) “The Query Side” of Akka Persistence
  52. 52. Persistence Query (experimental) Persistence Query Journals akka/akka-persistence-cassandra 0.16 akka/akka @ leveldb-journal 2.4.8 dnvriend/akka-persistence-jdbc 2.3.3 scullxbones/akka-persistence-mongo 1.2.5 …and more, that I likely forgot about. Implementation of “data-pump” pattern.
  53. 53. Akka + Kafka = BFF Reactive Kafka + Started by Krzysiek Ciesielski & Adam Warski @ SofwareMill.com
  54. 54. “ACKnowladged streams” happy ACKing!
  55. 55. Kafka + Akka = BFF Akka is Arbitrary processing. Kafka is somewhat more than a message queue, but very focused on “the log”. Spark shines with it’s data-science focus.
  56. 56. Kafka + Akka = BFF
  57. 57. Kafka + Akka = BFF
  58. 58. Streams talking to Actors && Actors talking to Streams
  59. 59. Streams <=> Actors inter-op Source.actorRef (no back-pressure) Source.queue (safe) Sink.actorRef (no back-pressure) Sink.actorRefWithAck (safe)
  60. 60. Exciting times ahead!
  61. 61. Next steps for Akka Completely new Akka Remoting (goal: 1M+ msg/s (!)), (it is built using Akka Streams). More integrations for Akka Streams stages, also dynamic fan-in/out A.K.A.“the Hub”. Reactive Kafka polishing and stable release with SoftwareMill. “Confirmed Streams” work from Reactive Kafka generalised. Akka Typed likely to progress again. Of course, continued maintenance of Cluster and others.
  62. 62. Upgrade your grey matter
 Two free O’Reilly eBooks by Lightbend DOWNLOAD NOWDOWNLOAD NOW
  63. 63. lightbend.com/pov Reactive Roundtable
 World Tour by Lightbend lightbend.com/reactive-roundtable Proof of Value Service
 Accelerate Project Success
  64. 64. Q/A ktoso @ lightbend.com twitter: ktosopl github: ktoso team blog: letitcrash.com home: akka.io

×