Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Reactive Streams at "Underscore" Scala User Group Tel Aviv, 2.7.2015

940 views

Published on

http://www.meetup.com/de/underscore/events/223584473/

Published in: Software
  • Be the first to comment

Reactive Streams at "Underscore" Scala User Group Tel Aviv, 2.7.2015

  1. 1. Reactive Streams and Akka Streams Lutz Hühnken (@lutzhuehnken)
  2. 2. Streams
  3. 3. Reactive Streams What is a Stream? • Ephemeral flow of data • Potentially unbounded in size • Processed by describing transformation of data 4
  4. 4. Reactive Streams Common Use of Streams • Bulk Data Transfer • Real Time Data Sources • Batch Processing of large data sets • Monitoring and Analytics 5
  5. 5. Reactive Streams
  6. 6. Reactive Streams Reactive 7
  7. 7. Reactive Streams Reactive Streams Overview • How do we: • Handle potentially infinite streams of data? • Handle data in a reactive manner? • Achieve asynchronous non-blocking data flow? • Avoid out of memory errors? 8
  8. 8. Reactive Streams Reactive Streams Projects / Companies • Typesafe • Akka Streams • Netflix • rxJava / rxScala • Pivotal • Spring Reactor • Redhat • Vert.x • Oracle 9
  9. 9. Reactive Streams Supply and Demand • Data Items Flow Downstream • Demand Flows Upstream • Data Items flow only when there is demand. 10
  10. 10. Reactive Streams Dynamic Push-Pull • “Push” behavior when consumer is faster • “Pull” behavior when producer is faster • Switches automatically between these • Batching demand allows batching data 11
  11. 11. Reactive Streams Reactive Streams Specification • Interface Specification • Java interfaces for implementations • TCK • Test Harness to validate implementations • Specification Website • http://www.reactive-streams.org/ 12
  12. 12. Reactive Streams Publisher package org.reactivestreams; public interface Publisher<T> { public void subscribe<T>( Subscriber<T> subscriber); } 13
  13. 13. Reactive Streams Subscriber public interface Subscriber<T> { public void onSubscribe( Subscription subscription); public void onNext(T element); public void onComplete(); public void onError(Throwable cause); } 14
  14. 14. Reactive Streams Subscription public interface Subscription { public void cancel(); public void request(long elements); } 15
  15. 15. Reactive Streams Backpressure • Downstream consumers pushing back on the producer to prevent flooding. • In reactive-streams: • Consumers stop requesting more elements until they are ready to process more. • Producers only fire elements if there is demand. 16
  16. 16. Reactive Streams Why Backpressure? • Explicitly design for system overload • Demand is propagated throughout the WHOLE flow • Can decide WHERE to handle overload • Limit the number of in-flight messages throughout the system • Bounded memory consumption • Bounded cpu contention • Recipient is in control of incoming data rate • Data in flight is bounded by signaled demand 17
  17. 17. Reactive Streams Buffering • We can prefetch stream elements by requesting more than we really need. • We can use this technique to ensure no "sputter" in the stream. • We can also use this technique to pull faster than downstream consumer. 18
  18. 18. Akka Streams & Actors
  19. 19. Reactive Streams Basic Actor 20 case class Greeting(who: String) class GreetingActor extends Actor with ActorLogging { def receive = { case Greeting(who) => log.info("Hello " + who) } } val system = ActorSystem("MySystem") val greeter: ActorRef = system.actorOf(Props[GreetingActor]) greeter ! Greeting("London Scala User Group")
  20. 20. Reactive Streams Properties of Actors • Message Based / Event Driven • Isolated State • Sane Concurrency Model • Isolated Failure Handling (Supervision) 21
  21. 21. Reactive Streams Akka Streams – A Bridge • Converts Publisher/Subscriber API into Actor messages • Simplify creating Publisher/Subscribers using Actors • Attach Reactive streams into existing Akka applications 22
  22. 22. Reactive Streams Creating an ActorSubscriber import akka.stream._ class PrintlnActor extends Actor with ActorSubscriber { val requestStrategy = OneByOneRequestStrategy def receive: Receive = { case ActorSubscriberMessage.OnNext(element) => println(element) } } val printlnActor:ActorRef = system.actorOf(Props[PrintlnActor], "println") val subscriber = ActorSubscriber(printlnActor) 23
  23. 23. Reactive Streams Creating an ActorPublisher import akka.stream._ class IntActor extends Actor with ActorPublisher[Int] { def receive: Receive = { case ActorPublisherMessage.Request(elements) => while (totalDemand > 0) { onNext(1) } } } val intActor: ActorRef = system.actorOf(Props[IntActor], "intActor") val publisher = ActorPublisher(intActor) 24
  24. 24. Reactive Streams Why use Actors as Publishers? • Actors are smart • They can keep internal state to track demand and supply • They can buffer data to meet anticipated demand • Actors are powerful • They can spin up child actors to meet demand • With Akka clustering, can spread load across multiple machines • Actors are resilient • On exception, actor can be killed and restarted by supervisor • Actor interaction is thread-safe, and actor state is private 25
  25. 25. Switch to code
  26. 26. Flows
  27. 27. Reactive Streams Linear Transformations 28 Demand Source Sinkmap …
  28. 28. Reactive Streams Linear Transformations 29 Source Demand Sink drop map
  29. 29. Reactive Streams Linear Stream Transformations • Deterministic (like for collections) • map, filter, collect, grouped, drop, take, groupBy, ... • Time-Based • takeWithin, dropWithin, groupedWithin, ... • Rate-Detached • expand, conflate, buffer, ... • asynchronous • mapAsync, mapAsyncUnordered, ... 30
  30. 30. Reactive Streams Non-linear Stream Transformations • Fan-In • merge, concat, zip • Fan-Out • broadcast, route, unzip 31
  31. 31. Reactive Streams Fan In 32
  32. 32. Reactive Streams Fan Out 33
  33. 33. Reactive Streams Materialization • Akka Streams separate the what from the how • declarative Source/Flow/Sink DSL to create blueprint • FlowMaterializer turns this into running Actors • this allows alternative materialization strategies • optimization • verification / validation • cluster deployment • only Akka Actors for now, but more to come! 34
  34. 34. Switch to code
  35. 35. Slick
  36. 36. Reactive Streams JDBC • Synchronous I/O • Will block the thread it’s running on. • This is true not only for JDBC, but any DB lib running on top of JDBC, Slick included. We like to avoid that. 37
  37. 37. Reactive Streams Imperfect Solution: Use Future / blocking combo • Put blocking database calls in Future(blocking( ... )) • Contention for Connections (but may be limited by the ExecutionContext) • A saturated thread pool blocks everything. 38
  38. 38. Reactive Streams Slick Every Database contains an AsyncExecutor that manages the thread pool for asynchronous execution of Database I/O Actions. 39 mydb = { dataSourceClass = "org.postgresql.ds.PGSimpleDataSource" properties = { databaseName = "mydb" user = "myuser" password = "secret" } numThreads = 10 }
  39. 39. Akka HTTP
  40. 40. Image by Mathias Doenitz
  41. 41. Interoperability
  42. 42. Image by Mathias Doenitz
  43. 43. Switch to code
  44. 44. Thanks @lutzhuehnken
  45. 45. ©Typesafe 2014 – All Rights Reserved

×