Reactive Streams 1.0.0 and Why You Should Care (webinar)

28,254 views

Published on

In this presentation, Akka Team Lead and author Roland Kuhn presents the freshly released final specification for Reactive Streams on the JVM. This work was done in collaboration with engineers representing Netflix, Red Hat, Pivotal, Oracle, Typesafe and others to define a standard for passing streams of data between threads in an asynchronous and non-blocking fashion. This is a common need in Reactive systems, where handling streams of "live" data whose volume is not predetermined.

The most prominent issue facing the industry today is that resource consumption needs to be controlled such that a fast data source does not overwhelm the stream destination. Asynchrony is needed in order to enable the parallel use of computing resources, on collaborating network hosts or multiple CPU cores within a single machine.

Here we'll review the mechanisms employed by Reactive Streams, discuss the applicability of this technology to a variety of problems encountered in day to day work on the JVM, and give an overview of the tooling ecosystem that is emerging around this young standard.

Published in: Technology
0 Comments
30 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
28,254
On SlideShare
0
From Embeds
0
Number of Embeds
20,492
Actions
Shares
0
Downloads
121
Comments
0
Likes
30
Embeds 0
No embeds

No notes for slide

Reactive Streams 1.0.0 and Why You Should Care (webinar)

  1. 1. Reactive Streams 1.0.0 and why you should care Dr. Roland Kuhn @rolandkuhn — Akka Tech Lead
  2. 2. Reactive Streams 1.0.0
  3. 3. Philosophical Background 3 “You cannot step twice into the same stream. For as you are stepping in, other waters are ever flowing on to you.” — Heraclitus
  4. 4. What is a Stream? • ephemeral flow of data • possibly unbounded in size • focused on describing transformation • can be formed into processing networks 4
  5. 5. Reactive Traits
  6. 6. The Problem: Getting Data across an Asynchronous Boundary without running OutOfMemory
  7. 7. Enter Reactive Streams 7 “Reactive Streams is an initiative to provide a standard for asynchronous stream processing with non-blocking back pressure on the JVM.” — reactive-streams.org
  8. 8. Participants • Engineers from (among others): • Netflix • Pivotal • Red Hat • Typesafe • Individuals like Doug Lea and Todd Montgomery 8
  9. 9. Motivation • all participants face the same basic problem • all are building tools for their community • a common solution benefits everybody • interoperability to make best use of efforts • proposal to include in JDK9 (j.u.c.Flow) 9
  10. 10. Recipe for Success • minimal interfaces • rigorous specification of semantics • full TCK for verification of implementation • complete freedom for many idiomatic APIs 10
  11. 11. Supply and Demand • data items flow downstream • demand flows upstream • data items flow only when there is demand • recipient is in control of incoming data rate • data in flight is bounded by signaled demand 11 Publisher Subscriber data demand
  12. 12. Dynamic Push–Pull • “push” behavior when consumer is faster • “pull” behavior when producer is faster • switches automatically between these • batching demand allows batching data 12 Publisher Subscriber data demand
  13. 13. Back-Pressure is Contagious • C is slow • B must slow down • A must slow down 13 CA B
  14. 14. Back-Pressure can be Propagated • TCP for example has it built-in 14 CA B networkhosts
  15. 15. The Meat 15 trait Publisher[T] { def subscribe(sub: Subscriber[T]): Unit } trait Subscription { def request(n: Int): Unit def cancel(): Unit } trait Subscriber[T] { def onSubscribe(s: Subscription): Unit def onNext(e: T): Unit def onError(t: Throwable): Unit def onComplete(): Unit }
  16. 16. How does it Connect? 16 SubscriberPublisher subscribe onSubscribeSubscription
  17. 17. How does it Flow? 17 SubscriberPublisher request onNextElements request onNextElements request
  18. 18. How does it Complete? 18 SubscriberPublisher request onNextElements onComplete
  19. 19. How does it Fail? 19 SubscriberPublisher request onNextElements onError
  20. 20. Reactive Streams • asynchronous non-blocking data flow • asynchronous non-blocking demand flow • minimal coordination and contention • message passing allows for distribution across
 applications, nodes, CPUs, threads, actors, … 20 http://reactiveEstreams.org/
  21. 21. What is it good for?
  22. 22. Streams are Ubiquitous • Streaming SIMD Extensions (SSE) — Intel,1999 • ingesting, transforming and emitting data • requests & responses flowing through a system • streams are graphical and intuitive 22
  23. 23. Moving Bulk Data over the Network • content streaming (e.g. video or audio) • data storage / backup • data replication between data centers 23
  24. 24. Mobile Devices • streams of updates from the server • streams of user commands from the client • bad connection quality—need for back-pressure 24
  25. 25. Internet of Things • ingesting large numbers of low-rate streams • conflating or extrapolating data • aggregating many streams • streaming towards the fleet of devices 25
  26. 26. Realtime Data Analysis • real-time business intelligence • complex event processing • temporal correlations (e.g. credit card fraud) 26
  27. 27. The Ecosystem
  28. 28. RxJava • «AnAPIforasynchronousprogrammingwith observablestreams» — reactivex.io • inspiration for Reactive Streams interfaces • RxJavaReactiveStreams provides RS compliant wrapper for Observable 28 io.reactivex:rxjava-reactive-streams:1.0.0
  29. 29. Project Reactor • «afoundationallibraryforbuildingreactivefast-data applicationsontheJVM» — projectreactor.io • Stream directly implements Publisher and offers a rich set of transformations 29 io.projectreactor:reactor-stream:2.0.3.RELEASE
  30. 30. Akka Streams • compose any kind of stream processing topology from immutable and reusable blueprints • creates RS compliant Publisher/Subscriber when materializing a flow graph into Actors • high-level flow graph DSL 30 com.typesafe.akka:akka-stream-experimental:1.0-RC3
  31. 31. ratpack • «RatpackisasetofJavalibrariesthatfacilitatefast, efficient,evolvableandwelltestedHTTP applications.» — ratpack.io • accepts Publishers of streams to be sent to the client • chunked responses, server-sent events, websockets 31 io.ratpack:ratpack-core:0.9.17
  32. 32. vert.x 3.0 • «Vert.xisalightweight,highperformanceapplication platformfortheJVMthat'sdesignedformodern mobile,web,andenterpriseapplications.» — vertx.io • Reactive Streams Integration Module
 (https://github.com/vert-x3/vertx-reactive-streams) 32 io.vertx:vertx-reactive-streams:3.0.0-milestone6
  33. 33. Slick 3.0 • «FunctionalRelationalMappingforScala»— slick.typesafe.com • database result sets can be streamed, returning a Publisher instead of a collection • no streaming inserts yet 33 com.typesafe.slick:slick_2.11:3.0.0
  34. 34. Kafka • «ApacheKafkaispublish-subscribemessaging rethoughtasadistributedcommitlog.» — kafka.apache.org • Reactive Kafka (by Softwaremill):
 consumer/producer are RS Subscriber/Publisher 34 com.softwaremill:reactive-kafka_2.11:0.6.0
  35. 35. AMQP • «Tobecomethestandardprotocolforinteroperability betweenallmessagingmiddleware» — amqp.org • https://github.com/ScalaConsultants/reactive-rabbit • fully asynchronous API • consumer/publisher are RS Subscriber/Publisher 35 io.scalac:reactive-rabbit_2.11:1.0.0
  36. 36. MongoDB • «TheofficialMongoDBReactiveStreamsJavaDriver, providingasynchronousstreamprocessingwithnon- blockingbackpressureforMongoDB.»—
 http://mongodb.github.io/mongo-java-driver- reactivestreams/ • all queries return Publishers • no streaming inserts yet 36 org.mongodb:mongodb-driver-reactivestreams:1.0.1
  37. 37. … and more are coming (TBC) • Cassandra • Spark • Riak 37
  38. 38. Reactive Streams in the Typesafe Reactive Platform
  39. 39. 39 O/S-level network stack Java NIO (JDK) Akka IO Akka HTTP Core Akka HTTP application level Akka Streams Database Driver Slick 3.0 Play Framework ReactiveStreamsIntegration: preview available
  40. 40. Streaming Database Results to the Client 40 import DefaultJsonProtocol._ implicit val denormOrderFormat = jsonFormat5(DenormalizedOrder.apply) val db = Database.forConfig("reportingDB") private def getFromDb(userId: Int): Publisher[DenormalizedOrder] = db.stream(denormalizedOrders.filter(_.userId === userId).result) Http().bindAndHandle( (get & path("orders" / IntNumber)) { userId => val pub = Source(getFromDb(userId)) .transform(() => new ToJsonArray) complete(HttpEntity.Chunked.fromData(`application/json`, pub)) }, "localhost", 8080)
  41. 41. REACTIVE PLATFORM
 Full Lifecycle Support for Play, Akka, Scala and Spark Give your project a boost with Reactive Platform: • Monitor Message-Driven Apps • Resolve Network Partitions Decisively • Integrate Easily with Legacy Systems • Eliminate Incompatibility & Security Risks • Protect Apps Against Abuse • Expert Support from Dedicated Product Teams Enjoy learning? See about the availability of on-site training for Scala, Akka, Play and Spark! Learn more about our offersCONTACT US
  42. 42. ©Typesafe 2015 – All Rights Reserved

×