Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Reactive OSGi meets Reactive Java - Don’t cross the streams - T Ward

176 views

Published on

OSGi Community Event 2017 Presentation by Tim Ward [Paremus]

Data collection, filtering and processing is a growing concern for all sorts of applications. A variety of different libraries have been created, all looking to solve the same sorts of problems. The OSGi Alliance has defined the PushStream with a simple streaming API, but Reactive Streams and Java 9’s Flow API use a the Observable model. Which is better? Do I have to use OSGi if I choose Push Streams? Do I have to choose one library or can I use both?

This talk will describe the basic design differences between Push Streams and Observable streams, showing side-by-side examples using both APIs. It will also show how one stream type can be converted into the other, and back meaning that you can always choose the appropriate tool for the job.

With Java 9 and OSGi Push Streams available to developers it has never been a better time to start learning what the new frontiers of streaming Java applications look like. Whether you are looking to process data events from IoT devices, collect metrics from cloud services, or simply to interact with messaging systems, reactive models let you build simple, high throughput systems with ease.

Published in: Technology
  • Be the first to comment

Reactive OSGi meets Reactive Java - Don’t cross the streams - T Ward

  1. 1. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Tim Ward
 tim.ward@paremus.com Reactive OSGi meets Reactive Java Don’t cross the streams?
  2. 2. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Who is Tim Ward? Chief Technology Officer at Paremus 9 years developing OSGi specifications Co-chair of the OSGi IoT Expert Group Interested in Asynchronous Distributed Systems Author of Manning’s Enterprise OSGi in Action http://www.manning.com/cummins @TimothyWard
  3. 3. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 A quick introduction
  4. 4. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 What is Reactive Programming? A set of principles for handling high volumes of “live” data Live data may be historic, but not stored for batch processing Changes should be “reactive” and reflected immediately Operations are stream-centric Resource utilisation must be controlled Asynchronous exchange of data to decouple processing Non-blocking back-pressure to limit the rate of event production Concepts described at https://www.reactivemanifesto.org Standardisation at http://www.reactive-streams.org
  5. 5. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Reactive Java specifications The Reactive Streams initiative Defines the org.reactivestreams API, a specification and a TCK Does not define an implementation OSGi PushStreams Defines the org.osgi.util.pushstream API The API is self-implementing The Java 9 Flow API A simple re-namespacing of the Reactive Streams API
  6. 6. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 The reactive primitives Reactive Streams / Java 9 PushStreams Event / Data Source Publisher<T> PushEventSource<T> Event / Data Sink Subscriber<T> PushEventConsumer<T> “Connection” Subscription AutoCloseable “Pipeline” Processor<T,R> PushStream<T>
  7. 7. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Being Reactive
  8. 8. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Reactive examples In the following examples: The data stream is a monotonic infinite sequence from 0 Data events occur every 50 ms The consumer wants to print out the first N events The consumer wants to print out when they are finished This is a fairly simple, but practical example case Short circuiting is necessary for infinite streams! Let’s compare OSGi Push Streams and Reactive Streams
  9. 9. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Push Streams Demo code…
  10. 10. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Push Streams (2) Highlights: The SimplePushEventSource is powerful It can multiplex multiple consumers from a single event stream It is internally buffered to protect against slow consumers It uses callbacks to allow lazy event generation The PushStream offers a familiar processing pipeline model There is no dependency on the rest of the OSGi API Easy to deploy in “standard Java”
  11. 11. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Reactive Streams Demo code…
  12. 12. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Reactive Streams (2) The code looks pretty horrible, right? This wasn’t really a fair fight… Reactive Streams is a pure API Any non-trivial application must use an implementation The same is true of Java 9’s Flow API! A fairer comparison would be to use a library RxJava was originally from Netflix, and is widely used Note that the example also ran faster…
  13. 13. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Rx Java Demo code…
  14. 14. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Rx Java (2) This example is super-concise It isn’t really a fair comparison either! The other examples use async “push” publication Most RxJava methods make data “on demand” This is one reason why the raw API version ran faster too! We need to move publication onto a different thread…
  15. 15. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Rx Java (3) Demo code…
  16. 16. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Rx Java (4) The code is now very similar to the Push Stream example This is unsurprising, as the primitives are pretty similar So what actually is the difference?
  17. 17. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 The differences
  18. 18. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Creating the connection Connecting a source and consumer is subtly different: AutoCloseable open(PushEventConsumer<T> pec) void subscribe(Subscriber<T> s) A Reactive Stream cannot be closed by the creator! The subscription can do this but it is hidden RxJava’s Disposable isn’t the same Reactive Streams are not (in general) Lambda friendly The Subscriber has four methods, and must be stateful Specific libraries try to work around this
  19. 19. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 The way events are propagated A Subscriber must request data before it can be sent The data cannot be delivered concurrently This is Thread Safe, and reduces CPU exhaustion Push Streams can push data whenever they like In general event delivery is concurrent (see sequential()) Pipeline functions should be stateless and/or Thread Safe Back pressure need not be honoured by the source In many cases these differences have no effect
  20. 20. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Publishers - Hot, Cold, Push and Pull Sources of events can have different characteristics Cold sources can be restarted from the beginning (e.g. a List) Hot sources provide live data and cannot restart (e.g. a sensor) Pull sources provide data “on request” Push sources provide data as a result of something Cold sources are almost always Pull Reactive Streams work well for Pull sources Data can only be sent after it is requested Push Streams API work well for Push sources
  21. 21. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Coping with slow consumers Event processing is easy when the consumer is fast If a Push source is faster than the consumer we have a problem! Reactive Streams must always use buffers for Push sources This is because Subscribers must be called on request Pushstreams use the incoming thread to receive data Pushed data on multiple threads is processed in parallel Long running/blocking operations should use their own thread(s) Buffering is still valuable as it releases the publisher thread
  22. 22. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Coping with slow consumers (2) Buffering is only part of the solution! Buffers cannot be infinite (OutOfMemoryError) Buffers solve “bursts” not a long term mismatch in speed Reactive Streams require you to “fan out” Spread the processing over multiple streams Often this uses a new process (see Akka) Pushstreams can simply use more threads! Buffers allow the number of workers to scale up or down Use the buffer to solve both bursts and speed up
  23. 23. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Coping with slow consumers (3) We can only get so far by speeding up the consumer We need to use back pressure to slow down the data source Reactive Streams back pressure is controlled by request This helps with Pull sources, but how does it help with Push? How long should we hold/drop data for? Pushstreams return a “delay” that should be observed Dropping messages for the period may make sense A periodic producer may be able to slow down A Push source with important messages is always a challenge
  24. 24. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Slow consumer examples In the following examples: The data stream is a monotonic infinite sequence from 0 The data stream produces events every 50 ms The consumer wants to print out the first 500 events The consumer takes 80 ms per event The examples may process the stream on N threads
  25. 25. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Slow consumer examples (2) Demo code…
  26. 26. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Library comparisons Both libraries are easy to deploy (small dependency graph) The default PushStream implementation is small (110kb) Rx Java 2.1.4 is 2100kb (> 10% the size of a custom Java 9 JVM!) The API decisions match the original use cases Handling Hot, Push data events from IoT sensors/message queues Rescaling video streams based on user backpressure This is why Push Streams work naturally with multiple threads The Reactive Streams API is useless, except as an SPI
  27. 27. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Making use of Reactive Streams
  28. 28. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Library interoperability The Reactive Streams API is good for library interoperation A standard PushStream can be treated as a Publisher A standard Publisher can be treated as a PushEventSource This means that any Stream can be linked to any other!
  29. 29. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Interoperation examples Demo code…
  30. 30. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 Questions?
  31. 31. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017 For more about OSGi... Specifications at http://www.osgi.org Enterprise OSGi in Action For more about the Push Streams https://www.osgi.org/developer/specifications/ http://oss.sonatype.org/content/groups/osgi/ For more about Reactive Streams/Rx Java https://github.com/ReactiveX/RxJava http://www.reactive-streams.org Thanks! http://www.paremus.com info@paremus.com http://www.manning.com/ cummins We look forward to seeing you tomorrow morning
  32. 32. Copyright © 2017 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. Reactive OSGi meets Reactive Java October 2017

×