Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
An event consumer may receive events on many threads

The event consumer may also be slow to process data

Buffering allows...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
O...
Upcoming SlideShare
Loading in …5
×

Scalable Event Processing – Pushing the limits with Push Streams! - Tim Ward

456 views

Published on

OSGi Community Event 2016 Presentation by Tim Ward (Paremus)

Data is being produced everywhere, there are sensors in thousands of homes, metrics collection from your cloud applications, industrial sensors manage the safe provision of water and electricity. The question is, what do we do with all of this data? How do you cope with thousands (or millions) of push-based data events per second?

A crucial part of any of any solution is to have the right primitives. An asynchronous, push-based event processing pipeline is essential. OSGi Push Streams offer the power and simplicity of Java 8 streams, but with the power of asynchronous push-based events. This talk will describe the work happening in OSGi’s Push Streams RFC, using streams and promises to build scalable event processing pipelines.

Background
The OSGi service platform has existed as a modular micro-service runtime for well over a decade, but more recently it has embraced asynchronous programming as a core part of the OSGi toolkit. With the introduction of the Async Service, and OSGi promises it is now easier than ever to build asynchronous applications. The focus so far has been on asynchronous requests, but what about streaming systems, ones that need to process or filter thousands, millions, or billions of events before an answer is reached, or perhaps there is no end to the stream at all!

Processing distributed events is the purpose of the Distributed Eventing RFC being discussed by the OSGi Enterprise Expert Group. As the lead author of the RFC, and the lead of the Asynchronous Service specification, Tim has a detailed understanding of the complexities of Asynchronous programming models. Tim will be able to describe the current prototyping around OSGi’s Asynchronous Streaming model, and how it fits with the other popular asynchronous standards.

Published in: Technology
  • Be the first to comment

Scalable Event Processing – Pushing the limits with Push Streams! - Tim Ward

  1. 1. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 OSGi Community Event 2016 co-located at EclipseCon Europe 2016 Scalable Event Processing Pushing the limits with Push Streams! Tim Ward http://www.paremus.com info@paremus.com
  2. 2. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 •Chief Technology Officer at Paremus •8 years developing OSGi specifications •Chair of the OSGi IoT Expert Group •Interested in Asynchronous Distributed Systems •Author of Manning’s Enterprise OSGi in Action •http://www.manning.com/cummins Who is Tim Ward? @TimothyWard
  3. 3. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Working with Data
  4. 4. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Iteration in Java We’ve probably all written this code: for(int i =0; i < list.size(); i ++) { MyData data = list.get(i); … } and this code: Iterator<MyData> it = list.iterator(); while(it.hasNext()) { MyData data = it.next(); … }and this code: for(MyData data : list) { … } Data is everywhere, and Java’s collections make processing data easy
  5. 5. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Problems with iteration Whilst Java iteration is easy, it’s still easy to make mistakes: This is bad for Linked Lists! for(int i =0; i < list.size(); i ++) { MyData data = list.get(i); … } “External” iteration also pushes control logic into your code! Parallelising the processing is a huge task Java 8 updated the Collections API to support “internal” iteration Powerful functional concepts were added via the Stream API
  6. 6. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Iteration in Java 8 Streaming collections is very simple: list.stream().forEach(data -> …); Internal iteration separates the “what” from the “how” in your code Parallel processing can occur implicitly if the list supports it Functional pipelines allow for easy processing list.stream() .map(MyData::getAge) .filter(i -> i < 15) .count(); intermediate operations terminal operation
  7. 7. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Properties of Java 8 Streams Java 8 Streams have a number of useful properties: They are lazy Streams only process data on-demand This is triggered by a “terminal operation” They can “short circuit” Some operations don’t need to see the whole stream findFirst() and findAny() can return if an element is found Importantly the Stream is “pulling” the data from the data structure
  8. 8. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Streams of data
  9. 9. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Streams - the overloaded concept Before Java 8, a “stream” of data was a java.io.InputStream Whilst you probably didn’t think of it that way, you still iterated int read; while((read = is.read()) != -1) { byte data = (byte) read; … } The big difference with an InputStream is that it may block A thread may get “stuck” waiting for user input, or a slow network Java NIO has non-blocking input, but is much harder to use
  10. 10. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Streams - the overloaded concept (2) An InputStream behaves like an ordered Collection of bytes Using a Java 8 Stream over these bytes could make sense is.forEach(data -> …) But the InputStream may block the thread indefinitely If the input is asynchronous then resources are wasted by waiting A slow function may not process data as fast as it arrives Rapid bursts of data may overload the consumer All of this is independent of the data, be it bytes or Objects Java 8 Streams aren’t able to cope with asynchronous data
  11. 11. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Push-based asynchronous streams
  12. 12. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Push-based Streams Push-based-streams are fundamentally different from pull-based streams The processing function is called when data arrives, not when the previous entry has been processed Asynchronous operation allows for high throughput and parallelisation Terminal operations must be asynchronous and non-blocking The Promise is the primitive of Asynchronous Programming Rather than returning values a push-based stream returns a Promise A Promise represents a delayed result that will be “resolved” later OSGi Promises are a good option here
  13. 13. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Mapping Java 8 Streams to a push model The Java 8 Stream has a rich and powerful API Making it push-based is easier than you might think! Change the return type of the terminal operations Pull Push long count() Promise<Long> count() boolean anyMatch() Promise<Boolean> anyMatch() boolean allMatch() Promise<Boolean> allMatch() Optional<T> min() Promise<Optional<T>> min() Optional<T> max() Promise<Optional<T>> max()
  14. 14. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Problems with this approach This model actually works very well, but there are some problems How do we know when an asynchronous stream has finished? A pull-based model can simply indicate that there is no more data Pull-based iteration offers a natural “brake” by processing elements in turn Push-based systems can be overwhelmed by “Event Storms” Even a single client thread can be problematic if it is too eager! How do we cope with this?
  15. 15. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Using Events to communicate Pushing the raw data into the consumer is simple, but insufficient Consumers need a pushed event to indicate the end of a stream Events should also be able to propagate failures An Event is therefore a simple wrapper for data or metadata public static enum EventType { DATA, ERROR, CLOSE }; public final class PushEvent<T> { public EventType getType() { … } public T getData() { … } public Exception getFailure{ … } }
  16. 16. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Learning lessons from Network Engineers Push-based streams share a lot of concepts with computer networks Asynchronous delivery Producer and Consumer may run at different rates TCP solves the “event storm” problem with back-pressure The producer gets faster, but backs off if the consumer’s ACK rate drops Our push-streams need to feed back to the data producer The simplest way to do this is simply to say “don’t call me for a while”
  17. 17. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 The AsyncConsumer A simple functional interface for consuming data: public interface PushEventConsumer { long accept(PushEvent<T> event); } End of Stream events can be detected and back-pressure returned If positive then the producer should wait at least that long If zero then continue as soon as possible If negative then close the stream
  18. 18. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Buffers, Windows and Circuit Breakers
  19. 19. An event consumer may receive events on many threads The event consumer may also be slow to process data Buffering allows a thread switch, freeing up the producer’s thread It also allows the consumer to scale up or down the number of workers Incoming data is queued until it can be processed The buffer can return back-pressure based on how full it is Buffering is a built-in feature of the push-based stream Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Buffering events 1723113119
  20. 20. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Buffering events (2) What happens when the buffer gets full? We could use an infinite buffer, but memory isn’t infinite… Blocking is a possibility, but not very asynchronous! A good option is simply to close the stream This is called a “circuit breaker” - it trips if the consumer falls too far behind This model protects against event storms 1723113119375?
  21. 21. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Windowing events An event consumer may wish to receive batches of events to process The consumer can then forward an aggregate event Batches can be defined using an absolute number, or a time window The underlying behaviour is similar to buffering 1723113119 17231131195 37Number Time
  22. 22. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Producing data events
  23. 23. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Producing events So far we’ve focussed on consuming and processing events This is not very useful unless we can produce events! Event producers are connected to consumers using the open() method public interface PushEventSource<T> { Closeable open(PushEventConsumer<? super T> event); } The returned Closeable can be used to “end” the stream This is useful when the data stream is infinite!
  24. 24. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Helpful behaviours Event Producers have to cope with multiple registrations They also have to handle back-pressure from the consumer Sometimes the producer has no choice about waiting! Writing a producer should not be hard, so we provide help The SimplePushEventSource provides buffered publishing The Buffer/Circuit breaker allows the producer to “ignore” back pressure A connect Promise allows event delivery to be lazily started
  25. 25. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Playing with streams
  26. 26. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Playing with push streams We need an asynchronous source of events The UK rail network has an open data API! Events are batched (for performance) and delivered using STOMP Events are JSON objects and so can easily be processed! Let’s have a play!
  27. 27. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016
  28. 28. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 •For more about OSGi... • Specifications at http://www.osgi.org • Enterprise OSGi in Action • http://www.manning.com/cummins •For more about the Push Streams • http://github.com/osgi/design •See the IoT Competition at 1745 today Questions? Thanks! http://www.paremus.com info@paremus.com http://www.manning.com/cummins
  29. 29. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Addendum
  30. 30. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 Demo Gremlins This morning I was testing my demo I couldn’t connect to the Network Rail Feed They had Deactivated my account! No demo = Unhappy Audience :( But OSGi is amazing technology from the future! I built a new module using MQTT to consume events from the OSGi Train 1 hour later - Working Demo. No other code had to change! Network Rail reactivated my account 10 minutes later…
  31. 31. Copyright © 2005 - 2016 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved. OSGi Community Event Nov 2016 www.paremus.com @Paremus info@paremus.com

×