Asynchronous Event Streams – when java.util.stream met org.osgi.util.promise! - Tim Ward

Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Asynchronous Event Streams – when
java.util.stream met org.osgi.util.promise!
Tim Ward
http://www.paremus.com
info@paremus.com

•Chief Technology Oﬃcer at Paremus

•7 years developing OSGi speciﬁcations

•Chair of the OSGi IoT Expert Group

•Interested in Asynchronous Distributed Systems

•Author of Manning’s Enterprise OSGi in Action

•http://www.manning.com/cummins
Who is Tim Ward?
@TimothyWard

Data Structure and Algorithms

You all remember Computer Science lectures, right?
Whether or not we think about them much, data structures are really important

Knowing when you want a List, Set or Map is a good start

Knowing the drawbacks of various implementation types is better!

Thankfully, most of the time it doesn’t matter which implementation you use

N is typically small enough for it not to matter

Modern computers are so fast that even a bad algorithm is “instant”

Most code paths aren’t super hot

The JIT Compiler is often indistinguishable from magic

Working with simple data structures
At their most basic level data structures are “holders” for data

All you can really do is add/remove a value, or look at the existing value(s)
17 23 11 1931
The order in which the values appear is typically determined by the structure

Some data structures restrict visibility to a ﬁrst/last value

We typically refer to processing the values in a data structure as “iteration”

Iteration in Java
We’ve probably all written this code:

for(int i =0; i < list.size(); i ++) {
MyData data = list.get(i);
…
} and this code:

Iterator<MyData> it = list.iterator();
while(it.hasNext()) {
MyData data = it.next();
…
}and this code:

for(MyData data : list) {
…
}
Java’s collections make iteration easy

Problems with iteration
Whilst Java iteration is easy, it’s still easy to make mistakes:
This is bad for Linked Lists!

for(int i =0; i < list.size(); i ++) {
MyData data = list.get(i);
…
}
“External” iteration also pushes control logic into your code!

Java 8 updated the Collections API to support “internal” iteration

Powerful functional concepts were added via the Stream API

Iteration in Java 8
Streaming collections is very simple:

list.stream().forEach(data -> …);
Internal iteration separates the “what” from the “how” in your code

Parallel processing can occur implicitly if the list supports it

Functional pipelines allow for easy processing

list.stream()
.map(MyData::getAge)
.filter(i -> i < 15)
.count();
intermediate operations

terminal operation

Properties of Java 8 Streams
Java 8 Streams have a number of useful properties:
They are lazy

Streams only process data on-demand

This is triggered by a “terminal operation”
They can “short circuit”

Some operations don’t need to see the whole stream

findFirst() and findAny() can return if an element is found
Importantly the Stream is “pulling” the data from the data structure

Streams of data

Streams - the overloaded concept
Before Java 8, a “stream” of data was a java.io.InputStream

Whilst you probably didn’t think of it that way, you still iterated

int read;
while((read = is.read()) != -1) {
byte data = (byte) read;
…
}
The big diﬀerence with an InputStream is that it may block

A thread may get “stuck” waiting for user input, or a slow network

Java NIO has non-blocking input, but is much harder to use

Streams - the overloaded concept (2)
An InputStream behaves like an ordered Collection of bytes

Using a Java 8 Stream over these bytes could make sense

is.forEach(data -> …)
But the InputStream may block the thread indeﬁnitely

If the input is asynchronous then resources are wasted by waiting

A slow function may not process data as fast as it arrives

Rapid bursts of data may overload the consumer

All of this is independent of the data, be it bytes or Objects

Java 8 Streams aren’t able to cope with asynchronous data

Push-based asynchronous streams

•Asynchronous systems oﬀer better performance

•This does assume that there isn’t a single bottleneck!

•Asynchronous Distributed systems typically scale better

•Less time spent waiting for high-latency calls

•More reasonable failure characteristics

•Parallelism is easier to exploit

•Network Infrastructure is intrinsically asynchronous

•Network Engineers have been working this way for decades
Advantages of Asynchronous Programming
100% 100%

Push-based Streams
Push-based-streams are fundamentally diﬀerent from pull-based streams

The processing function is called when data arrives, not when the
previous entry has been processed

Terminal operations must be asynchronous and non-blocking

The Promise is the primitive of Asynchronous Programming

Rather than returning values a push-based stream returns a Promise

A Promise represents a delayed result that will be “resolved” later

OSGi Promises are a good option here

Mapping Java 8 Streams to a push model
The Java 8 Stream has a rich and powerful API

Making it push-based is easier than you might think!

Change the return type of the terminal operations
Pull Push
long count() Promise<Long> count()
boolean anyMatch() Promise<Boolean> anyMatch()
boolean allMatch() Promise<Boolean> allMatch()
Optional<T> min() Promise<Optional<T>> min()
Optional<T> max() Promise<Optional<T>> max()

Problems with this approach
This model actually works very well, but there are some problems

How do we know when an asynchronous stream has ﬁnished?

A pull-based model can simply indicate that there is no more data

Pull-based iteration oﬀers a natural “brake” by processing elements in turn

Push-based systems can be overwhelmed by “Event Storms”

Even a single client thread can be problematic if it is too eager!

How do we cope with this?

Using Events to communicate
Pushing the raw data into the consumer is simple, but insuﬃcient

Consumers need a pushed event to indicate the end of a stream

Events should also be able to propagate failures

An Event is therefore a simple wrapper for data or metadata

public static enum EventType { DATA, ERROR, CLOSE };
public final class AsyncEvent<T> {
public EventType getType() { … }
public T getData() { … }
public Exception getFailure{ … }
}

Learning lessons from Network Engineers
Push-based streams share a lot of concepts with computer networks

Asynchronous delivery

Producer and Consumer may run at diﬀerent rates

TCP solves the “event storm” problem with back-pressure

The producer gets faster, but backs oﬀ if the consumer’s ACK rate drops

Our push-streams need to feed back to the data producer

The simplest way to do this is simply to say “don’t call me for a while”

The AsyncConsumer
A simple functional interface for consuming data:

public interface AsyncConsumer {
long accept(AsyncEvent<T> event);
}
End of Stream events can be detected and back-pressure returned

If positive then the producer should wait at least that long

If zero then continue as soon as possible

If negative then close the stream

Buffers, Windows and Circuit Breakers

An event consumer may receive events on many threads

The event consumer may also be slow to process data

Buffering allows a thread switch, freeing up the producer’s thread

It also allows the consumer to scale up or down the number of workers

Incoming data is queued until it can be processed

The buffer can return back-pressure based on how full it is

Buffering is a built-in feature of the push-based stream

Buffering events
1723113119

Buffering events (2)
What happens when the buffer gets full?

We could use an infinite buffer, but memory isn’t infinite…

Blocking is a possibility, but not very asynchronous!

A good option is simply to close the stream

This is called a “circuit breaker” - it trips if the consumer falls too far behind

This model protects against event storms
1723113119375?

Windowing events
An event consumer may wish to receive batches of events to process

The consumer can then forward an aggregate event

Batches can be deﬁned using an absolute number, or a time window

The underlying behaviour is similar to buﬀering
1723113119
17231131195 37Number
Time

Producing data events

Producing events
So far we’ve focussed on consuming and processing events

This is not very useful unless we can produce events!

Event producers are connected to consumers using the open() method

public interface AsyncEventSource<T> {
Closeable open(AsyncConsumer<? super T> event);
}
The returned Closeable can be used to “end” the stream

This is useful when the data stream is inﬁnite!

Helpful behaviours
Event Producers have to cope with multiple registrations

They also have to handle back-pressure from the consumer

Sometimes the producer has no choice about waiting!

Writing a producer should not be hard, so we provide help

Buﬀers/Circuit breakers mean that the producer can “ignore” back pressure

The RFC also describes a “helper” for managing multiple registrations

Playing with streams

Playing with push streams
Using push streams is a lot like creating Java 8 streams

Creating them is a little diﬀerent!

A Java 8 stream is created directly from a collection (or Spliterator)

Currently Push Streams use a factory to turn a Source into a Stream

The stream does not open the source until a terminal operation is invoked

No events are pushed before your pipeline is ready!

Examples

Example streams
All examples use an Event Source which sends Integers from 0 to N-1
Find the biggest?

createStream(10)
.max(Integer::compare);
9 (wrapped in a Promise!)
Find the ﬁrst odd number?

createStream(10)
.filter(x -> x % 2 == 1)
.findFirst();
1 (also in a Promise!)
Find the combined total?

createStream(10)
.reduce(0,(a,b)-> a + b);
45 (get the idea yet?)
Count the time windowed events?

createStream(100)
.window(200, Collection::size)
.collect(toList());
It depends!

Demo (sort of!)

•For more about OSGi...

• Speciﬁcations at http://www.osgi.org

• Enterprise OSGi in Action

• http://www.manning.com/cummins

•For more about the Push Streams

• http://github.com/osgi/design

•See the IoT Competition at 1745 today

Questions?
Thanks!
http://www.paremus.com
info@paremus.com
http://www.manning.com/cummins

Asynchronous Event Streams – when java.util.stream met org.osgi.util.promise! - Tim Ward

Recommended

Recommended

More Related Content

Similar to Asynchronous Event Streams – when java.util.stream met org.osgi.util.promise! - Tim Ward

Similar to Asynchronous Event Streams – when java.util.stream met org.osgi.util.promise! - Tim Ward (20)

More from mfrancis

More from mfrancis (20)

Recently uploaded

Recently uploaded (20)

Asynchronous Event Streams – when java.util.stream met org.osgi.util.promise! - Tim Ward