A presentation given to Overstock.com IT at annual conference. Twitter @TECHknO 2015. Goal of the presentation is to provide a good introduction to the reactive programming model with RxJava.
1. RxJava – Reactive Extensions for the JVM
An introduction to RxJava
By
Sanjay Acharya and Joel Weight
Twitter: @sanjayvacharya and @digitaljoel
1
RxJava 9/30/2015
2. Some reflection…
Only recently…
Large applications with 10s of servers
Seconds of response times
Hours of offline maintenance
Gigabytes of Data
Today’s Applications
Deployed on anything from Mobile to Cloud Clusters with thousands of
multi core processors
Millisecond response times
100% uptime
Petabytes of Data
Traditional Architectures fail to meet today’s demands
9/30/2015RxJava
2
3. So How do we meet today’s
demands ?
Reactive Systems
9/30/2015RxJava
3
7. Functional Reactive Programming
Values are pushed when ready in a non
blocking manner
Facilitates parallelism
Application of ‘functions’ on data stream
to transform data – Examples - map,
filter, zip
9/30/2015RxJava
7
Reactive Model is Push rather than Pull
8. ReactiveX
A library for composing asynchronous and event based
programs by using observable sequences
Created by Microsoft initially for the .NET platform
By Erik Meijer
Extends Observer Pattern to
Support sequence of data and/or events
Adds operators that allow you to compose sequences together
declaratively
Abstracts away concerns around
Threading and Thread Safety
Concurrent data structures and non blocking I/O.
RxJava is a port of Reactive Extensions created by Netflix
9/30/2015RxJava
8
11. rx.Observable
9/30/2015RxJava
11
public class Observable<T> { … }
Emits zero or more values
Life Cycle
Notifies Observer
onNext element
Completes with
onError
Or onCompletion
Observable Observer
onNext(T)
onComplete()
onError(Throwable)
12. Observable Creation
9/30/2015
12
Observable<Integer> o = Observable
.create(new Observable.OnSubscribe<Integer>(){ // Action
@Override
public void call(Subscriber<Integer> subscriber){
try {
for (int i = 0; i < 10
&& ! subscriber.isUnsubscribed(); i++) {
subscriber.onNext(i);
}
if (!subscriber.isUnsubscribed()) {
subscriber.onCompleted();
}
}
catch (RuntimeException e) {
if (!subscriber.isUnsubscribed()) {
subscriber.onError(e);
}
}
}
});
RxJava
Using JDK 7
Downstream Slide
examples will use
JDK 8 Lambdas
13. Subscribing (Observer)
9/30/2015
13
Observable<Integer> o = Observable.create(…);
Subscription s = o.subscribe(new Observer<Integer>() {
@Override
public void onNext(Integer i) {
System.out.print(i);
}
@Override
public void onCompleted() {
System.out.println(“nAll Done!”);
}
@Override
public void onError(Throwable t) {
t.printStackTrace();
}
});
Prints –
0 to 9!
Observable from
previous slide
RxJava
0123456789
All Done!
Prints –
All Done!
14. 9/30/2015RxJava
14
Operator Description Example
Observable.just(T) Simple values wrapped Observable.just(“HelloWorld”);
Observable.empty() Empty Sequence that
completes right away
Observable.empty();
Observable.from(Iterable<T>) Sequence from an
Iterable
Observable.from(Lists.newArrayList(“A”, “B”));
Observable.from(T []) Sequence from Array Observable.from(new Integer[] {0, 1, 2, 3, 4});
Observable.defer(ObservableFactory) Defer emitting of items
for each subscriber
Example shown later.
Observable.interval(long, TimeUnit) Emit a sequence of
numbers separated by
an interval
Observable.interval(100, TimeUnit.MILLISECONDS);
Observable.range(start, count) Emits a sequence of
integers in a range
Observable.range(100, 20);
Observable.create(OnSubscribe<T>) Most flexible and
powerful
Observable.create(new OnSubscribe…());
15. Observable Vs Iterator
Event Iterable (PULL) Observable (PUSH)
Retrieve Data T next(); onNext(T);
Discover the Error throws Exception onError(Throwable)
Complete !hasNext() onCompleted();
9/30/2015RxJava
15
20. Map
Transform the items
emitted by an
Observable by applying
a function to each item
9/30/2015RxJava
20
Observable<Integer> o = Observable.range(0,10);
Observable<Integer> timesTen = o.map(t -> t * 10);
21. Flatmap
Transforms items emitted from
an Observable into Observables
and then merges those
emissions into a single
Observable
9/30/2015RxJava
21
Observable<Integer> o = Observable.range(0, 10);
Observable<Integer> numbersAndNegatives
= o.flatmap(t -> Observable.just(t, -t));
22. Filtering
Operators that selectively emit items from a source
Observable
Commonly used
filter
distinct
take
first
9/30/2015RxJava
22
23. Filter
Emit only items from an
Observable that match a
predicate test
9/30/2015RxJava
23
Observable<Integer> o = Observable.range(0,10);
Observable<Integer> evenNos = o.filter(t -> (t % 2) == 0);
24. Take
Emit only the first n items
emitted by an Observable.
9/30/2015RxJava
24
Observable<Integer> o = Observable.range(0,10);
Observable<Integer> firstTwo = o.take(2);
25. Combining Observables
Operators that work with multiple source Observables to
create a single Observable
Commonly used
Merge
Concat
Zip
zipWith
9/30/2015RxJava
25
26. Merge
Combine multiple Observables by
merging their emissions
May interleave items
9/30/2015RxJava
26
Observable<Integer> first = Observable.range(0,10);
Observable<Integer> second = Observable.range(10, 20);
Observable<Integer> merged
= Observable.merge(first, second);
27. Concat
Similar to merge but combines
observable without interleaving
9/30/2015RxJava
27
Observable<Integer> first = Observable.range(0,10);
Observable<Integer> second = Observable.range(10, 20);
Observable<Integer> concat
= Observable.concat(first, second);
28. Zip
combine the emissions of multiple
Observables together via a
specified function and emit single
items for each combination based
on the results of this function
9/30/2015RxJava
28
Observable<Integer> first = Observable.just(1, 2, 3);
Observable<String> second = Observable.just(“A”,“B”, “C”, “D”);
Observable<String> zipped =
Observable.zip(first, second (x,y) -> String.valueOf(x)+y);
// on subscription emits “1A”, “2B”, “3C”
29. ZipWith
Instance-version of zip
9/30/2015RxJava
29
Observable<Integer> first = Observable.just(1, 2, 3);
Observable<String> second = Observable.just(“A”,“B”, “C”, “D”);
Observable<String> zipped =
first.zipWith(second, (x, y) -> String.valueOf(x)+y);
// on subscription emits “1A”, “2B”, “3C”
30. Decision Tree of Observables
9/30/2015RxJava
30
http://reactivex.io/documentation/operators.html#tree
68. Observable Product - Test
9/30/2015RxJava
68
Long productId = 1L;
Observable<Product> product = getProduct(productId);
// The Observable only starts doing something after
// an observer subscribes
product.subscribe(p -> LOG.debug(p));
71. Observable Creation
9/30/2015
71
Observable<Customer> getCustomers() {
return Observable.create(new Observable.OnSubscribe<Customer>(){
public void call(Subscriber<Customer> subscriber){
Connection conn = getConnection();
Stmt stmt = conn.createStatement();
String sql = “select id, name from customer”;
ResultSet rs = stmt.executeQuery(sql);
while (rs.next()) {
Long id = rs.getLong(“id”);
String name = rs.getString(“name”);
Customer c = new Customer(id, name);
subscriber.onNext(c);
}
subscriber.onComplete();
…
}
});
}
Observable<Customer> customer = getCustomers();
RxJava
72. Cold Observable
Is the more common types of Observables you
will deal with
Does nothing until subscribed to
In example shown, Database was not accessed
till a subscriber was present
9/30/2015RxJava
72
73. Hot Observable
Hot observables are "live" sequences that are occurring
whether they are being observed or not.
The canonical example is a mouse, which of course is
free to move regardless of whether you subscribe to an
Observable<MouseMoveEvent> or not.
9/30/2015RxJava
73
74. Hot Observable Example – Earth emitting Greetings
9/30/2015RxJava
74
Hello 1439838374206
Hola1439838374207
Namaste 1439838374210
Bonjour 1439838375707
Emitting Greetings in multiple languages + Timestamp
77. Hot Observable using - ConnectableObservable
A Connectable Observable resembles an ordinary
Observable, except that it does not begin emitting items
when it is subscribed to, but only when its connect()
method is called. Calling connect(), makes the
Observable ‘HOT’.
ConnectableObservables can be created from any
Observable by calling publish() or replay()
9/30/2015RxJava
77
87. PublishSubject to create a Hot Observable
Another way to create a Hot Observable
Left as an exercise for the viewer
An example provided in slide deck
addendum
9/30/2015RxJava
87
94. Unit Testing
rx.observers.TestSubscriber
Do not use in production code
Has a bunch of convenience methods
awaitTerminalEvent()
assertNoErrors()
assertReceivedOnNext(…)
assertUnsubscribed()
9/30/2015RxJava
94
95. Testing
9/30/2015RxJava
95
TestSubscriber<String> ts = new TestSubscriber<>();
Observable.interval(200, TimeUnit.MILLISECONDS)
.take(3)
.map(t -> {return “Hello:” + t})
.subscribe(ts);
ts.awaitTerminalEvent();
ts.assertNotErrors();
ts.assertReceivedOnNext(Lists.newArrayList(“Hello:0”,
“Hello:1”, “Hello:2”));
This is async!
96. What is the hardest part of
concurrent programming?
9/30/2015RxJava
96
97. Debugging
Never easy!
Not just a problem of Rx as much as simply the nature of it
Stepping through code is severely limited
Stack Trace can be difficult to understand
System.out.println or debug statements your friends
doOnNext(), doOnError()
9/30/2015RxJava
97
98. doOnNext()
@Test
public void doOnNextTest() {
List<String> received = Lists.newArrayList();
Observable<String> o = Observable.just(“a”,“b”,“c”) // Test Observable
.doOnNext( s -> received.add(s));
TestSubscriber<String> test = new TestSubscriber();
o.subscribe(test);
test.awaitTerminalEvent();
assertEquals(Lists.newArrayList(“a”,“b”,“c”), received);
}
9/30/2015RxJava
98
101. The Good
Makes Asynchronous code easy to develop
Lots of online resources and examples
Strong use of functional style of programming
Netflix used RxNetty and Reactive Programming
to considerable performance advantage over
Tomcat
Web Service Clients like Jersey and Retrofit
support RxJava
9/30/2015RxJava
101
102. The Bad
The Bad
Learning curve and mind set change to use
functional reactive programming
Appears like more code and more complex code
initially – Requires time for it to grow on you
If on java 7, anonymous classes would make this a
non-starter
Thank heavens for Java 8 Lambdas
9/30/2015RxJava
102
103. The Ugly
The Ugly
Debugging and Stepping through async
code is painful
Stack Traces are difficult to follow
Author’s recommend
System.out.println or Log.debug(..)
doOnNext() and doOnError()
9/30/2015RxJava
103
104. Resources
Intro to Rx - http://www.introtorx.com/
ReactiveX – http://reactivex.io
RxMarbles - http://rxmarbles.com/
RxJava - https://github.com/ReactiveX/RxJava
Air Traffic Control Simulation
https://github.com/digitaljoel/rxAir
9/30/2015RxJava
104
105. Thank you and Questions…
9/30/2015RxJava
105
https://github.com/digitaljoel/rxAir
108. Subject
Subject is an object that can be both Observable and
Observer
Bridge between Observable and Observer
Subject can subscribe to an Observable acting like an
Observer
Subject can emit new items like an Observable
9/30/2015RxJava
108
109. Publish Subject
Subject that, once an Observer
has subscribed, emits all
subsequently observed items to
the subscriber
9/30/2015RxJava
109
PublishSubject<Object> subject = PublishSubject.create();
subject.onNext(“one”);
subject.subscribe(t - > LOG.info(“Observer1:” + t));
subject.onNext(“two”);
subject.subscribe(t - > LOG.info(“Observer2:” + t));
subject.onNext(“three”);
110. Replay Subject
Subject that buffers all items it
observes and replays them to any
Observer that subscribes.
9/30/2015RxJava
110
ReplaySubject<Object> subject = ReplaySubject.create();
subject.onNext(“one”);
subject.onNext(“two”);
subject.onNext(“three”);
subject.onCompleted();
subject.subscribe(t - > LOG.info(“Observer1:” + t));
subject.subscribe(t - > LOG.info(“Observer2:” + t));
Responsive – React to users demands
Elastic – React to load
Resilient – React to Errors and Failures
Message Driven – React to events and messages
A Stream of Datum
Finite or Infinite
Disclaimer – All images used in this slide are property of original authors. We are simply using the same for demonstration.
Reactive Programming is a push based model rather than pull.
Events are pushed to you rather than you blocking and pulling for it
Promotes parallelism and rich transformation of data via pure and higher order functions
A behavior GOF Design Pattern
A way to message dependent objects of changes
Subject keeps a list of Objects that need to be notified of
ReactiveX extends the behavior of the Observer Pattern and provides the mentioned items
Observable and Subjects are two producing entities
- Observable is the source of data stream
Observer listens to emitted values
Observer subscribes to an Observable
Observer reacts to items emitted by Observable
Observable that emits 0 to 9.The observer is notified via the onNext and completed with onCompleted or onError
Some common creational operators mentioned above
Iterable consumer blocks synchronously and pulls values from producer
Observable producer asynchronously pushes values to the Observer whenever values are available
RxJava Enhances GOF Observer pattern with onCompleted and onError to notify Observer
Maps items to items of another type.
This example does Integer to Integer
You can go from any type to any other type using the function.
The key is that the result of the function used by flatmap is an Observable.
Resulting Observables are all merged together in true merge fashion (interleved.)
Map is item -> item.
FlatMap is item -> Observable
Filter - Filter based of a Predicate test
Distinct - Distinct elements from source
Take - Take ‘n’ items from source
First - – Take first item
Sample filters to even numbers only.
Notice that result of Func1 must be a Boolean.
Your service returns 50 recommendations but you know you only need to display 6 of them.
Merge – combine multiple observables into one by combining items
Concat – Combines multiple observables with another without interleaving
Zip – Combine emission of multiple observables together via a specified function and emit a single item for each combination of items
One possible result could include 0, 10, 1, 11…
Note that the first and second observables could be emitting concurrently, but the result of concat maintains the order.
Result would be 1..20, not interleaved.
Note different marble diagram here. This one is more clear than the one matching the other styles.
Note that we are calling zipWith on first, not Observable.zip
io() - > Scheduler for I/O bound work. Executor Thread pool grows as needed
Computation() - > Returns a Scheduler for computational work like loops, algorithmic etc.
newThread() - > Returns a Scheduler that spawns a new Thread for each unit of work
Trampoline() -> Queues work on the current thread to be executed after current work completes
Immediate() -> Creates and returns a scheduler that executes work immediately on the current thread
Test() -> Creates a Scheduler that allows for debugging
fromExecutor() - > From a custom Executor
Product Gateway Service
= Aggregates ( Base Product + Product Price + Inventory) to give a Product
Base Product Service provides a BaseProduct which contains id, description and BaseOptions. Each Base Option contains an id and description.
ProductGatewayService uses the three Web Service Clients to obtain back and aggregate the data to create a Product
In the case of our model. We have a few assumptions
- A Product itself has no direct Price or Inventory – they are inferred from Options
- The display Price of the product is the lowest Option Price
- The total Inventory of a Product is the sum of Option inventory
- The Base Option is augmented with inventory and price from the Inventory and Pricing web services accordingly to produce the Product
Blocking I/O bound call
Two simple ways of converting a blocking call to a deferred one.
Use the defer() operator that that an ObservableFactory to defer the creation of the observable.
Use the Observable.OnSubscribe() to make the invocation and notify.
Sanjay
A Hot Earth Observable emitting Greetings in multiple languages in the hope some ‘subscriber’ hears…
One observer picks up the signal and starts observing…
After a period of time a second Observer picks up the signal and starts hearing…
AlienGreetingOnSubscribe is responsible for emitting greetings periodically in all languages….
Replaying of all items emitted to subscribers
Joel
Exception thrown in the subscribe.
Don’t want to have to try/catch all of our observable operations.
If onError is called, then use the Observable returned by the Action passed to onErrorResumeNext.
If the source observable sends onError then resubscribe in the hopes that it will complete without error.
For instance, if it’s a webservice call that times-out give it a second chance (if it has a low timeout!)
This example is the verbose one. Without the throwable then it would track up to retryCount automatically.