A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone. In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available on the JVM. You will learn how to access web services using reactive observables (RxJava), which are asynchronous data streams. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.

1. Consuming web services
asynchronously with Futures
and Rx Observables
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com
@crichardson

2. Presentation goal
Learn how to use (Scala)
Futures and Rx
Observables to write simple
yet robust and scalable
concurrent code
@crichardson

3. About Chris
@crichardson

4. About Chris
Founder of a buzzword compliant (stealthy, social, mobile,
big data, machine learning, ...) startup
Consultant helping organizations improve how they
architect and deploy applications using cloud, micro
services, polyglot applications, NoSQL, ...
@crichardson

5. Agenda
The need for concurrency
Simplifying concurrent code with Futures
Consuming asynchronous streams with Reactive
Extensions
@crichardson

6. Let’s imagine you are building
an online store
@crichardson

7. Reviews
Product
Info
Shipping
Recomendations
@crichardson

8. Product
Info
Sales
ranking
Reviews
@crichardson

9. Related
books
Forum
Viewing
history
@crichardson

10. + mobile apps
@crichardson

11. Application architecture
Desktop
browser
REST
REST
HTML/
JSON
Product Info Service
Recomendation
Service
REST
Review Service
Front end server
Web Application
Mobile
browser
Native
mobile
client
REST
API gateway
JSON
@crichardson

12. How does the client get product
details?
Product Info Service
getProductInfo()
Browser/
Client
?
Front-end
server
getRecommendations()
Recommendations
Service
getReviews()
Review
Service
@crichardson

13. Product details - client-side
aggregation
Product Info Service
getProductInfo()
getProductInfo()
Browser/
Client
getRecommendations()
getReviews()
Front-end
server
getRecommendations()
Recommendations
Service
getReviews()
Requires
good network
performance
Review
Service
@crichardson

14. Product details - server-side
aggregation
Product Info Service
HTML or JSON
getProductInfo()
Browser/
Client
getProductDetails()
Front-end
server
getRecommendations()
Recommendations
Service
getReviews()
One roundtrip
Review
Service
@crichardson

15. Product details - server-side
aggregation: sequentially
Product Info Service
getProductInfo()
getProductDetails()
Front-end
server
getRecommendations()
Recommendations
Service
getReviews()
Higher response time :-(
Review
Service
@crichardson

16. Product details - server-side
aggregation: parallel
Product Info Service
getProductInfo()
getProductDetails()
Front-end
server
getRecommendations()
Recommendations
Service
getReviews()
Lower response
time :-)
Review
Service
@crichardson

17. Implementing a concurrent
REST client
Thread-pool based approach
executorService.submit(new Callable(...))
Simpler but less scalable - lots of idle threads
consuming memory
Event-driven approach
NIO with completion callbacks
More complex but more scalable
And it must handle partial failures
@crichardson

18. Agenda
The need for concurrency
Simplifying concurrent code with Futures
Consuming asynchronous streams with Reactive
Extensions
@crichardson

19. Futures are a great
concurrency abstraction
http://en.wikipedia.org/wiki/Futures_and_promises
@crichardson

20. How futures work
Main thread
Worker thread or
event-driven
initiates
Client
get
set
Asynchronous
operation
Outcome
Future
@crichardson

21. Benefits
Simple way for two concurrent activities to communicate safely
Abstraction:
Client does not know how the asynchronous operation is
implemented
Easy to implement scatter/gather:
Scatter: Client can invoke multiple asynchronous operations
and gets a Future for each one.
Gather: Get values from the futures
@crichardson

22. Front-end server design: handling
GetProductDetails request
ProductDetailsController
getProductDetails()
ProductDetailsService
Proxies
getProductDetails()
ProductInfoService
getProductInfo()
ReviewService
getReviews()
RecommendationService
getRecommendations()
RestTemplate
@crichardson

23. REST client using Spring @Async
trait ProductInfoService {
def getProductInfo(productId: Long):
java.util.concurrent.Future[ProductInfo]
}
@Component
class ProductInfoServiceImpl extends ProducInfoService {
val restTemplate : RestTemplate = ...
Execute
asynchronously in
thread pool
@Async
def getProductInfo(productId: Long) = {
new AsyncResult(restTemplate.getForObject(....)...)
}
}
A fulfilled Future
@crichardson

24. ProductDetailsService
@Component
class ProductDetailsService
@Autowired()(productInfoService: ProductInfoService,
reviewService: ReviewService,
recommendationService: RecommendationService) {
def getProductDetails(productId: Long): ProductDetails = {
val productInfoFuture = productInfoService.getProductInfo(productId)
val recommendationsFuture =
recommendationService.getRecommendations(productId)
val reviewsFuture = reviewService.getReviews(productId)
val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS)
val recommendations =
recommendationsFuture.get(10, TimeUnit.MILLISECONDS)
val reviews = reviewsFuture.get(10, TimeUnit.MILLISECONDS)
ProductDetails(productInfo, recommendations, reviews)
}
}
@crichardson

25. ProductController
@Controller
class ProductController
@Autowired()(productDetailsService : ProductDetailsService)
{
@RequestMapping(Array("/productdetails/{productId}"))
@ResponseBody
def productDetails(@PathVariable productId: Long) =
productDetailsService.getProductDetails(productId)
@crichardson

26. Not bad but...
class ProductDetailsService
def getProductDetails(productId: Long): ProductDetails = {
val productInfo =
productInfoFuture.get(300, TimeUnit.MILLISECONDS)
Not so scalable :-(
Gathering blocks Tomcat
thread until all Futures
complete
@crichardson

27. ... and also...
Java Futures work well for a single-level of asynchronous
execution
BUT
#fail for more complex, scalable scenarios
Difficult to compose and coordinate multiple concurrent
operations
See this blog post for more details:
http://techblog.netflix.com/2013/02/rxjava-netflix-api.html
@crichardson

28. Better: Futures with callbacks
no blocking!
def asyncSquare(x : Int)
: Future[Int] = ... x * x...
val f = asyncSquare(25)
Partial function applied to
successful outcome
f onSuccess {
case x : Int => println(x)
Applied to failed outcome
}
f onFailure {
case e : Exception => println("exception thrown")
}
Guava ListenableFutures, Spring 4 ListenableFuture
Java 8 CompletableFuture, Scala Futures
@crichardson

29. But
callback-based scatter/gather
Messy, tangled code
(aka. callback hell)
@crichardson

30. Composable futures hide the mess
Combines two futures
val fzip = asyncSquare(5) zip asyncSquare(7)
assertEquals((25, 49), Await.result(fzip, 1 second))
val fseq = Future.sequence((1 to 5).map { x =>
asyncSquare(x)
})
Transforms list of futures to a future
assertEquals(List(1, 4, 9, 16, 25),
Await.result(fseq, 1 second))
Scala, Java 8 CompletableFuture (partially)
@crichardson

31. zip() is asynchronous
Outcome1
f1
(o1, o2)
f3 = f1 zip f2
f3
Implemented using callbacks
Outcome2
f2
@crichardson

32. Transforming futures
def asyncPlus(x : Int, y : Int) = ... x + y ...
val future2 = asyncPlus(4, 5).map{ _ * 3 }
assertEquals(27, Await.result(future2, 1 second))
Asynchronously transforms
future
Scala, Java 8 CompletableFuture (partially)
@crichardson

33. Chaining asynchronous operations
Calls asyncSquare() with the eventual
outcome of asyncPlus()
val f2 = asyncPlus(5, 8).flatMap { x => asyncSquare(x) }
assertEquals(169, Await.result(f2, 1 second))
Scala, Java 8 CompletableFuture (partially)
@crichardson

34. Scala futures are Monads
Two calls execute in parallel
(asyncPlus(3, 5) zip asyncSquare(5))
.flatMap {
And then invokes asyncPlus()
case (a, b) =>
asyncPlus(a, b) map { _ * 2 }
}
result onSuccess { .... }
x2
Rewrite using ‘for’
@crichardson

35. Scala futures are Monads
Two calls execute in parallel
val result = for {
(a, b) <- asyncPlus(3, 5) zip asyncSquare(5);
c <- asyncPlus(a, b)
} yield c * 2
result onSuccess { .... }
‘for’ is shorthand for
map() and flatMap()
And then invokes
asyncPlus()
x2
@crichardson

36. ProductInfoService: using
Scala Futures
import scala.concurrent.Future
Scala Future
@Component
class ProductInfoService {
def getProductInfo(productId: Long): Future[ProductInfo]
= {
Future { restTemplate.getForObject(....) }
}
}
Executed in a threaded pool
@crichardson

37. ProductDetailsService: using
Scala Futures
class ProductDetailsService ...
Return a Scala Future
def getProductDetails(productId: Long) : Future[ProductDetails] = {
val productInfoFuture = productInfoService.getProductInfo(productId)
val recommendationsFuture =
recommendationService.getRecommendations(productId)
val reviewsFuture = reviewService.getReviews(productId)
for (((productInfo, recommendations), reviews) <productInfoFuture zip recommendationsFuture zip reviewsFuture)
yield ProductDetails(productInfo, recommendations, reviews)
}
}
Gathers data without blocking
@crichardson

38. Async ProductController: using
Spring MVC DeferredResult
@Controller
class ProductController ... {
@RequestMapping(Array("/productdetails/{productId}"))
@ResponseBody
def productDetails(@PathVariable productId: Long)
: DeferredResult[ProductDetails] = {
val productDetails =
productDetailsService.getProductDetails(productId)
val result = new DeferredResult[ProductDetails]
productDetails onSuccess {
case r => result.setResult(r)
}
productDetails onFailure {
case t => result.setErrorResult(t)
}
Spring MVC
DeferredResult
≅
Future
Convert Scala Future
to
DeferredResult
result
}
@crichardson

39. Servlet layer is asynchronous
BUT
the backend uses thread
pools
Need event-driven REST
client
@crichardson

40. Spring AsyncRestTemplate
New in Spring 4
Mirrors RestTemplate
Can use HttpComponents NIO-based AsyncHttpClient
Methods return a ListenableFuture
JDK 7 Future + callback methods
Yet another
“Future”!
@crichardson

41. ProductInfoService: using the
AsyncRestTemplate
class ProductInfoService {
val asyncRestTemplate = new AsyncRestTemplate(
new HttpComponentsAsyncClientHttpRequestFactory())
override def getProductInfo(productId: Long) = {
val listenableFuture =
asyncRestTemplate.getForEntity("{baseUrl}/productinfo/{productId}",
classOf[ProductInfo],
baseUrl, productId)
toScalaFuture(listenableFuture).map { _.getBody
}
}
Convert to Scala Future and get entity
http://hc.apache.org/httpcomponents-asyncclient-dev/
@crichardson

42. Converting ListenableFuture to
Scala Future
def toScalaFuture[T](lf : ListenableFuture[T]) :
Future[T] = {
val p = promise[T]()
Creates a promise = producer API
lf.addCallback(new ListenableFutureCallback[T] {
def onSuccess(result: T) { p.success(result)}
def onFailure(t: Throwable) { p.failure(t) }
})
p.future
}
Propagate outcome to promise
Return future
@crichardson

43. Now everything is nonblocking :-)
We have achieved scaling Nirvana
@crichardson

44. WT*#*# is my code doing?
Operations initiated in one thread but fail in another
Lack of a full stack trace can make debugging difficult
Inherent problem of async/event driven programming
Futures make it very easy to forget to handle errors
someFuture.foreach { handleTheHappyPath }
Error is quietly ignored: similar to an empty catch {}
block
@crichardson

45. Agenda
The need for concurrency
Simplifying concurrent code with Futures
Consuming asynchronous streams with Reactive
Extensions
@crichardson

46. Let’s imagine you have a
stream of trades
and
you need to calculate the 15
minute rolling average price of
each stock
@crichardson

47. Where is the high-level
abstraction that simplifies
solving this problem?
@crichardson

48. Future[List[T]]
Not applicable to infinite
streams
@crichardson

49. Pipes and Filters
e.g. Spring Integration
+
Complex event processing (CEP)
Not bad but tends to be an external DSL,
heavy weight, statically defined, ...
@crichardson

50. Introducing Reactive
Extensions (Rx)
The Reactive Extensions (Rx) is a library for composing
asynchronous and event-based programs using
observable sequences and LINQ-style query operators.
Using Rx, developers represent asynchronous data
streams with Observables , query asynchronous
data streams using LINQ operators , and .....
https://rx.codeplex.com/
@crichardson

51. About RxJava
Reactive Extensions (Rx) for the JVM
Original motivation for Netflix was to provide rich Futures
Implemented in Java
Adaptors for Scala, Groovy and Clojure
https://github.com/Netflix/RxJava
@crichardson

52. RxJava core concepts
An asynchronous
stream of items
trait Observable[T] {
def subscribe(observer : Observer[T]) : Subscription
...
}
Notifies
trait
def
def
def
}
Observer[T] {
onNext(value : T)
onCompleted()
onError(e : Throwable)
Used to
unsubscribe
* Note: with v0.17 subscribe() takes a Subscriber = Observer++
@crichardson

53. Comparing Observable to...
Observer pattern - similar but adds
Observer.onComplete()
Observer.onError()
Iterator pattern - mirror image
Push rather than pull
Future - similar but
Represents a stream of multiple values
@crichardson

54. So what?
@crichardson

55. Fun with observables
val oneItem = Observable.items(-1L)
val every10Seconds = Observable.interval(10 seconds)
val ticker = oneItem ++ every10Seconds
val subscription = ticker.subscribe ( new Observer[Long] {
override def onNext(value: Long) = { println("value=" + value)
})
...
subscription.unsubscribe()
-1
0
1
...
t=0
t=10
t=20
...
}
@crichardson

56. Creating observables
Observable.create({ observer: Observer[T] =>
...
Function
observer.onNext(...)
called when
...
Observer
observer.onCompleted()
subscribes
...
observer.onError(...)
Called when observer
...
unsubscribes
Subscription{ .... }
}
})
* Note: with v0.17 create() takes a Subscriber = Observer++
@crichardson

57. Creating observables from
Query AWS
Scala Futures
def getTableStatus(tableName: String) = {
val future = dynamoDbClient.describeTable(new DescribeTableRequest(tableName))
Observable.create({ observer : Observer[DynamoDbStatus] =>
future.onComplete {
case Success(response) =>
observer.onNext(DynamoDbStatus(response.getTable.getTableStatus))
observer.onCompleted()
case Failure(t: ResourceNotFoundException) =>
observer.onNext(DynamoDbStatus("NOT_FOUND"))
observer.onCompleted()
case Failure(someError) =>
observer.onError(someError)
}
Subscription({})
})
}
Propagate@crichardson
outcome

58. Transforming observables
val tableStatus = ticker.flatMap { i =>
logger.info("{}th describe table", i + 1)
getTableStatus(name)
}
Status1 Status2
t=0
t=10
Status3
...
t=20
...
+ Usual collection methods: map(), filter(), take(), drop(), ...
@crichardson

59. Back to the stream of Trades
averaging example...
@crichardson

60. Calculating averages
class AverageTradePriceCalculator {
def calculateAverages(trades: Observable[Trade]):
Observable[AveragePrice] = {
...
}
case class Trade(
symbol : String,
price : Double,
quantity : Int
...
)
case class AveragePrice(
symbol : String,
price : Double,
...)
@crichardson

61. Using groupBy()
Observable[Trade]
APPL : 401
100
IBM : 405
300
CAT : 405
150
APPL: 403
300
groupBy( (trade) => trade.symbol)
APPL : 401
APPL: 403
IBM : 405
CAT : 405
Observable[GroupedObservable[String, Trade]]
...
@crichardson

62. Using window()
Observable[Trade]
APPL : 401
APPL : 405
APPL : 405
...
window(...)
5 minutes
APPL : 401
APPL : 405
APPL : 405
APPL : 405
APPL : 403
30 secs
30 secs
APPL : 405
APPL : 405
...
Observable[Observable[Trade]]
@crichardson

63. Using foldLeft()
Observable[Trade]
APPL : 402
100
APPL : 405
200
APPL : 405
200
foldLeft(0.0)(_ + _.price)
/
foldLeft(0.0)(_ + _.quantity)
APPL : 404.4
Observable[AveragePrice]
Singleton
@crichardson

64. Using flatten()
Observable[Observable[AveragePrice]]
APPL : 401
APPL: 403
IBM : 405
CAT : 405
...
flatten()
APPL : 401
IBM : 405
Observable[AveragePrice]
CAT : 405
APPL: 403
@crichardson

65. Calculating average prices
def calculateAverages(trades: Observable[Trade]): Observable[AveragePrice] = {
trades.groupBy(_.symbol).map { symbolAndTrades =>
val (symbol, tradesForSymbol) = symbolAndTrades
val openingEverySecond =
Observable.items(-1L) ++ Observable.interval(1 seconds)
def closingAfterSixSeconds(opening: Any) =
Observable.interval(6 seconds).take(1)
tradesForSymbol.window(...).map {
windowOfTradesForSymbol =>
windowOfTradesForSymbol.fold((0.0, 0, List[Double]())) { (soFar, trade) =>
val (sum, count, prices) = soFar
(sum + trade.price, count + trade.quantity, trade.price +: prices)
} map { x =>
val (sum, length, prices) = x
AveragePrice(symbol, sum / length, prices)
}
}.flatten
}.flatten
}
@crichardson

66. Summary
Consuming web services asynchronously is essential
Scala-style Futures are a powerful concurrency abstraction
Rx Observables
even more powerful
unifying abstraction for a wide variety of use cases
@crichardson

67. @crichardson chris@chrisrichardson.net
Questions?
http://plainoldobjects.com
@crichardson