Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013)
Upcoming SlideShare
Loading in...5
×
 

Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013)

on

  • 1,879 views

A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store ...

A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone. In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available in Java, Scala and JavaScript. You will learn how to use reactive observables, which are asynchronous data streams, to access web services from both Java and JavaScript. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.

Statistics

Views

Total Views
1,879
Views on SlideShare
1,879
Embed Views
0

Actions

Likes
6
Downloads
55
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013) Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013) Presentation Transcript

  • @crichardson Consuming web services asynchronously with Futures and Rx Observables Chris Richardson Author of POJOs in Action Founder of the original CloudFoundry.com @crichardson chris@chrisrichardson.net http://plainoldobjects.com
  • @crichardson Presentation goal Learn how to use (Scala) Futures and Rx Observables to write simple yet robust and scalable concurrent code
  • @crichardson About Chris
  • @crichardson (About Chris)
  • @crichardson About Chris()
  • @crichardson About Chris
  • @crichardson About Chris http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
  • @crichardson About Chris
  • @crichardson Agenda The need for concurrency Simplifying concurrent code with Futures Taming callback hell with JavaScript promises Consuming asynchronous streams with Reactive Extensions
  • @crichardson Let’s imagine you are building an online store
  • @crichardson Shipping Recomendations Product Info Reviews
  • @crichardson Reviews Product Info Sales ranking
  • @crichardson Related books Viewing history Forum
  • @crichardson + mobile apps
  • @crichardson Application architecture Product Info Service Desktop browser Native mobile client REST REST REST Recomendation Service Review Service Front end server API gatewayREST Mobile browser Web Application HTML/ JSON JSON
  • @crichardson Handling getProductDetails() Front-end server Product Info Service Recommendations Service Review Service getProductInfo() getRecommendations() getReviews() getProductDetails()
  • @crichardson Handling getProductDetails() - sequentially Front-end server Product Info Service Recommendations Service Review Service getProductInfo() getRecommendations() getReviews() getProductDetails() Higher response time :-(
  • @crichardson Handling getProductDetails() - in parallel Front-end server Product Info Service Recommendations Service Review Service getProductInfo() getRecommendations() getReviews() getProductDetails() Lower response time :-)
  • @crichardson Implementing a concurrent REST client Thread-pool based approach executorService.submit(new Callable(...)) Simpler but less scalable - lots of idle threads consuming memory Event-driven approach NIO with completion callbacks More complex but more scalable
  • @crichardson The front-end server must handle partial failure of backend services Front-end server Product Info Service Recommendations Service Review Service getProductInfo() getRecommendations() getReviews() getProductDetails() X How to provide a good user experience?
  • @crichardson Agenda The need for concurrency Simplifying concurrent code with Futures Taming callback hell with JavaScript promises Consuming asynchronous streams with Reactive Extensions
  • @crichardson Futures are a great concurrency abstraction http://en.wikipedia.org/wiki/Futures_and_promises
  • @crichardson Worker threadMain thread How futures work Outcome Future Client get Asynchronous operation set initiates
  • @crichardson Benefits Client does not know how the asynchronous operation is implemented Client can invoke multiple asynchronous operations and gets a Future for each one.
  • @crichardson Handling ProductDetails request ProductDetailsController ProductDetailsService getProductDetails() ProductInfoService getProductInfo() ReviewService getReviews() RecommendationService getRecommendations() RestTemplate
  • @crichardson REST client using Spring @Async @Component class ProductInfoServiceImpl extends ProducInfoService { val restTemplate : RestTemplate = ... @Async def getProductInfo(productId: Long) = { new AsyncResult(restTemplate.getForObject(....)...) } } Execute asynchronously in thread pool A fulfilled Future trait ProductInfoService { def getProductInfo(productId: Long): java.util.concurrent.Future[ProductInfo] }
  • @crichardson ProductDetailsService @Component class ProductDetailsService @Autowired()(productInfoService: ProductInfoService, reviewService: ReviewService, recommendationService: RecommendationService) { def getProductDetails(productId: Long): ProductDetails = { val productInfoFuture = productInfoService.getProductInfo(productId) } } val recommendationsFuture = recommendationService.getRecommendations(pr val reviewsFuture = reviewService.getReviews(productId) val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS) val recommendations = recommendationsFuture.get(10, TimeUnit.MILLISECONDS) val reviews = reviewsFuture.get(10, TimeUnit.MILLISECONDS) ProductDetails(productInfo, recommendations, reviews)
  • @crichardson ProductController @Controller class ProductController @Autowired()(productDetailsService : ProductDetailsService) { @RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) = productDetailsService.getProductDetails(productId)
  • @crichardson Blocks Tomcat thread until Future completes Not bad but... class ProductDetailsService def getProductDetails(productId: Long): ProductDetails = { val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS) Not so scalable :-(
  • @crichardson ... and also... Java Futures work well for a single-level of asynchronous execution BUT #fail for more complex, scalable scenarios Difficult to compose and coordinate multiple concurrent operations See this blog post for more details: http://techblog.netflix.com/2013/02/rxjava-netflix-api.html
  • @crichardson Better: Futures with callbacks no blocking! val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") } Guava ListenableFutures, Spring 4 ListenableFuture Java 8 CompletableFuture, Scala Futures
  • @crichardson Even better: composable futures def asyncOperation() = Future { ... ; 1 } val r = asyncOperation().map{ _ * 2 } assertEquals(4, Await.result(r, 1 second)) Scala, Java 8 CompletableFuture (partially) Transforms Future
  • @crichardson map() is asynchronous - uses callbacks class Future[T] { def map[S](f: T => S): Future[S] = { val p = Promise[S]() onSuccess { case r => p success f(r) } onFailure { case t => p failure t } p.future } When ‘this’ completes propagate outcome to ‘p’ Return new Future
  • @crichardson Chaining using flatmap() def asyncOperation() = Future { ... ; 3 } def asyncSquare(x : Int) : Future[Int] = Future { x * x} val r = asyncOperation().flatMap { x => asyncSquare(x) } assertEquals(9, Await.result(r, 1 second)) Scala, Java 8 CompletableFuture (partially) Chaining asynchronous operations
  • @crichardson Combining futures val f1 = Future { ... ; 1} val f2 = Future { .... ; 2} val fzip = f1 zip f2 assertEquals((1, 2), Await.result(fzip, 1 second)) def asyncOp(x : Int) = Future { x * x} val f = Future.sequence((1 to 5).map { x => asyncOp(x) }) assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second)) Scala, Java 8 CompletableFuture (partially) Combines two futures Transforms list of futures to a future containing a list
  • @crichardson Scala futures are Monads def callB() : Future[...] = ... def callC() : Future[...] = ... def callD() : Future[...] = ... val result = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d result onSuccess { .... } Two calls execute in parallel And then invokes D Get the result of D
  • @crichardson ‘for’ is shorthand for map() and flatMap() (callB() zip callC()) flatMap { r1 => val (b, c) = r1 callD(b, c) map { d => d } } for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d
  • @crichardson Scala Future + RestTemplate import scala.concurrent.Future @Component class ProductInfoService { def getProductInfo(productId: Long): Future[ProductInfo] = { Future { restTemplate.getForObject(....) } } } Executes in thread pool Scala Future
  • @crichardson Scala Future + RestTemplate class ProductDetailsService @Autowired()(....) { def getProductDetails(productId: Long) = { val productInfoFuture = productInfoService.getProductInfo(productId) val recommendationsFuture = recommendationService.getRecommendations(productId) val reviewsFuture = reviewService.getReviews(productId) for (((productInfo, recommendations), reviews) <- productInfoFuture zip recommendationsFuture zip reviewsFuture) yield ProductDetails(productInfo, recommendations, reviews) } } Non-blocking!
  • @crichardson Async Spring MVC + Scala Futures @Controller class ProductController ... { @RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) = { val productDetails = productDetailsService.getProductDetails(productId) val result = new DeferredResult[ProductDetails] productDetails onSuccess { case r => result.setResult(r) } productDetails onFailure { case t => result.setErrorResult(t) } result } Convert Scala Future to Spring MVC DeferredResult
  • @crichardson Servlet layer is asynchronous but the backend uses thread pools Need event-driven REST client
  • @crichardson New in Spring 4 Mirrors RestTemplate Methods return a ListenableFuture ListenableFuture = JDK 7 Future + callback methods Can use HttpComponents NIO-based AsyncHttpClient Spring AsyncRestTemplate
  • @crichardson Using the AsyncRestTemplate http://hc.apache.org/httpcomponents-asyncclient-dev/ val asyncRestTemplate = new AsyncRestTemplate( new HttpComponentsAsyncClientHttpRequestFactory()) override def getProductInfo(productId: Long) = { val listenableFuture = asyncRestTemplate.getForEntity("{baseUrl}/productinfo/{productId}", classOf[ProductInfo], baseUrl, productId) toScalaFuture(listenableFuture).map { _.getBody } } Convert to Scala Future and get entity
  • @crichardson Converting ListenableFuture to Scala Future implicit def toScalaFuture[T](f : ListenableFuture[T]) : Future[T] = { val p = promise[T]() f.addCallback(new ListenableFutureCallback[T] { def onSuccess(result: T) { p.success(result)} def onFailure(t: Throwable) { p.failure(t) } }) p.future } The producer side of Scala Futures Supply outcome Return future
  • @crichardson Now everything is non- blocking :-)
  • @crichardson If recommendation service is down... Never responds front-end server waits indefinitely Consumes valuable front-end server resources Page never displayed and customer gives up Returns an error Error returned to the front-end server error page is displayed Customer gives up
  • @crichardson Fault tolerance at Netflix Network timeouts and retries Invoke remote services via a bounded thread pool Use the Circuit Breaker pattern On failure: return default/cached data return error to caller Implementation: https://github.com/Netflix/Hystrix http://techblog.netflix.com/2012/02/fault-tolerance-in-high- volume.html
  • @crichardson How to handling partial failures? productDetailsFuture zip recommendationsFuture zip reviewsFuture Fails if any Future has failed
  • @crichardson Handling partial failures recommendationService. getRecommendations(userId,productId) .recover { case _ => Recommendations(List()) } “catch-like” Maps Throwable to value
  • @crichardson Agenda The need for concurrency Simplifying concurrent code with Futures Taming callback hell with JavaScript promises Consuming asynchronous streams with Reactive Extensions
  • @crichardson Why solve this problem for JavaScript? Browser invokes web services Implement front-end server/API gateway using NodeJS
  • @crichardson What’s NodeJS? Designed for DIRTy apps
  • @crichardson NodeJS JavaScript Reactor pattern Modules
  • @crichardson Asynchronous JavaScript code = callback hell Scenarios: Sequential: A B C Fork and join: A and B C Code quickly becomes very messy
  • @crichardson Callback-based HTTP client request = require("request") handler = (error, clientResponse, body) -> if clientResponse.statusCode != 200 // ERROR else // SUCCESS request("http://.../productdetails/" + productId, handler)
  • @crichardson Messy callback code getProductDetails = (req, res) -> productId = req.params.productId result = {productId: productId} makeHandler = (key) -> (error, clientResponse, body) -> if clientResponse.statusCode != 200 res.status(clientResponse.statusCode) res.write(body) res.end() else result[key] = JSON.parse(body) if (result.productInfo and result.recommendations and result.reviews) res.json(result) request("http://localhost:3000/productdetails/" + productId, makeHandler("productInfo")) request("http://localhost:3000/recommendations/" + productId, makeHandler('recommendations')) request("http://localhost:3000/reviews/" + productId, makeHandler('reviews')) app.get('/productinfo/:productId', getProductInfo) Returns a callback function Have all three requests completed?
  • @crichardson Simplifying code with Promises (a.k.a. Futures) Functions return a promise - no callback parameter A promise represents an eventual outcome Use a library of functions for transforming and composing promises Promises/A+ specification - http://promises-aplus.github.io/promises-spec
  • @crichardson Basic promise API promise2 = promise1.then(fulfilledHandler, errorHandler, ...) called when promise1 is fulfilled returns a promise or value resolved with outcome of callback called when promise1 is rejected
  • About when.js Rich API = Promises spec + use full extensions Creation: when.defer() when.reject()... Combinators: all, some, join, map, reduce Other: Calling non-promise code timeout .... https://github.com/cujojs/when
  • @crichardson Simpler promise-based code rest = require("rest") @httpClient = rest.chain(mime).chain(errorCode); getProductInfo : (productId) -> @httpClient path: "http://.../productinfo/" + productId Returns a promise No ugly callbacks
  • @crichardson Simpler promise-based code class ProductDetailsService getProductDetails: (productId) -> makeProductDetails = (productInfo, recommendations, reviews) -> details = productId: productId productDetails: productInfo.entity recommendations: recommendations.entity reviews: reviews.entity details responses = [@getProductInfo(productId), @getRecommendations(productId), @getReviews(productId)] all(responses).spread(makeProductDetails) Array[Promise] Promise[Array]
  • @crichardson Simpler promise-based code: HTTP handler getProductDetails = (req, res) -> productId = req.params.productId succeed = (productDetails) -> res.json(productDetails) fail = (something) -> res.send(500, JSON.stringify(something.entity || something)) productDetailsService. getDetails(productId). then(succeed, fail) app.get('/productdetails/:productId', getProductDetails)
  • @crichardson Writing robust client code
  • @crichardson Recovering from failures getRecommendations(productId) .otherwise( -> {}) Invoked to return default value if getRecommendations() fails
  • @crichardson Agenda The need for concurrency Simplifying concurrent code with Futures Taming callback hell with JavaScript promises Consuming asynchronous streams with Reactive Extensions
  • @crichardson Let’s imagine you have a stream of trades and you need to calculate the rolling average price of each stock
  • @crichardson Spring Integration + Complex event processing (CEP) engine is a good choice
  • @crichardson But where is the high-level abstraction that solves this problem?
  • @crichardson Future[List[T]] Not applicable to infinite streams
  • @crichardson Introducing Reactive Extensions (Rx) The Reactive Extensions (Rx) is a library for composing asynchronous and event- based programs using observable sequences and LINQ-style query operators. Using Rx, developers represent asynchronous data streams with Observables , query asynchronous data streams using LINQ operators , and ..... https://rx.codeplex.com/
  • @crichardson About RxJava Reactive Extensions (Rx) for the JVM Implemented in Java Adaptors for Scala, Groovy and Clojure https://github.com/Netflix/RxJava
  • @crichardson RxJava core concepts class Observable<T> { Subscription subscribe(Observer<T> observer) ... } interface Observer<T> { void onNext(T args) void onCompleted() void onError(Throwable e) } Notifies An asynchronous stream of items Used to unsubscribe
  • @crichardson Comparing Observable to... Observer pattern - similar but adds Observer.onComplete() Observer.onError() Iterator pattern - mirror image Push rather than pull Future - similar but Represents a stream of multiple values
  • @crichardson So what?
  • @crichardson Transforming Observables class Observable<T> { Observable<T> take(int n); Observable<T> skip(int n); <R> Observable<R> map(Func1<T,R> func) <R> Observable<R> flatMap(Func1<T,Observable<R>> func) Observable<T> filter(Func1<T,java.lang.Boolean> predicate) ... } Scala-collection style methods
  • @crichardson Transforming observables class Observable<T> { <K> Observable<GroupedObservable<K,T>> groupBy(Func1<T,K> keySelector) ... } Similar to Scala groupBy except results are emitted as items arrive Stream of streams!
  • @crichardson Combining observables class Observable<T> { static <R,T1,T2> Observable<R> zip(Observable<T0> o1, Observable<T1> o2, Func2<T1,T2,R> reduceFunction) ... } Invoked with pairs of items from o1 and o2 Stream of results from reduceFunction
  • @crichardson Creating observables from data class Observable<T> { static Observable<T> from(Iterable<T> iterable]); static Observable<T> from(T items ...]); static Observable<T> from(Future<T> future]); ... }
  • @crichardson Arranges to call obs.onNext/ onComplete/... Creating observables from event sources Observable<T> o = Observable.create( new SubscriberFunc<T> () { Subscription onSubscribe(Observer<T> obs) { ... connect to event source.... return new Subscriber () { void unsubscribe() { ... }; }; }); Called once for each Observer Called when unsubscribing
  • @crichardson Using Rx Observables instead of Futures
  • @crichardson Rx-based ProductInfoService @Component class ProductInfoService override def getProductInfo(productId: Long) = { val baseUrl = ... val responseEntity = asyncRestTemplate.getForEntity(baseUrl + "/productinfo/{productId}", classOf[ProductInfo], productId) toRxObservable(responseEntity).map { (r : ResponseEntity[ProductInfo]) => r.getBody } }
  • @crichardson ListenableFuture Observable implicit def toRxObservable[T](future: ListenableFuture[T]) : Observable[T] = { def create(o: Observer[T]) = { future.addCallback(new ListenableFutureCallback[T] { def onSuccess(result: T) { o.onNext(result) o.onCompleted() } def onFailure(t: Throwable) { o.onError(t) } }) new Subscription { def unsubscribe() {} } } Observable.create(create _) } Supply single value Indicate failure Do nothing
  • @crichardson Rx - ProductDetailsService @Component class ProductDetailsService @Autowired() (productInfoService: ProductInfoService, reviewService: ReviewService, ecommendationService: RecommendationService) { def getProductDetails(productId: Long) = { val productInfo = productInfoService.getProductInfo(productId) val recommendations = recommendationService.getRecommendations(productId) val reviews = reviewService.getReviews(productId) Observable.zip(productInfo, recommendations, reviews, (p : ProductInfo, r : Recommendations, rv : Reviews) => ProductDetails(p, r, rv)) } } Just like Scala Futures
  • @crichardson Rx - ProductController @Controller class ProductController @RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) = val pd = productDetailsService.getProductDetails(productId) toDeferredResult(pd)
  • @crichardson Rx - Observable DeferredResult implicit def toDeferredResult[T](observable : Observable[T]) = { val result = new DeferredResult[T] observable.subscribe(new Observer[T] { def onCompleted() {} def onError(e: Throwable) { result.setErrorResult(e) } def onNext(r: T) { result.setResult(r.asInstanceOf[T]) } }) result }
  • @crichardson RxObservables vs. Futures Much better than JDK 7 futures Comparable to Scala Futures Rx Scala code is slightly more verbose
  • @crichardson Making this code robust Hystrix works with Observables (and thread pools) But For event-driven code we are on our own
  • @crichardson Back to the stream of Trades averaging example...
  • @crichardson Calculating averages Observable<AveragePrice> calculateAverages(Observable<Trade> trades) { ... } class Trade { private String symbol; private double price; class AveragePrice { private String symbol; private double averagePrice;
  • @crichardson Using groupBy() APPL : 401 IBM : 405 CAT : 405 APPL: 403 groupBy( (trade) => trade.symbol) APPL : 401 IBM : 405 CAT : 405 ... APPL: 403 Observable<GroupedObservable<String, Trade>> Observable<Trade>
  • @crichardson Using window() APPL : 401 APPL : 405 APPL : 405 ... APPL : 401 APPL : 405 APPL : 405 APPL : 405 APPL : 405 APPL : 403 APPL : 405 ... window(...) Observable<Trade> Observable<Observable<Trade>> N secs N secs M secs
  • @crichardson Using fold() APPL : 402 APPL : 405 APPL : 405 APPL : 404 fold(0)(sumCalc) / length Observable<Trade> Observable<AveragePrice> Singleton
  • @crichardson Using merge() merge() APPL : 401 IBM : 405 CAT : 405 ... APPL: 403 APPL : 401 IBM : 405 CAT : 405 APPL: 403 Observable<Observable<AveragePrice>> Observable<AveragePrice>
  • @crichardson Calculating average prices def calculateAverages(trades: Observable[Trade]): Observable[AveragePrice] = { trades.groupBy(_.symbol).map { symbolAndTrades => val (symbol, tradesForSymbol) = symbolAndTrades ... tradesForSymbol.window(...).map { windowOfTradesForSymbol => windowOfTradesForSymbol.fold((0.0, 0, List[Double]())) { (soFar, trade) => val (sum, count, prices) = soFar (sum + trade.price, count + 1, trade.price +: prices) } map { x => val (sum, length, prices) = x AveragePrice(symbol, sum / length, prices) } }.flatMap(identity) }.flatMap(identity) }
  • @crichardson RxJS / NodeJS example var rx = require("rx"); function tailFile (fileName) { var self = this; var o = rx.Observable.create(function (observer) { var watcher = self.tail(fileName, function (line) { observer.onNext(line); }); return function () { watcher.close(); } }); return o.map(parseLogLine); } function parseLogLine(line) { ... }
  • @crichardson Rx in the browser - implementing completion TextChanges(input) .DistinctUntilChanged() .Throttle(TimeSpan.FromMilliSeconds(10)) .Select(word=>Completions(word)) .Switch() .Subscribe(ObserveChanges(output)); Your Mouse is a Database http://queue.acm.org/detail.cfm?id=2169076 Ajax call returning Observable Change events Observable Display completions
  • @crichardson Summary Consuming web services asynchronously is essential Scala-style are a powerful concurrency abstraction Rx Observables are even more powerful Rx Observables are a unifying abstraction for a wide variety of use cases
  • @crichardson Questions? @crichardson chris@chrisrichardson.net http://plainoldobjects.com