Functional Reactive Programmingin the Netflix APIBen ChristensenSoftware Engineer – API Platform at Netflix@benjchristensenh...
InfoQ.com: News & Community Site• 750,000 unique visitors/month• Published in 4 languages (English, Chinese, Japanese and ...
Presented at QCon Londonwww.qconlondon.comPurpose of QCon- to empower software development by facilitating the spread ofkn...
onsdag den 6. marts 13
More than 33 million Subscribersin more than 50 Countries and Territoriesonsdag den 6. marts 13
Netflix accounts for 33% of Peak DownstreamInternet Traffic in North AmericaNetflix subscribers are watchingmore than 1 billi...
API traffic has grown from~20 million/day in 2010 to >2 billion/day05001000150020002010 2011 2012 TodaymillionsofAPIrequest...
Discovery Streamingonsdag den 6. marts 13
Netflix API Streamingonsdag den 6. marts 13
onsdag den 6. marts 13
Open API Netflix DevicesAPI Request Volume by Audienceonsdag den 6. marts 13
Netflix APIDependency ADependency DDependency GDependency JDependency MDependency PDependency BDependency EDependency HDepe...
/ps3/homeDependency F10 ThreadsDependency G10 ThreadsDependency H10 ThreadsDependency I5 ThreadsDependency J8 ThreadsDepen...
/ps3/homeDependency F10 ThreadsDependency G10 ThreadsDependency H10 ThreadsDependency I5 ThreadsDependency J8 ThreadsDepen...
/ps3/homeDependency F10 ThreadsDependency G10 ThreadsDependency H10 ThreadsDependency I5 ThreadsDependency J8 ThreadsDepen...
RxJava“a library for composingasynchronous and event-basedprograms using observablesequences for the Java VM”A Java port o...
Do we really need another way of“managing” concurrency?onsdag den 6. marts 13
Discovery of Rx began with a re-architecture ...onsdag den 6. marts 13
... that collapsed network traffic into coarse API calls ...onsdag den 6. marts 13
... that collapsed network traffic into coarse API calls ...Nested, conditional, parallel executiononsdag den 6. marts 13
... and wewanted to allowanybody tocreate endpoints,not just the“API Team”onsdag den 6. marts 13
onsdag den 6. marts 13
Concurrency withouteach engineer readingand re-reading this ->(awesome book ... everybody isn’tgoing to - or should have t...
Owner of API should retaincontrol of concurrency behavior.onsdag den 6. marts 13
public Data getData();What if the implementation needs to changefrom synchronous to asynchronous?How should the client exe...
public void getData(Callback<T> c);public Future<T> getData();public Future<List<Future<T>>> getData();What about ... ?ons...
IterablepullObservablepushT next()throws Exceptionreturns;onNext(T)onError(Exception)onCompleted()onsdag den 6. marts 13
IterablepullObservablepushT next()throws Exceptionreturns;onNext(T)onError(Exception)onCompleted()	  //	  Iterable<String>...
IterablepullObservablepushT next()throws Exceptionreturns;onNext(T)onError(Exception)onCompleted()	  //	  Iterable<String>...
Instead of blocking APIs ...class	  VideoService	  {	  	  	  def	  VideoList	  getPersonalizedListOfMovies(userId);	  	  	...
onsdag den 6. marts 13
onsdag den 6. marts 13
onsdag den 6. marts 13
onsdag den 6. marts 13
onsdag den 6. marts 13
onsdag den 6. marts 13
Observable.toObservable("one",	  "two",	  "three")	  	  	  	  	  .take(2)	  	  	  	  	  .subscribe((arg)	  -­‐>	  {	  	  	...
 	  	  	  Observable.create({	  observer	  -­‐>	  	  	  	  	  	  try	  {	  	  	  	  	  	  	  	  	  observer.onNext(new	  V...
 	  	  	  def	  Observable<VideoRating>	  getRating(userId,	  videoId)	  {	  	  	  	  	  	  	  	  //	  fetch	  the	  Video...
 	  	  	  def	  Observable<VideoRating>	  getRating(userId,	  videoId)	  {	  	  	  	  	  	  	  	  //	  fetch	  the	  Video...
Synchronous Observable with Multiple Values	  	  	  	  def	  Observable<Video>	  getVideos()	  {	  	  	  	  	  	  	  	  re...
Synchronous Observable with Multiple Values	  	  	  	  def	  Observable<Video>	  getVideos()	  {	  	  	  	  	  	  	  	  re...
Asynchronous Observable with Multiple Values	  def	  Observable<Video>	  getVideos()	  {	  	  	  	  return	  Observable.cr...
Asynchronous Observable with Multiple Values	  def	  Observable<Video>	  getVideos()	  {	  	  	  	  return	  Observable.cr...
Observable<SomeData> a = getDataA();Observable<SomeData> b = getDataB();Observable<SomeData> c = getDataC();Observable.mer...
Observable<SomeData> a = getDataA();Observable<String> b = getDataB();Observable<MoreData> c = getDataC();Observable.zip(a...
Observable<SomeData> a = getDataA();Observable<String> b = getDataB();Observable<MoreData> c = getDataC();Observable.zip(a...
Observable<SomeData> a = getDataA();Observable<String> b = getDataB();Observable<MoreData> c = getDataC().onErrorResumeNex...
def Observable getVideos(userId) {return VideoService.getVideos(userId)}onsdag den 6. marts 13
def Observable getVideos(userId) {return VideoService.getVideos(userId)}Asynchronous request thatreturns Observable<Video>...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
 	  	  	  Observable<R>	  b	  =	  Observable<T>.map({	  T	  t	  -­‐>	  	  	  	  	  	  	  R	  r	  =	  ...	  transform	  t	 ...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
 Observable<R>	  b	  =	  Observable<T>.mapMany({	  T	  t	  -­‐>	  	  	  	  	  Observable<R>	  r	  =	  ...	  transform	  t	...
 Observable<R>	  b	  =	  Observable<T>.mapMany({	  T	  t	  -­‐>	  	  	  	  	  Observable<R>	  r	  =	  ...	  transform	  t	...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
 	  	  	  Observable.zip(a,	  b,	  {	  a,	  b,	  -­‐>	  	  	  	  	  	  	  ...	  operate	  on	  values	  from	  both	  a	  ...
def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take...
onsdag den 6. marts 13
Observable<Video> emits n videos to onNext()onsdag den 6. marts 13
Takes first 10 then unsubscribes from origin.Returns Observable<Video> that emits 10 Videos.onsdag den 6. marts 13
For each of the 10 Video objects it transformsvia ‘mapMany’ function that does nested async calls.onsdag den 6. marts 13
For each Video ‘v’ it calls getMetadata()which returns Observable<VideoMetadata>These nested asyncrequests return Observab...
The Observable<VideoMetadata> is transformed viaa ‘map’ function to return a Map of key/values.onsdag den 6. marts 13
Same for Observable<VideoBookmark> andObservable<VideoRating>onsdag den 6. marts 13
The 3 ‘mapped’ Observables are combined with a‘zip’ function that emits a Map with all data.onsdag den 6. marts 13
The full sequence emits Observable<Map> thatemits a Map for each of 10 Videos.onsdag den 6. marts 13
Client code treats all interactionswith the API as asynchronousThe API implementation chooseswhether something isblocking ...
Example of latency reduction achieved by increasingnumber of threads used by Observables.onsdag den 6. marts 13
+Observable<User>	  u	  =	  new	  GetUserCommand(id).observe();Observable<Geo>	  g	  =	  new	  GetGeoCommand(request).obse...
<dependency	  org="com.netflix.rxjava"	  name="rxjava-­‐core"	  rev="x.y.z"	  /><dependency	  org="com.netflix.rxjava"	  n...
Functional Reactive in the Netflix API with RxJavahttp://techblog.netflix.com/2013/02/rxjava-netflix-api.htmlOptimizing the N...
Upcoming SlideShare
Loading in...5
×

Functional Reactive Programming in the Netflix API

5,073

Published on

Video and slides synchronized, mp3 and slide download available at http://bit.ly/XRdkqc.

Ben Christensen describes how Neflix has optimized their API using a functional reactive programming (modeled after Rx) in a polyglot Java stack. Filmed at qconlondon.com.

Ben Christensen is a software engineer on the Netflix API Platform team responsible for fault tolerance, performance, architecture and scale while enabling millions of customers to access the Netflix experience across more than 800 different device types. Prior to Netflix, Ben was at Apple in the iTunes division making iOS apps and media available to the world. Twitter: @benjchristensen

Published in: Technology
0 Comments
27 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,073
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
27
Embeds 0
No embeds

No notes for slide

Transcript of "Functional Reactive Programming in the Netflix API"

  1. 1. Functional Reactive Programmingin the Netflix APIBen ChristensenSoftware Engineer – API Platform at Netflix@benjchristensenhttp://www.linkedin.com/in/benjchristensenhttp://techblog.netflix.com/QCon London – March 6 2013onsdag den 6. marts 13
  2. 2. InfoQ.com: News & Community Site• 750,000 unique visitors/month• Published in 4 languages (English, Chinese, Japanese and BrazilianPortuguese)• Post content from our QCon conferences• News 15-20 / week• Articles 3-4 / week• Presentations (videos) 12-15 / week• Interviews 2-3 / week• Books 1 / monthWatch the video with slidesynchronization on InfoQ.com!http://www.infoq.com/presentations/netflix-functional-rx
  3. 3. Presented at QCon Londonwww.qconlondon.comPurpose of QCon- to empower software development by facilitating the spread ofknowledge and innovationStrategy- practitioner-driven conference designed for YOU: influencers ofchange and innovation in your teams- speakers and topics driving the evolution and innovation- connecting and catalyzing the influencers and innovatorsHighlights- attended by more than 12,000 delegates since 2007- held in 9 cities worldwide
  4. 4. onsdag den 6. marts 13
  5. 5. More than 33 million Subscribersin more than 50 Countries and Territoriesonsdag den 6. marts 13
  6. 6. Netflix accounts for 33% of Peak DownstreamInternet Traffic in North AmericaNetflix subscribers are watchingmore than 1 billion hours a monthonsdag den 6. marts 13
  7. 7. API traffic has grown from~20 million/day in 2010 to >2 billion/day05001000150020002010 2011 2012 TodaymillionsofAPIrequestsperdayonsdag den 6. marts 13
  8. 8. Discovery Streamingonsdag den 6. marts 13
  9. 9. Netflix API Streamingonsdag den 6. marts 13
  10. 10. onsdag den 6. marts 13
  11. 11. Open API Netflix DevicesAPI Request Volume by Audienceonsdag den 6. marts 13
  12. 12. Netflix APIDependency ADependency DDependency GDependency JDependency MDependency PDependency BDependency EDependency HDependency KDependency NDependency QDependency CDependency FDependency IDependency LDependency ODependency Ronsdag den 6. marts 13
  13. 13. /ps3/homeDependency F10 ThreadsDependency G10 ThreadsDependency H10 ThreadsDependency I5 ThreadsDependency J8 ThreadsDependency A10 ThreadsDependency B8 ThreadsDependency C10 ThreadsDependency D15 ThreadsDependency E5 ThreadsDependency K15 ThreadsDependency L4 ThreadsDependency M5 ThreadsDependency N10 ThreadsDependency O10 ThreadsDependency P10 ThreadsDependency Q8 ThreadsDependency R10 ThreadsDependency S8 ThreadsDependency T10 Threads/android/home/tv/homeFunctional Reactive Dynamic EndpointsAsynchronous Java APIonsdag den 6. marts 13
  14. 14. /ps3/homeDependency F10 ThreadsDependency G10 ThreadsDependency H10 ThreadsDependency I5 ThreadsDependency J8 ThreadsDependency A10 ThreadsDependency B8 ThreadsDependency C10 ThreadsDependency D15 ThreadsDependency E5 ThreadsDependency K15 ThreadsDependency L4 ThreadsDependency M5 ThreadsDependency N10 ThreadsDependency O10 ThreadsDependency P10 ThreadsDependency Q8 ThreadsDependency R10 ThreadsDependency S8 ThreadsDependency T10 Threads/android/home/tv/homeFunctional Reactive Dynamic EndpointsAsynchronous Java APIHystrixfault-isolation layeronsdag den 6. marts 13
  15. 15. /ps3/homeDependency F10 ThreadsDependency G10 ThreadsDependency H10 ThreadsDependency I5 ThreadsDependency J8 ThreadsDependency A10 ThreadsDependency B8 ThreadsDependency C10 ThreadsDependency D15 ThreadsDependency E5 ThreadsDependency K15 ThreadsDependency L4 ThreadsDependency M5 ThreadsDependency N10 ThreadsDependency O10 ThreadsDependency P10 ThreadsDependency Q8 ThreadsDependency R10 ThreadsDependency S8 ThreadsDependency T10 Threads/android/home/tv/homeFunctional Reactive Dynamic EndpointsAsynchronous Java APIonsdag den 6. marts 13
  16. 16. RxJava“a library for composingasynchronous and event-basedprograms using observablesequences for the Java VM”A Java port of Rx (Reactive Extensions)https://rx.codeplex.com (.Net and Javascript by Microsoft)onsdag den 6. marts 13
  17. 17. Do we really need another way of“managing” concurrency?onsdag den 6. marts 13
  18. 18. Discovery of Rx began with a re-architecture ...onsdag den 6. marts 13
  19. 19. ... that collapsed network traffic into coarse API calls ...onsdag den 6. marts 13
  20. 20. ... that collapsed network traffic into coarse API calls ...Nested, conditional, parallel executiononsdag den 6. marts 13
  21. 21. ... and wewanted to allowanybody tocreate endpoints,not just the“API Team”onsdag den 6. marts 13
  22. 22. onsdag den 6. marts 13
  23. 23. Concurrency withouteach engineer readingand re-reading this ->(awesome book ... everybody isn’tgoing to - or should have to - readit though, that’s the point)onsdag den 6. marts 13
  24. 24. Owner of API should retaincontrol of concurrency behavior.onsdag den 6. marts 13
  25. 25. public Data getData();What if the implementation needs to changefrom synchronous to asynchronous?How should the client execute that methodwithout blocking? spawn a thread?Owner of API should retaincontrol of concurrency behavior.onsdag den 6. marts 13
  26. 26. public void getData(Callback<T> c);public Future<T> getData();public Future<List<Future<T>>> getData();What about ... ?onsdag den 6. marts 13
  27. 27. IterablepullObservablepushT next()throws Exceptionreturns;onNext(T)onError(Exception)onCompleted()onsdag den 6. marts 13
  28. 28. IterablepullObservablepushT next()throws Exceptionreturns;onNext(T)onError(Exception)onCompleted()  //  Iterable<String>    //  that  contains  75  Strings  getDataFromLocalMemory()    .skip(10)    .take(5)    .map({  s  -­‐>        return  s  +  "_transformed"})    .forEach(          {  println  "next  =>  "  +  it})  //  Observable<String>    //  that  emits  75  Strings  getDataFromNetwork()    .skip(10)    .take(5)    .map({  s  -­‐>        return  s  +  "_transformed"})    .subscribe(          {  println  "onNext  =>  "  +  it})onsdag den 6. marts 13
  29. 29. IterablepullObservablepushT next()throws Exceptionreturns;onNext(T)onError(Exception)onCompleted()  //  Iterable<String>    //  that  contains  75  Strings  getDataFromLocalMemory()    .skip(10)    .take(5)    .map({  s  -­‐>        return  s  +  "_transformed"})    .forEach(          {  println  "onNext  =>  "  +  it})  //  Observable<String>    //  that  emits  75  Strings  getDataFromNetwork()    .skip(10)    .take(5)    .map({  s  -­‐>        return  s  +  "_transformed"})    .subscribe(          {  println  "onNext  =>  "  +  it})onsdag den 6. marts 13
  30. 30. Instead of blocking APIs ...class  VideoService  {      def  VideoList  getPersonalizedListOfMovies(userId);      def  VideoBookmark  getBookmark(userId,  videoId);      def  VideoRating  getRating(userId,  videoId);      def  VideoMetadata  getMetadata(videoId);}class  VideoService  {      def  Observable<VideoList>  getPersonalizedListOfMovies(userId);      def  Observable<VideoBookmark>  getBookmark(userId,  videoId);      def  Observable<VideoRating>  getRating(userId,  videoId);      def  Observable<VideoMetadata>  getMetadata(videoId);}... create Observable APIs:onsdag den 6. marts 13
  31. 31. onsdag den 6. marts 13
  32. 32. onsdag den 6. marts 13
  33. 33. onsdag den 6. marts 13
  34. 34. onsdag den 6. marts 13
  35. 35. onsdag den 6. marts 13
  36. 36. onsdag den 6. marts 13
  37. 37. Observable.toObservable("one",  "two",  "three")          .take(2)          .subscribe((arg)  -­‐>  {                    System.out.println(arg);          });Java8Observable.toObservable("one",  "two",  "three")    .take(2)    .subscribe((arg:  String)  =>  {            println(arg)    })Scala(-­‐>      (Observable/toObservable  ["one"  "two"  "three"])    (.take  2)      (.subscribe  (fn  [arg]  (println  arg))))Clojure    Observable.toObservable("one",  "two",  "three")        .take(2)          .subscribe({arg  -­‐>  println(arg)})Groovy    Observable.toObservable("one",  "two",  "three")        .take(2)          .subscribe(lambda  {  |arg|  puts  arg  })JRubyonsdag den 6. marts 13
  38. 38.        Observable.create({  observer  -­‐>            try  {                  observer.onNext(new  Video(id))                observer.onCompleted();            }  catch(Exception  e)  {                observer.onError(e);            }        })onsdag den 6. marts 13
  39. 39.        def  Observable<VideoRating>  getRating(userId,  videoId)  {                //  fetch  the  VideoRating  for  this  user  asynchronously                return  Observable.create({  observer  -­‐>                        executor.execute(new  Runnable()  {                                def  void  run()  {                                    try  {                                          VideoRating  rating  =  ...  do  network  call  ...                                        observer.onNext(rating)                                        observer.onCompleted();                                      }  catch(Exception  e)  {                                        observer.onError(e);                                      }                                      }                        })                })        }Asynchronous Observable with Single Valueonsdag den 6. marts 13
  40. 40.        def  Observable<VideoRating>  getRating(userId,  videoId)  {                //  fetch  the  VideoRating  for  this  user  asynchronously                return  Observable.create({  observer  -­‐>                        executor.execute(new  Runnable()  {                                def  void  run()  {                                    try  {                                          VideoRating  rating  =  ...  do  network  call  ...                                        observer.onNext(rating)                                        observer.onCompleted();                                      }  catch(Exception  e)  {                                        observer.onError(e);                                      }                                      }                        })                })        }Asynchronous Observable with Single Valueonsdag den 6. marts 13
  41. 41. Synchronous Observable with Multiple Values        def  Observable<Video>  getVideos()  {                return  Observable.create({  observer  -­‐>                      try  {                              for(v  in  videos)  {                                observer.onNext(v)                          }                          observer.onCompleted();                      }  catch(Exception  e)  {                          observer.onError(e);                      }                })        }Caution: This is eager and will always emit all valuesregardless of subsequent operators such as take(10)onsdag den 6. marts 13
  42. 42. Synchronous Observable with Multiple Values        def  Observable<Video>  getVideos()  {                return  Observable.create({  observer  -­‐>                      try  {                              for(v  in  videos)  {                                observer.onNext(v)                          }                          observer.onCompleted();                      }  catch(Exception  e)  {                          observer.onError(e);                      }                })        }Caution: This is eager and will always emit all valuesregardless of subsequent operators such as take(10)onsdag den 6. marts 13
  43. 43. Asynchronous Observable with Multiple Values  def  Observable<Video>  getVideos()  {        return  Observable.create({  observer  -­‐>              executor.execute(new  Runnable()  {                    def  void  run()  {                        try  {                                for(id  in  videoIds)  {                                  Video  v  =  ...  do  network  call  ...                                  observer.onNext(v)                              }                              observer.onCompleted();                          }  catch(Exception  e)  {                              observer.onError(e);                          }                      }              })        })  }onsdag den 6. marts 13
  44. 44. Asynchronous Observable with Multiple Values  def  Observable<Video>  getVideos()  {        return  Observable.create({  observer  -­‐>              executor.execute(new  Runnable()  {                    def  void  run()  {                        try  {                                for(id  in  videoIds)  {                                  Video  v  =  ...  do  network  call  ...                                  observer.onNext(v)                              }                              observer.onCompleted();                          }  catch(Exception  e)  {                              observer.onError(e);                          }                      }              })        })  }onsdag den 6. marts 13
  45. 45. Observable<SomeData> a = getDataA();Observable<SomeData> b = getDataB();Observable<SomeData> c = getDataC();Observable.merge(a, b, c).subscribe({ element -> println("data: " + element)},{ exception -> println("error occurred: "+ exception.getMessage())})Combining via Mergeonsdag den 6. marts 13
  46. 46. Observable<SomeData> a = getDataA();Observable<String> b = getDataB();Observable<MoreData> c = getDataC();Observable.zip(a, b, c, {x, y, z -> [x, y, z]}).subscribe({ triple -> println("a: " + triple[0]+ " b: " + triple[1]+ " c: " + triple[2])},{ exception -> println("error occurred: "+ exception.getMessage())})Combining via Ziponsdag den 6. marts 13
  47. 47. Observable<SomeData> a = getDataA();Observable<String> b = getDataB();Observable<MoreData> c = getDataC();Observable.zip(a, b, c, {x, y, z -> [x, y, z]}).subscribe({ triple -> println("a: " + triple[0]+ " b: " + triple[1]+ " c: " + triple[2])},{ exception -> println("error occurred: "+ exception.getMessage())})Error Handlingonsdag den 6. marts 13
  48. 48. Observable<SomeData> a = getDataA();Observable<String> b = getDataB();Observable<MoreData> c = getDataC().onErrorResumeNext(getFallbackForDataC());Observable.zip(a, b, c, {x, y, z -> [x, y, z]}).subscribe({ triple -> println("a: " + triple[0]+ " b: " + triple[1]+ " c: " + triple[2])},{ exception -> println("error occurred: "+ exception.getMessage())})Error Handlingonsdag den 6. marts 13
  49. 49. def Observable getVideos(userId) {return VideoService.getVideos(userId)}onsdag den 6. marts 13
  50. 50. def Observable getVideos(userId) {return VideoService.getVideos(userId)}Asynchronous request thatreturns Observable<Video>onsdag den 6. marts 13
  51. 51. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10)}onsdag den 6. marts 13
  52. 52. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10)}Reactive operator on the Observablethat takes the first 10 Video objectsthen unsubscribes.onsdag den 6. marts 13
  53. 53. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).map({ Video video ->// transform video object})}onsdag den 6. marts 13
  54. 54. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).map({ Video video ->// transform video object})}The ‘map’ operator allows transformingthe input value into a different output.onsdag den 6. marts 13
  55. 55.        Observable<R>  b  =  Observable<T>.map({  T  t  -­‐>              R  r  =  ...  transform  t  ...            return  r;        })onsdag den 6. marts 13
  56. 56. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->// for each video we want to fetch metadatadef m = video.getMetadata().map({ Map<String, String> md ->// transform to the data and format we wantreturn [title: md.get("title"),length: md.get("duration")]})// and its rating and bookmarkdef b ...def r ...})}onsdag den 6. marts 13
  57. 57. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->// for each video we want to fetch metadatadef m = video.getMetadata().map({ Map<String, String> md ->// transform to the data and format we wantreturn [title: md.get("title"),length: md.get("duration")]})// and its rating and bookmarkdef b ...def r ...})}We change to ‘mapMany’ which islike merge(map()) since we will returnan Observable<T> instead of T.onsdag den 6. marts 13
  58. 58.  Observable<R>  b  =  Observable<T>.mapMany({  T  t  -­‐>          Observable<R>  r  =  ...  transform  t  ...        return  r;  })onsdag den 6. marts 13
  59. 59.  Observable<R>  b  =  Observable<T>.mapMany({  T  t  -­‐>          Observable<R>  r  =  ...  transform  t  ...        return  r;  })onsdag den 6. marts 13
  60. 60. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->// for each video we want to fetch metadatadef m = video.getMetadata().map({ Map<String, String> md ->// transform to the data and format we wantreturn [title: md.get("title"),length: md.get("duration")]})// and its rating and bookmarkdef b ...def r ...})}Nested asynchronous callsthat return more Observables.onsdag den 6. marts 13
  61. 61. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->// for each video we want to fetch metadatadef m = video.getMetadata().map({ Map<String, String> md ->// transform to the data and format we wantreturn [title: md.get("title"),length: md.get("duration")]})// and its rating and bookmarkdef b ...def r ...})}Observable<VideoMetadata>Observable<VideoBookmark>Observable<VideoRating>onsdag den 6. marts 13
  62. 62. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->// for each video we want to fetch metadatadef m = video.getMetadata().map({ Map<String, String> md ->// transform to the data and format we wantreturn [title: md.get("title"),length: md.get("duration")]})// and its rating and bookmarkdef b ...def r ...})}Each Observable transformsits data using ‘map’onsdag den 6. marts 13
  63. 63. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->// for each video we want to fetch metadatadef m = video.getMetadata().map({ Map<String, String> md ->// transform to the data and format we wantreturn [title: md.get("title"),length: md.get("duration")]})// and its rating and bookmarkdef b ...def r ...// compose these together})}onsdag den 6. marts 13
  64. 64. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->def m ...def b ...def r ...// compose these together})}onsdag den 6. marts 13
  65. 65. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->def m ...def b ...def r ...// compose these togetherreturn Observable.zip(m, b, r, {metadata, bookmark, rating ->// now transform to complete dictionary// of data we want for each Videoreturn [id: video.videoId]<< metadata << bookmark << rating})})}onsdag den 6. marts 13
  66. 66. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->def m ...def b ...def r ...// compose these togetherreturn Observable.zip(m, b, r, {metadata, bookmark, rating ->// now transform to complete dictionary// of data we want for each Videoreturn [id: video.videoId]<< metadata << bookmark << rating})})}The ‘zip’ operator combines the 3asynchronous Observables into 1onsdag den 6. marts 13
  67. 67.        Observable.zip(a,  b,  {  a,  b,  -­‐>              ...  operate  on  values  from  both  a  &  b  ...            return  [a,  b];  //  i.e.  return  tuple        })onsdag den 6. marts 13
  68. 68. def Observable<Map> getVideos(userId) {return VideoService.getVideos(userId)// we only want the first 10 of each list.take(10).mapMany({ Video video ->def m ...def b ...def r ...// compose these togetherreturn Observable.zip(m, b, r, {metadata, bookmark, rating ->// now transform to complete dictionary// of data we want for each Videoreturn [id: video.videoId]<< metadata << bookmark << rating})})}return a single Map (dictionary)of transformed and combined datafrom 4 asynchronous callsonsdag den 6. marts 13
  69. 69. onsdag den 6. marts 13
  70. 70. Observable<Video> emits n videos to onNext()onsdag den 6. marts 13
  71. 71. Takes first 10 then unsubscribes from origin.Returns Observable<Video> that emits 10 Videos.onsdag den 6. marts 13
  72. 72. For each of the 10 Video objects it transformsvia ‘mapMany’ function that does nested async calls.onsdag den 6. marts 13
  73. 73. For each Video ‘v’ it calls getMetadata()which returns Observable<VideoMetadata>These nested asyncrequests return Observablesthat emit 1 value.onsdag den 6. marts 13
  74. 74. The Observable<VideoMetadata> is transformed viaa ‘map’ function to return a Map of key/values.onsdag den 6. marts 13
  75. 75. Same for Observable<VideoBookmark> andObservable<VideoRating>onsdag den 6. marts 13
  76. 76. The 3 ‘mapped’ Observables are combined with a‘zip’ function that emits a Map with all data.onsdag den 6. marts 13
  77. 77. The full sequence emits Observable<Map> thatemits a Map for each of 10 Videos.onsdag den 6. marts 13
  78. 78. Client code treats all interactionswith the API as asynchronousThe API implementation chooseswhether something isblocking or non-blockingandwhat resources it uses.onsdag den 6. marts 13
  79. 79. Example of latency reduction achieved by increasingnumber of threads used by Observables.onsdag den 6. marts 13
  80. 80. +Observable<User>  u  =  new  GetUserCommand(id).observe();Observable<Geo>  g  =  new  GetGeoCommand(request).observe();Observable.zip(u,  g,  {user,  geo  -­‐>                  return  [username:  user.getUsername(),                                  currentLocation:  geo.getCounty()]      })RxJava coming to Hystrixhttps://github.com/Netflix/Hystrixonsdag den 6. marts 13
  81. 81. <dependency  org="com.netflix.rxjava"  name="rxjava-­‐core"  rev="x.y.z"  /><dependency  org="com.netflix.rxjava"  name="rxjava-­‐groovy"  rev="x.y.z"  /><dependency  org="com.netflix.rxjava"  name="rxjava-­‐clojure"  rev="x.y.z"  /><dependency  org="com.netflix.rxjava"  name="rxjava-­‐scala"  rev="x.y.z"  /><dependency  org="com.netflix.rxjava"  name="rxjava-­‐jruby"  rev="x.y.z"  /><dependency>        <groupId>com.netflix.rxjava</groupId>        <artifactId>rxjava-­‐core</artifactId>        <version>x.y.z</version></dependency>... or for a different language ...To get started ...onsdag den 6. marts 13
  82. 82. Functional Reactive in the Netflix API with RxJavahttp://techblog.netflix.com/2013/02/rxjava-netflix-api.htmlOptimizing the Netflix APIhttp://techblog.netflix.com/2013/01/optimizing-netflix-api.htmlRxJavahttps://github.com/Netflix/RxJava@RxJavaBen Christensen@benjchristensenhttp://www.linkedin.com/in/benjchristensenNetflix is Hiringhttp://jobs.netflix.comonsdag den 6. marts 13

×