SlideShare a Scribd company logo
+
JSONG

JSONGPath

HTTP

Path Evaluators

Reactive REST
RESTful API for Application Data
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/netflix-reactive-rest

InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Presented at QCon San Francisco
www.qconsf.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
+

The is the story…
…of how we freed REST from HTTP...

…and discovered it was even more
Powerful than we thought it was.
+

Netflix
+

Question

What’s the most successful distributed system designed for
information browsing?
+

Answer
+

WWW Caching

index.html

Browser
Cache

index.html
index.html

cnn.com
+

REST


Unique ID for every resource



Idempotent VERBS


GET



PUT
+

REST Benefits


Cache Transparency



Cache Coherence
+

REST + HTTP: Two great tastes


Unique ID for every resource (URL)



Idempotent VERBS





GET
PUT

Cache Control Headers for Invalidation
+

Netflix is a web application…

Why not use the browser cache for our data?
+

Why not the Browser Cache?
http://netflix.com/videos/234234
http://netflix.com/videos/234234/rating
http://netflix.com/videos/54325
http://netflix.com/videos/54325/rating
http://netflix.com/videos/12356/name
http://netflix.com/videos/12356/rating
http://netflix.com/videos/876456/name
http://netflix.com/videos/876456/rating

Too many HTTP requests!
+

Challenge

Can we build a high-performance REST API for our data?
+

Introducing Falkor


RESTful Query API for Data



Coherent, Transparent, Managed Cache



Efficient bulk data transfer over HTTP



Query Optimization
+

Netflix Domain Model
+

Netflix Domain Model
http://netflix.com/user
{
videoLists: [
[
{
name: “Die hard”,
rating: 4.0
},
// more titles
],
[
{
name: “Die hard”,
rating: 4.0
},
// more titles
],
// more lists…
]
}
+

RESTful Data Access API
var userModel = {
videoLists: {
“0”: {
“name”: “Thrillers”,
“0”: {
id: 2654,
name: “Die hard”,
rating: 4.0
},
// more titles
length: 74,
}
}
}

userModel[‘videoLists’][0][0][‘name’]
[‘videoLists’, 0, 0, ‘name’]
+

Building a Proxy for a Remote Model

// Create a proxy or the current user’s domain model
var remoteModel =
new RemotePathEvaluator(“http://netflix.com/user”);

Server Path Evaluator

Falkor Server

Remote Path Evaluator
+

Retrieving Data from the Server

// Retrieve the name of the first title in the first genre list.
var remoteModel.
get([“videoLists”,0,0,”name”]).
forEach(function(pathBoundValue) {
console.log(pathBoundValue);
});
[“videoLists”,0,0,”name”]

{
path: [“videoLists”,0,0,”name”],
value: “Die Hard”
}

netflix.com/user
+

Retrieving Sub Graph

// Retrieve the names of the first 10 title in the first
// 10 genre lists.
var remoteModel.
get([“videoLists”,{from:0,to:9},{from:0,to:9},”name”]).
forEach(function(pathBoundValue) {
console.log(pathBoundValue);
});

>
>
>
>
>

{path: [“videoLists”,0,0,”name”],
{path: [“videoLists”,0,1,”name”],
// snip
{path: [“videoLists”,9,8,”name”],
{path: [“videoLists”,9,9,”name”],

value: “Die Hard”}
value: “Amelie”}
value: “The New Guys”}
value: “Animal House”}
+

Caching Data

// Create a proxy or the current user’s domain model
var remoteModel =
new RemotePathEvaluator(“http://netflix.com/user”);
// Create a path evaluator for a local, in-memory cache.
// Cache uses LRU to ensure size stays < 10000.
var localModel = new SizedMemoryPathEvaluator(10000, {});
// Create a cached path evaluator, using the remoteModel as the
// source and the in-memory model as the cache.
var cachedModel = remoteModel.cache(localModel);
+

Caching Data on the Client
Client Memory
Server Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

Cached Path Evaluator
+

Retrieving Cached Data

var cachedModel.
get([“videoLists”,0,0,”name”]).
forEach(function(pathBoundValue) {
console.log(pathBoundValue.value);
});

[“videoLists”,0,0,”name”]

{
path: [“videoLists”,0,0,”name”],
value: “Die Hard”
}

Local
Cache
Cache

[“videoLists”,0,0,”name”]
+

Retrieving Cached Data

var cachedModel.
get([“videoLists”,0,0,”name”]).
forEach(function(pathBoundValue) {
console.log(pathBoundValue.value);
});

[“videoLists”,0,0,”name”]

{
path: [“videoLists”,0,0,”name”],
value: “Die Hard”
}

Local
Cache
Cache
+

Query Optimization

// Query for another video property can be optimized!
var cachedModel.
get([“videoLists”,0,0,”rating”]).
forEach(function(pathBoundValue) {
console.log(pathBoundValue.value);
});
Query is transparently optimized!

[“videoLists”,0,0,”rating”]

{
path: [“videoLists”,0,0,”rating”],
value: “Die Hard”
}

Local
Cache
Cache

[“videos”,234234,”rating”]
+

Question

So how does it work?
+

Features


Efficient data transfer over HTTP



Cache Coherence



Cache Transparency



Query Optimization
+

One Resource/One Domain Model
http://netflix.com/videos/234234/name
http://netflix.com/videos/234234/rating
http://netflix.com/videos/54325/name
http://netflix.com/user?path=[“videos”,234,”name”]&path=…
http://netflix.com/videos/54325/rating

http://netflix.com/videos/12356/name
http://netflix.com/videos/12356/rating
http://netflix.com/videos/876456/name
http://netflix.com/videos/876456/rating
+

Features


Efficient data transfer over HTTP



Cache Coherence



Cache Transparency



Query Optimization
+

Netflix Domain Model
http://netflix.com/user
{
videoLists: {
“0”: {
“name”: “Thrillers”,
“0”: {
name: “Die hard”,
rating: 4.0
},
// more titles
length: 74,
},
“1”: {
“name”: “Action Movies”,
“0”: {
name: “Die hard”,
rating: 4.0
},
// more titles
length: 74
}, // more lists…
length: 25
}
}
+

What’s wrong with JSON?

“videoLists”

“0”

“0”

“name”

“Die Hard”

“1”

“0”

Same movie appears
twice in the same message!

“name”

“Die Hard”
+

The Problem

How to model a graph in JSON?
+

Introducing JSON Graph (JSONG)
+

JSONG


Graph representation language in JSON



Two types:
1.

Maps

2.

Values (includes Arrays)
+

JSONG Benefits


Serializable



Partitionable
+

JSON
http://netflix.com/user
{
videoLists: {
“0”: {
“name”: “Thrillers”,
“0”: {
id: 2654,
name: “Die hard”,
rating: 4.0
},
// more titles
length: 74,
},
“1”: {
“name”: “Action Movies”,
“0”: {
id: 2654,
name: “Die hard”,
rating: 4.0
},
// more titles
length: 74
}, // more lists…
length: 25
}

}
+

JSONG
http://netflix.com/user
{
videoLists: {
“0”: {
“name”: “Thrillers”,
“0”: [“videos”, 2654],
// more titles
length: 74,
},
“1”: {
“name”: “Action Movies”,
“0”: [“videos”, 2654],
// more titles
length: 74
}, // more lists…
length: 25
},
videos: {
2654: {
name: “Die hard”,
rating: 4.0
}
}
}
+

Okay so we’ve got a graph…

…now we need a convenient API for querying data.
+

Hierarchical API
var userModel = {
videoLists: {
“0”: {
“name”: “Thrillers”,
“0”: {
id: 2654,
name: “Die hard”,
rating: 4.0
},
// more titles
length: 74,
}
}
}

JSONG Path

userModel[‘videoLists’][0][0][‘name’]
[‘videoLists’, 0, 0, ‘name’]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

[“videoLists”,0,0,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

[“videoLists”,0,0,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

[“videoLists”,0,0,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

[“videoLists”,0,0,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

Array found before last key
evaluated…

[“videoLists”,0,0,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

Array must be a path!

[“videoLists”,0,0,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

Array must be a path!

[“videos”,234,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

Rewrite path, and start
evaluating from top node.

[“videos”,234,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

[“videos”,234,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

[“videos”,234,”name”]
+

JSONG Path Evaluation
var netflixUser = {
videoLists: {
“0”: {
name: “Thrillers”,
“0”: [“videos”,234],
// more videos…
length: 75
},
“1”: {
name: “Action Movies”,
“0”: [“videos”,234],
// more videos…
length: 75
},
// more lists…
length: 15
},
videos: {
234: {name: “Die Hard”, rating:5}
}
}

[“videos”,234,”name”]
+

Okay…

…how do we make this process transparent to the developer?
+

Features


Efficient data transfer over HTTP



Cache Coherence



Cache Transparency



Query Optimization
+

Path Evaluator

Path Evaluator

get(Observable<Path>): Observable<PathBoundValue>
set(Observable<PathBoundValue>): Observable<PathBoundValue>
delete(Observable<Path>): Observable<PathBoundValue>
+

Observable
 Object
 Open

that represents stream of data

Source Reactive Extensions Library

 Ported

to
 Javascript
 .NET
C
 Java (Netflix)
+

Observable.forEach
// “subscribe”
var subscription =
remoteModel.get([“videoLists”,0,0,”name”]).
forEach(
event => console.log(event),error
error => console.error(error),
() => console.log(“done”));

// “unsubscribe”
subscription.dispose();

optional
+

Path Evaluator


Proxy for a JSONG Model



Idempotent Operations:



Set





Get

Delete

Composable
+

Path Evaluator Implementations


RemotePathEvaluator



SizedMemoryPathEvaluator



CachedPathEvaluator



LocalStoragePathEvaluator?



etc
+

Accessing Data on the Client
// Create a Proxy path evaluator for the server JSONG domain model
var remoteModel =
new RemotePathEvaluator(“http://netflix.com/tvui/user”);`
+

Accessing Data on the Client
// Create a Proxy path evaluator for the server JSONG domain model
var remoteModel =
new RemotePathEvaluator(“http://netflix.com/tvui/user”);`

Server Path Evaluator

Remote Path Evaluator
+

Accessing Data on the Client
// Create a Proxy path evaluator for the server JSONG domain model
var remoteModel =
new RemotePathEvaluator(“http://netflix.com/tvui/user”);
remoteModel.get([“videoLists”,{to:5},{to:10},”name”]).
forEach(function(pathBoundValue) { /* do something */ });

Server Path Evaluator

[videoLists,{to:5},{to:10},’name’]

Remote Path Evaluator
+

Accessing Data on the Client
// Create a Proxy path evaluator for the server JSONG domain model
var remoteModel =
new RemotePathEvaluator(“http://netflix.com/tvui/user”);
remoteModel.get([“videoLists”,{to:5},{to:10},”name”]).
forEach(function(pathBoundValue) { /* do something */ });

Server Path Evaluator

{ path: [“videoLists”,0,0,”name”], value: {…} }
{ path: [“videoLists”,0,1,”name”], value: {…} }
…
{ path: [“videoLists”,5,10,”name”], value: {…} }

Remote Path Evaluator
+

Caching Data on the Client
// Create a Proxy path evaluator for the server JSONG domain model
var remoteModel =
new RemotePathEvaluator(“http://netflix.com/tvui/user”);
// Create a path evaluator for a local, in-memory cache.
// Cache stores the fragments of the remote model and stays within
// size of 10000.
var localModelCache = new SizedMemoryPathEvaluator(10000, {});
Client Memory
Server Path Evaluator

Sized Memory Path Evaluator

Remote Path Evaluator
+

Sized Memory Path Evaluator


Works like a web browser cache!



Caches resources by path



Removes least-recently used resources
+

Caching Data on the Client
// Create a Proxy path evaluator for the server JSONG domain model
var remoteModel =
new RemotePathEvaluator(“http://netflix.com/tvui/user”);
// Create a path evaluator for a local, in-memory cache.
// Cache stores the fragments of the remote model and stays within
// size of 10000.
var localModel = new SizedMemoryPathEvaluator(10000, {});
// Create a cached path evaluator, using the remoteModel as the
// source and the in-memory model as the cache.
var cachedModel = remoteModel.cache(localModel);
+

Caching Data on the Client
Client Memory
Server Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

Cached Path Evaluator
[videoLists,{to:5},{to:10},’name’]
+

Caching Data on the Client
Client Memory
Server Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

Cached Path Evaluator
+

Caching Data on the Client
Client Memory
Server Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

[videoLists,{to:5},{to:10},’name’]`

Cached Path Evaluator
+

Caching Data on the Client
Client Memory
Server Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

Nothing

Cached Path Evaluator
+

Caching Data on the Client
Client Memory
Server Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator
[videoLists,{to:5},{to:10},’name’]

Cached Path Evaluator
+

Caching Data on the Client
Client Memory
Server Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator
{ path: [“videoLists”,0,0,”name”], value: {…} }
{ path: [“videoLists”,0,1,”name”], value: {…} }
…
{ path: [“videoLists”,5,10,”name”], value: {…} }

Cached Path Evaluator
+

Caching Data on the Client
Client Memory
Server Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

{ path: [“videoLists”,0,0,”name”], value: {…} }
{ path: [“videoLists”,0,1,”name”], value: {…} }
…
{ path: [“videoLists”,5,10,”name”], value: {…} }

Cached Path Evaluator
+

Caching Data on the Client
Client Memory
var netflixMember = {
videoLists: {
name: “Action Movies”,
“0”: [“lists”, 54354334],
// more lists…
“length: 15
},
“videos”: {
“34123432”:
{ … }
}
}

Server Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

Cached Path Evaluator
{ path: [“videoLists”,0,0,”name”], value: {…} }
{ path: [“videoLists”,0,1,”name”], value: {…} }
…
{ path: [“videoLists”,5,10,”name”], value: {…} }
+

Features


Efficient data transfer over HTTP



Cache Coherence



Cache Transparency



Query Optimization
+

Query Optimization
+

Query Optimization
Client Memory
Server Path Evaluator

var netflixMember = {
videoLists: {
name: “Action Movies”,
“0”: [“lists”, 54354334],
// more lists…
“length: 15
},
“videos”: {
“34123432”:
{ … }
}
}

TV Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

Cached Path Evaluator

[videoLists,1,3,’rating’]
+

Query Optimization
Client Memory
Server Path Evaluator

var netflixMember = {
videoLists: {
name: “Action Movies”,
“0”: [“lists”, 54354334],
// more lists…
“length: 15
},
“videos”: {
“34123432”:
{ … }
}
}

TV Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

Cached Path Evaluator

[videoLists,1,3,’rating’]
+

Query Optimization
Client Memory
Server Path Evaluator

var netflixMember = {
videoLists: {
name: “Action Movies”,
“0”: [“lists”, 54354334],
// more lists…
“length: 15
},
“videos”: {
“34123432”:
{ … }
}
}

TV Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

[videoLists,1,3,’rating’]

Cached Path Evaluator

[videoLists,1,3,’rating’]
+

Query Optimization
Client Memory
Server Path Evaluator

var netflixMember = {
videoLists: {
name: “Action Movies”,
“0”: [“lists”, 54354334],
// more lists…
“length: 15
},
“videos”: {
“34123432”:
{ … }
}
}

TV Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator

{path: [videoLists,1], value: [lists,23432] }
{path: [lists,23432,0], value: [movies,234234] }

Cached Path Evaluator

[videoLists,1,3,’rating’]
+

Query Optimization
Client Memory
Server Path Evaluator

var netflixMember = {
videoLists: {
name: “Action Movies”,
“0”: [“lists”, 54354334],
// more lists…
“length: 15
},
“videos”: {
“34123432”:
{ … }
}
}

TV Path Evaluator

Falkor Server

Sized Memory Path Evaluator

Remote Path Evaluator
[movies,234234,rating]

Cached Path Evaluator

Path optimized!!
+

Query Optimization


Path partially evaluated in the cache



Path is rewritten and optimized before it is sent to server
+

Query Optimization Rocks!

Same API for local and remote data!
+

Features


Efficient data transfer over HTTP



Cache Coherence



Cache Transparency



Query Optimization
+

Roadmap


Currently Internal Software



Defensive Patent Filed



Plan to open-source 2014
+

Questions?
Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/netflixreactive-rest

More Related Content

Viewers also liked

Microsoft Big Data @ SQLUG 2013
Microsoft Big Data @ SQLUG 2013Microsoft Big Data @ SQLUG 2013
Microsoft Big Data @ SQLUG 2013
Nathan Bijnens
 
a real-time architecture using Hadoop and Storm at Devoxx
a real-time architecture using Hadoop and Storm at Devoxxa real-time architecture using Hadoop and Storm at Devoxx
a real-time architecture using Hadoop and Storm at Devoxx
Nathan Bijnens
 
Reactor 3.0, a reactive foundation for java 8 and Spring
Reactor 3.0, a reactive foundation for java 8 and SpringReactor 3.0, a reactive foundation for java 8 and Spring
Reactor 3.0, a reactive foundation for java 8 and Spring
Stéphane Maldini
 
A real-time (lambda) architecture using Hadoop & Storm (NoSQL Matters Cologne...
A real-time (lambda) architecture using Hadoop & Storm (NoSQL Matters Cologne...A real-time (lambda) architecture using Hadoop & Storm (NoSQL Matters Cologne...
A real-time (lambda) architecture using Hadoop & Storm (NoSQL Matters Cologne...
Nathan Bijnens
 
Rethink Async With RXJS
Rethink Async With RXJSRethink Async With RXJS
Rethink Async With RXJS
Ryan Anklam
 
Virdata: lessons learned from the Internet of Things and M2M Cloud Services @...
Virdata: lessons learned from the Internet of Things and M2M Cloud Services @...Virdata: lessons learned from the Internet of Things and M2M Cloud Services @...
Virdata: lessons learned from the Internet of Things and M2M Cloud Services @...
Nathan Bijnens
 
Functional Reactive Programming in the Netflix API
Functional Reactive Programming in the Netflix APIFunctional Reactive Programming in the Netflix API
Functional Reactive Programming in the Netflix API
C4Media
 
A real time architecture using Hadoop and Storm @ FOSDEM 2013
A real time architecture using Hadoop and Storm @ FOSDEM 2013A real time architecture using Hadoop and Storm @ FOSDEM 2013
A real time architecture using Hadoop and Storm @ FOSDEM 2013
Nathan Bijnens
 
Twitter Big Data
Twitter Big DataTwitter Big Data
Twitter Big Data
Colin Surprenant
 
Supercharged java 8 : with cyclops-react
Supercharged java 8 : with cyclops-reactSupercharged java 8 : with cyclops-react
Supercharged java 8 : with cyclops-react
John McClean
 
Practical Akka HTTP - introduction
Practical Akka HTTP - introductionPractical Akka HTTP - introduction
Practical Akka HTTP - introduction
Łukasz Sowa
 
Building Beautiful REST APIs with ASP.NET Core
Building Beautiful REST APIs with ASP.NET CoreBuilding Beautiful REST APIs with ASP.NET Core
Building Beautiful REST APIs with ASP.NET Core
Stormpath
 
Reducing Microservice Complexity with Kafka and Reactive Streams
Reducing Microservice Complexity with Kafka and Reactive StreamsReducing Microservice Complexity with Kafka and Reactive Streams
Reducing Microservice Complexity with Kafka and Reactive Streams
jimriecken
 
Reactive programming with examples
Reactive programming with examplesReactive programming with examples
Reactive programming with examples
Peter Lawrey
 
Java 8 Stream API and RxJava Comparison
Java 8 Stream API and RxJava ComparisonJava 8 Stream API and RxJava Comparison
Java 8 Stream API and RxJava Comparison
José Paumard
 
RestMQ - HTTP/Redis based Message Queue
RestMQ - HTTP/Redis based Message QueueRestMQ - HTTP/Redis based Message Queue
RestMQ - HTTP/Redis based Message Queue
Gleicon Moraes
 
Reactive Programming in Java 8 with Rx-Java
Reactive Programming in Java 8 with Rx-JavaReactive Programming in Java 8 with Rx-Java
Reactive Programming in Java 8 with Rx-Java
Kasun Indrasiri
 
Introduction to Functional Reactive Programming
Introduction to Functional Reactive ProgrammingIntroduction to Functional Reactive Programming
Introduction to Functional Reactive Programming
Edward Amsden
 

Viewers also liked (18)

Microsoft Big Data @ SQLUG 2013
Microsoft Big Data @ SQLUG 2013Microsoft Big Data @ SQLUG 2013
Microsoft Big Data @ SQLUG 2013
 
a real-time architecture using Hadoop and Storm at Devoxx
a real-time architecture using Hadoop and Storm at Devoxxa real-time architecture using Hadoop and Storm at Devoxx
a real-time architecture using Hadoop and Storm at Devoxx
 
Reactor 3.0, a reactive foundation for java 8 and Spring
Reactor 3.0, a reactive foundation for java 8 and SpringReactor 3.0, a reactive foundation for java 8 and Spring
Reactor 3.0, a reactive foundation for java 8 and Spring
 
A real-time (lambda) architecture using Hadoop & Storm (NoSQL Matters Cologne...
A real-time (lambda) architecture using Hadoop & Storm (NoSQL Matters Cologne...A real-time (lambda) architecture using Hadoop & Storm (NoSQL Matters Cologne...
A real-time (lambda) architecture using Hadoop & Storm (NoSQL Matters Cologne...
 
Rethink Async With RXJS
Rethink Async With RXJSRethink Async With RXJS
Rethink Async With RXJS
 
Virdata: lessons learned from the Internet of Things and M2M Cloud Services @...
Virdata: lessons learned from the Internet of Things and M2M Cloud Services @...Virdata: lessons learned from the Internet of Things and M2M Cloud Services @...
Virdata: lessons learned from the Internet of Things and M2M Cloud Services @...
 
Functional Reactive Programming in the Netflix API
Functional Reactive Programming in the Netflix APIFunctional Reactive Programming in the Netflix API
Functional Reactive Programming in the Netflix API
 
A real time architecture using Hadoop and Storm @ FOSDEM 2013
A real time architecture using Hadoop and Storm @ FOSDEM 2013A real time architecture using Hadoop and Storm @ FOSDEM 2013
A real time architecture using Hadoop and Storm @ FOSDEM 2013
 
Twitter Big Data
Twitter Big DataTwitter Big Data
Twitter Big Data
 
Supercharged java 8 : with cyclops-react
Supercharged java 8 : with cyclops-reactSupercharged java 8 : with cyclops-react
Supercharged java 8 : with cyclops-react
 
Practical Akka HTTP - introduction
Practical Akka HTTP - introductionPractical Akka HTTP - introduction
Practical Akka HTTP - introduction
 
Building Beautiful REST APIs with ASP.NET Core
Building Beautiful REST APIs with ASP.NET CoreBuilding Beautiful REST APIs with ASP.NET Core
Building Beautiful REST APIs with ASP.NET Core
 
Reducing Microservice Complexity with Kafka and Reactive Streams
Reducing Microservice Complexity with Kafka and Reactive StreamsReducing Microservice Complexity with Kafka and Reactive Streams
Reducing Microservice Complexity with Kafka and Reactive Streams
 
Reactive programming with examples
Reactive programming with examplesReactive programming with examples
Reactive programming with examples
 
Java 8 Stream API and RxJava Comparison
Java 8 Stream API and RxJava ComparisonJava 8 Stream API and RxJava Comparison
Java 8 Stream API and RxJava Comparison
 
RestMQ - HTTP/Redis based Message Queue
RestMQ - HTTP/Redis based Message QueueRestMQ - HTTP/Redis based Message Queue
RestMQ - HTTP/Redis based Message Queue
 
Reactive Programming in Java 8 with Rx-Java
Reactive Programming in Java 8 with Rx-JavaReactive Programming in Java 8 with Rx-Java
Reactive Programming in Java 8 with Rx-Java
 
Introduction to Functional Reactive Programming
Introduction to Functional Reactive ProgrammingIntroduction to Functional Reactive Programming
Introduction to Functional Reactive Programming
 

More from C4Media

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
C4Media
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy Mobile
C4Media
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020
C4Media
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java Applications
C4Media
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No Keeper
C4Media
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like Owners
C4Media
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
C4Media
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate Guide
C4Media
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CD
C4Media
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine Learning
C4Media
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at Speed
C4Media
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep Systems
C4Media
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.js
C4Media
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly Compiler
C4Media
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix Scale
C4Media
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's Edge
C4Media
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home Everywhere
C4Media
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing For
C4Media
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data Engineering
C4Media
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
C4Media
 

More from C4Media (20)

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy Mobile
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java Applications
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No Keeper
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like Owners
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate Guide
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CD
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine Learning
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at Speed
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep Systems
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.js
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly Compiler
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix Scale
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's Edge
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home Everywhere
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing For
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data Engineering
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
 

Recently uploaded

High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
aslasdfmkhan4750
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
digitalxplive
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
ssuser1915fe1
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
Kief Morris
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
313mohammedarshad
 
Best Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdfBest Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdf
Tatiana Al-Chueyr
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
Google Developer Group - Harare
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
Anant Gupta
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
Adam Dunkels
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
Axel Rennoch
 
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Torry Harris
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Networks
 
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyyActive Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
RaminGhanbari2
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
HackersList
 

Recently uploaded (20)

High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
[Talk] Moving Beyond Spaghetti Infrastructure [AOTB] 2024-07-04.pdf
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
 
Best Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdfBest Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdf
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
How to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptxHow to Build a Profitable IoT Product.pptx
How to Build a Profitable IoT Product.pptx
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
 
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...Evolution of iPaaS - simplify IT workloads to provide a unified view of  data...
Evolution of iPaaS - simplify IT workloads to provide a unified view of data...
 
IPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite SolutionIPLOOK Remote-Sensing Satellite Solution
IPLOOK Remote-Sensing Satellite Solution
 
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyyActive Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
Active Inference is a veryyyyyyyyyyyyyyyyyyyyyyyy
 
How Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdfHow Social Media Hackers Help You to See Your Wife's Message.pdf
How Social Media Hackers Help You to See Your Wife's Message.pdf
 

Reactive REST

  • 2. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /netflix-reactive-rest InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month
  • 3. Presented at QCon San Francisco www.qconsf.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  • 4. + The is the story… …of how we freed REST from HTTP... …and discovered it was even more Powerful than we thought it was.
  • 6. + Question What’s the most successful distributed system designed for information browsing?
  • 9. + REST  Unique ID for every resource  Idempotent VERBS  GET  PUT
  • 11. + REST + HTTP: Two great tastes  Unique ID for every resource (URL)  Idempotent VERBS    GET PUT Cache Control Headers for Invalidation
  • 12. + Netflix is a web application… Why not use the browser cache for our data?
  • 13. + Why not the Browser Cache? http://netflix.com/videos/234234 http://netflix.com/videos/234234/rating http://netflix.com/videos/54325 http://netflix.com/videos/54325/rating http://netflix.com/videos/12356/name http://netflix.com/videos/12356/rating http://netflix.com/videos/876456/name http://netflix.com/videos/876456/rating Too many HTTP requests!
  • 14. + Challenge Can we build a high-performance REST API for our data?
  • 15. + Introducing Falkor  RESTful Query API for Data  Coherent, Transparent, Managed Cache  Efficient bulk data transfer over HTTP  Query Optimization
  • 17. + Netflix Domain Model http://netflix.com/user { videoLists: [ [ { name: “Die hard”, rating: 4.0 }, // more titles ], [ { name: “Die hard”, rating: 4.0 }, // more titles ], // more lists… ] }
  • 18. + RESTful Data Access API var userModel = { videoLists: { “0”: { “name”: “Thrillers”, “0”: { id: 2654, name: “Die hard”, rating: 4.0 }, // more titles length: 74, } } } userModel[‘videoLists’][0][0][‘name’] [‘videoLists’, 0, 0, ‘name’]
  • 19. + Building a Proxy for a Remote Model // Create a proxy or the current user’s domain model var remoteModel = new RemotePathEvaluator(“http://netflix.com/user”); Server Path Evaluator Falkor Server Remote Path Evaluator
  • 20. + Retrieving Data from the Server // Retrieve the name of the first title in the first genre list. var remoteModel. get([“videoLists”,0,0,”name”]). forEach(function(pathBoundValue) { console.log(pathBoundValue); }); [“videoLists”,0,0,”name”] { path: [“videoLists”,0,0,”name”], value: “Die Hard” } netflix.com/user
  • 21. + Retrieving Sub Graph // Retrieve the names of the first 10 title in the first // 10 genre lists. var remoteModel. get([“videoLists”,{from:0,to:9},{from:0,to:9},”name”]). forEach(function(pathBoundValue) { console.log(pathBoundValue); }); > > > > > {path: [“videoLists”,0,0,”name”], {path: [“videoLists”,0,1,”name”], // snip {path: [“videoLists”,9,8,”name”], {path: [“videoLists”,9,9,”name”], value: “Die Hard”} value: “Amelie”} value: “The New Guys”} value: “Animal House”}
  • 22. + Caching Data // Create a proxy or the current user’s domain model var remoteModel = new RemotePathEvaluator(“http://netflix.com/user”); // Create a path evaluator for a local, in-memory cache. // Cache uses LRU to ensure size stays < 10000. var localModel = new SizedMemoryPathEvaluator(10000, {}); // Create a cached path evaluator, using the remoteModel as the // source and the in-memory model as the cache. var cachedModel = remoteModel.cache(localModel);
  • 23. + Caching Data on the Client Client Memory Server Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator Cached Path Evaluator
  • 24. + Retrieving Cached Data var cachedModel. get([“videoLists”,0,0,”name”]). forEach(function(pathBoundValue) { console.log(pathBoundValue.value); }); [“videoLists”,0,0,”name”] { path: [“videoLists”,0,0,”name”], value: “Die Hard” } Local Cache Cache [“videoLists”,0,0,”name”]
  • 25. + Retrieving Cached Data var cachedModel. get([“videoLists”,0,0,”name”]). forEach(function(pathBoundValue) { console.log(pathBoundValue.value); }); [“videoLists”,0,0,”name”] { path: [“videoLists”,0,0,”name”], value: “Die Hard” } Local Cache Cache
  • 26. + Query Optimization // Query for another video property can be optimized! var cachedModel. get([“videoLists”,0,0,”rating”]). forEach(function(pathBoundValue) { console.log(pathBoundValue.value); }); Query is transparently optimized! [“videoLists”,0,0,”rating”] { path: [“videoLists”,0,0,”rating”], value: “Die Hard” } Local Cache Cache [“videos”,234234,”rating”]
  • 28. + Features  Efficient data transfer over HTTP  Cache Coherence  Cache Transparency  Query Optimization
  • 29. + One Resource/One Domain Model http://netflix.com/videos/234234/name http://netflix.com/videos/234234/rating http://netflix.com/videos/54325/name http://netflix.com/user?path=[“videos”,234,”name”]&path=… http://netflix.com/videos/54325/rating http://netflix.com/videos/12356/name http://netflix.com/videos/12356/rating http://netflix.com/videos/876456/name http://netflix.com/videos/876456/rating
  • 30. + Features  Efficient data transfer over HTTP  Cache Coherence  Cache Transparency  Query Optimization
  • 31. + Netflix Domain Model http://netflix.com/user { videoLists: { “0”: { “name”: “Thrillers”, “0”: { name: “Die hard”, rating: 4.0 }, // more titles length: 74, }, “1”: { “name”: “Action Movies”, “0”: { name: “Die hard”, rating: 4.0 }, // more titles length: 74 }, // more lists… length: 25 } }
  • 32. + What’s wrong with JSON? “videoLists” “0” “0” “name” “Die Hard” “1” “0” Same movie appears twice in the same message! “name” “Die Hard”
  • 33. + The Problem How to model a graph in JSON?
  • 35. + JSONG  Graph representation language in JSON  Two types: 1. Maps 2. Values (includes Arrays)
  • 37. + JSON http://netflix.com/user { videoLists: { “0”: { “name”: “Thrillers”, “0”: { id: 2654, name: “Die hard”, rating: 4.0 }, // more titles length: 74, }, “1”: { “name”: “Action Movies”, “0”: { id: 2654, name: “Die hard”, rating: 4.0 }, // more titles length: 74 }, // more lists… length: 25 } }
  • 38. + JSONG http://netflix.com/user { videoLists: { “0”: { “name”: “Thrillers”, “0”: [“videos”, 2654], // more titles length: 74, }, “1”: { “name”: “Action Movies”, “0”: [“videos”, 2654], // more titles length: 74 }, // more lists… length: 25 }, videos: { 2654: { name: “Die hard”, rating: 4.0 } } }
  • 39. + Okay so we’ve got a graph… …now we need a convenient API for querying data.
  • 40. + Hierarchical API var userModel = { videoLists: { “0”: { “name”: “Thrillers”, “0”: { id: 2654, name: “Die hard”, rating: 4.0 }, // more titles length: 74, } } } JSONG Path userModel[‘videoLists’][0][0][‘name’] [‘videoLists’, 0, 0, ‘name’]
  • 41. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } [“videoLists”,0,0,”name”]
  • 42. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } [“videoLists”,0,0,”name”]
  • 43. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } [“videoLists”,0,0,”name”]
  • 44. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } [“videoLists”,0,0,”name”]
  • 45. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } Array found before last key evaluated… [“videoLists”,0,0,”name”]
  • 46. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } Array must be a path! [“videoLists”,0,0,”name”]
  • 47. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } Array must be a path! [“videos”,234,”name”]
  • 48. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } Rewrite path, and start evaluating from top node. [“videos”,234,”name”]
  • 49. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } [“videos”,234,”name”]
  • 50. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } [“videos”,234,”name”]
  • 51. + JSONG Path Evaluation var netflixUser = { videoLists: { “0”: { name: “Thrillers”, “0”: [“videos”,234], // more videos… length: 75 }, “1”: { name: “Action Movies”, “0”: [“videos”,234], // more videos… length: 75 }, // more lists… length: 15 }, videos: { 234: {name: “Die Hard”, rating:5} } } [“videos”,234,”name”]
  • 52. + Okay… …how do we make this process transparent to the developer?
  • 53. + Features  Efficient data transfer over HTTP  Cache Coherence  Cache Transparency  Query Optimization
  • 54. + Path Evaluator Path Evaluator get(Observable<Path>): Observable<PathBoundValue> set(Observable<PathBoundValue>): Observable<PathBoundValue> delete(Observable<Path>): Observable<PathBoundValue>
  • 55. + Observable  Object  Open that represents stream of data Source Reactive Extensions Library  Ported to  Javascript  .NET C  Java (Netflix)
  • 56. + Observable.forEach // “subscribe” var subscription = remoteModel.get([“videoLists”,0,0,”name”]). forEach( event => console.log(event),error error => console.error(error), () => console.log(“done”)); // “unsubscribe” subscription.dispose(); optional
  • 57. + Path Evaluator  Proxy for a JSONG Model  Idempotent Operations:   Set   Get Delete Composable
  • 59. + Accessing Data on the Client // Create a Proxy path evaluator for the server JSONG domain model var remoteModel = new RemotePathEvaluator(“http://netflix.com/tvui/user”);`
  • 60. + Accessing Data on the Client // Create a Proxy path evaluator for the server JSONG domain model var remoteModel = new RemotePathEvaluator(“http://netflix.com/tvui/user”);` Server Path Evaluator Remote Path Evaluator
  • 61. + Accessing Data on the Client // Create a Proxy path evaluator for the server JSONG domain model var remoteModel = new RemotePathEvaluator(“http://netflix.com/tvui/user”); remoteModel.get([“videoLists”,{to:5},{to:10},”name”]). forEach(function(pathBoundValue) { /* do something */ }); Server Path Evaluator [videoLists,{to:5},{to:10},’name’] Remote Path Evaluator
  • 62. + Accessing Data on the Client // Create a Proxy path evaluator for the server JSONG domain model var remoteModel = new RemotePathEvaluator(“http://netflix.com/tvui/user”); remoteModel.get([“videoLists”,{to:5},{to:10},”name”]). forEach(function(pathBoundValue) { /* do something */ }); Server Path Evaluator { path: [“videoLists”,0,0,”name”], value: {…} } { path: [“videoLists”,0,1,”name”], value: {…} } … { path: [“videoLists”,5,10,”name”], value: {…} } Remote Path Evaluator
  • 63. + Caching Data on the Client // Create a Proxy path evaluator for the server JSONG domain model var remoteModel = new RemotePathEvaluator(“http://netflix.com/tvui/user”); // Create a path evaluator for a local, in-memory cache. // Cache stores the fragments of the remote model and stays within // size of 10000. var localModelCache = new SizedMemoryPathEvaluator(10000, {}); Client Memory Server Path Evaluator Sized Memory Path Evaluator Remote Path Evaluator
  • 64. + Sized Memory Path Evaluator  Works like a web browser cache!  Caches resources by path  Removes least-recently used resources
  • 65. + Caching Data on the Client // Create a Proxy path evaluator for the server JSONG domain model var remoteModel = new RemotePathEvaluator(“http://netflix.com/tvui/user”); // Create a path evaluator for a local, in-memory cache. // Cache stores the fragments of the remote model and stays within // size of 10000. var localModel = new SizedMemoryPathEvaluator(10000, {}); // Create a cached path evaluator, using the remoteModel as the // source and the in-memory model as the cache. var cachedModel = remoteModel.cache(localModel);
  • 66. + Caching Data on the Client Client Memory Server Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator Cached Path Evaluator [videoLists,{to:5},{to:10},’name’]
  • 67. + Caching Data on the Client Client Memory Server Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator Cached Path Evaluator
  • 68. + Caching Data on the Client Client Memory Server Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator [videoLists,{to:5},{to:10},’name’]` Cached Path Evaluator
  • 69. + Caching Data on the Client Client Memory Server Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator Nothing Cached Path Evaluator
  • 70. + Caching Data on the Client Client Memory Server Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator [videoLists,{to:5},{to:10},’name’] Cached Path Evaluator
  • 71. + Caching Data on the Client Client Memory Server Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator { path: [“videoLists”,0,0,”name”], value: {…} } { path: [“videoLists”,0,1,”name”], value: {…} } … { path: [“videoLists”,5,10,”name”], value: {…} } Cached Path Evaluator
  • 72. + Caching Data on the Client Client Memory Server Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator { path: [“videoLists”,0,0,”name”], value: {…} } { path: [“videoLists”,0,1,”name”], value: {…} } … { path: [“videoLists”,5,10,”name”], value: {…} } Cached Path Evaluator
  • 73. + Caching Data on the Client Client Memory var netflixMember = { videoLists: { name: “Action Movies”, “0”: [“lists”, 54354334], // more lists… “length: 15 }, “videos”: { “34123432”: { … } } } Server Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator Cached Path Evaluator { path: [“videoLists”,0,0,”name”], value: {…} } { path: [“videoLists”,0,1,”name”], value: {…} } … { path: [“videoLists”,5,10,”name”], value: {…} }
  • 74. + Features  Efficient data transfer over HTTP  Cache Coherence  Cache Transparency  Query Optimization
  • 76. + Query Optimization Client Memory Server Path Evaluator var netflixMember = { videoLists: { name: “Action Movies”, “0”: [“lists”, 54354334], // more lists… “length: 15 }, “videos”: { “34123432”: { … } } } TV Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator Cached Path Evaluator [videoLists,1,3,’rating’]
  • 77. + Query Optimization Client Memory Server Path Evaluator var netflixMember = { videoLists: { name: “Action Movies”, “0”: [“lists”, 54354334], // more lists… “length: 15 }, “videos”: { “34123432”: { … } } } TV Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator Cached Path Evaluator [videoLists,1,3,’rating’]
  • 78. + Query Optimization Client Memory Server Path Evaluator var netflixMember = { videoLists: { name: “Action Movies”, “0”: [“lists”, 54354334], // more lists… “length: 15 }, “videos”: { “34123432”: { … } } } TV Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator [videoLists,1,3,’rating’] Cached Path Evaluator [videoLists,1,3,’rating’]
  • 79. + Query Optimization Client Memory Server Path Evaluator var netflixMember = { videoLists: { name: “Action Movies”, “0”: [“lists”, 54354334], // more lists… “length: 15 }, “videos”: { “34123432”: { … } } } TV Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator {path: [videoLists,1], value: [lists,23432] } {path: [lists,23432,0], value: [movies,234234] } Cached Path Evaluator [videoLists,1,3,’rating’]
  • 80. + Query Optimization Client Memory Server Path Evaluator var netflixMember = { videoLists: { name: “Action Movies”, “0”: [“lists”, 54354334], // more lists… “length: 15 }, “videos”: { “34123432”: { … } } } TV Path Evaluator Falkor Server Sized Memory Path Evaluator Remote Path Evaluator [movies,234234,rating] Cached Path Evaluator Path optimized!!
  • 81. + Query Optimization  Path partially evaluated in the cache  Path is rewritten and optimized before it is sent to server
  • 82. + Query Optimization Rocks! Same API for local and remote data!
  • 83. + Features  Efficient data transfer over HTTP  Cache Coherence  Cache Transparency  Query Optimization
  • 84. + Roadmap  Currently Internal Software  Defensive Patent Filed  Plan to open-source 2014
  • 86. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/netflixreactive-rest