SlideShare a Scribd company logo
1Rest.li: RESTful Service Architecture at Scale
SATURN 2018
14th Annual SEI Architecture Technology User Network Conference
MAY 7–10, 2018 | PLANO, TEXAS
Rest.li: RESTful Service Architecture at Scale
By: Min Chen (LinkedIn)
Email: mnchen@linkedin.com
Profile: www.linkedin.com/in/minchen07
2Rest.li: RESTful Service Architecture at Scale
A Brief History
Monolithic/Layered
Complex Orchestration
Complex Dependency from
Service Explosion
Incompatibilities
Slow Rollout
3Rest.li: RESTful Service Architecture at Scale
Paradigm Shift
Microservices
Fine-grained Service Boundary
Continuous Delivery
Standard Service Access Pattern
Backward-compatible service
evolution
4Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Challenges
 Standard way to describe service in service-oriented architecture.
 Standard access pattern by multi-language diverse clients.
 Asynchronous and scalable to handle heavy service load.
 Fast and easy experience even for non-REST expert developers.
 Support long-term service evolution and growth.
5Rest.li: RESTful Service Architecture at Scale
Monolithic
Data Model
Centric
Microservices
6Rest.li: RESTful Service Architecture at Scale
Rest.li is an open source REST framework for
building robust, scalable RESTful architectures
using asynchronous, non-blocking IO. It fills a
niche for applying RESTful principles at scale with
an end-to-end developer workflow that
promotes clean REST practices.
7Rest.li: RESTful Service Architecture at Scale
8Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Clean REST Practice
 End-to-End Developer Workflow
 Consistent data modeling
 Uniform interface design
 Type-safe client bindings
 Online API Hub
 API documentation
 API service catalog
 Compatibility Checker Enabled API Evolution
9Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Developer Flow
10Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Consistent Data Modeling
 Data schema in PDSC (Pegasus Data Schema)
 PDSC is inspired by Apache Avro specification.
 JSON serialization format
 Rich type support:
 Named: Record, Enum, Fixed, Typeref
 Unnamed: Union, Array, Map
 Custom type
 Custom validator and coercer
11Rest.li: RESTful Service Architecture at Scale
Data Schema Example
12Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Resource Class
Resource Class
Resource Class
3
Record Template
2
Generated
Developer Flow
13Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Uniform Interface Design
 Standardized RESTful resource pattern
 Annotated resource identifier
 Uniform service interface
 CRUD
 Finder
 Action
 BATCH
 REST specification IDL (Interface Description Language)
14Rest.li: RESTful Service Architecture at Scale
Rest.li Resource Patterns
Resource Type Semantics Interface Base Template
Collection
(@RestLiCollection)
Key/value map of
entites
CollectionResource
KeyValueResource
CollectionResourceTemplate
Simple
(@RestLiSimple)
Singleton in scope SimpleResource SimpleResourceTemplate
Association
(@RestLiAssociation)
Relationships
between entities
AssociationResource AssociationResourceTemplate
Sub-Resource Child resources All Above All Above
Action
(@RestliActions)
Flexible for non-
standard behavior
N/A N/A
15Rest.li: RESTful Service Architecture at Scale
Rest.li Resource Method (Basic)
Rest.li Request HTTP Method URI Example Entity Body
(V – Data Model)
CREATE POST /statuses V
GET GET /statuses/<id>
GET_ALL GET /statuses
UPDATE PUT /statuses/<id> V
PARTIAL_UPDATE POST /statuses/<id> Patch of V
DELETE DELETE /statuses/<id>
FINDER GET /statuses?q=search&keywords=li
nkedin
ACTION POST /statuses?action=purge Dynamic Record
16Rest.li: RESTful Service Architecture at Scale
Rest.li Resource Method (Batch)
Rest.li Request HTTP Method URI Example Entity Body
BATCH_CREATE POST /statuses {“elements”: […]}
BATCH_GET GET /statuses?ids=List(1,2,3)
BATCH_UPDATE PUT /statuses?ids=List(1,2,3) {“entities”: {“1”: {…}, “2” :
{…}}
BATCH_PARTIAL_UPDATE POST /statuses?ids=List(1,2,3) {“entities”: {“1”: {“patch”:
{…}}, “2” : {“patch”: {…}}
BATCH_DELETE DELETE /statuses?ids=List(1,2,3)
17Rest.li: RESTful Service Architecture at Scale
What Server Code I Need to Write
18Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Resource Class
Resource Class
Resource Class
3
Record Template
2
Generated
4
REST IDL
Generated
Developer Flow
19Rest.li: RESTful Service Architecture at Scale
REST Spec IDL
Service
Metadata
PDSC
curl -X OPTIONS <ServiceEndpoint>
20Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Server
5
Resource Class
Resource Class
Resource Class
3
Client
7
Record Template
2
Generated
4
REST IDL
Generated
6
Type-Safe
RequestBuilders
Generated
Developer Flow
21Rest.li: RESTful Service Architecture at Scale
What Client Code I Need to Write
Photo newPhoto = new Photo()
.setTitle("NewPhoto")
.setFormat(PhotoFormats.PNG)
.setExif(new Exif().setLocation(newLocation));
Request<?> createReq = new
PhotosRequestBuilder().create().input(newPhoto).build()build();
POST /photos -H "Content-Type: application/json”
-d {"title":"NewPhoto", "format": "PNG", "exif":{"location":...}}
Strong-typed data model
(auto-generated)
Type-safe request builders
(auto-generated)
22Rest.li: RESTful Service Architecture at Scale
Data Schema
1
Server
5
Resource Class
Resource Class
Resource Class
3
Client
7
Record Template
2
Generated
4
REST IDL
Generated
Human Readable
Documentation
Generated
8
6
Type-Safe
RequestBuilders
Generated
Developer Flow
23Rest.li: RESTful Service Architecture at Scale
Rest.li API Hub
• Lucene Powered full-text search
• Data model and interface
documentation
• Sample request generation
• Live interaction with real REST
endpoint
24Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Backward Compatibility
 Build-time Compatibility Checker
 equivalent - No changes to Resources or Data model.
 backwards - Changes considered backwards.
 ignore - Log the changes but allow changes to pass.
 off - The compatibility checker will not be run at all.
25Rest.li: RESTful Service Architecture at Scale
26Rest.li: RESTful Service Architecture at Scale
Rest.li Server Stack
Rest.li Data Layer and RESTful Operations.li Inversion-of-Control
D2 Dynamic Discovery and Load Balancing Fault tolerance
R2 Request/Response and Network
Communication
Async and
Non-blocking
27Rest.li: RESTful Service Architecture at Scale
Request Response (R2)
Non-Streaming Streaming
Message Communication Abstraction over HTTP
28Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Dynamic Discovery (D2)
 Apache Zookeeper (Announce and Discover)
 Horizontal Scaling
 Services, Servers (ephemeral nodes) and Clusters
 Client side load balancing
 Fault tolerance
 Backup request (tail-latency)
29Rest.li: RESTful Service Architecture at Scale
Rest.li Client
Implement and Interpret Rest.li Protocol
(Pluggable Transport Client)
30Rest.li: RESTful Service Architecture at Scale
Rest.li Client/Server Data Flow
31Rest.li: RESTful Service Architecture at Scale
Open and Extensible
 Filter Chain pipeline
 Polyglot transport client
 Custom Codec
(JSON, PSON, XML, etc)
32Rest.li: RESTful Service Architecture at Scale
Rich Eco-Systems
C#
Java
Android
Java
Objective-C
Swift
Python
33Rest.li: RESTful Service Architecture at Scale
Deco
Describe WHAT data you want, let Deco worry about
HOW to fetch them
posts
profiles
Companies
34Rest.li: RESTful Service Architecture at Scale
Deco Concept
/posts
/companies
/profiles
35Rest.li: RESTful Service Architecture at Scale
Deco Concept
deco://posts/123?projection=(title, content, author~(firstName,
lastName, company~(companyName)))
 Projection
 Decoration
 Resolution
Normalized Domain Modeling
36Rest.li: RESTful Service Architecture at Scale
37Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Asynchronous Challenges
 Callback Hell
 Thread Safety
 Indeterminism
 Debugging
 How to achieve all-async?
38Rest.li: RESTful Service Architecture at Scale
SATURN 2018
ParSeq Concept
Promise<T> Represents state of an asynchronous operation, also
a container for an asynchronous operation result.
Task<T> Main abstraction unit representing an asynchronous
operation.
Task Callable Main ingredient of Task<T>. It starts asynchronous
operation and returns a Promise<T> representing
the operation state.
Engine Harness to run a Task<T>.
39Rest.li: RESTful Service Architecture at Scale
SATURN 2018
ParSeq Programming Model
 Declarative and Functional (What not How)
 Task composition, chaining, transformation and error handling
 Task.async (task creation)
 Task.par (parallel composition)
 task.andThen (sequential composition)
 task.map and task.flatMap (transformation)
 task.recover and task.withTimeout (error handling)
 Ends up with one Task (execution plan) submitted to ParSeq engine to
execute
40Rest.li: RESTful Service Architecture at Scale
SATURN 2018
ParSeq Execution Model
 All task Callables are executed sequentially, such that Callable of a
previous Task in a sequence “happens before” (in Java Memory Model
sense) Callable of a next Task.
 No need to be concerned about:
 Synchronized, locks, volatile, atomic, etc
 Immutability
 3rd party libraries
 Thinking about Java Memory Model
 Thread safety does not mean atomicity
Task<Response<T>> task =
Task.async(name, () -> sendRequest(request, requestContext));
Callable
41Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Seamless Integration with Rest.li
 Rest.li supports resource method returning Task.
 ParSeqRestClient provides API to create Task from a rest.li request
@RestMethod.Get
public Task<Greeting> get(final Long id) {
// rest.li resource implementation
}
public <T> Task<Response<T>> createTask(final Request<T>
request, final RequestContext requestContext) {
}
42Rest.li: RESTful Service Architecture at Scale
ParSeq Trace Visualization
Power Tool for Debugging
43Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Multi-Language Support
C#
Java
Android
Java
Objective-C
Swift
Python
44Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Comparison Matrix
Rest.li Jersey DropWizard Olingo (Odata)
Data Schema yes no no yes
Data Model Auto Manual Manual Manual
Server Coding
URI Pattern Uniform Non-Uniform Non-Uniform Non-Uniform
Batch Support Yes No No Yes
Service Metadata Yes No No Yes
Dynamic
Discovery/Load
balancing
Yes No No No
Documentation Yes No No No
Service Catalog Yes No No No
Graph Query Yes No No Yes
45Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Future of Rest.li
 Object Streaming in Rest.li Layer
 Unstructured data support
 Multi-language support enhancement
 Go
 Cross-language test suites
 Open source Deco
 ParSeq
 Collection API
 ParSeq Trace Analytics
46Rest.li: RESTful Service Architecture at Scale
SATURN 2018
Useful Links
 Rest.li (https://github.com/linkedin/rest.li)
 Rest.li API Hub (https://github.com/linkedin/rest.li-api-hub)
 ParSeq (https://github.com/linkedin/parseq)

More Related Content

What's hot

Redis and it's data types
Redis and it's data typesRedis and it's data types
Redis and it's data types
Aniruddha Chakrabarti
 
Json Tutorial
Json TutorialJson Tutorial
Json Tutorial
Napendra Singh
 
Introduction to redis
Introduction to redisIntroduction to redis
Introduction to redis
Tanu Siwag
 
Redis overview for Software Architecture Forum
Redis overview for Software Architecture ForumRedis overview for Software Architecture Forum
Redis overview for Software Architecture Forum
Christopher Spring
 
From a monolith to microservices + REST: The evolution of LinkedIn's architec...
From a monolith to microservices + REST: The evolution of LinkedIn's architec...From a monolith to microservices + REST: The evolution of LinkedIn's architec...
From a monolith to microservices + REST: The evolution of LinkedIn's architec...
Karan Parikh
 
What is REST API? REST API Concepts and Examples | Edureka
What is REST API? REST API Concepts and Examples | EdurekaWhat is REST API? REST API Concepts and Examples | Edureka
What is REST API? REST API Concepts and Examples | Edureka
Edureka!
 
Deep Dive into Cassandra
Deep Dive into CassandraDeep Dive into Cassandra
Deep Dive into Cassandra
Brent Theisen
 
MongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective IndexingMongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB
 
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
Amazon Web Services Korea
 
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Edureka!
 
Intro to GraphQL
 Intro to GraphQL Intro to GraphQL
Intro to GraphQL
Rakuten Group, Inc.
 
Graphql
GraphqlGraphql
Graphql
Niv Ben David
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
Maarten Smeets
 
Rest API
Rest APIRest API
Gatling Performance Workshop
Gatling Performance WorkshopGatling Performance Workshop
Gatling Performance Workshop
Sai Krishna
 
Domain Driven Design and Hexagonal Architecture with Rails
Domain Driven Design and Hexagonal Architecture with RailsDomain Driven Design and Hexagonal Architecture with Rails
Domain Driven Design and Hexagonal Architecture with Rails
Declan Whelan
 
LMAX Architecture
LMAX ArchitectureLMAX Architecture
LMAX Architecture
Stephan Schmidt
 
KrakenD API Gateway
KrakenD API GatewayKrakenD API Gateway
KrakenD API Gateway
Albert Lombarte
 
REST vs. GraphQL: Critical Look
REST vs. GraphQL: Critical LookREST vs. GraphQL: Critical Look
REST vs. GraphQL: Critical Look
Nordic APIs
 
Gradle - the Enterprise Automation Tool
Gradle  - the Enterprise Automation ToolGradle  - the Enterprise Automation Tool
Gradle - the Enterprise Automation Tool
Izzet Mustafaiev
 

What's hot (20)

Redis and it's data types
Redis and it's data typesRedis and it's data types
Redis and it's data types
 
Json Tutorial
Json TutorialJson Tutorial
Json Tutorial
 
Introduction to redis
Introduction to redisIntroduction to redis
Introduction to redis
 
Redis overview for Software Architecture Forum
Redis overview for Software Architecture ForumRedis overview for Software Architecture Forum
Redis overview for Software Architecture Forum
 
From a monolith to microservices + REST: The evolution of LinkedIn's architec...
From a monolith to microservices + REST: The evolution of LinkedIn's architec...From a monolith to microservices + REST: The evolution of LinkedIn's architec...
From a monolith to microservices + REST: The evolution of LinkedIn's architec...
 
What is REST API? REST API Concepts and Examples | Edureka
What is REST API? REST API Concepts and Examples | EdurekaWhat is REST API? REST API Concepts and Examples | Edureka
What is REST API? REST API Concepts and Examples | Edureka
 
Deep Dive into Cassandra
Deep Dive into CassandraDeep Dive into Cassandra
Deep Dive into Cassandra
 
MongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective IndexingMongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
MongoDB .local Toronto 2019: Tips and Tricks for Effective Indexing
 
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
서버리스 앱 배포 자동화 (김필중, AWS 솔루션즈 아키텍트) :: AWS DevDay2018
 
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
Elasticsearch Tutorial | Getting Started with Elasticsearch | ELK Stack Train...
 
Intro to GraphQL
 Intro to GraphQL Intro to GraphQL
Intro to GraphQL
 
Graphql
GraphqlGraphql
Graphql
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Rest API
Rest APIRest API
Rest API
 
Gatling Performance Workshop
Gatling Performance WorkshopGatling Performance Workshop
Gatling Performance Workshop
 
Domain Driven Design and Hexagonal Architecture with Rails
Domain Driven Design and Hexagonal Architecture with RailsDomain Driven Design and Hexagonal Architecture with Rails
Domain Driven Design and Hexagonal Architecture with Rails
 
LMAX Architecture
LMAX ArchitectureLMAX Architecture
LMAX Architecture
 
KrakenD API Gateway
KrakenD API GatewayKrakenD API Gateway
KrakenD API Gateway
 
REST vs. GraphQL: Critical Look
REST vs. GraphQL: Critical LookREST vs. GraphQL: Critical Look
REST vs. GraphQL: Critical Look
 
Gradle - the Enterprise Automation Tool
Gradle  - the Enterprise Automation ToolGradle  - the Enterprise Automation Tool
Gradle - the Enterprise Automation Tool
 

Similar to Saturn 2018 rest.li

A COMPARATIVE STUDY BETWEEN GRAPH-QL RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL  RESTFUL SERVICES IN API MANAGEMENT OF S...A COMPARATIVE STUDY BETWEEN GRAPH-QL  RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL RESTFUL SERVICES IN API MANAGEMENT OF S...
Sophia Diaz
 
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
ijwscjournal
 
A Comparative Study between Graph-QL& Restful Services in API Management of S...
A Comparative Study between Graph-QL& Restful Services in API Management of S...A Comparative Study between Graph-QL& Restful Services in API Management of S...
A Comparative Study between Graph-QL& Restful Services in API Management of S...
ijwscjournal
 
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp RESTFUL SERVICES IN API MANAGEMENT ...
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp  RESTFUL SERVICES IN API MANAGEMENT ...A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp  RESTFUL SERVICES IN API MANAGEMENT ...
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp RESTFUL SERVICES IN API MANAGEMENT ...
Amanda Summers
 
Modeling REST API's Behaviour with Text, Graphics or Both?
Modeling REST API's Behaviour with Text, Graphics or Both?Modeling REST API's Behaviour with Text, Graphics or Both?
Modeling REST API's Behaviour with Text, Graphics or Both?
Ana Ivanchikj
 
REST & RESTful APIs: The State of Confusion
REST & RESTful APIs: The State of ConfusionREST & RESTful APIs: The State of Confusion
REST & RESTful APIs: The State of Confusion
Glenn Antoine
 
Rest API Automation with REST Assured
Rest API Automation with REST AssuredRest API Automation with REST Assured
Rest API Automation with REST Assured
TO THE NEW Pvt. Ltd.
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptxTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
kmani5
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).pptTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
HusseinWassof
 
Rest API and Client OM for Developer
Rest API and Client OM for DeveloperRest API and Client OM for Developer
Rest API and Client OM for Developer
InnoTech
 
Unit 2
Unit 2Unit 2
Unit 2
Ravi Kumar
 
ROC for Adaptive Systems
ROC for Adaptive SystemsROC for Adaptive Systems
ROC for Adaptive Systems
Tom Mueck
 
Lab7 paper
Lab7 paperLab7 paper
Toward data driven programming for RESTful Linked Data
Toward data driven programming for RESTful Linked DataToward data driven programming for RESTful Linked Data
Toward data driven programming for RESTful Linked Data
Steffen Stadtmüller
 
Cs 1023 lec 13 web (week 4)
Cs 1023 lec 13 web (week 4)Cs 1023 lec 13 web (week 4)
Cs 1023 lec 13 web (week 4)
stanbridge
 
IRJET- Rest API for E-Commerce Site
IRJET- Rest API for E-Commerce SiteIRJET- Rest API for E-Commerce Site
IRJET- Rest API for E-Commerce Site
IRJET Journal
 
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
apidays
 
The Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data StreamsThe Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data Streams
confluent
 
Representational State Transfer (REST) and HATEOAS
Representational State Transfer (REST) and HATEOASRepresentational State Transfer (REST) and HATEOAS
Representational State Transfer (REST) and HATEOAS
Guy K. Kloss
 
ALM iStack - Application Lifecycle Management using Linked Data
ALM iStack - Application Lifecycle Management using Linked Data ALM iStack - Application Lifecycle Management using Linked Data
ALM iStack - Application Lifecycle Management using Linked Data
Nandana Mihindukulasooriya
 

Similar to Saturn 2018 rest.li (20)

A COMPARATIVE STUDY BETWEEN GRAPH-QL RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL  RESTFUL SERVICES IN API MANAGEMENT OF S...A COMPARATIVE STUDY BETWEEN GRAPH-QL  RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL RESTFUL SERVICES IN API MANAGEMENT OF S...
 
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
A COMPARATIVE STUDY BETWEEN GRAPH-QL& RESTFUL SERVICES IN API MANAGEMENT OF S...
 
A Comparative Study between Graph-QL& Restful Services in API Management of S...
A Comparative Study between Graph-QL& Restful Services in API Management of S...A Comparative Study between Graph-QL& Restful Services in API Management of S...
A Comparative Study between Graph-QL& Restful Services in API Management of S...
 
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp RESTFUL SERVICES IN API MANAGEMENT ...
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp  RESTFUL SERVICES IN API MANAGEMENT ...A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp  RESTFUL SERVICES IN API MANAGEMENT ...
A COMPARATIVE STUDY BETWEEN GRAPH-QL Amp RESTFUL SERVICES IN API MANAGEMENT ...
 
Modeling REST API's Behaviour with Text, Graphics or Both?
Modeling REST API's Behaviour with Text, Graphics or Both?Modeling REST API's Behaviour with Text, Graphics or Both?
Modeling REST API's Behaviour with Text, Graphics or Both?
 
REST & RESTful APIs: The State of Confusion
REST & RESTful APIs: The State of ConfusionREST & RESTful APIs: The State of Confusion
REST & RESTful APIs: The State of Confusion
 
Rest API Automation with REST Assured
Rest API Automation with REST AssuredRest API Automation with REST Assured
Rest API Automation with REST Assured
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptxTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx
 
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).pptTransform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
Transform-to-Smart-ERP-using-Custom-Mobile-Apps.pptx (3).ppt
 
Rest API and Client OM for Developer
Rest API and Client OM for DeveloperRest API and Client OM for Developer
Rest API and Client OM for Developer
 
Unit 2
Unit 2Unit 2
Unit 2
 
ROC for Adaptive Systems
ROC for Adaptive SystemsROC for Adaptive Systems
ROC for Adaptive Systems
 
Lab7 paper
Lab7 paperLab7 paper
Lab7 paper
 
Toward data driven programming for RESTful Linked Data
Toward data driven programming for RESTful Linked DataToward data driven programming for RESTful Linked Data
Toward data driven programming for RESTful Linked Data
 
Cs 1023 lec 13 web (week 4)
Cs 1023 lec 13 web (week 4)Cs 1023 lec 13 web (week 4)
Cs 1023 lec 13 web (week 4)
 
IRJET- Rest API for E-Commerce Site
IRJET- Rest API for E-Commerce SiteIRJET- Rest API for E-Commerce Site
IRJET- Rest API for E-Commerce Site
 
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
apidays Australia 2023 - The Playful Bond Between REST And Data Streams, Warr...
 
The Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data StreamsThe Playful Bond Between REST And Data Streams
The Playful Bond Between REST And Data Streams
 
Representational State Transfer (REST) and HATEOAS
Representational State Transfer (REST) and HATEOASRepresentational State Transfer (REST) and HATEOAS
Representational State Transfer (REST) and HATEOAS
 
ALM iStack - Application Lifecycle Management using Linked Data
ALM iStack - Application Lifecycle Management using Linked Data ALM iStack - Application Lifecycle Management using Linked Data
ALM iStack - Application Lifecycle Management using Linked Data
 

Recently uploaded

Series of visio cisco devices Cisco_Icons.ppt
Series of visio cisco devices Cisco_Icons.pptSeries of visio cisco devices Cisco_Icons.ppt
Series of visio cisco devices Cisco_Icons.ppt
PauloRodrigues104553
 
ACEP Magazine edition 4th launched on 05.06.2024
ACEP Magazine edition 4th launched on 05.06.2024ACEP Magazine edition 4th launched on 05.06.2024
ACEP Magazine edition 4th launched on 05.06.2024
Rahul
 
A review on techniques and modelling methodologies used for checking electrom...
A review on techniques and modelling methodologies used for checking electrom...A review on techniques and modelling methodologies used for checking electrom...
A review on techniques and modelling methodologies used for checking electrom...
nooriasukmaningtyas
 
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
ihlasbinance2003
 
6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024)6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024)
ClaraZara1
 
132/33KV substation case study Presentation
132/33KV substation case study Presentation132/33KV substation case study Presentation
132/33KV substation case study Presentation
kandramariana6
 
Low power architecture of logic gates using adiabatic techniques
Low power architecture of logic gates using adiabatic techniquesLow power architecture of logic gates using adiabatic techniques
Low power architecture of logic gates using adiabatic techniques
nooriasukmaningtyas
 
ACRP 4-09 Risk Assessment Method to Support Modification of Airfield Separat...
ACRP 4-09 Risk Assessment Method to Support Modification of Airfield Separat...ACRP 4-09 Risk Assessment Method to Support Modification of Airfield Separat...
ACRP 4-09 Risk Assessment Method to Support Modification of Airfield Separat...
Mukeshwaran Balu
 
Exception Handling notes in java exception
Exception Handling notes in java exceptionException Handling notes in java exception
Exception Handling notes in java exception
Ratnakar Mikkili
 
2. Operations Strategy in a Global Environment.ppt
2. Operations Strategy in a Global Environment.ppt2. Operations Strategy in a Global Environment.ppt
2. Operations Strategy in a Global Environment.ppt
PuktoonEngr
 
Recycled Concrete Aggregate in Construction Part III
Recycled Concrete Aggregate in Construction Part IIIRecycled Concrete Aggregate in Construction Part III
Recycled Concrete Aggregate in Construction Part III
Aditya Rajan Patra
 
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...
University of Maribor
 
digital fundamental by Thomas L.floydl.pdf
digital fundamental by Thomas L.floydl.pdfdigital fundamental by Thomas L.floydl.pdf
digital fundamental by Thomas L.floydl.pdf
drwaing
 
Properties Railway Sleepers and Test.pptx
Properties Railway Sleepers and Test.pptxProperties Railway Sleepers and Test.pptx
Properties Railway Sleepers and Test.pptx
MDSABBIROJJAMANPAYEL
 
ML Based Model for NIDS MSc Updated Presentation.v2.pptx
ML Based Model for NIDS MSc Updated Presentation.v2.pptxML Based Model for NIDS MSc Updated Presentation.v2.pptx
ML Based Model for NIDS MSc Updated Presentation.v2.pptx
JamalHussainArman
 
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
IJECEIAES
 
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECTCHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
jpsjournal1
 
Modelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdfModelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdf
camseq
 
basic-wireline-operations-course-mahmoud-f-radwan.pdf
basic-wireline-operations-course-mahmoud-f-radwan.pdfbasic-wireline-operations-course-mahmoud-f-radwan.pdf
basic-wireline-operations-course-mahmoud-f-radwan.pdf
NidhalKahouli2
 
Embedded machine learning-based road conditions and driving behavior monitoring
Embedded machine learning-based road conditions and driving behavior monitoringEmbedded machine learning-based road conditions and driving behavior monitoring
Embedded machine learning-based road conditions and driving behavior monitoring
IJECEIAES
 

Recently uploaded (20)

Series of visio cisco devices Cisco_Icons.ppt
Series of visio cisco devices Cisco_Icons.pptSeries of visio cisco devices Cisco_Icons.ppt
Series of visio cisco devices Cisco_Icons.ppt
 
ACEP Magazine edition 4th launched on 05.06.2024
ACEP Magazine edition 4th launched on 05.06.2024ACEP Magazine edition 4th launched on 05.06.2024
ACEP Magazine edition 4th launched on 05.06.2024
 
A review on techniques and modelling methodologies used for checking electrom...
A review on techniques and modelling methodologies used for checking electrom...A review on techniques and modelling methodologies used for checking electrom...
A review on techniques and modelling methodologies used for checking electrom...
 
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
5214-1693458878915-Unit 6 2023 to 2024 academic year assignment (AutoRecovere...
 
6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024)6th International Conference on Machine Learning & Applications (CMLA 2024)
6th International Conference on Machine Learning & Applications (CMLA 2024)
 
132/33KV substation case study Presentation
132/33KV substation case study Presentation132/33KV substation case study Presentation
132/33KV substation case study Presentation
 
Low power architecture of logic gates using adiabatic techniques
Low power architecture of logic gates using adiabatic techniquesLow power architecture of logic gates using adiabatic techniques
Low power architecture of logic gates using adiabatic techniques
 
ACRP 4-09 Risk Assessment Method to Support Modification of Airfield Separat...
ACRP 4-09 Risk Assessment Method to Support Modification of Airfield Separat...ACRP 4-09 Risk Assessment Method to Support Modification of Airfield Separat...
ACRP 4-09 Risk Assessment Method to Support Modification of Airfield Separat...
 
Exception Handling notes in java exception
Exception Handling notes in java exceptionException Handling notes in java exception
Exception Handling notes in java exception
 
2. Operations Strategy in a Global Environment.ppt
2. Operations Strategy in a Global Environment.ppt2. Operations Strategy in a Global Environment.ppt
2. Operations Strategy in a Global Environment.ppt
 
Recycled Concrete Aggregate in Construction Part III
Recycled Concrete Aggregate in Construction Part IIIRecycled Concrete Aggregate in Construction Part III
Recycled Concrete Aggregate in Construction Part III
 
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...
 
digital fundamental by Thomas L.floydl.pdf
digital fundamental by Thomas L.floydl.pdfdigital fundamental by Thomas L.floydl.pdf
digital fundamental by Thomas L.floydl.pdf
 
Properties Railway Sleepers and Test.pptx
Properties Railway Sleepers and Test.pptxProperties Railway Sleepers and Test.pptx
Properties Railway Sleepers and Test.pptx
 
ML Based Model for NIDS MSc Updated Presentation.v2.pptx
ML Based Model for NIDS MSc Updated Presentation.v2.pptxML Based Model for NIDS MSc Updated Presentation.v2.pptx
ML Based Model for NIDS MSc Updated Presentation.v2.pptx
 
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...
 
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECTCHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
CHINA’S GEO-ECONOMIC OUTREACH IN CENTRAL ASIAN COUNTRIES AND FUTURE PROSPECT
 
Modelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdfModelagem de um CSTR com reação endotermica.pdf
Modelagem de um CSTR com reação endotermica.pdf
 
basic-wireline-operations-course-mahmoud-f-radwan.pdf
basic-wireline-operations-course-mahmoud-f-radwan.pdfbasic-wireline-operations-course-mahmoud-f-radwan.pdf
basic-wireline-operations-course-mahmoud-f-radwan.pdf
 
Embedded machine learning-based road conditions and driving behavior monitoring
Embedded machine learning-based road conditions and driving behavior monitoringEmbedded machine learning-based road conditions and driving behavior monitoring
Embedded machine learning-based road conditions and driving behavior monitoring
 

Saturn 2018 rest.li

  • 1. 1Rest.li: RESTful Service Architecture at Scale SATURN 2018 14th Annual SEI Architecture Technology User Network Conference MAY 7–10, 2018 | PLANO, TEXAS Rest.li: RESTful Service Architecture at Scale By: Min Chen (LinkedIn) Email: mnchen@linkedin.com Profile: www.linkedin.com/in/minchen07
  • 2. 2Rest.li: RESTful Service Architecture at Scale A Brief History Monolithic/Layered Complex Orchestration Complex Dependency from Service Explosion Incompatibilities Slow Rollout
  • 3. 3Rest.li: RESTful Service Architecture at Scale Paradigm Shift Microservices Fine-grained Service Boundary Continuous Delivery Standard Service Access Pattern Backward-compatible service evolution
  • 4. 4Rest.li: RESTful Service Architecture at Scale SATURN 2018 Challenges  Standard way to describe service in service-oriented architecture.  Standard access pattern by multi-language diverse clients.  Asynchronous and scalable to handle heavy service load.  Fast and easy experience even for non-REST expert developers.  Support long-term service evolution and growth.
  • 5. 5Rest.li: RESTful Service Architecture at Scale Monolithic Data Model Centric Microservices
  • 6. 6Rest.li: RESTful Service Architecture at Scale Rest.li is an open source REST framework for building robust, scalable RESTful architectures using asynchronous, non-blocking IO. It fills a niche for applying RESTful principles at scale with an end-to-end developer workflow that promotes clean REST practices.
  • 7. 7Rest.li: RESTful Service Architecture at Scale
  • 8. 8Rest.li: RESTful Service Architecture at Scale SATURN 2018 Clean REST Practice  End-to-End Developer Workflow  Consistent data modeling  Uniform interface design  Type-safe client bindings  Online API Hub  API documentation  API service catalog  Compatibility Checker Enabled API Evolution
  • 9. 9Rest.li: RESTful Service Architecture at Scale Data Schema 1 Developer Flow
  • 10. 10Rest.li: RESTful Service Architecture at Scale SATURN 2018 Consistent Data Modeling  Data schema in PDSC (Pegasus Data Schema)  PDSC is inspired by Apache Avro specification.  JSON serialization format  Rich type support:  Named: Record, Enum, Fixed, Typeref  Unnamed: Union, Array, Map  Custom type  Custom validator and coercer
  • 11. 11Rest.li: RESTful Service Architecture at Scale Data Schema Example
  • 12. 12Rest.li: RESTful Service Architecture at Scale Data Schema 1 Resource Class Resource Class Resource Class 3 Record Template 2 Generated Developer Flow
  • 13. 13Rest.li: RESTful Service Architecture at Scale SATURN 2018 Uniform Interface Design  Standardized RESTful resource pattern  Annotated resource identifier  Uniform service interface  CRUD  Finder  Action  BATCH  REST specification IDL (Interface Description Language)
  • 14. 14Rest.li: RESTful Service Architecture at Scale Rest.li Resource Patterns Resource Type Semantics Interface Base Template Collection (@RestLiCollection) Key/value map of entites CollectionResource KeyValueResource CollectionResourceTemplate Simple (@RestLiSimple) Singleton in scope SimpleResource SimpleResourceTemplate Association (@RestLiAssociation) Relationships between entities AssociationResource AssociationResourceTemplate Sub-Resource Child resources All Above All Above Action (@RestliActions) Flexible for non- standard behavior N/A N/A
  • 15. 15Rest.li: RESTful Service Architecture at Scale Rest.li Resource Method (Basic) Rest.li Request HTTP Method URI Example Entity Body (V – Data Model) CREATE POST /statuses V GET GET /statuses/<id> GET_ALL GET /statuses UPDATE PUT /statuses/<id> V PARTIAL_UPDATE POST /statuses/<id> Patch of V DELETE DELETE /statuses/<id> FINDER GET /statuses?q=search&keywords=li nkedin ACTION POST /statuses?action=purge Dynamic Record
  • 16. 16Rest.li: RESTful Service Architecture at Scale Rest.li Resource Method (Batch) Rest.li Request HTTP Method URI Example Entity Body BATCH_CREATE POST /statuses {“elements”: […]} BATCH_GET GET /statuses?ids=List(1,2,3) BATCH_UPDATE PUT /statuses?ids=List(1,2,3) {“entities”: {“1”: {…}, “2” : {…}} BATCH_PARTIAL_UPDATE POST /statuses?ids=List(1,2,3) {“entities”: {“1”: {“patch”: {…}}, “2” : {“patch”: {…}} BATCH_DELETE DELETE /statuses?ids=List(1,2,3)
  • 17. 17Rest.li: RESTful Service Architecture at Scale What Server Code I Need to Write
  • 18. 18Rest.li: RESTful Service Architecture at Scale Data Schema 1 Resource Class Resource Class Resource Class 3 Record Template 2 Generated 4 REST IDL Generated Developer Flow
  • 19. 19Rest.li: RESTful Service Architecture at Scale REST Spec IDL Service Metadata PDSC curl -X OPTIONS <ServiceEndpoint>
  • 20. 20Rest.li: RESTful Service Architecture at Scale Data Schema 1 Server 5 Resource Class Resource Class Resource Class 3 Client 7 Record Template 2 Generated 4 REST IDL Generated 6 Type-Safe RequestBuilders Generated Developer Flow
  • 21. 21Rest.li: RESTful Service Architecture at Scale What Client Code I Need to Write Photo newPhoto = new Photo() .setTitle("NewPhoto") .setFormat(PhotoFormats.PNG) .setExif(new Exif().setLocation(newLocation)); Request<?> createReq = new PhotosRequestBuilder().create().input(newPhoto).build()build(); POST /photos -H "Content-Type: application/json” -d {"title":"NewPhoto", "format": "PNG", "exif":{"location":...}} Strong-typed data model (auto-generated) Type-safe request builders (auto-generated)
  • 22. 22Rest.li: RESTful Service Architecture at Scale Data Schema 1 Server 5 Resource Class Resource Class Resource Class 3 Client 7 Record Template 2 Generated 4 REST IDL Generated Human Readable Documentation Generated 8 6 Type-Safe RequestBuilders Generated Developer Flow
  • 23. 23Rest.li: RESTful Service Architecture at Scale Rest.li API Hub • Lucene Powered full-text search • Data model and interface documentation • Sample request generation • Live interaction with real REST endpoint
  • 24. 24Rest.li: RESTful Service Architecture at Scale SATURN 2018 Backward Compatibility  Build-time Compatibility Checker  equivalent - No changes to Resources or Data model.  backwards - Changes considered backwards.  ignore - Log the changes but allow changes to pass.  off - The compatibility checker will not be run at all.
  • 25. 25Rest.li: RESTful Service Architecture at Scale
  • 26. 26Rest.li: RESTful Service Architecture at Scale Rest.li Server Stack Rest.li Data Layer and RESTful Operations.li Inversion-of-Control D2 Dynamic Discovery and Load Balancing Fault tolerance R2 Request/Response and Network Communication Async and Non-blocking
  • 27. 27Rest.li: RESTful Service Architecture at Scale Request Response (R2) Non-Streaming Streaming Message Communication Abstraction over HTTP
  • 28. 28Rest.li: RESTful Service Architecture at Scale SATURN 2018 Dynamic Discovery (D2)  Apache Zookeeper (Announce and Discover)  Horizontal Scaling  Services, Servers (ephemeral nodes) and Clusters  Client side load balancing  Fault tolerance  Backup request (tail-latency)
  • 29. 29Rest.li: RESTful Service Architecture at Scale Rest.li Client Implement and Interpret Rest.li Protocol (Pluggable Transport Client)
  • 30. 30Rest.li: RESTful Service Architecture at Scale Rest.li Client/Server Data Flow
  • 31. 31Rest.li: RESTful Service Architecture at Scale Open and Extensible  Filter Chain pipeline  Polyglot transport client  Custom Codec (JSON, PSON, XML, etc)
  • 32. 32Rest.li: RESTful Service Architecture at Scale Rich Eco-Systems C# Java Android Java Objective-C Swift Python
  • 33. 33Rest.li: RESTful Service Architecture at Scale Deco Describe WHAT data you want, let Deco worry about HOW to fetch them posts profiles Companies
  • 34. 34Rest.li: RESTful Service Architecture at Scale Deco Concept /posts /companies /profiles
  • 35. 35Rest.li: RESTful Service Architecture at Scale Deco Concept deco://posts/123?projection=(title, content, author~(firstName, lastName, company~(companyName)))  Projection  Decoration  Resolution Normalized Domain Modeling
  • 36. 36Rest.li: RESTful Service Architecture at Scale
  • 37. 37Rest.li: RESTful Service Architecture at Scale SATURN 2018 Asynchronous Challenges  Callback Hell  Thread Safety  Indeterminism  Debugging  How to achieve all-async?
  • 38. 38Rest.li: RESTful Service Architecture at Scale SATURN 2018 ParSeq Concept Promise<T> Represents state of an asynchronous operation, also a container for an asynchronous operation result. Task<T> Main abstraction unit representing an asynchronous operation. Task Callable Main ingredient of Task<T>. It starts asynchronous operation and returns a Promise<T> representing the operation state. Engine Harness to run a Task<T>.
  • 39. 39Rest.li: RESTful Service Architecture at Scale SATURN 2018 ParSeq Programming Model  Declarative and Functional (What not How)  Task composition, chaining, transformation and error handling  Task.async (task creation)  Task.par (parallel composition)  task.andThen (sequential composition)  task.map and task.flatMap (transformation)  task.recover and task.withTimeout (error handling)  Ends up with one Task (execution plan) submitted to ParSeq engine to execute
  • 40. 40Rest.li: RESTful Service Architecture at Scale SATURN 2018 ParSeq Execution Model  All task Callables are executed sequentially, such that Callable of a previous Task in a sequence “happens before” (in Java Memory Model sense) Callable of a next Task.  No need to be concerned about:  Synchronized, locks, volatile, atomic, etc  Immutability  3rd party libraries  Thinking about Java Memory Model  Thread safety does not mean atomicity Task<Response<T>> task = Task.async(name, () -> sendRequest(request, requestContext)); Callable
  • 41. 41Rest.li: RESTful Service Architecture at Scale SATURN 2018 Seamless Integration with Rest.li  Rest.li supports resource method returning Task.  ParSeqRestClient provides API to create Task from a rest.li request @RestMethod.Get public Task<Greeting> get(final Long id) { // rest.li resource implementation } public <T> Task<Response<T>> createTask(final Request<T> request, final RequestContext requestContext) { }
  • 42. 42Rest.li: RESTful Service Architecture at Scale ParSeq Trace Visualization Power Tool for Debugging
  • 43. 43Rest.li: RESTful Service Architecture at Scale SATURN 2018 Multi-Language Support C# Java Android Java Objective-C Swift Python
  • 44. 44Rest.li: RESTful Service Architecture at Scale SATURN 2018 Comparison Matrix Rest.li Jersey DropWizard Olingo (Odata) Data Schema yes no no yes Data Model Auto Manual Manual Manual Server Coding URI Pattern Uniform Non-Uniform Non-Uniform Non-Uniform Batch Support Yes No No Yes Service Metadata Yes No No Yes Dynamic Discovery/Load balancing Yes No No No Documentation Yes No No No Service Catalog Yes No No No Graph Query Yes No No Yes
  • 45. 45Rest.li: RESTful Service Architecture at Scale SATURN 2018 Future of Rest.li  Object Streaming in Rest.li Layer  Unstructured data support  Multi-language support enhancement  Go  Cross-language test suites  Open source Deco  ParSeq  Collection API  ParSeq Trace Analytics
  • 46. 46Rest.li: RESTful Service Architecture at Scale SATURN 2018 Useful Links  Rest.li (https://github.com/linkedin/rest.li)  Rest.li API Hub (https://github.com/linkedin/rest.li-api-hub)  ParSeq (https://github.com/linkedin/parseq)

Editor's Notes

  1. 5/9/2018
  2. Let’s first brief history of Rest.li. Before rest.li is born, LinkedIn service architecture is Monolithic and Layered. With service expansion and quick iteration, this Monolithic architecture brought too many headaches to us.
  3. To address these painpoints, we are attempting a paradigm shift from this monolithic architecture to modern microservices architecture. We are setting high expectations to our paradigm shift effort.
  4. However, It is quite challenging to meet such high expectations. First, we needed a standard way to describe the resources available in our service-oriented architecture. We wanted to focus on standardizing common API operations, while still allowing developers to create non-standard operations. Secondly, we would like to enable consistent access pattern by diverse clients written in any language Third, we needed our solution to be scalable and asynchronous. Many of our services receive several thousand queries per second per instance, and we needed our solution to work under that sort of load. Fourth, We wanted to ensure that writing APIs would be fast and easy for our developers, even if they are not well-versed with REST. Lastly, We also needed our system to support long-term evolution and growth of our interfaces.
  5. Rest.li is our key secret weapon for us to successfully move from Monolithic architecture at LinkedIn to Data Model centric microservices architecture.
  6. Rest.li is a REST framework that was created by LinkedIn and open sourced in 2012. It allows you to build rich, robust, scalable RESTful services using async, non-blocking I/O. It also works beautifully at scale by promoting CLEAN REST practices. Here, I particularly highlighted several keywords here: Clean REST practices/ Scalable and Open Source. Next, I will elaborate these two important points that differentiate Rest.li from other REST framework on the market.
  7. First, Clean REST practice.
  8. There is a famous Chinese Proverb: nothing can be accomplished without norms or standards. This is particularly true in building a large-scale microservice infrastructure. Rest.li aims to set these norms and standards by promoting clean REST practices from the following 3 aspects: End-to-End developer workflow, Online API hub for service documentation and catalog, and Compatibility checker controlled API evolution.
  9. So what does the end-to-end developer flow look like to build a rest.li application? Step 1. The first step is to define your data schema using Pegasus Data Schemas we call it PDSC. The Pegasus Data Schema format uses a simple Avro-like syntax.
  10. Consistent data modeling plays an important role to achieve data model centric microservices infrastructure.
  11. This is a simple data schema file example.
  12. Step 2, Rest.li code generator will automatically create a language-specific POJO-like class that represent the data model defined in Step 1, we call these POJO-like class RecordTemplate. These RecordTemplate classes serve as language bindings of the data model in both the server and client. Step 3, Developers now need to implement server REST Resource classes and define the operations they support. Rest.li provides a set of annotations and base classes that allow you to map Resource classes to REST endpoints and to specify methods of your Resource classes to respond to REST operations, such as GET or PUT. Your Resource classes are expected to use or return data using instances of the RecordTemplate classes generated in Step 2.
  13. Rest.li is intended to foster Uniform interface design, which makes developing a Restful service like a breeze even for non REST expert developers. Rest.li framework provides some standardized REST resource pattern and a set of annotations for developers to define RESTful endpoints and operations supported. We have categorized typical RESTful operations into CRUD, Finder, Action and BATCH to both standardize common API operations while still allowing developers to create non-standard operations. RESTful interface contract is formally communicated btw client and server through a well-defined IDL format.
  14. Each Rest.li resource endpoint is one of the following types: Collection – used to model key-vale map of entities. Simple – used to model singleton in a certain scope. Association – used to model relationships between entities. Action – used to accommodate some non-standard REST behaviors. Additionally, each resource endpoint may be a sub-resource of any other resource. We have provided out-of-box resource template class and interface with well-defined annotations for developer to customize for their own resource.
  15. Resources are the nouns in the Rest.li world, and Resource Methods are the verbs. Rest.li provides a standard set of Resource Methods that describe what clients may do with the resources. These verbs are different than the HTTP verbs (GET, POST, PUT, DELETE, and so on) but map onto them (CREATE is a POST, UPDATE is a PUT, FINDER is a GET). ACTION are provided for users to define some non-standard functions and actions.
  16. Rest.li also provided batch versioned resource methods to reduce round-trip to servers for performance consideration. Server developers can use BATCH to do some performance optimization like cache, database batch call, etc.
  17. With resource pattern and interface method standardized, server developer’s task to introduce a new rest.li endpoint becomes a piece of cake. You only need to extend a proper ResourceTemplate, and implement those resource methods you are planning to support using those POJO-like RecordTemplate classes.
  18. Step 4, After resources are implemented by developers, Rest.li will automatically generate an interface description (IDL) file that is considered the source of truth for the interface contract between the server resource implemented in Step 3 and its clients. The IDL itself is a language-agnostic JSON format.
  19. After resource class is implemented, our gradle build process will automatically generate a REST spec IDL file for your service which together with your data model PDSC file constitutes your official service metadata and contract to external parties. This service metadata can be dynamically discovered by your external parties using OPTIONS call.
  20. Step 5, Developer now can create your server application, which involves leveraging a few Rest.li classes to instantiate and configure the Rest.li server. Now we are done with server implementation, can move to client side development. Step 6, Based on REST IDL generated in step 4, Rest.li can auto-generate classes known as RequestBuilders that are used by clients to create requests to send to the server. Step 7, Together with the RecordTemplates, RequestBuilders provide convenient and type-safe mechanisms for developers to write client applications to working with the data models supported by the server by issuing rest.li request to rest.li server.
  21. Now let’s look at how client developers need to interact with this service. For example, photos is a resource server developer just exposed and which supports CREATE operation, we need to create a new Photo. To perform this operation, this is the client code you need to write. Auto-generated strong-typed Data model binding and type-safe request builders hide over-the-wire rest.li protocol from client developers and make it quite easy to send a valid rest.li request.
  22. Last but not the least, Rest.li uses this IDL along with the original data schema files to support automatically generating human-readable documentation, which can be requested from a server. This documentation is also used by rest.li framework to check backward compatibility during API evolution. Blue boxes represent code developers need to write, all other components are auto-generated from rest.li framework. Throughout this end-to-end developer flow, we are trying standardizing developer practice through 3 aspects: Consistent data modeling, Uniform interface design, Type-safe client bindings.
  23. To foster cooperation and cross-service communication in a large-scale system, an on-line service catalog becomes an indispensable part to facilitate smooth integration. Rest.li API Hub is a WEB-UI to browse and search a catalog of rest.li APIs along with their on-line documentation. It is empowered with the following features: Lucene powered full-text search. Data model and service interface documentation. Sample request generation and live interaction with real REST endpoint.
  24. Last point in CLEAN REST practice: service backward compatibility. In Restful world, API is not static and needs to evolve based on business needs. This may create many integration headaches if not done properly due to backward-incompatibility issues. Rest.li framework comes with a build-time compatibility checker to provide both data moel and interface compatibility issue at build-time and provide early warnings to API developers before real deployment based on their configured compatibility settings. We support various compatibility settings like equivalent, backwards, ignore, and off.
  25. Now let’s move to second highlight of Rest.li: Scale and Open. This will delve a bit into Rest.li server stack.
  26. Rest.li server stack consists of three layers: First: Rest.li layer. It includes data model layer and RESTful operations exposed through our uniform interface design. Internally it is a Java framework based on inversion-of-control model, which handles most of the data flow and client/server interaction transparently and calls code you supply at the appropriate time. Second: D2 layer. This is our dynamic discovery and client-side load balancing layer. This layer is optional for smaller service, but provide fault tolerance and scalability to large-scale service infrastructure like LinkedIn. Last: R2 layer. This is our transport layer abstraction in Java. It exposes a low-level message style communication inspired by HTTP and a high-performance asynchronous/non-blocking API. We have covered most stuff in Rest.li layer in explaining end-to-end developer flow, which makes Rest.li developer-friendly. Now let’s see how we achieve scalability through R2D2 layer.
  27. R2 stands for Request Response. It is a java abstraction of message-style communication over HTTP. To provide a high-performance asynchronous API with non-blocking back-pressure support, we would like a streaming model illustrated in the right figure, where the body of the request/response could be processed one chuck at a time, instead of fully buffered in the left. This is the EntityStream API embedded in our R2 implementation.
  28. D2 is our dynamic discovery layer built on top of Apache ZooKeeper. By dynamic discovery I mean that clients do not have to hardcode server URLs in their code. Clients simply need to specify what service or resource they want to talk to and D2 converts that name into a URL and directs to the right host. By grouping services into clusters and dynamic service announcement and shutdown with zookeeper ephemeral nodes, we can perform client side load-balancing, and backup request to improve tail-latency, of course, fault tolerance when one server is not healthy. We use ZooKeeper as a registry for information about available services and the hosts that provide them. Clients automatically get the latest information from ZooKeeper and apply load-balancing algorithms on the client side to distribute load evenly across servers and reduce load to overloaded servers. This layer is optional for smaller service, but provide fault tolerance and scalability to large-scale service infrastructure like LinkedIn.
  29. Rest.li defined its own over-the-wire protocol. It provides a client-side framework to shield this low-level wire protocol from users. Apart from type-safe client bindings we have mentioned earlier, the core of Rest.li client framework is RestLiClient interface with its pluggable transport client architecture. This RestLiClient interface will implement and interpret Rest.li protocol and delegate to a pluggable transport client (for example, Netty client, etc) to transport HTTP request/response message to and back from server.
  30. This diagram provides a high-level view of the interaction and data flow between a Rest.li client and server. The yellow arrows indicate the flow of requests out of the client and into the server, while dark blue arrows represent the server’s response.
  31. As an open source project, we are striving to provide great room for customization and extension. It provides several extension points for customization: Filter chain pipeline. Rest.li provides a mechanism to intercept incoming requests and outgoing responses via filters. Each filter contains methods that handle both requests and responses. Filter can be at both Rest.li level (dealing with data model entities) or at R2 level (dealing with Bytes) Polyglot transport client bindings. R2 layer only defined a message communication abstraction. Default out-of-box rest.li supports Netty client, Jetty server, but you are freely plug into their own transport client and servlet container. Custom codec. Out-of-box rest.li supports JSON and PSON content encoding, but it provides pluggable framework for you to easily plugin your custom codec to encode/decode request/response in your preferred format, for example, XML, etc.
  32. Building a large-scale application cannot be limited to just send individual REST requests. Most likely, you may need to deal with heterogenous sub-systems implemented in diverse languages, struggle to build a presentable view on top of our data models, also need to consider executing some downstream calls in some sequential/parallel fashion with complex workflow dependency. This is why our rich eco-systems built around Rest.li come into help, specifically, Deco, ParSeq, and Multi-language support.
  33. What is Deco? The name “Deco” comes from the word ”Decoration”. It is a Java library for projecting fields and fetching interconnected rest.li resources based on a specified query. It was designed to allow users to “Describe WHAT data you want, let Deco worry about HOW to fetch them”. Here shows a perfect real scenario where Deco is used inside LinkedIn to display a post created by a LinkedIn member.
  34. To fetch data for such a Post, we need to make service calls to 3 different services (/posts, /profiles, /companies), and somehow massage data into such a data model view. Deco provides a special query language (deco recipes) to achieve this.
  35. The deco recipe will be like this. Deco supports field projection, field decoration, and FK resolution. All these features enabled us for normalized domain modeling to avoid data redundancy.
  36. Next, I would like to introduce another rest.li eco-system: ParSeq which is also open sourced and well-adopted at LinkedIn in both frontend and midtier services. This is a framework to make asynchronous Java easier. It is well known fact that writing and using asynchronous code is tricky. It often adds accidental complexity that is unrelated to app or business logic code, but rather caused by the asynchronous mechanism.
  37. It is well known fact that writing and using asynchronous code is tricky. It often adds accidental complexity that is unrelated to app or business logic code, but rather caused by the asynchronous mechanism. Here lists many buzz words (Callback hell, Thread safety, Indeterminism, Async debugging) about pain-points of developing all-async code flow. Asynchronous operations can finish in different orders because the underlying operations can take varying amounts of time. This is due to both expected causes like fetching more data and executing more time-consuming operations, and unexpected and problematic causes such as garbage collection or server load. This behavior makes asynchronous operations hard to test and adds elements of non-determinism to asynchronous code. Meaningless stack traces make debugging difficult, and coordinating callbacks on various threads is challenging. Furthermore, developers must always consider the issue of thread safety. These factors, among others, make asynchrony quite complex.
  38. To address these challenges, we built ParSeq to make programming using asynchronous methods in Java easier. Here lists several main concepts in ParSeq. Task: It is the main abstraction unit in ParSeq system to represent an asynchronous operation whose result can be set asynchronously. Promise: represents the state of an asynchronous operation and also a container for the operation result. Task Callable: it is the main ingredient of Task, which is like a Callable starting asynchronous operation and returns a Promise. Finally, ParSeq engine, which is a harness responsible for running a Task.
  39. ParSeq programming model is declarative and functional, focusing on What you want to do, not How you get that. It provides developer-friendly API for coordinating asynchronous tasks, for example, task creation, parallel/sequential composition, transformation, error handling, etc. It ends up with one Task (internally represented as an execution plan) that will be submitted to ParSeq Engine to execute.
  40. ParSeq execution model maintains a very important invariant to relieve developers from various asynchronous programming concerns. That is, All Task Callables are executed sequentially, such that Callable of a previous Task in a sequence “happens before” (in Java Memory Model sense) Callable of a next Task. This is a real example of Task Callable from sending a rest.li request.
  41. Another valuable feature provided by ParSeq is its built-in tracing and visualization. This is very important for service monitoring and observability. Trace visualization provides us a powerful tool for debugging some problematic application flow, also can provide us performance metric on each step in your business logic.
  42. To promote our clean REST practices to heterogenous system and diverse clients, we are striving to expand our multi-language support. So far, we have already had support for Java desktop, Android Java, Objective-C/Swift for iOS, Python, C# support. Go client binding support is working in progress.
  43. Here is an overview matrix of comparing Rest.li framework with other popular RESTful framework on the market. I just picked some representatives, but they are more or less similar.
  44. The rest.li framework has been proven to create robust RESTful service architectures that can work at scale from its wide adoption at LinkedIn and Coursera.  There are still many interesting items on our road map, for example: - Object streaming and unstructured data support in rest.li Multi-language suppprt enhancement including Go and cross-language test suites. Open source our Deco library. In ParSeq, we are planning to enhance it with friendly collection API, and also power ParSeq Tracing with Intelligent data Analytics. We are looking forward to contributions from our audience here.