Downloaded 24 times
![16Rest.li: RESTful Service Architecture at Scale
Rest.li Resource Method (Batch)
Rest.li Request HTTP Method URI Example Entity Body
BATCH_CREATE POST /statuses {“elements”: […]}
BATCH_GET GET /statuses?ids=List(1,2,3)
BATCH_UPDATE PUT /statuses?ids=List(1,2,3) {“entities”: {“1”: {…}, “2” :
{…}}
BATCH_PARTIAL_UPDATE POST /statuses?ids=List(1,2,3) {“entities”: {“1”: {“patch”:
{…}}, “2” : {“patch”: {…}}
BATCH_DELETE DELETE /statuses?ids=List(1,2,3)](https://image.slidesharecdn.com/saturn2018rest-180509194755/75/Saturn-2018-rest-li-16-2048.jpg)
Uploaded byMin Chen
Saturn 2018 rest.li
Rest.li is an open source REST framework for building scalable RESTful architectures. It addresses challenges like standard service description, asynchronous scalability, and API evolution. Rest.li uses data modeling, resource patterns, and auto-generated artifacts to promote clean REST practices. It also provides tools for service discovery, backward compatibility, and asynchronous programming with ParSeq.
More Related Content
![[Image Results] Java Build Tools: Part 2 - A Decision Maker's Guide Compariso...](https://cdn.slidesharecdn.com/ss_thumbnails/java-build-tools-part-2-maven-gradle-ant-ivy-140121040954-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Saturn 2018 rest.li
![cloud enabling tecnologies , unit ii [cc]](https://cdn.slidesharecdn.com/ss_thumbnails/unit-ii-240619035331-08614007-thumbnail.jpg?width=640&height=640&fit=bounds)
Saturn 2018 rest.li
- 1. 1Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 14th Annual SEI Architecture Technology User Network Conference MAY 7–10, 2018 | PLANO, TEXAS Rest.li: RESTful Service Architecture at Scale By: Min Chen (LinkedIn) Email: mnchen@linkedin.com Profile: www.linkedin.com/in/minchen07
- 2. 2Rest.li: RESTful ServiceArchitecture at Scale A Brief History Monolithic/Layered Complex Orchestration Complex Dependency from Service Explosion Incompatibilities Slow Rollout
- 3. 3Rest.li: RESTful ServiceArchitecture at Scale Paradigm Shift Microservices Fine-grained Service Boundary Continuous Delivery Standard Service Access Pattern Backward-compatible service evolution
- 4. 4Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Challenges Standard way to describe service in service-oriented architecture. Standard access pattern by multi-language diverse clients. Asynchronous and scalable to handle heavy service load. Fast and easy experience even for non-REST expert developers. Support long-term service evolution and growth.
- 5. 5Rest.li: RESTful ServiceArchitecture at Scale Monolithic Data Model Centric Microservices
- 6. 6Rest.li: RESTful ServiceArchitecture at Scale Rest.li is an open source REST framework for building robust, scalable RESTful architectures using asynchronous, non-blocking IO. It fills a niche for applying RESTful principles at scale with an end-to-end developer workflow that promotes clean REST practices.
- 7. 7Rest.li: RESTful ServiceArchitecture at Scale
- 8. 8Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Clean REST Practice End-to-End Developer Workflow Consistent data modeling Uniform interface design Type-safe client bindings Online API Hub API documentation API service catalog Compatibility Checker Enabled API Evolution
- 9. 9Rest.li: RESTful ServiceArchitecture at Scale Data Schema 1 Developer Flow
- 10. 10Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Consistent Data Modeling Data schema in PDSC (Pegasus Data Schema) PDSC is inspired by Apache Avro specification. JSON serialization format Rich type support: Named: Record, Enum, Fixed, Typeref Unnamed: Union, Array, Map Custom type Custom validator and coercer
- 11. 11Rest.li: RESTful ServiceArchitecture at Scale Data Schema Example
- 12. 12Rest.li: RESTful ServiceArchitecture at Scale Data Schema 1 Resource Class Resource Class Resource Class 3 Record Template 2 Generated Developer Flow
- 13. 13Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Uniform Interface Design Standardized RESTful resource pattern Annotated resource identifier Uniform service interface CRUD Finder Action BATCH REST specification IDL (Interface Description Language)
- 14. 14Rest.li: RESTful ServiceArchitecture at Scale Rest.li Resource Patterns Resource Type Semantics Interface Base Template Collection (@RestLiCollection) Key/value map of entites CollectionResource KeyValueResource CollectionResourceTemplate Simple (@RestLiSimple) Singleton in scope SimpleResource SimpleResourceTemplate Association (@RestLiAssociation) Relationships between entities AssociationResource AssociationResourceTemplate Sub-Resource Child resources All Above All Above Action (@RestliActions) Flexible for non- standard behavior N/A N/A
- 15. 15Rest.li: RESTful ServiceArchitecture at Scale Rest.li Resource Method (Basic) Rest.li Request HTTP Method URI Example Entity Body (V – Data Model) CREATE POST /statuses V GET GET /statuses/<id> GET_ALL GET /statuses UPDATE PUT /statuses/<id> V PARTIAL_UPDATE POST /statuses/<id> Patch of V DELETE DELETE /statuses/<id> FINDER GET /statuses?q=search&keywords=li nkedin ACTION POST /statuses?action=purge Dynamic Record
- 16. 16Rest.li: RESTful ServiceArchitecture at Scale Rest.li Resource Method (Batch) Rest.li Request HTTP Method URI Example Entity Body BATCH_CREATE POST /statuses {“elements”: […]} BATCH_GET GET /statuses?ids=List(1,2,3) BATCH_UPDATE PUT /statuses?ids=List(1,2,3) {“entities”: {“1”: {…}, “2” : {…}} BATCH_PARTIAL_UPDATE POST /statuses?ids=List(1,2,3) {“entities”: {“1”: {“patch”: {…}}, “2” : {“patch”: {…}} BATCH_DELETE DELETE /statuses?ids=List(1,2,3)
- 17. 17Rest.li: RESTful ServiceArchitecture at Scale What Server Code I Need to Write
- 18. 18Rest.li: RESTful ServiceArchitecture at Scale Data Schema 1 Resource Class Resource Class Resource Class 3 Record Template 2 Generated 4 REST IDL Generated Developer Flow
- 19. 19Rest.li: RESTful ServiceArchitecture at Scale REST Spec IDL Service Metadata PDSC curl -X OPTIONS <ServiceEndpoint>
- 20. 20Rest.li: RESTful ServiceArchitecture at Scale Data Schema 1 Server 5 Resource Class Resource Class Resource Class 3 Client 7 Record Template 2 Generated 4 REST IDL Generated 6 Type-Safe RequestBuilders Generated Developer Flow
- 21. 21Rest.li: RESTful ServiceArchitecture at Scale What Client Code I Need to Write Photo newPhoto = new Photo() .setTitle("NewPhoto") .setFormat(PhotoFormats.PNG) .setExif(new Exif().setLocation(newLocation)); Request<?> createReq = new PhotosRequestBuilder().create().input(newPhoto).build()build(); POST /photos -H "Content-Type: application/json” -d {"title":"NewPhoto", "format": "PNG", "exif":{"location":...}} Strong-typed data model (auto-generated) Type-safe request builders (auto-generated)
- 22. 22Rest.li: RESTful ServiceArchitecture at Scale Data Schema 1 Server 5 Resource Class Resource Class Resource Class 3 Client 7 Record Template 2 Generated 4 REST IDL Generated Human Readable Documentation Generated 8 6 Type-Safe RequestBuilders Generated Developer Flow
- 23. 23Rest.li: RESTful ServiceArchitecture at Scale Rest.li API Hub • Lucene Powered full-text search • Data model and interface documentation • Sample request generation • Live interaction with real REST endpoint
- 24. 24Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Backward Compatibility Build-time Compatibility Checker equivalent - No changes to Resources or Data model. backwards - Changes considered backwards. ignore - Log the changes but allow changes to pass. off - The compatibility checker will not be run at all.
- 25. 25Rest.li: RESTful ServiceArchitecture at Scale
- 26. 26Rest.li: RESTful ServiceArchitecture at Scale Rest.li Server Stack Rest.li Data Layer and RESTful Operations.li Inversion-of-Control D2 Dynamic Discovery and Load Balancing Fault tolerance R2 Request/Response and Network Communication Async and Non-blocking
- 27. 27Rest.li: RESTful ServiceArchitecture at Scale Request Response (R2) Non-Streaming Streaming Message Communication Abstraction over HTTP
- 28. 28Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Dynamic Discovery (D2) Apache Zookeeper (Announce and Discover) Horizontal Scaling Services, Servers (ephemeral nodes) and Clusters Client side load balancing Fault tolerance Backup request (tail-latency)
- 29. 29Rest.li: RESTful ServiceArchitecture at Scale Rest.li Client Implement and Interpret Rest.li Protocol (Pluggable Transport Client)
- 30. 30Rest.li: RESTful ServiceArchitecture at Scale Rest.li Client/Server Data Flow
- 31. 31Rest.li: RESTful ServiceArchitecture at Scale Open and Extensible Filter Chain pipeline Polyglot transport client Custom Codec (JSON, PSON, XML, etc)
- 32. 32Rest.li: RESTful ServiceArchitecture at Scale Rich Eco-Systems C# Java Android Java Objective-C Swift Python
- 33. 33Rest.li: RESTful ServiceArchitecture at Scale Deco Describe WHAT data you want, let Deco worry about HOW to fetch them posts profiles Companies
- 34. 34Rest.li: RESTful ServiceArchitecture at Scale Deco Concept /posts /companies /profiles
- 35. 35Rest.li: RESTful ServiceArchitecture at Scale Deco Concept deco://posts/123?projection=(title, content, author~(firstName, lastName, company~(companyName))) Projection Decoration Resolution Normalized Domain Modeling
- 36. 36Rest.li: RESTful ServiceArchitecture at Scale
- 37. 37Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Asynchronous Challenges Callback Hell Thread Safety Indeterminism Debugging How to achieve all-async?
- 38. 38Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 ParSeq Concept Promise<T> Represents state of an asynchronous operation, also a container for an asynchronous operation result. Task<T> Main abstraction unit representing an asynchronous operation. Task Callable Main ingredient of Task<T>. It starts asynchronous operation and returns a Promise<T> representing the operation state. Engine Harness to run a Task<T>.
- 39. 39Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 ParSeq Programming Model Declarative and Functional (What not How) Task composition, chaining, transformation and error handling Task.async (task creation) Task.par (parallel composition) task.andThen (sequential composition) task.map and task.flatMap (transformation) task.recover and task.withTimeout (error handling) Ends up with one Task (execution plan) submitted to ParSeq engine to execute
- 40. 40Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 ParSeq Execution Model All task Callables are executed sequentially, such that Callable of a previous Task in a sequence “happens before” (in Java Memory Model sense) Callable of a next Task. No need to be concerned about: Synchronized, locks, volatile, atomic, etc Immutability 3rd party libraries Thinking about Java Memory Model Thread safety does not mean atomicity Task<Response<T>> task = Task.async(name, () -> sendRequest(request, requestContext)); Callable
- 41. 41Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Seamless Integration with Rest.li Rest.li supports resource method returning Task. ParSeqRestClient provides API to create Task from a rest.li request @RestMethod.Get public Task<Greeting> get(final Long id) { // rest.li resource implementation } public <T> Task<Response<T>> createTask(final Request<T> request, final RequestContext requestContext) { }
- 42. 42Rest.li: RESTful ServiceArchitecture at Scale ParSeq Trace Visualization Power Tool for Debugging
- 43. 43Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Multi-Language Support C# Java Android Java Objective-C Swift Python
- 44. 44Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Comparison Matrix Rest.li Jersey DropWizard Olingo (Odata) Data Schema yes no no yes Data Model Auto Manual Manual Manual Server Coding URI Pattern Uniform Non-Uniform Non-Uniform Non-Uniform Batch Support Yes No No Yes Service Metadata Yes No No Yes Dynamic Discovery/Load balancing Yes No No No Documentation Yes No No No Service Catalog Yes No No No Graph Query Yes No No Yes
- 45. 45Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Future of Rest.li Object Streaming in Rest.li Layer Unstructured data support Multi-language support enhancement Go Cross-language test suites Open source Deco ParSeq Collection API ParSeq Trace Analytics
- 46. 46Rest.li: RESTful ServiceArchitecture at Scale SATURN 2018 Useful Links Rest.li (https://github.com/linkedin/rest.li) Rest.li API Hub (https://github.com/linkedin/rest.li-api-hub) ParSeq (https://github.com/linkedin/parseq)
Editor's Notes
- #2 5/9/2018
- #3 Let’s first brief history of Rest.li. Before rest.li is born, LinkedIn service architecture is Monolithic and Layered. With service expansion and quick iteration, this Monolithic architecture brought too many headaches to us.
- #4 To address these painpoints, we are attempting a paradigm shift from this monolithic architecture to modern microservices architecture. We are setting high expectations to our paradigm shift effort.
- #5 However, It is quite challenging to meet such high expectations. First, we needed a standard way to describe the resources available in our service-oriented architecture. We wanted to focus on standardizing common API operations, while still allowing developers to create non-standard operations. Secondly, we would like to enable consistent access pattern by diverse clients written in any language Third, we needed our solution to be scalable and asynchronous. Many of our services receive several thousand queries per second per instance, and we needed our solution to work under that sort of load. Fourth, We wanted to ensure that writing APIs would be fast and easy for our developers, even if they are not well-versed with REST. Lastly, We also needed our system to support long-term evolution and growth of our interfaces.
- #6 Rest.li is our key secret weapon for us to successfully move from Monolithic architecture at LinkedIn to Data Model centric microservices architecture.
- #7 Rest.li is a REST framework that was created by LinkedIn and open sourced in 2012. It allows you to build rich, robust, scalable RESTful services using async, non-blocking I/O. It also works beautifully at scale by promoting CLEAN REST practices. Here, I particularly highlighted several keywords here: Clean REST practices/ Scalable and Open Source. Next, I will elaborate these two important points that differentiate Rest.li from other REST framework on the market.
- #8 First, Clean REST practice.
- #9 There is a famous Chinese Proverb: nothing can be accomplished without norms or standards. This is particularly true in building a large-scale microservice infrastructure. Rest.li aims to set these norms and standards by promoting clean REST practices from the following 3 aspects: End-to-End developer workflow, Online API hub for service documentation and catalog, and Compatibility checker controlled API evolution.
- #10 So what does the end-to-end developer flow look like to build a rest.li application? Step 1. The first step is to define your data schema using Pegasus Data Schemas we call it PDSC. The Pegasus Data Schema format uses a simple Avro-like syntax.
- #11 Consistent data modeling plays an important role to achieve data model centric microservices infrastructure.
- #12 This is a simple data schema file example.
- #13 Step 2, Rest.li code generator will automatically create a language-specific POJO-like class that represent the data model defined in Step 1, we call these POJO-like class RecordTemplate. These RecordTemplate classes serve as language bindings of the data model in both the server and client. Step 3, Developers now need to implement server REST Resource classes and define the operations they support. Rest.li provides a set of annotations and base classes that allow you to map Resource classes to REST endpoints and to specify methods of your Resource classes to respond to REST operations, such as GET or PUT. Your Resource classes are expected to use or return data using instances of the RecordTemplate classes generated in Step 2.
- #14 Rest.li is intended to foster Uniform interface design, which makes developing a Restful service like a breeze even for non REST expert developers. Rest.li framework provides some standardized REST resource pattern and a set of annotations for developers to define RESTful endpoints and operations supported. We have categorized typical RESTful operations into CRUD, Finder, Action and BATCH to both standardize common API operations while still allowing developers to create non-standard operations. RESTful interface contract is formally communicated btw client and server through a well-defined IDL format.
- #15 Each Rest.li resource endpoint is one of the following types: Collection – used to model key-vale map of entities. Simple – used to model singleton in a certain scope. Association – used to model relationships between entities. Action – used to accommodate some non-standard REST behaviors. Additionally, each resource endpoint may be a sub-resource of any other resource. We have provided out-of-box resource template class and interface with well-defined annotations for developer to customize for their own resource.
- #16 Resources are the nouns in the Rest.li world, and Resource Methods are the verbs. Rest.li provides a standard set of Resource Methods that describe what clients may do with the resources. These verbs are different than the HTTP verbs (GET, POST, PUT, DELETE, and so on) but map onto them (CREATE is a POST, UPDATE is a PUT, FINDER is a GET). ACTION are provided for users to define some non-standard functions and actions.
- #17 Rest.li also provided batch versioned resource methods to reduce round-trip to servers for performance consideration. Server developers can use BATCH to do some performance optimization like cache, database batch call, etc.
- #18 With resource pattern and interface method standardized, server developer’s task to introduce a new rest.li endpoint becomes a piece of cake. You only need to extend a proper ResourceTemplate, and implement those resource methods you are planning to support using those POJO-like RecordTemplate classes.
- #19 Step 4, After resources are implemented by developers, Rest.li will automatically generate an interface description (IDL) file that is considered the source of truth for the interface contract between the server resource implemented in Step 3 and its clients. The IDL itself is a language-agnostic JSON format.
- #20 After resource class is implemented, our gradle build process will automatically generate a REST spec IDL file for your service which together with your data model PDSC file constitutes your official service metadata and contract to external parties. This service metadata can be dynamically discovered by your external parties using OPTIONS call.
- #21 Step 5, Developer now can create your server application, which involves leveraging a few Rest.li classes to instantiate and configure the Rest.li server. Now we are done with server implementation, can move to client side development. Step 6, Based on REST IDL generated in step 4, Rest.li can auto-generate classes known as RequestBuilders that are used by clients to create requests to send to the server. Step 7, Together with the RecordTemplates, RequestBuilders provide convenient and type-safe mechanisms for developers to write client applications to working with the data models supported by the server by issuing rest.li request to rest.li server.
- #22 Now let’s look at how client developers need to interact with this service. For example, photos is a resource server developer just exposed and which supports CREATE operation, we need to create a new Photo. To perform this operation, this is the client code you need to write. Auto-generated strong-typed Data model binding and type-safe request builders hide over-the-wire rest.li protocol from client developers and make it quite easy to send a valid rest.li request.
- #23 Last but not the least, Rest.li uses this IDL along with the original data schema files to support automatically generating human-readable documentation, which can be requested from a server. This documentation is also used by rest.li framework to check backward compatibility during API evolution. Blue boxes represent code developers need to write, all other components are auto-generated from rest.li framework. Throughout this end-to-end developer flow, we are trying standardizing developer practice through 3 aspects: Consistent data modeling, Uniform interface design, Type-safe client bindings.
- #24 To foster cooperation and cross-service communication in a large-scale system, an on-line service catalog becomes an indispensable part to facilitate smooth integration. Rest.li API Hub is a WEB-UI to browse and search a catalog of rest.li APIs along with their on-line documentation. It is empowered with the following features: Lucene powered full-text search. Data model and service interface documentation. Sample request generation and live interaction with real REST endpoint.
- #25 Last point in CLEAN REST practice: service backward compatibility. In Restful world, API is not static and needs to evolve based on business needs. This may create many integration headaches if not done properly due to backward-incompatibility issues. Rest.li framework comes with a build-time compatibility checker to provide both data moel and interface compatibility issue at build-time and provide early warnings to API developers before real deployment based on their configured compatibility settings. We support various compatibility settings like equivalent, backwards, ignore, and off.
- #26 Now let’s move to second highlight of Rest.li: Scale and Open. This will delve a bit into Rest.li server stack.
- #27 Rest.li server stack consists of three layers: First: Rest.li layer. It includes data model layer and RESTful operations exposed through our uniform interface design. Internally it is a Java framework based on inversion-of-control model, which handles most of the data flow and client/server interaction transparently and calls code you supply at the appropriate time. Second: D2 layer. This is our dynamic discovery and client-side load balancing layer. This layer is optional for smaller service, but provide fault tolerance and scalability to large-scale service infrastructure like LinkedIn. Last: R2 layer. This is our transport layer abstraction in Java. It exposes a low-level message style communication inspired by HTTP and a high-performance asynchronous/non-blocking API. We have covered most stuff in Rest.li layer in explaining end-to-end developer flow, which makes Rest.li developer-friendly. Now let’s see how we achieve scalability through R2D2 layer.
- #28 R2 stands for Request Response. It is a java abstraction of message-style communication over HTTP. To provide a high-performance asynchronous API with non-blocking back-pressure support, we would like a streaming model illustrated in the right figure, where the body of the request/response could be processed one chuck at a time, instead of fully buffered in the left. This is the EntityStream API embedded in our R2 implementation.
- #29 D2 is our dynamic discovery layer built on top of Apache ZooKeeper. By dynamic discovery I mean that clients do not have to hardcode server URLs in their code. Clients simply need to specify what service or resource they want to talk to and D2 converts that name into a URL and directs to the right host. By grouping services into clusters and dynamic service announcement and shutdown with zookeeper ephemeral nodes, we can perform client side load-balancing, and backup request to improve tail-latency, of course, fault tolerance when one server is not healthy. We use ZooKeeper as a registry for information about available services and the hosts that provide them. Clients automatically get the latest information from ZooKeeper and apply load-balancing algorithms on the client side to distribute load evenly across servers and reduce load to overloaded servers. This layer is optional for smaller service, but provide fault tolerance and scalability to large-scale service infrastructure like LinkedIn.
- #30 Rest.li defined its own over-the-wire protocol. It provides a client-side framework to shield this low-level wire protocol from users. Apart from type-safe client bindings we have mentioned earlier, the core of Rest.li client framework is RestLiClient interface with its pluggable transport client architecture. This RestLiClient interface will implement and interpret Rest.li protocol and delegate to a pluggable transport client (for example, Netty client, etc) to transport HTTP request/response message to and back from server.
- #31 This diagram provides a high-level view of the interaction and data flow between a Rest.li client and server. The yellow arrows indicate the flow of requests out of the client and into the server, while dark blue arrows represent the server’s response.
- #32 As an open source project, we are striving to provide great room for customization and extension. It provides several extension points for customization: Filter chain pipeline. Rest.li provides a mechanism to intercept incoming requests and outgoing responses via filters. Each filter contains methods that handle both requests and responses. Filter can be at both Rest.li level (dealing with data model entities) or at R2 level (dealing with Bytes) Polyglot transport client bindings. R2 layer only defined a message communication abstraction. Default out-of-box rest.li supports Netty client, Jetty server, but you are freely plug into their own transport client and servlet container. Custom codec. Out-of-box rest.li supports JSON and PSON content encoding, but it provides pluggable framework for you to easily plugin your custom codec to encode/decode request/response in your preferred format, for example, XML, etc.
- #33 Building a large-scale application cannot be limited to just send individual REST requests. Most likely, you may need to deal with heterogenous sub-systems implemented in diverse languages, struggle to build a presentable view on top of our data models, also need to consider executing some downstream calls in some sequential/parallel fashion with complex workflow dependency. This is why our rich eco-systems built around Rest.li come into help, specifically, Deco, ParSeq, and Multi-language support.
- #34 What is Deco? The name “Deco” comes from the word ”Decoration”. It is a Java library for projecting fields and fetching interconnected rest.li resources based on a specified query. It was designed to allow users to “Describe WHAT data you want, let Deco worry about HOW to fetch them”. Here shows a perfect real scenario where Deco is used inside LinkedIn to display a post created by a LinkedIn member.
- #35 To fetch data for such a Post, we need to make service calls to 3 different services (/posts, /profiles, /companies), and somehow massage data into such a data model view. Deco provides a special query language (deco recipes) to achieve this.
- #36 The deco recipe will be like this. Deco supports field projection, field decoration, and FK resolution. All these features enabled us for normalized domain modeling to avoid data redundancy.
- #37 Next, I would like to introduce another rest.li eco-system: ParSeq which is also open sourced and well-adopted at LinkedIn in both frontend and midtier services. This is a framework to make asynchronous Java easier. It is well known fact that writing and using asynchronous code is tricky. It often adds accidental complexity that is unrelated to app or business logic code, but rather caused by the asynchronous mechanism.
- #38 It is well known fact that writing and using asynchronous code is tricky. It often adds accidental complexity that is unrelated to app or business logic code, but rather caused by the asynchronous mechanism. Here lists many buzz words (Callback hell, Thread safety, Indeterminism, Async debugging) about pain-points of developing all-async code flow. Asynchronous operations can finish in different orders because the underlying operations can take varying amounts of time. This is due to both expected causes like fetching more data and executing more time-consuming operations, and unexpected and problematic causes such as garbage collection or server load. This behavior makes asynchronous operations hard to test and adds elements of non-determinism to asynchronous code. Meaningless stack traces make debugging difficult, and coordinating callbacks on various threads is challenging. Furthermore, developers must always consider the issue of thread safety. These factors, among others, make asynchrony quite complex.
- #39 To address these challenges, we built ParSeq to make programming using asynchronous methods in Java easier. Here lists several main concepts in ParSeq. Task: It is the main abstraction unit in ParSeq system to represent an asynchronous operation whose result can be set asynchronously. Promise: represents the state of an asynchronous operation and also a container for the operation result. Task Callable: it is the main ingredient of Task, which is like a Callable starting asynchronous operation and returns a Promise. Finally, ParSeq engine, which is a harness responsible for running a Task.
- #40 ParSeq programming model is declarative and functional, focusing on What you want to do, not How you get that. It provides developer-friendly API for coordinating asynchronous tasks, for example, task creation, parallel/sequential composition, transformation, error handling, etc. It ends up with one Task (internally represented as an execution plan) that will be submitted to ParSeq Engine to execute.
- #41 ParSeq execution model maintains a very important invariant to relieve developers from various asynchronous programming concerns. That is, All Task Callables are executed sequentially, such that Callable of a previous Task in a sequence “happens before” (in Java Memory Model sense) Callable of a next Task. This is a real example of Task Callable from sending a rest.li request.
- #43 Another valuable feature provided by ParSeq is its built-in tracing and visualization. This is very important for service monitoring and observability. Trace visualization provides us a powerful tool for debugging some problematic application flow, also can provide us performance metric on each step in your business logic.
- #44 To promote our clean REST practices to heterogenous system and diverse clients, we are striving to expand our multi-language support. So far, we have already had support for Java desktop, Android Java, Objective-C/Swift for iOS, Python, C# support. Go client binding support is working in progress.
- #45 Here is an overview matrix of comparing Rest.li framework with other popular RESTful framework on the market. I just picked some representatives, but they are more or less similar.
- #46 The rest.li framework has been proven to create robust RESTful service architectures that can work at scale from its wide adoption at LinkedIn and Coursera. There are still many interesting items on our road map, for example: - Object streaming and unstructured data support in rest.li Multi-language suppprt enhancement including Go and cross-language test suites. Open source our Deco library. In ParSeq, we are planning to enhance it with friendly collection API, and also power ParSeq Tracing with Intelligent data Analytics. We are looking forward to contributions from our audience here.