Principles of REST API Design
•Download as PPTX, PDF•
2 likes•1,584 views
Presented by Amy Wasik, Two Sigma Investments, LP
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Principles of REST API Design
Similar to Principles of REST API Design (20)
More from Two Sigma
More from Two Sigma (20)
Recently uploaded
Recently uploaded (20)
Principles of REST API Design
- 1. www.twosigma.com Principles of REST API Design May 19, 2017 Presented By: Amy Wasik Two Sigma Investments, LP Important Legal Information: This document is being distributed for informational and educational purposes only and is not an offer to sell or the solicitation of an offer to buy any securities or other instruments. The information contained herein is not intended to provide, and should not be relied upon for investment advice. The views expressed herein are not necessarily the views of Two Sigma Investments, LP or any of its affiliates (collectively, “Two Sigma”). Such views reflect significant assumptions and subjective of the author(s) of the document and are subject to change without notice. The document may employ data derived from third-party sources. No representation is made as to the accuracy of such information and the use of such information in no way implies an endorsement of the source of such information or its validity. The copyrights and/or trademarks in some of the images, logos or other material used herein may be owned by entities other than Two Sigma. If so, such copyrights and/or trademarks are most likely owned by the entity that created the material and are used purely for identification and comment as fair use under international copyright and/or trademark laws. Use of such image, copyright or trademark does not imply any association with such organization (or endorsement of such organization) by Two Sigma, nor vice versa.
- 2. Software Architecture May 19, 2017 Single Server Services REST Services
- 3. Our Application May 19, 2017 Application Job Job Job Job Host
- 4. Our Application May 19, 2017 Application Job Job Job Job Host Application Job Job Job Job Host Application Job Job Job Job Host Application Job Job Job Job Host Application Job Job Job Job Host Application Job Job Job Job Host Application Job Job Job Host Application Job Job Job Job Host
- 5. Our Application May 19, 2017 Application Job Job Job Job Host Monitor Monitor Host Host
- 6. Our Application May 19, 2017 Client API Client Client Client Host Host Host Host Scheduler Job Job Job Job Host Monitor Monitor Host Host
- 7. Software Architecture May 19, 2017 Single Server Services REST Services
- 8. Services Architecture May 19, 2017 • Low Coupling • Maintainable • Interoperable • Language Agnostic • Shareable • Scalable • Resilient
- 9. Services Architecture May 19, 2017 • Low Coupling • Maintainable • Interoperable • Language Agnostic • Shareable • Scalable • Resilient Client API Client Client Client Host Host Host Host Scheduler Job Job Job Job Host Monitor Monitor Host Host
- 10. Services Architecture May 19, 2017 • Low Coupling • Maintainable • Interoperable • Language Agnostic • Shareable • Scalable • Resilient Client API Client Client Client Host Host Host Host Scheduler Job Job Job Job Host Monitor Monitor Host Host
- 11. Services Architecture May 19, 2017 • Low Coupling • Maintainable • Interoperable • Language Agnostic • Shareable • Scalable • Resilient High-level Javascript Low-level Client API Client Client Client Host Host Host Host Scheduler Job Job Job Job Host Monitor Monitor Host Host
- 12. Services Architecture May 19, 2017 • Low Coupling • Maintainable • Interoperable • Language Agnostic • Shareable • Scalable • Resilient Application 2 Client API Client Client Client Host Host Host Host Scheduler Job Job Job Job Host Monitor Monitor Host Host
- 13. Services Architecture May 19, 2017 • Low Coupling • Maintainable • Interoperable • Language Agnostic • Shareable • Scalable • Resilient Client API Client Client Client Host Host Host Host Scheduler Host Job Job Job Host Monitor Monitor Host
- 14. Services Architecture May 19, 2017 • Low Coupling • Maintainable • Interoperable • Language Agnostic • Shareable • Scalable • Resilient Client API Client Client Client Host Host Host Host Scheduler Job Job Job Job Host Monitor Monitor Host Host
- 15. Communication May 19, 2017 Single Server: Local API Method Calls Services: Inter-process Communication (IPC)
- 16. Inter-Process Communication (IPC) May 19, 2017 • Remote Procedure Calls (RPC) • Use a library to convert local calls to remote ones public interface Jobsystem { Job createAJob(JobDetails details); void submitJob(Job j); List<Job> getJobs(String namePattern); List<Job> getMyJobs(String user); List<Job> getJobsOther(String query); Job getJob(int id); void updateJob(JobDetails details); }
- 17. Software Architecture May 19, 2017 Single Server Services REST Services
- 18. REST Architecture May 19, 2017 Additional Constraints Benefits Stateless Scalability • Representational State Transfer (REST)
- 19. REST Architecture May 19, 2017 Additional Constraints Benefits Stateless Scalability Cacheable Increased Capacity • Representational State Transfer (REST)
- 20. REST Architecture May 19, 2017 Additional Constraints Benefits Stateless Scalability Cacheable Increased Capacity Layered Low Coupling/Interoperability • Representational State Transfer (REST)
- 21. REST Architecture May 19, 2017 https://martinfowler.com/articles/richardsonMaturityModel.html 4 Levels of Adherence Benefits 0 – HTTP Transport 1 – Resource Oriented Design 2 – HTTP Verbs as actions on resources 3 – Hypertext as the Engine of Application State (HATEOAS) • Representational State Transfer (REST) • API Constraints
- 22. REST Architecture May 19, 2017 https://martinfowler.com/articles/richardsonMaturityModel.html 4 Levels of Adherence Benefits 0 – HTTP Transport Standard Interface 1 – Resource Oriented Design Easier-to-Use API 2 – HTTP Verbs as actions on resources Complete API 3 – Hypertext as the Engine of Application State (HATEOAS) Easy-to-Learn API • Representational State Transfer (REST) • API Constraints
- 23. REST Level 0 May 19, 2017 HTTP Transport Readable object encoding (typically JSON) Standard URI format
- 24. Level 0: RPC over HTTP May 19, 2017 public interface Jobsystem { Job createAJob(JobDetails details); void submitJob(Job j); List<Job> getJobs(String namePattern); List<Job> getMyJobs(String user); List<Job> getJobsOther(String query); Job getJob(int id); void updateJob(JobDetails details); } GET http://example.com/createAJob?name=t&... GET http://example.com/submitJob?id=123 GET http://example.com/getJobs?name=test GET http://example.com/getMyJobs?user=me GET http://example.com/getJobs?query=... GET http://example.com/getJob?id=123 GET http://example.com/updateJob?id=123&
- 25. Level 0: RPC over HTTP May 19, 2017 GET http://example.com/createAJob?name=t&user=userA... HTTP/1.1 200 OK [other headers] { "id": 123 } GET http://example.com/submitJob?id=123 HTTP/1.1 200 OK [other headers] { "error" : "no permission" }
- 26. REST Level 1 May 19, 2017 Resource Oriented Design • Divide and conquer • Easy to understand and navigate API Standard URI Format • /{resource} • /{resource}/{resource-id} • /{resource}/{resource-id}/{sub-resource} • /{resource}/{resource-id}/{sub-resource}/{sub-resource-id}
- 27. Object Oriented Design May 19, 2017 GET http://example.com/createAJob?name= GET http://example.com/submitJob?id=12 GET http://example.com/getJobs?name= GET http://example.com/getMyJobs?user= GET http://example.com/getJobs?query= GET http://example.com/getJob?id=123 GET http://example.com/updateJob?id=123 GET http://example.com/jobs/create?name=t&user=me GET http://example.com/jobs/get?name=test GET http://example.com/jobs/getMy?user=me GET http://example.com/jobs/get?query=... GET http://example.com/jobs/123 GET http://example.com/jobs/123/update?name=t2... GET http://example.com/jobs/123/instances/start GET http://example.com/jobs/123/instances
- 28. REST Level 2 May 19, 2017 HTTP Verbs Represent Actions • More complete and structured APIs Common Verbs • GET – Read (Nullipotent) • PUT – Update (Idempotent) • POST - Create • DELETE – Remove (Idempotent)
- 29. REST Level 2 May 19, 2017 Standard HTTP Response Codes • Standard results of actions Success Client Error Server Error 200 OK 400 Bad Request 500 Internal Server Error 201 Created 401 Unauthorized (authentication failure) 204 No Content 403 Forbidden (not allowed access) 404 Not Found
- 30. HTTP Verbs for Actions May 19, 2017 GET http://example.com/jobs/create?name=t&user=me GET http://example.com/jobs/get?name=test GET http://example.com/jobs/getMy?user=me GET http://example.com/jobs/get?query=... GET http://example.com/jobs/123 GET http://example.com/jobs/123/update?name=t2... GET http://example.com/jobs/123/instances/start GET http://example.com/jobs/123/instances POST http://example.com/jobs -d '{"name":"test", "user": "me", …}' GET http://example.com/jobs?name=test GET http://example.com/jobs?user=me GET http://example.com/jobs?query=... GET http://example.com/jobs/123 PUT http://example.com/jobs/123 -d '{"name":"job" …}' POST http://example.com/jobs/123/instances GET http://example.com/jobs/123/instances
- 31. HTTP Verbs for Actions May 19, 2017 POST http://example.com/jobs -d '{"name":"test", "user": "me", …}' HTTP/1.1 201 Created [other headers] { "id": 123 } POST http://example.com/jobs/123/instances HTTP/1.1 403 Forbidden [other headers] { "errorCode" : 10, "moreInfo" : "no permission to run this job" }
- 32. REST Level 3 May 19, 2017 REST API Documentation and API Discoverability • Hypertext As The Engine Of Application State (HATEOAS) — Adds links to response that indicate useful actions • Open API — Provides language-agnostic way to describe REST API — Lots of tooling for automation
- 33. Open API May 19, 2017
- 34. Open API May 19, 2017
- 35. Evolution of the API May 19, 2017 public interface Jobsystem { Job createAJob(JobDetails details); void submitJob(Job j); List<Job> getJobs(String namePattern); List<Job> getMyJobs(String user); List<Job> getJobsOther(String query); Job getJob(int id); void updateJob(JobDetails details); }
- 36. Evolution of the API May 19, 2017
- 37. Conclusion May 19, 2017 • Modern day best practices • Services architectures • REST APIs • Resource Oriented Design • Self-documenting code • Next steps • Evolving APIs • Complex operations • Error handling, Standard response types
- 38. Additional Resources May 19, 2017 http://swagger.io/ https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm https://martinfowler.com/articles/richardsonMaturityModel.html Published API Guides: https://pages.apigee.com/rs/apigee/images/api-design-ebook-2012-03.pdf https://github.com/paypal/api-standards/blob/master/api-style-guide.md https://cloud.google.com/apis/design/ Thank you!
Editor's Notes
- Do you consider yourself a programmer, scientist, architect, designer, writer? We have many roles and titles but what we all have in common is that we like to use technology to solve challenging problems. Introduce - Myself – undergrad and masters in computer science, then came to Two Sigma – a technology company that does finance Worked on many levels of software, from low level performance tuning, beautiful Uis, to looking at software from higher level architecture Now work on our platform for building, deploying, and managing services, and some generally useful services such as the one I’m going to describe now
- Looking at software from a high level we talk about the software architecture – how the app is designed In college you build apps that run on a single server. In this talk I’ll take you through the why and how you go from that single server app to a services architecture which is what you commonly see in industry, to a REST services architecture which is what I call beauty.
- Example: Our application runs jobs on a schedule. A job is a program that runs for a short to medium time on the order of minutes to hours and can span from everything from downloading, processing and transforming data, generating reports, starting and priming critical systems, and running all kinds of verification data and system checks. Typically you start with an application as a single program that you run on a single machine But then you realize you have problems- It gets really popular or big and can’t fit on one machine You could replicate this over a second machine and have one type of jobs on each and remember where they all are. This gets annoying when you have 10 machines and have 10 interactive terminals open to each. It gets impossible when you have 1000 machines.
- You could replicate this over a second machine and have one type of jobs on each and remember where they all are. This gets annoying when you have 10 machines and have 10 interactive terminals open to each. It gets impossible when you have 1000 machines. At TS we run 100s of thousands of jobs a day on 1Ks of machines.
- So you start to break out different pieces of the software to run on separate machines and with multiple copies Now we can interact with one application and it knows how to interact with jobs on all our machines.
- And we may want to separate the part that interacts with the processes and that part that interact with clients. Oh and it’s nice to have a browser interface so the client is actually running in a browser on the user machine.
- This is called a services software architecture
- It has many benefits…
- Low coupling: few interaction points between different components strongly enforced interfaces Makes your system less complex Eg. when running the app on one host you could make assumptions about the host your tasks run on, the location of certain files, etc. this can be very brittle and error prone, separating the interfaces across hosts makes it very difficult or impossible to get around the interface and make bad assumptions
- Maintainable: can upgrade each piece independently Interoperable: can easily replace each piece independently
- Language agnostic: can write different pieces in different languages
- Shareable: can reuse service components with other apps
- Scalable: can scale each piece independently
- Resilient: can provide degraded functionality in cases of failure Lots of moving pieces, means more complexity.
- Services architecture brings in new complications, one of them being Inter process communication, which I’ll focus on.
- How do the different components pass data and state around? Basic approach is RPC make a call as you would normally to another object We thought RPC was the answer to make it easy and transparent you were in a services architecture Problem: You have to know how to deal with failures, which don't exist in local calls Not opinionated to get full benefits of services architecture No structure can lead to spaghetti code
- The answer is REST
- REST Adds more constraints to IPC that helps you design your API to gain the benefits of Services 3 Constraints on the communication Stateless - no client state is stored on the server, all information for a request is passed in the request, eg. can’t support a call like nextLine Makes it so you can scale
- Cacheable - responses must state themselves as cacheable or not, clients and intermediaries may cache results
- Layered - clients may not be directly connected to server, and neither should care
- Additionally there are constraints on the API – these are not always followed, but modern implementations use this as a way to define how "RESTful" an API is A model (developed by Leonard Richardson - RMM defines different levels of adherence to REST principles in the implementation Each of these are additive
- REST APIs can follow any level of these, and even within these there are best practices. The more adherence to these levels and best practices, the more beautiful your API Like making music in an orchestra the constraints allow us to be creative within a framework that helps us easily make beautiful music. We’ll look at them in more detail next
- Level 0 HTTP as a transport is a common standard, lots of standard tooling, using standard readable text formats like JSON makes it easy to debug and test in a browser or using basic curl commands Standards for how to access the methods, read the data, and understand responses HTTP is a long lasting standard. RPC libraries and protocols come and go, HTTP stands the test of time.
- Level 0 is also sometimes called RPC over HTTP. It is not very RESTful
- Level 0 is also sometimes called RPC over HTTP. It is not very RESTful
- Level 1 tackles the question of handling complexity by using divide and conquer, breaking a large service endpoint down into multiple resources. Splitting the API into a hierarchy of resources makes each of the sections small, simple and easier to navigate The API is split using a URI hierarchy with common best practices for how to structure it
- Think of your resources as concepts you can take action on. They often map to tables in a database, but don’t necessarily. Especially if you map transient transactions as a resource. For example, we can think of a job as two separate resources, one is the job definition, and one is the job instance, a specific run of the job definition There are common URI structures that are best practices rather than enforced
- Level 2 introduces a standard set of verbs so that we handle similar situations in the same way, removing unnecessary variation. Usage the HTTP verbs to represent the action rather than random names in the URI Verbs have useful meanings, mostly obvious GET read or retrieve a resource. Nullipotent - GET requests have no side effects. PUT is idempotent – used for update a resource or update or add if not found (if client provides ID) POST create new resources. POST assigns an ID to the new resource. POST is not idempotent. Two identical POST requests will result in two resources with same information. DELETE is used to delete a resource, if already deleted can return 204 The side effects of these actions are really useful for clients in how to handle errors
- Common response codes
- Now we can really clean up our API
- Now we can really clean up our API
- Level 3 introduces discoverability, providing a way of making a protocol more self-documenting. Most don’t follow HATEOS, alternative with same benefit of discoverability is Open API, FKA Swagger Language agnostic way to describe REST APIs in a YAML or JSON text file Why OpenAPI? Provides an alternative easy to write/read way to document your API in its entirety Lots of tooling for automating server and client code generation, and visualizing documentation
- As a reminder we started with this… Used to think APIs were from computers
- … to this But really they're for humans. A more complete, well thought out, easy to navigate, easy to learn about, and easy to use from anywhere API
- What we’ve learned – some modern day best practices There is more we don’t have time to go into about conventions of best practices in REST API design including Standards for naming conventions of resources and handling complex operations Standard ways to return error responses with more context and detailed messages Standards for returning common complex objects such as lists and pages of items