Recommended
Recommended
More Related Content
Similar to Consuming REST APIs for all interpretations of REST
Similar to Consuming REST APIs for all interpretations of REST (20)
More from darrelmiller71
Recently uploaded
Recently uploaded (20)
Consuming REST APIs for all interpretations of REST
- 2. CONSUMING REST APIS FOR ALL INTERPRETATIONS OF REST @darrel_miller
- 7. TODAY’S MENU API Economy Client SDKs Appetizers Main Course Dessert Interpretations of REST Change is inevitable and good Code reuse on the web The secret of REST clients Samples and Libraries to explore
- 8. API ECONOMY
- 10. QUALITY + QUANTITY = $$$
- 12. DON’T ASSUME THE PROVIDED SDK IS THE BEST OPTION CHEF’S SUGGESTION UNDERSTAND YOUR DEPLOYMENT CONSTRAINTS
- 13. WHAT DO YOU MEAN BY REST?
- 14. DESIRED EFFECT?
- 15. DESIRED EFFECT? Client/Server Stateless Caching Uniform Interface Layered Code On Demand Scalable Evolvable Fault Tolerant Composable
- 18. APIS DONE OUR WAY application/vnd.github.v3+json application/vnd.heroku+json; version=3
- 20. UNDERSTAND THE PURPOSE OF MEDIA TYPES CHEF’S SUGGESTION BE AWARE OF THE EFFECTS OF YOUR DESIGN CHOICES
- 24. DATA ORIENTED APIS ARE GREAT FOR QUICK, EASY AND INTERNAL OR GOVERNMENT DATA CHEF’S SUGGESTION SCENARIO ORIENTED IF YOU WANT TO MAKE MONEY
- 25. CHANGE IS GOOD
- 26. START SMALL, EVOLVE QUICKLY
- 27. START SMALL, EVOLVE QUICKLY
- 28. START SMALL, EVOLVE QUICKLY
- 29. VERSIONING HURTS
- 30. A LITTLE PARANOIA CAN BE USEFUL
- 31. CHANGE IS GOOD CHEF’S SUGGESTION VERSION AS A LAST RESORT
- 32. SOFTWARE’S HOLY GRAIL Achieving reuse on the web is different…
- 33. API REUSE VERSUS WEB REUSE
- 36. MAYBE ONE DAY…
- 37. BEWARE MEDIA TYPE EXPLOSION
- 38. SUPPORT MULTIPLE MEDIA TYPES
- 39. CONSIDER HYPERMEDIA TO ENABLE CLIENT CODE REUSE CHEF’S SUGGESTION UNDERSTAND THE CONTRACTS YOU ARE USING
- 40. THE SECRET OF REST CLIENTS Client code dictates coupling
- 41. BREAKING REQUEST FROM RESPONSE Create Request Process Request Handle Response
- 42. CENTRALIZED RESPONSE HANDLING Create Request Process Request HTTP Response Machine
- 43. CONTEXT IS KING Process Request HTTP Response Machine Dispatch on : Status Code Media Type Link Relation Type
- 45. CLIENT STATE MANAGEMENT Create Request Process Request HTTP Response Machine Client State Model Client View Application Controller
- 46. BREAKING REQUEST/RESPONSE COUPLING BRINGS FLEXIBILITY CHEF’S SUGGESTION LINK RELATION TYPES ARE YOUR FRIEND APPLY RESPONSES TO TRANSFORM CLIENT STATE
- 47. Using Hypermedia to avoid the app store – https://www.youtube.com/watch?v=LbSM8U21YkM Crafting Evolvable API Representations – https://vimeo.com/131643022 Succeeding In Failing – https://vimeo.com/131632607 Dot Net Fringe - ??? RELATED TALKS
Editor's Notes
- Welcome. Let’s first thank all of the sponsors who have made this event possible. Thank you for taking the time out of your weekend to come and enhance your knowledge.
- Thank you for talking time out of your weekend to come and participate. This talk is about consuming REST apis for all interpretions of REST. There are many talks you can hear about how to build REST APIS, but not many people are talking about how to consume them Possible because people think there is nothing to talk about. Hopefully, this talk will convince you that it’s worth talking about it. Whether your are a full blown RESTafarian, or you think Swagger is pure awesome. This talk will provide you with some food for thought. My name is Darrel Miller and…
- I am Canadian Do we have Canadians in the audience? But I was born in England.
- Any Brits here? Being an Canadian Brit, just means, when I say sorry, I don’t really mean it. This is my first time here in Australia. I’m very excited about that. I have a cousin who moved here recently, who I’m going to see on Sunday. I haven’t seen in 17 years. Well, it’s not like I was going to go back to England to visit her Australia, England, Canada…. You know what they have in common, right?
- They all use http! This is where my true allegiance lies. Sorry Lizzy. Almost 10 years ago, and I was getting tired of chasing frameworks. I had just upgraded .net framework. Had ASMX services that were consumed by VB6 on XP Embedded and.net CF on CE devices I discovered I had to upgrade the Web Services SOAP Toolkit on clients… sigh Longing for WCF to solve all my distributed systems problems… yeah… HTTP hadn’t changed in a long time and it didn’t look like it was going to change any time soon. So I decided to invest some time in really learning how it works. It has paid off tremendously. HTTP is still unchanged. HTTP/2 does not change its semantics, simply makes it more efficient to use. For the next 40 mins we are going to talk about consuming HTTP APIs, what we are doing wrong, and how I think we can do it better.
- And in the spirit of my love for HTTP, No talk would be complete without a reference to an obscure status code. The content I will be presented should be considered as coming with a 203 status code. <click> 203 means that the content has been, transformed, processed, interpreted, in some way. In this case, by me. You have been warned….
- As the talk is called consuming REST APIs, and because I always seem to gravitate to food analogies when talking about APIs, I decided we should make this a theme. So, for appetizers, we are going to discuss where we are today The state of the API Economy The use of client SDKs to consume APIs Main Course, we will discuss challenges and solutuioins - We will discuss the challenges of consuming a REST api when people have so many different interpretations of what REST means. We will consider the effect of change. The reasons for it and why you should protect your clients from changing APIs Code reuse is always a high priority goal in software development. We will discuss the constructs of HTTP that deliver re-use most effectively. One of the primary goals of REST is to allow independent evolution of system components though the use of loose coupling. But the client has some unique properties in this relationship. We will discuss some of those secrets. We will discuss some architecture concepts to consider when building clients that will help to make your clients more adaptable to change. Dessert - Some additional resources you can use to dig down deeper into the technical aspects of the concepts
- The API is the product Sold to companies … integrate external functionality into own Stripe is the canonical example… Products can now be build by combining services provides by other companies. Just like our delicious looking soup is made by combining ingredients to make a meal. It is becoming a massive industry 13,000 publicly accessible APIs listed on Programmable Web Salesforce generates 50% of its $3 billion revenue from its APIs Distributed APIs as ans integration technique has become so popular, we now have microservices which is bringing the technique within product development itself.
- So how can we access the API Economy. Most developers consume APIs via client SDKs Eating a salad is healthy choice until you cover it with dressing and croutons. Don’t hide the goodness SDKs are considered essential by API providers for developer adoption - Unfortunately SDKs, depending on how they are built, bring their own set of problems - Can negate benefits of HTTP API by being an RPC layer that sits between the client application and the HTTP API - RPC is not a characteristic of an API, but a style of mechanism to invoke the remote API - explain - Moving point of coupling to the SDK - This can create tight coupling between the client application and the version of the SDK. - Change the SDK, break the client application. - I believe it is possible it build tools to help users consume an HTTP API without hiding the HTTP
- Building lots of client SDKs is hard to do well. - Need to build so many! - Focus on the primary platforms and let the rest be second class citizens. Last to update Many choose to use code generators, tied to API description languages. It’s really hard to code gen good client code. Especially across multiple platforms. Can end up with Lowest common denominator. The question is, how can afford to build good quality libraries and still many platforms
- Sometimes tightly coupled client SDKs are the right solution If you can maintain complete control of both the ingredients in the salad dressing and how much is put on the salad, then maybe you are OK. In terms of SDKs: Who owns each end of the wire? Can deployments be coordinated? Same team? App Store Can a user choose not to update? What happens during the change over? JS clients consuming their own API What about next week? But what about when a 3rd party starts to consume
- O
- While consuming APIs…many different defns of REST. Useful to understand why… Chocolate Chip cookie analogy: Many different recipes for CC Cookies. So what defines a CC cookie? (answer) Consider those the constraints Challenge is that devs wants a recipe. Framework designers to the rescue. They create a recipe for REST API Devs mistake the recipe for REST New rules get invented based on framework conventions and limitations. The effect is what is important…
- Doesn’t this kid look happy? The recipe is not so important as long as the constraints are respected. REST is the same, just different constraints and effects.
- REST defined by constraints -> Effects are what you want -> Sometimes, not all effects required, => some constraints can be dropped. Unfortunately term REST used to describe subset of constraints. Leads to confusion because developers don’t get all effects Need to focus on the effects we desire and the constraints that achieve it, and stop getting hung up on the name. Remember: Kids love Oatmeal too. And there are no chocolate chips in those. Understand the costs and benefits.
- Probably the most common style of “REST APIs” you will run into is what I call JSON-REST. Poutine – A Quebec delicacy of French fries, cheese curds and gravy They are quick, easy and cheap to make. Long term consumption may be detrimental to your health Documentation tells you everything about a specific URL: What will come back, what to send, what methods, what status codes to expect. The documentation is effectively the contract. We are used to this approach in local computing. But Distributed systems are different. Description languages are being introduced because managing this contract is hard. Servers can’t changing without breaking clients, so versioning is necessary. Technically it isn’t what was originally described as REST because… Failure of the self-description constraint and affects evolvability. That may not be a problem for you.
- Hypermedia can help evolution by adding a layer of decoupling But application/json as content-type still leaves a lots of prearrangement Aka out of band coupling La Banquise
- Some APIs are starting to document their API conventions as a media type This is great step forward as media types are the contracts in REST. But doesn’t exactly promote sharing and re-use outside of the company’s walls Accept header!
- Another approach you may run into while consuming APIs involves the use of Generic hypermedia types Also, help to solve the problem of creating and documenting API conventions. Links, forms, lists, embedded resources Application Semantics are missing Schemas, Profiles, Vocabularies, Ontologies,
- Understand how the constraints you choose to apply affect the system you are building Even if you only ever plan on using application/json, know what the cost is.
- You need to understand the style of API that you are accessing. Data Oriented APIs are the buffets of APIs. Also referred to as CRUD Apis. The primary purpose of this type of API is to give you simple access to all the data owned by the API There is no assumptions about why you want to access it, or how you might combine that data. It’s fairly easy to do, doesn’t add much value to the data itself, and has performance challenges, due to the open ended way you can access the data.
- At the other end of the API spectrum you have scenario based APIs. Some foods are designed for special occasions, like this birthday cake It is easier to focus on solving a single problem and add new scenarios as required Best way to keep control over performance
- Hypermedia works well for this type of API The hypermedia guides the API consumer through the application workflow However, clients are very different because they must follow a ritual to achieve the intended goal Clients should only make requests when presented with links to follow Link relations are the point of coupling for discovering available links Fairly steep learning curve… and Catch-22. Designing a good hypermedia API requires experience consuming one. Getting good experience consuming one requires access to a good one.
- Often
- Whereas restaurants change menus for variety, To encourage regular visitors to come back. APIs change because they are actively being used and users want to do more. This is a good thing.
- Most good APIs should start small and grow based on customer feedback. If you can do this, then it is easy to progress and do this…
- And before too long you will have…
- This. Trying to build a complete solution up front is the API equivalent of a waterfall design. It’s much safer to take the agile approach where we build a MVP first and then enhance. Guarantees we give what the customer needs And continue to provide good performance But it requires accepting that change is the norm. Unfortunately, …
- We have a false assumption that versioning is the primary way of dealing with change. But this comes from our local code mindset Versioning is painful – Versioning is a way for server developers to push their problems onto client developers Unless you are going to stay on an old API version, then you are going to have to update client code. Why not write the client to withstand changes and then versions become irrelevant? A little extra effort in building clients can go a long way to making the client resilient to change.
- Be paranoid about your server side API developers. Do not trust them, no matter what promises they make. Be defensive in how you build clients: Don’t assume ordering or presence of properties Don’t assume returned content type Don’t assume URLs will not change It’s not that API developers are evil. They just have internal forces that make them do things they were not expecting to do, and stuff out of their control. Change happens.
- Holy Grail of the software industry Path to higher productivity The web has been successful because of code re-use. Browser/javascript API world is struggling to achieve significant levels of re-use Partly because applying local system code reuse techniques Code based systems use inheritance, interfaces and libraries with shared types But API consumers and providers have different platforms, languages and type systems. Still searching for solutions Many Efforts to produce description languages, search engines and even patents Few examples of people re-using APIs in the wild. Metablog APIs are not an effective unit of reuse on the web
- 2 servers, 2 API defns. All or nothing. What about partial? What about a readonly? Composition can be ugly Client integration. Media Types and link relations are unit of re-use Integrations can be presented to client Server boundaries are transparent We can reuse a single resource. To maximize reuse we must try and minimize the knowledge contained in a message. Client is on a need to know basis. Can you interact with something without knowing the “meaning” of the content? Let me give an example
- Example from electronics industry. LittleBits The sensors don't care what is producing the light or heat, or why. only care if the measurement exceeds a threshold, at which point a switch is flipped. Bad habit of expose everything because we don't know what a client wants. This severely inhibits reuse. Achieving reuse is about focusing on the things that matter and avoiding things that don’t.
- Mentioned that m/ts are unit of reuse Useful analogy for role of m/ts is cooking equipment Can you cook with only one type of pan? Different pots and pans have different uses But not limited to a single food. Boil potatoes and heat soup in a pot, Fry an egg, or cook bacon in a pan Each address a class of problem. Optimized for simplicity and reusibility
- Star-Trek Food Replicator One device that can produce any kind of food. Generic media types become complex And that’s before we’ve even started to address application semantics For that we need profiles, schemas and namespaces Until we get there, let’s not forget we can use simple pots and pans to get the job done.
- Obviously you can take this too far. Too much spec writing Be wary of overly specific media types. Sometimes it does make sense, making a waffle without a waffle maker is hard. They make re-use hard.
- Consequence is clients must support diff media types Media types used can change over time / Over time move to new format Helps with backward compatibility Can be used to manipulate workflow…. Question example
- You’ll be surprised what you can achieve Implicit or explicit contracts
- Clients can tightly couple to hypermedia driven server Clients can minimize coupling with JSON REST API Client code is key for evolution. Server code helps to enable it. A V2 API doesn’t necessarily mean you need to Create a whole new client version Stop supporting V1 Supporting multiple “versions” in a single client is useful Prod, vs stage, vs test. Multi-tenancy This is why it is so important to build flexible clients
- Omakase – A style of ordering in Sushi restaurants Customers express intent by requesting Omakase style Chef gets to implement that request in the most effective way This flexibility gives the possibility for the chef to deliver the highest quality food in stock for less than the regular price. Letting go of some level of control brings flexibility Breaking request creation and response handling can bring flexibility to client applications. Responses must be self-descriptive Naturally asynchronous Testing is easier Facilitates centralized response handling
- Uniform interface shared by all components We can take advantage of that Get code reuse by centralizing handling HTTP Response Machine Every status code should be handled, for all requests One place for Redirects, Retries, Authentication challenges Inline handling can easily introduce hidden coupling Pet peeve of declaring status codes Support ranges of status codes Account for all codes. Don’t assume what is being returned Forget to look at headers
- Link Relations provide context to your content Responses are handled independent of context. That’s not quite true. The link relation is the identifier that provides you with the necessary context e.g. Stylesheet Media Types and Link Relations when combined can be very powerful Angry dog When building clients, don’t forget to take advantage of this context Your client doesn’t need to care what is in the href, the rel is what is important. Type is only a hint.
- APIs that implement HATEOAS require a fairly different way of thinking about building clients. One of the most useful tools it to an implement a client side class that encapsulates the entire client state Consider it like a ViewModel for your entire application if you are familiar with that term.
- Application Controller waits for some input. Creates a HTTP request Sends it the HTTP connector HttpResponseMachine handles the response Applies the returned representation to the client state. Client state fires events based on changes that happen Client view is updated. Application Controller waits for some input Using this approach, the server can completely change its responses to requests and the client doesn’t break. Example if time…. (Question, expense receipts)
- Often
- Covered a large range of concepts If you want to dig into more details here some links Demo of hypermedia as the engine of application state to reduce client updates More talk about avoiding the need to versioning Standardized handling and testing error responses Talk and demo on the tooling
- Over the past few years I’ve produced a number of different open source tools for working with HTTP APIs The burden of managing the nugets for each of these, along with all the supported platforms (incl portable) and tracking the dependencies between them has been sucking up my time. So, I decided to aggregate all the relevant libraries together and produce a single, simpler, kit for building client SDKs. This is still very much a work in progress, but hopefully one day in the near future, this will be available for anyone to quickly build high quality, robust and feature rich SDKs.
- Warning: Under construction!
- Bon Appetit and Thank you for listening. I will be around all day, if you have any questions, or just want to comment on or challenge, my assertions, I am open to all feedback.