7. TODAY’S MENU
Interpretations of REST
Change is inevitable and good
Code reuse on the web
The secret of REST clients
Samples and Libraries to
47. Using Hypermedia to avoid the app store –
Crafting Evolvable API Representations –
Succeeding In Failing –
Dot Net Fringe - ???
Let’s first thank all of the sponsors who have made this event possible.
Thank you for taking the time out of your weekend to come and enhance your knowledge.
Thank you for talking time out of your weekend to come and participate.
This talk is about consuming REST apis for all interpretions of REST.
There are many talks you can hear about how to build REST APIS, but not many people are talking about how to consume them
Possible because people think there is nothing to talk about.
Hopefully, this talk will convince you that it’s worth talking about it.
Whether your are a full blown RESTafarian, or you think Swagger is pure awesome.
This talk will provide you with some food for thought.
My name is Darrel Miller and…
I am Canadian
Do we have Canadians in the audience?
But I was born in England.
Any Brits here?
Being an Canadian Brit, just means, when I say sorry, I don’t really mean it.
This is my first time here in Australia. I’m very excited about that.
I have a cousin who moved here recently, who I’m going to see on Sunday.
I haven’t seen in 17 years. Well, it’s not like I was going to go back to England to visit her
Australia, England, Canada…. You know what they have in common, right?
They all use http!
This is where my true allegiance lies. Sorry Lizzy.
Almost 10 years ago, and I was getting tired of chasing frameworks.
I had just upgraded .net framework.
Had ASMX services that were consumed by VB6 on XP Embedded and.net CF on CE devices
I discovered I had to upgrade the Web Services SOAP Toolkit on clients… sigh
Longing for WCF to solve all my distributed systems problems… yeah…
HTTP hadn’t changed in a long time and
it didn’t look like it was going to change any time soon.
So I decided to invest some time in really learning how it works.
It has paid off tremendously. HTTP is still unchanged.
HTTP/2 does not change its semantics, simply makes it more efficient to use.
For the next 40 mins we are going to talk about consuming HTTP APIs, what we are doing wrong, and how I think we can do it better.
And in the spirit of my love for HTTP,
No talk would be complete without a reference to an obscure status code.
The content I will be presented should be considered as coming with a 203 status code.
203 means that the content has been, transformed, processed, interpreted, in some way.
In this case, by me.
You have been warned….
As the talk is called consuming REST APIs, and because I always seem to gravitate to food analogies when talking about APIs, I decided we should make this a theme.
So, for appetizers, we are going to discuss where we are today
The state of the API Economy
The use of client SDKs to consume APIs
Main Course, we will discuss challenges and solutuioins
- We will discuss the challenges of consuming a REST api when people have so many different interpretations of what REST means.
We will consider the effect of change. The reasons for it and why you should protect your clients from changing APIs
Code reuse is always a high priority goal in software development. We will discuss the constructs of HTTP that deliver re-use most effectively.
One of the primary goals of REST is to allow independent evolution of system components though the use of loose coupling. But the client has some unique properties in this relationship. We will discuss some of those secrets.
We will discuss some architecture concepts to consider when building clients that will help to make your clients more adaptable to change.
- Some additional resources you can use to dig down deeper into the technical aspects of the concepts
The API is the product
Sold to companies … integrate external functionality into own
Stripe is the canonical example…
Products can now be build by combining services provides by other companies. Just like our delicious looking soup is made by combining ingredients to make a meal.
It is becoming a massive industry
13,000 publicly accessible APIs listed on Programmable Web
Salesforce generates 50% of its $3 billion revenue from its APIs
Distributed APIs as ans integration technique has become so popular, we now have microservices which is bringing the technique within product development itself.
So how can we access the API Economy.
Most developers consume APIs via client SDKs
Eating a salad is healthy choice until you cover it with dressing and croutons.
Don’t hide the goodness
SDKs are considered essential by API providers for developer adoption
- Unfortunately SDKs, depending on how they are built, bring their own set of problems
- Can negate benefits of HTTP API by being an RPC layer that sits between the client application and the HTTP API
- RPC is not a characteristic of an API, but a style of mechanism to invoke the remote API - explain
- Moving point of coupling to the SDK
- This can create tight coupling between the client application and the version of the SDK.
- Change the SDK, break the client application.
- I believe it is possible it build tools to help users consume an HTTP API without hiding the HTTP
Building lots of client SDKs is hard to do well.
- Need to build so many!
- Focus on the primary platforms and let the rest be second class citizens. Last to update
Many choose to use code generators, tied to API description languages.
It’s really hard to code gen good client code. Especially across multiple platforms.
Can end up with Lowest common denominator.
The question is, how can afford to build good quality libraries and still many platforms
Sometimes tightly coupled client SDKs are the right solution
If you can maintain complete control of both the ingredients in the salad dressing
and how much is put on the salad, then maybe you are OK.
In terms of SDKs:
Who owns each end of the wire?
Can deployments be coordinated?
Can a user choose not to update?
What happens during the change over?
JS clients consuming their own API
What about next week?
But what about when a 3rd party starts to consume
While consuming APIs…many different defns of REST.
Useful to understand why…
Chocolate Chip cookie analogy:
Many different recipes for CC Cookies.
So what defines a CC cookie? (answer)
Consider those the constraints
Challenge is that devs wants a recipe.
Framework designers to the rescue.
They create a recipe for REST API
Devs mistake the recipe for REST
New rules get invented based on framework conventions and limitations.
The effect is what is important…
Doesn’t this kid look happy?
The recipe is not so important as long as the constraints are respected.
REST is the same, just different constraints and effects.
REST defined by constraints ->
Effects are what you want ->
Sometimes, not all effects required,
=> some constraints can be dropped.
Unfortunately term REST used to describe subset of constraints.
Leads to confusion because developers don’t get all effects
Need to focus on the effects we desire and the constraints that achieve it, and stop getting hung up on the name.
Remember: Kids love Oatmeal too. And there are no chocolate chips in those.
Understand the costs and benefits.
Probably the most common style of “REST APIs” you will run into is what I call JSON-REST.
Poutine – A Quebec delicacy of French fries, cheese curds and gravy
They are quick, easy and cheap to make.
Long term consumption may be detrimental to your health
Documentation tells you everything about a specific URL:
What will come back, what to send, what methods, what status codes to expect.
The documentation is effectively the contract.
We are used to this approach in local computing. But Distributed systems are different.
Description languages are being introduced because managing this contract is hard.
Servers can’t changing without breaking clients, so versioning is necessary.
Technically it isn’t what was originally described as REST because…
Failure of the self-description constraint and affects evolvability. That may not be a problem for you.
Hypermedia can help evolution by adding a layer of decoupling
But application/json as content-type still leaves a lots of prearrangement
Aka out of band coupling
Some APIs are starting to document their API conventions as a media type
This is great step forward as media types are the contracts in REST.
But doesn’t exactly promote sharing and re-use outside of the company’s walls
Another approach you may run into while consuming APIs involves the use of Generic hypermedia types
Also, help to solve the problem of creating and documenting API conventions.
Links, forms, lists, embedded resources
Application Semantics are missing
Schemas, Profiles, Vocabularies, Ontologies,
Understand how the constraints you choose to apply affect the system you are building
Even if you only ever plan on using application/json, know what the cost is.
You need to understand the style of API that you are accessing.
Data Oriented APIs are the buffets of APIs. Also referred to as CRUD Apis.
The primary purpose of this type of API is to give you simple access to all the data owned by the API
There is no assumptions about why you want to access it, or how you might combine that data.
It’s fairly easy to do, doesn’t add much value to the data itself,
and has performance challenges, due to the open ended way you can access the data.
At the other end of the API spectrum you have scenario based APIs.
Some foods are designed for special occasions, like this birthday cake
It is easier to focus on solving a single problem and add new scenarios as required
Best way to keep control over performance
Hypermedia works well for this type of API
The hypermedia guides the API consumer through the application workflow
However, clients are very different because they must follow a ritual to achieve the intended goal
Clients should only make requests when presented with links to follow
Link relations are the point of coupling for discovering available links
Fairly steep learning curve… and Catch-22.
Designing a good hypermedia API requires experience consuming one.
Getting good experience consuming one requires access to a good one.
Whereas restaurants change menus for variety,
To encourage regular visitors to come back.
APIs change because they are actively being used and users want to do more.
This is a good thing.
Most good APIs should start small and grow based on customer feedback.
If you can do this, then it is easy to progress and do this…
And before too long you will have…
Trying to build a complete solution up front is the API equivalent of a waterfall design.
It’s much safer to take the agile approach where we build a MVP first and then enhance.
Guarantees we give what the customer needs
And continue to provide good performance
But it requires accepting that change is the norm.
We have a false assumption that versioning is the primary way of dealing with change.
But this comes from our local code mindset
Versioning is painful – Versioning is a way for server developers to push their problems onto client developers
Unless you are going to stay on an old API version, then you are going to have to update client code.
Why not write the client to withstand changes and then versions become irrelevant?
A little extra effort in building clients can go a long way to making the client resilient to change.
Be paranoid about your server side API developers.
Do not trust them, no matter what promises they make.
Be defensive in how you build clients:
Don’t assume ordering or presence of properties
Don’t assume returned content type
Don’t assume URLs will not change
It’s not that API developers are evil. They just have internal forces that make them do things they were not expecting to do, and stuff out of their control.
Holy Grail of the software industry
Path to higher productivity
API world is struggling to achieve significant levels of re-use
Partly because applying local system code reuse techniques
Code based systems use inheritance, interfaces and libraries with shared types
But API consumers and providers have different platforms, languages and type systems.
Still searching for solutionsMany Efforts to produce description languages, search engines and even patents
Few examples of people re-using APIs in the wild. Metablog
APIs are not an effective unit of reuse on the web
2 servers, 2 API defns.
All or nothing. What about partial? What about a readonly?
Composition can be ugly
Media Types and link relations are unit of re-use
Integrations can be presented to client
Server boundaries are transparent
We can reuse a single resource.
To maximize reuse we must try and minimize the knowledge contained in a message.
Client is on a need to know basis.
Can you interact with something without knowing the “meaning” of the content?
Let me give an example
Example from electronics industry. LittleBits
The sensors don't care what is producing the light or heat, or why.
only care if the measurement exceeds a threshold, at which point a switch is flipped.
Bad habit of expose everything
because we don't know what a client wants.
This severely inhibits reuse.
Achieving reuse is about focusing on the things that matter and avoiding things that don’t.
Mentioned that m/ts are unit of reuse
Useful analogy for role of m/ts is cooking equipment
Can you cook with only one type of pan?
Different pots and pans have different uses
But not limited to a single food. Boil potatoes and heat soup in a pot,
Fry an egg, or cook bacon in a pan
Each address a class of problem.
Optimized for simplicity and reusibility
Star-Trek Food Replicator
One device that can produce any kind of food.
Generic media types become complex
And that’s before we’ve even started to address application semantics
For that we need profiles, schemas and namespaces
Until we get there, let’s not forget we can use simple pots and pans to get the job done.
Obviously you can take this too far.
Too much spec writing
Be wary of overly specific media types.
Sometimes it does make sense, making a waffle without a waffle maker is hard.
They make re-use hard.
Consequence is clients must support diff media types
Media types used can change over time / Over time move to new format
Helps with backward compatibility
Can be used to manipulate workflow…. Question example
You’ll be surprised what you can achieve
Implicit or explicit contracts
Clients can tightly couple to hypermedia driven server
Clients can minimize coupling with JSON REST API
Client code is key for evolution.
Server code helps to enable it.
A V2 API doesn’t necessarily mean you need to
Create a whole new client version
Stop supporting V1
Supporting multiple “versions” in a single client is useful
Prod, vs stage, vs test.
This is why it is so important to build flexible clients
Omakase – A style of ordering in Sushi restaurants
Customers express intent by requesting Omakase style
Chef gets to implement that request in the most effective way
This flexibility gives the possibility for the chef to deliver the highest quality food in stock for less than the regular price.
Letting go of some level of control brings flexibility
Breaking request creation and response handling can bring flexibility to client applications.
Responses must be self-descriptive
Testing is easier
Facilitates centralized response handling
Uniform interface shared by all components
We can take advantage of that
Get code reuse by centralizing handling
HTTP Response Machine
Every status code should be handled, for all requests
One place for Redirects, Retries, Authentication challenges
Inline handling can easily introduce hidden coupling
Pet peeve of declaring status codes
Support ranges of status codes
Account for all codes.
Don’t assume what is being returned
Forget to look at headers
Link Relations provide context to your content
Responses are handled independent of context. That’s not quite true.
The link relation is the identifier that provides you with the necessary context
Media Types and Link Relations when combined can be very powerful
When building clients, don’t forget to take advantage of this context
Your client doesn’t need to care what is in the href, the rel is what is important.
Type is only a hint.
APIs that implement HATEOAS require a fairly different way of thinking about building clients.
One of the most useful tools it to an implement a client side class that encapsulates the entire client state
Consider it like a ViewModel for your entire application if you are familiar with that term.
Application Controller waits for some input.
Creates a HTTP request
Sends it the HTTP connector
HttpResponseMachine handles the response
Applies the returned representation to the client state.
Client state fires events based on changes that happen
Client view is updated.
Application Controller waits for some input
Using this approach, the server can completely change its responses to requests and the client doesn’t break.
Example if time…. (Question, expense receipts)
Covered a large range of concepts
If you want to dig into more details here some links
Demo of hypermedia as the engine of application state to reduce client updates
More talk about avoiding the need to versioning
Standardized handling and testing error responses
Talk and demo on the tooling
Over the past few years I’ve produced a number of different open source tools for working with HTTP APIs
The burden of managing the nugets for each of these, along with all the supported platforms (incl portable) and tracking the dependencies between them has been sucking up my time.
So, I decided to aggregate all the relevant libraries together and produce a single, simpler, kit for building client SDKs.
This is still very much a work in progress, but hopefully one day in the near future, this will be available for anyone to quickly build high quality, robust and feature rich SDKs.
Warning: Under construction!
Bon Appetit and Thank you for listening.
I will be around all day, if you have any questions, or just want to comment on or challenge, my assertions, I am open to all feedback.