Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Crafting Evolvable API Responses
Similar to Crafting Evolvable API Responses (20)
Recently uploaded
Recently uploaded (20)
Crafting Evolvable API Responses
- 2. Who am I? • Twitter: @darrel_miller • http://www.bizcoder.com/ Solve API Problems Fast
- 3. Our Journey Today • Focus on API responses • Versioning is painful • Why we think we need it? • How can we avoid it? • What if you can’t?
- 4. Objects over the wire
- 5. We have been here before • CORBA, DCOM • SOAP, WSDL • DTOs • JSON
- 6. The ASP.NET Web API Project Template public class ValuesController : ApiController { // GET api/values public IEnumerable<string> Get() { return new string[] { "value1", "value2" };} // GET api/values/5 public string Get(int id) { return "value"; } // POST api/values public void Post([FromBody]string value) { } // PUT api/values/5 public void Put(int id, [FromBody]string value) { } // DELETE api/values/5 public void Delete(int id) { } }
- 7. The ASP.NET Web API Starter Tutorial public class ProductsController : ApiController { //… public IEnumerable<Product> GetAllProducts() { return products; } public IHttpActionResult GetProduct(int id) { var product = products.FirstOrDefault((p) => p.Id == id); if (product == null) { return NotFound(); } return Ok(product); } } http://www.asp.net/web-api/overview/getting-started-with-aspnet-web-api/tutorial-your-first-web-api
- 8. ServiceStack public class ReqstarsService : Service { public List<Reqstar> Get(AllReqstars request) { return Db.Select<Reqstar>(); } }
- 9. NancyFX public class SampleModule : Nancy.NancyModule { public SampleModule() { Get["/"] = _ => "Hello World!"; } }
- 10. Python Flask @app.route('/todo/api/v1.0/tasks', methods=['GET']) def get_tasks(): return jsonify({'tasks': tasks})
- 11. Rails # Returns the resource from the created instance variable # @return [Object] def get_resource instance_variable_get("@#{resource_name}") end
- 12. But, what’s the problem?
- 13. Which objects to map? • Domain objects • How do we hide content we don’t want to expose? • How do we create different views of data? • Changes to domain model cause changes to API • DTOs • whole lot of busy work • Only indirect control over serialization process
- 14. Automatic Serialization • All the properties • Missing values, unrecognized properties • Non standard data types: datetime, timespan • Does null mean unspecified, or explicitly null? • Empty collection or no collection • Capitalization • Links • Cycles • Changes to behavior in the frameworks • Security Risks
- 15. The whole app is useless because the API added a new preference
- 16. APIs Break because Serialization Libraries Change/Fix things
- 18. Use a DOM to build your document dynamic jspeaker = new JObject(); jspeaker.name = speakerInfo.Name; jspeaker.bio = speakerInfo.Bio; dynamic links = new JObject(); dynamic iconLink = new JObject(); iconLink.href = speakerInfo.ImageUrl; links.icon = iconLink; dynamic sessionsLink = new JObject(); sessionsLink.href = SessionsLinkHelper.CreateLink(request, speakerInfo).Target; links[LinkHelper.GetLinkRelationTypeName<SessionsLink>()] = sessionsLink; jspeaker["_links"] = links; return new DynamicHalContent(jspeaker);
- 19. What to do with your new found freedom?
- 21. Anatomy of an HTTP representation 200 OK HTTP/1.1 Server: Microsoft-HTTPAPI/2.0 Content-Length: 44 Content-Type: text/plain Acme-perf-database-cost: 120ms The quick brown fox jumped over the lazy dog
- 22. Let’s build some payloads
- 23. { "description" :"There is a hole in my bucket" } The smallest thing that is actionable
- 24. { "description" :"There is a hole in my bucket", "steps_to_reproduce" : "Pour water in bucket. Lift bucket off ground. Look for water dripping", "date_reported": "2012-04-21T18:25:43-05:00" } Just enough data to solve the problem
- 25. { "description" :"The font is too big", "application_name" : "Wordament", "application_version" : "1.2.0.2392", "environment_osversion" : "NT4.0", "environment_memory_free" : "100 MB", "environment_diskspace_free" : "1.5 GB", "reported_by_user" : “Bob Bing” } Why do they need that data?
- 26. { "description" :"The font is too big", "application_name" : "Wordament", "application_version" : "1.2.0.2392", "environment" : { "osversion" : "NT4.0", "memory_free" : "100 MB", "diskspace_free" : "1.5 GB" } } Attribute Groups
- 27. { "description" :"The font is too big", "history" : [ {"status" : "reported", "date" :"2014-02-01"}, {"status" : "triaged", "date" :"2014-02-04"}, {"status" : "assigned", "date" :"2014-02-12"}, {"status" : "resolved", "date" :"2014-02-19"}, ] } Attribute Groups for multiple instances
- 28. { "description" :"The font is too big", "reported_by_user" : { "name" : "Bob Bing", "email" : "bob@acme.com", "twitter" : "@bobbing", "date_hired" : "2001-01-21" } } Attribute Groups for related data
- 29. { "description" :"The font is too big", "reported_by_user_url" : "http://api.acme.com/users/75" } Linking related data
- 30. { "description" :"The font is too big", “links" : [ { "href" :"http://api.acme.com/users/75", "rel": "reportedbyuser" }, { "href" :"http://api.acme.com/users/24", "rel": "assignedtouser" } ] } Multiple Links
- 31. { "issues" : [ { "description" :"There is a hole in my bucket" } ] } Beware of structural changes { "issue" : { "description" :"There is a hole in my bucket" } } Item in a Array Item as a Property
- 32. Naming Your Conventions • Empty arrays • Nulls • Single item as array or object • Camel case, snake case • Vocabulary • Protocols
- 33. Media Type Scope application/json application/vnd.acme.issue+json application/issue+json Too broad Too narrow Ideal application/hal+json Partial application/vnd.acmeapi+json Compromise Vocabularies : Profiles, Namespaces, Schemas, Ontologies
- 35. { "description" :"The font is too big", "_embedded" : { "reportedByUser" : { "name" : "Bob Bing", "email" : "bob@acme.com", "_links" : { "self" : {"href" :"http://api.acme.com/users/75"}} } } Meet application/hal+json
- 36. { "collection": { "links": [], "items": [ { "data": [ { "name": "Title", "value": "rntttLearning from Noda Time: a case study in API design and open source (good, bad and ugly)rntt“ }, { "name": "Timeslot", "value": "04 December 2013 16:20 - 17:20“ }, { "name": "Speaker", "value": "Jon Skeet“ } ], "links": [ { "href": "http://conference.hypermediaapi.com/speaker/6", "rel": "http://tavis.net/rels/speaker" }, { "href": "http://conference.hypermediaapi.com/session/133/topics", "rel": "http://tavis.net/rels/topics" } ], "href": "http://conference.hypermediaapi.com/session/133" } ], "query": [], "template": { "data": [] }, "version": "1.0" } } Meet application/vnd.collection+json
- 37. Version as a Last Resort • Version the payload • HTML DOCTYPE, Collection+Json • Version the resource • /api/documents/invoice.v2/758 • Version the media type • application/vnd.github.v3+json • Version the API • /api/v2/documents/invoice/758
- 38. Wrap up • Understand the limitations of “objects over the wire” • Consider taking back control of your representations • Think in terms of messages, instead of objects • Build software that is designed to survive change • Believe that versioning is an admission of failure
- 40. Image Credits • Package - https://flic.kr/p/3mrNyn • Freedom - https://flic.kr/p/4vwRDw • Treasure map - https://flic.kr/p/7jDJwi • Handshake - https://flic.kr/p/nbAu8Y • Telephone - https://flic.kr/p/7Q8bMd • Blindfolded Typing - https://flic.kr/p/53Q3JE • Magic Trick - https://flic.kr/p/7T8zk5 • Donut machine - https://flic.kr/p/anncxf • GT-R Steering Wheel - https://flic.kr/p/fDUSDk • Anatomy - https://flic.kr/p/6bfUZn • Shapes - https://flic.kr/p/3aKUAq • Payloaders - https://flic.kr/p/dTU9sN • Birds on a Wire - https://flic.kr/p/4YdfK
Editor's Notes
- - Developer advocate for Runscope. - Cloud based solutions for API performance monitoring Microsoft MVP Book
- Two parts to API design, resource identification and representation design designing representations aka HTTP response. - Versioning sucks So many ways to do it Starting to hear the message, don’t version Roy says middle finger Layer7 say don’t. The problem is we all make mistakes. - Why do we think we need it
- One reason we think we need versioning is because we do Objects over the wire Pass object graph Infrastructure converts it Hope the client understands it Pre-arranged agreement between client/server The details are key to avoiding versioning. But Web APIs are not the same as local APIs Objects over the wire. The idea is that you construct some kind of object graph in your favourite language in your web application, that contains the data that you want to deliver to the clients. And then you pass that object graph to some piece of infrastructure code that converts it into a wire format that hopefully will be consumable by the client. The challenge is, for this to work, there has to be some kind of pre-arranged agreement between sender and receiver. The details of that agreement are the key to avoiding versioning.
- - Trying to do this since the mid 1990’s CORBA, DCOM - When the web won, SOAP was invented simplified to DTOs - REST was rediscovered and redefined to use JSON to send objects over the wire JSON has helped. It makes the easy stuff really easy. But the more work we do in JSON the more we are starting to see the re-invention of the complexity. JSON-Schema, JSON Pointer, JSON Path, patch formats, namespacing. Fallacy of REST has no contracts. We are just pretending they are there, but they are implicit and unwritten. So what does “objects over the wire” look like in code, in the current incarnation. We have been trying to do this since the mid 1990’s. CORBA, DCOM were attempts to allow access to remote objects. When it was recognized that the web has won, SOAP was invented to try and do Objects over the wire on HTTP Some smart people realized that “objects over the wire” was never really going to succeed so they simplified to DTOs over the wire. Types over the wire, but sharing types creates coupling. SOAP used a similar concept but called them data contracts Only works if you have tight control over both sides of the wire.
- - The default behavior is to return CLR types let the framework decide how to convert it to representation on the wire What does Ienumerable<string> look like on the wire? Does every platform do it the same way? What does string look like on the wire? JSON spec has changed recently for simple values. POSTing string. So many StackOverflow questions on posting string of XML that doesn’t work. The worlds simplest API and multiple interoperability issues.
- In this example we move from returning a native CLR data type to a custom object. Product Note IHTTPActionResult Starts to break down when the information we are trying to send is not in the payload Returning a object limits how we can craft the response. But this is not just Microsoft doing silly things
- What are the odds that List<Reqstars> creates the same wire representation as List<Reqstars> in Web API Returning objects is hiding a critical information needed to ensure interoperability
- Even the really cool frameworks. I wonder what this will look like on the wire. It is a very common pattern. Every framework provider will tell you, “oh but you can customize the response if you want” …but that’s not where they lead you.
- It is also not a problem that is limited to the .net space. This python example is slightly better, because at least we know it is going to be sent as a JSON representation and the mapping from objects to JSON is fairly straight forward
- And just as one last example, rails does the same thing.
- So what, you say… most of those are minor implementation details. What’s the fundamental problem with objects over the wire?
- Even with DTOs, you only have indirect control over the serialization process
- If you address supports suite no you are going to get it whether there is a value or not. Sending data, what does it expect? Mandrill How do you format dates? iJson was a new standard developed to help interoperability In a geneology app, if Date of death is null, are we missing that data or is he just not dead yet? How you handle links in your server implementation is your business, how you format them on the wire is everyone’s business .Net objects don’t follow capitalization rules of JSON. What about word separation? Cycles can cause some serializers to die randomly. The great YAML fiasco When dealing with Automatic Serialization there are so many unknowns Being aware of what are the rules of your representations is absolutely critical. When you know it is a rule, you start to think about the long term effects and the cross platform concerns. When you are “just serializing your objects as JSON” you get a false sense of security.
- This is what happens when you are not clear about your contract.
- The poor guys writing the serializers. They break people when they fix things.
- What’s a DOM…. - Here is just one example of using the JSON.NET Jobject and dynamic. - There are many ways to do this. - helper methods allow you to establish conventions in your representation formats. - Instead of domain objects to DTOs, - Do domain objects/linq projections to DOM. - You define your own conventions instead of a serializer doing it for you. And maybe you don’t even have to define your own conventions, Maybe the conventions of an existing media type meet your needs
- The problem with not using serializers, now you have the problem of having think about how to structure your responses. So, let’s start with some fundamentals.
- Clear up confusion around terms. Mapping Resources to entities can be limiting to your design. A different URI should almost always be a different resource. Ideally if you want two URLs for the same resource then one should redirect to the other. Prevents cache pollution A Resource can have multiple representations, or just a single one. There is nothing wrong with creating distinct resources for available formats Resources can be completely dynamic concepts
- You can change that reason phrase… Headers are metadata about the response Performance – Headers can be processed and interpreted without having to interpret the entire request body. Cross cutting concerns Timing – Headers arrive first and can actually be used to abort the reception of the body, or prepare for the body, whilst downloading bytes Naming – don’t use x-, do use company- for custom headers Content - Links, json, Unicode? In the future http/2 will support header compression
- Often at the root of an API. Another example : Atom service document
- Building an Issue Tracking System Really simple minimal documents make it really easy to get started
- - Optional attributes can be added to the root object - Consider the use-case, not the server object model. - What would a serializer do? - Objects include everything. Guide client devs down the right path - You can always add, it’s much hard to take away. - There are reasons to group attributes, until you have a reason, probably not worth it. Structural changes are tricky. - Clients shouldn’t depend on order - Name it based on most natural conventions for the format
- How much semantics does the client really need? Is it going to do calcs on the memory free? Consider cultural issues, like formatting, language. Should a date be pretty formatted and localized? Should an address be concatenated or separate fields Should a name be two fields, or formatted firstname + lastname or last name comma first name
- Grouping together can have a variety of advantages saved a few bytes potentially more human readable allows definition of mandatory fields within a group, without the group itself being mandatory - group becomes a candidate for linking instead of embedding. It might map to an object, it might not.
- Groups are useful for supporting multiple instances of a set of data.
- Embed related data to reduce round trips When updating an issue, we would not update the contents of the reportedByUser. - Related information can also be included by providing a link. - Consider volatility and reusability of data. - beware of embedding short lived data inside long lived data.
- // Links can be very handy for pointing at information that changes rarely and can easily be cached locally. // There are many ways of representing links. Sometimes a link is presented as an object.
- Multiple objects can be displayed like this…
- Making a your concept a property allows for a issue.v2, …or allows for defining a media type that can hold many different concepts Sometimes you might want to make something a collection, even though today there is only one. Don’t make the root of your JSON object an array (must Ignore policy)
- “Media Type” is the name you give to your conventions Content-Type is the header to identify conventions to the client There is a lot of nitty gritty details to think about when defining a media type from scratch Better to layer new types on old ones. Profile is an attempt to separate the efforts.
- Distinction between what is here and what is elsewhere
- A list of things