Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Beautiful REST+JSON APIs with Ion

3,278 views

Published on

At Stormpath we spent 18 months researching API design best practices. Join Les Hazlewood, Stormpath CTO and Apache Shiro Chair, as he explains how to design a secure REST API, the right way. He'll also hang out for a live Q&A session at the end.

Sign up for Stormpath: https://api.stormpath.com/register
More from Stormpath: http://www.stormpath.com/blog

Les will cover:
REST + JSON API Design
Base URL design tips
API Security
Versioning for APIs
API Resource Formatting
API Return Values and Content Negotiation
API References (Linking)
API Pagination, Parameters, & Errors
Method Overloading
Resource Expansion and Partial Responses
Error Handling
Multi-tenancy

Published in: Software

Beautiful REST+JSON APIs with Ion

  1. 1. Beautiful REST+JSON APIs with Ion Les Hazlewood @lhazlewood CTO, Stormpath, stormpath.com
  2. 2. @lhazlewood @goStormpath .com • User Management API for Developers • Password security • Authentication and Authorization • LDAP/AD/Social/SAML/OAuth support • Instant-on, scalable, and highly available • Free for developers
  3. 3. @lhazlewood @goStormpath Why REST? • Scalability • Generality • Independence • Latency (Caching) • Security • Encapsulation
  4. 4. @lhazlewood @goStormpath Why JSON? • Ubiquity • Simplicity • Readability • Scalability • Flexibility
  5. 5. @lhazlewood @goStormpath REST Is Easy
  6. 6. @lhazlewood @goStormpath REST Is *&@#$! Hard (for providers)
  7. 7. @lhazlewood @goStormpath JSON – No Spec
  8. 8. @lhazlewood @goStormpath REST can be easy (if you follow some guidelines)
  9. 9. @lhazlewood @goStormpath HATEOAS • Hypermedia • As • The • Engine • Of • Application • State
  10. 10. @lhazlewood @goStormpath HATEOAS • Links • State Transitions
  11. 11. @lhazlewood @goStormpath Fielding’s Requirements roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext- driven: • Communication Protocol Independent • Media Type Centric • No Fixed Names / Hierarchies • Dynamically Typed! • ZERO OUT-OF-BAND KNOWLEDGE
  12. 12. @lhazlewood @goStormpath How do we meet these requirements?
  13. 13. @lhazlewood @goStormpath How do we...? Base URL Versioning Resource Format Return Values Content Negotiation References (Linking) Pagination Query Parameters Create/Update Search Associations Errors IDs Method Overloading Resource Expansion Partial Responses Caching & Etags Security Multi Tenancy Maintenance Batch Operations
  14. 14. @lhazlewood @goStormpath Fundamentals
  15. 15. @lhazlewood @goStormpath Resources Nouns, not Verbs Coarse Grained, not Fine Grained Architectural style for use-case scalability
  16. 16. @lhazlewood @goStormpath What If? /getAccount /createDirectory /updateGroup /verifyAccountEmailAddress
  17. 17. @lhazlewood @goStormpath What If? /getAccount /getAllAccounts /searchAccounts /createDirectory /createLdapDirectory /updateGroup /updateGroupName /findGroupsByDirectory /searchGroupsByName /verifyAccountEmailAddress /verifyAccountEmailAddressByToken … Smells like bad RPC. DON’T DO THIS.
  18. 18. @lhazlewood @goStormpath Keep It Simple
  19. 19. @lhazlewood @goStormpath The Answer Fundamentally two types of resources: Collection Resource Instance Resource
  20. 20. @lhazlewood @goStormpath Collection Resource /applications
  21. 21. @lhazlewood @goStormpath Instance Resource /applications/a1b2c3
  22. 22. @lhazlewood @goStormpath Behavior • GET • PUT • POST • DELETE • HEAD
  23. 23. @lhazlewood @goStormpath Behavior POST, GET, PUT, DELETE ≠ 1:1 Create, Read, Update, Delete
  24. 24. @lhazlewood @goStormpath Behavior As you would expect: GET = Read DELETE = Delete HEAD = Headers, no Body
  25. 25. @lhazlewood @goStormpath Behavior Not so obvious: PUT and POST can both be used for Create and Update
  26. 26. @lhazlewood @goStormpath PUT for Create Identifier is known by the client: PUT /applications/clientSpecifiedId { … }
  27. 27. @lhazlewood @goStormpath PUT for Update Full Replacement PUT /applications/existingId { “name”: “Best App Ever”, “description”: “Awesomeness” }
  28. 28. @lhazlewood @goStormpath PUT Idempotent
  29. 29. @lhazlewood @goStormpath POST as Create On a parent resource POST /applications { “name”: “Best App Ever” } Response: 201 Created Location: https://api.stormpath.com/applications/a1b2c3
  30. 30. @lhazlewood @goStormpath POST as Update On instance resource POST /applications/a1b2c3 { “name”: “Best App Ever. Srsly.” } Response: 200 OK
  31. 31. @lhazlewood @goStormpath POST NOT Idempotent
  32. 32. @lhazlewood @goStormpath Media Types • Format Specification + Parsing Rules • Request: Accept header • Response: Content-Type header • application/json • application/ion+json • application/ion+json;v=2 • …
  33. 33. @lhazlewood @goStormpath Design Time!
  34. 34. @lhazlewood @goStormpath HATEAOS in JSON: Ion
  35. 35. @lhazlewood @goStormpath Resources
  36. 36. @lhazlewood @goStormpath Object { “firstName”: “Bob”, “lastName”: “Smith”, “birthDate”: “1980-01-23”, }
  37. 37. @lhazlewood @goStormpath Ion Object { “meta”: { ... }, “firstName”: “Bob”, “lastName”: “Smith”, “birthDate”: “1980-01-23”, }
  38. 38. @lhazlewood @goStormpath Naïve collection (not recommended) “items”: []
  39. 39. @lhazlewood @goStormpath Collection Object { “items”: [] }
  40. 40. @lhazlewood @goStormpath Ion Collection { “meta”: { ... }, “items”: [] }
  41. 41. @lhazlewood @goStormpath Linking
  42. 42. @lhazlewood @goStormpath HREF • Distributed Hypermedia is paramount! • Every accessible Resource has a canonical unique URL • Replaces IDs (IDs exist, but are opaque). • Critical for linking
  43. 43. @lhazlewood @goStormpath Linking in JSON? • XML has it (XLink), JSON doesn’t • How do we do it?
  44. 44. @lhazlewood @goStormpath Naïve linking
  45. 45. @lhazlewood @goStormpath Naïve linking - instance GET /accounts/x7y8z9 200 OK { “givenName”: “Tony”, “surname”: “Stark”, …, “directory”: ???? }
  46. 46. @lhazlewood @goStormpath Naïve linking - instance cont’d GET /accounts/x7y8z9 200 OK { “givenName”: “Tony”, “surname”: “Stark”, …, “directory”: { “href”: “https://api.stormpath.com/v1/directories/g4h5i6” } }
  47. 47. @lhazlewood @goStormpath Naïve linking - collection GET /accounts/x7y8z9 200 OK { “givenName”: “Tony”, “surname”: “Stark”, …, “groups”: { “href”: “https://api.stormpath.com/accounts/x7y8z9/groups” } }
  48. 48. @lhazlewood @goStormpath Ion linking (recommended)
  49. 49. @lhazlewood @goStormpath Ion meta href GET /accounts/x7y8z9 200 OK { “meta”: { “href”: “https://api.stormpath.com/accounts/x7y8z9” }, “givenName”: “Tony”, “surname”: “Stark”, … }
  50. 50. @lhazlewood @goStormpath Ion link GET /accounts/x7y8z9 200 OK { “meta”: { ... }, “givenName”: “Tony”, “surname”: “Stark”, …, “directory”: { “meta”:{ “href”: https://api.stormpath.com/directories/g4h5i6” } } }
  51. 51. @lhazlewood @goStormpath Ion link (collection) GET /accounts/x7y8z9 200 OK { “meta”: { ... }, “givenName”: “Tony”, “surname”: “Stark”, …, “groups”: { “meta”: { “href”: “https://api.stormpath.com/accounts/x7y8z9/groups”, “rel”: [“collection”] } } }
  52. 52. @lhazlewood @goStormpath What about HAL? • Linking focus • Forces links to be separate from context/content – (when was the last time you had to put all of your anchors at the bottom of an html paragraph? Right... never.) • In contrast to HAL, Ion is much more like HTML – it’s all in one convenient spec. • Ion transliteration to/from HTML is far easier (by design)
  53. 53. @lhazlewood @goStormpath State Transitions
  54. 54. @lhazlewood @goStormpath Creating And Updating
  55. 55. @lhazlewood @goStormpath Remember HATEOS?
  56. 56. @lhazlewood @goStormpath “Let me go read the docs to figure out how to POST”
  57. 57. @lhazlewood @goStormpath NO! This is not HATEOS.
  58. 58. @lhazlewood @goStormpath How do browsers work?
  59. 59. @lhazlewood @goStormpath Forms
  60. 60. @lhazlewood @goStormpath Forms: Create { “foo”: “bar”, “baz”: “boo”, ... “createAccount”: { “meta”: {“href”: “https://foo.io/users”, “rel”: [“create-form”], “method”: “POST”}, “items”: [ {“name”: “login”, “required”: “true”, “label”: “Username or Email” }, {“name”: “password”, “secret”: “true”, “required”: “true”, “label”: “Password”} ] } }
  61. 61. @lhazlewood @goStormpath Forms: Create cont’d { "meta": {"href": "https://example.io/users", "rel":["create-form"], "method":"POST"}, "items": [ {"name":"username"}, {"name": "password", "secret": true }, {"name": "visitedContinents","type": "set", "minitems": 1,"maxitems": 7,"options": { "items": [ {"label": "Africa", "value": "af" }, {"label": "North America", "value": "na" }, {"label": "South America", "value": "sa" }, {"label": "Europe", "value": "eu" }, {"label": "Asia", "value": "as" }, {"label": "Oceania", "value": "oc" }, {"label": "Antarctica", "value": "an" }, ] } } ] }
  62. 62. @lhazlewood @goStormpath Forms: Search / Query{ ... “findAccounts”: { “meta”: { “href”: “https://foo.io/users”, “rel”: [“search-form”], “method”: “GET” }, “items”: [ {“name”: “username”, “label”: “Username”}, {“name”: “email”, “type”: “email”, “label”: “Email” }, {“name”: “givenName”, “label”: “First Name” }, {“name”: “surname”, “label”: “Last Name”}, ] } }
  63. 63. @lhazlewood @goStormpath What about Schemas / json-schema ? Not needed. REST != RDBMS (are schemas necessary for browsers/HTML?) Forms do the same thing and are more flexible/powerful Remember Fielding’s REST Rule about Dynamic Typing
  64. 64. @lhazlewood @goStormpath HTTP Protocol Semantics
  65. 65. @lhazlewood @goStormpath Base URL
  66. 66. @lhazlewood @goStormpath http(s)://foo.io vs http://www.foo.com/dev/service/api/rest
  67. 67. @lhazlewood @goStormpath http(s)://foo.io Rest Client vs Browser
  68. 68. @lhazlewood @goStormpath Versioning
  69. 69. @lhazlewood @goStormpath URL https://api.stormpath.com/v1 vs. Media-Type application/json application/ion+json
  70. 70. @lhazlewood @goStormpath Content Negotiation
  71. 71. @lhazlewood @goStormpath Header • Accept header • Header values comma delimited • q param determines precedence, defaults to 1, then conventionally by list order GET /applications/a1b2c3 Accept: application/json, text/plain;q=0.8
  72. 72. @lhazlewood @goStormpath Resource Extension /applications/a1b2c3.json /applications/a1b2c3.csv … Conventionally overrides Accept header
  73. 73. @lhazlewood @goStormpath Style Guide
  74. 74. @lhazlewood @goStormpath camelCase ‘JS’ in ‘JSON’ = JavaScript myArray.forEach Not myArray.for_each account.givenName Not account.given_name Underscores for property/function names are unconventional for JS. Stay consistent.
  75. 75. @lhazlewood @goStormpath Dates & Times
  76. 76. @lhazlewood @goStormpath Dates & Times There’s already a standard. Use it: ISO 8601 “createdAt”: “2013-07-10T18:02:24.343Z” Use UTC! This is represented in Ion as a field types of date, time, datetime, etc.
  77. 77. @lhazlewood @goStormpath createdAt / updatedAt Most people will want this at some point { …, “createdAt”: “2013-07-10T18:02:24.343Z”, “updatedAt”: “2014-09-29T07:02:48.761Z” } Use UTC!
  78. 78. @lhazlewood @goStormpath Reference Expansion (aka Entity Expansion, Link Expansion)
  79. 79. @lhazlewood @goStormpath Account and its Directory?
  80. 80. @lhazlewood @goStormpath GET /accounts/x7y8z9?expand=directory 200 OK { “meta”: {..., “expandable”: [“directory”,...] }, “givenName”: “Tony”, “surname”: “Stark”, …, “directory”: { “meta”: { ... }, “name”: “Avengers”, “description”: “Hollywood’s plan for more $”, “createdAt”: “2012-07-01T14:22:18.029Z”, … } }
  81. 81. @lhazlewood @goStormpath Collection Pagination
  82. 82. @lhazlewood @goStormpath Ensure Collection Resources support query params: • Offset + Limit vs Cursor …/applications?offset=50&limit=25 • Don’t require the user to query for these – provide OOTB links
  83. 83. @lhazlewood @goStormpath GET /accounts/x7y8z9/groups 200 OK { “meta”: { ... }, “offset”: 0, “limit”: 25, “first”: { “meta”:{“href”: “…/accounts/x7y8z9/groups?offset=0”}}, “previous”: null, “next”: { “meta”:{“href”: “…/accounts/x7y8z9/groups?offset=25”}}, “last”: { “meta”:{“href”: “…”}}, “items”: [ { “meta”: { “href”: “…”, ...} }, { “meta”: { “href”: “…”, ...} }, … ] }
  84. 84. @lhazlewood @goStormpath Sorting
  85. 85. @lhazlewood @goStormpath GET .../accounts? orderBy=surname,givenName%20desc
  86. 86. @lhazlewood @goStormpath Search
  87. 87. @lhazlewood @goStormpath “Find all accounts with a ‘company.com’ email address that can login to a particular application”
  88. 88. @lhazlewood @goStormpath GET /applications/x7y8z9/accounts?email=*company.com 200 OK { “meta”: { ... }, “offset”: 0, “limit”: 25, “first”: { “meta”:{ “href”:“/applications/x7y8z9/accounts?email=*company.com&offset=0”} }, “previous”: null, “next”: { “meta”:{ “href”: “/applications/x7y8z9/accounts?email=*company.com&offset=25”} }, “last”: { “meta”:{“href”: “…”}}, “items”: [ { “meta”: { “href”: “…”, ...} }, { “meta”: { “href”: “…”, ...} }, … ] }
  89. 89. @lhazlewood @goStormpath Search cont’d • Filter search .../accounts?q=some+value • Attribute Search .../accounts?surname=Joe&email=*company.com
  90. 90. @lhazlewood @goStormpath Search cont’d • Starts with ?email=joe* • Ends with ?email=*company.com • Contains (warning! Bad performance) ?email=*foo*
  91. 91. @lhazlewood @goStormpath Search cont’d • Range queries via Ion min and max field members “all accounts created between September 1st and the 15th” Form fields example: {“name”: “createdAtBegin”, “min”: “2014-01-01”,”max=“2014-12-31”} {“name”: “createdAtEnd”, “min”: “2014-01-01”,”max=“2014-12-31”} Ion TBD: range type: .../accounts?createdAt=[2014-09-01,2014-09-15]
  92. 92. @lhazlewood @goStormpath Search cont’d • Use Ion forms and the pattern form field member to represent search expressions
  93. 93. @lhazlewood @goStormpath Many To Many
  94. 94. @lhazlewood @goStormpath Group to Account • A group can have many accounts • An account can be in many groups • Each mapping is a resource: GroupMembership
  95. 95. @lhazlewood @goStormpath GET /groupMemberships/23lk3j2j3 200 OK { “meta”:{“href”: “…/groupMemberships/23lk3j2j3”}, “account”: { “meta”:{“href”: “…”} }, “group”: { “meta”{“href”: “…”} }, … }
  96. 96. @lhazlewood @goStormpath GET /accounts/x7y8z9 200 OK { “meta”:{“href”: “…/accounts/x7y8z9”}, “givenName”: “Tony”, “surname”: “Stark”, …, “groups”: { “meta”:{“href”: “…/accounts/x7y8z9/groups” “rel”: [“collection”]} }, “groupMemberships”: { “meta”:{“href”: “…/groupMemberships?accountId=x7y8z9”,”rel”:[“collection”]} } }
  97. 97. @lhazlewood @goStormpath Async or Long-Lived Operations
  98. 98. @lhazlewood @goStormpath POST /emails { “from”: me@somewhere.com, “subject”: “Hi!” “body”: “...” }
  99. 99. @lhazlewood @goStormpath 204 Accepted Location: /emails/23Sd932sSl { “status”: “queued”, ... }
  100. 100. @lhazlewood @goStormpath GET /emails/23Sd932sSl Expires: 2014-09-29T18:00:00.000Z { “status”: “sent”, ... }
  101. 101. @lhazlewood @goStormpath Batch Operations
  102. 102. @lhazlewood @goStormpath • Each batch is represented as a resource • Batches are likely to be a collection • Batches are likely to have a status • Downside: problematic regarding HTTP caching
  103. 103. @lhazlewood @goStormpath Batch Delete “Delete all company.com accounts” DELETE /accounts? email=*@company.com
  104. 104. @lhazlewood @goStormpath Batch Create / Update Already have a Collection concept. Use it.
  105. 105. @lhazlewood @goStormpath Batch Create or Update POST /accounts { “meta”: { ... }, “items”: [ { ... account 1 ... }, { ... account 2 ... }, ... ] }
  106. 106. @lhazlewood @goStormpath Batch Operations: The ‘Catch’ HTTP Caching is bypassed entirely 
  107. 107. @lhazlewood @goStormpath 204 Accepted Location: /batches/a1b2c3 { “status”: “processing”, //overall status “size”: “n”, “limit”: 25, ..., “items”: { { response 1 (w/ individual status) ...}, { response 2 (w/ individual status) ...}, ... } }
  108. 108. @lhazlewood @goStormpath Errors
  109. 109. @lhazlewood @goStormpath • As descriptive as possible • As much information as possible • Developers are your customers
  110. 110. @lhazlewood @goStormpath POST /directories 409 Conflict { “status”: 409, “code”: 40924, “property”: “name”, “message”: “A Directory named ‘Avengers’ already exists.”, “developerMessage”: “A directory named ‘Avengers’ already exists. If you have a stale local cache, please expire it now.”, “moreInfo”: “https://www.stormpath.com/docs/api/errors/40924” }
  111. 111. @lhazlewood @goStormpath Security
  112. 112. @lhazlewood @goStormpath Avoid sessions when possible Authenticate every request if necessary Stateless Authorize based on resource content, NOT URL! Use Existing Protocol: Oauth 1.0a, Oauth2, Basic over SSL only Custom Authentication Scheme: Only if you provide client code / SDK Only if you really, really know what you’re doing Use API Keys and/or JWTs instead of Username/Passwords
  113. 113. @lhazlewood @goStormpath 401 vs 403 • 401 “Unauthorized” really means Unauthenticated “You need valid credentials for me to respond to this request” • 403 “Forbidden” really means Unauthorized “Sorry, you’re not allowed!”
  114. 114. @lhazlewood @goStormpath HTTP Authentication Schemes • Server response to issue challenge: WWW-Authenticate: <scheme name> realm=“Application Name” • Client request to submit credentials: Authorization: <scheme name> <data>
  115. 115. @lhazlewood @goStormpath API Keys • Entropy • Password Reset • Independence • Scope • Speed • Limited Exposure • Traceability
  116. 116. @lhazlewood @goStormpath IDs
  117. 117. @lhazlewood @goStormpath • IDs should be opaque • Should be globally unique • Avoid sequential numbers (contention, fusking) • Good candidates: UUIDs, ‘Url64’
  118. 118. @lhazlewood @goStormpath HTTP Method Overrides
  119. 119. @lhazlewood @goStormpath POST /accounts/x7y8z9?_method=DELETE
  120. 120. @lhazlewood @goStormpath Caching & Concurrency Control
  121. 121. @lhazlewood @goStormpath Server (initial response): ETag: "686897696a7c876b7e” Client (later request): If-None-Match: "686897696a7c876b7e” Server (later response): 304 Not Modified
  122. 122. @lhazlewood @goStormpath Maintenance
  123. 123. @lhazlewood @goStormpath Use HTTP Redirects Create abstraction layer / endpoints when migrating Use well defined custom Media Types
  124. 124. @lhazlewood @goStormpath IETF RFC?
  125. 125. @lhazlewood @goStormpath Ion Media Type ionwg.org/draft-ion.html
  126. 126. @lhazlewood @goStormpath .com • Free for developers • Eliminate months of development • Automatic security best practices • Single Sign On • Social/OAuth/SAML/Multi-factor/etc • API Authentication & Key Management • Token Authentication for SPAs / Mobile • Authorization & Multi-tenancy for your apps Libraries and integrations: https://docs.stormpath.com

×