Don't screw it up: how to build durable web apis

7,449 views
6,902 views

Published on

Presentation given during the phpDay 2014 in Verona.
It's about how to build durable web apis based on the experience gained at Namshi while we were developing our SOA architecture

Published in: Internet, Technology

Don't screw it up: how to build durable web apis

  1. Don’t screw it up! @cirpo @_odino_
  2. How to build durable web APIs
  3. How to build durable web APIs 1. Can you predict the future?
  4. Dubai Marina, ~2000
  5. Dubai Marina, 2014
  6. Can you really predict the future?
  7. If there’s one thing we learned over the past 5 years of development...
  8. Monoliths are disappearing
  9. Full stack is dead Microservice Architecture, [...] a particular way of designing software applications as suites of independently deployable services http://martinfowler.com/articles/microservices.html “ ”
  10. Full stack is dead Microservice Architecture, [...] a particular way of designing software applications as suites of independently deployable services http://martinfowler.com/articles/microservices.html “ ” SERVICE-ORIENTED ARCHITECTURES
  11. LEGO, something new in a geek presentation...
  12. FROM a single page application written in
  13. TO an hybrid solution
  14. In TWO weeks!
  15. HOW???
  16. APIs written in PHP <3
  17. Everyone wants APIs
  18. Everyday normal services
  19. dev-oriented services
  20. API maniacs
  21. 2. HTTP is here to stay
  22. GET vs POST “The difference is that in a GET request you have the parameters in the url , with a POST the parameters are in the request’s body”
  23. GET vs POST
  24. HTTP FUNDAMENTALS
  25. HTTP FUNDAMENTALS GET POST
  26. HTTP FUNDAMENTALS GET POST PUT HEAD DELETEPATCH OPTIONS
  27. HTTP FUNDAMENTALS HEADERS Accept Accept-Encoding Accept-Language Cookie Content-Type Referer If-Modified-Since If-None-Match Origin User-Agent Cache-Control
  28. HTTP FUNDAMENTALS CUSTOM HEADERS N-LocationN-Locale N-Device N-Platform N-App N-Theme
  29. WAKA “A new protocol designed to match the efficiency of well-designed Web Applications” http://tools.ietf.org/agenda/83/slides/slides-83-httpbis-5.pdf
  30. SPDY/1..3 A protocol “invented” by Google, which supports: extended compression multiplexing prioritization server push
  31. SPDY/1..3 A protocol “invented” by Google, which supports: extended compression multiplexing prioritization server push
  32. SPDY/1..3 A protocol “invented” by Google, which supports: extended compression multiplexing prioritization server push
  33. SPDY/1..3 A protocol “invented” by Google, which supports: extended compression multiplexing prioritization server push
  34. SPDY/1..3 A protocol “invented” by Google, which supports: extended compression multiplexing prioritization server push
  35. HTTP/2.0
  36. HTTP/2.0 based on SPDY
  37. HTTP/2.0 which is a faster version of HTTPS
  38. HTTP/2.0 which is a safer version of HTTP
  39. HTTP is definitely here to stay, semantics won’t change
  40. 3. Plan for failure
  41. Work around bugs https://gist.github.com/odino/11295759/revisions
  42. Work around bugs https://gist.github.com/odino/11295759/revisions
  43. Failover HTTP/1.1 200 OK Date: Fri, 25 Apr 2014 16:52:37 GMT Content-Type: application/json Transfer-Encoding: chunked Connection: keep-alive Vary: Accept-Encoding Cache-Control: stale-if-error=3600, stale-while-revalidate=6000 Age: 0 Via: 1.1 varnish X-Cache: MISS Alternate-Protocol: 443:npn-spdy/2
  44. Failover HTTP/1.1 200 OK Date: Fri, 25 Apr 2014 16:52:37 GMT Content-Type: application/json Transfer-Encoding: chunked Connection: keep-alive Vary: Accept-Encoding Cache-Control: stale-if-error=3600, stale-while-revalidate=6000 Age: 0 Via: 1.1 varnish X-Cache: MISS Alternate-Protocol: 443:npn-spdy/2
  45. Failover HTTP/1.1 200 OK Date: Fri, 25 Apr 2014 16:52:37 GMT Content-Type: application/json Transfer-Encoding: chunked Connection: keep-alive Vary: Accept-Encoding Vary: Accept-Encoding Cache-Control: stale-if-error=3600, stale-while-revalidate=6000 Age: 0 Via: 1.1 varnish X-Cache: MISS alternate-protocol: : 443:npn-spdy/2 Alternate-Protocol: 443:npn-spdy/2 cache available if the backend is down
  46. Design mistakes?
  47. Versioning to the rescue
  48. Versioning to the rescue https://gist.github.com/odino/bf4c7468cba8b16c6493
  49. Versioning to the rescue https://gist.github.com/odino/f820dda941bf44aa7605
  50. Versioning to the rescue https://gist.github.com/odino/b5d963d8f8aec904d76c
  51. Versioning to the rescue https://gist.github.com/odino/0fbb5be8113deed752fc
  52. How to detect the version?
  53. How to detect the version? api.domain.org/v1/...
  54. How to detect the version? SIMPLE
  55. How to detect the version? ...but how to detect it?
  56. Detecting the version https://gist.github.com/odino/f5a1026449e35cfa8a29
  57. Detecting the version https://gist.github.com/odino/f5a1026449e35cfa8a29 Here it belongs to the route/controller, you need it at the Request level
  58. Detecting the version https://gist.github.com/odino/f5a1026449e35cfa8a29 Use a header!
  59. Detecting the version https://gist.github.com/odino/bf4c7468cba8b16c6493
  60. Can’t test it easily!
  61. Let Nginx do the dirty work https://gist.github.com/odino/6750004f735c8d08687d
  62. Let Nginx do the dirty work https://gist.github.com/odino/6750004f735c8d08687d example.org/v1/customers/1
  63. Let Nginx do the dirty work https://gist.github.com/odino/6750004f735c8d08687d example.org/customers/1 Api-Version: 1
  64. Let Nginx do the dirty work https://gist.github.com/odino/6750004f735c8d08687d $req->getHeader(‘Api-Version’)
  65. Let Nginx do the dirty work https://gist.github.com/odino/6750004f735c8d08687d Without polluting routing and controllers
  66. “I beg to differ”
  67. “I beg to differ” URL, subdomain, media type, header...
  68. “I beg to differ” Picking a wrong implementation doesn’t matter
  69. “I beg to differ” Picking a wrong implementation doesn’t matter AT ALL.
  70. “I beg to differ” How it impacts the design of your software matters
  71. “I beg to differ” #NoSilverBullet
  72. 4. Be Pragmatic
  73. /login GET or POST?
  74. 5. Testing
  75. cURL is your best friend curl -X GET https://api.namshi.com/products curl -X POST https://api.namshi.com/order -data=”{...}” curl -X DELETE ... curl -X PATCH ...
  76. cURL is your best friend
  77. cURL is your best friend
  78. cURL is your best friend
  79. cURL is your best friend https://docs.python.org/2/library/json.html
  80. httparty https://docs.python.org/2/library/json.html
  81. httpie https://github.com/jkbr/httpie
  82. smoke tests made easy
  83. consuming/testing apis locally https://gist.github.com/cirpo/92fa22d4c45fddf0ccfa
  84. consuming/testing apis locally https://gist.github.com/cirpo/c6d497c5654094904306
  85. testing apis
  86. Android 2.3 native browser
  87. testing apis
  88. testing apis you can even decrypt the https responses :)
  89. 6. Design
  90. An API is a layer on top of your domain
  91. Pick the layer that is most suitable to your needs
  92. HTTP APIs are a good start
  93. REST is a DREAM
  94. POST or PUT? HTTP METHOD
  95. PUT or PATCH? HTTP METHOD
  96. /users/johnny/tags USER TAGS
  97. USER TAGS to remove a tag PUT, PATCH or DELETE?
  98. USER TAGS deleting a non-existent tag 200 or 204 or 404? http://stackoverflow. com/questions/2342579/http-status-code- for-update-and-delete
  99. USER TAGS deleting a non-existent tag 200 or 204 or 404? http://stackoverflow. com/questions/2342579/http-status-code- for-update-and-delete ON STACKOVERFLOW THEY’RE STILL FIGHTING http://stackoverflow.com/questions/2342579/http-status-code-for-update-and-delete
  100. be consistent
  101. NAMING /user/1 /users /order/1 /orders
  102. NAMING /city/1 /cities /curriculum/1 /curricula
  103. NAMING /user/1 /users /order/1 /orders /city/1 /cities /curriculum/1 /curricula
  104. NAMING /user/1 /users /order/1 /orders /city/1 /cities /curriculum/1 /curricula not good AT ALL!
  105. STICK WITH PLURALS
  106. NAMING /users/1 /users /orders/1 /orders /cities/1 /cities /curricula/1 /curricula
  107. UNIQUE RESOURCES /users/1 /users/cirpo /users/A323K833
  108. UNIQUE RESOURCES /orders/15 /orders/A323K833
  109. UNIQUE RESOURCES AVOID INCREMENTAL NUMBER (if it’s business critical)
  110. Unstructured APIs = API aggregation
  111. api.example.org/v1/latest-news
  112. latest news + metatags + banners + navigation yada yada yada
  113. Sort of a “wild” API for your whole app
  114. The client receives a GET on /something and will let the API figure out what /u/something actually is
  115. Orchestration Layers
  116. https://engineering.groupon.com/2013/misc/i-tier-dismantling-the-monoliths/
  117. “Most APIs are designed by the API provider with the goal of maintaining data model purity. When building an OL, be prepared to sometimes abandon purity in favor of optimizations and/or performance.” Daniel Jacobson, director of engineering for the Netflix API http://www.infoq.com/presentations/API-Revolution
  118. DOMAIN users orders stock images
  119. DOMAIN Think about collections not controllers
  120. DOMAIN PUT/PATCH try to always plan for full updates
  121. uniform responses
  122. codebase organization
  123. codebase organization one bundle for each api? one bundle for each application? one app for each sets of api?
  124. codebase organization start with an app organize bundles semantically create shared bundles
  125. codebase organization BUNDLES product checkout warehouse generic entity
  126. 7. Scalability
  127. CACHE ALL THE THINGS!
  128. Middlewares to the rescue!
  129. CONNECT https://gist.github.com/cirpo/e9ec20871e2e8d433f8d
  130. STACK https://gist.github.com/cirpo/11296317
  131. STACK https://gist.github.com/odino/b3fdacceaa0cce65fbce
  132. Avoid sessions
  133. Everything as a resource http://leaphly.org/
  134. 8. We have a problem
  135. CORS
  136. CORS
  137. iFrames to the rescue!
  138. iFrames to the rescue! domain.org includes an iframe from api.domain.org
  139. iFrames to the rescue! then sends it a message through the postMessage API
  140. iFrames to the rescue! the iFrame triggers the ajax request on its own domain with the parameters in the message
  141. iFrames to the rescue! and sends the result back to the caller
  142. iFrames to the rescue! and sends the result back to the caller #ghetto
  143. CORS xDomain, cross-browser without CORS https://github.com/jpillora/xdomain
  144. CORS great idea, but Jaime is alone :(
  145. CORS poor file upload support
  146. CORS no automated tests
  147. CORS not a long-term solution :’-(
  148. CORS xAuth, a standard https://github.com/xauth/xauth
  149. CORS initially thought to provide a decentralized auth service
  150. CORS on the centralized xauth.org http://hueniverse.com/2010/06/05/xauth-a-terrible-horrible-no-good-very-bad-idea/
  151. CORS Dead.
  152. CORS DEAD.
  153. CORS Use an API proxy
  154. CORS example.org/api/
  155. (silly) browsers
  156. (silly) browsers if a cross-domain request is cacheable, the android browser goes nuts
  157. (silly) browsers The request does not include the Origin header
  158. (silly) browsers Status code: 0
  159. (silly) browsers WHAT. THE. HECK. http://opensourcehacker.com/2011/03/20/android-webkit-xhr-status-code-0-and-expires- headers/
  160. “Standards”
  161. Don’t play with fire
  162. Don’t play with fire 1 API, N clients consuming it
  163. Don’t play with fire desktop browser, mobile browser, ios app, android app...
  164. Don’t play with fire Keep as much logic as possible on the server
  165. Don’t play with fire Less things to implement on every client and centralized implementations
  166. Don’t play with fire make it easy for the API clients
  167. Don’t play with fire POST https://api.example.com/login 200 OK date: Thu, 01 May 2014 21:52:33 GMT content-type: application/json transfer-encoding: chunked connection: close set-cookie: login=...; cache-control: no-cache { "email"=>"alessandro.nadalin@gmail.com", "firstName"=>"Alex", "lastName"=>"Nadalin", "birthday"=>"21/10/1988", }
  168. Security matters
  169. Security matters [ "odino@gmail.com", "cirpo@gmail.com", ... ]
  170. Security matters for(;;);[ "odino@gmail.com", "cirpo@gmail.com", ... ]
  171. Security matters while(1);[ "odino@gmail.com", "cirpo@gmail.com", ... ]
  172. Security matters while(1);[ "odino@gmail.com", "cirpo@gmail.com", ... ] http://bit.ly/why-does-google
  173. Security matters Avoid [...] http://bit.ly/json-hijacking
  174. Security matters Use {...}
  175. That’s all folks
  176. github.com/cirpo twitter.com/cirpo cirpo.org github.com/odino twitter.com/_odino_ odino.org Namshi Lead Developer Namshi VP Technology
  177. github.com/cirpo cirpo.org thank you! joind.in/11310
  178. we are hiring! tech.namshi.com/join-us github.com/namshi twitter.com/TechNamshi tech.namshi.com
  179. CREDITS http://www.panoramio.com/photo/30329016 https://farm3.staticflickr.com/2199/2365883747_3a5c753719_o.jpg http://news.buzzbuzzhome.com/2013/04/top-7-aerial-photos-cities.html https://www.flickr.com/photos/superlekker/5917559189/sizes/l https://www.flickr.com/photos/derekbruff/12336187505/sizes/l https://www.flickr.com/photos/chberge/3803475294/sizes/l https://www.flickr.com/photos/neilsingapore/8057578769 https://www.flickr.com/photos/dionnehartnett/6805481856/sizes/l https://www.flickr.com/photos/thomashawk/186339737 https://www.flickr.com/photos/cesarastudillo/3981364314/sizes/l https://www.flickr.com/photos/an_untrained_eye/6630719431 https://www.flickr.com/photos/30835738@N03/7936491790/sizes/l https://www.flickr.com/photos/deboni/2959228565/sizes/l https://www.flickr.com/photos/ghalog/6782751111/sizes/l https://www.flickr.com/photos/timzim/177640262/sizes/o/ https://www.flickr.com/photos/innoxiuss/2824204305 https://www.flickr.com/photos/hawk59/6038847752/sizes/l https://www.flickr.com/photos/remydwd/5487417702/sizes/l https://www.flickr.com/photos/rammorrison/4359793666/sizes/o/ https://www.flickr.com/photos/piers_nye/2501994750/sizes/o/ https://www.flickr.com/photos/danielygo/7559750132/sizes/l https://www.flickr.com/photos/msc72/2600035028/sizes/l https://www.flickr.com/photos/sicilianitaliano/3609275241/sizes/l https://www.flickr.com/photos/scottmontreal/7235110028/sizes/l https://www.flickr.com/photos/piet_musterd/6170853224/sizes/l https://www.flickr.com/photos/music_embassy/7137413247/sizes/l http://upload.wikimedia.org/wikipedia/commons/9/9c/William_James_b1842c.jpg http://theverybesttop10.files.wordpress.com/2013/08/the-world_s-top-10-things-no-person-with-a-ocd-should-see-1. jpg https://www.flickr.com/photos/62244271@N03/8553590682/sizes/l

×