I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem

5,310 views

Published on

The Ruby ecosystem is pretty awesome when it comes to developing or
consuming HTTP APIs. On the publishing front, the Rails framework is
an attractive option because it supports publishing what are popularly
(but inaccurately) referred to as 'RESTful' APIs quickly and
effortlessly. On the consumer side, the Ruby ecosystem provides
several very fluent and powerful libraries that make it easy to
consume HTTP based APIs.

Since a significant proportion of projects today require that APIs be
both published and consumed, many of them wind up choosing Ruby as a
platform for the reasons mentioned above. This talk is targeted at
folks that are currently on such projects, or anticipate being on such
projects in the future.

We will cover:
Consuming HTTP APIs:
1) The basics of making HTTP calls with Ruby
2) The strengths and weaknesses of Ruby's Net::HTTP across 1.8, 1.9
and JRuby (possibly Rubinius if we have the time to do research)
3) Popular HTTP libraries that either make it easier to do HTTP by
providing better APIs, make it faster by using libCurl or both
4) Different approaches to deserializing popular encoding formats such
as XML and JSON and the pitfalls thereof

Producing HTTP APIs using Rails:
1) The basics of REST
2) What Rails gives you out of the box - content-type negotiation,
deserialization etc. and the limitations thereof
3) What Rails fails to give you out of the box - hypermedia controls etc.
4) What Rails does wrong - wrong PUT semantics, no support for PATCH,
error handling results in responses that violate the clients Accepts
header constraints etc.
4) How one can achieve Level 2 on the Richardson Maturity Model of
REST using Rails
5) Writing tests for all of this

At the end of this, our audience will understand how you can both
consume and produce HTTP APIs in the Ruby ecosystem. They will also
have a clear idea of what the limitations of such systems are and what
the can do to work around the limitations.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,310
On SlideShare
0
From Embeds
0
Number of Embeds
1,894
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Opensource we have done in this area\n
  • This talk is not about NetHTTP vs libcurl\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • This talk does have *some* structure; we’ll look at APIs from the perspective of the producer and the consumer. With a little bit left over for stuff that bridges the gap.\n
  • \n
  • Everyone talks about how easy it is to produce APIs with Sinatra and such like, but lets be honest. There’s an elephant in the room - those of you that freelance know exactly how many requests we get for APIs on Sinatra versus APIs on...\n
  • [Handoff]\n\n...Rails. Love it or hate it, Rails has a standard way of producing APIs, and understanding what Rails can or cannot do for you is important. Sinatra does not have these constraints, so you’re free to roll APIs any way you want, so we’ll focus on the framework that does impose significant patterns.\n
  • [achamian]\n\nBut before we launch into Rails’ capabilities, it’s important to put things into context first by talking a little about REST\n
  • What is a RESTful API?\n
  • For this we turn to Leonard Richardson who came up with a maturity model that has gotten so famous it has an acronym and looks like...\n
  • ...this, popularly known as the Richardson Maturity Model, or RMM. \nIt begins with Level 0, which uses plain old xml (or a similar encoding format) to tunnel requests to a HTTP endpoint. Think RPC. Clearly Rails is better than this.\nNext comes Level 1, which introduces the concept of Resources - every URI represents a resource. Easy peasy. Rails makes it easy to do this - but you’re free to deviate.\nThen comes Level 2, which defines a certain set of constraints around how HTTP verbs are used. Rails violates the semantics by using PUT instead of PATCH - this was fixed on May 07 with Pull request #348, so future releases should be L2 compliant.\nFinally, we come to L3 which involves using hypermedia to define the edges of the state graph of an application using hypermedia links. This definitely isn’t in Rails. Yet.\n
  • ...this, popularly known as the Richardson Maturity Model, or RMM. \nIt begins with Level 0, which uses plain old xml (or a similar encoding format) to tunnel requests to a HTTP endpoint. Think RPC. Clearly Rails is better than this.\nNext comes Level 1, which introduces the concept of Resources - every URI represents a resource. Easy peasy. Rails makes it easy to do this - but you’re free to deviate.\nThen comes Level 2, which defines a certain set of constraints around how HTTP verbs are used. Rails violates the semantics by using PUT instead of PATCH - this was fixed on May 07 with Pull request #348, so future releases should be L2 compliant.\nFinally, we come to L3 which involves using hypermedia to define the edges of the state graph of an application using hypermedia links. This definitely isn’t in Rails. Yet.\n
  • ...this, popularly known as the Richardson Maturity Model, or RMM. \nIt begins with Level 0, which uses plain old xml (or a similar encoding format) to tunnel requests to a HTTP endpoint. Think RPC. Clearly Rails is better than this.\nNext comes Level 1, which introduces the concept of Resources - every URI represents a resource. Easy peasy. Rails makes it easy to do this - but you’re free to deviate.\nThen comes Level 2, which defines a certain set of constraints around how HTTP verbs are used. Rails violates the semantics by using PUT instead of PATCH - this was fixed on May 07 with Pull request #348, so future releases should be L2 compliant.\nFinally, we come to L3 which involves using hypermedia to define the edges of the state graph of an application using hypermedia links. This definitely isn’t in Rails. Yet.\n
  • So what level of compliance can we achieve with a minimum of effort?\n
  • [Handoff]\n\nI’d say an L2 is quite easily do-able in rails without too much effort. Now lets talk about how.\n
  • [kaiwren]\n\nSo after that little digression, lets go back to what we were originally talking about - how to build APIs on Rails.\n
  • [kaiwren]\n\nThis is one of the easiest thing to forget - APIs are for machines and software, websites are for people. Mixing the two is Not Good.\n
  • \n
  • \n
  • Hollywood has shown us the consequence of mixing people stuff with machine stuff time and again, and it isn’t good.\n
  • \n
  • \n
  • \n
  • and REST cares only about state transitions. If you mix these, then remember that this only works in the most trivial of use cases. Beyond that...\n
  • \n
  • \n
  • Your app doesn’t have a single HTML page. This is especially useful if you expect 3rd parties to use your APIs - you’re dogfooding your own stuff from day one.\n
  • [Handoff]\n\nif your api is being consumed by 3rd parties, be aware that they will use a fairly eclectic collection of HTTP libs. What’s convenient/intuitive when using AR may be extremely obscure when using simple HTTP calls.\n
  • [achamian]\n\nThe next thing everybody should keep an eye on when building Rails APIs are the HTTP status codes. There are many of them, and your APIs should respect them.\n
  • The first thing everybody ignores when building Rails APIs are the HTTP status codes. There are many of them, and your APIs should respect them.\n
  • Adhere to these defaults in the code you write. They’re there for a reason. Don’t, for example, return a 200 with an error message in the body - that’s bad form.\n
  • There are, however, a couple of codes they deal with weirdly, or not at all\n
  • The handling of 404s can be strange, driven as it is in certain cases by the ActiveRecord::RecordNotFound exception. Make sure that if a resource is not found, you’re always returning a 404 status code and you write a spec to expect it.\n
  • Have a unique constraint on a resource identifier which is part of the input and not system assigned? When it’s violated, return a 409-Conflict and not a 422-Unprocessable entity.\n
  • It’s important to lock these codes in because they mean a lot to the outside world. When building apis, make sure all your controller specs include assertions on the response code. I’ve got a simple gem that can help if you use rspec. Try it out. That asserts that a response has a code of 418.\n
  • \n
  • \n
  • This way your application doesn’t need to deal with your own users through one auth mechanism and those that come through fb/twitter/github/sf through another\n
  • Oauth 2 requires SSL, so do keep that in mind.\n
  • \n
  • \n
  • If you have resources that don’t need to be refreshed as soon as they change, you should use an appropriate expires header to allow intermediate nodes on the network to cache them for a specified period of time. This can massively reduce load on your servers.\n
  • \n
  • \n
  • Just like any library or gem, your users require visibility and stability. Whether you use URL based versioning (by including v1/v2/v3) in the API url, or by headers that specify the version, make sure your versioning roadmap is clear, consistent and sensible. Try maintaining backward compatibility as far as is reasonable.\n
  • Content Types are rarely done wrong, so I won’t talk about them for too long. There’s only one thing that I’ve observed out here...\n
  • ... is folks tunneling application/x-www-form-urlencoded over application/xml or application/json.\n
  • \n
  • \n
  • a suffix of .xml or .json in the URI is to make it easier to look at api responses in the browser. While consuming an API you should stick to the Accept header.\n
  • There are a couple of important things when dealing with APIs that I’ve never done and have no first-hand experience with\n
  • Solved using rack middleware\n
  • [Handoff]\n
  • [kaiwren]\n
  • Are you talking to a rails app (typically within your own walled garden) or to an API out on the internets?\n
  • Let’s talk about the consumer that ships with Rails - ActiveResource\n
  • ActiveResource follows the 60/40 rule not 80/20 - it works for the straightforward use cases, and does so smoothly and well\n
  • Trying accessing the header information for a response\n
  • \n
  • \n
  • Passing parameters for a post request. Accepting access token as a parameter to a Get over https \n
  • \n
  • \n
  • \n
  • You don’t need gems for OAuth2 clients. Just roll your own - it takes 20 minutes tops.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Can I figure out the verb, the connection (if keep alive), corelate the req and response a situation where parallel threads are logging to the same log\n
  • These are the bits that both producers and consumers deal with\n
  • A significant proportion of the time an API takes to process something is spent doing serialisation and deserialization\n
  • Remember to switch your deserialization backend from REXML to Nokogiri which uses the native libxml library. It’s way way faster.\n
  • The default XML serializer in Rails is written in pure Ruby. It’s pretty fast, and there are drop in native alternatives based off libxml that are upto 50% faster, but that aren’t particularly popular.\n
  • The big gotcha with builder occurs when you’re generating custom XML\n
  • you want to run all the custom xml you create using builder’s DSLs through Nokogiri’s deserialise to make sure its valid XML in your specs\n
  • In addition to controller specs, it makes sense to have full, live tests hitting a real server, especially if you’re integrating across multiple APIs that you’ve authored\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • I can haz HTTP - Consuming and producing HTTP APIs in the Ruby ecosystem

    1. 1. I can haz HTTPConsuming and producing HTTP APIs in theRuby ecosystem
    2. 2. sidu ponnappa
    3. 3. kaiwren
    4. 4. @ponnappa
    5. 5. niranjan paranjape
    6. 6. achamian
    7. 7. @niranjan_p
    8. 8. Engineeringhttp://github.com/c42
    9. 9. Open Source (gem install) wrest(gem install) pox_paginate
    10. 10. Not about HTTP libs not per se...
    11. 11. More about...
    12. 12. REST
    13. 13. caching
    14. 14. serialisation & deserialisation
    15. 15. authentication
    16. 16. stuff like that...
    17. 17. Producer / Consumer
    18. 18. Producer
    19. 19. digression
    20. 20. RESTful?
    21. 21. Leonard Richardson
    22. 22. L0: Plain Old Xml
    23. 23. L1: ResourcesL0: Plain Old Xml
    24. 24. L2: HTTP Verbs L1: ResourcesL0: Plain Old Xml
    25. 25. L3: Hypermedia L2: HTTP Verbs L1: ResourcesL0: Plain Old Xml
    26. 26. reasonable compliance?
    27. 27. L2 is achievable with minimal effort in Rails
    28. 28. getting back to the point
    29. 29. Don’t mix machine stuff with human stuff
    30. 30. APIs are for machines
    31. 31. mixing the two causes trouble
    32. 32. basically, doing this causes troubledef index respond_to do |format| e format.html format.json { render :json => Project.all.to_json } endend
    33. 33. because, people like usable apps
    34. 34. separate API controllers from website controllers
    35. 35. or beyond even that...
    36. 36. your app only exposes APIs your website is a separate applicationthat consumes these APIs; think NewTwitter
    37. 37. don’t design exclusively for ActiveResource it isn’t a standard
    38. 38. HTTP status codes
    39. 39. rails has nice defaults 201 for creates 401 for auth violations 405 for L2 violations 406 for Content-Type violations 422 for invalid data 500 for server faults 200 for everything else
    40. 40. let rails help you
    41. 41. what they missed
    42. 42. 404
    43. 43. 409
    44. 44. assert on status codes gem install rspec-http response.should be_im_a_teapot
    45. 45. Authentication
    46. 46. OAuth 2 is nice
    47. 47. be an OAuth provider consistency makes life easier
    48. 48. https://
    49. 49. Caching
    50. 50. go beyond Rails’ local caches
    51. 51. the Cache-Control header is your friend
    52. 52. Versioning
    53. 53. successful APIs change
    54. 54. good versioning is criticalpublish clear roadmaps and deprecate sensibly
    55. 55. Content Types
    56. 56. don’t tunnelapplication/x-www-form-urlencoded
    57. 57. Accept headers
    58. 58. “http://localhost:3000/user.xml”
    59. 59. “http://localhost:3000/user.xml”
    60. 60. rails respects it you should use it
    61. 61. Important stuff that we’ve never done
    62. 62. Throttlinghttps://github.com/dambalah/api-throttling (thanks @godfoca)
    63. 63. Metering
    64. 64. Consumer
    65. 65. rails API vs. everything else
    66. 66. ActiveResource
    67. 67. 60:40
    68. 68. the rest of the time you monkeypatch but that isn’t so easy, so be careful
    69. 69. ActiveResource works best inside a walled garden (IMHO)
    70. 70. Producer maturity
    71. 71. some APIs are weird
    72. 72. everyone’s standardizing on REST but it’s what they call REST
    73. 73. suffice to say, one size doesn’t fit all except for OAuth 2, which everyone’s adopting
    74. 74. building a clean domain model for the API is key
    75. 75. OAuth 2 is ridiculously easy to write a client for #justSaying https://github.com/kaiwren/wrest/tree/master/ examples/facebook_auth
    76. 76. authorisation on the otherhand still has no standard implementation
    77. 77. What should a good HTTP client do for you?
    78. 78. this one needs bullets• Verbs - should support GET, POST, PUT & DELETE.• If it supports PATCH & OPTIONS, epic.• Transparent deserialization of XML and JSON, with the option to easily add more• HTTP caching (RFC 2616)• Keep-alive connections (because negotiating SSL can get expensive)• Thread safety• Transparent compression/decompression using Accept-Encoding• A fluent, easy to use API
    79. 79. most importantly...
    80. 80. verbs
    81. 81. serializationI rather like ActiveSupport
    82. 82. caching RFC 2616
    83. 83. thread safety
    84. 84. fluent apiis it easy to work with uris, because you’ll do a fair bit of that when building domains
    85. 85. is it easy to build a domain model using it?
    86. 86. logging/debugging
    87. 87. Shared stuff
    88. 88. serialization/deserialization
    89. 89. libxmlActiveSupport::XmlMini.backend = Nokogiri
    90. 90. BuilderPure Ruby XML serializer. Pretty Fast. Just does String concats.
    91. 91. gotcha
    92. 92. no DOM for Builderso no guarantee that you get valid XML
    93. 93. testingthis can be tricky
    94. 94. Conclusion
    95. 95. respect HTTP
    96. 96. respect REST constraints
    97. 97. spend less time messing with plumbing
    98. 98. focus on building a clean domain model
    99. 99. Photo creditsDavid Blackwell, Elephant in Room: http://www.flickr.com/photos/mobilestreetlife
    100. 100. Funny picture?cat with mausambi hat
    101. 101. 42 Sidu Ponnappa Niranjan Paranjapehttp://twitter.com/ponnappa http://twitter.com/niranjan_phttps://github.com/kaiwren https://github.com/achamian

    ×