Your SlideShare is downloading. ×
Ruby HTTP clients comparison
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Ruby HTTP clients comparison

11,843
views

Published on


4 Comments
38 Likes
Statistics
Notes
No Downloads
Views
Total Views
11,843
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
79
Comments
4
Likes
38
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Ruby HTTP clients comparison Hiroshi Nakamura nahi at Twitter, github Technical Architect at Appirio Japan CRuby and JRuby committer Asakusa.rb: http://qwik.jp/asakusarb/
  • 2. Ruby HTTP Clients Matrixadvantages-and-disadvantages comparison ofHTTP client libraries http://bit.ly/RubyHTTPClients2012 Disclaimer: Im the author of "httpclient"
  • 3. Agendanet/http16 libraries I pickedRuby HTTP Clients Matrix API style Compatibility Supported featuresPerformance ComparisonsMy Recommendations
  • 4. net/http Net::HTTP::Proxy? net/https?
  • 5. Ruby HTTP Client libraries
  • 6. HTTP client libraries I didn’t evaluateCannot evaluate• activeresource (Rails specific)• http (under development)• http_request.rb (test doesnt pass)• nestful (no test)• typhoeus (under heavy rewrite)Obsolete• eventmachine (built-in client is obsolete)• right_http_connection (no update)• simplehttp (no update)• rfuzz (no update)
  • 7. Evaluation AxisProject StatsAPI styleCompatibility: CRuby, JRuby, RubiniusSupported features Connection features Basic HTTP features Development support Advanced features http://bit.ly/RubyHTTPClientsFeatureTest (test/unit scripts)
  • 8. Project Stats
  • 9. Project Stats
  • 10. API style
  • 11. sync API (1/5) - Client instancenet/http, mechanize, httpclient, patron, curb, faraday client = Net::HTTP.new(host, port) p client.get(path).body client = Mechanize.new client = HTTPClient.new client = Patron::Session.new p client.get(url).body curl = Curl::Easy.new(url) curl.http_get p curl.body_str client = Faraday.new(:url => baseurl) p client.get(path).body
  • 12. sync API (2/5) - Client class restfulie, excon, httpi p Restfulie.at(url).get!.body p Excon.get(url).body p HTTPI.get(url).body
  • 13. sync API (3/5) - Resource rest-client, rufus-verbsrs = RestClient::Resource.new(http://example.com)rs[posts/1/comments].post Good article., :content_type => text/plainep = EndPoint.new( :host => "resta.farian.host", :port => 80, :resource => "inventory/tools")res = ep.get :id => 1res = ep.get :id => 2
  • 14. sync API (4/5) - Include & Customize httparty, weary client = Class.new { include HTTParty } p client.get(url) class WearyClient < Weary::Client domain http://api.target.org/ get :retrieve, {path} end p WearyClient.new.fetch(:path => path).perform.body
  • 15. sync API (5/5) - Others open-uri.rb, wrest# open-uri.rbp open(url) { |f| f.read }# wrestp http://www.google.co.jp/.to_uri.get.body
  • 16. async API(1/2) - Callback em-http-request body = nil EM.run do req = EM::HttpRequest.new( http://www.google.com/).get req.callback do body = req.response EM.stop end req.errback do body = nil end end p body
  • 17. async API(2/2) - Polling httpclient, weary, wrest client = HTTPClient.new conn = client.get_async(url) conn.finished? # => false # ... io = conn.pop.content while str = io.read(4096) p str end
  • 18. parallel API - curbresponses = []m = Curl::Multi.newurls.each do |url| responses[url] = m.add(Curl::Easy.new(url) { |curl| curl.on_body { |data| responses[url] << data data.bytesize } })endm.performp responses.map { |e| e.bytesize }
  • 19. API style
  • 20. Compatibility
  • 21. Compatibility
  • 22. Connection features
  • 23. 3 types of HTTP connections
  • 24. Keep-Alive in em-http-request body = [] EM.run do conn = EventMachine::HttpRequest.new(server.url) req1 = conn.get(:keepalive => true) req1.callback { body << req1.response req2 = conn.get(:keepalive => true) req2.callback { body << req2.response req3 = conn.get(:keepalive => true) req3.callback { body << req3.response req4 = conn.get(:keepalive => true) req4.callback { body << req4.response EM.stop req4.errback { ... }} req3.errback { ... }} req2.errback { ... }} req1.errback { ... } end
  • 25. Pipelining in em-http-request body = [] EM.run do conn = EventMachine::HttpRequest.new(server.url) req1 = conn.get(:keepalive => true) req2 = conn.get(:keepalive => true) req3 = conn.get(:keepalive => true) req4 = conn.get() req1.callback { body << req1.response } req2.callback { body << req2.response } req3.callback { body << req3.response } req4.callback { body << req4.response; EM.stop } req1.errback { ... } req2.errback { ... } req3.errback { ... } req4.errback { ... } end
  • 26. NO verification by default?!if http.use_ssl? http.verify_mode = OpenSSL::SSL::VERIFY_NONE if options[:ssl_ca_file] http.ca_file = options[:ssl_ca_file] http.verify_mode = OpenSSL::SSL::VERIFY_PEER endendoptions[:ssl_ca_file] == nil => VERIFY_NONE
  • 27. Connection features
  • 28. Basic HTTP features
  • 29. IRI: Internationalized Resource Identifierclient.get("http://www.ebooks.com/797059/some-kind-of-peace/grebe-camilla-träff-åsa-norlen-paul/") uri.rb doesnt support IRI addressable/uri does
  • 30. "Cross-site Cooking" bug in httpclientSet-Cookie: TEST=test; path=/; domain=.com httpclient eats this cookie and send it to all *.com site Mechanize handles it properly like browsers http://en.wikipedia.org/wiki/Cross-site_cooking
  • 31. Streaming upload/download# Chunked upload with Patronclient = Patron::Session.newres = client.request(:post, url, {}, :file => path_to_upload)# Chunked downloadclient.get_file(url, path_to_write)# Chunked upload with em-http-requestreq = EM::HttpRequest.new(url).post :file => path_to_upload# Chunked downloadreq = EM::HttpRequest.new(url).getreq.stream do |chunk| p chunkend
  • 32. Basic HTTP features
  • 33. Development Support
  • 34. Response stubbing# Stubbing response bodyclient = HTTPClient.newclient.test_loopback_response << Hello!client.get(http://www.example.com/hello).body#=> "Hello!"# Stubbing HTTP responseclient.test_loopback_http_response << "HTTP/1.0 302 Found¥r¥nLocation: http://foo/¥r¥n¥r¥n" << "HTTP/1.0 200 OK¥r¥n¥r¥nHello!"client.post(http://www.example.com/todo, :follow_redirect => true, :body => {}).body#=> "Hello!"
  • 35. IRB like shell rest-client, httpclient, wrest% restclient https://example.com user pass>> delete /private/resource% httpclient>> get "https://www.google.com", :q => :ruby% wrest>> http://www.google.com?q=ruby.to_uri.get
  • 36. Replayable log rest-client% RESTCLIENT_LOG=/tmp/restclient.log restclient>> RestClient.get "https://www.google.com/"...% cat /tmp/restclient.logRestClient.get "https://www.google.com/", "Accept"=>"*/*;q=0.5, application/xml", "Accept-Encoding"=>"gzip, deflate"# => 200 OK | text/html 13354 bytes
  • 37. Development Support
  • 38. Advanced features
  • 39. HTML form handling of Mechanize agent = Mechanize.new page = agent.get(url) form = page.form(login) form.email = nahi@ruby-lang.org form.password = jKH.P945wruV*qh3 page = agent.submit(form, form.button(submit))
  • 40. Advanced features
  • 41. Testing your clientwebmock by Bartosz Blimke (bblimke) Library for stubbing and setting expectations on HTTP requests in Ruby.vcr by Myron Marston (myronmarston) Record your test suites HTTP interactions and replay them during future test runs for fast, deterministic, accurate tests.
  • 42. Performance ComparisonsServer Linode Xen VPS (Linode 512) at Fremont, CA Ubuntu 10.10 Apache 2.2, KeepAlive OnClient AWS EC2 (m1.small) at North Virginia (us-east-1b) Ubuntu 12.04 HTTP clients w/ CRuby 1.9.3p286Multiple downloads of 177B.html and 24MB.zipDont take it serious! http://bit.ly/RubyHTTPClientsBenchmarkScript
  • 43. Multiple 177B downloads [sec]
  • 44. [sec]
  • 45. [sec]
  • 46. My Recommendations• Speed is the king => em-http-request, curb w/ multi• HTML operation, Cookies => Mechanize• API client => Faraday and adapter based impls• SSL, Connectivity => httpclientCheck the matrix before you use the librariesPlease let me know when you find incorrect cell
  • 47. Development Timeline