0-60 with Goliath: Building High Performance Ruby Web-Services

0-60 with Goliath
building high-performance (Ruby) web services
Ilya Grigorik
@igrigorik

conn =EM::HttpRequest.new('http://gogaruco.com')
r1 =conn.get :path => "speakers.html", :keepalive => true# 250 ms
r2 =conn.get :path => "schedule.html"# 300 ms
# wait until done …
Answer:
All of the above!
Total execution time is:
250 ms
300 ms
550 ms
~ 65% truthiness
~ 25% truthiness *
~ 10% truthiness **
HTTP Quiz
this is not a trick question…

“SPDY? We can’t even get HTTP right…”
the why, how, and how-to of Goliath

Server
Client
20 ms
TCP handshake
HTTP Request
headers, body
40 ms
processing
Multi-part body (*)
Terminate connection
HTTP 1.0
RFC 1945 (1996)
+ 40ms TCP setup (network)
+ 20ms request (network)
+ 40ms processing
+ 20ms response (network)
66% of time in network overhead

Benchmark client RTT, not just the server processing time
A public service announcement…

Keep-alive
Re-use open connection
No multiplexing, serial
Default to “on”
Pipelining
No multiplexing
Parallel requests
HTTP 1.1
RFC 2616 (1999)

+ 40ms processing
x 40ms TCP setup (network)
+ 40ms processing
200ms for two requests
Small win over HTTP 1.0
Keep-alive
RFC 2616 (1999)
* One gotcha…

Net:HTTP
Keep-alive
RFC 2616 (1999)
Connection: close < ugh!

+ 40ms processing
60% of time in network overhead
120ms for two requests – 50% improvement!
Pipelining
RFC 2616 (1999)

Connection setup: 50ms
Request 1: 300ms
Request 2: 250ms
Total time:
~250 ms
~300 ms
~350 ms
~600 ms
Pipelining Quiz
RFC 2616 (1999)

There is just one small gotcha…
Making HTTP Pipelining Usable on the Open Web
http://tools.ietf.org/html/draft-nottingham-http-pipeline-01

conn =EM::HttpRequest.new('http://gogaruco.com')
r1 =conn.get :path => "speakers.html", :keepalive => true# 250 ms
r2 =conn.get :path => "schedule.html"# 300 ms
Total execution time is:
250 ms
300 ms
550 ms
Keep-alive what? HTTP 1.0!
~ 65% truthiness
~ 25% truthiness *
~ 10% truthiness **
Good: Keep-alive + Pipelining
Bad: Keep-alive + Garbage
“I’m confused”
HTTP in the wild
it’s a sad state of affairs
Keep-alive: mostly works – yay!
Pipelining: disabled (except in Opera)

HTTP can be a high-performance transport
Goliath is our attempt to make it work

Goliath == v3 API stack
Open source in 2011
Growing community
“Social Analytics”
Rails frontend
Ruby backend
95%+ of traffic via API’s
+
+
Brief History
Goliath @ PostRank

Rails
…
HTTP API
HTTP API
HTTP API
HTTP API
SQL
SQL
SQL
…
SQL
SQL
Solr
SQL
Separate logical & physical services
Easy to tune, easy to maintain, easy to “scale”
Stable code, fault-tolerance
PRO
Higher upfront ops cost
Lots of cross-service communication
CON

www.goliath.io
Single responsibility web-services
AsyncHTTP response streaming + progressive notifications
AsyncHTTP request streaming + progressive notifications
Multiple requests within the same VM
Keep-alive support
Pipelining support
Ruby API & “X-Ruby friendly”
Easy to maintain & test
… lower ops costs
… full HTTP 1.1 support
… Ruby polyglot!

Client API
(optional) Fibers
optional async
“Sync API”
Routing
Middleware
async-rack
0.3 ms
(streaming) HTTP Parser
Ruby, JRuby, Rubinius …
HTTP 1.1
EventMachine
Network
Goliath
Optimize bottom up + minimal client API

Goliath: Hands on…

require'goliath'
classHello< Goliath::API
defresponse(env)
[200, {}, "Hello World"]
end
end
$>ruby hello.rb-sv –p 8000 –e production
Hello World
Simple Goliath server

classHello< Goliath::API
use Goliath::Rack::Params
use Goliath::Rack::JSONP
use Goliath::Rack::Validation::RequestMethod, %w(GET)
use Goliath::Rack::Validation::RequiredParam, {:key => 'echo'}
defresponse(env)
[200, {}, {pong: params['echo’]}]
end
end
Middleware
No rackup file

classBonjour< Goliath::API
defresponse(env)
[200, {}, "bonjour!"]
end
end
classRackRoutes< Goliath::API
map '/version'do
run Proc.new { |env| [200, {}, ["Version 0.1"]] }
end
get "/bonjour", Bonjour
not_found('/') do
# run Proc. new { ... }
end
end
Routing
simple, but powerful

classAsyncUpload< Goliath::API
defon_headers(env, headers)
env.logger.info'received headers: '+headers
end
defon_body(env, data)
env.logger.info'received data chunk: '+ data
end
defon_close(env)
env.logger.info'closing connection'
end
defresponse(env)
# called when request processing is complete
end
end
Async Request Processing
don’t need to wait for the full request…

classStream< Goliath::API
defresponse(env)
pt=EM.add_periodic_timer(1) { env.stream_send("hello") }
EM.add_timer(10) do
pt.cancel
env.stream_send("goodbye!")
env.stream_close
end
streaming_response 202, {'X-Stream' => 'Goliath’}
end
end
Async/Streaming Response
don’t need to render full response…

* Goliath::SPDY
classWebsocket< Goliath::WebSocket
defon_open(env)
env.logger.info”WebSocket opened”
end
defon_message(env, msg)
env.logger.info”WebSocket message: #{msg}”
end
defon_close(env)
env.logger.info”WebSocket closed”
end
defon_error(env, error)
env.logger.error error
end
end
Web-Sockets
simple backend extension

Ruby 1.9: Fibers

defresponse(env)
conn =EM::HttpRequest.new(’http://google.com/')
r1 =conn.get :query => {:q => 'gogaruco'}
r1.callback do
r2 =conn.get :query => {:q => 'rubyconf'}
r2.callback { ... }
r2.errback { ... }
end
r2.errback { ... }
streaming_response 200, {}
end
Async fun
{ { {} } } …

defresponse(env)
conn =EM::HttpRequest.new(’http://google.com/')
r1 =conn.get :query => {:q => 'gogaruco'}
if r1.error?
r2 =conn.get :query => {:q => 'rubyconf'}
else
# ...
end
[200, {}, r2.response]
end
Async + Fibers
Ruby gives us a choice

EM-Synchrony:
http://github.com/igrigorik/em-synchrony
em-http-request
em-memcached
em-mongo
em-jack
mysql2
redis
…
ConnectionPool
MultiRequest
Iterators
Inline async
TCPSocket
…
multi =EventMachine::Synchrony::Multi.new
multi.add :a, db.aquery("select sleep(1)")
multi.add :b, db.aquery("select sleep(1)")
res =multi.perform

describe HttpLogdo
it 'forwards to our API server'do
with_api(HttpLog, api_options) do |api|
get_request({}, err) do |c|
c.response_header.status.should== 200
c.response_header[’X-Header'].should =='Header'
c.response.should=='Hello from Responder'
end
end
end
end
Integration Testing
simple end-to-end testing

Goliath
http://github.com/postrank-labs/goliath
http://goliath.io/
Google Group: http://groups.google.com/group/goliath-io
Examples: GIT repo / examples
thoughts, questions?
now or later: ilya@igvita.com

http://www.igvita.com	22509
http://feeds.igvita.com	166
http://igvita.com	88
http://www.newsblur.com	59
http://bgror.com	54
http://localhost	21
http://translate.googleusercontent.com	15
http://lanyrd.com	14
http://coderwall.com	11
http://webcache.googleusercontent.com	9
http://201.217.50.130	7
http://www.hanrss.com	5
http://xianguo.com	5
http://beta.igvita.com	3
http://feeds.feedburner.com	3
http://tweetree.com	3
http://protopage.com	2
http://newsblur.com	2
http://igvita.com HTTP	2
http://www.directrss.co.il	2
http://213.8.145.174	1
http://127.0.0.1	1
http://207.46.192.232	1
http://www.slideshare.net	1

0-60 with Goliath: Building High Performance Ruby Web-Services

by Ilya Grigorik

Accessibility

Categories

Upload Details

Usage Rights

24 Embeds 22,984

Statistics

0-60 with Goliath: Building High Performance Ruby Web-Services Presentation Transcript