Ruby Proxies for Scale, Performance, and Monitoring

Ruby Proxies for Scale, Performance and Monitoring Ilya Grigorik @igrigorik Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

postrank.com/topic/ruby The slides… Twitter My blog Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Code + Examples EventMachine Misc Proxies Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy Love Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

“Rails, Django, Seaside, Grails…” cant scale. Myth: Slow Frameworks Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

The Proxy Solution Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

The “More” Proxy Solution Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Transparent Scalability Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Load Balancer Reverse Proxy App Server MySQL Proxy Proxy as Middleware middleware ftw! Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

90% use case %w*Transparent Intercepting Caching …+ There are many different types! Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Transparent HAProxy App server A App server B Transparent, Cut-Through Proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Transparent Proxy = Scalability Power Tool Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy Proxy App server A App server B App server C Problem: Staging Environment Production Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Simulating traffic? Proxy Duplication App server C “Representative Load / Staging” Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

github.com/igrigorik/autoperf Replay log data, rinse, repeat Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Profile of queries has changed Fail Load on production has changed Fail Parallel environment Fail Slower release cycle Fail Staging fail. Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Duplex Ruby Proxy, FTW! Production Real (production) traffic Benchmark Benchmarking Proxy flash of the obvious Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

github.com/igrigorik/em-proxy Proxy DSL FTW! Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

EventMachine: Speed + Convenience building high performance network apps in Ruby Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

p quot;Startingquot; while true do EM.run do timers p quot;Running in EM reactorquot; network_io end other_io end puts quot;Almost donequot; EventMachine Reactor concurrency without threads Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

C++ core Easy concurrency without threading EventMachine Reactor concurrency without threads Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

http = EM::HttpRequest.new('http://site.com/').get http.callback { p http.response } # ... do other work, until callback fires. Event = IO event + block or lambda call EventMachine Reactor concurrency without threads Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

EM.run do EM.add_timer(1) { p quot;1 second laterquot; } EM.add_periodic_timer(5) { p quot;every 5 secondsquot;} EM.defer { long_running_task() } end class Server < EM::Connection def receive_data(data) send_data(quot;Pong; #{data}quot;) end def unbind p [:connection_completed] end end EM.run do EM.start_server quot;0.0.0.0quot;, 3000, Server end Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

EM.run do EM.add_timer(1) { p quot;1 second laterquot; } EM.add_periodic_timer(5) { p quot;every 5 secondsquot;} EM.defer { long_running_task() } end class Server < EM::Connection def receive_data(data) send_data(quot;Pong; #{data}quot;) Connection Handler end def unbind p [:connection_completed] end end EM.run do Start Reactor EM.start_server quot;0.0.0.0quot;, 3000, Server end Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

http://bit.ly/aiderss-eventmachine by Dan Sinclair (Twitter: @dj2sincl) Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxies for Monitoring, Performance and Scale welcome to the wonderful world of… Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn| conn.server :name, :host => quot;127.0.0.1quot;, :port => 81 conn.on_data do |data| # ... end Relay Server conn.on_response do |server, resp| # ... end conn.on_finish do # ... end end EM-Proxy www.github.com/igrigorik/em-proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn| conn.server :name, :host => quot;127.0.0.1quot;, :port => 81 conn.on_data do |data| # ... end Process incoming data conn.on_response do |server, resp| # ... end conn.on_finish do # ... end end EM-Proxy www.github.com/igrigorik/em-proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn| conn.server :name, :host => quot;127.0.0.1quot;, :port => 81 conn.on_data do |data| # ... end Process response data conn.on_response do |server, resp| # ... end conn.on_finish do # ... end end EM-Proxy www.github.com/igrigorik/em-proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn| conn.server :name, :host => quot;127.0.0.1quot;, :port => 81 conn.on_data do |data| # ... end conn.on_response do |server, resp| # ... end Post-processing step conn.on_finish do # ... end end EM-Proxy www.github.com/igrigorik/em-proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

%w[ <Transparent> Intercepting Caching … + solution for every problem Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn| conn.server :srv, :host => quot;127.0.0.1quot;, :port => 81 # modify / process request stream conn.on_data do |data| p [:on_data, data] data end No data modifications # modify / process response stream conn.on_response do |server, resp| p [:on_response, server, resp] resp end end Port-Forwarding transparent proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn| conn.server :srv, :host => quot;127.0.0.1quot;, :port => 81 conn.on_data do |data| data end conn.on_response do |backend, resp| Alter response resp.gsub(/hello/, 'good bye') end end Port-Forwarding + Alter transparent proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

%w[ Transparent <Intercepting> Caching … + solution for every problem Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn| @start = Time.now @data = Hash.new(quot;quot;) Prod + Test conn.server :prod, :host => quot;127.0.0.1quot;, :port => 81 conn.server :test, :host => quot;127.0.0.1quot;, :port => 82 conn.on_data do |data| data.gsub(/User-Agent: .*?rn/, 'User-Agent: em-proxyrn') end conn.on_response do |server, resp| @data[server] += resp resp if server == :prod end conn.on_finish do p [:on_finish, Time.now - @start] p @data end end Duplex HTTP: Benchmarking Intercepting proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn| @start = Time.now @data = Hash.new(quot;quot;) conn.server :prod, :host => quot;127.0.0.1quot;, :port => 81 conn.server :test, :host => quot;127.0.0.1quot;, :port => 82 conn.on_data do |data| data.gsub(/User-Agent: .*?rn/, 'User-Agent: em-proxyrn') end conn.on_response do |server, resp| Respond from production @data[server] += resp resp if server == :prod end conn.on_finish do p [:on_finish, Time.now - @start] p @data end end Duplex HTTP: Benchmarking Intercepting proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 80) do |conn| @start = Time.now @data = Hash.new(quot;quot;) conn.server :prod, :host => quot;127.0.0.1quot;, :port => 81 conn.server :test, :host => quot;127.0.0.1quot;, :port => 82 conn.on_data do |data| data.gsub(/User-Agent: .*?rn/, 'User-Agent: em-proxyrn') end conn.on_response do |server, resp| @data[server] += resp resp if server == :prod end Run post-processing conn.on_finish do p [:on_finish, Time.now - @start] p @data end end Duplex HTTP: Benchmarking Intercepting proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

[ilya@igvita] > ruby examples/appserver.rb 81 [ilya@igvita] > ruby examples/appserver.rb 82 [ilya@igvita] > ruby examples/line_interceptor.rb [ilya@igvita] > curl localhost >> [:on_finish, 1.008561] >> {:prod=>quot;HTTP/1.1 200 OKrnConnection: closernDate: Fri, 01 May 2009 04:20:00 GMTrnContent-Type: text/plainrnrnhello world: 0quot;, :test=>quot;HTTP/1.1 200 OKrnConnection: closernDate: Fri, 01 May 2009 04:20:00 GMTrnContent-Type: text/plainrnrnhello world: 1quot;} Duplex HTTP: Benchmarking Intercepting proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

[ilya@igvita] > ruby examples/appserver.rb 81 [ilya@igvita] > ruby examples/appserver.rb 82 [ilya@igvita] > ruby examples/line_interceptor.rb [ilya@igvita] > curl localhost STDOUT [:on_finish, 1.008561] {:prod=>quot;HTTP/1.1 200 OKrnConnection: closernDate: Fri, 01 May 2009 04:20:00 GMTrnContent-Type: text/plainrnrnhello world: 0quot;, :test=>quot;HTTP/1.1 200 OKrnConnection: closernDate: Fri, 01 May 2009 04:20:00 GMTrnContent-Type: text/plainrnrnhello world: 1quot;} Duplex HTTP: Benchmarking Intercepting proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Same response, different turnaround time Different response body! Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Woops! Validating Proxy easy, real-time diagnostics Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Hacking SMTP for fun and profit Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 2524) do |conn| conn.server :srv, :host => quot;127.0.0.1quot;, :port => 2525 # RCPT TO:<name@address.com>rn RCPT_CMD = /RCPT TO:<(.*)?>rn/ Intercept Addressee conn.on_data do |data| if rcpt = data.match(RCPT_CMD) if rcpt[1] != quot;ilya@igvita.comquot; conn.send_data quot;550 No such user herenquot; data = nil end end data end conn.on_response do |backend, resp| resp end Defeating SMTP Wildcards end Intercepting proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 2524) do |conn| conn.server :srv, :host => quot;127.0.0.1quot;, :port => 2525 # RCPT TO:<name@address.com>rn RCPT_CMD = /RCPT TO:<(.*)?>rn/ conn.on_data do |data| Allow: ilya@igvita.com if rcpt = data.match(RCPT_CMD) if rcpt[1] != quot;ilya@igvita.comquot; conn.send_data quot;550 No such user herenquot; data = nil 550 Error otherwise end end data end conn.on_response do |backend, resp| resp end Defeating SMTP Wildcards end Intercepting proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

[ilya@igvita] > mailtrap run –p 2525 –f /tmp/mailtrap.log [ilya@igvita] > ruby examples/smtp_whitelist.rb > require 'net/smtp‘ > smtp = Net::SMTP.start(quot;localhostquot;, 2524) > smtp.send_message quot;Hello World!quot;, quot;ilya@aiderss.comquot;, quot;ilya@igvita.comquot; => #<Net::SMTP::Response:0xb7dcff5c @status=quot;250quot;, @string=quot;250 OKnquot;> > smtp.finish => #<Net::SMTP::Response:0xb7dcc8d4 @status=quot;221quot;, @string=quot;221 Seeyanquot;> > smtp.send_message quot;Hello World!quot;, quot;ilya@aiderss.comquot;, “missing_user@igvita.comquot; => Net::SMTPFatalError: 550 No such user here Duplex HTTP: Benchmarking Intercepting proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

[ilya@igvita] > mailtrap run –p 2525 –f /tmp/mailtrap.log [ilya@igvita] > ruby examples/smtp_whitelist.rb To: ilya@igvita.com > require 'net/smtp‘ > smtp = Net::SMTP.start(quot;localhostquot;, 2524) > smtp.send_message quot;Hello World!quot;, quot;ilya@aiderss.comquot;, quot;ilya@igvita.comquot; => #<Net::SMTP::Response:0xb7dcff5c @status=quot;250quot;, @string=quot;250 OKnquot;> > smtp.finish => #<Net::SMTP::Response:0xb7dcc8d4 @status=quot;221quot;, @string=quot;221 Seeyanquot;> > smtp.send_message quot;Hello World!quot;, quot;ilya@aiderss.comquot;, “missing_user@igvita.comquot; => Net::SMTPFatalError: 550 No such user here Denied! Duplex HTTP: Benchmarking Intercepting proxy Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

“Hacking SMTP”.gsub(/Hacking/, ’Kung-fu’) DIY spam filtering with Defensio Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 2524) do |conn| conn.server :srv, :host => quot;127.0.0.1quot;, :port => 2525 RCPT_CMD = /RCPT TO:<(.*)?>rn/ FROM_CMD = /MAIL FROM:<(.*)?>rn/ MSG_CMD = /354 Start your message/ MSGEND_CMD = /^.rn/ Intercept commands conn.on_data do |data| #… end conn.on_response do |server, resp| p [:resp, resp] if resp.match(MSG_CMD) @buffer = true @msg = quot;quot; end resp SMTP + SPAM Filtering end end building a state-machine Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 2524) do |conn| conn.server :srv, :host => quot;127.0.0.1quot;, :port => 2525 RCPT_CMD = /RCPT TO:<(.*)?>rn/ FROM_CMD = /MAIL FROM:<(.*)?>rn/ MSG_CMD = /354 Start your message/ MSGEND_CMD = /^.rn/ conn.on_data do |data| #… end conn.on_response do |server, resp| p [:resp, resp] if resp.match(MSG_CMD) Flag & Buffer message @buffer = true @msg = quot;quot; end resp SMTP + SPAM Filtering end end building a state-machine Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Save data conn.on_data do |data| @from = data.match(FROM_CMD)[1] if data.match(FROM_CMD) @rcpt = data.match(RCPT_CMD)[1] if data.match(RCPT_CMD) @done = true if data.match(MSGEND_CMD) if @buffer @msg += data Buffer data = nil end if @done #… end data SMTP + SPAM Filtering end building a state-machine Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

conn.on_data do |data| @from = data.match(FROM_CMD)[1] if data.match(FROM_CMD) @rcpt = data.match(RCPT_CMD)[1] if data.match(RCPT_CMD) @done = true if data.match(MSGEND_CMD) if @buffer @msg += data Flag end of message data = nil end if @done #… Process message end data SMTP + SPAM Filtering end building a state-machine Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

@buffer = false uri = URI.parse('http://api.defensio.com/app/1.2/audit/key.yaml') res = Net::HTTP.post_form(uri, { quot;owner-urlquot; => quot;http://www.github.com/igrigorik/em-proxyquot;, quot;user-ipquot; => quot;216.16.254.254quot;, quot;article-datequot; => quot;2009/05/01quot;, quot;comment-authorquot; => @from, Defensio API quot;comment-typequot; => quot;commentquot;, quot;comment-contentquot; => @msg}) defensio = YAML.load(res.body)['defensio-result'] p [:defensio, quot;SPAM: #{defensio['spam']}quot;] if defensio['spam'] conn.send_data quot;550 No such user herenquot; else data = @msg end SMTP + SPAM Filtering building a state-machine Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

@buffer = false uri = URI.parse('http://api.defensio.com/app/1.2/audit/key.yaml') res = Net::HTTP.post_form(uri, { quot;owner-urlquot; => quot;http://www.github.com/igrigorik/em-proxyquot;, quot;user-ipquot; => quot;216.16.254.254quot;, quot;article-datequot; => quot;2009/05/01quot;, quot;comment-authorquot; => @from, quot;comment-typequot; => quot;commentquot;, quot;comment-contentquot; => @msg}) defensio = YAML.load(res.body)['defensio-result'] p [:defensio, quot;SPAM: #{defensio['spam']}quot;] if defensio['spam'] Pass / Deny Message conn.send_data quot;550 No such user herenquot; else data = @msg end SMTP + SPAM Filtering building a state-machine Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Protocol Trace [:relay_from_backend, :srv, quot;354 Start your messagequot;] [:resp, quot;354 Start your messagequot;] [:srv, quot;nquot;] [:relay_from_backend, :srv, quot;nquot;] [:resp, quot;nquot;] [:connection, quot;Hello Worldrnquot;] [:connection, quot;.rnquot;] [:defensio, quot;SPAM: false, Spaminess: 0.4quot;] [:srv, quot;250 OKnquot;] [:relay_from_backend, :srv, quot;250 OKnquot;] [:resp, quot;250 OKnquot;] SMTP + SPAM Filtering building a state-machine Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

@PostRank: Beanstalkd + Ruby Proxy because RAM is still expensive Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

~ 93 Bytes of overhead per job ~300 Bytes of data / job x 80,000,000 jobs in memory ~ 30 GB of RAM = 2 X-Large EC2 instances Oi, expensive! Beanstalkd Math Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Observations: 1. Each job is rescheduled several times 2. > 95% are scheduled for > 3 hours into the future Memory is wasted… 3. Beanstalkd does not have overflow page-to-disk We’ll add it ourselves! Extending Beanstalkd Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

1 “Medium” EC2 Instance Intercepting Proxy MySQL EM-Proxy Beanstalkd @PostRank: “Chronos Scheduler” Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 11300) do |conn| conn.server :srv, :host => quot;127.0.0.1quot;, :port => 11301 PUT_CMD = /put (d+) (d+) (d+) (d+)rn/ conn.on_data do |data| if put = data.match(PUT_CMD) Intercept PUT command if put[2].to_i > 600 p [:put, :archive] # INSERT INTO .... conn.send_data quot;INSERTED 9999rnquot; data = nil end end data end conn.on_response do |backend, resp| resp end end Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Proxy.start(:host => quot;0.0.0.0quot;, :port => 11300) do |conn| conn.server :srv, :host => quot;127.0.0.1quot;, :port => 11301 PUT_CMD = /put (d+) (d+) (d+) (d+)rn/ conn.on_data do |data| if put = data.match(PUT_CMD) if put[2].to_i > 600 If over 10 minutes… p [:put, :archive] # INSERT INTO .... conn.send_data quot;INSERTED 9999rnquot; data = nil end end Archive & Reply data end conn.on_response do |backend, resp| resp end end Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Overload the protocol MySQL EM-Proxy PUT RESERVE, PUT, … put job, 900 Beanstalkd @PostRank: “Chronos Scheduler” Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

400% cheaper + extensible! ~79,000,000 jobs, 4GB RAM MySQL EM-Proxy PUT Upcoming jobs: ~ 1M RESERVE, PUT, … Beanstalkd @PostRank: “Chronos Scheduler” Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

%w[ Transparent <Intercepting> Caching … + solution for every problem Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Slides: http://bit.ly/railsconf-proxy Code: http://github.com/igrigorik/em-proxy Twitter: @igrigorik Thanks. Questions? The slides… Twitter My blog Ruby Proxies + EventMachine http://bit.ly/railsconf-proxy @igrigorik #railsconf

Ruby Proxies for Scale, Performance, and Monitoring

by Ilya Grigorik

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 56

Statistics

Ruby Proxies for Scale, Performance, and Monitoring Presentation Transcript

http://www.slideshare.net	48
http://coderwall.com	8