Your SlideShare is downloading. ×
0
No Callbacks, No Threads & Ruby 1.9<br />async & co-operative web-servers<br />Ilya Grigorik<br />@igrigorik<br />
The slides…<br />Twitter<br />My blog<br />
The state of art is not good enough.<br />(we’ve been stuck in the same local minima for several years)<br />
require"active_record”<br />ActiveRecord::Base.establish_connection(<br />  :adapter => "mysql",<br />  :username => "root...
require"active_record”<br />ActiveRecord::Base.establish_connection(<br />  :adapter => "mysql",<br />  :username => "root...
BHP<br />% power <br />    loss<br />WHP<br />
> 50% power loss!?<br />
Mongo<br />Couch<br />MySQL<br />PSQL<br />…<br />Drivers<br />Threads<br />Ruby VM<br />GIL<br />Fibers<br />…<br />Netwo...
2<br />Mongo<br />Couch<br />MySQL<br />PSQL<br />…<br />Drivers<br />Threads<br />Ruby VM<br />1<br />4<br />GIL<br />Fib...
Global Interpreter Lock is a mutual exclusion lock held by a programming language interpreter thread to avoid sharing code...
N-M thread pool in Ruby 1.9…<br />Better but still the same problem!<br />Concurrency is a myth in Ruby<br />still no conc...
Nick – tomorrow @ 11:45am<br />Concurrency is a myth in Ruby<br />still no concurrency in Ruby 1.9<br />http://bit.ly/ruby...
Blocks entire<br />Ruby VM<br />Not as bad, but<br />avoid it still..<br />Avoid locking interpreter threads at all costs<...
Will fetch_xyz() block the VM?<br />when was the last time you asked yourself this question?<br />
require 'rubygems’<br />require 'sequel'DB = Sequel.connect('mysql://root@localhost/test')while trueDB['select sleep(1)']....
Blocking calls to mysql_real_query<br />mysql_real_query requires an OS thread<br />Blocking on mysql_real_query blocks th...
static VALUE async_query(intargc, VALUE* argv, VALUE obj) {<br />  ...<br />send_query( obj, sql );<br />  ...<br />schedu...
spinning in select<br />mysqlplus.gem + ruby select<br />
Step 1: Fix the drivers<br />Many of our DB drivers don’t respect the <br />underlying Ruby VM. Don’t blame the VM.<br />
Drivers<br />Ruby VM<br />Network<br />WEBRick<br />Mongrel made Rails viable<br />still powers a lot of Rails apps today<...
Rails rediscovers Apache<br />the worker/forker model… <br />
*nix IPC is fast!  Woo!<br />…<br />Full Ruby VM<br />An exclusive Ruby VM for EACH request<br />am I the only one who thi...
“Does not care if your application is thread-safe or not, workers all run within their own isolated address space and only...
Step 2: consider entire stack<br />The driver, the web-server, and the network<br />must all work together.<br />
Node imposes the full-stack requirements<br />Node imposes async drivers<br />Node imposes async frameworks<br />Surprise:...
We can ignore the performance<br />issues at our own peril<br />or, we can just fix the problem<br />
><br />I’ll take Ruby over JS<br />gem install eventmachine<br />
p "Starting"EM.rundop"Running in EM reactor"endp”won’t get here"<br />whiletruedo<br />       timersnetwork_ioother_io<br ...
Non-blocking IO requires non-blocking drivers:<br />AMQP                        http://github.com/tmm1/amqp<br />MySQLPlus...
gem install em-mysqlplus<br />EventMachine.rundo<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br />  query =con...
non-blocking driver<br />require'mysqlplus'<br />defconnect(opts)<br />conn=connect_socket(opts)<br />EM.watch(conn.socket...
Features:<br /><ul><li> Maintains C-based mysql gem API
Deferrables for every query with callback & errback
 Connection query queue - pile 'em up!
 Auto-reconnect on disconnects
 Auto-retry on deadlocks</li></ul>http://github.com/igrigorik/em-mysqlplus<br />em-mysqlplus<br />mysqlplus + reactor loop...
and this callback goes to…<br />
We can do better than node.js<br />all the benefits of evented code without the drawbacks<br />
Ruby 1.9 Fibers are a means of creating code blocks which can be paused and resumed by our application (think lightweight ...
Fibers vs Threads: creation time much lower<br />Fibers vs Threads: memory usage is much lower<br />Ruby 1.9 Fibers<br />a...
defquery(sql)<br />f = Fiber.current<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br />q = conn.query(sql)<br /...
defquery(sql)<br />f = Fiber.current<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br />q = conn.query(sql)<br /...
defquery(sql)<br />f=Fiber.current<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br />q = conn.query(sql)<br />c...
defquery(sql)<br />f=Fiber.current<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br />q = conn.query(sql)<br />c...
Good news, you don’t even have to muck around with Fibers!<br />gem install em-synchrony<br />http://github.com/igrigorik/...
 Multi request interface which accepts any callback enabled client
 Fibered iterator to allow concurrency control & mixing of sync / async
em-http-request: .get, etc are synchronous, while .aget, etc are async
em-mysqlplus: .query is synchronous, while .aquery is async
Upcoming SlideShare
Loading in...5
×

No Callbacks, No Threads - RailsConf 2010

18,670

Published on

Multi-threaded servers compete for the global interpreter lock (GIL) and incur the cost of continuous context switching, potential deadlocks, or plain wasted cycles. Asynchronous servers, on the other hand, create a mess of callbacks and errbacks, complicating the code. But, what if, you could get all the benefits of asynchronous programming, while preserving the synchronous look and feel of the code – no threads, no callbacks?

Published in: Technology
1 Comment
87 Likes
Statistics
Notes
No Downloads
Views
Total Views
18,670
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
400
Comments
1
Likes
87
Embeds 0
No embeds

No notes for slide
  • The blame game.
  • The blame game.
  • coroutines
  • coroutines
  • Transcript of "No Callbacks, No Threads - RailsConf 2010"

    1. 1. No Callbacks, No Threads & Ruby 1.9<br />async & co-operative web-servers<br />Ilya Grigorik<br />@igrigorik<br />
    2. 2. The slides…<br />Twitter<br />My blog<br />
    3. 3. The state of art is not good enough.<br />(we’ve been stuck in the same local minima for several years)<br />
    4. 4. require"active_record”<br />ActiveRecord::Base.establish_connection(<br /> :adapter => "mysql",<br /> :username => "root",<br /> :database => "database",<br /> :pool => 5<br />)<br />threads = []<br />10.times do |n| <br /> threads <<Thread.new {<br />ActiveRecord::Base.connection_pool.with_connectiondo |conn|<br /> res =conn.execute("select sleep(1)")<br />end<br /> }<br />end<br />threads.each { |t| t.join }<br />The “Experiment”<br />vanilla everything…<br />
    5. 5. require"active_record”<br />ActiveRecord::Base.establish_connection(<br /> :adapter => "mysql",<br /> :username => "root",<br /> :database => "database",<br /> :pool => 5<br />)<br />threads = []<br />10.times do |n| <br /> threads <<Thread.new {<br />ActiveRecord::Base.connection_pool.with_connectiondo |conn|<br /> res =conn.execute("select sleep(1)")<br />end<br /> }<br />end<br />threads.each { |t| t.join }<br />5 shared connections<br /># time ruby activerecord-pool.rb<br />#<br /># real 0m10.663s<br /># user 0m0.405s<br /># sys 0m0.201s<br />
    6. 6.
    7. 7. BHP<br />% power <br /> loss<br />WHP<br />
    8. 8.
    9. 9.
    10. 10. > 50% power loss!?<br />
    11. 11. Mongo<br />Couch<br />MySQL<br />PSQL<br />…<br />Drivers<br />Threads<br />Ruby VM<br />GIL<br />Fibers<br />…<br />Network<br />Mongrel<br />Unicorn<br />Passenger<br />…<br />
    12. 12. 2<br />Mongo<br />Couch<br />MySQL<br />PSQL<br />…<br />Drivers<br />Threads<br />Ruby VM<br />1<br />4<br />GIL<br />Fibers<br />…<br />Network<br />Mongrel<br />Unicorn<br />3<br />We’re as fast as the slowest component<br />Passenger<br />…<br />
    13. 13. Global Interpreter Lock is a mutual exclusion lock held by a programming language interpreter thread to avoid sharing code that is not thread-safe with other threads. <br />There is always one GIL for one interpreter process.<br />Concurrency is a myth in Ruby<br />(with a few caveats, of course)<br />http://bit.ly/ruby-gil<br />
    14. 14. N-M thread pool in Ruby 1.9…<br />Better but still the same problem!<br />Concurrency is a myth in Ruby<br />still no concurrency in Ruby 1.9<br />http://bit.ly/ruby-gil<br />
    15. 15. Nick – tomorrow @ 11:45am<br />Concurrency is a myth in Ruby<br />still no concurrency in Ruby 1.9<br />http://bit.ly/ruby-gil<br />
    16. 16. Blocks entire<br />Ruby VM<br />Not as bad, but<br />avoid it still..<br />Avoid locking interpreter threads at all costs<br />let’s say you’re writing an extension…<br />
    17. 17. Will fetch_xyz() block the VM?<br />when was the last time you asked yourself this question?<br />
    18. 18. require 'rubygems’<br />require 'sequel'DB = Sequel.connect('mysql://root@localhost/test')while trueDB['select sleep(1)'].select.firstend<br />Blocking 1s call!<br />ltrace –ttTg -xmysql_real_query –p ./example.rb<br />mysql.gem under the hood<br />22:10:00.218438 mysql_real_query(0x02740000, "select sleep(1)", 15) = 0 <1.001100>22:10:01.241679 mysql_real_query(0x02740000, "select sleep(1)", 15) = 0 <1.000812><br />http://bit.ly/c3Pt3f<br />
    19. 19. Blocking calls to mysql_real_query<br />mysql_real_query requires an OS thread<br />Blocking on mysql_real_query blocks the Ruby VM<br />Aka, “select sleep(1)” blocks the entire Ruby runtime for 1s<br />(ouch)<br />gem install mysqlwhat you didn’t know…<br />
    20. 20. static VALUE async_query(intargc, VALUE* argv, VALUE obj) {<br /> ...<br />send_query( obj, sql );<br /> ...<br />schedule_query( obj, timeout);<br /> ...<br />returnget_result(obj); <br />}<br />staticvoidschedule_query(VALUEobj, VALUE timeout) {<br /> ...<br />structtimevaltv = { tv_sec: timeout, tv_usec: 0 };<br />for(;;){<br />FD_ZERO(&read);<br />FD_SET(m->net.fd, &read);<br /> ret = rb_thread_select(m->net.fd + 1, &read, NULL, NULL, &tv);<br /> ...<br />if (m->status == MYSQL_STATUS_READY)<br />break;<br /> }<br />}<br />send query and block<br />Ruby: select() = C: rb_thread_select()<br />mysqlplus.gem under the hood<br />
    21. 21. spinning in select<br />mysqlplus.gem + ruby select<br />
    22. 22. Step 1: Fix the drivers<br />Many of our DB drivers don’t respect the <br />underlying Ruby VM. Don’t blame the VM.<br />
    23. 23. Drivers<br />Ruby VM<br />Network<br />WEBRick<br />Mongrel made Rails viable<br />still powers a lot of Rails apps today<br />
    24. 24. Rails rediscovers Apache<br />the worker/forker model… <br />
    25. 25. *nix IPC is fast! Woo!<br />…<br />Full Ruby VM<br />An exclusive Ruby VM for EACH request<br />am I the only one who thinks this is terrible? <br />
    26. 26. “Does not care if your application is thread-safe or not, workers all run within their own isolated address space and only serve one client at a time for maximum robustness.”<br />Robustness? That sounds like a bug.<br />An exclusive Ruby VM for EACH request<br />am I the only one who thinks this is terrible? <br />
    27. 27.
    28. 28. Step 2: consider entire stack<br />The driver, the web-server, and the network<br />must all work together.<br />
    29. 29. Node imposes the full-stack requirements<br />Node imposes async drivers<br />Node imposes async frameworks<br />Surprise: Node is “fast”<br />
    30. 30. We can ignore the performance<br />issues at our own peril<br />or, we can just fix the problem<br />
    31. 31. ><br />I’ll take Ruby over JS<br />gem install eventmachine<br />
    32. 32. p "Starting"EM.rundop"Running in EM reactor"endp”won’t get here"<br />whiletruedo<br /> timersnetwork_ioother_io<br />end<br />EventMachine Reactor<br />concurrency without thread<br />EventMachine: The Speed DemonWednesday @ 11:45am – Aman Gupta<br />
    33. 33. Non-blocking IO requires non-blocking drivers:<br />AMQP http://github.com/tmm1/amqp<br />MySQLPlushttp://github.com/igrigorik/em-mysqlplus<br />Memcachedhttp://github.com/astro/remcached<br />DNS http://github.com/astro/em-dns<br />Redishttp://github.com/madsimian/em-redis<br />MongoDBhttp://github.com/tmm1/rmongo<br />HTTPRequesthttp://github.com/igrigorik/em-http-request<br />WebSockethttp://github.com/igrigorik/em-websocket<br />Amazon S3 http://github.com/peritor/happening<br />And many others: <br />http://wiki.github.com/eventmachine/eventmachine/protocol-implementations<br />
    34. 34. gem install em-mysqlplus<br />EventMachine.rundo<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br /> query =conn.query("select sleep(1)")<br />query.callback { |res| pres.all_hashes }<br />query.errback { |res| pres.all_hashes }<br /> puts ”executing…”<br />end<br /># > ruby em-mysql-test.rb<br />#<br /># executing…<br /># [{"sleep(1)"=>"0"}]<br />callback fired 1s after “executing”<br />em-mysqlplus: example<br />asyncMySQL driver<br />
    35. 35. non-blocking driver<br />require'mysqlplus'<br />defconnect(opts)<br />conn=connect_socket(opts)<br />EM.watch(conn.socket, EventMachine::MySQLConnection, conn, opts, self)<br />end<br />defconnect_socket(opts)<br />conn=Mysql.init<br />conn.real_connect(host, user, pass, db, port, socket, ...)<br />conn.reconnect=false<br />conn<br />end<br />EM.watch: reactor will poll & notify<br />em-mysqlplus: under the hood<br />mysqlplus + reactor loop<br />
    36. 36. Features:<br /><ul><li> Maintains C-based mysql gem API
    37. 37. Deferrables for every query with callback & errback
    38. 38. Connection query queue - pile 'em up!
    39. 39. Auto-reconnect on disconnects
    40. 40. Auto-retry on deadlocks</li></ul>http://github.com/igrigorik/em-mysqlplus<br />em-mysqlplus<br />mysqlplus + reactor loop<br />
    41. 41. and this callback goes to…<br />
    42. 42. We can do better than node.js<br />all the benefits of evented code without the drawbacks<br />
    43. 43. Ruby 1.9 Fibers are a means of creating code blocks which can be paused and resumed by our application (think lightweight threads, minus the thread scheduler and less overhead). <br />f=Fiber.new {<br />whiletruedo<br />Fiber.yield"Hi”<br />end<br />}<br />pf.resume# => Hi<br />pf.resume# => Hi<br />pf.resume# => Hi<br />Manual / cooperative scheduling!<br />Ruby 1.9 Fibers<br />and cooperative scheduling<br />http://bit.ly/d2hYw0<br />
    44. 44. Fibers vs Threads: creation time much lower<br />Fibers vs Threads: memory usage is much lower<br />Ruby 1.9 Fibers<br />and cooperative scheduling<br />http://bit.ly/aesXy5<br />
    45. 45. defquery(sql)<br />f = Fiber.current<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br />q = conn.query(sql)<br />c.callback { f.resume(conn) }<br />c.errback { f.resume(conn) }<br />return Fiber.yield<br />end<br />EventMachine.rundo<br />Fiber.new {<br /> res =query('select sleep(1)')<br /> puts "Results: #{res.fetch_row.first}"<br />}.resume<br />end<br />Exception, async!<br />Untangling Evented Code with Fibers<br />http://bit.ly/d2hYw0<br />
    46. 46. defquery(sql)<br />f = Fiber.current<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br />q = conn.query(sql)<br />c.callback { f.resume(conn) }<br />c.errback { f.resume(conn) }<br /> return Fiber.yield<br />end<br />EventMachine.rundo<br />Fiber.new{<br /> res =query('select sleep(1)')<br /> puts "Results: #{res.fetch_row.first}"<br /> }.resume<br />end<br />1. Wrap into a continuation<br />Untangling Evented Code with Fibers<br />http://bit.ly/d2hYw0<br />
    47. 47. defquery(sql)<br />f=Fiber.current<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br />q = conn.query(sql)<br />c.callback { f.resume(conn) }<br />c.errback { f.resume(conn) }<br />returnFiber.yield<br />end<br />EventMachine.rundo<br />Fiber.new{<br /> res =query('select sleep(1)')<br /> puts "Results: #{res.fetch_row.first}"<br /> }.resume<br />end<br />2. Pause the continuation<br />Untangling Evented Code with Fibers<br />http://bit.ly/d2hYw0<br />
    48. 48. defquery(sql)<br />f=Fiber.current<br />conn=EventMachine::MySQL.new(:host => 'localhost')<br />q = conn.query(sql)<br />c.callback { f.resume(conn) }<br />c.errback { f.resume(conn) }<br />returnFiber.yield<br />end<br />EventMachine.rundo<br />Fiber.new{<br /> res =query('select sleep(1)')<br /> puts "Results: #{res.fetch_row.first}"<br /> }.resume<br />end<br />3. Resume the continuation<br />Untangling Evented Code with Fibers<br />http://bit.ly/d2hYw0<br />
    49. 49. Good news, you don’t even have to muck around with Fibers!<br />gem install em-synchrony<br />http://github.com/igrigorik/em-synchrony<br /><ul><li> Fiber aware connection pool with sync/async query support
    50. 50. Multi request interface which accepts any callback enabled client
    51. 51. Fibered iterator to allow concurrency control & mixing of sync / async
    52. 52. em-http-request: .get, etc are synchronous, while .aget, etc are async
    53. 53. em-mysqlplus: .query is synchronous, while .aquery is async
    54. 54. remcached: .get, etc, and .multi_* methods are synchronous</li></ul>em-synchrony: simple evented programming<br />best of both worlds…<br />
    55. 55. require"em-synchrony/em-mysqlplus"<br />EventMachine.synchronydo<br /> db =EventMachine::MySQL.new(host:"localhost")<br /> res =db.query("select sleep(1)")<br /> puts res<br />EventMachine.stop<br />end<br />Async under the hood<br />Untangling Evented Code with Fibers<br />http://bit.ly/d2hYw0<br />
    56. 56. require "em-synchrony/em-mysqlplus"<br />EventMachine.synchrony do<br /> db = EventMachine::MySQL.new(host: "localhost")<br /> res =db.query("select sleep(1)")<br /> puts res<br />EventMachine.stop<br />end<br />Untangling Evented Code with Fibers<br />http://bit.ly/d2hYw0<br />
    57. 57. EM-HTTP, EM-MySQL, EM-Jack, etc.<br />Drivers<br />Async-rack<br />Ruby VM<br />Fibers<br />Network<br />Goliath<br />Thin<br />One VM, full concurrency, network-bound<br />Ruby 1.9, Fibers, Thin: in production!<br />
    58. 58. git clone git://github.com/igrigorik/em-mysqlplus.git<br />git checkout activerecord<br />rake install<br />database.yml<br />development:<br />adapter:em_mysqlplus<br />database:widgets<br />pool: 5<br />timeout: 5000<br />environment.rb<br />require 'em-activerecord’<br />require 'rack/fiber_pool'<br /># Run each request in a Fiber<br />config.middleware.useRack::FiberPool<br />config.threadsafe!<br />Async Rails<br />with EventMachine & MySQL<br />
    59. 59. classWidgetsController< ApplicationController<br />defindex<br />Widget.find_by_sql("select sleep(1)")<br />render:text => "Oh hai”<br />end<br />end<br />ab –c 5 –n 10 http://127.0.0.1:3000/widgets<br />Server Software: thin<br />Server Hostname: 127.0.0.1<br />Server Port: 3000<br />Document Path: /widgets/<br />Document Length: 6 bytes<br />Concurrency Level: 5<br />Time taken for tests: 2.210 seconds<br />Complete requests: 10<br />Failed requests: 0<br />Requests per second: 4.53 [#/sec] (mean)<br />woot! Fiber DB pool at work.<br />Async Rails<br />with EventMachine & MySQL<br />
    60. 60. git clone git://…./igrigorik/mysqlplus<br />git checkout activerecord<br />rake install<br />Not only is it doable… it already works.<br />
    61. 61. Ruby 1.9 + Rails 3 + new stack<br />=<br />Order of magnitude better performance<br />(aka, enough of a reason to actually switch)<br />
    62. 62. The state of art is not good enough, in fact, it’s terrible!<br />Let’s fix it.<br />Fibers & Cooperative Scheduling in Ruby:<br />http://www.igvita.com/2009/05/13/fibers-cooperative-scheduling-in-ruby/<br />Untangling Evented Code with Ruby Fibers:<br />http://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers/<br />EM-Synchrony:<br />http://github.com/igrigorik/em-synchrony<br />What do you think?<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×