The document discusses using fibers and evented programming with Ruby to achieve asynchronous and concurrent behavior without threads. It introduces EventMachine and EM-Synchrony for implementing asynchronous drivers and applications. Code examples show wrapping database queries in fibers to make them non-blocking. This allows running queries concurrently using a threadpool and achieving higher throughput than blocking implementations. The approach is applied to build an asynchronous version of Rails using EventMachine, fibers and the Thin web server.

1. No Callbacks, No Threads & Ruby 1.9async & co-operative web-serversIlya Grigorik@igrigorik

2. Reactor PatternReview: State of the artAsync PrimitivesCallbacks & FibersAsync Rails?

3. require"active_record”ActiveRecord::Base.establish_connection( :adapter => "mysql", :username => "root", :database => "database", :pool => 5)threads = []10.times do |n| threads <<Thread.new {ActiveRecord::Base.connection_pool.with_connectiondo |conn| res =conn.execute("select sleep(1)")end }endthreads.each { |t| t.join }The “Experiment”vanilla everything…

4. require"active_record”ActiveRecord::Base.establish_connection( :adapter => "mysql", :username => "root", :database => "database", :pool => 5)threads = []10.times do |n| threads <<Thread.new {ActiveRecord::Base.connection_pool.with_connectiondo |conn| res =conn.execute("select sleep(1)")end }endthreads.each { |t| t.join }5 shared connections# time ruby activerecord-pool.rb## real 0m10.663s# user 0m0.405s# sys 0m0.201s

6. BHP% power lossWHP

10. MongoCouchMySQLPSQL…DriversThreadsRuby VMGILFibers…NetworkMongrelUnicornPassenger…

11. Global Interpreter Lock is a mutual exclusion lock held by a programming language interpreter thread to avoid sharing code that is not thread-safe with other threads. There is always one GIL for one interpreter process.Concurrency is a myth in Ruby(with a few caveats, of course)http://bit.ly/ruby-gil

12. N-M thread pool in Ruby 1.9…Better but still the same problem!Concurrency is a myth in Rubystill no concurrency in Ruby 1.9http://bit.ly/ruby-gil

13. Concurrency is a myth in Rubystill no parallelism in Ruby 1.9http://bit.ly/ruby-gil

14. Blocks entireRuby VMNot as bad, butavoid it still..Avoid locking interpreter threads at all costslet’s say you’re writing an extension…

15. DriversRuby VMNetworkWEBRickMongrel made Rails viablestill powers a lot of Rails apps today

16. Rails rediscovers Apachethe worker/forker model…

17. *nix IPC is fast! …Full Ruby VMAn exclusive Ruby VM for EACH requestam I the only one who thinks this is terrible?

18. “Does not care if your application is thread-safe or not, workers all run within their own isolated address space and only serve one client at a time for maximum robustness.”Robustness? That sounds like a bug.An exclusive Ruby VM for EACH requestam I the only one who thinks this is terrible?

20. …?

21. Node imposes the full-stack requirementsNode imposes async driversNode imposes async frameworks*Surprise*: Node is “fast”

22. =I’ll take Ruby over JSgem install eventmachine

23. p "Starting"EM.rundop"Running in EM reactor"endp”won’t get here"whiletruedo timersnetwork_ioother_ioendEventMachine Reactorconcurrency without thread

24. gem install em-mysqlplusEventMachine.rundoconn=EventMachine::MySQL.new(:host => 'localhost') query =conn.query("select sleep(1)")query.callback { |res| pres.all_hashes }query.errback { |res| pres.all_hashes } puts ”executing…”endcallback fired 1s after “executing”# > ruby em-mysql-test.rb## executing…# [{"sleep(1)"=>"0"}]em-mysqlplus: exampleasyncMySQL driver

25. Non-blocking IO requires non-blocking drivers:AMQP http://github.com/tmm1/amqpMySQLPlushttp://github.com/igrigorik/em-mysqlplusMemcachedhttp://github.com/astro/remcachedDNS http://github.com/astro/em-dnsRedishttp://github.com/madsimian/em-redisMongoDBhttp://github.com/tmm1/rmongoHTTPRequesthttp://github.com/igrigorik/em-http-requestWebSockethttp://github.com/igrigorik/em-websocketAmazon S3 http://github.com/peritor/happeningAnd many others: http://wiki.github.com/eventmachine/eventmachine/protocol-implementations

26. httpA=EventMachine::HttpRequest.new(URL).gethttpA.errback {# retry logic?}httpA.callback {Memcached.set(:key => URL, :value => httpA.response) { |response|caseresponse[:status]whenMemcached::Errors::NO_ERROR# That's goodwhenMemcached::Errors::DISCONNECTED# Maybe stop filling the cache for now?else# What could've gone wrong?end }}http =get(url)memc=save(url, http.response)ifmemc.error?# That's goodelse# Err?endaccidental complexityasync is harder to manage

27. and this callback goes to…

28. We can do better than node.jsall the benefits of evented code without the drawbacks

29. Ruby 1.9 Fibers are a means of creating code blocks which can be paused and resumed by our application (think lightweight threads, minus the thread scheduler and less overhead). f=Fiber.new {whiletruedoFiber.yield"Hi”end}pf.resume# => Hipf.resume# => Hipf.resume# => HiManual / cooperative scheduling!Ruby 1.9 Fibersand cooperative schedulinghttp://bit.ly/d2hYw0

30. Fibers vs Threads: creation time much lowerFibers vs Threads: memory usage is much lowerRuby 1.9 Fibersand cooperative schedulinghttp://bit.ly/aesXy5

31. defquery(sql)f = Fiber.currentconn=EventMachine::MySQL.new(:host => 'localhost')q = conn.query(sql)c.callback { f.resume(conn) }c.errback { f.resume(conn) }return Fiber.yieldendEventMachine.rundoFiber.new { res =query('select sleep(1)') puts "Results: #{res.fetch_row.first}"}.resumeendException, async!Untangling Evented Code with Fibershttp://bit.ly/d2hYw0

32. defquery(sql)f = Fiber.currentconn=EventMachine::MySQL.new(:host => 'localhost')q = conn.query(sql)c.callback { f.resume(conn) }c.errback { f.resume(conn) } return Fiber.yieldendEventMachine.rundoFiber.new{ res =query('select sleep(1)') puts "Results: #{res.fetch_row.first}" }.resumeend1. Wrap into a continuationUntangling Evented Code with Fibershttp://bit.ly/d2hYw0

33. defquery(sql)f=Fiber.currentconn=EventMachine::MySQL.new(:host => 'localhost')q = conn.query(sql)c.callback { f.resume(conn) }c.errback { f.resume(conn) }returnFiber.yieldendEventMachine.rundoFiber.new{ res =query('select sleep(1)') puts "Results: #{res.fetch_row.first}" }.resumeend2. Pause the continuationUntangling Evented Code with Fibershttp://bit.ly/d2hYw0

34. defquery(sql)f=Fiber.currentconn=EventMachine::MySQL.new(:host => 'localhost')q = conn.query(sql)c.callback { f.resume(conn) }c.errback { f.resume(conn) }returnFiber.yieldendEventMachine.rundoFiber.new{ res =query('select sleep(1)') puts "Results: #{res.fetch_row.first}" }.resumeend3. Resume the continuationUntangling Evented Code with Fibershttp://bit.ly/d2hYw0

35. Good news, you don’t even have to muck around with Fibers!gem install em-synchronyhttp://github.com/igrigorik/em-synchrony Fiber aware connection pool with sync/async query support

36. Multi request interface which accepts any callback enabled client

37. Fibered iterator to allow concurrency control & mixing of sync / async

38. em-http-request: .get, etc are synchronous, while .aget, etc are async

39. em-mysqlplus: .query is synchronous, while .aquery is async

40. remcached: .get, etc, and .multi_* methods are synchronousem-synchrony: simple evented programmingbest of both worlds…

41. EventMachine.synchronydo multi =EventMachine::Synchrony::Multi.newmulti.add:a, EventMachine::HttpRequest.new(url1).agetmulti.add:b, EventMachine::HttpRequest.new(url2).apost res =multi.performp“No callbacks, and parallel HTTP requests!"p resEventMachine.stopendParallel IO with Fibersmulti interface example

42. Persistent connectionsEventMachine.synchronydo db =EventMachine::Synchrony::ConnectionPool.new(size:2) doEventMachine::MySQL.new(host:"localhost")end multi =EventMachine::Synchrony::Multi.newmulti.add:a, db.aquery("select sleep(1)")multi.add:b, db.aquery("select sleep(1)") res =multi.performEventMachine.stopendFiber Poolsaccessing shared services

43. EM.synchronydo concurrency =2urls= ['http://url.1.com', 'http://url2.com'] results =EM::Synchrony::Iterator.new(urls, concurrency).mapdo |url, iter| http =EventMachine::HttpRequest.new(url).agethttp.callback { iter.return(http) }endp results # all completed requestsEventMachine.stopendExplicit iterators.each, .map, .inject, …asynchronous iterators

44. EM.synchronydo result =EM::Synchrony.syncEventMachine::HttpRequest.new(URL).getpresultEM.stopendInline synchronizationInline synchronizationad-hoc synchronization

45. require"em-synchrony/em-mysqlplus"EventMachine.synchronydo db =EventMachine::MySQL.new(host:"localhost") res =db.query("select sleep(1)") puts resEventMachine.stopendAsync under the hoodUntangling Evented Code with Fibershttp://bit.ly/d2hYw0

46. require "em-synchrony/em-mysqlplus"EventMachine.synchrony do db = EventMachine::MySQL.new(host: "localhost") res =db.query("select sleep(1)") puts resEventMachine.stopendUntangling Evented Code with Fibershttp://bit.ly/d2hYw0

47. Async web-stack in Ruby?

48. em-synchrony, connection pool, iterators…Driversasync-rackRuby VMFibersNetworkThinhttp://code.macournoyer.com/thin/One VM, full concurrency, network-boundRuby 1.9, Fibers, Thin: in production!

49. git clonegit://github.com/igrigorik/async-rails.gitdatabase.ymldevelopment:adapter:em_mysqlplusdatabase:widgetspool: 5timeout: 5000environment.rbrequire 'em-activerecord’require 'rack/fiber_pool'# Run each request in a Fiberconfig.middleware.useRack::FiberPoolconfig.threadsafe!AsyncRails 3with EventMachine & MySQLthin –D start

50. classWidgetsController< ApplicationControllerdefindexWidget.find_by_sql("select sleep(1)")render:text => "Oh hai”endendab –c 5 –n 10 http://127.0.0.1:3000/widgetsServer Software: thinServer Hostname: 127.0.0.1Server Port: 3000Document Path: /widgets/Document Length: 6 bytesConcurrency Level: 5Time taken for tests: 2.210 secondsComplete requests: 10Failed requests: 0Requests per second: 4.53 [#/sec] (mean)Async DB! Async Railswith EventMachine & MySQL

51. http://gist.github.com/445603 concurrency time 75 73.426 60 66.411 50 65.502 40 78.105 30 106.624Async Railswith EventMachine & MySQL

52. Async Rails 3 demo:http://github.com/igrigorik/async-rails/Fibers & Cooperative Scheduling in Ruby:http://www.igvita.com/2009/05/13/fibers-cooperative-scheduling-in-ruby/Untangling Evented Code with Ruby Fibers:http://www.igvita.com/2010/03/22/untangling-evented-code-with-ruby-fibers/EM-Synchrony:http://github.com/igrigorik/em-synchronyWhat do you think?