Concurrent programming with      http://github.com/celluloid           Tony Arcieri      MountainWest RubyConf        Marc...
About Me
About Melibev binding for Ruby (1 year before Node)
About Me     Actors +“Fibered” I/O(2 years before em-synchrony)
About MeRuby-Flavored Erlang(2 years before Elixir)
http://elixir-lang.org/
About MeDidn’t talk about Revactor or Reia at             MWRC 2008
What next?
If I can’t drag Erlang  halfway to Ruby...
Perhaps I can get Ruby halfway to    Erlang...
Rubyists don’t like threads      I want to change that
x86 CPU Trends (Not Entirely to Scale)
Multicoreis the future
Threadsare important
No GIL!RubiniusThread-level parallelism  Threads = Multicore
Parallel Blocking I/O            YARV  But only one Ruby thread at a time :(          Threads != Multicore
When everyone has 100 core CPUs...
will we run 100virtual machines?
or one?
Sidekiq    What if 1 Sidekiq process coulddo the work of 20 Resque processes?   http://mperham.github.com/sidekiq/
A little about Revactor...
Revactor predates:• Neverblock• Dramatis• Rack::FiberPool• em-synchrony
Inspired by:• Erlang• Omnibus Concurrency (MenTaLguY)• Kamaelia (Python)• Eventlet (Python)
Single-Threaded
Bad API
listener = Actor::TCP.listen(HOST, PORT, :filter => :line)puts "Listening on #{HOST}:#{PORT}"# The main loop handles incom...
WAT
listener = Actor::TCP.listen(HOST, PORT, :filter => :line)puts "Listening on #{HOST}:#{PORT}"# The main loop handles incom...
Procedural!
Ugly!
WTF is T?filter.when(T[:tcp, sock]) do |_, _, message|  server << T[:say, Actor.current, message]  sock.active = :onceend
Less Erlang
More Objects
Can WeDo Better?
YES
What is Celluloid?
Celluloid isa general purpose concurrency framework                for Ruby
A Contrived Example
require threadclass ConcurrentNestedHash  def initialize    @outer = {}    @mutex = Mutex.new  end  def [](*keys)    @mute...
>> h = ConcurrentNestedHash.new => #<ConcurrentNestedHash:0x007f99ed735f08 @outer={}, @mutex=#<Mutex:0x007f99ed735e90>>>> ...
-require thread+require celluloid class ConcurrentNestedHash+ include Celluloid      def initialize        @outer = {}-   ...
require celluloidclass ConcurrentNestedHash  include Celluloid  def initialize    @outer = {}  end  def [](*keys)    keys....
How?
MAGIC
Automatic locking?
Locks are hard• Dining Philosophers Problem• Sleeping Barber Problem• Cigarette Smokers Problem
OOP + Concurrency
“I thought of objects being like biologicalcells and/or individual computers on anetwork, only able to communicate withmes...
OOPTools ClassesInheritanceMessages
Concurrency   Tools   Threads    Locks   Queues
OOP           ConcurrencyTools            Tools              + Classes          ThreadsInheritance        LocksMessages   ...
=
Concurrent Objects
CommunicatingSequential Processes • Do One Thing At a Time • Communicate With Messages • Encapsulate Local State
Concurrent Objects   do more...
Generalized Deadlock-freeSynchronization
Pythons did it!
1997
1999
Web vs Objects   (Not to Scale)
How doesCelluloid work?
>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>
>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>
 # Class methods added to classes which include Celluloidmodule ClassMethods  # Create a new actor  def new(*args, &block)...
Synchronous   Calls
EXTREME Late   Binding
Synchronous Calls>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> h...
Asynchronous    Calls
Asynchronous Calls
Asynchronous Calls>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> ...
Asynchronous Calls>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> ...
Asynchronous Calls>> h.inspect! => nil
Asynchronous Calls  Kind of like “next tick”
Asynchronous CallsHow do I get the value returned?
Futures
Futures
Futures>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> future = h....
Futures>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> future = h....
Basic Ingredients• Regular method calls• Async calls (“next tick”)• Futures
SecretSauce
How do weprevent deadlocks?
Why do deadlocks   happen?
Waiting forsomething that never    happens...
...instead of what’s      important
How can we wait oneverything at once?
FIBERS!
WAT?
SUBLIMINAL  MESSAGE:DONALD KNUTH  IS YODA
No really...
CommunicatingSequential Processes    Do one thing at a time
FibersCheap suspendable/resumable     execution context
CommunicatingSequential Processes          +    Do one thing at a time         Fibers Cheap suspendable/resumable      exe...
Internal Concurrency      for Actors
Don’t Block, Suspend  to the Scheduler
Example!
require celluloidclass Person  include Celluloid  def name    self.class.to_s  end  def greet(interested_party)    "Hello ...
OutputHello Joe, Im MikeHello Mike, Im Joe
But it is acool story!
Circular Call Graph
CommunicatingSequential Processes    Do one thing at a time
DEADLOCK!
require celluloidclass Person  include Celluloid  def name    self.class.to_s  end  def greet(interested_party)    "Hello ...
Multitasking Fibers
Waiting taskssuspend themselves   so ready tasks can run
Every method call creates a Fiber
Slow?
Call Benchmark   Core i7 2.0GHz (OS X 10.7.3)              16815/s (60µs) - JRuby 1.6.7              20899/s (50µs) - JRub...
What if an actor  crashes?
Fault Tolerance• Supervisors & Supervision Trees• “Fail early”, restart in a clean state• Do what Erlang does• No seriousl...
Evented I/O for Celluloidhttp://github.com/celluloid/celluloid-io
Now that we have threads licked...    how about I/O?
USE BLOCKING I/O
Blocking IO is OK!* No central event loop to block
*But be carefulLocks in external services            = Deadlocks in Celluloid
But how will I serve my roflmillions of      users?
Evented IO• Large numbers of connections (>1000)• Mostly idle connections• Mostly IO-bound problems• Websockets are an ide...
Actors areevent loops
Normal Actors
Celluloid::IO Actors
nio4r-powered   reactor
nio4r    http://github.com/tarcieri/nio4r/• Quasi-inspired by Java NIO• Smallest API possible• libev C extension for CRuby...
Celluloid::IO::TCPSocket • Uses fibered I/O • “Duck-type” of ::TCPSocket • Evented inside Celluloid::IO actors • Blocking I...
Evented IO     AND Threaded IOYou don’t have to choose!
Transparent handlesyou can pass around  Kind of like file descriptors!
Other replacement       classes• Celluloid::IO::TCPServer• Celluloid::IO::UDPSocket• No Celluloid::IO::UnixSocket yet,  so...
Echo Server Example
class EchoServer  include Celluloid::IO  def initialize(host, port)    puts "*** Starting echo server on #{host}:#{port}" ...
Running out of gas after slide #150
^^^ LET’S USE THIS
Dependency Injection
MyClient.new(‘myhost.domain’, 1234, :socket => Celluloid::IO::TCPSocket)
Easy Peasy!
Celluloid::IO-powered web server    http://github.com/celluloid/reel
Hello World        Benchmark# httperf --num-conns=50 --num-calls=1000Ruby Version        Throughput    Latency------------...
Hello World        ComparisonWeb Server         Throughput     Latency----------         ----------     -------Goliath (0....
0MQ TOO!
Celluloid::ZMQ• Built on ffi-rzmq• But exposes a higher level API• Celluloid::IO for ZeroMQ
Distributed Celluloid over 0MQhttp://github.com/celluloid/reel
Mostly ready to use!
Probably needs its   own talk :(
Celluloid-powered Web Framework http://github.com/celluloid/lattice
Vaporware!
Goals• Reuse parts of Rails• Multithreaded development mode• Easy scatter/gather for SOA
That’s all, folks!
Bye!
Links• http://github.com/celluloid• http://twitter.com/bascule• http://unlimitednovelty.com/
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Concurrent programming with Celluloid (MWRC 2012)
Upcoming SlideShare
Loading in …5
×

Concurrent programming with Celluloid (MWRC 2012)

6,237 views

Published on

Threads versus events: which should you choose? How about both? In this talk you'll learn about the Celluloid concurrency framework, which combines OOP and the Actor Model to give you concurrent Ruby objects. You'll also learn about how Celluloid lets you combine blocking I/O and asynchronous evented I/O, offering you all the benefits of EventMachine without the restrictions of a single event loop. The talk will also provide a brief introduction to DCell, a distributed extension to Celluloid.

Published in: Technology
0 Comments
21 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,237
On SlideShare
0
From Embeds
0
Number of Embeds
76
Actions
Shares
0
Downloads
71
Comments
0
Likes
21
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • Mention Omnibus, Rubinius, MenTaLguY\n
  • My most popular project is probably Reia, where I attempted to build an immutable Ruby-like language on top of the Erlang VM, then quit after Jose Valim built a better language called Elixir\n
  • \n
  • \n
  • \n
  • I got into Erlang in 2007 has been a huge influence in the way I think\nIt blew my mind when I first discovered it\nMany of my subsequent ideas I owe to Erlang\n
  • The Erlang guys were ahead of their time (that&amp;#x2019;s Joe Armstrong, Erlang&amp;#x2019;s creator there in the photo)\nI wrote a nasty blog post about the warts in Erlang as a language, but their ideas are still phenomenal and continue to influence me to this day\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • I&amp;#x2019;ll give you a second to look at this to form an opinion, but my opinion is...\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • YO YO GET READY\n
  • This is my favorite quote from Alan Kay. What particularly interests me is the idea that objects are like biological cells or networked computers, self-contained, encapsulated, and running in parallel. Object oriented programming, in Alan Kay&amp;#x2019;s conception, naturally predisposes itself to concurrency.\n
  • If we take the object oriented tools (LIKE, EXPOUND)...\n
  • ...and the tools for concurrency (LIKE, EXPOUND)...\n
  • ...and combine them...\n
  • we get\n
  • concurrent objects. (*pause*) I&amp;#x2019;ll dig into this slide a bit later. Celluloid builds on the ideas of the actor model and communicating...\n
  • ...sequential processes\n
  • Concurrent objects represent a higher level of abstraction for solving concurrency problems, and one I think is unique\n
  • Concurrent objects represent a higher level of abstraction for solving concurrency problems, and one I think is unique\n
  • it&amp;#x2019;s one not\n
  • Erlang\n
  • or\n
  • Scala can give you\n
  • At first I thought this was something I&amp;#x2019;d invented independently, but after doing some research on the subject, I discovered...\n
  • \n
  • I found a paper detailing a nearly identical system to Celluloid in Python\nIt felt like a validation I&amp;#x2019;m not off in the wilderness somewhere, someone has explored these ideas before\nAnd what&amp;#x2019;s more...\n
  • They did it in 1997\n
  • I&amp;#x2019;m not sure how well the rest of you remember 1997, but to refresh your memory, computers kind of sucked back then.\nIt wasn&amp;#x2019;t the greatest time to be working on projects targeting concurrency or massively multicore CPUs.\n
  • Flash forward to 1999, yours truly was working on revolutionizing the state of the art of\n
  • X11 CD players. But just for the record, I was into rounded corners and gradients before they were popular\n
  • In the late &amp;#x2018;80s and early &amp;#x2018;90s academic research into concurrent and distributed objects was booming, by the late &amp;#x2018;90s research in the field was virtually nonexistent and the number of papers published on the topic had practically ground to a halt. I believe this was due to the overpowering effect of the web and HTTP as the universal abstraction.\n
  • While the web is an amazing thing, I think it may have distracted us from some good ideas, and that concurrent objects probably deserve another look. So let&amp;#x2019;s take a look at\n
  • ...how Celluloid works. If we go back to the Celluloid version of the ConcurrentNestedHash example\n
  • and make a concurrent NestedHash object, this is what we see\nPay particular attention to...\n
  • the Celluloid::Actor part there\n
  • Celluloid hijacks the new method, encapsulating newly created objects inside of actors, and hands you a proxy object to talk to them. This all happens before it even calls initialize\n
  • Let&amp;#x2019;s go back to that concurrent object diagram I showed you earlier...\n
  • \n
  • \n
  • Another kind of call pattern Celluloid provides is the \n
  • asynchronous call pattern. When you don&amp;#x2019;t need a result, this is the pattern to use, because it provides a nice, clean, fast path to schedule work in another actor.\n
  • To send another object an asynchronous call, \n
  • pay particular attention to bang/nil\nasync calls end in a bang and always return nil because they&amp;#x2019;re async\n
  • there&amp;#x2019;s an async bang method counterpart for every method, so we can pointlessly call inspect bang\nbang means &amp;#x201C;dangerous&amp;#x201D;\nasync methods are dangerous because you have no guarantees they&amp;#x2019;ll complete, but that&amp;#x2019;s OK\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • So now that we know the basic ingredients, what&amp;#x2019;s the secret sauce?\n
  • The part I&amp;#x2019;m sure you&amp;#x2019;re dying to know is how Celluloid prevents\n
  • DEADLOCKS. This problem bothered me for many years. There&amp;#x2019;s many pretenders to the throne here, like\n
  • ERLANG\n
  • Erlang emphasizes messaging as the universal abstraction. This is a beautiful concept, but there&amp;#x2019;s a problem. Erlang&amp;#x2019;s gen_server manages to get around deadlocks... by totally punting\n
  • \n
  • Erlang\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • If communicating sequential processes can only do one thing at a time, then we&amp;#x2019;d expect a deadlock...\n
  • ...here, but we don&amp;#x2019;t get one. Let&amp;#x2019;s look at the code again.\n
  • Take another look at the code paths here and then I&amp;#x2019;ll show you the solution... all right, you ready?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • ERLANG\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Concurrent programming with Celluloid (MWRC 2012)

    1. 1. Concurrent programming with http://github.com/celluloid Tony Arcieri MountainWest RubyConf March 15th, 2012
    2. 2. About Me
    3. 3. About Melibev binding for Ruby (1 year before Node)
    4. 4. About Me Actors +“Fibered” I/O(2 years before em-synchrony)
    5. 5. About MeRuby-Flavored Erlang(2 years before Elixir)
    6. 6. http://elixir-lang.org/
    7. 7. About MeDidn’t talk about Revactor or Reia at MWRC 2008
    8. 8. What next?
    9. 9. If I can’t drag Erlang halfway to Ruby...
    10. 10. Perhaps I can get Ruby halfway to Erlang...
    11. 11. Rubyists don’t like threads I want to change that
    12. 12. x86 CPU Trends (Not Entirely to Scale)
    13. 13. Multicoreis the future
    14. 14. Threadsare important
    15. 15. No GIL!RubiniusThread-level parallelism Threads = Multicore
    16. 16. Parallel Blocking I/O YARV But only one Ruby thread at a time :( Threads != Multicore
    17. 17. When everyone has 100 core CPUs...
    18. 18. will we run 100virtual machines?
    19. 19. or one?
    20. 20. Sidekiq What if 1 Sidekiq process coulddo the work of 20 Resque processes? http://mperham.github.com/sidekiq/
    21. 21. A little about Revactor...
    22. 22. Revactor predates:• Neverblock• Dramatis• Rack::FiberPool• em-synchrony
    23. 23. Inspired by:• Erlang• Omnibus Concurrency (MenTaLguY)• Kamaelia (Python)• Eventlet (Python)
    24. 24. Single-Threaded
    25. 25. Bad API
    26. 26. listener = Actor::TCP.listen(HOST, PORT, :filter => :line)puts "Listening on #{HOST}:#{PORT}"# The main loop handles incoming connectionsloop do  # Spawn a new actor for each incoming connection  Actor.spawn(listener.accept) do |sock|    puts "#{sock.remote_addr}:#{sock.remote_port} connected"    # Connection handshaking    begin      sock.write "Please enter a nickname:"      nickname = sock.read      server << T[:register, Actor.current, nickname]            # Flip the socket into asynchronous "active" mode      # This means the Actor can receive messages from      # the socket alongside other events.      sock.controller = Actor.current      sock.active = :once          # Main message loop      loop do        Actor.receive do |filter|          filter.when(T[:tcp, sock]) do |_, _, message|            server << T[:say, Actor.current, message]            sock.active = :once          end
    27. 27. WAT
    28. 28. listener = Actor::TCP.listen(HOST, PORT, :filter => :line)puts "Listening on #{HOST}:#{PORT}"# The main loop handles incoming connectionsloop do  # Spawn a new actor for each incoming connection  Actor.spawn(listener.accept) do |sock|    puts "#{sock.remote_addr}:#{sock.remote_port} connected"    # Connection handshaking    begin      sock.write "Please enter a nickname:"      nickname = sock.read      server << T[:register, Actor.current, nickname]            # Flip the socket into asynchronous "active" mode      # This means the Actor can receive messages from      # the socket alongside other events.      sock.controller = Actor.current      sock.active = :once          # Main message loop      loop do        Actor.receive do |filter|          filter.when(T[:tcp, sock]) do |_, _, message|            server << T[:say, Actor.current, message]            sock.active = :once          end
    29. 29. Procedural!
    30. 30. Ugly!
    31. 31. WTF is T?filter.when(T[:tcp, sock]) do |_, _, message|  server << T[:say, Actor.current, message]  sock.active = :onceend
    32. 32. Less Erlang
    33. 33. More Objects
    34. 34. Can WeDo Better?
    35. 35. YES
    36. 36. What is Celluloid?
    37. 37. Celluloid isa general purpose concurrency framework for Ruby
    38. 38. A Contrived Example
    39. 39. require threadclass ConcurrentNestedHash  def initialize    @outer = {}    @mutex = Mutex.new  end  def [](*keys)    @mutex.synchronize { keys.inject(@outer) { |h,k| h[k] } }  end  def []=(*args)    @mutex.synchronize do      value = args.pop      raise ArgumentError, "wrong number of arguments (1 for 2)" if args.empty?      key = args.pop      hash = args.inject(@outer) { |h,k| h[k] ||= {} }      hash[key] = value    end  end  def inspect; @mutex.synchronize { super }; endend
    40. 40. >> h = ConcurrentNestedHash.new => #<ConcurrentNestedHash:0x007f99ed735f08 @outer={}, @mutex=#<Mutex:0x007f99ed735e90>>>> h[:foo, :bar, :baz] = 42 => 42>> h => #<ConcurrentNestedHash:0x007f99ed735f08 @outer={:foo=>{:bar=>{:baz=>42}}},@mutex=#<Mutex:0x007f99ed735e90>>>> h[:foo, :bar, :baz] => 42
    41. 41. -require thread+require celluloid class ConcurrentNestedHash+ include Celluloid def initialize @outer = {}- @mutex = Mutex.new end def [](*keys)- @mutex.synchronize { keys.inject(@outer) { |h,k| h[k] } }+ keys.inject(@outer) { |h,k| h[k] } end def []=(*args)- @mutex.synchronize do value = args.pop raise ArgumentError, "wrong number of arguments (1 for 2)" if args.empty? key = args.pop hash = args.inject(@outer) { |h,k| h[k] ||= {} } hash[key] = value- end end-- def inspect; @mutex.synchronize { super }; end end
    42. 42. require celluloidclass ConcurrentNestedHash  include Celluloid  def initialize    @outer = {}  end  def [](*keys)    keys.inject(@outer) { |h,k| h[k] }  end  def []=(*args)    value = args.pop    raise ArgumentError, "wrong number of arguments (1 for 2)" if args.empty?    key = args.pop    hash = args.inject(@outer) { |h,k| h[k] ||= {} }    hash[key] = value  endend
    43. 43. How?
    44. 44. MAGIC
    45. 45. Automatic locking?
    46. 46. Locks are hard• Dining Philosophers Problem• Sleeping Barber Problem• Cigarette Smokers Problem
    47. 47. OOP + Concurrency
    48. 48. “I thought of objects being like biologicalcells and/or individual computers on anetwork, only able to communicate withmessages”- Alan Kay, creator of Smalltalk, on the meaning of "objectoriented programming"
    49. 49. OOPTools ClassesInheritanceMessages
    50. 50. Concurrency Tools Threads Locks Queues
    51. 51. OOP ConcurrencyTools Tools + Classes ThreadsInheritance LocksMessages Queues
    52. 52. =
    53. 53. Concurrent Objects
    54. 54. CommunicatingSequential Processes • Do One Thing At a Time • Communicate With Messages • Encapsulate Local State
    55. 55. Concurrent Objects do more...
    56. 56. Generalized Deadlock-freeSynchronization
    57. 57. Pythons did it!
    58. 58. 1997
    59. 59. 1999
    60. 60. Web vs Objects (Not to Scale)
    61. 61. How doesCelluloid work?
    62. 62. >> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>
    63. 63. >> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>
    64. 64.  # Class methods added to classes which include Celluloidmodule ClassMethods  # Create a new actor  def new(*args, &block)    proxy = Actor.new(allocate).proxy    proxy._send_(:initialize, *args, &block)    proxy  end ...
    65. 65. Synchronous Calls
    66. 66. EXTREME Late Binding
    67. 67. Synchronous Calls>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> h.inspect => “#<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>”
    68. 68. Asynchronous Calls
    69. 69. Asynchronous Calls
    70. 70. Asynchronous Calls>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> h.send!(:[]=, :foo, :bar, :baz, 42) => nil>> h[:foo, :bar, :baz] => 42
    71. 71. Asynchronous Calls>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> h.send !(:[]=, :foo, :bar, :baz, 42) => nil>> h[:foo, :bar, :baz] => 42
    72. 72. Asynchronous Calls>> h.inspect! => nil
    73. 73. Asynchronous Calls Kind of like “next tick”
    74. 74. Asynchronous CallsHow do I get the value returned?
    75. 75. Futures
    76. 76. Futures
    77. 77. Futures>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> future = h.future :inspect => #<Celluloid::Future:0x3ff3b953821a>>> 41 + 1 # roflscale computation => 42>> future.value => “#<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>”
    78. 78. Futures>> h = ConcurrentNestedHash.new => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>>> future = h.future :inspect => #<Celluloid::Future:0x3ff3b953821a>>> 41 + 1 # roflscale computation => 42>> future.value => “#<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>”
    79. 79. Basic Ingredients• Regular method calls• Async calls (“next tick”)• Futures
    80. 80. SecretSauce
    81. 81. How do weprevent deadlocks?
    82. 82. Why do deadlocks happen?
    83. 83. Waiting forsomething that never happens...
    84. 84. ...instead of what’s important
    85. 85. How can we wait oneverything at once?
    86. 86. FIBERS!
    87. 87. WAT?
    88. 88. SUBLIMINAL MESSAGE:DONALD KNUTH IS YODA
    89. 89. No really...
    90. 90. CommunicatingSequential Processes Do one thing at a time
    91. 91. FibersCheap suspendable/resumable execution context
    92. 92. CommunicatingSequential Processes + Do one thing at a time Fibers Cheap suspendable/resumable execution context
    93. 93. Internal Concurrency for Actors
    94. 94. Don’t Block, Suspend to the Scheduler
    95. 95. Example!
    96. 96. require celluloidclass Person  include Celluloid  def name    self.class.to_s  end  def greet(interested_party)    "Hello #{interested_party.name}, Im #{name}"  endendclass Joe < Person; endclass Mike < Person  def greet(other)    super << "n" << other.greet(current_actor)  endendmike = Mike.newjoe = Joe.newputs mike.greet(joe)
    97. 97. OutputHello Joe, Im MikeHello Mike, Im Joe
    98. 98. But it is acool story!
    99. 99. Circular Call Graph
    100. 100. CommunicatingSequential Processes Do one thing at a time
    101. 101. DEADLOCK!
    102. 102. require celluloidclass Person  include Celluloid  def name    self.class.to_s  end  def greet(interested_party)    "Hello #{interested_party.name}, Im #{name}"  endendclass Joe < Person; endclass Mike < Person  def greet(other)    super << "n" << other.greet(current_actor)  endendmike = Mike.newjoe = Joe.newputs mike.greet(joe)
    103. 103. Multitasking Fibers
    104. 104. Waiting taskssuspend themselves so ready tasks can run
    105. 105. Every method call creates a Fiber
    106. 106. Slow?
    107. 107. Call Benchmark Core i7 2.0GHz (OS X 10.7.3) 16815/s (60µs) - JRuby 1.6.7 20899/s (50µs) - JRuby HEADRubinius 15260/s (65µs) - rbx HEAD YARV 9367/s (107µs) - Ruby 1.9.3
    108. 108. What if an actor crashes?
    109. 109. Fault Tolerance• Supervisors & Supervision Trees• “Fail early”, restart in a clean state• Do what Erlang does• No seriously, do what Erlang does
    110. 110. Evented I/O for Celluloidhttp://github.com/celluloid/celluloid-io
    111. 111. Now that we have threads licked... how about I/O?
    112. 112. USE BLOCKING I/O
    113. 113. Blocking IO is OK!* No central event loop to block
    114. 114. *But be carefulLocks in external services = Deadlocks in Celluloid
    115. 115. But how will I serve my roflmillions of users?
    116. 116. Evented IO• Large numbers of connections (>1000)• Mostly idle connections• Mostly IO-bound problems• Websockets are an ideal case
    117. 117. Actors areevent loops
    118. 118. Normal Actors
    119. 119. Celluloid::IO Actors
    120. 120. nio4r-powered reactor
    121. 121. nio4r http://github.com/tarcieri/nio4r/• Quasi-inspired by Java NIO• Smallest API possible• libev C extension for CRuby/rbx• Java extension for JRuby• Pure Ruby version too!
    122. 122. Celluloid::IO::TCPSocket • Uses fibered I/O • “Duck-type” of ::TCPSocket • Evented inside Celluloid::IO actors • Blocking IO elsewhere (Ruby Threads, normal Celluloid actors)
    123. 123. Evented IO AND Threaded IOYou don’t have to choose!
    124. 124. Transparent handlesyou can pass around Kind of like file descriptors!
    125. 125. Other replacement classes• Celluloid::IO::TCPServer• Celluloid::IO::UDPSocket• No Celluloid::IO::UnixSocket yet, sorry :(
    126. 126. Echo Server Example
    127. 127. class EchoServer  include Celluloid::IO  def initialize(host, port)    puts "*** Starting echo server on #{host}:#{port}"    # Since we included Celluloid::IO, were actually making a    # Celluloid::IO::TCPServer here    @server = TCPServer.new(host, port)    run!  end  def finalize    @server.close if @server  end  def run    loop { handle_connection! @server.accept }  end  def handle_connection(socket)    _, port, host = socket.peeraddr    puts "*** Received connection from #{host}:#{port}"    loop { socket.write socket.readpartial(4096) }  rescue EOFError    puts "*** #{host}:#{port} disconnected"    socket.close  endendsupervisor = EchoServer.supervise("127.0.0.1", 1234)trap("INT") { supervisor.terminate; exit }sleep
    128. 128. Running out of gas after slide #150
    129. 129. ^^^ LET’S USE THIS
    130. 130. Dependency Injection
    131. 131. MyClient.new(‘myhost.domain’, 1234, :socket => Celluloid::IO::TCPSocket)
    132. 132. Easy Peasy!
    133. 133. Celluloid::IO-powered web server http://github.com/celluloid/reel
    134. 134. Hello World Benchmark# httperf --num-conns=50 --num-calls=1000Ruby Version Throughput Latency------------ ---------- -------JRuby HEAD 5650 reqs/s (0.2 ms/req)Ruby 1.9.3 5263 reqs/s (0.2 ms/req)JRuby 1.6.7 4303 reqs/s (0.2 ms/req)rbx HEAD 2288 reqs/s (0.4 ms/req)
    135. 135. Hello World ComparisonWeb Server Throughput Latency---------- ---------- -------Goliath (0.9.4) 2058 reqs/s (0.5 ms/req)Thin (1.2.11) 7502 reqs/s (0.1 ms/req)Node.js (0.6.5) 11735 reqs/s (0.1 ms/req)
    136. 136. 0MQ TOO!
    137. 137. Celluloid::ZMQ• Built on ffi-rzmq• But exposes a higher level API• Celluloid::IO for ZeroMQ
    138. 138. Distributed Celluloid over 0MQhttp://github.com/celluloid/reel
    139. 139. Mostly ready to use!
    140. 140. Probably needs its own talk :(
    141. 141. Celluloid-powered Web Framework http://github.com/celluloid/lattice
    142. 142. Vaporware!
    143. 143. Goals• Reuse parts of Rails• Multithreaded development mode• Easy scatter/gather for SOA
    144. 144. That’s all, folks!
    145. 145. Bye!
    146. 146. Links• http://github.com/celluloid• http://twitter.com/bascule• http://unlimitednovelty.com/

    ×