SlideShare a Scribd company logo
1 of 172
Concurrent programming with



      http://github.com/celluloid

           Tony Arcieri
      MountainWest RubyConf
        March 15th, 2012
About Me
About Me




libev binding for Ruby
 (1 year before Node)
About Me




     Actors +“Fibered” I/O
(2 years before em-synchrony)
About Me




Ruby-Flavored Erlang
(2 years before Elixir)
http://elixir-lang.org/
About Me




Didn’t talk about Revactor or Reia at
             MWRC 2008
What next?
If I can’t drag Erlang
  halfway to Ruby...
Perhaps I can get
 Ruby halfway to
    Erlang...
Rubyists don’t like threads
      I want to change that
x86 CPU Trends
 (Not Entirely to Scale)
Multicore
is the future
Threads
are important
No GIL!



Rubinius
Thread-level parallelism
  Threads = Multicore
Parallel Blocking I/O


            YARV
  But only one Ruby thread at a time :(
          Threads != Multicore
When everyone has
 100 core CPUs...
will we run 100
virtual machines?
or one?
Sidekiq
    What if 1 Sidekiq process could
do the work of 20 Resque processes?

   http://mperham.github.com/sidekiq/
A little about
 Revactor...
Revactor predates:

• Neverblock
• Dramatis
• Rack::FiberPool
• em-synchrony
Inspired by:

• Erlang
• Omnibus Concurrency (MenTaLguY)
• Kamaelia (Python)
• Eventlet (Python)
Single-Threaded
Bad API
listener = Actor::TCP.listen(HOST, PORT, :filter => :line)
puts "Listening on #{HOST}:#{PORT}"

# The main loop handles incoming connections
loop do
  # Spawn a new actor for each incoming connection
  Actor.spawn(listener.accept) do |sock|
    puts "#{sock.remote_addr}:#{sock.remote_port} connected"

    # Connection handshaking
    begin
      sock.write "Please enter a nickname:"
      nickname = sock.read

      server << T[:register, Actor.current, nickname]
      
      # Flip the socket into asynchronous "active" mode
      # This means the Actor can receive messages from
      # the socket alongside other events.
      sock.controller = Actor.current
      sock.active = :once
    
      # Main message loop
      loop do
        Actor.receive do |filter|
          filter.when(T[:tcp, sock]) do |_, _, message|
            server << T[:say, Actor.current, message]
            sock.active = :once
          end
WAT
listener = Actor::TCP.listen(HOST, PORT, :filter => :line)
puts "Listening on #{HOST}:#{PORT}"

# The main loop handles incoming connections
loop do
  # Spawn a new actor for each incoming connection
  Actor.spawn(listener.accept) do |sock|
    puts "#{sock.remote_addr}:#{sock.remote_port} connected"

    # Connection handshaking
    begin
      sock.write "Please enter a nickname:"
      nickname = sock.read

      server << T[:register, Actor.current, nickname]
      
      # Flip the socket into asynchronous "active" mode
      # This means the Actor can receive messages from
      # the socket alongside other events.
      sock.controller = Actor.current
      sock.active = :once
    
      # Main message loop
      loop do
        Actor.receive do |filter|
          filter.when(T[:tcp, sock]) do |_, _, message|
            server << T[:say, Actor.current, message]
            sock.active = :once
          end
Procedural!
Ugly!
WTF is T?

filter.when(T[:tcp, sock]) do |_, _, message|
  server << T[:say, Actor.current, message]
  sock.active = :once
end
Less Erlang
More Objects
Can We
Do Better?
YES
What is Celluloid?
Celluloid is
a general purpose concurrency framework
                for Ruby
A Contrived Example
require 'thread'

class ConcurrentNestedHash
  def initialize
    @outer = {}
    @mutex = Mutex.new
  end

  def [](*keys)
    @mutex.synchronize { keys.inject(@outer) { |h,k| h[k] } }
  end

  def []=(*args)
    @mutex.synchronize do
      value = args.pop
      raise ArgumentError, "wrong number of arguments (1 for 2)" if args.empty?

      key = args.pop
      hash = args.inject(@outer) { |h,k| h[k] ||= {} }
      hash[key] = value
    end
  end

  def inspect; @mutex.synchronize { super }; end
end
>> h = ConcurrentNestedHash.new
 => #<ConcurrentNestedHash:0x007f99ed735f08 @outer={}, @mutex=#<Mutex:
0x007f99ed735e90>>

>> h[:foo, :bar, :baz] = 42
 => 42

>> h
 => #<ConcurrentNestedHash:0x007f99ed735f08 @outer={:foo=>{:bar=>{:baz=>42}}},
@mutex=#<Mutex:0x007f99ed735e90>>

>> h[:foo, :bar, :baz]
 => 42
-require 'thread'
+require 'celluloid'

 class ConcurrentNestedHash
+ include Celluloid

      def initialize
        @outer = {}
-       @mutex = Mutex.new
      end

      def [](*keys)
-       @mutex.synchronize { keys.inject(@outer) { |h,k| h[k] } }
+       keys.inject(@outer) { |h,k| h[k] }
      end

      def []=(*args)
-       @mutex.synchronize do
        value = args.pop
        raise ArgumentError, "wrong number of arguments (1 for 2)" if args.empty?
        key = args.pop
        hash = args.inject(@outer) { |h,k| h[k] ||= {} }
        hash[key] = value
-       end
      end
-
-     def inspect; @mutex.synchronize { super }; end
    end
require 'celluloid'

class ConcurrentNestedHash
  include Celluloid

  def initialize
    @outer = {}
  end

  def [](*keys)
    keys.inject(@outer) { |h,k| h[k] }
  end

  def []=(*args)
    value = args.pop
    raise ArgumentError, "wrong number of arguments (1 for 2)" if args.empty?

    key = args.pop
    hash = args.inject(@outer) { |h,k| h[k] ||= {} }
    hash[key] = value
  end
end
How?
MAGIC
Automatic locking?
Locks are hard

• Dining Philosophers Problem
• Sleeping Barber Problem
• Cigarette Smokers Problem
OOP + Concurrency
“I thought of objects being like biological
cells and/or individual computers on a
network, only able to communicate with
messages”

- Alan Kay, creator of Smalltalk, on the meaning of "object
oriented programming"
OOP
Tools
 Classes


Inheritance


Messages
Concurrency
   Tools
   Threads


    Locks


   Queues
OOP           Concurrency
Tools            Tools



              +
 Classes          Threads


Inheritance        Locks


Messages          Queues
=
Concurrent Objects
Communicating
Sequential Processes
 • Do One Thing At a Time
 • Communicate With Messages
 • Encapsulate Local State
Concurrent Objects
   do more...
Generalized
 Deadlock-free
Synchronization
Pythons did it!
1997
1999
Web vs Objects
   (Not to Scale)
How does
Celluloid work?
>> h = ConcurrentNestedHash.new
 => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>
>> h = ConcurrentNestedHash.new
 => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>
 
# Class methods added to classes which include Celluloid
module ClassMethods
  # Create a new actor
  def new(*args, &block)
    proxy = Actor.new(allocate).proxy
    proxy._send_(:initialize, *args, &block)
    proxy
  end

  ...
Synchronous
   Calls
EXTREME Late
   Binding
Synchronous Calls


>> h = ConcurrentNestedHash.new
 => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>

>> h.inspect
 => “#<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>”
Asynchronous
    Calls
Asynchronous Calls
Asynchronous Calls

>> h = ConcurrentNestedHash.new
 => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>

>> h.send!(:[]=, :foo, :bar, :baz, 42)
 => nil

>> h[:foo, :bar, :baz]
 => 42
Asynchronous Calls

>> h = ConcurrentNestedHash.new
 => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>


>> h.send   !(:[]=,   :foo, :bar, :baz, 42)
 =>   nil
>> h[:foo, :bar, :baz]
 => 42
Asynchronous Calls


>> h.inspect!
 => nil
Asynchronous Calls


  Kind of like “next tick”
Asynchronous Calls


How do I get the value returned?
Futures
Futures
Futures

>> h = ConcurrentNestedHash.new
 => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>

>> future = h.future :inspect
 => #<Celluloid::Future:0x3ff3b953821a>

>> 41 + 1 # roflscale computation
 => 42

>> future.value
 => “#<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>”
Futures

>> h = ConcurrentNestedHash.new
 => #<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>

>> future = h.future :inspect
 => #<Celluloid::Future:0x3ff3b953821a>

>> 41 + 1 # roflscale computation
 => 42

>> future.value
 => “#<Celluloid::Actor(ConcurrentNestedHash:0x3ff3b952df7c) @outer={}>”
Basic Ingredients

• Regular method calls
• Async calls (“next tick”)
• Futures
Secret
Sauce
How do we
prevent deadlocks?
Why do deadlocks
   happen?
Waiting for
something that never
    happens...
...instead of what’s
      important
How can we wait on
everything at once?
FIBERS!
WAT?
SUBLIMINAL
  MESSAGE:


DONALD KNUTH
  IS YODA
No really...
Communicating
Sequential Processes
    Do one thing at a time
Fibers
Cheap suspendable/resumable
     execution context
Communicating
Sequential Processes


          +
    Do one thing at a time




         Fibers
 Cheap suspendable/resumable
      execution context
Internal Concurrency
      for Actors
Don’t Block, Suspend
  to the Scheduler
Example!
require 'celluloid'

class Person
  include Celluloid

  def name
    self.class.to_s
  end

  def greet(interested_party)
    "Hello #{interested_party.name}, I'm #{name}"
  end
end

class Joe < Person; end

class Mike < Person
  def greet(other)
    super << "n" << other.greet(current_actor)
  end
end

mike = Mike.new
joe = Joe.new

puts mike.greet(joe)
Output

Hello Joe, I'm Mike
Hello Mike, I'm Joe
But it is a
cool story!
Circular Call Graph
Communicating
Sequential Processes
    Do one thing at a time
DEADLOCK!
require 'celluloid'

class Person
  include Celluloid

  def name
    self.class.to_s
  end

  def greet(interested_party)
    "Hello #{interested_party.name}, I'm #{name}"
  end
end

class Joe < Person; end

class Mike < Person
  def greet(other)
    super << "n" << other.greet(current_actor)
  end
end

mike = Mike.new
joe = Joe.new

puts mike.greet(joe)
Multitasking Fibers
Waiting tasks
suspend themselves
   so ready tasks can run
Every method call
 creates a Fiber
Slow?
Call Benchmark
   Core i7 2.0GHz (OS X 10.7.3)

              16815/s (60µs) - JRuby 1.6.7

              20899/s (50µs) - JRuby HEAD


Rubinius      15260/s (65µs) - rbx HEAD



  YARV        9367/s (107µs) - Ruby 1.9.3
What if an actor
  crashes?
Fault Tolerance

• Supervisors & Supervision Trees
• “Fail early”, restart in a clean state
• Do what Erlang does
• No seriously, do what Erlang does
Evented I/O for Celluloid
http://github.com/celluloid/celluloid-io
Now that we have
 threads licked...
    how about I/O?
USE BLOCKING I/O
Blocking IO is OK!*
 No central event loop to block
*But be careful
Locks in external services
            =
 Deadlocks in Celluloid
But how will I serve
 my roflmillions of
      users?
Evented IO

• Large numbers of connections (>1000)
• Mostly idle connections
• Mostly IO-bound problems
• Websockets are an ideal case
Actors are
event loops
Normal Actors
Celluloid::IO Actors
nio4r-powered
   reactor
nio4r
    http://github.com/tarcieri/nio4r/

• Quasi-inspired by Java NIO
• Smallest API possible
• libev C extension for CRuby/rbx
• Java extension for JRuby
• Pure Ruby version too!
Celluloid::IO::TCPSocket

 • Uses fibered I/O
 • “Duck-type” of ::TCPSocket
 • Evented inside Celluloid::IO actors
 • Blocking IO elsewhere (Ruby
   Threads, normal Celluloid actors)
Evented IO
     AND
 Threaded IO
You don’t have to choose!
Transparent handles
you can pass around
  Kind of like file descriptors!
Other replacement
       classes
• Celluloid::IO::TCPServer
• Celluloid::IO::UDPSocket
• No Celluloid::IO::UnixSocket yet,
  sorry :(
Echo Server
 Example
class EchoServer
  include Celluloid::IO

  def initialize(host, port)
    puts "*** Starting echo server on #{host}:#{port}"

    # Since we included Celluloid::IO, we're actually making a
    # Celluloid::IO::TCPServer here
    @server = TCPServer.new(host, port)
    run!
  end

  def finalize
    @server.close if @server
  end

  def run
    loop { handle_connection! @server.accept }
  end

  def handle_connection(socket)
    _, port, host = socket.peeraddr
    puts "*** Received connection from #{host}:#{port}"
    loop { socket.write socket.readpartial(4096) }
  rescue EOFError
    puts "*** #{host}:#{port} disconnected"
    socket.close
  end
end

supervisor = EchoServer.supervise("127.0.0.1", 1234)
trap("INT") { supervisor.terminate; exit }
sleep
Running out of gas after slide #150
^^^ LET’S USE THIS
Dependency
 Injection
MyClient.new(‘myhost.domain’, 1234, :socket => Celluloid::IO::TCPSocket)
Easy Peasy!
Celluloid::IO-powered web server
    http://github.com/celluloid/reel
Hello World
        Benchmark
# httperf --num-conns=50 --num-calls=1000

Ruby Version        Throughput    Latency
------------        ----------    -------
JRuby HEAD          5650 reqs/s   (0.2 ms/req)
Ruby 1.9.3          5263 reqs/s   (0.2 ms/req)
JRuby 1.6.7         4303 reqs/s   (0.2 ms/req)
rbx HEAD            2288 reqs/s   (0.4 ms/req)
Hello World
        Comparison
Web Server         Throughput     Latency
----------         ----------     -------
Goliath (0.9.4)    2058 reqs/s    (0.5 ms/req)
Thin    (1.2.11)   7502 reqs/s    (0.1 ms/req)
Node.js (0.6.5)    11735 reqs/s   (0.1 ms/req)
0MQ TOO!
Celluloid::ZMQ


• Built on ffi-rzmq
• But exposes a higher level API
• Celluloid::IO for ZeroMQ
Distributed Celluloid over 0MQ
http://github.com/celluloid/reel
Mostly ready to use!
Probably needs its
   own talk :(
Celluloid-powered Web Framework
 http://github.com/celluloid/lattice
Vaporware!
Goals

• Reuse parts of Rails
• Multithreaded development mode
• Easy scatter/gather for SOA
That’s all, folks!
Bye!
Links

• http://github.com/celluloid
• http://twitter.com/bascule
• http://unlimitednovelty.com/

More Related Content

What's hot

Elegant Solutions For Everyday Python Problems - Nina Zakharenko
Elegant Solutions For Everyday Python Problems - Nina ZakharenkoElegant Solutions For Everyday Python Problems - Nina Zakharenko
Elegant Solutions For Everyday Python Problems - Nina ZakharenkoNina Zakharenko
 
Rust Mozlando Tutorial
Rust Mozlando TutorialRust Mozlando Tutorial
Rust Mozlando Tutorialnikomatsakis
 
Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...
Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...
Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...Charles Nutter
 
Introduction to clojure
Introduction to clojureIntroduction to clojure
Introduction to clojureAbbas Raza
 
JVMLS 2016. Coroutines in Kotlin
JVMLS 2016. Coroutines in KotlinJVMLS 2016. Coroutines in Kotlin
JVMLS 2016. Coroutines in KotlinAndrey Breslav
 
Introduction to kotlin + spring boot demo
Introduction to kotlin + spring boot demoIntroduction to kotlin + spring boot demo
Introduction to kotlin + spring boot demoMuhammad Abdullah
 
Rust concurrency tutorial 2015 12-02
Rust concurrency tutorial 2015 12-02Rust concurrency tutorial 2015 12-02
Rust concurrency tutorial 2015 12-02nikomatsakis
 
Clojure: Practical functional approach on JVM
Clojure: Practical functional approach on JVMClojure: Practical functional approach on JVM
Clojure: Practical functional approach on JVMsunng87
 
50 new things we can do with Java 8
50 new things we can do with Java 850 new things we can do with Java 8
50 new things we can do with Java 8José Paumard
 
Lambdas and Streams Master Class Part 2
Lambdas and Streams Master Class Part 2Lambdas and Streams Master Class Part 2
Lambdas and Streams Master Class Part 2José Paumard
 
Introduction to Rust language programming
Introduction to Rust language programmingIntroduction to Rust language programming
Introduction to Rust language programmingRodolfo Finochietti
 
DRYing to Monad in Java8
DRYing to Monad in Java8DRYing to Monad in Java8
DRYing to Monad in Java8Dhaval Dalal
 
Lambda and Stream Master class - part 1
Lambda and Stream Master class - part 1Lambda and Stream Master class - part 1
Lambda and Stream Master class - part 1José Paumard
 
ConFess Vienna 2015 - Metaprogramming with Groovy
ConFess Vienna 2015 - Metaprogramming with GroovyConFess Vienna 2015 - Metaprogramming with Groovy
ConFess Vienna 2015 - Metaprogramming with GroovyIván López Martín
 
Oral presentation v2
Oral presentation v2Oral presentation v2
Oral presentation v2Yeqi He
 

What's hot (20)

Elegant Solutions For Everyday Python Problems - Nina Zakharenko
Elegant Solutions For Everyday Python Problems - Nina ZakharenkoElegant Solutions For Everyday Python Problems - Nina Zakharenko
Elegant Solutions For Everyday Python Problems - Nina Zakharenko
 
Rust Mozlando Tutorial
Rust Mozlando TutorialRust Mozlando Tutorial
Rust Mozlando Tutorial
 
Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...
Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...
Øredev 2011 - JVM JIT for Dummies (What the JVM Does With Your Bytecode When ...
 
Introduction to clojure
Introduction to clojureIntroduction to clojure
Introduction to clojure
 
JVMLS 2016. Coroutines in Kotlin
JVMLS 2016. Coroutines in KotlinJVMLS 2016. Coroutines in Kotlin
JVMLS 2016. Coroutines in Kotlin
 
Introduction to kotlin + spring boot demo
Introduction to kotlin + spring boot demoIntroduction to kotlin + spring boot demo
Introduction to kotlin + spring boot demo
 
Rust concurrency tutorial 2015 12-02
Rust concurrency tutorial 2015 12-02Rust concurrency tutorial 2015 12-02
Rust concurrency tutorial 2015 12-02
 
Clojure: Practical functional approach on JVM
Clojure: Practical functional approach on JVMClojure: Practical functional approach on JVM
Clojure: Practical functional approach on JVM
 
java sockets
 java sockets java sockets
java sockets
 
Rust-lang
Rust-langRust-lang
Rust-lang
 
Don't do this
Don't do thisDon't do this
Don't do this
 
50 new things we can do with Java 8
50 new things we can do with Java 850 new things we can do with Java 8
50 new things we can do with Java 8
 
Lambdas and Streams Master Class Part 2
Lambdas and Streams Master Class Part 2Lambdas and Streams Master Class Part 2
Lambdas and Streams Master Class Part 2
 
Introduction to Rust language programming
Introduction to Rust language programmingIntroduction to Rust language programming
Introduction to Rust language programming
 
Akka in-action
Akka in-actionAkka in-action
Akka in-action
 
Groovy!
Groovy!Groovy!
Groovy!
 
DRYing to Monad in Java8
DRYing to Monad in Java8DRYing to Monad in Java8
DRYing to Monad in Java8
 
Lambda and Stream Master class - part 1
Lambda and Stream Master class - part 1Lambda and Stream Master class - part 1
Lambda and Stream Master class - part 1
 
ConFess Vienna 2015 - Metaprogramming with Groovy
ConFess Vienna 2015 - Metaprogramming with GroovyConFess Vienna 2015 - Metaprogramming with Groovy
ConFess Vienna 2015 - Metaprogramming with Groovy
 
Oral presentation v2
Oral presentation v2Oral presentation v2
Oral presentation v2
 

Similar to Concurrent programming with Celluloid libev binding for Ruby actors and fibered I/O

What can be done with Java, but should better be done with Erlang (@pavlobaron)
What can be done with Java, but should better be done with Erlang (@pavlobaron)What can be done with Java, but should better be done with Erlang (@pavlobaron)
What can be done with Java, but should better be done with Erlang (@pavlobaron)Pavlo Baron
 
Principles of the Play framework
Principles of the Play frameworkPrinciples of the Play framework
Principles of the Play frameworkBernhard Huemer
 
Clojure 1.1 And Beyond
Clojure 1.1 And BeyondClojure 1.1 And Beyond
Clojure 1.1 And BeyondMike Fogus
 
Everything is Permitted: Extending Built-ins
Everything is Permitted: Extending Built-insEverything is Permitted: Extending Built-ins
Everything is Permitted: Extending Built-insAndrew Dupont
 
Paradigma FP y OOP usando técnicas avanzadas de Programación | Programacion A...
Paradigma FP y OOP usando técnicas avanzadas de Programación | Programacion A...Paradigma FP y OOP usando técnicas avanzadas de Programación | Programacion A...
Paradigma FP y OOP usando técnicas avanzadas de Programación | Programacion A...Víctor Bolinches
 
Kotlin coroutines and spring framework
Kotlin coroutines and spring frameworkKotlin coroutines and spring framework
Kotlin coroutines and spring frameworkSunghyouk Bae
 
Postobjektové programovanie v Ruby
Postobjektové programovanie v RubyPostobjektové programovanie v Ruby
Postobjektové programovanie v RubyJano Suchal
 
Stackless Python 101
Stackless Python 101Stackless Python 101
Stackless Python 101guest162fd90
 
Blocks by Lachs Cox
Blocks by Lachs CoxBlocks by Lachs Cox
Blocks by Lachs Coxlachie
 
Clojure concurrency
Clojure concurrencyClojure concurrency
Clojure concurrencyAlex Navis
 
.NET Multithreading/Multitasking
.NET Multithreading/Multitasking.NET Multithreading/Multitasking
.NET Multithreading/MultitaskingSasha Kravchuk
 
Asynchronous Orchestration DSL on squbs
Asynchronous Orchestration DSL on squbsAsynchronous Orchestration DSL on squbs
Asynchronous Orchestration DSL on squbsAnil Gursel
 
Server side JavaScript: going all the way
Server side JavaScript: going all the wayServer side JavaScript: going all the way
Server side JavaScript: going all the wayOleg Podsechin
 
Actor Clustering with Docker Containers and Akka.Net in F#
Actor Clustering with Docker Containers and Akka.Net in F#Actor Clustering with Docker Containers and Akka.Net in F#
Actor Clustering with Docker Containers and Akka.Net in F#Riccardo Terrell
 
Dataflow: Declarative concurrency in Ruby
Dataflow: Declarative concurrency in RubyDataflow: Declarative concurrency in Ruby
Dataflow: Declarative concurrency in RubyLarry Diehl
 

Similar to Concurrent programming with Celluloid libev binding for Ruby actors and fibered I/O (20)

What can be done with Java, but should better be done with Erlang (@pavlobaron)
What can be done with Java, but should better be done with Erlang (@pavlobaron)What can be done with Java, but should better be done with Erlang (@pavlobaron)
What can be done with Java, but should better be done with Erlang (@pavlobaron)
 
Principles of the Play framework
Principles of the Play frameworkPrinciples of the Play framework
Principles of the Play framework
 
Clojure 1.1 And Beyond
Clojure 1.1 And BeyondClojure 1.1 And Beyond
Clojure 1.1 And Beyond
 
Everything is Permitted: Extending Built-ins
Everything is Permitted: Extending Built-insEverything is Permitted: Extending Built-ins
Everything is Permitted: Extending Built-ins
 
Paradigma FP y OOP usando técnicas avanzadas de Programación | Programacion A...
Paradigma FP y OOP usando técnicas avanzadas de Programación | Programacion A...Paradigma FP y OOP usando técnicas avanzadas de Programación | Programacion A...
Paradigma FP y OOP usando técnicas avanzadas de Programación | Programacion A...
 
An introduction to Ruby
An introduction to RubyAn introduction to Ruby
An introduction to Ruby
 
Ruby
RubyRuby
Ruby
 
Kotlin coroutines and spring framework
Kotlin coroutines and spring frameworkKotlin coroutines and spring framework
Kotlin coroutines and spring framework
 
Postobjektové programovanie v Ruby
Postobjektové programovanie v RubyPostobjektové programovanie v Ruby
Postobjektové programovanie v Ruby
 
Stackless Python 101
Stackless Python 101Stackless Python 101
Stackless Python 101
 
ssh.isdn.test
ssh.isdn.testssh.isdn.test
ssh.isdn.test
 
Blocks by Lachs Cox
Blocks by Lachs CoxBlocks by Lachs Cox
Blocks by Lachs Cox
 
Ruby basics
Ruby basicsRuby basics
Ruby basics
 
Clojure concurrency
Clojure concurrencyClojure concurrency
Clojure concurrency
 
.NET Multithreading/Multitasking
.NET Multithreading/Multitasking.NET Multithreading/Multitasking
.NET Multithreading/Multitasking
 
Asynchronous Orchestration DSL on squbs
Asynchronous Orchestration DSL on squbsAsynchronous Orchestration DSL on squbs
Asynchronous Orchestration DSL on squbs
 
Current State of Coroutines
Current State of CoroutinesCurrent State of Coroutines
Current State of Coroutines
 
Server side JavaScript: going all the way
Server side JavaScript: going all the wayServer side JavaScript: going all the way
Server side JavaScript: going all the way
 
Actor Clustering with Docker Containers and Akka.Net in F#
Actor Clustering with Docker Containers and Akka.Net in F#Actor Clustering with Docker Containers and Akka.Net in F#
Actor Clustering with Docker Containers and Akka.Net in F#
 
Dataflow: Declarative concurrency in Ruby
Dataflow: Declarative concurrency in RubyDataflow: Declarative concurrency in Ruby
Dataflow: Declarative concurrency in Ruby
 

Recently uploaded

Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 

Recently uploaded (20)

Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 

Concurrent programming with Celluloid libev binding for Ruby actors and fibered I/O

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. Mention Omnibus, Rubinius, MenTaLguY\n
  5. My most popular project is probably Reia, where I attempted to build an immutable Ruby-like language on top of the Erlang VM, then quit after Jose Valim built a better language called Elixir\n
  6. \n
  7. \n
  8. \n
  9. I got into Erlang in 2007 has been a huge influence in the way I think\nIt blew my mind when I first discovered it\nMany of my subsequent ideas I owe to Erlang\n
  10. The Erlang guys were ahead of their time (that&amp;#x2019;s Joe Armstrong, Erlang&amp;#x2019;s creator there in the photo)\nI wrote a nasty blog post about the warts in Erlang as a language, but their ideas are still phenomenal and continue to influence me to this day\n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. I&amp;#x2019;ll give you a second to look at this to form an opinion, but my opinion is...\n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. YO YO GET READY\n
  53. This is my favorite quote from Alan Kay. What particularly interests me is the idea that objects are like biological cells or networked computers, self-contained, encapsulated, and running in parallel. Object oriented programming, in Alan Kay&amp;#x2019;s conception, naturally predisposes itself to concurrency.\n
  54. If we take the object oriented tools (LIKE, EXPOUND)...\n
  55. ...and the tools for concurrency (LIKE, EXPOUND)...\n
  56. ...and combine them...\n
  57. we get\n
  58. concurrent objects. (*pause*) I&amp;#x2019;ll dig into this slide a bit later. Celluloid builds on the ideas of the actor model and communicating...\n
  59. ...sequential processes\n
  60. Concurrent objects represent a higher level of abstraction for solving concurrency problems, and one I think is unique\n
  61. Concurrent objects represent a higher level of abstraction for solving concurrency problems, and one I think is unique\n
  62. it&amp;#x2019;s one not\n
  63. Erlang\n
  64. or\n
  65. Scala can give you\n
  66. At first I thought this was something I&amp;#x2019;d invented independently, but after doing some research on the subject, I discovered...\n
  67. \n
  68. I found a paper detailing a nearly identical system to Celluloid in Python\nIt felt like a validation I&amp;#x2019;m not off in the wilderness somewhere, someone has explored these ideas before\nAnd what&amp;#x2019;s more...\n
  69. They did it in 1997\n
  70. I&amp;#x2019;m not sure how well the rest of you remember 1997, but to refresh your memory, computers kind of sucked back then.\nIt wasn&amp;#x2019;t the greatest time to be working on projects targeting concurrency or massively multicore CPUs.\n
  71. Flash forward to 1999, yours truly was working on revolutionizing the state of the art of\n
  72. X11 CD players. But just for the record, I was into rounded corners and gradients before they were popular\n
  73. In the late &amp;#x2018;80s and early &amp;#x2018;90s academic research into concurrent and distributed objects was booming, by the late &amp;#x2018;90s research in the field was virtually nonexistent and the number of papers published on the topic had practically ground to a halt. I believe this was due to the overpowering effect of the web and HTTP as the universal abstraction.\n
  74. While the web is an amazing thing, I think it may have distracted us from some good ideas, and that concurrent objects probably deserve another look. So let&amp;#x2019;s take a look at\n
  75. ...how Celluloid works. If we go back to the Celluloid version of the ConcurrentNestedHash example\n
  76. and make a concurrent NestedHash object, this is what we see\nPay particular attention to...\n
  77. the Celluloid::Actor part there\n
  78. Celluloid hijacks the new method, encapsulating newly created objects inside of actors, and hands you a proxy object to talk to them. This all happens before it even calls initialize\n
  79. Let&amp;#x2019;s go back to that concurrent object diagram I showed you earlier...\n
  80. \n
  81. \n
  82. Another kind of call pattern Celluloid provides is the \n
  83. asynchronous call pattern. When you don&amp;#x2019;t need a result, this is the pattern to use, because it provides a nice, clean, fast path to schedule work in another actor.\n
  84. To send another object an asynchronous call, \n
  85. pay particular attention to bang/nil\nasync calls end in a bang and always return nil because they&amp;#x2019;re async\n
  86. there&amp;#x2019;s an async bang method counterpart for every method, so we can pointlessly call inspect bang\nbang means &amp;#x201C;dangerous&amp;#x201D;\nasync methods are dangerous because you have no guarantees they&amp;#x2019;ll complete, but that&amp;#x2019;s OK\n
  87. \n
  88. \n
  89. \n
  90. \n
  91. \n
  92. \n
  93. \n
  94. So now that we know the basic ingredients, what&amp;#x2019;s the secret sauce?\n
  95. The part I&amp;#x2019;m sure you&amp;#x2019;re dying to know is how Celluloid prevents\n
  96. DEADLOCKS. This problem bothered me for many years. There&amp;#x2019;s many pretenders to the throne here, like\n
  97. ERLANG\n
  98. Erlang emphasizes messaging as the universal abstraction. This is a beautiful concept, but there&amp;#x2019;s a problem. Erlang&amp;#x2019;s gen_server manages to get around deadlocks... by totally punting\n
  99. \n
  100. Erlang\n
  101. \n
  102. \n
  103. \n
  104. \n
  105. \n
  106. \n
  107. \n
  108. \n
  109. \n
  110. \n
  111. \n
  112. \n
  113. \n
  114. \n
  115. \n
  116. \n
  117. \n
  118. \n
  119. \n
  120. \n
  121. \n
  122. If communicating sequential processes can only do one thing at a time, then we&amp;#x2019;d expect a deadlock...\n
  123. ...here, but we don&amp;#x2019;t get one. Let&amp;#x2019;s look at the code again.\n
  124. Take another look at the code paths here and then I&amp;#x2019;ll show you the solution... all right, you ready?\n
  125. \n
  126. \n
  127. \n
  128. \n
  129. \n
  130. \n
  131. \n
  132. \n
  133. ERLANG\n
  134. \n
  135. \n
  136. \n
  137. \n
  138. \n
  139. \n
  140. \n
  141. \n
  142. \n
  143. \n
  144. \n
  145. \n
  146. \n
  147. \n
  148. \n
  149. \n
  150. \n
  151. \n
  152. \n
  153. \n
  154. \n
  155. \n
  156. \n
  157. \n
  158. \n
  159. \n
  160. \n
  161. \n
  162. \n
  163. \n
  164. \n
  165. \n
  166. \n
  167. \n
  168. \n
  169. \n
  170. \n
  171. \n
  172. \n