Celluloid - Beyond Sidekiq
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Celluloid - Beyond Sidekiq

on

  • 2,007 views

My RS on Rails 2013 keynote, talking about Rubinius project and main features of Celluloid.

My RS on Rails 2013 keynote, talking about Rubinius project and main features of Celluloid.

Statistics

Views

Total Views
2,007
Views on SlideShare
1,979
Embed Views
28

Actions

Likes
3
Downloads
7
Comments
0

2 Embeds 28

https://twitter.com 14
http://lanyrd.com 14

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Celluloid - Beyond Sidekiq Presentation Transcript

  • 1. Celluloid: Beyond Sidekiq Marcelo Pinheiro @salizzar - https://github.com/salizzar Sunday, October 20, 13
  • 2. Keynote Soundtrack Sunday, October 20, 13
  • 3. Topics Intro to Actor Model Rubinius: cores 4 Ruby Celluloid: AM in Ruby How it works Supervisors, Supervision Groups Linking, Observers Futures, Pools, Notifications Celluloid + Rails = ? Simple Fractal Demo Sunday, October 20, 13
  • 4. Intro to Actor Model Carl Hewitt paper from 1973 [OT] Two processors with native support: J-Machine Cosmic Cube Inspired well-known languages: Smalltalk Erlang (embraces at all) Sunday, October 20, 13
  • 5. Intro to Actor Model Actor Model provides a abstraction to concurrency Threads Locks Gained popularity today with multi-core programming challenge Better CPU use More CPUs Massive concurrency Sunday, October 20, 13
  • 6. Intro to Actor Model Everything is a actor Actors communicates between self by asynchronous message exchange Does sound familiar? Mailbox to buffer incoming messages Mailbox processing with pattern matching Each actor runs as independent, lightweight process No shared state Sunday, October 20, 13
  • 7. Intro to Actor Model It sounds good, but is not a silver bullet (as anything in Planet Earth) Data *must* be immutable Requires multi-core support to be effective You need to be comfortable to change your way of thinking :) Sunday, October 20, 13
  • 8. Rubinius: cores 4 Ruby Sunday, October 20, 13
  • 9. Rubinius: cores 4 Ruby Created by Evan Phoenix Implements concurrency support for Ruby (no GIL) Better Garbage Collector Uses LLVM for aggressive bytecode optimization with JIT (Just-In-Time) machine code compiler Written mainly in C++ Ruby STDLIB in pure Ruby (rubysl*) Wraps native gems with FFI Sunday, October 20, 13
  • 10. Rubinius: cores 4 Ruby Supports MRI 2.0 features, focusing on 2.1 horizon Rubinius implementation was created RubySpec, a executable specification for Ruby Programming Language Used by JRuby too Sunday, October 20, 13
  • 11. Rubinius: cores 4 Ruby To install in your computer: $ rbenv install rbx-2.0.0 $ rvm install rbx To install in your server: No available OSs packages at this moment :’( But I create one for Debian Wheezy 64 bits :D https://github.com/salizzar/rubinius-debian Next target: CentOS ;) Sunday, October 20, 13
  • 12. Rubinius: cores 4 Ruby HOT NEWS: Rubinius X was created in last week Created by Brian Shirai Roadmap to modernize Ruby @polemiquinho: Ruby is dying Sunday, October 20, 13
  • 13. Celluloid: AM in Ruby Sunday, October 20, 13
  • 14. Celluloid: AM in Ruby Created by Tony Arcieri Inspired from Erlang concurrency approach Gooby pls, only thread-safe libs Requires Ruby 1.9 support MRI >= 1.9 JRuby >= 1.6 Rubinius >= 1.2 Sunday, October 20, 13
  • 15. Celluloid: AM in Ruby Celluloid contains a lot of features, check on Github Wiki Page Main Features Automatic Synchronization Celluloid manages method dispatch and threads Abstraction layer to Threads / Fibers, don’t worry to manage it Fault-Tolerance Let it crash Erlang philosophy Celluloid offers mechanisms to handle crashed actors Sunday, October 20, 13
  • 16. Celluloid: AM in Ruby require 'celluloid' class FredFlinstone include Celluloid def scream(to) @scream = "#{to}#{to[-1] * 10}" @screamed_at = Time.now end def resume "Screamed [#{@scream}] at #{@screamed_at}" end end Sunday, October 20, 13
  • 17. Celluloid: AM in Ruby [1] pry(main)> fred = FredFlinstone.new => #<Celluloid::ActorProxy(FredFlinstone:0x1164f94)> [2] pry(main)> fred.async.scream("Wilma") => nil [3] pry(main)> fred.resume => "Screamed [Wilmaaaaaaaaaaaa] at 2013-10-13 17:12:59 -0300" Sunday, October 20, 13
  • 18. Celluloid: AM in Ruby Fault-Tolerance Let-it-crash Erlang philosophy A crashed actor *must* be handled or your application will be down Celluloid have the following mechanisms: Supervisors Supervision Groups Linking Sunday, October 20, 13
  • 19. Celluloid: Supervisors How actors crash? Simple, unhandled exceptions Warning #1: async calls that raises an error crashes the message receiver; posterior calls NOT RAISES ANYTHING Warning #2: each actor spawn a native Thread, that is not automatically cleaned by GC; you MUST explicitly terminate this if not crashed Supervise to the rescue Sunday, October 20, 13
  • 20. Celluloid: Supervisors require 'celluloid' class Devops include Celluloid def initialize(name) @name = name end def up_to_no_good @bad_cmd = 'rm-f /' @command = `#{@bad_cmd}`, @executed_at = Time.now end end Sunday, October 20, 13
  • 21. Celluloid: Supervisors [1] pry(main)> supervisor = Devops.supervise "salizzar" => #<Celluloid::ActorProxy(Celluloid::SupervisionGroup:0x104af14) (...)> [2] pry(main)> salizzar = supervisor.actors.first => #<Celluloid::ActorProxy(Devops:0x104c33c) @name="salizzar"> [3] pry(main)> salizzar.async.up_to_no_good => nil [4] pry(main)> E, [2013-10-13T17:37:37.296348 #4829] ERROR -- : Devops crashed! Errno::ENOENT: No such file or directory - rm-f / ! (pry):9:in ``' ! (pry):9:in `up_to_no_good' ! (... backtrace error here ...) [5] pry(main)> salizzar => #<Celluloid::ActorProxy:0x104be64> [6] pry(main)> salizzar.terminate Celluloid::DeadActorError: actor already terminated from /vagrant/vendor/bundle/ruby/1.9.1/gems/celluloid-0.15.2/lib/celluloid/ proxies/actor_proxy.rb:66:in `terminate!' [7] pry(main)> salizzar = supervisor.actors.first => #<Celluloid::ActorProxy(Devops:0xe81368) @name="salizzar"> Sunday, October 20, 13
  • 22. Celluloid: Supervision Groups Supervise many actors at once Able to supervise other groups too You can create pools of supervised actors Transparent GC cleaning (automagically terminates all supervised actors) Sunday, October 20, 13
  • 23. Celluloid: Supervision Groups require 'celluloid' class QuarryWorker include Celluloid def initialize(sound) ; @sound = sound ; end def explode ; puts @sound.upcase ; end end class EyeOfSauron < Celluloid::SupervisionGroup supervise FredFlinstone, as: :fred pool QuarryWorker, as: :quarry_pool, args: [ 'boom' ] end Sunday, October 20, 13
  • 24. Celluloid: Supervision Groups [1] pry(main)> eye_of_sauron = EyeOfSauron.run! => #<Celluloid::ActorProxy(EyeOfSauron:0x17b608c) (...)> [2] pry(main)> fred = Celluloid::Actor[:fred] => #<Celluloid::ActorProxy(FredFlinstone:0x17abf38)> [3] pry(main)> quarry_pool = Celluloid::Actor[:quarry_pool] => #<Celluloid::ActorProxy(QuarryWorker:0x14a2cc4) @sound="boom"> [4] pry(main)> quarry_pool.async.explode BOOOM => nil [5] pry(main)> fred.async.scream("Barney") => nil [6] pry(main)> fred.resume => "Screamed [Barneyyyyyyyyyyy] at 2013-10-13 18:40:12 -0300" Sunday, October 20, 13
  • 25. Celluloid: Linking Suppose that you have two interdependent actors and wants to be notified if one fails Association by linking actor that commonly dies and the receiver enables a simple callback when failure occurs Useful to catch exceptions Sunday, October 20, 13
  • 26. Celluloid: Linking require 'celluloid' class RobertoBaggio include Celluloid class KickedFarAwayError < StandardError; end def kick_penalty raise(KickedFarAwayError, "OH MAMMA MIA! :'(") end end Sunday, October 20, 13
  • 27. Celluloid: Linking require 'celluloid' class GalvaoBueno include Celluloid trap_exit :penalty_kick def penalty_kick(player, reason) puts "#{player.inspect} will kick and... #{reason.class}!" 2.times { puts "ACABOOOOOOU! "; sleep(1) } 3.times { puts "EH TETRAAAA! "; sleep(1) } end end Sunday, October 20, 13
  • 28. Celluloid: Linking [1] pry(main)> galvao = GalvaoBueno.new => #<Celluloid::ActorProxy(GalvaoBueno:0x19808e0)> [2] pry(main)> baggio = RobertoBaggio.new => #<Celluloid::ActorProxy(RobertoBaggio:0x1984fe4)> [3] pry(main)> galvao.link(baggio) => #<Celluloid::ActorProxy(RobertoBaggio:0x1984fe4)> [4] pry(main)> baggio.async.kick_penalty E, [2013-10-13T18:46:41.512914 #4972] ERROR -- : RobertoBaggio crashed! RobertoBaggio::KickedFarAwayError: OH MAMMA MIA! :'( ! (...) backtrace here (...) #<Celluloid::ActorProxy(RobertoBaggio) dead> will kick and... RobertoBaggio::KickedFarAwayError! ACABOOOOOOU! => nil ACABOOOOOOU! EH TETRAAAA! EH TETRAAAA! EH TETRAAAA! Sunday, October 20, 13
  • 29. Celluloid: Futures Lazy computation: calling method with .future returns a Future object, that will be executed when .value is called When value is required, Celluloid synchronously call method and returns Transparent error raising Sunday, October 20, 13
  • 30. Celluloid: Futures require 'celluloid' require 'restclient' class LazyConsumer include Celluloid def retrieve RestClient.get('http://www.locaweb.com.br').body end end Sunday, October 20, 13
  • 31. Celluloid: Futures [1] pry(main)> consumer = LazyConsumer.new => #<Celluloid::ActorProxy(LazyConsumer:0xf38284)> [2] pry(main)> future = consumer.future.retrieve => #<Celluloid::Future:0x00000001f925d0> [3] pry(main)> future.value => "<!DOCTYPE html>rn<html dir="ltr" lang="pt-BR">r n<head>rnt <meta http-equiv="content-type" content= "text/html; charset=utf-8" />rn <meta name="robots" content="index, follow" /> (...) HTML (...)" Sunday, October 20, 13
  • 32. Celluloid: Pools Generalized pool mechanism Default size: number of processors (Celluloid.cores) Delegates method call to a worker in pool to execute In MRI, performance is OK with async I/O Two tips: Synchronous call with concurrent actors accessing pool Asynchronous call or Futures with parallel computation Sunday, October 20, 13
  • 33. Celluloid: Pools require 'celluloid' require 'complex' class ComplexFactory def create(seed) factor = seed + 0.1 a, b = 2.times.collect { srand() % factor } Complex(a, b) end end Sunday, October 20, 13
  • 34. Celluloid: Pools [1] pry(main)> pool = ComplexFactory.pool => #<Celluloid::ActorProxy(ComplexFactory:0x12d21d8)> [2] pry(main)> (1..8).collect do |i| [3] pry(main)* pool.future.create(i) [4] pry(main)* end.collect(&:value) => [ (0.9320213390604204+0.6495975396265634i), (0.2192850934232697+1.3652569533895615i), (0.9544363040629293+1.5916591822692894i), (1.9770264674909868+2.9614915840843354i), (3.2629217852664585+1.3563783097433732i), (3.4472746283258061+2.7785173492567807i), (6.3050303460818099+0.3322862800371151i), (4.1685797344882974+4.5646328695454597i), (7.4257865062947812+6.2157473844634782i) ] Sunday, October 20, 13
  • 35. Celluloid: Notifications typeof Observer Pattern Subscribe / Publish topics to be handled by actors without need to explicitly make them known Ideal for long-lived subscriptions, be careful with short-lived Sunday, October 20, 13
  • 36. Celluloid: Notifications require 'celluloid' class HardWorker include Celluloid, Celluloid::Notifications def work(factor) fibo = lambda do |x| return x if (0..1).include?(x) fibo[x - 1] + fibo[x - 2] end result = fibo.call(factor) publish('factorial_created', factor: factor, value: result) true end end Sunday, October 20, 13
  • 37. Celluloid: Notifications require 'celluloid' class LazyStudent include Celluloid, Celluloid::Notifications def on_creation(*args) data = args.last puts "Factorial of #{data[:factor]} is #{data[:value]}" end end Sunday, October 20, 13
  • 38. Celluloid: Notifications [1] pry(main)> student = LazyStudent.new => #<Celluloid::ActorProxy(LazyStudent:0xda16c8)> [2] pry(main)> student.subscribe('factorial_created', :on_creation) => #<Celluloid::Notifications::Subscriber:0x00000001ac20f0 @actor=#<Celluloid::ActorProxy(LazyStudent:0xda16c8)>, @method=:on_creation, @pattern="factorial_created"> [3] pry(main)> worker = HardWorker.new => #<Celluloid::ActorProxy(HardWorker:0xce6d64)> [4] pry(main)> fibo = [ 30, 25, 28, 34, 11, 5 ] => [30, 25, 28, 34, 11, 5] [5] pry(main)> fibo.each { |i| worker.async.work(i) } => [30, 25, 28, 34, 11, 5] Factorial of 30 is 832040 Factorial of 25 is 75025 Factorial of 28 is 317811 Factorial of 34 is 5702887 Factorial of 11 is 89 Factorial of 5 is 5 Sunday, October 20, 13
  • 39. Celluloid: and Rails? + Sunday, October 20, 13 = ?
  • 40. Celluloid: and Rails? Honestly? Forget it when using MRI. Rack not handle Fibers well Since Celluloid uses Fibers a lot, it may be a big problem Fibers in MRI 1.9 have 4kb of stack size, when exceeds Celluloid mad With apps that uses ActiveRecord, you will work hard due for concurrency issues related to DB connection management Sidekiq have a AR middleware to handle DB connections Some people runs Rails apps mounted in a Reel (Celluloid::IO webserver) using reel-rack, but only with MRI 2.0 because of Fiber stack size Sunday, October 20, 13
  • 41. Celluloid: and Rails? It sounds very bad for MRI, but if you use JRuby or Rubinius... No fear, but checks Celluloid Gotchas Github Page for your sanity :) In general, you can use it if: Precise Timing is not a requirement (fire and forget) Work is CPU bound Work can be parallelizable Sunday, October 20, 13
  • 42. Celluloid: and Rails? Here are some useful links talking about it: http://rubyrogues.com/088-rr-concurrency-andcelluloid-with-tony-arcieri/ https://groups.google.com/d/topic/celluloidruby/y5gSm2VjVJw https://github.com/celluloid/celluloid/wiki/ Gotchas Sunday, October 20, 13
  • 43. Celluloid: Fractal Demo Time to show! http://fractal.rubinius.salizzar.net Source code available on: https://github.com/salizzar/celluloid-fractal Sunday, October 20, 13
  • 44. Questions? Sunday, October 20, 13
  • 45. Thank you! Sunday, October 20, 13