0
Bringing Concurrency
to Ruby
Charles Oliver Nutter	

@headius
Me, about 40 weeks of the year
Me, the other 12 weeks of the year
Concurrency
Parallelism?
Concurrency
• Two or more jobs	

• Making progress	

• Over a given time span
Parallelism
• Two or more computations	

• Executing at the same moment in time
Examples
• Thread APIs: concurrency	

• Actore APIs: concurrency	

• Native thread, process: parallelism	

• If the underl...
You Need Both
• Work that can split into concurrent jobs	

• Platform that runs those jobs in parallel	

• In an ideal wor...
Process-level
Concurrency
• Separate processes running concurrently	

• As parallel as OS/CPU can make them	

• Low risk d...
Thread-level
Concurrency
• Threads in-process running concurrently	

• As parallel as OS/CPU can make them	

• Higher risk...
Popular Platforms
Concurrency Parallelism GC Notes
MRI 1.8.7 ✔ ✘ Single thread, stop-
the-world
Large C core would
need mu...
Idle System
MRI 1.8.7, One Thread
MRI 1.8.7, Eight Threads
Timeslicing
Thread 1
Thread 2
Thread 3
Thread 4
Native thread
Native thread
Native thread
Native thread
“Green” or “virtua...
MRI 1.8.7, Eight Threads
MRI 1.9.3+, Eight Threads
GVL: GlobalVM Lock
Thread 1
Thread 2
Thread 3
Thread 4
CPU
CPU
CPU
CPU
In 1.9+, each thread gets its own native thread,	

...
GVL: GlobalVenue Lock
MRI 1.9.3+, Eight Threads
MRI 1.9.3+, Eight Threads
JRuby, One Thread
Why Do We See
Parallelism?
• Hotspot JVM has many background threads	

• GC with concurrent and parallel options	

• JIT t...
JRuby, One Thread
JRuby, Eight Threads
Time Matters Too
0
1.75
3.5
5.25
7
Time per iteration
MRI 1.8.7 MRI 1.9.3 JRuby
Nearly 10x	

faster than	

1.9.3
Rules of Concurrency
1. Don’t do it, if you don’t have to.	

2. If you must do it, don’t share data.	

3. If you must shar...
#1: Don’t
• Many problems won’t benefit	

• Explicitly sequential things, e.g	

• Bad code can get worse	

• Multiply perf,...
I’m Not Perfect
• Wrote a naive algorithm	

• Measured it taking N seconds	

• Wrote the concurrent version	

• Measured i...
Fix Single-thread First!Timeinseconds
0
5
10
15
20
big_list time
v1 v2 v3 v4
String slice instead of unpack/pack
Simpler l...
Timeinseconds
0
17.5
35
52.5
70
Processing 23M word file
Non-threaded Two threads Four threads
Before Conc Work
• Fix excessive allocation (and GC)	

• Fix algorithmic complexity	

• Test on the runtime you want to ta...
Concurrency won’t help code
that’s using up all hardware
resources.
#2: Don’t Share Data
• Process-level concurrency	

• …have to sync up eventually, though	

• Threads with their own data o...
#3: Immutable Data
• In other words…	

• Data can be shared	

• Threads can pass it around safely	

• Cross-thread view of...
Object#freeze
• Simplest mechanism for immutability	

• For read-only: make changes, freeze	

• Read-mostly: dup, change, ...
Immutable Data
Structure
• Designed to avoid visible mutation but still
have good performance characteristics	

• Copy-on-...
Persistent?
• Collection you have a reference to is
guaranteed never to change	

• Modifications return a new reference	

•...
Hamster
• Pure-Ruby persistent data structures	

• Set, List, Stack, Queue,Vector, Hash	

• Based on Clojure’s Ctrie colle...
person = Hamster.hash(!
:name => “Simon",!
:gender => :male)!
# => {:name => "Simon", :gender => :male}!
 !
person[:name]!...
Coming Soon
• Reimplementation by Smit Shah	

• Mostly “native” impl of Ctrie	

• Considerably better perf than Hamster	

...
Other Techniques
• Known-immutable data like Symbol, Fixnum	

• Mutate for a while, then freeze	

• Hand-off: if you pass ...
#4: Synchronize Mutation
• Trickiest to get right; usually best perf	

• Fully-immutable generates lots of garbage	

• Loc...
Locks
• Avoid concurrent operations	

• Read + write, in general	

• Many varieties: reentrant, read/write	

• Many implem...
Mutex
• Simplest form of lock	

• Acquire, do work, release	

• Not reentrant
semaphore = Mutex.new!
...!
a = Thread.new {...
ConditionVariable
• Release mutex temporarily	

• Signal others waiting on the mutex	

• …and be signaled	

• Similar to w...
resource = ConditionVariable.new!
 !
a = Thread.new {!
mutex.synchronize {!
# Thread 'a' now needs the resource!
resource....
Monitor
• Reentrancy	

• “try” acquire	

• Mix-in for convenience	

• Java synchronization = CondVar + Monitor
Monitor
require 'monitor'!
 !
lock = Monitor.new!
lock.synchronize do!
# exclusive access!
end
Monitor
require 'monitor'!
 !
class SynchronizedArray < Array!
 !
include MonitorMixin!
 !
alias :old_shift :shift!
 !
def...
Atomics
• Without locking…	

• …replace a value only if unchanged	

• …increment, decrement safely	

• Thread-safe code ca...
atomic
• Atomic operations for Ruby	

• https://github.com/headius/ruby-atomic
require 'atomic'!
 !
my_atomic = Atomic.new(0)!
my_atomic.value!
# => 0!
my_atomic.value = 1!
my_atomic.swap(2)!
# => 1!
m...
Specialized Collections
• thread_safe gem	

• Fully-synchronized Array and Hash	

• Atomic-based hash impl (“Cache”)	

• j...
Queues
• Thread-safe Queue and SizedQueue	

• Pipeline data to/from threads	

• Standard in all Ruby impls
thread_count = (ARGV[2] || 1).to_i!
queue = SizedQueue.new(thread_count * 4)!
!
word_file.each_line.each_slice(50) do |wor...
threads = thread_count.times.map do |i|!
Thread.new do!
while true!
words = queue.pop!
if words.nil? # terminating conditi...
Putting It All Together
• These are a lot of tools to sort out	

• Others have sorted them out for you
Celluloid
• Actor model implementation	

• OO/Ruby sensibilities	

• Normal classes, normal method calls	

• Async support...
class Sheen!
include Celluloid!
 !
def initialize(name)!
@name = name!
end!
 !
def set_status(status)!
@status = status!
e...
>> charlie = Sheen.new "Charlie Sheen"!
=> #<Celluloid::Actor(Sheen:0x00000100a312d0) @name="Char
>> charlie.set_status "w...
Sidekiq
• Simple, efficient background processing	

• Think Resque or DelayedJob but better	

• Normal-looking Ruby class i...
class HardWorker!
include Sidekiq::Worker!
 !
def perform(name, count)!
puts 'Doing hard work'!
end!
end!
!
...later, in a...
Concurrent Ruby
• Grab bag of concurrency patterns	

• Actor,Agent, Channel, Future, Promise,
ScheduledTask,TimerTask, Sup...
…all the examples I’ve shown you and more
Recap
• The future of Ruby is concurrent	

• The tools are there to help you	

• Let’s all help move Ruby forward
Thank you!
• Charles Oliver Nutter	

• headius@headius.com	

• @headius
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Bringing Concurrency to Ruby - RubyConf India 2014
Upcoming SlideShare
Loading in...5
×

Bringing Concurrency to Ruby - RubyConf India 2014

2,815

Published on

A talk on the why and how of high-concurrency Ruby.

Published in: Technology
0 Comments
21 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,815
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
45
Comments
0
Likes
21
Embeds 0
No embeds

No notes for slide

Transcript of "Bringing Concurrency to Ruby - RubyConf India 2014"

  1. 1. Bringing Concurrency to Ruby Charles Oliver Nutter @headius
  2. 2. Me, about 40 weeks of the year
  3. 3. Me, the other 12 weeks of the year
  4. 4. Concurrency
  5. 5. Parallelism?
  6. 6. Concurrency • Two or more jobs • Making progress • Over a given time span
  7. 7. Parallelism • Two or more computations • Executing at the same moment in time
  8. 8. Examples • Thread APIs: concurrency • Actore APIs: concurrency • Native thread, process: parallelism • If the underlying system supports it • SIMD, GPU, vector operations: parallelism
  9. 9. You Need Both • Work that can split into concurrent jobs • Platform that runs those jobs in parallel • In an ideal world, scales with job count • In our world, each job adds overhead
  10. 10. Process-level Concurrency • Separate processes running concurrently • As parallel as OS/CPU can make them • Low risk due to isolated memory space • High memory requirements • High communication overhead
  11. 11. Thread-level Concurrency • Threads in-process running concurrently • As parallel as OS/CPU can make them • Higher risk due to shared memory space • Lower memory requirements • Low communication overhead
  12. 12. Popular Platforms Concurrency Parallelism GC Notes MRI 1.8.7 ✔ ✘ Single thread, stop- the-world Large C core would need much work MRI 1.9+ ✔ ✘ Single thread, stop- the-world Few changes since 1.9.3 JRuby (JVM) ✔ ✔ Many concurrent and parallel options JVM is the “best” platform for conc Rubinius ✔ ✔ Single thread, stop- the-world, partial concurrent old gen Promising, but a long road ahead Topaz ✘ ✘ Single thread, stop- the-world Incomplete impl Node.js (V8) ✘ ✘ Single thread, stop- the-world No threads in JS CPython ✔ ✘ Reference-counting Reference counting kills parallelism Pypy ✔ ✘ Single thread, stop- the-world Exploring STM to enable concurrency
  13. 13. Idle System
  14. 14. MRI 1.8.7, One Thread
  15. 15. MRI 1.8.7, Eight Threads
  16. 16. Timeslicing Thread 1 Thread 2 Thread 3 Thread 4 Native thread Native thread Native thread Native thread “Green” or “virtual” or “userspace” threads share a single native thread.The CPU then schedules that thread on available CPUs. Time’s up Time’s up Time’s up
  17. 17. MRI 1.8.7, Eight Threads
  18. 18. MRI 1.9.3+, Eight Threads
  19. 19. GVL: GlobalVM Lock Thread 1 Thread 2 Thread 3 Thread 4 CPU CPU CPU CPU In 1.9+, each thread gets its own native thread, but a global lock prevents concurrent execution. Time slices are finer grained and variable, but threads still can’t run in parallel. Lock xfer CPU Lock xfer Lock xfer
  20. 20. GVL: GlobalVenue Lock
  21. 21. MRI 1.9.3+, Eight Threads
  22. 22. MRI 1.9.3+, Eight Threads
  23. 23. JRuby, One Thread
  24. 24. Why Do We See Parallelism? • Hotspot JVM has many background threads • GC with concurrent and parallel options • JIT threads • Signal handling • Monitoring and management
  25. 25. JRuby, One Thread
  26. 26. JRuby, Eight Threads
  27. 27. Time Matters Too 0 1.75 3.5 5.25 7 Time per iteration MRI 1.8.7 MRI 1.9.3 JRuby Nearly 10x faster than 1.9.3
  28. 28. Rules of Concurrency 1. Don’t do it, if you don’t have to. 2. If you must do it, don’t share data. 3. If you must share data, make it immutable. 4. If it must be mutable, coordinate all access.
  29. 29. #1: Don’t • Many problems won’t benefit • Explicitly sequential things, e.g • Bad code can get worse • Multiply perf, GC, alloc overhead by N • Fixes may not be easy (esp. in Ruby) • The risks can get tricky to address
  30. 30. I’m Not Perfect • Wrote a naive algorithm • Measured it taking N seconds • Wrote the concurrent version • Measured it taking roughly N seconds • Returned to original to optimize
  31. 31. Fix Single-thread First!Timeinseconds 0 5 10 15 20 big_list time v1 v2 v3 v4 String slice instead of unpack/pack Simpler loops Stream from file
  32. 32. Timeinseconds 0 17.5 35 52.5 70 Processing 23M word file Non-threaded Two threads Four threads
  33. 33. Before Conc Work • Fix excessive allocation (and GC) • Fix algorithmic complexity • Test on the runtime you want to target • If serial perf is still poor after optimization, the task, runtime, or system may not be appropriate for a concurrent version.
  34. 34. Concurrency won’t help code that’s using up all hardware resources.
  35. 35. #2: Don’t Share Data • Process-level concurrency • …have to sync up eventually, though • Threads with their own data objects • Rails request objects, e.g. • APIs with a “master” object, usually • Weakest form of concurrency
  36. 36. #3: Immutable Data • In other words… • Data can be shared • Threads can pass it around safely • Cross-thread view of data can’t mutate • Threads can’t see concurrent mutations as they happen, avoiding data races
  37. 37. Object#freeze • Simplest mechanism for immutability • For read-only: make changes, freeze • Read-mostly: dup, change, freeze, replace • Write-mostly: same, but O(n) complexity
  38. 38. Immutable Data Structure • Designed to avoid visible mutation but still have good performance characteristics • Copy-on-write is poor-man’s IDS • Better: persistent data structures like Ctrie http://en.wikipedia.org/wiki/Ctrie
  39. 39. Persistent? • Collection you have a reference to is guaranteed never to change • Modifications return a new reference • …and only duplicate affected part of trie
  40. 40. Hamster • Pure-Ruby persistent data structures • Set, List, Stack, Queue,Vector, Hash • Based on Clojure’s Ctrie collections • https://github.com/hamstergem/hamster
  41. 41. person = Hamster.hash(! :name => “Simon",! :gender => :male)! # => {:name => "Simon", :gender => :male}!  ! person[:name]! # => "Simon"! person.get(:gender)! # => :male!  ! friend = person.put(:name, "James")! # => {:name => "James", :gender => :male}! person! # => {:name => "Simon", :gender => :male}! friend[:name]! # => "James"! person[:name]! # => "Simon"
  42. 42. Coming Soon • Reimplementation by Smit Shah • Mostly “native” impl of Ctrie • Considerably better perf than Hamster • https://github.com/Who828/persistent_data_structures
  43. 43. Other Techniques • Known-immutable data like Symbol, Fixnum • Mutate for a while, then freeze • Hand-off: if you pass mutable data, assume you can’t mutate it anymore • Sometimes enforced by runtime, e.g. “thread-owned objects”
  44. 44. #4: Synchronize Mutation • Trickiest to get right; usually best perf • Fully-immutable generates lots of garbage • Locks, atomics, and specialized collections
  45. 45. Locks • Avoid concurrent operations • Read + write, in general • Many varieties: reentrant, read/write • Many implementations
  46. 46. Mutex • Simplest form of lock • Acquire, do work, release • Not reentrant semaphore = Mutex.new! ...! a = Thread.new {! semaphore.synchronize {! # access shared resource! }! }
  47. 47. ConditionVariable • Release mutex temporarily • Signal others waiting on the mutex • …and be signaled • Similar to wait/notify/notifyAll in Java
  48. 48. resource = ConditionVariable.new!  ! a = Thread.new {! mutex.synchronize {! # Thread 'a' now needs the resource! resource.wait(mutex)! # 'a' can now have the resource! }! }!  ! b = Thread.new {! mutex.synchronize {! # Thread 'b' has finished using the resource! resource.signal! }! }!
  49. 49. Monitor • Reentrancy • “try” acquire • Mix-in for convenience • Java synchronization = CondVar + Monitor
  50. 50. Monitor require 'monitor'!  ! lock = Monitor.new! lock.synchronize do! # exclusive access! end
  51. 51. Monitor require 'monitor'!  ! class SynchronizedArray < Array!  ! include MonitorMixin!  ! alias :old_shift :shift!  ! def shift(n=1)! self.synchronize do! self.old_shift(n)! end! end! ...
  52. 52. Atomics • Without locking… • …replace a value only if unchanged • …increment, decrement safely • Thread-safe code can use atomics instead of locks, usually with better performance
  53. 53. atomic • Atomic operations for Ruby • https://github.com/headius/ruby-atomic
  54. 54. require 'atomic'!  ! my_atomic = Atomic.new(0)! my_atomic.value! # => 0! my_atomic.value = 1! my_atomic.swap(2)! # => 1! my_atomic.compare_and_swap(2, 3)! # => true, updated to 3! my_atomic.compare_and_swap(2, 3)! # => false, current is not 2
  55. 55. Specialized Collections • thread_safe gem • Fully-synchronized Array and Hash • Atomic-based hash impl (“Cache”) • java.util.concurrent • Numerous tools for concurrency
  56. 56. Queues • Thread-safe Queue and SizedQueue • Pipeline data to/from threads • Standard in all Ruby impls
  57. 57. thread_count = (ARGV[2] || 1).to_i! queue = SizedQueue.new(thread_count * 4)! ! word_file.each_line.each_slice(50) do |words|! queue << words! end! queue << nil # terminating condition
  58. 58. threads = thread_count.times.map do |i|! Thread.new do! while true! words = queue.pop! if words.nil? # terminating condition! queue.shutdown! break! end! words.each do |word|! # analyze the word
  59. 59. Putting It All Together • These are a lot of tools to sort out • Others have sorted them out for you
  60. 60. Celluloid • Actor model implementation • OO/Ruby sensibilities • Normal classes, normal method calls • Async support • Growing ecosystem • Celluloid-IO and DCell (distributed actors) • https://github.com/celluloid/celluloid
  61. 61. class Sheen! include Celluloid!  ! def initialize(name)! @name = name! end!  ! def set_status(status)! @status = status! end!  ! def report! "#{@name} is #{@status}"! end! end
  62. 62. >> charlie = Sheen.new "Charlie Sheen"! => #<Celluloid::Actor(Sheen:0x00000100a312d0) @name="Char >> charlie.set_status "winning!"! => "winning!"! >> charlie.report! => "Charlie Sheen is winning!"! >> charlie.async.set_status "asynchronously winning!"! => nil! >> charlie.report! => "Charlie Sheen is asynchronously winning!"
  63. 63. Sidekiq • Simple, efficient background processing • Think Resque or DelayedJob but better • Normal-looking Ruby class is the job • Simple call to start it running in background • http://mperham.github.io/sidekiq/
  64. 64. class HardWorker! include Sidekiq::Worker!  ! def perform(name, count)! puts 'Doing hard work'! end! end! ! ...later, in a controller...!  ! HardWorker.perform_async('bob', 5)
  65. 65. Concurrent Ruby • Grab bag of concurrency patterns • Actor,Agent, Channel, Future, Promise, ScheduledTask,TimerTask, Supervisor • Thread pools, executors, timeouts, conditions, latches, atomics • May grow into a central lib for conc stuff • https://github.com/jdantonio/concurrent-ruby
  66. 66. …all the examples I’ve shown you and more
  67. 67. Recap • The future of Ruby is concurrent • The tools are there to help you • Let’s all help move Ruby forward
  68. 68. Thank you! • Charles Oliver Nutter • headius@headius.com • @headius
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×