Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
CONCURRENCYAND DISTRIBUTEDSYSTEMS IN JRUBY       @iconara
speakerdeck.com/u/iconara          (real time!)
Theo / @iconara
Chief Architect at
big data with Ruby since 2009
MAKING THIS
INTO THIS
big data with Ruby since 2009
APRIL 2011   stuck
MAY 2011 let’s try JRuby
JUNE 2011wrapping Java libraries for fun and for profit
HOTBUNNIESnatural Ruby API for RabbitMQ
JULY 2011threads are better than processes
JRUBY
JRUBY
J.U.C(java.util.concurrent)
SHORTCUTJava Concurrency in Practice
EXECUTORS configurable thread pools, a betterabstraction than using Thread directly
thread_pool = Executors.new_fixed_thread_pool(16)all_the_things.each do |thing|  thread_pool.submit do    crunch_crunch_cr...
BLOCKING QUEUES  easy producer/consumer patterns
J.U.C.ATOMIC   optimistic locking primitives,       compare-and-swap,thread safe non-blocking counters
MORE AWESOMES   ConcurrentHashMap, CountDownLatch,ForkJoinPool, Google Guava, LMAX distruptor
TL; DRuse j.u.c and avoid mutable state     and you’re safe, mostly
THREADS enough rope
AUGUST 2011threads are a mess, let’s try actors
ACTORSa higher level abstraction of concurrency
AKKAconcurrency library for Scala,famous actor implementation
MIKKAAkka wrapper for JRuby
DO ALL THE THINGS Programming Concurrency on the JVM
AUGUST 2011 async systems with no back pressuremechanism will die of memory starvation
INPUTAGGREGATION      TIMER   STATE      PERSISTENCE                OUTPUT
INPUT    AGGREGATION      TIMER!      STATE      PERSISTENCE                    OUTPUT
AUGUST 2011   actors are cool, but not really worth it when all the arrows in yourflow chart point in the same direction
DIVIDE DIVIDE DIVIDErun multiple independent, synchronous workers,     tune to the number of CPUs & IO wait
SEPTEMBER 2011   blocking all the way down,  back pressure all the way up
AUGUST 2011(short rant about MongoDB’s global write lock)
NEW SHINY TOYS for building new shiny products
CASSANDRAdistributed database, mostly magic
ColumnFamilyUpdater<String, String> updater = template.createUpdater("a key");updater.setString("domain", "www.datastax.co...
new_data = {:domain => www.datastax.com, :time => Time.now.to_i}column_family.update(a key, new_data)         EURYDICE    ...
STORMstream processing framework written in Clojure
STORM(it does the arrows, you do the blobs)
REDSTORM Storm for JRuby
YOUR CODE       DON’T DO  THIS AT HOME KIDS!   the key to building concurrent and distributedsystems is to separate what y...
JRUBY IS A SLIPPERY  SLOPE TO JBOSS    let there be no doubt about it
SCALING IS HARD  go to Srdan’s talk tomorrow
KTHXBAI        @iconara   github.com/iconaraarchitecturalatrocities.com       burtcorp.com
Upcoming SlideShare
Loading in …5
×

Concurrency and Distributed Systems Using JRuby

889 views

Published on

Presented at JRubyConf EU August 2012

Published in: Technology
  • Be the first to comment

Concurrency and Distributed Systems Using JRuby

  1. 1. CONCURRENCYAND DISTRIBUTEDSYSTEMS IN JRUBY @iconara
  2. 2. speakerdeck.com/u/iconara (real time!)
  3. 3. Theo / @iconara
  4. 4. Chief Architect at
  5. 5. big data with Ruby since 2009
  6. 6. MAKING THIS
  7. 7. INTO THIS
  8. 8. big data with Ruby since 2009
  9. 9. APRIL 2011 stuck
  10. 10. MAY 2011 let’s try JRuby
  11. 11. JUNE 2011wrapping Java libraries for fun and for profit
  12. 12. HOTBUNNIESnatural Ruby API for RabbitMQ
  13. 13. JULY 2011threads are better than processes
  14. 14. JRUBY
  15. 15. JRUBY
  16. 16. J.U.C(java.util.concurrent)
  17. 17. SHORTCUTJava Concurrency in Practice
  18. 18. EXECUTORS configurable thread pools, a betterabstraction than using Thread directly
  19. 19. thread_pool = Executors.new_fixed_thread_pool(16)all_the_things.each do |thing| thread_pool.submit do crunch_crunch_crunch(thing) endend
  20. 20. BLOCKING QUEUES easy producer/consumer patterns
  21. 21. J.U.C.ATOMIC optimistic locking primitives, compare-and-swap,thread safe non-blocking counters
  22. 22. MORE AWESOMES ConcurrentHashMap, CountDownLatch,ForkJoinPool, Google Guava, LMAX distruptor
  23. 23. TL; DRuse j.u.c and avoid mutable state and you’re safe, mostly
  24. 24. THREADS enough rope
  25. 25. AUGUST 2011threads are a mess, let’s try actors
  26. 26. ACTORSa higher level abstraction of concurrency
  27. 27. AKKAconcurrency library for Scala,famous actor implementation
  28. 28. MIKKAAkka wrapper for JRuby
  29. 29. DO ALL THE THINGS Programming Concurrency on the JVM
  30. 30. AUGUST 2011 async systems with no back pressuremechanism will die of memory starvation
  31. 31. INPUTAGGREGATION TIMER STATE PERSISTENCE OUTPUT
  32. 32. INPUT AGGREGATION TIMER! STATE PERSISTENCE OUTPUT
  33. 33. AUGUST 2011 actors are cool, but not really worth it when all the arrows in yourflow chart point in the same direction
  34. 34. DIVIDE DIVIDE DIVIDErun multiple independent, synchronous workers, tune to the number of CPUs & IO wait
  35. 35. SEPTEMBER 2011 blocking all the way down, back pressure all the way up
  36. 36. AUGUST 2011(short rant about MongoDB’s global write lock)
  37. 37. NEW SHINY TOYS for building new shiny products
  38. 38. CASSANDRAdistributed database, mostly magic
  39. 39. ColumnFamilyUpdater<String, String> updater = template.createUpdater("a key");updater.setString("domain", "www.datastax.com");updater.setLong("time", System.currentTimeMillis());try { template.update(updater);} catch (HectorException e) { // hurgh} CASSANDRA very verbose drivers
  40. 40. new_data = {:domain => www.datastax.com, :time => Time.now.to_i}column_family.update(a key, new_data) EURYDICE JRuby nails it
  41. 41. STORMstream processing framework written in Clojure
  42. 42. STORM(it does the arrows, you do the blobs)
  43. 43. REDSTORM Storm for JRuby
  44. 44. YOUR CODE DON’T DO THIS AT HOME KIDS! the key to building concurrent and distributedsystems is to separate what you want to do from the concurrent- and distributed-ness, and let someone else take care of that
  45. 45. JRUBY IS A SLIPPERY SLOPE TO JBOSS let there be no doubt about it
  46. 46. SCALING IS HARD go to Srdan’s talk tomorrow
  47. 47. KTHXBAI @iconara github.com/iconaraarchitecturalatrocities.com burtcorp.com

×