Concurrency and Distributed Systems Using JRuby

746 views
667 views

Published on

Presented at JRubyConf EU August 2012

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
746
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
8
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Concurrency and Distributed Systems Using JRuby

  1. 1. CONCURRENCYAND DISTRIBUTEDSYSTEMS IN JRUBY @iconara
  2. 2. speakerdeck.com/u/iconara (real time!)
  3. 3. Theo / @iconara
  4. 4. Chief Architect at
  5. 5. big data with Ruby since 2009
  6. 6. MAKING THIS
  7. 7. INTO THIS
  8. 8. big data with Ruby since 2009
  9. 9. APRIL 2011 stuck
  10. 10. MAY 2011 let’s try JRuby
  11. 11. JUNE 2011wrapping Java libraries for fun and for profit
  12. 12. HOTBUNNIESnatural Ruby API for RabbitMQ
  13. 13. JULY 2011threads are better than processes
  14. 14. JRUBY
  15. 15. JRUBY
  16. 16. J.U.C(java.util.concurrent)
  17. 17. SHORTCUTJava Concurrency in Practice
  18. 18. EXECUTORS configurable thread pools, a betterabstraction than using Thread directly
  19. 19. thread_pool = Executors.new_fixed_thread_pool(16)all_the_things.each do |thing| thread_pool.submit do crunch_crunch_crunch(thing) endend
  20. 20. BLOCKING QUEUES easy producer/consumer patterns
  21. 21. J.U.C.ATOMIC optimistic locking primitives, compare-and-swap,thread safe non-blocking counters
  22. 22. MORE AWESOMES ConcurrentHashMap, CountDownLatch,ForkJoinPool, Google Guava, LMAX distruptor
  23. 23. TL; DRuse j.u.c and avoid mutable state and you’re safe, mostly
  24. 24. THREADS enough rope
  25. 25. AUGUST 2011threads are a mess, let’s try actors
  26. 26. ACTORSa higher level abstraction of concurrency
  27. 27. AKKAconcurrency library for Scala,famous actor implementation
  28. 28. MIKKAAkka wrapper for JRuby
  29. 29. DO ALL THE THINGS Programming Concurrency on the JVM
  30. 30. AUGUST 2011 async systems with no back pressuremechanism will die of memory starvation
  31. 31. INPUTAGGREGATION TIMER STATE PERSISTENCE OUTPUT
  32. 32. INPUT AGGREGATION TIMER! STATE PERSISTENCE OUTPUT
  33. 33. AUGUST 2011 actors are cool, but not really worth it when all the arrows in yourflow chart point in the same direction
  34. 34. DIVIDE DIVIDE DIVIDErun multiple independent, synchronous workers, tune to the number of CPUs & IO wait
  35. 35. SEPTEMBER 2011 blocking all the way down, back pressure all the way up
  36. 36. AUGUST 2011(short rant about MongoDB’s global write lock)
  37. 37. NEW SHINY TOYS for building new shiny products
  38. 38. CASSANDRAdistributed database, mostly magic
  39. 39. ColumnFamilyUpdater<String, String> updater = template.createUpdater("a key");updater.setString("domain", "www.datastax.com");updater.setLong("time", System.currentTimeMillis());try { template.update(updater);} catch (HectorException e) { // hurgh} CASSANDRA very verbose drivers
  40. 40. new_data = {:domain => www.datastax.com, :time => Time.now.to_i}column_family.update(a key, new_data) EURYDICE JRuby nails it
  41. 41. STORMstream processing framework written in Clojure
  42. 42. STORM(it does the arrows, you do the blobs)
  43. 43. REDSTORM Storm for JRuby
  44. 44. YOUR CODE DON’T DO THIS AT HOME KIDS! the key to building concurrent and distributedsystems is to separate what you want to do from the concurrent- and distributed-ness, and let someone else take care of that
  45. 45. JRUBY IS A SLIPPERY SLOPE TO JBOSS let there be no doubt about it
  46. 46. SCALING IS HARD go to Srdan’s talk tomorrow
  47. 47. KTHXBAI @iconara github.com/iconaraarchitecturalatrocities.com burtcorp.com

×