Concurrency with JRuby and the JVM


Published on

Actors using the Akka library in JRuby in, along with some patterns of use and techniques to help manage concurrency including java.util.concurrent, Futures, and software transactional memory.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • \n
  • actors will be meat of presentation\n
  • Concurrency is a popular topic lately\n
  • \n
  • \n
  • \n
  • cr\n
  • \n
  • \n
  • Need to know where to add locks. Use cases might change.\nHard to know what threads will be accessing\n
  • Need to know where to add locks. Use cases might change.\nHard to know what threads will be accessing\nDifferent dimension\n
  • IO you have lock ineffient\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Started out doing Java development\nCorporate\nUnderstandable we don’t want to go back\nBeen there done that\n
  • Reaction is to Java as a language, not the platform\n
  • \n
  • mix evented and threaded code\n
  • \n
  • many of the libraries use this instead of lower level threads\n
  • abstract away from thread creation\n
  • \n
  • Once you abstract away operations, need a way to join threads\nCountdown latch - allow threads to wait for set of operations to complete\nSet number of operations, thread mark off, and you can wait\nReadWriteLock - separate out readers from writers\n
  • synchronized in more intelligent manner\n
  • \n
  • Top - jruby imports. Can have in a module somewhere\nAtomicInteger - using. Don’t have to synchronize\nLatch - once you abstract away threads, don’t have thread.join\n
  • \n
  • \n
  • \n
  • * popularized by erlang\n* adopted by other frameworks such as Celluloid / Akka etc.\n* Original model was from 1970s\n
  • Thanks to Jesse\n
  • \n
  • State encapsulated in these different silos which are actors\nCommunicate asynchronously via their mailboxes\nDoesn’t have to be class oriented\nSimplified diagram\n
  • * A lot of the concepts taken from Erlang and can apply throughout\n* Celluloid is object based - uses fibers to suspend and enable actors\n* Akka and rubinius - message processing based - reactor loop\n
  • \n
  • \n
  • Message processing \n - create other actors \n - send messages to other actors\n - change state\n
  • - asynch\n
  • \n
  • \n
  • Isolates actor from specific instance. \nCan restart and fail and messages are still there and address is the same\n
  • \n
  • Issues with raw threads - exception handling\n
  • Issues with raw threads - exception handling\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • patterns that I like - \n
  • Another pattern I like - concept of failure zones\nIndependent failure and recovery\n
  • \n
  • \n
  • Once you send a message, don’t change it\nOnly give up actor reference never your internal class\nDon’t send behavior\n\n
  • Celluloid also has an ActorRef method\nAvoid these when language doesn’t enforce this\n
  • \n
  • \n
  • Typed actors - similar to celluloid\n
  • The network is reliable.\nLatency is zero.\nBandwidth is infinite.\nThe network is secure.\nTopology doesn't change.\nThere is one administrator.\nTransport cost is zero.\nThe network is homogeneous.\nprefer more explicit style (erlang) vs object style\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Note- this is patched code for Mikka. Can be nicer.\nNice thing is there is no blocking! \nCheck if a future is ready\nget first completed\n
  • every time you do an operation it creates a brand new object\n
  • Atomic\nConsistency\nIsolatation\n
  • \n
  • \n
  • Reaction is to Java as a language, not the platform\n
  • Had to use RC1\nNo side effects - you can’t use Array without clone\n
  • \n
  • \n
  • advantages and disadvantages - try them out and see\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Concurrency with JRuby and the JVM

    1. 1. Concurrency using JRuby and the JVM Portland Ruby Brigade October 2, 2012 Alex Kira @AlexKira
    2. 2. What we will cover• Background• Survey of concurrency techniques with JRuby / JVM • java.util.concurrent • Actors (in more detail) • Futures • STM
    3. 3. Concurrency• Number of CPU cores going up• We would like to take advantage of it
    4. 4. Let’s start with a basic example
    5. 5. counter = 5000threads = (1..5).collect do do 1000.times do counter -= 1 end endendthreads.each {|t| t.join }puts counter Not 0 !
    6. 6. Well, let’s add a mutex
    7. 7. semaphore = Mutex.newcounter = 5000threads = (1..5).collect do do 1000.times do semaphore.synchronize do counter -= 1 end end endendthreads.each {|t| t.join }puts counter This time it’s 0
    8. 8. That seemed to work. What’s theproblem?
    9. 9. OK, think about this...
    10. 10. In an OO based program, we use to classes toencapsulate data Class State Methods Class Class Class State State State Methods Methods Methods Class State Methods
    11. 11. Threads, however, clobber your encapsulation Class State Methods Thread 1 Thread 3 Class Class Class State State State Methods Methods Methods Thread 2 Class State Methods
    12. 12. Hard to verify correctnessHard to deal with lock contentionHard to get right lock granularityHard to reason aboutDeadlock, starvation, livelockexception handling
    13. 13. Need more tools we can use
    14. 14. When deciding, we should look at:Concurrency frameworks and toolsThread safe libraries available
    15. 15. MRI• GIL - Not true multicore (but could be enough for IO heavy tasks)• Many gems are written using EventMachine - mixing with threads
    16. 16. Sometimes can feel like:
    17. 17. What about the JVM?
    18. 18. May bring back some Memories <Boilerplate Code> <XML Hell> <Week long IDE Setup> <EJB> <Over-engineering>
    19. 19. Maybe we can look past that
    20. 20. Other languages are using JVM successfully (Clojure, Scala) What can we learn from them? Can we leverage their libraries?
    21. 21. JVM is solid and has many threadsafe libraries and concurrencyoptions availableRuby is an awesome language, whynot consider it with the JVM?
    22. 22. Also Other Options Out There• Rubinius• Maglev• But for this presentation we will look at the JVM...
    23. 23. java.util.concurrentLibrary of higher level concurrency primitives built on top of threads
    24. 24. juc - Thread Management• Threads are not reusable - typically we end up managing our own thread pools at a certain scale (tasks can outnumber threads)• Executor framework abstracts away task execution
    25. 25. juc - Thread Management• ExecutorService • Few different flavors available • FixedThreadPool, CachedThreadPool, ScheduledThreadPool, SingleThread• ForkJoin • Dynamically manage thread pool based on resources • Threads attempt to steal subtasks
    26. 26. juc - Locks• Going beyond a mutex and thread.join• Synchronization primitives: • CountDownLatch, CyclicBarrier • ReentrantLock, ReadWriteLock
    27. 27. juc - Collections• Efficient data structures that be shared by multiple threads • ConcurrentHashMap • CopyOnWriteArrayList • CopyOnWriteArraySet • AtomicInteger, AtomicBoolean, AtomicReference • (‘atomic’ gem based on this)
    28. 28. juc - Queues• How can we share data and synchronize workflow between threads ?• BlockingQueue - queue and dequeue operations can trigger blocking depending on flavor being used• ArrayBlockingQueue, DelayQueue, LinkedBlockingQueue, PriorityBlockingQueue, SynchronousQueue
    29. 29. counter = = Executors.new_fixed_thread_pool(5)latch = do executor.submit do 1000.times do counter.add_and_get(-1) end latch.count_down endendlatch.awaitputs counter 0 againexecutor.shutdown
    30. 30. jucGood to know about. Many of the javalibraries use the java.util.concurrent package
    31. 31. Shared Mutable StateWe still need to be aware of state that can beaccessed concurrently throughout yourobjects
    32. 32. What if we can eliminatelocking and not have to deal with shared state?
    33. 33. Actor Model• Isolate state, • Don’t allow anyone to access same state concurrently• Communicate state via messages• No longer need explicit locking
    34. 34. Formal Definition of Actors • Originated in1973 • Actor is a unit satisfying: • Processing • Storage • CommunicationResource:
    35. 35. Formal Definition of Actors • When an actor receives message, it can: • Create actors • Send messages to actors with address (sends are asynchronous) • Designate what to do with next message • Process one message at a timeResource:
    36. 36. Actor DiagramActor Boundaries Actor Boundaries Thread 1 Thread 2 Messages Actor Actor Class Class State State Methods Messages Methods Class Class State State Methods Methods
    37. 37. Actor Model Libraries (Ruby / JRuby) • Celluloid (Object based) • Rubinius actors • AkkaTip: Putting aside implementation choice, general concepts can be applied across libraries
    38. 38. We will be using Akka as an example•• Scala / Java API• Fully featured and highly configurable• Has been really solid, currently at V2• Explicit message processing like Erlang
    39. 39. Mikka•• We will be using with Mikka wrapper gem• (It works, but you may have to get hands dirty with Akka API at times)
    40. 40. Basic Actor Usage with Mikka# define an actorclass MyActor < Mikka::Actor def receive(message) # act on message # typically based on message type endend
    41. 41. Basic Actor Usage with Mikka# Create an actor system to use# We can have multiple isolated actor systemsactor_system = Mikka.create_actor_system(system)# create an actoractor = actor_system.actor_of(Mikka::Props[MyActor], OptionalActorName)# send it a message - Asynchronousactor << :a_messageactor <<“do some work”)
    42. 42. Actor - Addresses in Akka• An ActorRef is a Proxy for the object• Can’t get to internal state (more info later)• Isolates actors from each other• If actor restarts • Address stays the same • Messages are still there
    43. 43. Actor - Addresses in Akka• Optionally name actors for lookup (or pass them around as references)• Can lookup / address via tree structure• Can also put routers behind an address • RoundRobin / LeastFull
    44. 44. ActorRefs Restart Actor ActorRef Actor Router ActorActorRef Actor
    45. 45. Erlang:Let It Crash
    46. 46. Fault Tolerance• Problem: How do you know if your threads die?• How do you communicate these errors?
    47. 47. Fault Tolerance• Erlang Model: Don’t program defensively, let your actors crash• Have a supervisor keep track of it - separates error handling from business logic
    48. 48. Fault Tolerance• Supervisor watches actor and decides what to do with failure scenarios: • Restart it (with fresh state). • Stop • Resume • Escalate failure to its supervisor
    49. 49. Supervisors• In Akka, the Actor that creates children becomes their supervisor• Specifies their restart / failure strategy• Can also “watch” actors to get termination message
    50. 50. Hierarchical Supervision in Akka Supervisor A /user/Supervisor WorkerA A /user/Supervisor/Worker A A Child1 Child2/user/Supervisor/Worker/Child1 /user/Supervisor/Worker/Child2 Relative Lookup: ../Child1
    51. 51. Supervisors• Supervision strategy: • AllForOne - if a child crashes, take down all the other children • OneForOne - if a child crashes, only take down the specific actor • Specify duration - let it restart 5 times in 10 minutes, otherwise it terminates
    52. 52. Supervision Diagram 5 times in 1 Minute A 10 times in 20 seconds * Restart * Restart * Resume * Resume * Escalate * Escalate * Stop * Stop A AAll For One One For One A A A A A A
    53. 53. Fault Tolerance - Patterns• Actor is restarted with initial state on crash• Error Kernel - identify valuable state • Keep core simple • Layer actors around core (Onion Layer) • For risky actions, use another actor
    54. 54. Fault Tolerance - patterns• Identify failure zones• Within zones can contain or escalate failure within zones• Cascading failure
    55. 55. Failure Zones Zone 3 A Zone 2 A A Zone 1A A A A A A
    56. 56. Actor - pitfalls• Languages like Erlang enforce certain guarantees with language level features• In Ruby this has to be by convention
    57. 57. Actor - pitfalls• Pitfalls to watch when using Ruby: • Mutating messages • Leaking underlying actor reference • Leaking actor state via function closure • Try not to block in an actor
    58. 58. # Pitfalls in Akka (also applies to Celluloid)class DangerousActor < Mikka::Actor attr_reader :local_state def receive(message) message.balance -= 10 # Mutate message # leak internal actor ref # (In Mikka, for actor reference use get_self) response = response.on_success = lambda { @local_state = "bam" # leak state via closure } wait_for_something(message) # blocking call another_actor << response endend
    59. 59. Actors vs Threads• Actor != Thread• Many actors can share a single thread• Can scale up better than using threads• Abstract specific configuration and tuning options from code
    60. 60. Actor Model - Intricacies• Know your message guarantees: • Once or at most once?• Message ordering• Mailbox durability
    61. 61. Other Akka Actor Features• Scheduler - schedule future messages to actors• Typed actors - ActiveObjects• FSM - model actors as FSM• become - change reactor dynamically• EventBus - pub/sub across actors
    62. 62. Distributed Actors• Actor model also applies well to distributed computing• If all we have is an actor reference, they can be remote as well as local• Beware of - “Eight fallacies of distributed computing”
    63. 63. Distributed Akka• Supports remote actors• Dynamo style clustering coming in V2.1 • Partitioning / Handoff • Leader election • Gossip protocol with Vector clocks
    64. 64. Downside to Actors?• Different programming model & thought process• Can’t get snapshot of whole system • State of system = actor state + messages• Can be harder to test
    65. 65. Even more tools we can use
    66. 66. Brief survey of these• Futures• Immutable data structures• STM - software transactional memory
    67. 67. Futures• Object representing a result that is yet to be computed. (Like an IOU).• Can end up with result or exception• Composable futures - multiple operations can be chained together without blocking• Great for interacting and composing multiple services together - non blocking result Resource:
    68. 68. @system = Mikka.create_actor_system(System)futures = (0..10).collect do |i| Mikka::Future.future(@system.dispatcher) do sleep(rand(10) + 5) "future #{i}" endendfutures ={ |f|{ |v| " #{v} modified"} }reduced = Mikka::Future.reduce(futures, @system.dispatcher) do |r, v| r+vendputs "waiting..."puts Await.result(reduced, Mikka::Duration["1 minute"])============================================ Output:waiting... future 0 modified future 1 modified......
    69. 69. Immutable Data Structures• Can be safely shared and know that another thread will not change your copy• Hamster gem •
    70. 70. Software Transactional Memory (STM) • Manage identity via software transaction, similar to database transactions • ACI(D) • Retry functional piece of code if conflictResource: Rich-Hickey
    71. 71. STM • Optimistic locking • If code has side effects they will be executed multiple times on retry • Should have pure functional codeResource: Rich-Hickey Resource:
    72. 72. Clojure STM • Ref - manages a reference • Ref.deref, Ref.set • Functional - Ref.alter and Ref.commute • To change a Ref, have to be inside a transactionResource: Rich-Hickey Resource:
    73. 73. An example that uses Hamster and Clojure STM
    74. 74. john_books = of the Flies, Brave New World))larry_books = Great Gatsby, The Catcher in the Rye))jill_books = Farm))def borrow_book(description, lender_list, borrower_list, book) LockingTransaction.run_in_transaction do puts "Starting tx #{description}" raise "Book not available " if !lender_list.deref.include?(book) # remove book from lender lender_list.set(lender_list.deref.delete(book)) sleep(1) # add a sleep to mess up tx # add book to borrower borrower_list.set(borrower_list.deref.add(book)) endend
    75. 75. Successful Transaction (with retries)t1 = do borrow_book("John to larry", john_books, larry_books, Lord of the Flies)endt2 = do borrow_book("John to jill", john_books, jill_books, Brave New World)endt1.joint2.join======================================================================Output:Starting tx John to larryStarting tx John to jillStarting tx John to jillStarting tx John to larryStarting tx John to larryStarting tx John to larryJohns books: []Larrys books: ["The Catcher in the Rye", "The Great Gatsby", "Lord of the Flies"]Jills books: ["Brave New World", "Animal Farm"]
    76. 76. Failed Transactiont1 = do borrow_book("John to larry", john_books, larry_books, Lord of the Flies)endt2 = do borrow_book("John to jill", john_books, jill_books, Lord of the Flies)endt1.joint2.join======================================================================Output:Starting tx from john to jillStarting tx from john to larryStarting tx from john to jillStarting tx from john to jillStarting tx from john to jillStarting tx from john to jillStarting tx from john to jillBook not availableJohns books: ["Brave New World"]Larrys books: ["The Catcher in the Rye", "Lord of the Flies", "The Great Gatsby"]Jills books: ["Animal Farm"]
    77. 77. Phew... We covered:• Survey of concurrency techniques with the JVM • java.util.concurrent • Actors • Futures • STM
    78. 78. Combining Techniques• Can combine these techniques • Futures can be used standalone or with actors (actor response can be a future) • Can use STM or Akka Transactors for system consensus • java.util.concurrent for lower level functionality or Akka extensions
    79. 79. Takeaways:Concurrency is HardWe need more tools to helpEvaluate and choose the right tool forthe situationThese general strategies can beapplicable across libraries and languages
    80. 80. Resources• Book: Programming Concurrency on the JVM•••• STM -•
    81. 81. Image Credits
    82. 82. Thank you! Alex Kira @AlexKira