Jobim: an ActorsLibrary for the ClojureProgramming Language         Antonio Garrote      María N. Moreno García
Clojure• New Lisp dialect• Hosted: JVM, CLR• Focus on concurrency: (STM, Agents,  futures, fork-join, java.util.concurrent...
Distributed concurrentapplications in Clojure?“Given the diversity, sophistication, maturity,interoperability,robustness e...
Distributed applications      on the JVM• JINI• Java Spaces• JMS• Terracota• RabbitMQ• ZeroMQ• Gearman
Problems• Different computational models• Hard to port code from one solution to  other• Friction with Clojure semantics
Extending Clojure• Computational model?• Suitable notation?• Underlying implementation?
Computational Model
Actors: components• Named channel (PID)      PID• Message box                    MBox                                     ...
Actors: mobile                processesPID3                                        PID1                 Execution         ...
Notation
Actors: minimal     interface(def *pid* (spawn f))(is (= *pid* (self)))(send! *pid* msg)(def msg (receive *pid*))
Selective Reception(let [p (promise)      pid (spawn           #(let [a (receive odd?)                  b (receive even?)]...
Implementation
Message Box• Lamina - Z. Tellman (https://github.com/  ztellman/lamina)• Transforms Clojure sequences into event-  driven ...
PIDs/Channel Names• Plain Java strings: easy to exchange• Globally unique identifiers for actors across  all the nodes in a...
Execution context• Threaded actors: - Java thread per actor - End of thread execution, releases     resources -   Threaded...
Execution context• Evented actors (Scala): - Actors context = closure + callback     functions -   Small number of reactor...
(spawn             #(loop []               (let [[from msg] (receive)]Threaded          (send! from msg)                  ...
react-loop               Callbacks+Contexts queuesEvents                                                  listen-once     ...
Distribution
3 problemsMessaging    Coordination   Serialization
Modular solution            Protocols: jobim.services.*Messaging         Coordination      Serialization                  ...
Coordination• Name service• Membership groups: presence• Distributed agreement: 2PC protocol
Coordination• Apache ZooKeeper plugin - Light-weight - Scalable - Small set of primitives to build    sophisticated coordi...
Coordination: name service - group  membership(nodes)(resolve-node-name node-name)(register-name name *pid*)(def *pid* (re...
Coordination: 2PC -group membership   (link *pid1* *pid2*)         Signal
Messaging• TCP plugin: Netty, Z.Tellman’s Aleph  [https://github.com/ztellman/aleph]• RabbitMQ plugin• ZeroMQ plugin
Serialization• Java Serialization plugin• JSON plugin• Kryo serialization library plugin (Yahoo S4)
Behaviours
Reusing distributed     components?• Encapsulate distributed patterns• Hide message passing logic• Building blocks for lar...
Behaviours• Supervisor• Generic Server• FSM• Event Manager / Event Handler• Generic TCP server
;; FSM Lock type(def-fsm Lock  (init [this code] [:locked {:so-far [] :code code}])  (next-transition [this state-name sta...
Demo
Future work• Benchmarking + Performance• Packaging / deployment / managing of  distributed apps• Missing functionality
Code+deps• https://github.com/antoniogarrote/jobim   [jobim-core "0.1.2-SNAPSHOT"] [jobim-rabbitmq "0.1.1-SNAPSHOT"] [jobi...
4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language
Upcoming SlideShare
Loading in …5
×

4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language

1,514 views
1,403 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,514
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

4th European Lisp Symposium: Jobim: an Actors Library for the Clojure Programming Language

  1. 1. Jobim: an ActorsLibrary for the ClojureProgramming Language Antonio Garrote María N. Moreno García
  2. 2. Clojure• New Lisp dialect• Hosted: JVM, CLR• Focus on concurrency: (STM, Agents, futures, fork-join, java.util.concurrent.*)
  3. 3. Distributed concurrentapplications in Clojure?“Given the diversity, sophistication, maturity,interoperability,robustness etc of these options, itsunlikely Im going to fiddle around with somelanguage-specific solution.” Rich Hickey
  4. 4. Distributed applications on the JVM• JINI• Java Spaces• JMS• Terracota• RabbitMQ• ZeroMQ• Gearman
  5. 5. Problems• Different computational models• Hard to port code from one solution to other• Friction with Clojure semantics
  6. 6. Extending Clojure• Computational model?• Suitable notation?• Underlying implementation?
  7. 7. Computational Model
  8. 8. Actors: components• Named channel (PID) PID• Message box MBox Execution Context• Execution contextComputation based on the exchange of messages
  9. 9. Actors: mobile processesPID3 PID1 Execution Execution MBox MBox Context Context Msg PID3 PID3 PID2 Execution MBox Context
  10. 10. Notation
  11. 11. Actors: minimal interface(def *pid* (spawn f))(is (= *pid* (self)))(send! *pid* msg)(def msg (receive *pid*))
  12. 12. Selective Reception(let [p (promise) pid (spawn #(let [a (receive odd?) b (receive even?)] (deliver p [a b])))] (send! pid 2) (send! pid 1) (is (= @p [1 2])))
  13. 13. Implementation
  14. 14. Message Box• Lamina - Z. Tellman (https://github.com/ ztellman/lamina)• Transforms Clojure sequences into event- driven channels• Synchronous and asynchronous interface
  15. 15. PIDs/Channel Names• Plain Java strings: easy to exchange• Globally unique identifiers for actors across all the nodes in a Jobim cluster• Generated by Jobim’s runtime• GUID node + process counter
  16. 16. Execution context• Threaded actors: - Java thread per actor - End of thread execution, releases resources - Threaded actors do not scale (~2000 threads per node max.)
  17. 17. Execution context• Evented actors (Scala): - Actors context = closure + callback functions - Small number of reactor threads execute all the evented threads
  18. 18. (spawn #(loop [] (let [[from msg] (receive)]Threaded (send! from msg) (recur)))) (spawn-evented #(react-loop [] (react [[from msg]]Evented (send! from msg) (react-recur))))
  19. 19. react-loop Callbacks+Contexts queuesEvents listen-once Reactor1 ... ReactorN publish react-recur Multiplexer react-future Messages thread- publish handler pool
  20. 20. Distribution
  21. 21. 3 problemsMessaging Coordination Serialization
  22. 22. Modular solution Protocols: jobim.services.*Messaging Coordination Serialization Plugins
  23. 23. Coordination• Name service• Membership groups: presence• Distributed agreement: 2PC protocol
  24. 24. Coordination• Apache ZooKeeper plugin - Light-weight - Scalable - Small set of primitives to build sophisticated coordination protocols
  25. 25. Coordination: name service - group membership(nodes)(resolve-node-name node-name)(register-name name *pid*)(def *pid* (resolve-name name))
  26. 26. Coordination: 2PC -group membership (link *pid1* *pid2*) Signal
  27. 27. Messaging• TCP plugin: Netty, Z.Tellman’s Aleph [https://github.com/ztellman/aleph]• RabbitMQ plugin• ZeroMQ plugin
  28. 28. Serialization• Java Serialization plugin• JSON plugin• Kryo serialization library plugin (Yahoo S4)
  29. 29. Behaviours
  30. 30. Reusing distributed components?• Encapsulate distributed patterns• Hide message passing logic• Building blocks for larger distributed systems• Built using Clojure protocols• Threaded and evented versions
  31. 31. Behaviours• Supervisor• Generic Server• FSM• Event Manager / Event Handler• Generic TCP server
  32. 32. ;; FSM Lock type(def-fsm Lock (init [this code] [:locked {:so-far [] :code code}]) (next-transition [this state-name state-data message] (let [[topic _] message] (condp = [state-name topic] [:locked :button] handle-button [:open :lock] handle-lock action-ignore))) (handle-info [this current-state current-data message] (do (cond-match [[?from :state] message] (send! from current-state)) (action-next-state current-state current-data))))
  33. 33. Demo
  34. 34. Future work• Benchmarking + Performance• Packaging / deployment / managing of distributed apps• Missing functionality
  35. 35. Code+deps• https://github.com/antoniogarrote/jobim [jobim-core "0.1.2-SNAPSHOT"] [jobim-rabbitmq "0.1.1-SNAPSHOT"] [jobim-zeromq "0.1.1-SNAPSHOT"] *

×