Clojure:
Functional
Concurrency for
the JVM
Howard M. Lewis Ship

Director of Open Source Technology
Formos Software Devel...
http://xkcd.com/297/




Clojure: The Language

                       © 2009 Formos Software Development
Rich Hickey




              © 2009 Formos Software Development
Code is Data


                 Quoted list of
'(1 2 3)         numbers




(biggest 5 42)                Function call

 ...
Read Eval Print Loop


         user=> (defn biggest
            "Find the maximum of two numbers"
            [x y]
     ...
There Is No Interpreter



                                                 Source Code

Repl Input                 Clojur...
Clojure Literals



           user=> 42
           42
           user=> "A Clojure String"
           "A Clojure String"
...
Clojure Literals


           user=> 5
           5
           user=> 5.001
           5.001
           user=> 22/7
      ...
Java Interop


factory.setNamespaceAware(true)               (.setNamespaceAware factory true)


new StringBuffer()       ...
Java Interop



frame = new JFrame();

frame.add(panel, BorderLayout.CENTER);
frame.add(greetButton, BorderLayout.SOUTH);
...
Clojure Collections: Lists




                            4
                                lst
user=> (def lst `(1 2 3))...
Clojure Collections: Vectors


          user=> (def v [:moe :larry :curly])
          #=(var user/v)
          user=> v
 ...
Clojure Collections: Map




 user=> (def m {:first-name "Howard" :last-name "Lewis Ship"})
 #=(var user/m)
 user=> m
 {:l...
Clojure Collections: Sets


 user=> (def s #{"Howard" "Suzanne" "Molly" "Jim"})
 #=(var user/s)
 user=> s
 #{"Howard" "Jim...
❝For alumni of other languages,
beginning to use Lisp may be like stepping
onto a skating rink for the first time. It’s
act...
Functional Programming

                  © 2009 Formos Software Development
No
Mutable
 State© 2009 Formos Software Development
No Side
Effects
     © 2009 Formos Software Development
First Class
Functions
         © 2009 Formos Software Development
Functional
Composition
         © 2009 Formos Software Development
Functional Programming in Java



public void saveOrUpdate(final Employee employee)
{
  HibernateCallback callback = new H...
Functional Java Collections

public interface Predicate<T>
{
  boolean accept(T value);
}

public static <T> Collection<T>...
Functional Clojure Collections

                              Function
            Anonymous        parameter
            ...
First Class Functions
     (filter #(not (.startsWith % ".")) names)




                           function as
          ...
Life without the for loop

public static int sum(int[] vals)
{
  int total = 0;
                                          ...
Life without the for loop

  public static String[] formatDoubles(double[] inputs)
  {
    String[] output = new String[in...
for: list comprehension

user=> (range 0 5)
(0 1 2 3 4)
user=> (for [x (range 0 10) :when (even? x)]
         x)
(0 2 4 6 ...
Laziness is
                                             a Virtue




user=>   (take   20 (iterate inc 1))
(1 2 3   4 5 6 ...
Laziness
           © 2009 Formos Software Development
Java: Data Encapsulated in Objects

              Person               Person                Person                Person
...
Clojure: Data in Transformable Collections
      :first-name    Howard            :first-name      Scott            :first-na...
❝Somehow the idea of reusability got
attached to object-oriented
programming in the 1980s, and no
amount of evidence to th...
Clojure Concurrency

                      © 2009 Formos Software Development
Solving Deadlocks:
Timeout & Retry




                     © 2009 Formos Software Development
Solving
Deadlocks:
Specific Lock
Order



               © 2009 Formos Software Development
Solving Deadlocks:
Coarse Locks




                     © 2009 Formos Software Development
Locks are
the Enemy
       © 2009 Formos Software Development
Clojure: Software Transactional Memory

(def savings (ref 1000.))
(def checking (ref 2000.))
(def mm (ref 7000.))

(defn t...
Retries: Transactions Are Speculative

(def savings (ref 1000.))
(def checking (ref 2000.))
(def mm (ref 7000.))

(defn tr...
No
Blocking
       © 2009 Formos Software Development
No Locks
       © 2009 Formos Software Development
Persistent
Collections
         © 2009 Formos Software Development
Managing Mutation
• What can change?


  • Reference types: atom, var, agent, ref


• When can they change?


• When are c...
Atoms
• Shared, Global


• Changes are atomic, synchronous, & non-blocking


• (swap!): Pass value to function yielding ne...
Vars — Per-Thread Mutables
• (def) sets global binding


• (binding) to set up a per-thread override


• (set!) if per-thr...
Interfacing with Java APIs

 (def *tokens*)

 (defn add-token
   [token]
   (set! *tokens* (conj *tokens* token)))

 (def ...
Everything's a Var!




                                              Function


      Namespace                         V...
Functions are stored in Vars


   user=> (defn say-hello [] (println "Hello"))
   #'user/say-hello
   user=> (say-hello)
 ...
Agents — Single Thread Writes


user=> (def savings (agent 1000.))
#=(var user/savings)
user=> (def checking (agent 2000.)...
Refs — Software Transactional Memory
 (def savings (ref 1000.))
 (def checking (ref 2000.))
 (def mm (ref 7000.))

 (defn ...
Concurrency Notes
• All reference types can have a validator
  function


• All refs can have a watcher: an agent notified
...
❝The key to performance is elegance,
not battalions of special cases.❞




Jon Bentley and Doug McIlroy
                  ...
Wrap Up

          © 2009 Formos Software Development
Clojure
• 1.0 release: May 4 2009


• Simple, regular syntax


• Improves on Lisp: vectors, maps, sets


• Fully integrate...
Stuart Halloway

                             Pragmatic Bookshelf




http://pragprog.com/titles/shcloj/programming-clojur...
http://jnb.ociweb.com/jnb/jnbMar2009.html




                                   © 2009 Formos Software Development
Object Oriented




                  © 2009 Formos Software Development
Functional




             © 2009 Formos Software Development
Picture Credits
   © 2007 Jon Fife
   http://flickr.com/photos/good-karma/577632972/

                                     ...
Upcoming SlideShare
Loading in...5
×

Clojure: Functional Concurrency for the JVM (presented at OSCON)

3,138

Published on

A high-level overview of the Clojure functional programming language for the JVM, as presented on Jul 23 2009 at OSCON.

Published in: Technology, Education
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,138
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
110
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Clojure: Functional Concurrency for the JVM (presented at OSCON)

  1. 1. Clojure: Functional Concurrency for the JVM Howard M. Lewis Ship Director of Open Source Technology Formos Software Development howard.lewis.ship@formos.com © 2009 Formos Software Development
  2. 2. http://xkcd.com/297/ Clojure: The Language © 2009 Formos Software Development
  3. 3. Rich Hickey © 2009 Formos Software Development
  4. 4. Code is Data Quoted list of '(1 2 3) numbers (biggest 5 42) Function call Function definition (defn biggest "Find the maximum of two numbers" [x y] (if (> x y) x y)) © 2009 Formos Software Development
  5. 5. Read Eval Print Loop user=> (defn biggest "Find the maximum of two numbers" [x y] (if (> x y) x y)) #=(var user/biggest) user=> (biggest 5 42) 42 user=> (doc biggest) ------------------------- user/biggest ([x y]) Find the maximum of two numbers nil user=> '(1 2 3) (1 2 3) user=> '(biggest 5 42) (biggest 5 42) user=> (first '(biggest 5 42)) biggest user=> (eval '(biggest 5 42)) 42 © 2009 Formos Software Development
  6. 6. There Is No Interpreter Source Code Repl Input Clojure User Classes Java Evaluator Compiler Clojure Source Files Java Libraries JVM Operating System © 2009 Formos Software Development
  7. 7. Clojure Literals user=> 42 42 user=> "A Clojure String" "A Clojure String" user=> nil nil user=> :balance :balance user=> true true user=> false false © 2009 Formos Software Development
  8. 8. Clojure Literals user=> 5 5 user=> 5.001 5.001 user=> 22/7 22/7 user=> (* 2 22/7) 44/7 user=> (* 100000 100000 100000) 1000000000000000 user=> (+ 5. 0.000000000000000001) 5.0 user=> (+ 5.0M 0.000000000000000001M) 5.000000000000000001M © 2009 Formos Software Development
  9. 9. Java Interop factory.setNamespaceAware(true) (.setNamespaceAware factory true) new StringBuffer() (new StringBuffer) (StringBuffer.) factory.newSAXParser().parse(src, handler) (.. factory newSAXParser (parse src handler)) myObject.ivar = "foo"; (set! (. myObject ivar) "foo") © 2009 Formos Software Development
  10. 10. Java Interop frame = new JFrame(); frame.add(panel, BorderLayout.CENTER); frame.add(greetButton, BorderLayout.SOUTH); frame.pack(); frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); frame.setVisible(true); (doto (JFrame.) (.add panel BorderLayout/CENTER) (.add greet-button BorderLayout/SOUTH) (.pack) (.setDefaultCloseOperation JFrame/EXIT_ON_CLOSE) (.setVisible true)) © 2009 Formos Software Development
  11. 11. Clojure Collections: Lists 4 lst user=> (def lst `(1 2 3)) #=(var user/lst) user=> lst (1 2 3) user=> (first lst) 1 1 user=> (rest lst) (2 3) user=> (conj lst 4) 2 (4 1 2 3) user=> (cons 4 lst) (4 1 2 3) 3 © 2009 Formos Software Development
  12. 12. Clojure Collections: Vectors user=> (def v [:moe :larry :curly]) #=(var user/v) user=> v [:moe :larry :curly] user=> (first v) :moe user=> (rest v) (:larry :curly) user=> (conj v :shemp) [:moe :larry :curly :shemp] user=> (cons :shemp v) (:shemp :moe :larry :curly) user=> v [:moe :larry :curly] user=> (v 1) :larry vector is a function of its indexes © 2009 Formos Software Development
  13. 13. Clojure Collections: Map user=> (def m {:first-name "Howard" :last-name "Lewis Ship"}) #=(var user/m) user=> m {:last-name "Lewis Ship", :first-name "Howard"} user=> (get m :last-name) "Lewis Ship" map is a user=> (m :last-name) function of "Lewis Ship" user=> (assoc m :company "Formos") its keys {:company "Formos", :last-name "Lewis Ship", :first-name "Howard"} user=> m {:last-name "Lewis Ship", :first-name "Howard"} user=> (:first-name m) "Howard" user=> (:ssn m) nil Keywords are functions, too! © 2009 Formos Software Development
  14. 14. Clojure Collections: Sets user=> (def s #{"Howard" "Suzanne" "Molly" "Jim"}) #=(var user/s) user=> s #{"Howard" "Jim" "Molly" "Suzanne"} user=> (contains? s "Howard") true user=> (contains? s "howard") false set is a user=> (s "Howard") function of "Howard" its elements user=> (s "Rhys") nil user=> (conj s "Howard") #{"Howard" "Jim" "Molly" "Suzanne"} user=> (conj s "Scott") #{"Howard" "Jim" "Molly" "Suzanne" "Scott"} © 2009 Formos Software Development
  15. 15. ❝For alumni of other languages, beginning to use Lisp may be like stepping onto a skating rink for the first time. It’s actually much easier to get around on ice than it is on dry land—if you use skates. Till then you will be left wondering what people see in this sport.❞ Paul Graham © 2009 Formos Software Development
  16. 16. Functional Programming © 2009 Formos Software Development
  17. 17. No Mutable State© 2009 Formos Software Development
  18. 18. No Side Effects © 2009 Formos Software Development
  19. 19. First Class Functions © 2009 Formos Software Development
  20. 20. Functional Composition © 2009 Formos Software Development
  21. 21. Functional Programming in Java public void saveOrUpdate(final Employee employee) { HibernateCallback callback = new HibernateCallback() { public Object doInHibernate(Session session) throws HibernateException,SQLException { session.saveOrUpdate(employee); return null; } }; hibernateTemplate.execute(callback); } Outer function controls the SwingUtilities.invokeLater(new Runnable() context: { public void run() • Thread { • Exception handling progressBar.setValue(progressBar.getValue() + 1); } • Parameters }); © 2009 Formos Software Development
  22. 22. Functional Java Collections public interface Predicate<T> { boolean accept(T value); } public static <T> Collection<T> filter(Predicate<T> pred, Collection<T> coll) { Collection<T> out = new ArrayList<T>(); for (T item : coll) { if (pred.accept(item)) out.add(item); } return out; } return CollectionUtils.filter(new Predicate<String>() { public boolean accept(String value) { return !value.startsWith("."); } }, names); © 2009 Formos Software Development
  23. 23. Functional Clojure Collections Function Anonymous parameter function (filter #(not (.startsWith % ".")) names) Member access form user=> (def names ["fred" "barney" ".hidden" "wilma"]) #=(var user/names) user=> (filter #(not (.startsWith % ".")) names) ("fred" "barney" "wilma") user=> (remove #(.startsWith % ".") names) ("fred" "barney" "wilma") user=> © 2009 Formos Software Development
  24. 24. First Class Functions (filter #(not (.startsWith % ".")) names) function as parameter to function (defn require-extension [ext] (fn [file-name] (= ext (last (split-string file-name "."))))) function as return value (defn filter-by-extension [ext coll] (filter (require-extension ext) coll)) composing functions © 2009 Formos Software Development
  25. 25. Life without the for loop public static int sum(int[] vals) { int total = 0; col for (int val : vals) total += val; return total; } x y (defn sum z [col] (reduce + 0 col)) ➠ 0 + col[0] + col[1] + col[2] ... © 2009 Formos Software Development
  26. 26. Life without the for loop public static String[] formatDoubles(double[] inputs) { String[] output = new String[inputs.length]; for (int i = 0; i < input.length; i++) output[i] = String.format("%9.2f", inputs[i]); return output; } (defn format-doubles f [col] (map #(format "%9.2f" %) col)) f Apply function to each item, forming f new seq user=> (format-doubles '(2.5 3.7 -22.7)) (" 2.50" " 3.70" " -22.70") © 2009 Formos Software Development
  27. 27. for: list comprehension user=> (range 0 5) (0 1 2 3 4) user=> (for [x (range 0 10) :when (even? x)] x) (0 2 4 6 8) user=> user=> (for [suit [:heart :spade :diamond :club] rank [:ace 2 3 4]] [suit rank]) ([:heart :ace] [:heart 2] [:heart 3] [:heart 4] [:spade :ace] [:spade 2] [:spade 3] [:spade 4] [:diamond :ace] [:diamond 2] [:diamond 3] [:diamond 4] [:club :ace] [:club 2] [:club 3] [:club 4]) user=> user=> (for [x (range 1 5) y (range 0 x)] [x y]) ([1 0] [2 0] [2 1] [3 0] [3 1] [3 2] [4 0] [4 1] [4 2] [4 3]) user=> © 2009 Formos Software Development
  28. 28. Laziness is a Virtue user=> (take 20 (iterate inc 1)) (1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20) user=> (take 20 (map * (iterate inc 1) (iterate inc 1))) (1 4 9 16 25 36 49 64 81 100 121 144 169 196 225 256 289 324 361 400) © 2009 Formos Software Development
  29. 29. Laziness © 2009 Formos Software Development
  30. 30. Java: Data Encapsulated in Objects Person Person Person Person firstName: "Howard" firstName: "Scott" firstName: "Molly" firstName: "David" lastName: "Lewis Ship" lastName: "Simon" lastName: "Newman" lastName: "Goldman" age: 42 age: 44 age: 29 age: 42 public double averageAge(Collection<Person> persons) { double total = 0.0; for (Person p : persons) total += p.getAge(); return total / persons.size(); } © 2009 Formos Software Development
  31. 31. Clojure: Data in Transformable Collections :first-name Howard :first-name Scott :first-name Molly { :last-name Lewis Ship } { :last-name Simon }{ :last-name Newman } :age 42 :age 44 :age 29 user=> persons [{:first-name "Howard", :last-name "Lewis Ship", :age 42} {:first-name "Scott", :last-name "Simon", :age 44} {:first-name "Molly", :last-name "Newman", :age 29}] user=> (map :age persons) (42 44 29) user=> (apply + (map :age persons)) 115 user=> (defn avg-age [coll] (/ (apply + (map :age coll)) (count coll))) (defn avg (defn avg-age [f coll] [coll] (/ (apply + (map f coll))) (count coll))) (avg :age coll)) (avg #(count (:last-name %)) persons) © 2009 Formos Software Development
  32. 32. ❝Somehow the idea of reusability got attached to object-oriented programming in the 1980s, and no amount of evidence to the contrary seems to be able to shake it free.❞ Paul Graham © 2009 Formos Software Development
  33. 33. Clojure Concurrency © 2009 Formos Software Development
  34. 34. Solving Deadlocks: Timeout & Retry © 2009 Formos Software Development
  35. 35. Solving Deadlocks: Specific Lock Order © 2009 Formos Software Development
  36. 36. Solving Deadlocks: Coarse Locks © 2009 Formos Software Development
  37. 37. Locks are the Enemy © 2009 Formos Software Development
  38. 38. Clojure: Software Transactional Memory (def savings (ref 1000.)) (def checking (ref 2000.)) (def mm (ref 7000.)) (defn transfer "Transaction to transfer money from one account to another." [from to amount] (dosync (alter from - amount) (alter to + amount))) (transfer checking savings 500.) (transfer mm checking 300.) - 500. + 500. - 300. + 300. Checking 1000. 500. 500. Savings 2000. 2500. 2800. 2800. MM 7000. 6700. 6700. © 2009 Formos Software Development
  39. 39. Retries: Transactions Are Speculative (def savings (ref 1000.)) (def checking (ref 2000.)) (def mm (ref 7000.)) (defn transfer "Transaction to transfer money from one account to another." [from to amount] (dosync (alter from - amount) (alter to + amount))) (transfer checking savings 500.) (transfer mm checking 300.) + 300. - 500. - 300. + 500. - 300. + 300. Checking 1000. 500. X 500. Savings 2000. 2500. 2300. 2500. 2800. 2800. MM 7000. 6700. 7000. 6700. 6700. © 2009 Formos Software Development
  40. 40. No Blocking © 2009 Formos Software Development
  41. 41. No Locks © 2009 Formos Software Development
  42. 42. Persistent Collections © 2009 Formos Software Development
  43. 43. Managing Mutation • What can change? • Reference types: atom, var, agent, ref • When can they change? • When are changes visible to other threads? © 2009 Formos Software Development
  44. 44. Atoms • Shared, Global • Changes are atomic, synchronous, & non-blocking • (swap!): Pass value to function yielding new value • (reset!): Force new value, regardless of existing value user=> (def queue (atom [])) #'user/queue user=> @queue [] user=> (swap! queue conj {:parse "http://www.clojure.org/"}) [{:parse "http://www.clojure.org/"}] user=> @queue [{:parse "http://www.clojure.org/"}] user=> (reset! queue []) [] user=> @queue [] user=> © 2009 Formos Software Development
  45. 45. Vars — Per-Thread Mutables • (def) sets global binding • (binding) to set up a per-thread override • (set!) if per-thread binding user=> (def x 1) #=(var user/x) user=> x 1 user=> (defn manipulate-x [] (binding [x 2] (printf "Local x is %d" x) (set! x 3) (printf "nLocal x is now %dn" x))) #=(var user/manipulate-x) user=> (.run (Thread. manipulate-x)) Local x is 2 Local x is now 3 nil user=> x 1 © 2009 Formos Software Development
  46. 46. Interfacing with Java APIs (def *tokens*) (defn add-token [token] (set! *tokens* (conj *tokens* token))) (def sax-handler (proxy [DefaultHandler] [] (startElement [uri local-name q-name attrs] (flush-text) (add-token …)) … )) (defn tokenize-xml [src] (binding [*tokens* []] (let [factory (SAXParserFactory/newInstance)] (.setNamespaceAware factory true) (.. factory newSAXParser (parse src sax-handler)) *tokens*))) © 2009 Formos Software Development
  47. 47. Everything's a Var! Function Namespace Var Value ... Symbol such as x or map © 2009 Formos Software Development
  48. 48. Functions are stored in Vars user=> (defn say-hello [] (println "Hello")) #'user/say-hello user=> (say-hello) Hello nil user=> (binding [say-hello #(println "Goodbye")] (say-hello)) Goodbye nil user=> (say-hello) Hello nil user=> © 2009 Formos Software Development
  49. 49. Agents — Single Thread Writes user=> (def savings (agent 1000.)) #=(var user/savings) user=> (def checking (agent 2000.)) #=(var user/checking) user=> @savings 1000 user=> @checking 2000 user=> (send savings - 300.) #<clojure.lang.Agent@c3233b> user=> (send checking + 300.) #<clojure.lang.Agent@67e5a7> user=> @savings 700 user=> @checking 2300 •Asynchonous •Single threaded •Non-transactional © 2009 Formos Software Development
  50. 50. Refs — Software Transactional Memory (def savings (ref 1000.)) (def checking (ref 2000.)) (def mm (ref 7000.)) (defn transfer "Transaction to transfer money from one account to another." [from to amount] (dosync (alter from - amount) (alter to + amount))) (transfer checking savings 500.) (transfer mm checking 300.) user=> @checking 2000 user=> @savings 1000 user=> (transfer checking savings 500.) 1500 user=> @checking 1500 user=> @savings 1500 user=> (ref-set savings 2000.) java.lang.IllegalStateException: No transaction running (NO_SOURCE_FILE:0) © 2009 Formos Software Development
  51. 51. Concurrency Notes • All reference types can have a validator function • All refs can have a watcher: an agent notified of changes • (send) inside (dosync) waits until successful completion • No guarantees when calling Java objects © 2009 Formos Software Development
  52. 52. ❝The key to performance is elegance, not battalions of special cases.❞ Jon Bentley and Doug McIlroy © 2009 Formos Software Development
  53. 53. Wrap Up © 2009 Formos Software Development
  54. 54. Clojure • 1.0 release: May 4 2009 • Simple, regular syntax • Improves on Lisp: vectors, maps, sets • Fully integrates with Java http://www.clojure.org • Impressive functional & concurrency support • Many features not covered here © 2009 Formos Software Development
  55. 55. Stuart Halloway Pragmatic Bookshelf http://pragprog.com/titles/shcloj/programming-clojure © 2009 Formos Software Development
  56. 56. http://jnb.ociweb.com/jnb/jnbMar2009.html © 2009 Formos Software Development
  57. 57. Object Oriented © 2009 Formos Software Development
  58. 58. Functional © 2009 Formos Software Development
  59. 59. Picture Credits © 2007 Jon Fife http://flickr.com/photos/good-karma/577632972/ © 2008 Marcin Wichary http://flickr.com/photos/mwichary/2222776430/ © 2007 James Manners http://flickr.com/photos/jmanners/443421045/ © 2008 Daniel Chan http://flickr.com/photos/chanchan222/2847443980/ © 2005 Jack Keene http://www.flickr.com/photos/whatknot/3118124/ © A. Lipson 2003 http://www.andrewlipson.com/escher/relativity.html © 2007 Alan Chia http://flickr.com/photos/seven13avenue/2080281038/ © 2007 Woodley Wonderworks http://flickr.com/photos/wwworks/2222523486/ © 2009 Formos Software Development
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×