:clojure/south - São Paulo, 2019
From Java to parallel Clojure
Leonardo Borges

@leonardo_borges

www.leonardoborges.com

www.recordpoint.com
A bit about me
• Head of Engineering at RecordPoint

• Founder of the Sydney Clojure User Group

• Open-source contributor 

• Author of bouncer and imminent

• Author of Clojure Reactive Programming
A bit about me
• 2nd edition is out now!
What about you?
What we’ll talk about
• Being a Lisp on the JVM

• Functional Programming strengths

• Concurrency and parallelism
Java has come a long way…
• ForkJoin;

• Lambda Expressions;

• Method References;

• CompletableFutures;

• JShell;

• Reactive Streams / Stream API;

• …and more!
So why would you invest in
Clojure?
Here’s a few reasons
Here’s a few reasons
Classes and Interfaces

• Minimize the accessibility of
classes and members

• In public classes, use
accessor methods, not public
fields

• Minimize mutability
Immutability
(def ages [10 20 30])
(def names ["Leo" "Liv" "Bruce"])
Immutability
(def ages [10 20 30])
(def names ["Leo" "Liv" "Bruce"])
(defrecord Person [fname age])
(def leo (->Person "Leo" 10))
;; {:fname "Leo", :age 10}
Immutability
(def ages [10 20 30])
(def names ["Leo" "Liv" "Bruce"])
(defrecord Person [fname age])
(map (fn [fname age]
(->Person fname age)) names ages)
;; ({:fname "Leo", :age 10}
;; {:fname "Liv", :age 20}
;; {:fname "Bruce", :age 30})
Immutability
(def ages [10 20 30])
(def names ["Leo" "Liv" "Bruce"])
(defrecord Person [fname age])
(map ->Person names ages)
;; ({:fname "Leo", :age 10}
;; {:fname "Liv", :age 20}
;; {:fname "Bruce", :age 30})
What if we want to add new
ages and names?
Adding new elements to vectors
(def new-ages (conj ages 40)) ;; [10 20 30 40]
ages ;; [10 20 30]
(def new-names (conj names "Gwen")) ;; ["Leo" "Liv"
"Bruce" "Gwen"]
names ;; ["Leo" "Liv" "Bruce"]
Is that slow?
Persistent data structures
(def xs ‘(0 1 2))
(def ys ‘(3 4 5))
Persistent data structures
(def xs ‘(0 1 2))
(def ys ‘(3 4 5))
(def zs (concat xs ys))
Persistent data structures
(def xs ‘(0 1 2))
(def ys ‘(3 4 5))
(def zs (concat xs ys))
Here’s a few reasons
Lambdas and Streams

• Prefer lambdas to anonymous
classes

• Prefer method references to
lambdas

• Favor the use of standard
functional interfaces
Prefer lambdas to anonymous classes
Collections.sort(names, new Comparator<String>() {
@Override
public int compare(String o1, String o2) {
return o1.compareTo(o2);
}
});
Prefer lambdas to anonymous classes
Collections.sort(names, (o1, o2) -> o1.compareTo(o2));
Collections.sort(names, new Comparator<String>() {
@Override
public int compare(String o1, String o2) {
return o1.compareTo(o2);
}
});
Prefer method references to lambdas
Collections.sort(names, String::compareTo);
Collections.sort(names, (o1, o2) -> o1.compareTo(o2));
The Clojure way
(sort names) ;; ("Bruce" "Leo" “Liv”)
(sort-by #(count %) names) ;; ("Bruce" "Leo" “Liv")
(sort-by count names) ;; ("Bruce" "Leo" "Liv")
Anonymous functions
(sort-by (fn [s] (count s)) names) ;; ("Bruce" "Leo" "Liv")
(sort-by #(count %) names) ;; ("Bruce" "Leo" "Liv")
(sort-by count names) ;; ("Bruce" "Leo" "Liv")
Concurrency

• Synchronize access to shared
mutable data

• Avoid excessive
synchronization
Here’s a few reasons
Synchronise access to shared mutable data
class StopThread {
private static boolean stopRequested;
private static synchronized void requestStop() {
stopRequested = true;
}
private static synchronized boolean stopRequested() {
return stopRequested;
}
public static void example3() throws InterruptedException {
Thread backgroundThread = new Thread(() -> {
while (!stopRequested())
System.out.println("going....");
});
backgroundThread.start();
TimeUnit.SECONDS.sleep(1);
requestStop();
}
}
Synchronise access to shared mutable references
(def stop-requested (atom false))
(defn request-stop! []
(reset! stop-requested true))
(defn stop-requested? []
@stop-requested)
(defn example-3 []
(let [background-thread (java.lang.Thread. (fn []
(while (not (stop-requested?))
(prn "going..."))))]
(.start background-thread)
(Thread/sleep 1000)
(request-stop!)))
What about multiple shared
references?
STM - Software Transactional Memory
(def account-a (ref 100))
(def account-b (ref 250))
(defn transfer [amount from to]
(dosync
(alter from #(- % amount))
(alter to #(+ % amount))))
(transfer 25 account-a account-b)
@account-a ;; 75
@account-b ;; 275
Clojure makes it easy to do the
right thing
Let’s revisit example-3
(defn example-3 []
(let [background-thread (java.lang.Thread. (fn []
(while (not @stop-requested)
(prn "going..."))))]
(.start background-thread)
(Thread/sleep 1000)
(request-stop!)))
(defn example-3[]
(future
(while (not @stop-requested)
(prn "going...")))
(Thread/sleep 1000)
(request-stop!))
Concurrency with futures
(def doubler (partial * 2))
(defn service-a [n]
(future
(Thread/sleep 1000)
n))
(defn service-b [n]
(future
(Thread/sleep 1000)
(Math/pow n 2)))
(defn service-c [n]
(future
(Thread/sleep 1000)
(Math/pow n 3)))
(defn service-d [n]
(future
(Thread/sleep 1000)
(Math/pow n 4)))
(let [doubled (doubler @(service-a 10))]
(+ @(service-b doubled)
@(service-c doubled)
@(service-d doubled)))
;; Elapsed time: 4013.746558 msecs
(let [a (service-a 10)
doubled (doubler @a)
b (service-b doubled)
c (service-c doubled)
d (service-d doubled)]
(+ @b @c @d))
Concurrency with futures
(def doubler (partial * 2))
(defn service-a [n]
(future
(Thread/sleep 1000)
n))
(defn service-b [n]
(future
(Thread/sleep 1000)
(Math/pow n 2)))
(defn service-c [n]
(future
(Thread/sleep 1000)
(Math/pow n 3)))
(defn service-d [n]
(future
(Thread/sleep 1000)
(Math/pow n 4)))
(let [doubled (doubler @(service-a 10))]
(+ @(service-b doubled)
@(service-c doubled)
@(service-d doubled)))
;; Elapsed time: 4013.746558 msecs
(let [a (service-a 10)
doubled (doubler @a)
b (service-b doubled)
c (service-c doubled)
d (service-d doubled)]
(+ @b @c @d))
Blocks main thread!
Concurrency with CompletableFutures
static Integer doubler(Integer n) {
return 2 * n;
}
static CompletableFuture<Integer> serviceA(Integer n) {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
return n;
});
}
static CompletableFuture<Integer> serviceB(Integer n) {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
return Double.valueOf(Math.pow(n, 2)).intValue();
});
}
static CompletableFuture<Integer> serviceC(Integer n) {
return CompletableFuture.supplyAsync(() -> {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
return Double.valueOf(Math.pow(n, 3)).intValue();
});
}
Concurrency with CompletableFutures
final CompletableFuture<Integer> doubled = serviceA(10).thenApply(CompletableFutures::doubler);
final CompletableFuture<Integer> resultB = doubled.thenCompose(CompletableFutures::serviceB);
final CompletableFuture<Integer> resultC = doubled.thenCompose(CompletableFutures::serviceC);
CompletableFuture<Void> allFutures = CompletableFuture.allOf(resultB, resultC);
allFutures.whenComplete((v, ex) -> {
try {
System.out.println("Result: " + resultB.get() + " - " + resultC.get());
} catch (Exception e) {
}
});
Concurrency with CompletableFutures
final CompletableFuture<Integer> doubled = serviceA(10).thenApply(CompletableFutures::doubler);
final CompletableFuture<Integer> resultB = doubled.thenCompose(CompletableFutures::serviceB);
final CompletableFuture<Integer> resultC = doubled.thenCompose(CompletableFutures::serviceC);
CompletableFuture<Void> allFutures = CompletableFuture.allOf(resultB, resultC);
allFutures.whenComplete((v, ex) -> {
try {
System.out.println("Result: " + resultB.get() + " - " + resultC.get());
} catch (Exception e) {
}
});
What about Clojure?
What if?
;; Wouldn't it be great to be able to write:
(let [doubled (doubler (service-a 10))
b (service-b doubled)
c (service-c doubled)
d (service-d doubled)]
;; then once that's all done concurrently...
(+ b c d))
Concurrency with imminent
(require '[imminent.core :as i])
(defn service-a [n]
(i/future
(Thread/sleep 1000)
n))
(defn service-b [n]
(i/future
(Thread/sleep 1000)
(Math/pow n 2)))
(defn service-c [n]
(i/future
(Thread/sleep 1000)
(Math/pow n 3)))
(defn service-d [n]
(i/future
(Thread/sleep 1000)
(Math/pow n 4)))
Concurrency with imminent
(let [doubled (i/map (service-a 10) doubler)
b (i/bind doubled service-b)
c (i/bind doubled service-c)
d (i/bind doubled service-d)
result (i/sequence [b c d])]
(i/map result
(fn [[b c d]]
(+ b c d))))
;; Elapsed time: 2025.446899 msecs
Concurrency with imminent
(let [doubled (i/map (service-a 10) doubler)
b (i/bind doubled service-b)
c (i/bind doubled service-c)
d (i/bind doubled service-d)
result (i/sequence [b c d])]
(i/map result
(fn [[b c d]]
(+ b c d))))
;; Elapsed time: 2025.446899 msecs
Concurrency with imminent
(defn f-doubler [n]
(i/const-future (* n 2)))
(i/mdo [a (service-a 10)
doubled (f-doubler a)
b (service-b doubled)
c (service-c doubled)
d (service-d doubled)]
(i/return (+ b c d)))
Concurrency with imminent
(bind
(service-a 10)
(fn* ([a]
(bind
(f-doubler a)
(fn* ([doubled]
(bind
(service-b doubled)
(fn* ([b]
(bind
(service-c doubled) (fn* ([c]
(bind
(service-d doubled)
(fn* ([d]
(i/return (+ b c d))))))))))))))))))
Concurrency with imminent
(bind
(service-a 10)
(fn* ([a]
(bind
(f-doubler a)
(fn* ([doubled]
(bind
(service-b doubled)
(fn* ([b]
(bind
(service-c doubled) (fn* ([c]
(bind
(service-d doubled)
(fn* ([d]
(i/return (+ b c d))))))))))))))))))
;; Elapsed time: 4017.578654 msecs
Concurrency with imminent
(def a+ (i/alift +))
(i/mdo [a (service-a 10)
doubled (f-doubler a)]
(a+ (service-b doubled)
(service-c doubled)
(service-d doubled)))
;; Elapsed time: 2010.171729 msecs
Concurrency with imminent
(def a+ (i/alift +))
(i/mdo [a (service-a 10)
doubled (f-doubler a)]
(a+ (service-b doubled)
(service-c doubled)
(service-d doubled)))
;; Elapsed time: 2010.171729 msecs
What’s with map, bind and
alift?
The algebra of library design
i/map => Functor
i/bind => Monad
i/alift => Applicative
References
• Clojure Reactive Programming - http://bit.ly/cljRp
• Imminent - http://bit.ly/immi-clj
• The Algebra of Library Design - http://bit.ly/2HBBJwJ
• Purely Functional Data Structures - https://amzn.to/2zGZizS
• Java 8 CompletableFuture - http://bit.ly/j8Future
• Java 8 Streams - http://bit.ly/j8stream
• Category Theory - http://amzn.to/1NfL08U
Thank you!
Leonardo Borges

@leonardo_borges

www.leonardoborges.com

www.recordpoint.com
:clojure/south - São Paulo, 2019

From Java to Parellel Clojure - Clojure South 2019

  • 1.
    :clojure/south - SãoPaulo, 2019 From Java to parallel Clojure Leonardo Borges @leonardo_borges www.leonardoborges.com www.recordpoint.com
  • 2.
    A bit aboutme • Head of Engineering at RecordPoint • Founder of the Sydney Clojure User Group • Open-source contributor • Author of bouncer and imminent • Author of Clojure Reactive Programming
  • 3.
    A bit aboutme • 2nd edition is out now!
  • 4.
  • 5.
    What we’ll talkabout • Being a Lisp on the JVM • Functional Programming strengths • Concurrency and parallelism
  • 6.
    Java has comea long way… • ForkJoin; • Lambda Expressions; • Method References; • CompletableFutures; • JShell; • Reactive Streams / Stream API; • …and more!
  • 7.
    So why wouldyou invest in Clojure?
  • 8.
  • 9.
    Here’s a fewreasons Classes and Interfaces • Minimize the accessibility of classes and members • In public classes, use accessor methods, not public fields • Minimize mutability
  • 10.
    Immutability (def ages [1020 30]) (def names ["Leo" "Liv" "Bruce"])
  • 11.
    Immutability (def ages [1020 30]) (def names ["Leo" "Liv" "Bruce"]) (defrecord Person [fname age]) (def leo (->Person "Leo" 10)) ;; {:fname "Leo", :age 10}
  • 12.
    Immutability (def ages [1020 30]) (def names ["Leo" "Liv" "Bruce"]) (defrecord Person [fname age]) (map (fn [fname age] (->Person fname age)) names ages) ;; ({:fname "Leo", :age 10} ;; {:fname "Liv", :age 20} ;; {:fname "Bruce", :age 30})
  • 13.
    Immutability (def ages [1020 30]) (def names ["Leo" "Liv" "Bruce"]) (defrecord Person [fname age]) (map ->Person names ages) ;; ({:fname "Leo", :age 10} ;; {:fname "Liv", :age 20} ;; {:fname "Bruce", :age 30})
  • 14.
    What if wewant to add new ages and names?
  • 15.
    Adding new elementsto vectors (def new-ages (conj ages 40)) ;; [10 20 30 40] ages ;; [10 20 30] (def new-names (conj names "Gwen")) ;; ["Leo" "Liv" "Bruce" "Gwen"] names ;; ["Leo" "Liv" "Bruce"]
  • 16.
  • 17.
    Persistent data structures (defxs ‘(0 1 2)) (def ys ‘(3 4 5))
  • 18.
    Persistent data structures (defxs ‘(0 1 2)) (def ys ‘(3 4 5)) (def zs (concat xs ys))
  • 19.
    Persistent data structures (defxs ‘(0 1 2)) (def ys ‘(3 4 5)) (def zs (concat xs ys))
  • 20.
    Here’s a fewreasons Lambdas and Streams • Prefer lambdas to anonymous classes • Prefer method references to lambdas • Favor the use of standard functional interfaces
  • 21.
    Prefer lambdas toanonymous classes Collections.sort(names, new Comparator<String>() { @Override public int compare(String o1, String o2) { return o1.compareTo(o2); } });
  • 22.
    Prefer lambdas toanonymous classes Collections.sort(names, (o1, o2) -> o1.compareTo(o2)); Collections.sort(names, new Comparator<String>() { @Override public int compare(String o1, String o2) { return o1.compareTo(o2); } });
  • 23.
    Prefer method referencesto lambdas Collections.sort(names, String::compareTo); Collections.sort(names, (o1, o2) -> o1.compareTo(o2));
  • 24.
    The Clojure way (sortnames) ;; ("Bruce" "Leo" “Liv”) (sort-by #(count %) names) ;; ("Bruce" "Leo" “Liv") (sort-by count names) ;; ("Bruce" "Leo" "Liv")
  • 25.
    Anonymous functions (sort-by (fn[s] (count s)) names) ;; ("Bruce" "Leo" "Liv") (sort-by #(count %) names) ;; ("Bruce" "Leo" "Liv") (sort-by count names) ;; ("Bruce" "Leo" "Liv")
  • 26.
    Concurrency • Synchronize accessto shared mutable data • Avoid excessive synchronization Here’s a few reasons
  • 27.
    Synchronise access toshared mutable data class StopThread { private static boolean stopRequested; private static synchronized void requestStop() { stopRequested = true; } private static synchronized boolean stopRequested() { return stopRequested; } public static void example3() throws InterruptedException { Thread backgroundThread = new Thread(() -> { while (!stopRequested()) System.out.println("going...."); }); backgroundThread.start(); TimeUnit.SECONDS.sleep(1); requestStop(); } }
  • 28.
    Synchronise access toshared mutable references (def stop-requested (atom false)) (defn request-stop! [] (reset! stop-requested true)) (defn stop-requested? [] @stop-requested) (defn example-3 [] (let [background-thread (java.lang.Thread. (fn [] (while (not (stop-requested?)) (prn "going..."))))] (.start background-thread) (Thread/sleep 1000) (request-stop!)))
  • 29.
    What about multipleshared references?
  • 30.
    STM - SoftwareTransactional Memory (def account-a (ref 100)) (def account-b (ref 250)) (defn transfer [amount from to] (dosync (alter from #(- % amount)) (alter to #(+ % amount)))) (transfer 25 account-a account-b) @account-a ;; 75 @account-b ;; 275
  • 31.
    Clojure makes iteasy to do the right thing
  • 32.
  • 33.
    (defn example-3 [] (let[background-thread (java.lang.Thread. (fn [] (while (not @stop-requested) (prn "going..."))))] (.start background-thread) (Thread/sleep 1000) (request-stop!)))
  • 34.
    (defn example-3[] (future (while (not@stop-requested) (prn "going..."))) (Thread/sleep 1000) (request-stop!))
  • 35.
    Concurrency with futures (defdoubler (partial * 2)) (defn service-a [n] (future (Thread/sleep 1000) n)) (defn service-b [n] (future (Thread/sleep 1000) (Math/pow n 2))) (defn service-c [n] (future (Thread/sleep 1000) (Math/pow n 3))) (defn service-d [n] (future (Thread/sleep 1000) (Math/pow n 4))) (let [doubled (doubler @(service-a 10))] (+ @(service-b doubled) @(service-c doubled) @(service-d doubled))) ;; Elapsed time: 4013.746558 msecs (let [a (service-a 10) doubled (doubler @a) b (service-b doubled) c (service-c doubled) d (service-d doubled)] (+ @b @c @d))
  • 36.
    Concurrency with futures (defdoubler (partial * 2)) (defn service-a [n] (future (Thread/sleep 1000) n)) (defn service-b [n] (future (Thread/sleep 1000) (Math/pow n 2))) (defn service-c [n] (future (Thread/sleep 1000) (Math/pow n 3))) (defn service-d [n] (future (Thread/sleep 1000) (Math/pow n 4))) (let [doubled (doubler @(service-a 10))] (+ @(service-b doubled) @(service-c doubled) @(service-d doubled))) ;; Elapsed time: 4013.746558 msecs (let [a (service-a 10) doubled (doubler @a) b (service-b doubled) c (service-c doubled) d (service-d doubled)] (+ @b @c @d)) Blocks main thread!
  • 37.
    Concurrency with CompletableFutures staticInteger doubler(Integer n) { return 2 * n; } static CompletableFuture<Integer> serviceA(Integer n) { return CompletableFuture.supplyAsync(() -> { try { Thread.sleep(1000); } catch (InterruptedException e) { } return n; }); } static CompletableFuture<Integer> serviceB(Integer n) { return CompletableFuture.supplyAsync(() -> { try { Thread.sleep(1000); } catch (InterruptedException e) { } return Double.valueOf(Math.pow(n, 2)).intValue(); }); } static CompletableFuture<Integer> serviceC(Integer n) { return CompletableFuture.supplyAsync(() -> { try { Thread.sleep(1000); } catch (InterruptedException e) { } return Double.valueOf(Math.pow(n, 3)).intValue(); }); }
  • 38.
    Concurrency with CompletableFutures finalCompletableFuture<Integer> doubled = serviceA(10).thenApply(CompletableFutures::doubler); final CompletableFuture<Integer> resultB = doubled.thenCompose(CompletableFutures::serviceB); final CompletableFuture<Integer> resultC = doubled.thenCompose(CompletableFutures::serviceC); CompletableFuture<Void> allFutures = CompletableFuture.allOf(resultB, resultC); allFutures.whenComplete((v, ex) -> { try { System.out.println("Result: " + resultB.get() + " - " + resultC.get()); } catch (Exception e) { } });
  • 39.
    Concurrency with CompletableFutures finalCompletableFuture<Integer> doubled = serviceA(10).thenApply(CompletableFutures::doubler); final CompletableFuture<Integer> resultB = doubled.thenCompose(CompletableFutures::serviceB); final CompletableFuture<Integer> resultC = doubled.thenCompose(CompletableFutures::serviceC); CompletableFuture<Void> allFutures = CompletableFuture.allOf(resultB, resultC); allFutures.whenComplete((v, ex) -> { try { System.out.println("Result: " + resultB.get() + " - " + resultC.get()); } catch (Exception e) { } });
  • 40.
  • 41.
    What if? ;; Wouldn'tit be great to be able to write: (let [doubled (doubler (service-a 10)) b (service-b doubled) c (service-c doubled) d (service-d doubled)] ;; then once that's all done concurrently... (+ b c d))
  • 42.
    Concurrency with imminent (require'[imminent.core :as i]) (defn service-a [n] (i/future (Thread/sleep 1000) n)) (defn service-b [n] (i/future (Thread/sleep 1000) (Math/pow n 2))) (defn service-c [n] (i/future (Thread/sleep 1000) (Math/pow n 3))) (defn service-d [n] (i/future (Thread/sleep 1000) (Math/pow n 4)))
  • 43.
    Concurrency with imminent (let[doubled (i/map (service-a 10) doubler) b (i/bind doubled service-b) c (i/bind doubled service-c) d (i/bind doubled service-d) result (i/sequence [b c d])] (i/map result (fn [[b c d]] (+ b c d)))) ;; Elapsed time: 2025.446899 msecs
  • 44.
    Concurrency with imminent (let[doubled (i/map (service-a 10) doubler) b (i/bind doubled service-b) c (i/bind doubled service-c) d (i/bind doubled service-d) result (i/sequence [b c d])] (i/map result (fn [[b c d]] (+ b c d)))) ;; Elapsed time: 2025.446899 msecs
  • 45.
    Concurrency with imminent (defnf-doubler [n] (i/const-future (* n 2))) (i/mdo [a (service-a 10) doubled (f-doubler a) b (service-b doubled) c (service-c doubled) d (service-d doubled)] (i/return (+ b c d)))
  • 46.
    Concurrency with imminent (bind (service-a10) (fn* ([a] (bind (f-doubler a) (fn* ([doubled] (bind (service-b doubled) (fn* ([b] (bind (service-c doubled) (fn* ([c] (bind (service-d doubled) (fn* ([d] (i/return (+ b c d))))))))))))))))))
  • 47.
    Concurrency with imminent (bind (service-a10) (fn* ([a] (bind (f-doubler a) (fn* ([doubled] (bind (service-b doubled) (fn* ([b] (bind (service-c doubled) (fn* ([c] (bind (service-d doubled) (fn* ([d] (i/return (+ b c d)))))))))))))))))) ;; Elapsed time: 4017.578654 msecs
  • 48.
    Concurrency with imminent (defa+ (i/alift +)) (i/mdo [a (service-a 10) doubled (f-doubler a)] (a+ (service-b doubled) (service-c doubled) (service-d doubled))) ;; Elapsed time: 2010.171729 msecs
  • 49.
    Concurrency with imminent (defa+ (i/alift +)) (i/mdo [a (service-a 10) doubled (f-doubler a)] (a+ (service-b doubled) (service-c doubled) (service-d doubled))) ;; Elapsed time: 2010.171729 msecs
  • 50.
    What’s with map,bind and alift?
  • 51.
    The algebra oflibrary design i/map => Functor i/bind => Monad i/alift => Applicative
  • 52.
    References • Clojure ReactiveProgramming - http://bit.ly/cljRp • Imminent - http://bit.ly/immi-clj • The Algebra of Library Design - http://bit.ly/2HBBJwJ • Purely Functional Data Structures - https://amzn.to/2zGZizS • Java 8 CompletableFuture - http://bit.ly/j8Future • Java 8 Streams - http://bit.ly/j8stream • Category Theory - http://amzn.to/1NfL08U
  • 53.