(defrecord Assistant [name id]) (updatePersonalInfo ) Manager: (defrecord Manager [name id employees]) (raise ) (extend-type Assistant Employee (roles [this] "assistant")) (extend-type Manager Employee (roles [this] (str "manager of " (count employees)))) 85 The Expression Problem 86 The Expression Problem Add a new data type Add a new operation Without changing: - Existing data types - Existing operations 87 The Expression Problem Add Employee Add raise() Without changing: - Assistant

1. The Curious
Clojure-ist
1

2. Agenda
Data
Data as Code
Destructuring
Macros
Protocols
The Expression Problem
Concurrency
2

3. Data
3

4. edn Extensible Data Notation
https://github.com/edn-format/edn
data
4

5. edn Person
5

6. edn characteristics
edn ⊇ Clojure syntax
used by Datomic and others as data transfer format
language/implementation neutral
edn is a system for the conveyance of values.
6

7. edn is a system for the conveyance of values.
a type system
NOT:
schema based
a system for representing objects
7

8. Scalars
nil
nil, null, or nothing
booleans
true or false
strings
enclosed in “double quotes”
may span multiple lines
t r n supported
characters
c
newline, return, space and tab
8

9. Scalars
integers
0-9
negative
floating point
64-bit (double) precision is expected.
9

10. Names
symbols
used to represent identifiers
should map to something other than strings
may include namespace prefixs:
my-namespace/foo
keywords
identifiers that designate themselves
semantically akin to enumeration values
symbols that must start with :
:fred or :my/fred
10

11. Collections
lists
a sequence of values
zero or more elements within ()
(a b 42)
vectors
a sequence of values…
…that supports random access
zero or more elements within []
[a b 42]
11

12. Collections
maps
collection of key/value associations
every key should appear only once
unordered
zero or more elements within {}
{:a 1, "foo" :bar, [1 2 3] four}
sets
collection of unique values
unordered
heterogeneous
zero or more elements within #{}
#{a b [1 2 3]}
12

13. Data as Code
13

14. Clojure Syntax
edn + …
14

15. Functions
semantics:
structure:
fn call
symbol
arg
string
list
15

16. Operators (No Different than Functions)
fn call
args
list
16

17. Defining Functions
17

18. defn Semantics
define a
fn name
fn
docstring
arguments
fn body
18

19. defn Structure
symbol
symbol
string
vector
list
19

20. Multi-arity
function
meta-data
20

21. Control Flow
21

22. Decisions
true
branch
false
branch
22

23. Decisions
23

24. Refactor
More Arities
24

25. Don’t Forget…
25

26. (source …)
26

27. Namespaces
27

28. Namespace Declaration
(ns com.example.foo)
names correspond to Java packages,
imply same directory structure
28

29. Namespace Declaration
(ns com.example.foo
(:require clojure.data.generators
clojure.test.generative))
load some libs
29

30. Namespace Declaration
(ns com.example.foo
(:require
[clojure.data.generators :as gen]
[clojure.test.generative :as test]))
provide short aliases
for other libs
30

31. Namespace Declaration
(ns ^{:author "Stuart Halloway"
:doc "Data generators for Clojure."}
clojure.data.generators
(:refer-clojure :exclude [byte char long ...])
(:require [clojure.core :as core]))
namespace metadata
31

32. Don’t Do This
(ns com.example.foo
(:use clojure.test.generative))
“:use” makes all names
unqualified
32

33. Seqs
33

34. Sequences
Abstraction of traditional Lisp lists
(seq coll)
if collection is non-empty, return seq
object on it, else nil
(first seq)
returns the first element
(rest seq)
returns a sequence of the rest of the
elements
34

35. Laziness
Most of the core library functions that produce
sequences do so lazily
e.g. map, filter etc
And thus if they consume sequences, do so
lazily as well
Avoids creating full intermediate results
Create only as much as you consume
Work with infinite sequences, datasets larger
than memory
35

36. Sequences
(drop 2 [1 2 3 4 5]) -> (3 4 5)
(take 9 (cycle [1 2 3 4]))
-> (1 2 3 4 1 2 3 4 1)
(interleave [:a :b :c :d :e] [1 2 3 4 5])
-> (:a 1 :b 2 :c 3 :d 4 :e 5)
(partition 3 [1 2 3 4 5 6 7 8 9])
-> ((1 2 3) (4 5 6) (7 8 9))
(map vector [:a :b :c :d :e] [1 2 3 4 5])
-> ([:a 1] [:b 2] [:c 3] [:d 4] [:e 5])
(apply str (interpose , "asdf"))
-> "a,s,d,f"
(reduce + (range 100)) -> 4950
36

37. Seq Cheat Sheet
clojure.org/cheatsheet
37

38. Vectors
38

39. Vectors
(def v [42 :rabbit [1 2 3]])
(v 1) -> :rabbit
(peek v) -> [1 2 3]
(pop v) -> [42 :rabbit]
(subvec v 1) -> [:rabbit [1 2 3]]
(contains? v 0) -> true
; subtle
(contains? v 42) -> false
; subtle
39

40. Maps
40

41. Maps
(def m {:a 1 :b 2 :c 3})
(m :b) -> 2 ;also (:b m)
(keys m) -> (:a :b :c)
(assoc m :d 4 :c 42) -> {:d 4, :a 1, :b 2, :c 42}
(dissoc m :d) -> {:a 1, :b 2, :c 3}
(merge-with + m {:a 2 :b 3}) -> {:a 3, :b 5, :c 3}
41

42. Nested Structures
(def jdoe {:name "John Doe",
:address {:zip 27705, ...}})
(get-in jdoe [:address :zip])
-> 27705
(assoc-in jdoe [:address :zip] 27514)
-> {:name "John Doe", :address {:zip 27514}}
(update-in jdoe [:address :zip] inc)
-> {:name "John Doe", :address {:zip 27706}}
42

43. Sets
(use clojure.set)
(def colors #{"red" "green" "blue"})
(def moods #{"happy" "blue"})
(disj colors "red")
-> #{"green" "blue"}
(difference colors moods)
-> #{"green" "red"}
(intersection colors moods)
-> #{"blue"}
bonus: all relational
algebra primitives
supported for
sets-of-maps
(union colors moods)
-> #{"happy" "green" "red" "blue"}
43

44. Destructuring
44

45. Pervasive Destructuring
DSL for binding names
Works with abstract structure
Available wherever names are made*
Vector binding forms destructure sequential things
Map binding forms destructure associative things
45

46. Why Destructure?
without destructuring,
next-fib-pair is dominated
by code to “pick apart” pair
(defn next-fib-pair
[pair]
[(second pair) (+ (first pair) (second pair))])
(iterate next-fib-pair [0 1])
-> ([0 1] [1 1] [1 2] [2 3] [3 5] [5 8] [8 13]...)
destructure
it yourself…
46

47. Sequential Destructure
…or you can do the same
thing with a simple []
(defn next-fib-pair
[[a b]]
[b (+ a b)])
(iterate next-fib-pair [0 1])
-> ([0 1] [1 1] [1 2] [2 3] [3 5] [5 8] [8 13] ...)
47

48. Simple Things Inline
which makes next-fib-pair
so simple that you will
probably inline it away!
(defn fibs
[]
(map first
(iterate (fn [[a b]] [b (+ a b)]) [0 1])))
48

49. Associative Data
same problem as before:
code dominated by
picking apart person
(defn format-name
[person]
(str/join " " [(:salutation person)
(:first-name person)
(:last-name person)]))
(format-name
{:salutation "Mr." :first-name "John" :last-name "Doe"})
-> "Mr. John Doe"
49

50. Associative Destructure
pick apart name
(defn format-name
[name]
(let [{salutation :salutation
first-name :first-name
last-name :last-name}
name]
(str/join " " [salutation first-name last-name]))
(format-name
{:salutation "Mr." :first-name "John" :last-name "Doe"})
-> "Mr. John Doe"
50

51. The :keys Option
a common scenario:
parameter names and key
names are the same, so say
them only once
(defn format-name
[{:keys [salutation first-name last-name]}]
(str/join " " [salutation first-name last-name]))
(format-name
{:salutation "Mr." :first-name "John" :last-name "Doe"})
-> "Mr. John Doe"
51

52. Optional Keyword Args
not a language feature, simply a
consequence of variable arity fns
plus map destructuring
(defn game
[planet & {:keys [human-players computer-players]}]
(println "Total players: "
(+ human-players computer-players)))
(game "Mars” :human-players 1 :computer-players 2)
Total players: 3
52

53. Platform Interop
53

54. Java new
java
new Widget("foo")
clojure sugar
(Widget. "red")
54

55. Access Static Members
java
Math.PI
clojure sugar
Math/PI
55

56. Access Instance Members
java
clojure sugar
rnd.nextInt()
(.nextInt rnd)
56

57. Chaining Access
java
person.getAddress().getZipCode()
clojure sugar
(.. person getAddress getZipCode)
57

58. Parenthesis Count
java
()()()()
clojure
()()()
58

59. all forms are created equal !
interpretation is everything
59

60. all forms are created equal !
form
syntax
example
function
list
(println "hello")
operator
list
(+ 1 2)
method call
list
(.trim " hello ")
import
list
(require 'mylib)
metadata
list
(with-meta obj m)
control flow
list
(when valid? (proceed))
scope
list
(dosync (alter ...))
60

61. Special Forms
61

62. Special Forms
(def symbol init?)
(if test then else?)
(do exprs*)
(quote form)
(fn name? [params*] exprs*)
(fn name? ([params*] exprs*)+)
(let [bindings*] exprs*)
(loop [bindings*] exprs*)
(recur exprs*)
(throw expr)
(try expr* catch-clause* finally-clause?)
62

63. Macros
63

64. Programs writing
Programs
Code
Text
characters
Effect
Reader
data structures
characters
evaluator/
compiler
bytecode
JVM
data structures
You
Program
data structures
Program
(macro)
64

65. Inside Out?
{:name "Jonathan"}
(assoc {:name "Jonathan"} :nickname "Jon")
(dissoc
(assoc
{:name "Jonathan" :password "secret"}
:nickname "Jon")
:password)
65

66. Thread First ->
(dissoc
(assoc
{:name "Jonathan" :password "secret"}
:nickname "Jon")
:password)
(-> {:name "Jonathan" :password "secret"}
(assoc :nickname "Jon")
(dissoc :password))
66

67. Syntactic Abstraction
Code
Text
characters
Effect
Reader
data structures
characters
evaluator/
compiler
bytecode
JVM
data structures
You
Program
data structures
Program
(macro)
67

68. Seq Ops Inside Out
(range 10)
(map inc (range 10))
(filter odd? (map inc (range 10)))
(reduce + (filter odd? (map inc (range 10))))
68

69. Thread Last
->>
(reduce + (filter odd? (map inc (range 10))))
(->> (range 10)
(map inc)
(filter odd?)
(reduce +))
69

70. defrecord
70

71. Callability
Var
Symbol
Ref
IFn
ifn?
MultiFn
Keyword
APersistentSet
AFn
APersistentMap
APersistentVector
Fn
fn?
AFunction
RestFn
71

72. From Maps...
(def stu {:fname "Stu"
:lname "Halloway"
:address {:street "200 N Mangum"
:city "Durham"
:state "NC"
:zip 27701}})
(:lname stu)
=> "Halloway"
(-> stu :address :city)
=> "Durham"
data oriented
keyword access
nested access
(assoc stu :fname "Stuart")
=> {:fname "Stuart", :lname "Halloway",
:address ...}
(update-in stu [:address :zip] inc)
=> {:address {:street "200 N Mangum",
:zip 27702 ...} ...}
update
nested
update
72

73. ...to Records!
(defrecord Person [fname lname address])
(defrecord Address [street city state zip])
(def stu (Person. "Stu" "Halloway"
(Address. "200 N Mangum"
"Durham"
"NC"
27701)))
still data-oriented:
(:lname stu)
=> "Halloway"
(-> stu :address :city)
=> "Durham"
object
oriented
type is there
when you
everything works
as before
care
(assoc stu :fname "Stuart")
=> :user.Person{:fname "Stuart", :lname"Halloway",
:address ...}
(update-in stu [:address :zip] inc)
=> :user.Person{:address {:street "200 N Mangum",
:zip 27702 ...} ...}
73

74. defrecord
named type
(defrecord Foo [a b c])
-> user.Foo
with slots
(def f (Foo. 1 2 3))
-> #'user/f
positional
constructor
(:b f)
-> 2
(class f)
-> user.Foo
keyword
access
plain ol'
casydht*
class
(supers (class f))
-> #{clojure.lang.IObj clojure.lang.IKeywordLookup java.util.Map
clojure.lang.IPersistentMap clojure.lang.IMeta java.lang.Object
java.lang.Iterable clojure.lang.ILookup clojure.lang.Seqable
clojure.lang.Counted clojure.lang.IPersistentCollection
clojure.lang.Associative}
*Clojure abstracts so you don't have to
74

75. Protocols
75

76. Protocols
(defprotocol AProtocol
"A doc string for AProtocol abstraction"
(bar [a b] "bar docs")
(baz [a] "baz docs"))
Named set of generic functions
Polymorphic on type of first argument
No implementation
Define fns in the same namespaces as protocols
76

77. Extending Protocols
77

78. Extend Protocols Inline
(defrecord Bar [a b c]
AProtocol
(bar [this b] "Bar bar")
(baz [this] (str "Bar baz " c)))
(def b (Bar. 5 6 7))
(baz b)
=> "Bar baz 7"
78

79. Extend Protocols Inline
from ClojureScript
browser.clj
79

80. Extending to a Type
(baz "a")
java.lang.IllegalArgumentException:
No implementation of method: :baz
of protocol: #'user/AProtocol
found for class: java.lang.String
(extend-type String
AProtocol
(bar [s s2] (str s s2))
(baz [s] (str "baz " s)))
(baz "a")
=> "baz a"
80

81. Extending to Many Types
note extend
to nil
from Clojure
reducers.clj
81

82. Extending to Many Protocols
from ClojureScript
core.cljs
82

83. Composition with Extend
from Clojure java/io.clj
the “DSL” for advanced reuse is maps and assoc
83

84. Reify
instantiate an
unnamed type
implement 0
or more
protocols
(let [x 42
or interfaces
r (reify AProtocol
(bar [this b] "reify bar")
(baz [this ] (str "reify baz " x)))]
(baz r))
=> "reify baz 42"
closes over
environment
like fn
84

85. Code Structure
package com.acme.employees;
interface Employee {
}
(namespace com.acme.employees)
(defprotocol Employee )
Employee
(updatePersonalInfo )
raise()
roles()
(roles )
updatePersonalInfo()
Manager
roles()
(raise )
(approvalProfile )
approvalProfile()
85

86. The Expression Problem
86

87. The Expression Problem
A
abstraction
concretion
B
A should be able to work with B's
abstractions, and vice versa,
without modification of
the original code
87

88. Is This Really a Problem?
A
abstraction
concretion
B
just use interfaces
for abstraction (??)
88

89. Example: ArrayList vs.
the Abstractions
java.util.List
ArrayList
clojure.lang.Counted
?
clojure.lang.Seqable
89

90. Example: String vs.
the Abstractions
java.util.List
String
?
clojure.lang.Counted
clojure.lang.Seqable
90

91. A Can't Inherit from B
B is newer than A
A is hard to change
We don’t control A
happens even within a single library!
91

92. Some Approaches
to the Expression
Problem
92

93. 1. Roll-your-own
if/then instanceof? logic
closed
93

94. A Closed World
94

95. 2. Wrappers
java.util.Collection
strings are
so make a
NiftyString
not
that is
collections
java.util.List
String
NiftyString
95

96. Wrappers = Complexity
Ruin identity
Ruin Equality
Cause nonlocal defects
Don’t compose: AB + AC ≠ ABC
Have bad names
96

97. 3. Monkey Patching
strings are
java.util.Collection
sneak in
not
and change
collections
them!
java.util.List
String
common in e.g. ruby
not possible in java
97

98. Monkey Patching = Complexity
Preserves identity (mostly)
Ruins namespacing
Causes nonlocal defects
Forbidden in some languages
98

99. 4. Generic Functions (CLOS)
polymorphism
lives in the
fns
count
String
reduce
don't touch existing
implementation,
just use it
map
99

100. Generic Functions
Decouple polymorphism & types
Polymorphism in the fns, not the types
no “isa” requirement
no type intrusion necessary
100

101. protocols = generic functions
- arbitrary dispatch
+ speed
+ grouping
(and still powerful enough to
solve the expression problem!)
101

102. Concurrency
102

103. concurrency, coincidence of events or
space
parallelism, the execution of operations
concurrently by separate parts of a computer
103

104. Our Tools
threads
104

105. Our Tools
42
places
105

106. Our Tools
42
42
42
critical sections
106

107. memory, the capacity ... for returning to a
previous state when the cause of the
transition from that state is removed
record, the fact or condition of having been
written down as evidence...
... an authentic or official report
107

108. Memory, Records = Places?
Memory is small and expensive
Storage is small and expensive
Machines are precious, dedicated resources
Applications are control centers
108

109. A Different Approach
New memories use new places
New records use new places
New moments use new places
“In-place” changes encapsulated by constructors
109

110. Values
110

111. Values
Immutable
Maybe lazy
Cacheable (forever!)
Can be arbitrarily large
Share structure
111

112. What Can Be a Value?
42
112

113. What Can Be a Value?
42
{:first-name "Stu",
:last-name "Halloway"}
113

114. What Can Be a Value?
42
{:first-name "Stu",
:last-name "Halloway"}
114

115. What Can Be a Value?
42
{:first-name "Stu",
:last-name "Halloway"}
115

116. What Can Be a Value?
42
{:first-name "Stu",
:last-name "Halloway"}
Anything?
116

117. References
Refer to values (or other references)
Permit atomic, functional succession
Model time and identity
Compatible with a wide variety of update semantics
117

118. Epochal Time Model
values
v1
v2
v3
118

119. Epochal Time Model
f1
v1
f2
v2
v3
functions
119

120. Epochal Time Model
f1
v1
f2
v2
v3
atomic succession
120

121. Epochal Time Model
f1
v1
f2
v2
v3
reference
121

122. Epochal Time Model
f1
f2
observers perceive identity, can
v1
v2
remember and record
v3
122

123. Epochal Time Model
f1
f2
observers do not
coordinate
v1
v2
v3
123

124. Epochal Time Model
f1
v1
f2
v2
v3
124

125. Atoms
125

126. Atoms
(def a (atom 0))
(swap! a inc)
=> 1
(compare-and-set! a 0 42)
=> false
(compare-and-set! a 1 7)
=> true
126

127. Atoms
(def a (atom 0))
functional succession
(swap! a inc)
=> 1
(compare-and-set! a 0 42)
=> false
(compare-and-set! a 1 7)
=> true
127

128. Atoms
(def a (atom 0))
(swap! a inc)
=> 1
(compare-and-set! a 0 42)
=> false
optimistic
concurrency
(compare-and-set! a 1 7)
=> true
128

129. Software Transactional
Memory
129

130. Software Transactional Memory
Refs can change only within a transaction
Provides the ACI in ACID
Transactions are speculative, will be retried
130

131. v1
v1
v3
v2
v2
v3
v4
v4
v1
v2
v3
v4
v1
v2
v3
v4
F
F
F
F
F
F
F
F
F
F
F
F
Transactions
131

132. Transactions
(defn transfer
[from to amount]
(dosync
(alter from - amount)
(alter to + amount)))
(alter from - 1)
=> IllegalStateException No transaction running
132

133. Transactions
(defn transfer
[from to amount]
(dosync
(alter from - amount)
(alter to + amount)))
scope transaction
(alter from - 1)
=> IllegalStateException No transaction running
133

134. Transactions
(defn transfer
[from to amount]
(dosync
(alter from - amount)
(alter to + amount)))
functional succession
(alter from - 1)
=> IllegalStateException No transaction running
134

135. Transactions
(defn transfer
[from to amount]
(dosync
(alter from - amount)
(alter to + amount)))
coordination
guaranteed!
(alter from - 1)
=> IllegalStateException No transaction running
135

136. STM Details
Uses locks, latches internally to avoid churn
Deadlock detection and barging
No read tracking
Readers never impede writers
Nobody impedes readers
136

137. Summary
137

138. Summary
Serious Lisp on the JVM
Built as a destination
Advanced language features
Advanced implementation
Secret weapon?
138

139. Clojure in the wild?
“We
re-coded our flagship application XXXXXX
from Java to Clojure about a year ago.
NEVER looked back. Why?
Reduced our lines of code down by at
least half.
Support and bugs have likewise been cut
by about 65-70%.
We have large enterprise clients.
How did we get Clojure into these very
old guard environments????
139

140. Between you and me.... we
lie.
We don't talk about Clojure.
We talk about "Java Extensions" or
"the Clojure Java Extension".
No one is the wiser.
Clients LOVE us for our blistering
fast turn around.
We present ourselves as a larger
company with fake Linkedin employees.
We actually only have 4 real
employees.
But with Clojure we do the same
work as if we had 20.”
140

141. ?’s
The preceding work is licensed under the Creative
Commons Attribution-Share Alike 3.0 License.
http://creativecommons.org/licenses/by-sa/3.0/us/
Clojure (inside out)
Neal Ford, Stuart Halloway
bit.ly/clojureinsideout
Functional Thinking
bit.ly/nf_ftvideo
Presentation Patterns
Neal Ford, Matthew McCullough, Nathaniel Schutta
http://presentationpatterns.com
141