Clojureのstm実装について

Clojure STM

@marblejenka

2011 5 24

• Clojure STM
• STM STM

•
Clojure/Java java.util.concurrent.*

2

2011 5 24

Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
3

2011 5 24

Clojure

• Clojure
–
–
•
–

4

2011 5 24

Clojure

•

• Clojure

• ref STM:Software Transactional Memory

– ref
atom/agent/var

5

2011 5 24

Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
6

2011 5 24

STM
• Software Transactional Memory
• ACID ACI
– Atomic
•
– Consistency
•
– Isolation
•

– Durabilty
• STM
D
• DB
• MVCC
7

2011 5 24

STM
reference
F
v=42, t=0

The Joy of Clojure Figure 10.1 8

2011 5 24

Tx A STM
reference
F
v=42, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read

F
v=42, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read

F
v=42, t=0 calc


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read

F
v=42, t=0 calc

local write


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read

F
v=42, t=0 calc

local write

F
v=9, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc

local write

F
v=9, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write

F
v=9, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read

F
v=9, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F
v=9, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0

local write


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0

local write

F
v=27, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conﬂict?

local write

F
v=27, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conﬂict?

NO local write

F
v=27, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conﬂict?

NO local write

F
commit v=27, t=0


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conﬂict?

NO local write

F
commit v=27, t=0
F
v=9, t=1


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conﬂict?

NO local write

F
commit v=27, t=0
F
v=9, t=1 conﬂict?


2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conﬂict?

NO local write

F
commit v=27, t=0
F
v=9, t=1 conﬂict?

YES

2011 5 24

Tx A STM
reference
A
F
v=42, t=0
read Tx B
F
v=42, t=0 calc B
local write read
F
v=42, t=0
F calc
v=9, t=0 conﬂict?

NO local write

F
commit v=27, t=0
F
v=9, t=1 conﬂict?

YES
retry!

2011 5 24

STM
• STM
–
•

– Transactional
• atmic
• STM
–
–
I/O
–

9

2011 5 24

Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
10

2011 5 24

Clojure STM
•
– ref
– Clojure immutable

– ref
• ref all-or-noting
– dosync
– ref
all-of-nothing
– 11

2011 5 24

Clojure STM
• a,b
• def
– a,b
– a b
– a b
– a,b
• def dosync
– a,b
– a,b
12

2011 5 24

Clojure STM

(def a 0)
(def b 1)
(inc a) ; a +1
(dec b) ; b -1
(println a b) ; -> 1 0

• a,b
• Clojure immutable println
a,b

13

2011 5 24

Clojure STM

(def a (ref 0))
(def b (ref 1))
(dosync
(alter a inc)
(alter b dec))
(println @a @b)

• a,b all-or-nothing
• ref println
1,0

14

2011 5 24

Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
15

2011 5 24

Clojure STM API
• API(Clojure STM
)

• Java

• URL
– https://github.com/clojure/clojure/blob/master/src/clj/clojure/
core.clj#L1831

16

2011 5 24

API
• ref
– ref

• deref
–

• ref-set
– ref

• alter
– ref

• commute
– alter

• ensure
– in-transaction-value
17

2011 5 24

ref
(defn ref
{:added "1.0"
:static true
}
([x] (new clojure.lang.Ref x))
([x & options]
(let [r ^clojure.lang.Ref (setup-reference (ref x) options)
opts (apply hash-map options)]
(when (:max-history opts)
(.setMaxHistory r (:max-history opts)))
(when (:min-history opts)
(.setMinHistory r (:min-history opts)))
r)))

•
• ref dosync

• 18

2011 5 24

deref
(defn deref
"Also reader macro: @ref/@agent/@var/@atom/@delay/@future/@promise. Within a transaction,
returns the in-transaction-value of ref, else returns the
most-recently-committed value of ref. When applied to a var, agent
or atom, returns its current state. When applied to a delay, forces
it if not already forced. When applied to a future, will block if
computation not complete. When applied to a promise, will block
until a value is delivered. The variant taking a timeout can be
used for blocking references (futures and promises), and will return
timeout-val if the timeout (in milliseconds) is reached before a
value is available. See also - realized?."
{:added "1.0"
:static true}
([^clojure.lang.IDeref ref] (.deref ref))
([^clojure.lang.IBlockingDeref ref timeout-ms timeout-val] (.deref ref timeout-ms timeout-val)))

• ref
– dosync dosync in-
transaction-value
• future
19

2011 5 24

ref-set

(defn ref-set
"Must be called in a transaction. Sets the value of ref.
Returns val."
{:added "1.0"
:static true}
[^clojure.lang.Ref ref val]
(. ref (set val)))

• ref

20

2011 5 24

alter
(defn alter
"Must be called in a transaction. Sets the in-transaction-value of
ref to:

(apply fun in-transaction-value-of-ref args)

and returns the in-transaction-value of ref."
{:added "1.0"
:static true}
[^clojure.lang.Ref ref fun & args]
(. ref (alter fun args)))

• ref
•

21

2011 5 24

commute
(defn commute
"Must be called in a transaction. Sets the in-transaction-value of
ref to:

(apply fun in-transaction-value-of-ref args)

and returns the in-transaction-value of ref.

At the commit point of the transaction, sets the value of ref to be:

(apply fun most-recently-committed-value-of-ref args)

Thus fun should be commutative, or, failing that, you must accept
last-one-in-wins behavior. commute allows for more concurrency than
ref-set."
{:added "1.0"
:static true}

[^clojure.lang.Ref ref fun & args]
(. ref (commute fun args)))

• ref
• alter
22

2011 5 24

ensure

(defn ensure
"Must be called in a transaction. Protects the ref from modification
by other transactions. Returns the in-transaction-value of
ref. Allows for more concurrency than (ref-set ref @ref)"
{:added "1.0"
:static true}
[^clojure.lang.Ref ref]
(. ref (touch))
(. ref (deref)))

• in-
transaction-value
• ref ref
ref
23

2011 5 24

dosync

(defmacro dosync
"Runs the exprs (in an implicit do) in a transaction that encompasses
exprs and any nested calls. Starts a transaction if none is already
running on this thread. Any uncaught exception will abort the
transaction and flow out of dosync. The exprs may be run more than
once, but any effects on Refs will be atomic."
{:added "1.0"}
[& exprs]
`(sync nil ~@exprs))

•

24

2011 5 24

sync
(defmacro sync
"transaction-flags => TBD, pass nil for now

Runs the exprs (in an implicit do) in a transaction that encompasses
exprs and any nested calls. Starts a transaction if none is already
running on this thread. Any uncaught exception will abort the
transaction and flow out of sync. The exprs may be run more than
once, but any effects on Refs will be atomic."
{:added "1.0"}
[flags-ignored-for-now & body]
`(. clojure.lang.LockingTransaction
(runInTransaction (fn [] ~@body))))

•
• “ﬂags-ignored-for-now”

25

2011 5 24

io!
(defmacro io!
"If an io! block occurs in a transaction, throws an
IllegalStateException, else runs body in an implicit do. If the
first expression in body is a literal string, will use that as the
exception message."
{:added "1.0"}
[& body]
(let [message (when (string? (first body)) (first body))
body (if message (next body) body)]
`(if (clojure.lang.LockingTransaction/isRunning)
(throw (new IllegalStateException ~(or message "I/O in transaction")))
(do ~@body))))

• dosync io
• STM I/O

• I/O sync
26

2011 5 24

set/get-validator
(defn set-validator!
"Sets the validator-fn for a var/ref/agent/atom. validator-fn must be nil or a
side-effect-free fn of one argument, which will be passed the intended
new state on any state change. If the new state is unacceptable, the
validator-fn should return false or throw an exception. If the current state
(root
value if var) is not acceptable to the new validator, an exception
will be thrown and the validator will not be changed."
{:added "1.0"
:static true}
[^clojure.lang.IRef iref validator-fn] (. iref (setValidator validator-fn)))

(defn get-validator
"Gets the validator-fn for a var/ref/agent/atom."
{:added "1.0"
:static true}
[^clojure.lang.IRef iref] (. iref (getValidator)))

•
•

27

2011 5 24

alter/reset-meta
(defn alter-meta!
"Atomically sets the metadata for a namespace/var/ref/agent/atom to be:

(apply f its-current-meta args)

f must be free of side-effects"
{:added "1.0"
:static true}
[^clojure.lang.IReference iref f & args] (.alterMeta iref f args))

(defn reset-meta!
"Atomically resets the metadata for a namespace/var/ref/agent/atom"
{:added "1.0"
:static true}
[^clojure.lang.IReference iref metadata-map] (.resetMeta iref metadata-map))

•
•

28

2011 5 24

ref-history
(defn ref-history-count
"Returns the history count of a ref" • ref
{:added "1.1"
:static true}
[^clojure.lang.Ref ref]
(.getHistoryCount ref))
•
(defn ref-min-history
"Gets the min-history of a ref, or sets it and returns the ref"
{:added "1.1"
:static true}
([^clojure.lang.Ref ref]
(.getMinHistory ref))
([^clojure.lang.Ref ref n]
(.setMinHistory ref n)))

(defn ref-max-history
"Gets the max-history of a ref, or sets it and returns the ref"
{:added "1.1"
:static true}
([^clojure.lang.Ref ref]
(.getMaxHistory ref))
([^clojure.lang.Ref ref n]
(.setMaxHistory ref n)))
29

2011 5 24

Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
30

2011 5 24

Clojure STM
• STM
• STM

31

2011 5 24

Ref
• STM

–
TVal

• Ref TVal

34

2011 5 24

Ref.Tval
• ref

•

35

2011 5 24

LockingTransaction
•

•

– 10000000nano second
kill

• vals vals Ref

• sets/commutes/ensures

– sets ref comutes commute

– alter ref-set ref-set

– ensures commute

36

2011 5 24

LockingTransaction.Info
• ref
ref
transaction ref

•

• running/commiting/retry/killed/
commited

•

37

2011 5 24

LockingTransaction.RetryEx

•

•

38

2011 5 24

LockingTransaction.AbortException

•

39

2011 5 24

• ref-set

40

2011 5 24

(def current-track (ref "initial state of current track"))
(dosync (ref-set current-track "changed"))

• ref
• dosync ref-set

41

2011 5 24

transaction run

43

2011 5 24

ref-set

44

2011 5 24

Agenda
• Clojure
• STM
• Clojure STM
• Clojure STM API
• Clojure STM
• Clojure STM
46

2011 5 24

Clojure STM Transaction

• Multiversion Concurrency Control
–

• Snapshot Isolation
–

– Oracle/PostgereSQL RDBMS
SERIALIZABLE
47

2011 5 24

• Clojure

– conj

–

•

– Map Idel Hash Tree

• STM
– Snapshot Isolation
ref

48

2011 5 24

STM I/O
• No I/O in STM transactions?
– http://groups.google.com/group/clojure/
browse_thread/thread/
d967171ed4b0f20c/9f55aead761f3e99?
lnk=gst&q=STM#9f55aead761f3e99
• Agent

49

2011 5 24

• Extending Clojure's STM with external
transactions
aa22a709501a64ac
• RDBMS JTA

–
– 50

2011 5 24

• Long-running STM updates will never
complete... is there a solution?
7edd12e22eb120ea/207888a0d12ade38?
lnk=gst&q=STM#207888a0d12ade38
• ensure
–

51

2011 5 24

• LL tiger clojure
–

– http://www.slideshare.net/makingx/ll-tiger-clojure

• The Art of Multiprocessor Programming

– 18 Transactional Memory STM
Java Java

– http://amzn.to/kJCkyT

52

2011 5 24

Clojureのstm実装について

Recommended

Recommended

More Related Content

More from Shingo Furuyama

More from Shingo Furuyama (7)

Recently uploaded

Recently uploaded (20)

Clojureのstm実装について