Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 1
(not= DSL macros)
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 2
Writing DSLs is hard
●
At least for me
●
Enlive, Mo...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 3
So, what's a DSL?
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 4
DSL in Clojure
●
“mini-languages” in core:
●
destru...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 5
What's a DSL?
●
Domain Specific Language
●
Scope-li...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 6
What's a DSL?
●
Succinct notation for DS things
●
D...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 7
What's a DSL?
●
Succinct notation for DS things
●
D...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 8
A blurry line
●
A continuum
●
from data schemas
●
t...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 9
Complexity
Enlive users complained about:
●
selecto...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 10
Complexity
Enlive users complained about:
●
select...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 11
Complexity
ClojureQL users complained about:
●
que...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 12
Complexity
ClojureQL users complained about:
●
que...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 13
Complexity
Do you spot a pattern?
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 14
Spoilt users!
●
Users don't want a DSL
●
Users wan...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 15
Spoilt users!
●
Users don't want a DSL
●
Users wan...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 16
Limiting complexity
●
Limit scope
●
Don't reinvent...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 17
Start easy, simple, humble
●
Don't rush for macros...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 18
Data
●
Focus on DS values rather than DS Language
...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 19
But I really need macros!
— anonymous macro addict
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 20
You sure? Dynamicity is good...
●
Macros as premat...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 21
Ok, Macros
Decouple binding, step by step:
●
Desig...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 22
??? explained
●
Create a “binding specs capturing ...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 23
Example: a DSL for regexes
simplified version of g...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 24
Regex DSL
●
A DSL for regexes in Clojure
●
Compile...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 25
Giving Regex semantics to types
●
Literals
●
Seque...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 26
Giving Regex semantics to types
●
Literals
●
Seque...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 27
Evaluating to Java Patterns
(defprotocol RegexNota...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 28
Evaluating to Java Patterns
;; literals, sequences...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 29
Evaluating to Java Patterns
(regex (let [d {0 9}
d...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 30
We need new types!
(defrecord Repetition [spec min...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 31
Helper fns
(defn join
([spec separator] (join spec...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 32
Helper fns
The whole truth is:
●
regex is eval for...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 33
Closing the loop
(extend-protocol RegexNotation
Pa...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 34
Static optimization
Detecting constant fragments (...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 35
Static optimizations
●
Compiling constant fragment...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 36
That's all folks!
Questions?
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 37
STOP
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 38
Complexity
Data
form
at
Functions
M
acros
Com
pile...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 39
DSL Design
●
Code is data
●
Or the other way round...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 40
DSL Design
●
List forms are still cumbersome when ...
First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 41
Macros
Decouple binding example, Moustache routes:...
Upcoming SlideShare
Loading in …5
×

(not= DSL macros)

3,727 views

Published on

Talk given at (first clojure-conj)

  • Be the first to comment

(not= DSL macros)

  1. 1. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 1 (not= DSL macros)
  2. 2. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 2 Writing DSLs is hard ● At least for me ● Enlive, Moustache, Parsley: ● Several iterations ● And counting! ● Main sin: using macros ● Learnt some lessons along the way ● Applied here: github.com/cgrand/regex
  3. 3. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 3 So, what's a DSL?
  4. 4. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 4 DSL in Clojure ● “mini-languages” in core: ● destructuring, seq comprehensions, anonymous fns, syntax quote, pre and post-conditions, -> and ->> etc. ● ClojureQL, Clout, Hiccup, Matchure, Enlive, Moustache etc.
  5. 5. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 5 What's a DSL? ● Domain Specific Language ● Scope-limited ● not a General Purpose Language ● not always Turing-complete ● Internal DSL: ● Written in the host language (Clojure) ● Mostly intended for developers
  6. 6. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 6 What's a DSL? ● Succinct notation for DS things ● Data ● Logic ● DSL benefits ● Usually more declarative ● Reduce incidental complexity
  7. 7. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 7 What's a DSL? ● Succinct notation for DS things ● Data ● Logic ● DSL benefits ● Usually more declarative ● Reduce incidental complexity Succinct declarative notation for DS data & logic
  8. 8. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 8 A blurry line ● A continuum ● from data schemas ● to compiler-in-a-macro ● Every API is a DSL Data form at Functions M acros Com pilerin a m acro Complexity
  9. 9. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 9 Complexity Enlive users complained about: ● selectors not being first class ● and being hard to compose
  10. 10. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 10 Complexity Enlive users complained about: ● selectors not being first class ● and being hard to compose I rewrote it and removed macros
  11. 11. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 11 Complexity ClojureQL users complained about: ● queries being too suprising ● and hard to compose
  12. 12. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 12 Complexity ClojureQL users complained about: ● queries being too suprising ● and hard to compose Kotarak and Lau are rewriting it and removing macros
  13. 13. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 13 Complexity Do you spot a pattern?
  14. 14. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 14 Spoilt users! ● Users don't want a DSL ● Users want to use all the power of Clojure ● Conclusion: Users want values
  15. 15. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 15 Spoilt users! ● Users don't want a DSL ● Users want to use all the power of Clojure ● Conclusion: Users want values Values + functional core (+ macros as icing-on-the-cake) = DSL success!
  16. 16. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 16 Limiting complexity ● Limit scope ● Don't reinvent lexical scoping or control flow ● Rely on Clojure for these ● Limit syntax ● Use closures ● Use higher order functions ● Ugly? Syntax is icing, add it later! ● No macros
  17. 17. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 17 Start easy, simple, humble ● Don't rush for macros! ● Premature optimization ● Too much rope ● Start humble: ● Data ● a handful of functions
  18. 18. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 18 Data ● Focus on DS values rather than DS Language ● Give DS semantics to datatypes ● Clojure's ones ● Your own ones ● Except lists and symbols – Avoid confusion – Not pretty (need to be quoted) – Visual escape to regular Clojure
  19. 19. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 19 But I really need macros! — anonymous macro addict
  20. 20. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 20 You sure? Dynamicity is good... ● Macros as premature optimization ● Example: Enlive's commit 944312b1621 ● Make selectors first class ● Delete 12 defmacros out of 24 ● Allowed for simpler optimizations (caching) Net result: faster and more dynamic
  21. 21. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 21 Ok, Macros Decouple binding, step by step: ● Design your DSL (values and core fns) without binding (but provision it) but with capturing ● ??? ● Profit!!! Legitimate usecases Tentative workarounds Control flow Closures, delays Binding Decouple binding
  22. 22. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 22 ??? explained ● Create a “binding specs capturing specs” fn→ ● Create a “binding specs binding forms” fn→ Decouple binding and capturing (defmacro when-match [pattern value & body] (let [matcher (capturing pattern) bindings (extract-bindings pattern)] `(when-let [~bindings (capture ~matcher value)] ~@body)))
  23. 23. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 23 Example: a DSL for regexes simplified version of github.com/cgrand/regex (e.g. no named groups)
  24. 24. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 24 Regex DSL ● A DSL for regexes in Clojure ● Compile to host regexes ● Not that useful: regexes are already a DSL ● External and composable though ● Basic building blocks of regexes: ● Literals, sequences, alternatives, char ranges, repetitions and a wildcard
  25. 25. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 25 Giving Regex semantics to types ● Literals ● Sequences ● Alternatives ● Char ranges ● Repetitions ● Wildcard Notations?
  26. 26. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 26 Giving Regex semantics to types ● Literals ● Sequences ● Alternatives ● Char ranges ● Repetitions ● Wildcard "abc", d [this then-that and-also] #{this or-that or-also} {a z, A Z, 0 9} (re/repeat this min? max?) re/any
  27. 27. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 27 Evaluating to Java Patterns (defprotocol RegexNotation (pattern [this] "Returns a corresponding pattern (as String).")) regex is the user fn: Then, define pattern as a protocol fn (defn regex [spec] (-> spec pattern Pattern/compile))
  28. 28. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 28 Evaluating to Java Patterns ;; literals, sequences, alternatives and char ranges (extend-protocol RegexNotation String (pattern [s] (Pattern/quote s)) Character (pattern [c] (pattern (str c))) clojure.lang.APersistentVector (pattern [v] (str/join (map pattern v))) clojure.lang.APersistentSet (pattern [s] (str "(?:" (str/join "|" (map pattern s)) ")")) clojure.lang.APersistentMap (pattern [m] (str "[" (str/join (for [[from to] m] (str from "-" to))) "]")))
  29. 29. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 29 Evaluating to Java Patterns (regex (let [d {0 9} d2 [d d] d4 [d2 d2]] ["date: " d4 - d2 - d2])) ;; #"Qdate: E[0-9][0-9][0-9][0-9]Q-E[0-9][0-9]Q-E[0-9][0-9]" (let [d {0 9} d2 [d d] d4 [d2 d2]] (regex ["date: " d4 - d2 - d2])) ;; or (let [d {0 9} d2 [d d] date (into ["date: " d2] (interpose - [[d2 d2] d2 d2]))] (regex date)) ;; are equivalents ;; DSL fragments are first class, ;; you can use all clojure to build them!
  30. 30. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 30 We need new types! (defrecord Repetition [spec min max] RegexNotation (pattern [r] (str "(?:" (pattern spec) ")" "{" (or min 0) "," (or max "") "}"))) (defn repeat ([spec] (repeat spec nil nil)) ([spec min] (repeat spec min nil)) ([spec min max] (Repetition. spec min max))) ;; any is the only one of its kind -> reify (def any (reify RegexNotation (pattern [_] "."))) Repetition and any need their own types:
  31. 31. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 31 Helper fns (defn join ([spec separator] (join spec separator nil nil)) ([spec separator min] (join spec separator min nil)) ([spec separator min max] (if (zero? (or min 0)) (repeat (join spec separator 1 max) 0 1) [spec (repeat [separator spec] (dec min) (when max (dec max)))]))) => (regex (join [{0 9} {0 9}] - 1 3)) #"[0-9][0-9](?:Q-E[0-9][0-9]){0,2}" You can easily define helper fns:
  32. 32. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 32 Helper fns The whole truth is: ● regex is eval for your DSL ● Helper fns act like macros for your DSL
  33. 33. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 33 Closing the loop (extend-protocol RegexNotation Pattern (pattern [p] (.pattern p))) => (regex #{"hello" "world"}) #"(?:QhelloE|QworldE)" => (regex (regex #{"hello" "world"})) #"(?:QhelloE|QworldE)" Making regex idempotent Why does it matter? ● You can canonicalize your inputs ● You can "pre-compile" or cache fragments.
  34. 34. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 34 Static optimization Detecting constant fragments (macros ok...) ● Simplifying them away ● Recognizing your own functions: Don't check syms, check vars! (when-not (contains? &env sym) (resolve sym)) ; 1.2 (resolve &env sym) ; 1.3
  35. 35. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 35 Static optimizations ● Compiling constant fragments ● “eval” them (e.g. call re/regex) ● Expand to the value when readable or to a factory form
  36. 36. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 36 That's all folks! Questions?
  37. 37. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 37 STOP
  38. 38. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 38 Complexity Data form at Functions M acros Com pilerin a m acro Complexity ● Complexity for the implementor ● The joy of tree-walking ● Complexity for the user ● Different from regular Clojure code ● What is first class? What's not?
  39. 39. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 39 DSL Design ● Code is data ● Or the other way round ● Rich range of literals – Vectors, maps, sets, keywords, numbers, regexes ● No need to quote, walk, extract/unquote – arguments evaluation without call semantics [:operator arg1 arg2] (operator arg1 arg2) – Unless you use lists
  40. 40. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 40 DSL Design ● List forms are still cumbersome when doing static analysis ● Always use a conservative default “When in doubt, don't” – Benjamin Franklin ● &env relieves the pain ● (when-not (contains? &env sym) (resolve sym)) ● test against vars, not syms ● no more shadowing problems somewhat opaque
  41. 41. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 41 Macros Decouple binding example, Moustache routes: ● match-route is functional ● I could have done without derive-simple-route Legitimate macros Tentative workarounds Control flow Closures, delays Binding Decouple binding (let [bindings (derive-bindings route-spec) simple-route (derive-simple-route route-spec)] `(when-let [~bindings (match-route ~url ~simple-route)] ...))

×