Successfully reported this slideshow.

(not= DSL macros)

3

Share

1 of 41
1 of 41

More Related Content

Related Audiobooks

Free with a 14 day trial from Scribd

See all

(not= DSL macros)

  1. 1. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 1 (not= DSL macros)
  2. 2. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 2 Writing DSLs is hard ● At least for me ● Enlive, Moustache, Parsley: ● Several iterations ● And counting! ● Main sin: using macros ● Learnt some lessons along the way ● Applied here: github.com/cgrand/regex
  3. 3. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 3 So, what's a DSL?
  4. 4. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 4 DSL in Clojure ● “mini-languages” in core: ● destructuring, seq comprehensions, anonymous fns, syntax quote, pre and post-conditions, -> and ->> etc. ● ClojureQL, Clout, Hiccup, Matchure, Enlive, Moustache etc.
  5. 5. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 5 What's a DSL? ● Domain Specific Language ● Scope-limited ● not a General Purpose Language ● not always Turing-complete ● Internal DSL: ● Written in the host language (Clojure) ● Mostly intended for developers
  6. 6. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 6 What's a DSL? ● Succinct notation for DS things ● Data ● Logic ● DSL benefits ● Usually more declarative ● Reduce incidental complexity
  7. 7. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 7 What's a DSL? ● Succinct notation for DS things ● Data ● Logic ● DSL benefits ● Usually more declarative ● Reduce incidental complexity Succinct declarative notation for DS data & logic
  8. 8. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 8 A blurry line ● A continuum ● from data schemas ● to compiler-in-a-macro ● Every API is a DSL Data form at Functions M acros Com pilerin a m acro Complexity
  9. 9. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 9 Complexity Enlive users complained about: ● selectors not being first class ● and being hard to compose
  10. 10. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 10 Complexity Enlive users complained about: ● selectors not being first class ● and being hard to compose I rewrote it and removed macros
  11. 11. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 11 Complexity ClojureQL users complained about: ● queries being too suprising ● and hard to compose
  12. 12. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 12 Complexity ClojureQL users complained about: ● queries being too suprising ● and hard to compose Kotarak and Lau are rewriting it and removing macros
  13. 13. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 13 Complexity Do you spot a pattern?
  14. 14. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 14 Spoilt users! ● Users don't want a DSL ● Users want to use all the power of Clojure ● Conclusion: Users want values
  15. 15. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 15 Spoilt users! ● Users don't want a DSL ● Users want to use all the power of Clojure ● Conclusion: Users want values Values + functional core (+ macros as icing-on-the-cake) = DSL success!
  16. 16. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 16 Limiting complexity ● Limit scope ● Don't reinvent lexical scoping or control flow ● Rely on Clojure for these ● Limit syntax ● Use closures ● Use higher order functions ● Ugly? Syntax is icing, add it later! ● No macros
  17. 17. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 17 Start easy, simple, humble ● Don't rush for macros! ● Premature optimization ● Too much rope ● Start humble: ● Data ● a handful of functions
  18. 18. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 18 Data ● Focus on DS values rather than DS Language ● Give DS semantics to datatypes ● Clojure's ones ● Your own ones ● Except lists and symbols – Avoid confusion – Not pretty (need to be quoted) – Visual escape to regular Clojure
  19. 19. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 19 But I really need macros! — anonymous macro addict
  20. 20. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 20 You sure? Dynamicity is good... ● Macros as premature optimization ● Example: Enlive's commit 944312b1621 ● Make selectors first class ● Delete 12 defmacros out of 24 ● Allowed for simpler optimizations (caching) Net result: faster and more dynamic
  21. 21. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 21 Ok, Macros Decouple binding, step by step: ● Design your DSL (values and core fns) without binding (but provision it) but with capturing ● ??? ● Profit!!! Legitimate usecases Tentative workarounds Control flow Closures, delays Binding Decouple binding
  22. 22. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 22 ??? explained ● Create a “binding specs capturing specs” fn→ ● Create a “binding specs binding forms” fn→ Decouple binding and capturing (defmacro when-match [pattern value & body] (let [matcher (capturing pattern) bindings (extract-bindings pattern)] `(when-let [~bindings (capture ~matcher value)] ~@body)))
  23. 23. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 23 Example: a DSL for regexes simplified version of github.com/cgrand/regex (e.g. no named groups)
  24. 24. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 24 Regex DSL ● A DSL for regexes in Clojure ● Compile to host regexes ● Not that useful: regexes are already a DSL ● External and composable though ● Basic building blocks of regexes: ● Literals, sequences, alternatives, char ranges, repetitions and a wildcard
  25. 25. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 25 Giving Regex semantics to types ● Literals ● Sequences ● Alternatives ● Char ranges ● Repetitions ● Wildcard Notations?
  26. 26. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 26 Giving Regex semantics to types ● Literals ● Sequences ● Alternatives ● Char ranges ● Repetitions ● Wildcard "abc", d [this then-that and-also] #{this or-that or-also} {a z, A Z, 0 9} (re/repeat this min? max?) re/any
  27. 27. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 27 Evaluating to Java Patterns (defprotocol RegexNotation (pattern [this] "Returns a corresponding pattern (as String).")) regex is the user fn: Then, define pattern as a protocol fn (defn regex [spec] (-> spec pattern Pattern/compile))
  28. 28. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 28 Evaluating to Java Patterns ;; literals, sequences, alternatives and char ranges (extend-protocol RegexNotation String (pattern [s] (Pattern/quote s)) Character (pattern [c] (pattern (str c))) clojure.lang.APersistentVector (pattern [v] (str/join (map pattern v))) clojure.lang.APersistentSet (pattern [s] (str "(?:" (str/join "|" (map pattern s)) ")")) clojure.lang.APersistentMap (pattern [m] (str "[" (str/join (for [[from to] m] (str from "-" to))) "]")))
  29. 29. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 29 Evaluating to Java Patterns (regex (let [d {0 9} d2 [d d] d4 [d2 d2]] ["date: " d4 - d2 - d2])) ;; #"Qdate: E[0-9][0-9][0-9][0-9]Q-E[0-9][0-9]Q-E[0-9][0-9]" (let [d {0 9} d2 [d d] d4 [d2 d2]] (regex ["date: " d4 - d2 - d2])) ;; or (let [d {0 9} d2 [d d] date (into ["date: " d2] (interpose - [[d2 d2] d2 d2]))] (regex date)) ;; are equivalents ;; DSL fragments are first class, ;; you can use all clojure to build them!
  30. 30. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 30 We need new types! (defrecord Repetition [spec min max] RegexNotation (pattern [r] (str "(?:" (pattern spec) ")" "{" (or min 0) "," (or max "") "}"))) (defn repeat ([spec] (repeat spec nil nil)) ([spec min] (repeat spec min nil)) ([spec min max] (Repetition. spec min max))) ;; any is the only one of its kind -> reify (def any (reify RegexNotation (pattern [_] "."))) Repetition and any need their own types:
  31. 31. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 31 Helper fns (defn join ([spec separator] (join spec separator nil nil)) ([spec separator min] (join spec separator min nil)) ([spec separator min max] (if (zero? (or min 0)) (repeat (join spec separator 1 max) 0 1) [spec (repeat [separator spec] (dec min) (when max (dec max)))]))) => (regex (join [{0 9} {0 9}] - 1 3)) #"[0-9][0-9](?:Q-E[0-9][0-9]){0,2}" You can easily define helper fns:
  32. 32. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 32 Helper fns The whole truth is: ● regex is eval for your DSL ● Helper fns act like macros for your DSL
  33. 33. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 33 Closing the loop (extend-protocol RegexNotation Pattern (pattern [p] (.pattern p))) => (regex #{"hello" "world"}) #"(?:QhelloE|QworldE)" => (regex (regex #{"hello" "world"})) #"(?:QhelloE|QworldE)" Making regex idempotent Why does it matter? ● You can canonicalize your inputs ● You can "pre-compile" or cache fragments.
  34. 34. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 34 Static optimization Detecting constant fragments (macros ok...) ● Simplifying them away ● Recognizing your own functions: Don't check syms, check vars! (when-not (contains? &env sym) (resolve sym)) ; 1.2 (resolve &env sym) ; 1.3
  35. 35. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 35 Static optimizations ● Compiling constant fragments ● “eval” them (e.g. call re/regex) ● Expand to the value when readable or to a factory form
  36. 36. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 36 That's all folks! Questions?
  37. 37. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 37 STOP
  38. 38. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 38 Complexity Data form at Functions M acros Com pilerin a m acro Complexity ● Complexity for the implementor ● The joy of tree-walking ● Complexity for the user ● Different from regular Clojure code ● What is first class? What's not?
  39. 39. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 39 DSL Design ● Code is data ● Or the other way round ● Rich range of literals – Vectors, maps, sets, keywords, numbers, regexes ● No need to quote, walk, extract/unquote – arguments evaluation without call semantics [:operator arg1 arg2] (operator arg1 arg2) – Unless you use lists
  40. 40. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 40 DSL Design ● List forms are still cumbersome when doing static analysis ● Always use a conservative default “When in doubt, don't” – Benjamin Franklin ● &env relieves the pain ● (when-not (contains? &env sym) (resolve sym)) ● test against vars, not syms ● no more shadowing problems somewhat opaque
  41. 41. First Clojure Conj, Oct 22 2010 (not= DSL macros) / Christophe Grand 41 Macros Decouple binding example, Moustache routes: ● match-route is functional ● I could have done without derive-simple-route Legitimate macros Tentative workarounds Control flow Closures, delays Binding Decouple binding (let [bindings (derive-bindings route-spec) simple-route (derive-simple-route route-spec)] `(when-let [~bindings (match-route ~url ~simple-route)] ...))

×