Tommi Reiman (https://twitter.com/ikitommi) will be presenting Malli (https://github.com/metosin/malli) is a fresh new data-driven data validation and specification library for Clojure/Script. In this talk, Tommi will give a quick introduction to Malli, compare it to prior art including Plumatic Schema and clojure.spec and demonstrate how to elegantly solve real-world problems with it. Also, peek beyond the runtime validation.

1. Designing
with
Malli
London Clojurians
12.08.2020
Tommi Reiman
@ikitommi

2. a new data validation and specification
library for Clojure/Script. It provides unified
tools for building data-driven schema systems
covering schema definition, validation, (human)
errors, value and schema transformation, value and
schema generation, registries and much more.
https://github.com/metosin/malli
Malli

3. (def address
{:id "123"
:tags #{:garage :chicken}
:address {:street "Jl. Pacar No.15, Sumerta Kauh"
:city "Densapar"
:zip 80236
:latlon [-8.652037 115.226808]}})
data

4. (def Address
{:id s/Str
:tags #{s/Keyword}
:address {:street s/Str
:city s/Str
:zip s/Int
:latlon [(s/one Double "lat")
(s/one Double "lon")]}})
plumatic schema

5. (require '[clojure.spec.alpha :as s])
(s/def ::id string?)
(s/def ::tags (s/coll-of keyword? :kind set?))
(s/def ::street string?)
(s/def ::city string?)
(s/def ::zip int?)
(s/def ::latlon (s/tuple double? double?))
(s/def :address/address
(s/keys :req-un [::street ::city ::zip ::latlon]))
(s/def ::address
(s/keys :req-un [::id ::tags :address/address]))
clojure.spec

6. (def Address
[:map
[:id string?]
[:tags [:set keyword?]]
[:address
[:map
[:street string?]
[:city string?]
[:zip int?]
[:latlon [:tuple double? double?]]]]])
Hiccup syntax
Just data and functions
Maps are open
by default
Keys are
required by
default
Malli
Ordered
Keys

7. (def Address
[:map
[:id string?]
[:tags [:set keyword?]]
[:address
[:map
[:street string?]
[:city string?]
[:zip int?]
[:latlon [:tuple double? double?]]]]])
Malli

8. (def Address
[:map {:title “Address”
:summary “An address"}
[:id string?]
[:tags {:optional true} [:set keyword?]]
[:address
[:map {:closed true}
[:street string?]
[:city string?]
[:zip {:default "33800"} int?]
[:latlon [:tuple double? double?]]]]])
Malli

9. Schema applications
• Validation and Machine-readable errors
• Spell-checking and human-readable errors
• Two-way value transformations
• Schema transformations (walkers, visitors, utils, JSON Schema, DOT, ..)
• GeneratingValues & Schemas
• (Parsing)

10. (def invalid-address
{:id 123
:tags #{:garage :chicken}
:address {:streetz "Jl. Pacar No.15"
:city "Densapar"
:latlon [-8.652037 115.226808]}})
(-> Address
(m/explain invalid-address)
(me/with-spell-checking)
(me/humanize))
;{:id ["should be a string"]
; :address {:zip ["missing required key"]
; :streetz ["should be spelled :street"]}}

11. (require '[malli.generator :as mg])
(mg/generate Address {:size 10, :seed 20})
;{:id "A1f05U4uE5",
; :tags #{:FqNN/Ti. :b3/h. :Ni/h? :Yg*-/GW*Y :jhL/k! :_/QnR},
; :address {:street "ABM74Q96S4"
; :city "k6z3OD4cy"
; :zip -1
; :latlon [-0.32421875 -1.8125]}}

12. The
Registry

13. Registries
• Schema types are looked from a registry
• Registry is passed as explicit option into schema functions
• Default registry has ~100 types and is immutable
• Other registry types: mutable, dynamic, lazy, local and composite
• Special JVM / Closure compiler option to swap the default registry
• Default to global mutable registry in Malli 1.0.0?

14. (def Order
[:schema
{:registry {"Country" [:map
[:name [:enum :FI :PO]]
[:neighbors [:vector [:ref "Country"]]]]
"Burger" [:map
[:name string?]
[:description {:optional true} string?]
[:origin [:maybe "Country"]]
[:price pos-int?]]
"OrderLine" [:map
[:burger "Burger"]
[:amount int?]]
"Order" [:map
[:lines [:vector "OrderLine"]]
[:delivery [:map
[:delivered boolean?]
[:address [:map
[:street string?]
[:zip int?]
[:country "Country"]]]]]]}}
"Order"])

16. Decomplect
maps, keys
and values
::id int?
::id int?

17. (s/def :address/id string?)
(s/def :address/tags (s/coll-of keyword? :kind set?))
(s/def :address/street string?)
(s/def :address/city string?)
(s/def :address/zip int?)
(s/def :address/latlon (s/tuple double? double?))
(s/def :address.address/address
(s/keys :req [::street ::city ::zip ::latlon]))
(s/def :address/address
(s/keys :req [::id ::tags :address/address]))

18. [:schema
{:registry {::id string?
::tags [:set keyword?]
::street string?
::city string?
::zip int?
::latlon [:tuple double? double?]
:address/address [:map ::street ::city ::zip ::latlon]
::address [:map ::id ::tags :address/address]}}
::address]

19. [:map
{:registry {::id string?
::tags [:set keyword?]
::address [:map
{:registry {::street string?
::city string?
::zip int?
::latlon [:tuple double? double?]}}
::street ::city ::zip ::latlon]}}
::id ::tags ::address]

20. In Real
World

21. Coercion
• We are dealing with various data sources with different capabilities,
e.g. JSON,YAML and XML data
• We need to decode from X to EDN and encode from EDN to X’
• … or just apply default values. Or convert strings into keyword keys
• Malli has fast two-way interceptor-based transformers
• Transforming based on Schema type and properties

22. (def Address
[:map {:title “Address”
:summary “An address"}
[:id string?]
[:tags {:optional true} [:set keyword?]]
[:address
[:map {:closed true}
[:street string?]
[:city string?]
[:zip {:default "33800"} int?]
[:latlon [:tuple double? double?]]]]])

23. (require '[malli.transform :as mt])
(m/decode
Address
{:id "123"
:tags [“garage” “chicken”]
:address {:street "Jl. Pacar No.15"
:city "Densapar"
:zip 80236
:latlon [-8.652037 115.226808]}}
(mt/json-transformer))
;{:id "123"
; :tags #{:garage :chicken},
; :address {:street "Jl. Pacar No.15, Sumerta Kauh, Kec."
; :city "Densapar"
; :zip 80236
; :latlon [-8.652037 115.226808]}}

24. (m/decode
Address
{:id "Lillan"
:EVIL "LYN"
:tags ["coffee" "artesan" "garden"]
:address {:street "Ahlmanintie 29"
:DARK "ORKO"
:city "Tampere"
:lonlat [61.4858322 23.7854658]}}
(mt/transformer
(mt/strip-extra-keys-transformer)
(mt/default-value-transformer)
(mt/json-transformer)))
;{:id "Lillan"
; :tags #{:coffee :artesan :garden}
; :address {:street "Ahlmanintie 29"
; :city "Tampere"
; :zip "33800"}}

25. (def LegacyString
[:string {:decode/string '(fn [x] (str/lower-case (subs x 4)))
:encode/string '(fn [x] (str/upper-case (str "HAL_" x)))}])
(m/decode LegacyString "HAL_KIKKA" (mt/string-transformer))
; => "kikka"
(m/encode LegacyString "kikka" (mt/string-transformer))
; => "HAL_KIKKA"
(m/decode
[:string {:decode/string {:enter '#(str "olipa_" %)
:leave '#(str % "_avaruus")}}]
"kerran"
(mt/string-transformer))
; => "olipa_kerran_avaruus"

27. Inferring values
• Opposite of GeneratingValues - return a Schema that validates
against the sample values
• Crawled legacy MongoDB collection (5+ years of accumulated stuff)
• For each value, inferred a schema + accumulated schema union
• All documents have timestamps —>Temporal Schema index!

28. (require '[malli.provider :as mp])
(mp/provide
[{:id "123"
:tags #{:garage :chicken}
:address {:street "Jl. Pacar No.15, Sumerta Kauh, Kec."
:city "Densapar"
:zip 80236
:latlon [-8.652037 115.226808]}}
{:id "Lillan"
:tags #{:coffee :artesan :garden}
:address {:street "Ahlmanintie 29"
:city "Tampere"
:lonlat [61.4858322 23.7854658]}}])
;[:map
; [:id string?]
; [:tags [:set keyword?]]
; [:address
; [:map
; [:street string?]
; [:city string?]
; [:zip {:optional true} int?]
; [:latlon {:optional true} [:vector double?]]
; [:lonlat {:optional true} [:vector double?]]]]]

29. Serialising Schemas
• Thanks to SCI (https://github.com/borkdude/sci) and Edamame
(https://github.com/borkdude/edamame), mostly all schemas can be
• Thanks to SCI (https://github.com/borkdude/sci) and Edamame
(https://github.com/borkdude/edamame), mostly all schemas can be
persisted (and read back) to EDN
• Works with all of clj (JVM & GraalVM) and cljs, no eval needed
• Big leverage in building truly dynamic systems
• Playground at malli.io

30. (def Adult
[:map {:registry {::age [:and int? [:> 18]]}}
[:age ::age]])
(-> Adult
(malli.edn/write-string)
(malli.edn/read-string)
(m/validate {:age 46}))
; => true

31. Programming
with Schemas

32. malli.util
• Schemas can be transformed, just like data
• Pre- & postwalk for generic walking over schemas
• update-properties, get, assoc, update, get-in, assoc-in, update-in, dissoc,
select-keys, …
• subschemas for
• Backed by the Schema & LensSchema protocols

35. Dev-
tooling

36. Function Schemas
• (Higher order) functions can also be described with Malli
• Any existing ClojureVar can be annotated with Malli Schemas
• Support multi-rarities with custom return values for each
• Does not support varargs (WIP, the sequence schemas)
• Integrates to CLJ-kondo, plumatic schema style inline schemas WIP
• Aave, Ghostwheel-port for Malli (https://github.com/teknql/aave)

37. CLJS
• Designed to be light, DCE can remove most of the unneeded core
• Starts from few kilos zipped (validation with minimal registry)

38. GraalVM
• Works on GraalVM too, binaries starting from 14mb

40. Malli 1.0.0

41. Malli 1.0.0
• Support for sequence schemas (WIP, Pauli)
• Support for Schema parsing
• Protocol-based Fast Schema Inferrer
• Describe Malli Schemas with Malli (recall spec-of-specs)
• Expound-grade pretty printer (WIP)
• Proper documentation
• Re-visit some design decision (including the global registry)

42. Wrap-up

43. Designing with Malli
• Where was the designing part??? ;)
• Malli in a clean slate fusion of the best parts of existing libraries
(plumatic schema and clojure.spec)
• In alpha, under active development
• Lot’s of ideas, so little (extra) time, help welcome
• Try reitit-malli to get started…