Wieldy remote apis	with
Kekkonen:ClojureD 2016
Tommi Reiman & JuhoTeperi
@ikitommi & @JuhoTeperi
???
Query,	Body,	
Path,	
Header,	Fom
-parameters
Status	codes
GET,	POST,	PUT,	
PATCH,	DELETE,	
HEAD,	OPTIONS
uris
Resources
functions,	
namespaces	
&	data
functions,	
namespaces	
&	data
http://netflix.github.io/falcor/starter/why-falcor.html
Possible options
• REST / web apis
– Beautiful apis,but for whom?
– Extra abstracting to HTTP terms –worth it?
• Just RPC
– Simple, is it open enough (for 3rd parties)?
• Data-driven(Falcor, Relay, Om Next)
– Separate reads & mutations,bundled operations
– Great for reads, controlled mutations? Maturity?
• Mix & match?
– Commands & Queries, great api-docs
– best parts of data-driven
Kekkonen
• Small library for generic message handling
– “Functions processing requests via a dispatcher”
• Data-driven, no macros, wieldy via helpers
• Uncoupled from HTTP/REST, just data & functions
– Ring-adapter with Swagger-docs
– Future:Web Sockets, messaging, command line, whatever, ...
• Key concepts:
– Basics: Context, Handler, Dispatcher, Namespace, Interceptor
– Adapters and APIs
Handlers 1/3
• Purpose is to validate & process contexts
• Internal representation is just data
{:name :plus
:type :handler
:description "Adds to numbers together"
:input {:data {:y s/Int
:x s/Int
s/Keyword s/Any}
s/Keyword s/Any}
:output s/Int
:handler (fn [{{:keys [x y]} :data}]
(+ x y))}
Handlers 3/3
• Plumbing fnk-notation with Schemas
(defnk ^:handler plus :- s/Int
"Adds to numbers together"
[[:data x - s/Int, y :- s/Int]]
(+ x y))
Dispatcher
• Registry of handlers
– Compiles handlers into dispatch-table & interceptor-chains
• “a better multimethod”
• Functions to work with handlers
– check, validate, invoke
– some-handler, all-handlers, available-handlers,
dispatch-handlers
(defnk ^:handler increment
"Stateful counter"
[counter]
(swap! counter inc))
(defnk ^:handler plus
"Adds two numbers together"
[[:data x - s/Int, y :- s/Int]]
(+ x y))
(def d (k/dispatcher
{:handlers {:math [#'increment #'plus]}
:context {:counter (atom 0)}}))
(k/invoke d :math/plus) ; CoerceionError {:data missing-required-key}
(k/invoke d :math/plus {:data {:x 1, :y 2}}) ; => 3
(k/invoke d :math/increment) ; => 1
(k/invoke d :math/increment) ; => 2
(k/invoke d :math/increment) ; => 3
(k/invoke d :math/increment {:counter (atom 41)}) ; => 42
Extending
• Custom meta-data to handlers & namespaces
– compile down to Interceptors
(defnk ^:command close-application
"Closes the application”
{:roles #{:applicant}
:states #{:open :draft}
:interceptors [notify-on-success]}
[db, [:data id :- s/Int]]
(success (application/close db id)))
But where’s my
WEB APIs?
Kebab time!
The cool stuff
• Validate handler input without executing body
• Handler(mass-)availabilitywith partial contexts
• Speculative transactions*
• Client-side bundled transactional contexts*
• Extract handler-data to clients for local reasoning*
• Safe and dynamic api-docs
• Command-logging
*	demo,	not	in	the	core	yet
complex simulated real-life case
example showcase project
https://github.com/lupapiste/lupapiste
Problem
• Digitalized building permits in
Finland
• Multiple roles using the app,
collaborating in real-time
– Single application
• Role-based authorization
• Audit-trail
UI DEMO
Actions DEMO
(defn create-api [{:keys [state chord]}]
(cqrs-api
{:swagger {:info {:title "Building Permit application"
:description "a complex simulated real-life
case example showcase project for http://kekkonen.io"}
:securityDefinitions {:api_key {:type "apiKey", :name "x-apikey", :in "header"}}}
:swagger-ui {:validator-url nil
:path "/api-docs"}
:core {:handlers {building-permit-ns 'backend.building-permit
users-ns 'backend.users
:session 'backend.session}
:user [[:require-session app-session/require-session]
[:load-current-user app-session/load-current-user]
[:requires-role app-session/requires-role]
[::building-permit/retrieve-permit building-permit/retrieve-permit]
[::building-permit/requires-state building-permit/requires-state]
[::building-permit/requires-claim building-permit/requires-claim]]
:context {:state state
:chord chord}}}))
Api
(defnk ^:command approve
"Approve a permit"
{:requires-role #{:authority}
::requires-claim true
::retrieve-permit true
::requires-state #{:submitted}
:interceptors [broadcast-update]}
[[:state permits archive-id-seq]
[:entities [:permit permit-id]]]
(swap! permits update permit-id assoc
:state :approved
:archive-id (swap! archive-id-seq inc))
(success {:status :ok}))
Command
Findings
• Action availability logic on backend
– Backend has all the facts, single query with Kekkonen
– Not all data can be sent to client for local reasoning
• Modelling commands based on user intent
– UI-actions map mostly 1:1 to api actions
– Automatic audit trail
Links
• https://github.com/metosin/kekkonen-building-
permit-example
• https://building-permits.herokuapp.com/
Next steps?
• Create handlers-trees from external sources / spec (db, file)
• Kekkonen overWebsockets
• (ClojureScript) Api-docs beyond Swagger
• Om Next Remotes
• ClojureScript client
• RE-Kekkonen
• CQRS-template with Eventing
• Handler mutations & hot-swapping
• Graph-based dependency management
• Pulsar-backend, extract api-docs, ping @andreiursan
• Hiccup-style syntax for namespace-trees
Summary
• Kekkonen is a fresh new api library for clj(s)
• Simple, data-driven, free from the http
– Your domain functions & data
• Enables cool new ways to interact with apis
• Get involved
– https://kekkonen.io & #kekkonen at Slack
Special thanks to
• Prismatic Schema & Plumbing
• Pedestal for Interceptors
• Elegance of fnhouse
• Ring-swagger
• Best parts of compojure-api
• Schema-tools
• Kebabs
Q&Ahttp://kekkonen.io
#kekkonen at	Slack
@metosin at	Twitter
join@metosin.fi
Extra	slides
https://www.youtube.com/watch?v=3oQTSP4FngY
Zach	Tellman - Always	Be	Composing
Context
• Execution context, client input under :data
• Otherwise works mostly like in Pedestal
; simple context
{:data {:x 1, :y 1}}
Handlers 2/3
• Clojure functions with extra meta-data
(defn plus
"Adds to numbers together"
{:type :handler
:input {:data {:y s/Int
:x s/Int
s/Keyword s/Any}
s/Keyword s/Any}
:output s/Int}
[{{:keys [x y]} :data}]
(+ x y))
(Virtual) Namespace
• Just like Clojure namespaces, but uncoupled to
allow internal refactoring
{:name :admin
:type :namespace
:description "Admin-operations"
:interceptors [[require-role :admin]]}
Interceptors
• Like middleware in Ring
• In the end, (mostly) everything is an interceptor
• Pedestal <3
{:name "logging interceptor"
:enter (fn [ctx] (log/info ctx) ctx)
:leave (fn [ctx] (log/info ctx) ctx)}
https://twitter.com/dr4goonis/status/476617165463105536
Interceptors
Ring-adapter
• Create a ring-handler from dispatcher & options
(def app
(r/ring-handler
(k/dispatcher
{:handlers {:math [#'increment #'plus]}
:context {:counter (atom 0)}})))
(app {:uri "/":request-method :get}) => nil
(app {:uri "/math/plus"
:request-method :post
:body-params {:x 1, :y 2}}) => 3
API
• Public http-entrypoint in Kekkonen
– Wires ring-adapter, middleware & swagger artifacts
– Ships with good defaults
(def app
(a/api
{:core {:handlers {:math [#'increment
#'plus]}
:context {:counter (atom 0)}}}))
(server/run-server #'app {:port 5000})
source code for api
(defn api [options]
(s/with-fn-validation
(let [options (s/validate Options (kc/deep-merge +default-options+ options))
swagger (merge (:swagger options) (mw/api-info (:mw options)))
dispatcher (-> (k/dispatcher (:core options))
(k/inject (-> options :api :handlers))
(k/inject (ks/swagger-handler swagger options)))]
(mw/wrap-api
(r/routes
[(r/ring-handler dispatcher (:ring options))
(ks/swagger-ui (:swagger-ui options))])
(:mw options)))))
Create your own api styles!
• New styles via Dispatcher & Api configuration
• Ships with RPC, HTTP and CQRS Api styles
(defn cqrs-api [options]
(a/api
(kc/deep-merge
{:core {:type-resolver (k/type-resolver :command :query)}
:swagger {:info {:title "Kekkonen CQRS API"}}
:ring {:types {:query {:methods #{:get}
:parameters {[:data] [:request :query-params]}}
:command {:methods #{:post}
:parameters {[:data] [:request :body-params]}}}}}
options)))
(s/defschema Kebab
{:id s/Int
:name s/Str
:type (s/enum :doner :shish :souvlaki)})
(s/defschema NewKebab
(dissoc Kebab :id))
(defnk ^:query get-kebabs
"Retrieves all kebabs"
{:output [Kebab]}
[db]
(success (vals @db)))
(defnk ^:command add-kebab
"Adds an kebab to database"
{:output Kebab}
[db, ids, data :- NewKebab]
(let [item (assoc data :id (swap! ids inc))]
(swap! db assoc (:id item) item)
(success item)))
(def app
(cqrs-api
{:core {:handlers {:kebabs [#'get-kebabs #'add-kebab]}
:context {:db (atom {})
:ids (atom 0)}}}))
(server/run-server #'app {:port 4001})

Wieldy remote apis with Kekkonen - ClojureD 2016

  • 1.
    Wieldy remote apis with Kekkonen:ClojureD2016 Tommi Reiman & JuhoTeperi @ikitommi & @JuhoTeperi
  • 2.
  • 3.
  • 4.
    Possible options • REST/ web apis – Beautiful apis,but for whom? – Extra abstracting to HTTP terms –worth it? • Just RPC – Simple, is it open enough (for 3rd parties)? • Data-driven(Falcor, Relay, Om Next) – Separate reads & mutations,bundled operations – Great for reads, controlled mutations? Maturity? • Mix & match? – Commands & Queries, great api-docs – best parts of data-driven
  • 5.
    Kekkonen • Small libraryfor generic message handling – “Functions processing requests via a dispatcher” • Data-driven, no macros, wieldy via helpers • Uncoupled from HTTP/REST, just data & functions – Ring-adapter with Swagger-docs – Future:Web Sockets, messaging, command line, whatever, ... • Key concepts: – Basics: Context, Handler, Dispatcher, Namespace, Interceptor – Adapters and APIs
  • 6.
    Handlers 1/3 • Purposeis to validate & process contexts • Internal representation is just data {:name :plus :type :handler :description "Adds to numbers together" :input {:data {:y s/Int :x s/Int s/Keyword s/Any} s/Keyword s/Any} :output s/Int :handler (fn [{{:keys [x y]} :data}] (+ x y))}
  • 7.
    Handlers 3/3 • Plumbingfnk-notation with Schemas (defnk ^:handler plus :- s/Int "Adds to numbers together" [[:data x - s/Int, y :- s/Int]] (+ x y))
  • 8.
    Dispatcher • Registry ofhandlers – Compiles handlers into dispatch-table & interceptor-chains • “a better multimethod” • Functions to work with handlers – check, validate, invoke – some-handler, all-handlers, available-handlers, dispatch-handlers
  • 9.
    (defnk ^:handler increment "Statefulcounter" [counter] (swap! counter inc)) (defnk ^:handler plus "Adds two numbers together" [[:data x - s/Int, y :- s/Int]] (+ x y)) (def d (k/dispatcher {:handlers {:math [#'increment #'plus]} :context {:counter (atom 0)}})) (k/invoke d :math/plus) ; CoerceionError {:data missing-required-key} (k/invoke d :math/plus {:data {:x 1, :y 2}}) ; => 3 (k/invoke d :math/increment) ; => 1 (k/invoke d :math/increment) ; => 2 (k/invoke d :math/increment) ; => 3 (k/invoke d :math/increment {:counter (atom 41)}) ; => 42
  • 10.
    Extending • Custom meta-datato handlers & namespaces – compile down to Interceptors (defnk ^:command close-application "Closes the application” {:roles #{:applicant} :states #{:open :draft} :interceptors [notify-on-success]} [db, [:data id :- s/Int]] (success (application/close db id)))
  • 11.
  • 12.
  • 13.
    The cool stuff •Validate handler input without executing body • Handler(mass-)availabilitywith partial contexts • Speculative transactions* • Client-side bundled transactional contexts* • Extract handler-data to clients for local reasoning* • Safe and dynamic api-docs • Command-logging * demo, not in the core yet
  • 14.
    complex simulated real-lifecase example showcase project https://github.com/lupapiste/lupapiste
  • 15.
    Problem • Digitalized buildingpermits in Finland • Multiple roles using the app, collaborating in real-time – Single application • Role-based authorization • Audit-trail
  • 16.
  • 17.
  • 18.
    (defn create-api [{:keys[state chord]}] (cqrs-api {:swagger {:info {:title "Building Permit application" :description "a complex simulated real-life case example showcase project for http://kekkonen.io"} :securityDefinitions {:api_key {:type "apiKey", :name "x-apikey", :in "header"}}} :swagger-ui {:validator-url nil :path "/api-docs"} :core {:handlers {building-permit-ns 'backend.building-permit users-ns 'backend.users :session 'backend.session} :user [[:require-session app-session/require-session] [:load-current-user app-session/load-current-user] [:requires-role app-session/requires-role] [::building-permit/retrieve-permit building-permit/retrieve-permit] [::building-permit/requires-state building-permit/requires-state] [::building-permit/requires-claim building-permit/requires-claim]] :context {:state state :chord chord}}})) Api
  • 19.
    (defnk ^:command approve "Approvea permit" {:requires-role #{:authority} ::requires-claim true ::retrieve-permit true ::requires-state #{:submitted} :interceptors [broadcast-update]} [[:state permits archive-id-seq] [:entities [:permit permit-id]]] (swap! permits update permit-id assoc :state :approved :archive-id (swap! archive-id-seq inc)) (success {:status :ok})) Command
  • 20.
    Findings • Action availabilitylogic on backend – Backend has all the facts, single query with Kekkonen – Not all data can be sent to client for local reasoning • Modelling commands based on user intent – UI-actions map mostly 1:1 to api actions – Automatic audit trail
  • 21.
  • 22.
    Next steps? • Createhandlers-trees from external sources / spec (db, file) • Kekkonen overWebsockets • (ClojureScript) Api-docs beyond Swagger • Om Next Remotes • ClojureScript client • RE-Kekkonen • CQRS-template with Eventing • Handler mutations & hot-swapping • Graph-based dependency management • Pulsar-backend, extract api-docs, ping @andreiursan • Hiccup-style syntax for namespace-trees
  • 23.
    Summary • Kekkonen isa fresh new api library for clj(s) • Simple, data-driven, free from the http – Your domain functions & data • Enables cool new ways to interact with apis • Get involved – https://kekkonen.io & #kekkonen at Slack
  • 24.
    Special thanks to •Prismatic Schema & Plumbing • Pedestal for Interceptors • Elegance of fnhouse • Ring-swagger • Best parts of compojure-api • Schema-tools • Kebabs
  • 25.
  • 26.
  • 27.
  • 28.
    Context • Execution context,client input under :data • Otherwise works mostly like in Pedestal ; simple context {:data {:x 1, :y 1}}
  • 29.
    Handlers 2/3 • Clojurefunctions with extra meta-data (defn plus "Adds to numbers together" {:type :handler :input {:data {:y s/Int :x s/Int s/Keyword s/Any} s/Keyword s/Any} :output s/Int} [{{:keys [x y]} :data}] (+ x y))
  • 30.
    (Virtual) Namespace • Justlike Clojure namespaces, but uncoupled to allow internal refactoring {:name :admin :type :namespace :description "Admin-operations" :interceptors [[require-role :admin]]}
  • 31.
    Interceptors • Like middlewarein Ring • In the end, (mostly) everything is an interceptor • Pedestal <3 {:name "logging interceptor" :enter (fn [ctx] (log/info ctx) ctx) :leave (fn [ctx] (log/info ctx) ctx)}
  • 32.
  • 33.
    Ring-adapter • Create aring-handler from dispatcher & options (def app (r/ring-handler (k/dispatcher {:handlers {:math [#'increment #'plus]} :context {:counter (atom 0)}}))) (app {:uri "/":request-method :get}) => nil (app {:uri "/math/plus" :request-method :post :body-params {:x 1, :y 2}}) => 3
  • 34.
    API • Public http-entrypointin Kekkonen – Wires ring-adapter, middleware & swagger artifacts – Ships with good defaults (def app (a/api {:core {:handlers {:math [#'increment #'plus]} :context {:counter (atom 0)}}})) (server/run-server #'app {:port 5000})
  • 35.
    source code forapi (defn api [options] (s/with-fn-validation (let [options (s/validate Options (kc/deep-merge +default-options+ options)) swagger (merge (:swagger options) (mw/api-info (:mw options))) dispatcher (-> (k/dispatcher (:core options)) (k/inject (-> options :api :handlers)) (k/inject (ks/swagger-handler swagger options)))] (mw/wrap-api (r/routes [(r/ring-handler dispatcher (:ring options)) (ks/swagger-ui (:swagger-ui options))]) (:mw options)))))
  • 36.
    Create your ownapi styles! • New styles via Dispatcher & Api configuration • Ships with RPC, HTTP and CQRS Api styles (defn cqrs-api [options] (a/api (kc/deep-merge {:core {:type-resolver (k/type-resolver :command :query)} :swagger {:info {:title "Kekkonen CQRS API"}} :ring {:types {:query {:methods #{:get} :parameters {[:data] [:request :query-params]}} :command {:methods #{:post} :parameters {[:data] [:request :body-params]}}}}} options)))
  • 37.
    (s/defschema Kebab {:id s/Int :names/Str :type (s/enum :doner :shish :souvlaki)}) (s/defschema NewKebab (dissoc Kebab :id)) (defnk ^:query get-kebabs "Retrieves all kebabs" {:output [Kebab]} [db] (success (vals @db))) (defnk ^:command add-kebab "Adds an kebab to database" {:output Kebab} [db, ids, data :- NewKebab] (let [item (assoc data :id (swap! ids inc))] (swap! db assoc (:id item) item) (success item))) (def app (cqrs-api {:core {:handlers {:kebabs [#'get-kebabs #'add-kebab]} :context {:db (atom {}) :ids (atom 0)}}})) (server/run-server #'app {:port 4001})