Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scrap your query boilerplate with specql

408 views

Published on

Slides from the talk I gave at ClojuTRE 2017 about the specql library.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Scrap your query boilerplate with specql

  1. 1. SCRAP YOUR QUERY BOILERPLATE WITH SPECQL 2.9.2017 ClojuTRE Tatu Tarvainen (@tatut)
  2. 2. WHAT IS SPECQL? › The combination of clojure.spec, PostgreSQL and the power of macros clojure.spec PostgreSQL λ
  3. 3. SOME BACKGROUND › Yesql/Jeesql/Hugsql are all fine and specql does not try to ”solve” SQL so that you don’t have to use it anymore › When you have multiple slightly different queries, there’s duplication or fetching too much › Adding dynamic WHERE clauses leads to ugly NULL checking of bound parameters
  4. 4. EXAMPLE › SELECT it.foo, it.bar, it.baz › FROM interesting_table it › WHERE it.archived = FALSE › AND it.organization = :user_org_id › AND it.category IN (:categories)
  5. 5. EXAMPLE › SELECT it.foo, it.bar, it.baz › FROM interesting_table it › WHERE it.archived = FALSE › AND it.organization = :user_org_id › AND it.category IN (:categories) How to make variants without duplication or selecting too much?
  6. 6. EXAMPLE › SELECT it.foo, it.bar, it.baz › FROM interesting_table it › WHERE it.archived = FALSE › AND it.organization = :user_org_id › AND it.category IN (:categories) How to add/remove clauses dynamically?
  7. 7. EXAMPLE › SELECT it.foo, it.bar, it.baz, › o.name AS organization_name › FROM interesting_table it › JOIN organization o ON it.organization_id = o.id › WHERE it.archived = FALSE › AND it.organization = :user_org_id › AND it.category IN (:categories)
  8. 8. EXAMPLE › SELECT it.foo, it.bar, it.baz, › o.name AS organization_name › FROM interesting_table it › JOIN organization o ON it.organization_id = o.id › WHERE it.archived = FALSE › AND it.organization = :user_org_id › AND it.category IN (:categories) How to cleanly add these without code duplication?
  9. 9. THE SPECQL WAY: DEFINE › (define-tables db › [”interesting_table” ::it/interesting › {::it/organization (rel/has-one › ::it/organization_id › ::org/organization › ::org/id)}])
  10. 10. THE SPECQL WAY: DEFINE › (define-tables db › [”interesting_table” ::it/interesting › {::it/organization (rel/has-one › ::it/organization_id › ::org/organization › ::org/id)}]) The main macro for defining things
  11. 11. THE SPECQL WAY: DEFINE › (define-tables db › [”interesting_table” ::it/interesting › {::it/organization (rel/has-one › ::it/organization_id › ::org/organization › ::org/id)}]) The name of the table in the database
  12. 12. THE SPECQL WAY: DEFINE › (define-tables db › [”interesting_table” ::it/interesting › {::it/organization (rel/has-one › ::it/organization_id › ::org/organization › ::org/id)}]) The namespaced keyword for the table
  13. 13. THE SPECQL WAY: DEFINE › (define-tables db › [”interesting_table” ::it/interesting › {::it/organization (rel/has-one › ::it/organization_id › ::org/organization › ::org/id)}]) Additional definitions for columns
  14. 14. THE SPECQL WAY: FETCH › (defn fetch-interesting [db org-id categories] › (fetch db ::it/interesting › #{::it/foo ::it/bar ::it/baz} › {::it/archived false › ::it/organization_id org-id › ::it/category (op/in categories)}))
  15. 15. THE SPECQL WAY: FETCH › (defn fetch-interesting [db org-id categories] › (fetch db ::it/interesting › #{::it/foo ::it/bar ::it/baz} › {::it/archived false › ::it/organization_id org-id › ::it/category (op/in categories)})) The table to fetch from
  16. 16. THE SPECQL WAY: FETCH › (defn fetch-interesting [db org-id categories] › (fetch db ::it/interesting › #{::it/foo ::it/bar ::it/baz} › {::it/archived false › ::it/organization_id org-id › ::it/category (op/in categories)})) Set of columns to retrieve
  17. 17. THE SPECQL WAY: FETCH › (defn fetch-interesting [db org-id categories] › (fetch db ::it/interesting › #{::it/foo ::it/bar ::it/baz} › {::it/archived false › ::it/organization_id org-id › ::it/category (op/in categories)})) The WHERE clause map
  18. 18. THE SPECQL WAY: RESULT › (fetch-interesting db 1 #{”stuff” ”things”}) › ;; => ({::it/foo 1 › ::it/bar ”example” › ::it/baz true} …) ›
  19. 19. THE SPECQL WAY: JOIN › (fetch db ::it/interesting › #{::it/foo ::it/bar ::it/baz › [::it/organization #{::org/id ::org/name}]} › {::it/archived false › ::it/organization_id org-id › ::it/category (op/in categories)}))
  20. 20. THE SPECQL WAY: JOIN › (fetch db ::it/interesting › #{::it/foo ::it/bar ::it/baz › [::it/organization #{::org/id ::org/name}]} › {::it/archived false › ::it/organization_id org-id › ::it/category (op/in categories)})) Nested column definition for a JOINed tables
  21. 21. THE SPECQL WAY: JOIN › ;; => ({::it/foo 1 › ::it/bar ”example” › ::it/baz true › ::it/organization {::org/id 1 › ::org/name ”Acme Inc”} › …)
  22. 22. THE SPECQL WAY: JOIN › ;; => ({::it/foo 1 › ::it/bar ”example” › ::it/baz true › ::it/organization {::org/id 1 › ::org/name ”Acme Inc”} › …) Joined entity is available in a nested map
  23. 23. SPECQL BENEFITS › Clojure data! › The column set is just data and can be easily manipulated • Even given as parameters from the frontend › Where clauses can be easily added and are (mostly) just data as well › Every table and column has a single ns keyword definition • No more ”order” vs ”order-id” vs ”ord” differences in returned query results • Namespaced keys are the way of the future
  24. 24. SPECQL BENEFITS › Clojure data! › The column set is just data and can be easily manipulated • Even given as parameters from the frontend › Where clauses can be easily added and are (mostly) just data as well › Every table and column has a single ns keyword definition • No more ”order” vs ”order-id” vs ”ord” differences in returned query results • Namespaced keys are the way of the future HERE TO STAY
  25. 25. SPECQL BENEFITS › Works with ClojureScript • The spec generation part, put your specs in .cljc files and enjoy the same specs and have your db definitions drive your frontend as well › Works well with ”typed document storage” pattern • A column can be a user defined type or array › Provides generic upsert! • No need to branch code, let the db handle it › Works with database VIEWs
  26. 26. STATUS › The github project page (https://github.com/tatut/specql) still says EXPERIMENTAL • Current 0.7 alpha stage, should be ready this year • We are already using it in production in Harja › No concrete promises before the experimental flag is gone, but the API should only grow • The test suite is comprehensive and should not break
  27. 27. STATUS › 0.7 is coming and adds support for stored procedures • Macro for defining a stored procedure as a function • Better JOIN handling • Currently fetching multiple ”has many” collections at once doesn’t work properly › How to help • Please try it out if you are using PostgreSQL • Report rough edges, I try to provide good error messages
  28. 28. › EXAMPLE
  29. 29. THANK YOU github.com/tatut/specql twitter.com/tatut

×