Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Exploring Four 
Datomic 
Superpowers 
Lucas Cavalcanti & Edward Wible
2 
Why build a bank 
from scratch 
in Brazil 
using Clojure and 
Datomic?
3
3
4 
1 Audit Trail
5 
event stream 
01 NOV 10:00 Customer joins waiting list for a card
5 
event stream 
01 NOV 11:00 Robot 437aae3 approves R$3K limit 
01 NOV 10:00 Customer joins waiting list for a card
5 
event stream 
09 NOV 08:00 Mastercard purchase, Starbucks, R$100 
01 NOV 11:00 Robot 437aae3 approves R$3K limit 
01 NO...
5 
event stream 
15 NOV 15:00 Support agent increases limit to R$5K 
09 NOV 08:00 Mastercard purchase, Starbucks, R$100 
0...
5 
event stream 
15 NOV 17:05 Customer blocks card 
15 NOV 15:00 Support agent increases limit to R$5K 
09 NOV 08:00 Maste...
facts over time 
6 
15 NOV 17:05 
15 NOV 15:00 
09 NOV 08:00 
01 NOV 11:00 
01 NOV 10:00 
[<card> :card/status :card.statu...
facts over time 
6 
15 NOV 17:05 
15 NOV 15:00 
09 NOV 08:00 
01 NOV 11:00 
01 NOV 10:00 
[<card> :card/status :card.statu...
7
7 
don’t 
lose data
7 
don’t 
lose data 
what 
changed 
when
7 
don’t 
lose data 
what 
changed 
when 
as-of 
since
7 
don’t 
lose data 
what 
changed 
when 
as-of 
since 
fork 
merge
8 
What was the initial limit for the card?
8 
What was the initial limit for the card? 
At the time the Starbucks transaction occurred, which 
fraud triggers would h...
8 
What was the initial limit for the card? 
At the time the Starbucks transaction occurred, which 
fraud triggers would h...
8 
What was the initial limit for the card? 
At the time the Starbucks transaction occurred, which 
fraud triggers would h...
9 
What sequence of 
events resulted in the 
Starbucks transaction 
being persisted?
9 
What sequence of 
events resulted in the 
Starbucks transaction 
being persisted? 
iOS 
http-in 
kafka-out 
kafka-in 
i...
9 
iOS 
http-in 
kafka-out 
kafka-in 
iZb 
iZb.jnA 
iZb.jnA.9Cd 
iZb.jnA.9Cd.l9A 
What sequence of 
events resulted in the...
Who was responsible 
for the credit limit 
increase? 
9 
iOS 
http-in 
kafka-out 
kafka-in 
iZb 
iZb.jnA 
iZb.jnA.9Cd 
iZb...
Who was responsible 
for the credit limit 
increase? 
9 
iOS 
http-in 
kafka-out 
kafka-in 
iZb 
iZb.jnA 
iZb.jnA.9Cd 
iZb...
Who was responsible 
for the credit limit 
increase? 
9 
iOS 
http-in 
kafka-out 
kafka-in 
iZb 
iZb.jnA 
iZb.jnA.9Cd 
iZb...
Who was responsible 
for the credit limit 
increase? 
9 
What sequence of 
events resulted in the 
Starbucks transaction 
...
Who was responsible 
for the credit limit 
increase? 
9 
What sequence of 
events resulted in the 
Starbucks transaction 
...
Who was responsible 
for the credit limit 
increase? 
9 
What sequence of 
events resulted in the 
Starbucks transaction 
...
(defn block-card [card reason] 
(d/transact 
(conn) 
[[:db/add card :card/status :card.status/blocked] 
])) 
10 
storing t...
(defn block-card [card reason] 
(d/transact 
(conn) 
[[:db/add card :card/status :card.status/blocked] 
{:db/id (d/tempid ...
11 
transactions over time with metadata 
[<card> :card/status :card.status/blocked] 
[<card> :card/status :card.status/ac...
11 
transactions over time with metadata 
[<card> :card/status :card.status/blocked] 
[<card> :card/status :card.status/ac...
12 
querying transaction metadata 
(defn who-blocked-the-card? [card] 
(d/q '{:find [?user ?cid ?tags] 
:in [$ ?status ?ca...
12 
querying transaction metadata 
(defn who-blocked-the-card? [card] 
(d/q '{:find [?user ?cid ?tags] 
:in [$ ?status ?ca...
12 
querying transaction metadata 
(defn who-blocked-the-card? [card] 
(d/q '{:find [?user ?cid ?tags] 
:in [$ ?status ?ca...
12 
querying transaction metadata 
(defn who-blocked-the-card? [card] 
(d/q '{:find [?user ?cid ?tags] 
:in [$ ?status ?ca...
12 
querying transaction metadata 
(defn who-blocked-the-card? [card] 
(d/q '{:find [?user ?cid ?tags] 
:in [$ ?status ?ca...
13 
2 Authorization
should we return data? 
GET /purchases/1337/comments
should we return data? 
GET /purchases/1337/comments 
200 OK 401 Unauthorized
customer 
account 
payment 
card purchase 
comment 
should we return data? 
GET /purchases/1337/comments 
200 OK 401 Unaut...
customer 
account 
payment 
card purchase 
comment 
should we return data? 
GET /purchases/1337/comments 
200 OK 401 Unaut...
customer 
account 
payment 
card purchase 
comment 
should we return data? 
GET /purchases/1337/comments 
200 OK 401 Unaut...
15 
explicit relationship traversal 
(defn owns? [customer-id purchase-id db] 
(d/q '{:find [?pur .] 
:in [$ ?customer-id ...
15 
explicit relationship traversal 
(defn owns? [customer-id purchase-id db] 
(d/q '{:find [?pur .] 
:in [$ ?customer-id ...
generic relationship traversal 
16 
recursive rule 
(logical OR)
generic relationship traversal 
16 
recursive rule 
(logical OR) 
(def owner-rules 
'[[(owns? ?cus-id ?e) 
[?e :customer/i...
generic relationship traversal 
16 
recursive rule 
(logical OR) 
(def owner-rules 
'[[(owns? ?cus-id ?e) 
[?e :customer/i...
generic relationship traversal 
16 
recursive rule 
(logical OR) 
(def owner-rules 
'[[(owns? ?cus-id ?e) 
[?e :customer/i...
generic relationship traversal 
16 
recursive rule 
(logical OR) 
(def owner-rules 
'[[(owns? ?cus-id ?e) 
[?e :customer/i...
generic relationship traversal 
16 
recursive rule 
(logical OR) 
(def owner-rules 
'[[(owns? ?cus-id ?e) 
[?e :customer/i...
generic relationship traversal 
16 
recursive rule 
(logical OR) 
(def owner-rules 
'[[(owns? ?cus-id ?e) 
[?e :customer/i...
generic relationship traversal 
16 
recursive rule 
(logical OR) 
query 
using rule 
(def owner-rules 
'[[(owns? ?cus-id ?...
generic relationship traversal 
:in [$ ?cus-id ?e %] 
:where [(owns? ?cus-id ?e)]} 
db customer-id entity 
owner-rules)) 
...
generic relationship traversal 
:in [$ ?cus-id ?e %] 
:where [(owns? ?cus-id ?e)]} 
db customer-id entity 
owner-rules)) 
...
17 
Recursion rules! 
But can we go further? 
Can we prevent queries from 
returning entities owned by 
other customers?
all owned entities 
18 
query binding 
attribute
all owned entities 
18 
query binding 
attribute 
(defn owns? [customer-id entity db] 
(d/q '{:find [?e .] 
:in [$ ?cus-id...
all owned entities 
18 
query binding 
attribute 
(defn owns? [customer-id entity db] 
(d/q '{:find [?e .] 
:in [$ ?cus-id...
all owned entities 
18 
query binding 
attribute 
attribute 
unbound 
(defn owns? [customer-id entity db] 
(d/q '{:find [?...
all owned entities 
18 
query binding 
attribute 
attribute 
unbound 
(defn owns? [customer-id entity db] 
(d/q '{:find [?...
all owned entities 
18 
query binding 
attribute 
attribute 
unbound 
(defn owns? [customer-id entity db] 
(d/q '{:find [?...
all owned entities 
18 
query binding 
attribute 
attribute 
unbound 
(defn owns? [customer-id entity db] 
(d/q '{:find [?...
filtered db (as value) 
19 
(defn owned-db [customer-id db] 
(let [owned (set (owned-entities customer-id db))] 
(d/filter...
filtered db (as value) 
19 
(defn owned-db [customer-id db] 
(let [owned (set (owned-entities customer-id db))] 
(d/filter...
filtered db (as value) 
19 
(defn owned-db [customer-id db] 
(let [owned (set (owned-entities customer-id db))] 
(d/filter...
filtered db (as value) 
19 
(defn owned-db [customer-id db] 
(let [owned (set (owned-entities customer-id db))] 
(d/filter...
filtered db (as value) 
19 
(defn owned-db [customer-id db] 
(let [owned (set (owned-entities customer-id db))] 
(d/filter...
filtered db (as value) 
(let [owned (set (owned-entities customer-id db))] 
(d/filter db 
19 
(defn owned-db [customer-id ...
passing a filtered db is transparent for queries 
20 
(defn all-purchases [account-id db] 
(d/q '{:find [[(pull ?p [*]) .....
passing a filtered db is transparent for queries 
20 
(defn all-purchases [account-id db] 
(d/q '{:find [[(pull ?p [*]) .....
21 
We’re Mobile! 
Bandwidth is costly
22 
3 HTTP Cache
23 
http last modified header 
GET /accounts/1234/purchases 
200 OK 
Last-Modified: Fri, 14 Nov 2014 14:28:50 UTC 
{"purch...
http last modified header 
If-Modified-Since: Fri, 14 Nov 2014 14:28:50 UTC 
GET /accounts/1234/purchases 
304 Not Modifie...
24 
How to keep track of a 
good last-modified date 
for a URL?
24 
How to keep track of a 
good last-modified date 
for a URL? 
The last time any owned 
entity changed!
If no customer-owned data changed, 304 
25 
(defn last-modified [customer-id db] 
(d/q '{:find [(max ?time) .] 
:in [$ ?cu...
If no customer-owned data changed, 304 
25 
(defn last-modified [customer-id db] 
(d/q '{:find [(max ?time) .] 
:in [$ ?cu...
If no customer-owned data changed, 304 
25 
(defn last-modified [customer-id db] 
(d/q '{:find [(max ?time) .] 
:in [$ ?cu...
If no customer-owned data changed, 304 
25 
(defn last-modified [customer-id db] 
(d/q '{:find [(max ?time) .] 
:in [$ ?cu...
If no customer-owned data changed, 304 
25 
(defn last-modified [customer-id db] 
(d/q '{:find [(max ?time) .] 
:in [$ ?cu...
If no customer-owned data changed, 304 
25 
(defn last-modified [customer-id db] 
(d/q '{:find [(max ?time) .] 
:in [$ ?cu...
If no customer-owned data changed, 304 
25 
(defn last-modified [customer-id db] 
(d/q '{:find [(max ?time) .] 
:in [$ ?cu...
If no customer-owned data changed, 304 
25 
(defn last-modified [customer-id db] 
(d/q '{:find [(max ?time) .] 
:in [$ ?cu...
If no customer-owned data changed, 304 
25 
(defn last-modified [customer-id db] 
(d/q '{:find [(max ?time) .] 
:in [$ ?cu...
26 
Cache hits are awesome, 
but what about cache 
misses?
27 
4 Mobile Sync
desired hypermedia from the API 
28 
GET /accounts/1223/purchases 
200 OK 
{"purchases": [...] 
"_links": { 
"updates": "h...
serve purchases added or modified since our last sync 
29 
(defn updated-purchases [account-id db since] 
(d/q '{:find [[(...
serve purchases added or modified since our last sync 
29 
(defn updated-purchases [account-id db since] 
(d/q '{:find [[(...
serve purchases added or modified since our last sync 
29 
(defn updated-purchases [account-id db since] 
(d/q '{:find [[(...
serve purchases added or modified since our last sync 
29 
(defn updated-purchases [account-id db since] 
(d/q '{:find [[(...
serve purchases added or modified since our last sync 
29 
(defn updated-purchases [account-id db since] 
(d/q '{:find [[(...
serve purchases added or modified since our last sync 
29 
(defn updated-purchases [account-id db since] 
(d/q '{:find [[(...
serve purchases added or modified since our last sync 
29 
(defn updated-purchases [account-id db since] 
(d/q '{:find [[(...
30 
Bonus
31 
5 Future DBs
generating a virtual db 
(defn db-with-virtual-charges [purchases db] 
(:db-after 
(d/with db 
(mapcat purchase->virtual-c...
generating a virtual db 
(defn db-with-virtual-charges [purchases db] 
(:db-after 
(d/with db 
(mapcat purchase->virtual-c...
generating a virtual db 
(defn db-with-virtual-charges [purchases db] 
(:db-after 
(d/with db 
(mapcat purchase->virtual-c...
generating a virtual db 
(defn db-with-virtual-charges [purchases db] 
(:db-after 
(d/with db 
(mapcat purchase->virtual-c...
generating a virtual db 
(defn db-with-virtual-charges [purchases db] 
(:db-after 
(d/with db 
(mapcat purchase->virtual-c...
33 
6 Testing
db testing - locally scoped 
34 
(defn virtual-db [updates db] 
(:db-after 
(d/with db updates))) 
(let [one-account {...}...
db testing - locally scoped 
34 
(defn virtual-db [updates db] 
(:db-after 
(d/with db updates))) 
(let [one-account {...}...
db testing - locally scoped 
34 
(defn virtual-db [updates db] 
(:db-after 
(d/with db updates))) 
(let [one-account {...}...
db testing - locally scoped 
34 
(defn virtual-db [updates db] 
(:db-after 
(d/with db updates))) 
(let [one-account {...}...
db testing - locally scoped 
34 
(defn virtual-db [updates db] 
(:db-after 
(d/with db updates))) 
(let [one-account {...}...
db testing - locally scoped 
34 
(defn virtual-db [updates db] 
(:db-after 
(d/with db updates))) 
(let [one-account {...}...
db testing - vector of datoms 
35 
(fact "on last-modification-for" 
(let [customer-id (uuid) 
db [[42 :account/customer-i...
db testing - vector of datoms 
35 
(fact "on last-modification-for" 
(let [customer-id (uuid) 
db [[42 :account/customer-i...
36 
7 Schema Extension
schema is data! 
37 
(d/transact conn [{:db/valueType :db.type/string, 
:db.install/_attribute :db.part/db, 
:db/id (d/tem...
:nubank/transform :pii 
schema is data! 
37 
(d/transact conn [{:db/valueType :db.type/string, 
:db.install/_attribute :db...
38 
8 Sharding Reads
39 
sharding to the transactor 
customers 
accounts 
notification 
acquisition 
processor 
auth
39 
sharding to the transactor 
customers 
accounts 
notification 
acquisition 
processor 
auth
customers 
1 2 … 
39 
sharding to the transactor 
customers 
accounts 
notification 
acquisition 
processor 
auth
customers 
1 
2 … 
39 
sharding to the transactor 
customers 
accounts 
notification 
acquisition 
processor 
auth 
shard-...
customers 
1 
2 
… 
39 
sharding to the transactor 
customers 
accounts 
notification 
acquisition 
processor 
auth 
shard...
40 
9 Db Aggregation
41 
data science aggregation 
customers 
accounts 
notification 
acquisition 
processor 
auth 
data 
science
42 
querying multiple databases 
(defn multi-join [customer-id cus-db acc-db acq-db ntf-db] 
(d/q '{:find [...] 
:in [$cus...
42 
querying multiple databases 
(defn multi-join [customer-id cus-db acc-db acq-db ntf-db] 
(d/q '{:find [...] 
:in [$cus...
43 
1 Audit Trail 
2 Authorization 
3 HTTP Cache 
4 Mobile Sync 
5 Future DBs 
6 Testing 
7 Schema Extension 
8 Sharding R...
44 
We’re hiring
45 
We’re hiring
Thanks! 
Lucas Cavalcanti & Edward Wible 
@lucascs 
lucas@nubank.com.br 
edward@nubank.com.br
Upcoming SlideShare
Loading in …5
×

of

Exploring four Datomic superpowers Slide 1 Exploring four Datomic superpowers Slide 2 Exploring four Datomic superpowers Slide 3 Exploring four Datomic superpowers Slide 4 Exploring four Datomic superpowers Slide 5 Exploring four Datomic superpowers Slide 6 Exploring four Datomic superpowers Slide 7 Exploring four Datomic superpowers Slide 8 Exploring four Datomic superpowers Slide 9 Exploring four Datomic superpowers Slide 10 Exploring four Datomic superpowers Slide 11 Exploring four Datomic superpowers Slide 12 Exploring four Datomic superpowers Slide 13 Exploring four Datomic superpowers Slide 14 Exploring four Datomic superpowers Slide 15 Exploring four Datomic superpowers Slide 16 Exploring four Datomic superpowers Slide 17 Exploring four Datomic superpowers Slide 18 Exploring four Datomic superpowers Slide 19 Exploring four Datomic superpowers Slide 20 Exploring four Datomic superpowers Slide 21 Exploring four Datomic superpowers Slide 22 Exploring four Datomic superpowers Slide 23 Exploring four Datomic superpowers Slide 24 Exploring four Datomic superpowers Slide 25 Exploring four Datomic superpowers Slide 26 Exploring four Datomic superpowers Slide 27 Exploring four Datomic superpowers Slide 28 Exploring four Datomic superpowers Slide 29 Exploring four Datomic superpowers Slide 30 Exploring four Datomic superpowers Slide 31 Exploring four Datomic superpowers Slide 32 Exploring four Datomic superpowers Slide 33 Exploring four Datomic superpowers Slide 34 Exploring four Datomic superpowers Slide 35 Exploring four Datomic superpowers Slide 36 Exploring four Datomic superpowers Slide 37 Exploring four Datomic superpowers Slide 38 Exploring four Datomic superpowers Slide 39 Exploring four Datomic superpowers Slide 40 Exploring four Datomic superpowers Slide 41 Exploring four Datomic superpowers Slide 42 Exploring four Datomic superpowers Slide 43 Exploring four Datomic superpowers Slide 44 Exploring four Datomic superpowers Slide 45 Exploring four Datomic superpowers Slide 46 Exploring four Datomic superpowers Slide 47 Exploring four Datomic superpowers Slide 48 Exploring four Datomic superpowers Slide 49 Exploring four Datomic superpowers Slide 50 Exploring four Datomic superpowers Slide 51 Exploring four Datomic superpowers Slide 52 Exploring four Datomic superpowers Slide 53 Exploring four Datomic superpowers Slide 54 Exploring four Datomic superpowers Slide 55 Exploring four Datomic superpowers Slide 56 Exploring four Datomic superpowers Slide 57 Exploring four Datomic superpowers Slide 58 Exploring four Datomic superpowers Slide 59 Exploring four Datomic superpowers Slide 60 Exploring four Datomic superpowers Slide 61 Exploring four Datomic superpowers Slide 62 Exploring four Datomic superpowers Slide 63 Exploring four Datomic superpowers Slide 64 Exploring four Datomic superpowers Slide 65 Exploring four Datomic superpowers Slide 66 Exploring four Datomic superpowers Slide 67 Exploring four Datomic superpowers Slide 68 Exploring four Datomic superpowers Slide 69 Exploring four Datomic superpowers Slide 70 Exploring four Datomic superpowers Slide 71 Exploring four Datomic superpowers Slide 72 Exploring four Datomic superpowers Slide 73 Exploring four Datomic superpowers Slide 74 Exploring four Datomic superpowers Slide 75 Exploring four Datomic superpowers Slide 76 Exploring four Datomic superpowers Slide 77 Exploring four Datomic superpowers Slide 78 Exploring four Datomic superpowers Slide 79 Exploring four Datomic superpowers Slide 80 Exploring four Datomic superpowers Slide 81 Exploring four Datomic superpowers Slide 82 Exploring four Datomic superpowers Slide 83 Exploring four Datomic superpowers Slide 84 Exploring four Datomic superpowers Slide 85 Exploring four Datomic superpowers Slide 86 Exploring four Datomic superpowers Slide 87 Exploring four Datomic superpowers Slide 88 Exploring four Datomic superpowers Slide 89 Exploring four Datomic superpowers Slide 90 Exploring four Datomic superpowers Slide 91 Exploring four Datomic superpowers Slide 92 Exploring four Datomic superpowers Slide 93 Exploring four Datomic superpowers Slide 94 Exploring four Datomic superpowers Slide 95 Exploring four Datomic superpowers Slide 96 Exploring four Datomic superpowers Slide 97 Exploring four Datomic superpowers Slide 98 Exploring four Datomic superpowers Slide 99 Exploring four Datomic superpowers Slide 100 Exploring four Datomic superpowers Slide 101 Exploring four Datomic superpowers Slide 102 Exploring four Datomic superpowers Slide 103 Exploring four Datomic superpowers Slide 104 Exploring four Datomic superpowers Slide 105 Exploring four Datomic superpowers Slide 106 Exploring four Datomic superpowers Slide 107 Exploring four Datomic superpowers Slide 108 Exploring four Datomic superpowers Slide 109 Exploring four Datomic superpowers Slide 110 Exploring four Datomic superpowers Slide 111 Exploring four Datomic superpowers Slide 112 Exploring four Datomic superpowers Slide 113 Exploring four Datomic superpowers Slide 114 Exploring four Datomic superpowers Slide 115 Exploring four Datomic superpowers Slide 116 Exploring four Datomic superpowers Slide 117 Exploring four Datomic superpowers Slide 118 Exploring four Datomic superpowers Slide 119 Exploring four Datomic superpowers Slide 120 Exploring four Datomic superpowers Slide 121 Exploring four Datomic superpowers Slide 122 Exploring four Datomic superpowers Slide 123 Exploring four Datomic superpowers Slide 124 Exploring four Datomic superpowers Slide 125 Exploring four Datomic superpowers Slide 126 Exploring four Datomic superpowers Slide 127 Exploring four Datomic superpowers Slide 128 Exploring four Datomic superpowers Slide 129 Exploring four Datomic superpowers Slide 130 Exploring four Datomic superpowers Slide 131
Upcoming SlideShare
Low latency in java 8 by Peter Lawrey
Next
Download to read offline and view in fullscreen.

15 Likes

Share

Download to read offline

Exploring four Datomic superpowers

Download to read offline

https://www.youtube.com/watch?v=7lm3K8zVOdY

This session will explore four common problems, and the unique and surprising tools Datomic provides to solve them elegantly: HTTP caching - How to generically generate and validate Last-Modified and If-Modified-Since headers Audit trail - how to extend Datomic’s immutable transaction log to include arbitrary audit related metadata Mobile database sync - trivial implementation of an incremental update API for high latency/low bandwidth clients Authorization - easily determine resource ownership, and centrally isolate users from data they are not allowed to see

These problems have certainly been solved before using other databases, but Datomic provides features that make the proposed implementations concise, generic, and purely functional

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Exploring four Datomic superpowers

  1. 1. Exploring Four Datomic Superpowers Lucas Cavalcanti & Edward Wible
  2. 2. 2 Why build a bank from scratch in Brazil using Clojure and Datomic?
  3. 3. 3
  4. 4. 3
  5. 5. 4 1 Audit Trail
  6. 6. 5 event stream 01 NOV 10:00 Customer joins waiting list for a card
  7. 7. 5 event stream 01 NOV 11:00 Robot 437aae3 approves R$3K limit 01 NOV 10:00 Customer joins waiting list for a card
  8. 8. 5 event stream 09 NOV 08:00 Mastercard purchase, Starbucks, R$100 01 NOV 11:00 Robot 437aae3 approves R$3K limit 01 NOV 10:00 Customer joins waiting list for a card
  9. 9. 5 event stream 15 NOV 15:00 Support agent increases limit to R$5K 09 NOV 08:00 Mastercard purchase, Starbucks, R$100 01 NOV 11:00 Robot 437aae3 approves R$3K limit 01 NOV 10:00 Customer joins waiting list for a card
  10. 10. 5 event stream 15 NOV 17:05 Customer blocks card 15 NOV 15:00 Support agent increases limit to R$5K 09 NOV 08:00 Mastercard purchase, Starbucks, R$100 01 NOV 11:00 Robot 437aae3 approves R$3K limit 01 NOV 10:00 Customer joins waiting list for a card
  11. 11. facts over time 6 15 NOV 17:05 15 NOV 15:00 09 NOV 08:00 01 NOV 11:00 01 NOV 10:00 [<card> :card/status :card.status/blocked] [<card> :card/status :card.status/active] [<account> :account/limit 5000] [<account> :account/limit 3000] [<purchase> :purchase/card <card>] [<purchase> :purchase/amount 100] [<purchase> :purchase/merchant “Starbucks”] [<account> :account/customer <customer>] [<account> :account/limit 3000] [<card> :card/account <account>] [<card> :card/status :card.status/active] [<customer> :customer/id #uuid “b2c90…”]
  12. 12. facts over time 6 15 NOV 17:05 15 NOV 15:00 09 NOV 08:00 01 NOV 11:00 01 NOV 10:00 [<card> :card/status :card.status/blocked] [<card> :card/status :card.status/active] [<account> :account/limit 5000] [<account> :account/limit 3000] [<purchase> :purchase/card <card>] [<purchase> :purchase/amount 100] [<purchase> :purchase/merchant “Starbucks”] [<account> :account/customer <customer>] [<account> :account/limit 3000] [<card> :card/account <account>] [<card> :card/status :card.status/active] [<customer> :customer/id #uuid “b2c90…”] entity attribute value
  13. 13. 7
  14. 14. 7 don’t lose data
  15. 15. 7 don’t lose data what changed when
  16. 16. 7 don’t lose data what changed when as-of since
  17. 17. 7 don’t lose data what changed when as-of since fork merge
  18. 18. 8 What was the initial limit for the card?
  19. 19. 8 What was the initial limit for the card? At the time the Starbucks transaction occurred, which fraud triggers would have activated?
  20. 20. 8 What was the initial limit for the card? At the time the Starbucks transaction occurred, which fraud triggers would have activated? How long did the customer spend on each stage of the acquisition funnel?
  21. 21. 8 What was the initial limit for the card? At the time the Starbucks transaction occurred, which fraud triggers would have activated? How long did the customer spend on each stage of the acquisition funnel? How frequently do we see amount changes on Starbucks transactions?
  22. 22. 9 What sequence of events resulted in the Starbucks transaction being persisted?
  23. 23. 9 What sequence of events resulted in the Starbucks transaction being persisted? iOS http-in kafka-out kafka-in iZb iZb.jnA iZb.jnA.9Cd iZb.jnA.9Cd.l9A
  24. 24. 9 iOS http-in kafka-out kafka-in iZb iZb.jnA iZb.jnA.9Cd iZb.jnA.9Cd.l9A What sequence of events resulted in the Starbucks transaction being persisted? correlation-id
  25. 25. Who was responsible for the credit limit increase? 9 iOS http-in kafka-out kafka-in iZb iZb.jnA iZb.jnA.9Cd iZb.jnA.9Cd.l9A What sequence of events resulted in the Starbucks transaction being persisted? correlation-id
  26. 26. Who was responsible for the credit limit increase? 9 iOS http-in kafka-out kafka-in iZb iZb.jnA iZb.jnA.9Cd iZb.jnA.9Cd.l9A What sequence of events resulted in the Starbucks transaction being persisted? correlation-id #uuid “b2c90…” “lucas@nubank.com.br” :kafka-LIMIT-CHANGED :robot-437aae3
  27. 27. Who was responsible for the credit limit increase? 9 iOS http-in kafka-out kafka-in iZb iZb.jnA iZb.jnA.9Cd iZb.jnA.9Cd.l9A What sequence of events resulted in the Starbucks transaction being persisted? correlation-id user #uuid “b2c90…” “lucas@nubank.com.br” :kafka-LIMIT-CHANGED :robot-437aae3
  28. 28. Who was responsible for the credit limit increase? 9 What sequence of events resulted in the Starbucks transaction being persisted? Why was the customer’s card blocked? correlation-id iOS http-in kafka-out kafka-in iZb iZb.jnA iZb.jnA.9Cd iZb.jnA.9Cd.l9A user #uuid “b2c90…” “lucas@nubank.com.br” :kafka-LIMIT-CHANGED :robot-437aae3
  29. 29. Who was responsible for the credit limit increase? 9 What sequence of events resulted in the Starbucks transaction being persisted? Why was the customer’s card blocked? correlation-id iOS http-in kafka-out kafka-in iZb iZb.jnA iZb.jnA.9Cd iZb.jnA.9Cd.l9A :fraud-preventative :recurring-scheduled :late-payment :data-migration user #uuid “b2c90…” “lucas@nubank.com.br” :kafka-LIMIT-CHANGED :robot-437aae3
  30. 30. Who was responsible for the credit limit increase? 9 What sequence of events resulted in the Starbucks transaction being persisted? Why was the customer’s card blocked? correlation-id iOS http-in kafka-out kafka-in iZb iZb.jnA iZb.jnA.9Cd iZb.jnA.9Cd.l9A user #uuid “b2c90…” “lucas@nubank.com.br” :kafka-LIMIT-CHANGED :robot-437aae3 :fraud-preventative reason :recurring-scheduled :late-payment :data-migration
  31. 31. (defn block-card [card reason] (d/transact (conn) [[:db/add card :card/status :card.status/blocked] ])) 10 storing transaction metadata
  32. 32. (defn block-card [card reason] (d/transact (conn) [[:db/add card :card/status :card.status/blocked] {:db/id (d/tempid :db.part/tx) :audit/user "lucas@nubank.com.br" :audit/cid "iZb.jnA.9Cd.l9A" :audit/tags :fraud-preventative} ])) 10 storing transaction metadata
  33. 33. 11 transactions over time with metadata [<card> :card/status :card.status/blocked] [<card> :card/status :card.status/active] 15 NOV 17:05 [<account> :account/limit 5000M] 15 NOV 15:00 [<account> :account/limit 3000M] [<purchase> :purchase/card <card>] [<purchase> :purchase/amount 100M] [<purchase> :purchase/merchant “Starbucks”] 09 NOV 08:00 [<account> :account/customer <customer>] [<account> :account/limit 3000M] [<card> :card/account <account>] [<card> :card/status :card.status/active] 01 NOV 11:00 01 NOV 10:00 [<customer> :customer/id #uuid “b2c90…”]
  34. 34. 11 transactions over time with metadata [<card> :card/status :card.status/blocked] [<card> :card/status :card.status/active] 15 NOV 17:05 [<account> :account/limit 5000M] 15 NOV 15:00 [<account> :account/limit 3000M] [<purchase> :purchase/card <card>] [<purchase> :purchase/amount 100M] [<purchase> :purchase/merchant “Starbucks”] 09 NOV 08:00 [<account> :account/customer <customer>] [<account> :account/limit 3000M] [<card> :card/account <account>] [<card> :card/status :card.status/active] 01 NOV 11:00 01 NOV 10:00 [<customer> :customer/id #uuid “b2c90…”]
  35. 35. 12 querying transaction metadata (defn who-blocked-the-card? [card] (d/q '{:find [?user ?cid ?tags] :in [$ ?status ?card] :where [[?card :card/status ?status ?tx] [?tx :audit/user ?user] [?tx :audit/cid ?cid] [?tx :audit/tags ?tags]]} db :card.status/blocked card))
  36. 36. 12 querying transaction metadata (defn who-blocked-the-card? [card] (d/q '{:find [?user ?cid ?tags] :in [$ ?status ?card] :where [[?entity attribute value txn [?card :card/status ?status ?tx] [?tx :audit/user ?user] [?tx :audit/cid ?cid] [?tx :audit/tags ?tags]]} db :card.status/blocked card))
  37. 37. 12 querying transaction metadata (defn who-blocked-the-card? [card] (d/q '{:find [?user ?cid ?tags] :in [$ ?status ?card] :where [[?card :card/status ?status ?tx] [?tx :audit/user ?user] [?tx :audit/cid ?cid] [?tx :audit/tags ?tags]]} db :card.status/blocked card))
  38. 38. 12 querying transaction metadata (defn who-blocked-the-card? [card] (d/q '{:find [?user ?cid ?tags] :in [$ ?status ?card] :where [[?card :card/status ?status ?tx] [?tx :audit/user ?user] [?tx :audit/cid ?cid] [?tx :audit/tags ?tags]]} db :card.status/blocked card))
  39. 39. 12 querying transaction metadata (defn who-blocked-the-card? [card] (d/q '{:find [?user ?cid ?tags] :in [$ ?status ?card] :where [[?card :card/status ?status ?tx] [?tx :audit/user ?user] [?tx :audit/cid ?cid] [?tx :audit/tags ?tags]]} db :card.status/blocked card))
  40. 40. 13 2 Authorization
  41. 41. should we return data? GET /purchases/1337/comments
  42. 42. should we return data? GET /purchases/1337/comments 200 OK 401 Unauthorized
  43. 43. customer account payment card purchase comment should we return data? GET /purchases/1337/comments 200 OK 401 Unauthorized
  44. 44. customer account payment card purchase comment should we return data? GET /purchases/1337/comments 200 OK 401 Unauthorized
  45. 45. customer account payment card purchase comment should we return data? GET /purchases/1337/comments 200 OK 401 Unauthorized
  46. 46. 15 explicit relationship traversal (defn owns? [customer-id purchase-id db] (d/q '{:find [?pur .] :in [$ ?customer-id ?purchase-id] :where [[?pur :purchase/id ?purchase-id] [?pur :purchase/account ?acc] [?acc :account/customer ?cus] [?cus :customer/id ?customer-id]]} db customer-id purchase-id))
  47. 47. 15 explicit relationship traversal (defn owns? [customer-id purchase-id db] (d/q '{:find [?pur .] :in [$ ?customer-id ?purchase-id] :where [[?pur :purchase/id ?purchase-id] [?pur :purchase/account ?acc] [?acc :account/customer ?cus] [?cus :customer/id ?customer-id]]} db customer-id purchase-id))
  48. 48. generic relationship traversal 16 recursive rule (logical OR)
  49. 49. generic relationship traversal 16 recursive rule (logical OR) (def owner-rules '[[(owns? ?cus-id ?e) [?e :customer/id ?cus-id]] [(owns? ?cus-id ?e) [?e ?ref-attr ?r] (owns? ?cus-id ?r)]])
  50. 50. generic relationship traversal 16 recursive rule (logical OR) (def owner-rules '[[(owns? ?cus-id ?e) [?e :customer/id ?cus-id]] [(owns? ?cus-id ?e) [?e ?ref-attr ?r] (owns? ?cus-id ?r)]])
  51. 51. generic relationship traversal 16 recursive rule (logical OR) (def owner-rules '[[(owns? ?cus-id ?e) [?e :customer/id ?cus-id]] [(owns? ?cus-id ?e) [?e ?ref-attr ?r] (owns? ?cus-id ?r)]])
  52. 52. generic relationship traversal 16 recursive rule (logical OR) (def owner-rules '[[(owns? ?cus-id ?e) [?e :customer/id ?cus-id]] [(owns? ?cus-id ?e) [?e ?ref-attr ?r] (owns? ?cus-id ?r)]])
  53. 53. generic relationship traversal 16 recursive rule (logical OR) (def owner-rules '[[(owns? ?cus-id ?e) [?e :customer/id ?cus-id]] [(owns? ?cus-id ?e) [?e ?ref-attr ?r] (owns? ?cus-id ?r)]])
  54. 54. generic relationship traversal 16 recursive rule (logical OR) (def owner-rules '[[(owns? ?cus-id ?e) [?e :customer/id ?cus-id]] [(owns? ?cus-id ?e) [?e ?ref-attr ?r] (owns? ?cus-id ?r)]])
  55. 55. generic relationship traversal 16 recursive rule (logical OR) query using rule (def owner-rules '[[(owns? ?cus-id ?e) [?e :customer/id ?cus-id]] [(owns? ?cus-id ?e) [?e ?ref-attr ?r] (owns? ?cus-id ?r)]])
  56. 56. generic relationship traversal :in [$ ?cus-id ?e %] :where [(owns? ?cus-id ?e)]} db customer-id entity owner-rules)) 16 recursive rule (logical OR) query using rule (def owner-rules '[[(owns? ?cus-id ?e) [?e :customer/id ?cus-id]] [(owns? ?cus-id ?e) [?e ?ref-attr ?r] (owns? ?cus-id ?r)]]) (defn owns? [customer-id entity db] (d/q '{:find [?e .] (owns? customer-id [:purchase/id id] db)
  57. 57. generic relationship traversal :in [$ ?cus-id ?e %] :where [(owns? ?cus-id ?e)]} db customer-id entity owner-rules)) 16 recursive rule (logical OR) query using rule (def owner-rules '[[(owns? ?cus-id ?e) [?e :customer/id ?cus-id]] [(owns? ?cus-id ?e) [?e ?ref-attr ?r] (owns? ?cus-id ?r)]]) (defn owns? [customer-id entity db] (d/q '{:find [?e .] (owns? customer-id [:purchase/id id] db)
  58. 58. 17 Recursion rules! But can we go further? Can we prevent queries from returning entities owned by other customers?
  59. 59. all owned entities 18 query binding attribute
  60. 60. all owned entities 18 query binding attribute (defn owns? [customer-id entity db] (d/q '{:find [?e .] :in [$ ?cus-id ?e %] :where [(owns? ?cus-id ?e)]} db customer-id entity] owner-rules))
  61. 61. all owned entities 18 query binding attribute (defn owns? [customer-id entity db] (d/q '{:find [?e .] :in [$ ?cus-id ?e %] :where [(owns? ?cus-id ?e)]} db customer-id entity] owner-rules))
  62. 62. all owned entities 18 query binding attribute attribute unbound (defn owns? [customer-id entity db] (d/q '{:find [?e .] :in [$ ?cus-id ?e %] :where [(owns? ?cus-id ?e)]} db customer-id entity] owner-rules))
  63. 63. all owned entities 18 query binding attribute attribute unbound (defn owns? [customer-id entity db] (d/q '{:find [?e .] :in [$ ?cus-id ?e %] :where [(owns? ?cus-id ?e)]} db customer-id entity] owner-rules)) (defn owned-entities [customer-id db] (d/q '{:find [[?e ...]] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e)]} db customer-id owner-rules))
  64. 64. all owned entities 18 query binding attribute attribute unbound (defn owns? [customer-id entity db] (d/q '{:find [?e .] :in [$ ?cus-id ?e %] :where [(owns? ?cus-id ?e)]} db customer-id entity] owner-rules)) (defn owned-entities [customer-id db] (d/q '{:find [[?e ...]] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e)]} db customer-id owner-rules))
  65. 65. all owned entities 18 query binding attribute attribute unbound (defn owns? [customer-id entity db] (d/q '{:find [?e .] :in [$ ?cus-id ?e %] :where [(owns? ?cus-id ?e)]} db customer-id entity] owner-rules)) (defn owned-entities [customer-id db] (d/q '{:find [[?e ...]] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e)]} db customer-id owner-rules))
  66. 66. filtered db (as value) 19 (defn owned-db [customer-id db] (let [owned (set (owned-entities customer-id db))] (d/filter db (fn [_ [e a v t :as datom]] (when (or (= e t) (contains? owned e)) datom)))))
  67. 67. filtered db (as value) 19 (defn owned-db [customer-id db] (let [owned (set (owned-entities customer-id db))] (d/filter db (fn [_ [e a v t :as datom]] (when (or (= e t) (contains? owned e)) datom)))))
  68. 68. filtered db (as value) 19 (defn owned-db [customer-id db] (let [owned (set (owned-entities customer-id db))] (d/filter db (fn [_ [e a v t :as datom]] (when (or (= e t) (contains? owned e)) datom)))))
  69. 69. filtered db (as value) 19 (defn owned-db [customer-id db] (let [owned (set (owned-entities customer-id db))] (d/filter db (fn [_ [e a v t :as datom]] (when (or (= e t) (contains? owned e)) datom)))))
  70. 70. filtered db (as value) 19 (defn owned-db [customer-id db] (let [owned (set (owned-entities customer-id db))] (d/filter db (fn [_ [e a v t :as datom]] (when (or (= e t) (contains? owned e)) datom)))))
  71. 71. filtered db (as value) (let [owned (set (owned-entities customer-id db))] (d/filter db 19 (defn owned-db [customer-id db] (fn [_ [e a v t :as datom]] (when (or (= e t) (contains? owned e)) datom))))) filtered db will not contain datoms from other owners!
  72. 72. passing a filtered db is transparent for queries 20 (defn all-purchases [account-id db] (d/q '{:find [[(pull ?p [*]) ...]] :in [$ ?acc] :where [[?p :purchase/account ?acc]]} db [:account/id account-id])) (all-purchases account-id db) (all-purchases account-id (owned-db customer-id db))
  73. 73. passing a filtered db is transparent for queries 20 (defn all-purchases [account-id db] (d/q '{:find [[(pull ?p [*]) ...]] :in [$ ?acc] :where [[?p :purchase/account ?acc]]} db [:account/id account-id])) (all-purchases account-id db) (all-purchases account-id (owned-db customer-id db))
  74. 74. 21 We’re Mobile! Bandwidth is costly
  75. 75. 22 3 HTTP Cache
  76. 76. 23 http last modified header GET /accounts/1234/purchases 200 OK Last-Modified: Fri, 14 Nov 2014 14:28:50 UTC {"purchases": [...]}
  77. 77. http last modified header If-Modified-Since: Fri, 14 Nov 2014 14:28:50 UTC GET /accounts/1234/purchases 304 Not Modified 23 GET /accounts/1234/purchases 200 OK Last-Modified: Fri, 14 Nov 2014 14:28:50 UTC {"purchases": [...]}
  78. 78. 24 How to keep track of a good last-modified date for a URL?
  79. 79. 24 How to keep track of a good last-modified date for a URL? The last time any owned entity changed!
  80. 80. If no customer-owned data changed, 304 25 (defn last-modified [customer-id db] (d/q '{:find [(max ?time) .] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e) [?e ?a ?v ?tx] [?tx :db/txInstant ?time]]} db customer-id owner-rules))
  81. 81. If no customer-owned data changed, 304 25 (defn last-modified [customer-id db] (d/q '{:find [(max ?time) .] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e) [?e ?a ?v ?tx] [?tx :db/txInstant ?time]]} db customer-id owner-rules))
  82. 82. If no customer-owned data changed, 304 25 (defn last-modified [customer-id db] (d/q '{:find [(max ?time) .] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e) [?e ?a ?v ?tx] [?tx :db/txInstant ?time]]} db customer-id owner-rules))
  83. 83. If no customer-owned data changed, 304 25 (defn last-modified [customer-id db] (d/q '{:find [(max ?time) .] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e) [?e ?a ?v ?tx] [?tx :db/txInstant ?time]]} db customer-id owner-rules))
  84. 84. If no customer-owned data changed, 304 25 (defn last-modified [customer-id db] (d/q '{:find [(max ?time) .] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e) [?e ?a ?v ?tx] [?tx :db/txInstant ?time]]} db customer-id owner-rules))
  85. 85. If no customer-owned data changed, 304 25 (defn last-modified [customer-id db] (d/q '{:find [(max ?time) .] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e) [?e ?a ?v ?tx] [?tx :db/txInstant ?time]]} db customer-id owner-rules))
  86. 86. If no customer-owned data changed, 304 25 (defn last-modified [customer-id db] (d/q '{:find [(max ?time) .] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e) [?e ?a ?v ?tx] [?tx :db/txInstant ?time]]} db customer-id owner-rules))
  87. 87. If no customer-owned data changed, 304 25 (defn last-modified [customer-id db] (d/q '{:find [(max ?time) .] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e) [?e ?a ?v ?tx] [?tx :db/txInstant ?time]]} db customer-id owner-rules))
  88. 88. If no customer-owned data changed, 304 25 (defn last-modified [customer-id db] (d/q '{:find [(max ?time) .] :in [$ ?cus-id %] :where [(owns? ?cus-id ?e) [?e ?a ?v ?tx] [?tx :db/txInstant ?time]]} db customer-id owner-rules))
  89. 89. 26 Cache hits are awesome, but what about cache misses?
  90. 90. 27 4 Mobile Sync
  91. 91. desired hypermedia from the API 28 GET /accounts/1223/purchases 200 OK {"purchases": [...] "_links": { "updates": "https://...?since=2014-11-10T11:12:13Z" } }
  92. 92. serve purchases added or modified since our last sync 29 (defn updated-purchases [account-id db since] (d/q '{:find [[(pull ?pur [*]) ...]] :in [$ $since ?acc] :where [[?pur :purchase/account ?acc] [$since ?pur]]} db (d/history (d/since db since)) [:account/id account-id])) (updated-purchases account-id db #inst "2014-11-14"))
  93. 93. serve purchases added or modified since our last sync 29 (defn updated-purchases [account-id db since] (d/q '{:find [[(pull ?pur [*]) ...]] :in [$ $since ?acc] :where [[?pur :purchase/account ?acc] [$since ?pur]]} db (d/history (d/since db since)) [:account/id account-id])) (updated-purchases account-id db #inst "2014-11-14"))
  94. 94. serve purchases added or modified since our last sync 29 (defn updated-purchases [account-id db since] (d/q '{:find [[(pull ?pur [*]) ...]] :in [$ $since ?acc] :where [[?pur :purchase/account ?acc] [$since ?pur]]} db (d/history (d/since db since)) [:account/id account-id])) (updated-purchases account-id db #inst "2014-11-14"))
  95. 95. serve purchases added or modified since our last sync 29 (defn updated-purchases [account-id db since] (d/q '{:find [[(pull ?pur [*]) ...]] :in [$ $since ?acc] :where [[?pur :purchase/account ?acc] [$since ?pur]]} db (d/history (d/since db since)) [:account/id account-id])) (updated-purchases account-id db #inst "2014-11-14"))
  96. 96. serve purchases added or modified since our last sync 29 (defn updated-purchases [account-id db since] (d/q '{:find [[(pull ?pur [*]) ...]] :in [$ $since ?acc] :where [[?pur :purchase/account ?acc] [$since ?pur]]} db (d/history (d/since db since)) [:account/id account-id])) (updated-purchases account-id db #inst "2014-11-14"))
  97. 97. serve purchases added or modified since our last sync 29 (defn updated-purchases [account-id db since] (d/q '{:find [[(pull ?pur [*]) ...]] :in [$ $since ?acc] :where [[?pur :purchase/account ?acc] [$since ?pur]]} db (d/history (d/since db since)) [:account/id account-id])) (updated-purchases account-id db #inst "2014-11-14"))
  98. 98. serve purchases added or modified since our last sync 29 (defn updated-purchases [account-id db since] (d/q '{:find [[(pull ?pur [*]) ...]] :in [$ $since ?acc] :where [[?pur :purchase/account ?acc] [$since ?pur]]} db (d/history (d/since db since)) [:account/id account-id])) (updated-purchases account-id db #inst "2014-11-14"))
  99. 99. 30 Bonus
  100. 100. 31 5 Future DBs
  101. 101. generating a virtual db (defn db-with-virtual-charges [purchases db] (:db-after (d/with db (mapcat purchase->virtual-charges purchases)))) 32
  102. 102. generating a virtual db (defn db-with-virtual-charges [purchases db] (:db-after (d/with db (mapcat purchase->virtual-charges purchases)))) 32
  103. 103. generating a virtual db (defn db-with-virtual-charges [purchases db] (:db-after (d/with db (mapcat purchase->virtual-charges purchases)))) 32
  104. 104. generating a virtual db (defn db-with-virtual-charges [purchases db] (:db-after (d/with db (mapcat purchase->virtual-charges purchases)))) 32
  105. 105. generating a virtual db (defn db-with-virtual-charges [purchases db] (:db-after (d/with db (mapcat purchase->virtual-charges purchases)))) 32
  106. 106. 33 6 Testing
  107. 107. db testing - locally scoped 34 (defn virtual-db [updates db] (:db-after (d/with db updates))) (let [one-account {...} one-card {...} one-purchase {...} wrong-purchase {...} db (virtual-db [one-account one-card one-purchase wrong-purchase] db)] (my-weird-db-query db) => [one-purchase])
  108. 108. db testing - locally scoped 34 (defn virtual-db [updates db] (:db-after (d/with db updates))) (let [one-account {...} one-card {...} one-purchase {...} wrong-purchase {...} db (virtual-db [one-account one-card one-purchase wrong-purchase] db)] (my-weird-db-query db) => [one-purchase])
  109. 109. db testing - locally scoped 34 (defn virtual-db [updates db] (:db-after (d/with db updates))) (let [one-account {...} one-card {...} one-purchase {...} wrong-purchase {...} db (virtual-db [one-account one-card one-purchase wrong-purchase] db)] (my-weird-db-query db) => [one-purchase])
  110. 110. db testing - locally scoped 34 (defn virtual-db [updates db] (:db-after (d/with db updates))) (let [one-account {...} one-card {...} one-purchase {...} wrong-purchase {...} db (virtual-db [one-account one-card one-purchase wrong-purchase] db)] (my-weird-db-query db) => [one-purchase])
  111. 111. db testing - locally scoped 34 (defn virtual-db [updates db] (:db-after (d/with db updates))) (let [one-account {...} one-card {...} one-purchase {...} wrong-purchase {...} db (virtual-db [one-account one-card one-purchase wrong-purchase] db)] (my-weird-db-query db) => [one-purchase])
  112. 112. db testing - locally scoped 34 (defn virtual-db [updates db] (:db-after (d/with db updates))) (let [one-account {...} one-card {...} one-purchase {...} wrong-purchase {...} db (virtual-db [one-account one-card one-purchase wrong-purchase] db)] (my-weird-db-query db) => [one-purchase])
  113. 113. db testing - vector of datoms 35 (fact "on last-modification-for" (let [customer-id (uuid) db [[42 :account/customer-id customer-id 99] [99 :db/txInstant #inst "2014-10-10T12:00:00.909Z" 99] [1337 :purchase/account 42 100] [1337 :purchase/id (uuid) 100] [1337 :purchase/merchant-name "Coiso" 100] [100 :db/txInstant #inst "2014-10-11T12:00:00.888Z" 100] [9001 :line-item/account 42 101] [9001 :charge/id (uuid) 101] [9001 :line-item/precise-amount 100.00M 101] [101 :db/txInstant #inst "2014-10-12T13:00:00.777Z" 101] [45 :account/customer-id (uuid) 102] [102 :db/txInstant #inst "2014-10-15T12:00:00.909Z" 102]]] (last-modification-for customer-id db) => #nu/time "2014-10-12T13:00:00.777Z"))
  114. 114. db testing - vector of datoms 35 (fact "on last-modification-for" (let [customer-id (uuid) db [[42 :account/customer-id customer-id 99] [99 :db/txInstant #inst "2014-10-10T12:00:00.909Z" 99] [1337 :purchase/account 42 100] [1337 :purchase/id (uuid) 100] [1337 :purchase/merchant-name "Coiso" 100] [100 :db/txInstant #inst "2014-10-11T12:00:00.888Z" 100] [9001 :line-item/account 42 101] [9001 :charge/id (uuid) 101] [9001 :line-item/precise-amount 100.00M 101] [101 :db/txInstant #inst "2014-10-12T13:00:00.777Z" 101] [45 :account/customer-id (uuid) 102] [102 :db/txInstant #inst "2014-10-15T12:00:00.909Z" 102]]] (last-modification-for customer-id db) => #nu/time "2014-10-12T13:00:00.777Z"))
  115. 115. 36 7 Schema Extension
  116. 116. schema is data! 37 (d/transact conn [{:db/valueType :db.type/string, :db.install/_attribute :db.part/db, :db/id (d/tempid :db.part/db), :db/cardinality :db.cardinality/one, :db/ident :customer/name }])
  117. 117. :nubank/transform :pii schema is data! 37 (d/transact conn [{:db/valueType :db.type/string, :db.install/_attribute :db.part/db, :db/id (d/tempid :db.part/db), :db/cardinality :db.cardinality/one, :db/ident :customer/name }])
  118. 118. 38 8 Sharding Reads
  119. 119. 39 sharding to the transactor customers accounts notification acquisition processor auth
  120. 120. 39 sharding to the transactor customers accounts notification acquisition processor auth
  121. 121. customers 1 2 … 39 sharding to the transactor customers accounts notification acquisition processor auth
  122. 122. customers 1 2 … 39 sharding to the transactor customers accounts notification acquisition processor auth shard-specific peer cache
  123. 123. customers 1 2 … 39 sharding to the transactor customers accounts notification acquisition processor auth shard-specific peer cache inter-shard ACID transactions
  124. 124. 40 9 Db Aggregation
  125. 125. 41 data science aggregation customers accounts notification acquisition processor auth data science
  126. 126. 42 querying multiple databases (defn multi-join [customer-id cus-db acc-db acq-db ntf-db] (d/q '{:find [...] :in [$cus $acc $acq $ntf ?cus-id] :where [[$cus ?cus :customer/id ?cus-id] [$acc ?acc :account/customer-id ?cus-id] [$acq ?ar :request/customer-id ?cus-id] [$ntf ?evt :event/customer-id ?cus-id] [...]]} cus-db acc-db acq-db ntf-db customer-id))
  127. 127. 42 querying multiple databases (defn multi-join [customer-id cus-db acc-db acq-db ntf-db] (d/q '{:find [...] :in [$cus $acc $acq $ntf ?cus-id] :where [[$cus ?cus :customer/id ?cus-id] [$acc ?acc :account/customer-id ?cus-id] [$acq ?ar :request/customer-id ?cus-id] [$ntf ?evt :event/customer-id ?cus-id] [...]]} cus-db acc-db acq-db ntf-db customer-id))
  128. 128. 43 1 Audit Trail 2 Authorization 3 HTTP Cache 4 Mobile Sync 5 Future DBs 6 Testing 7 Schema Extension 8 Sharding Reads 9 Db Aggregation
  129. 129. 44 We’re hiring
  130. 130. 45 We’re hiring
  131. 131. Thanks! Lucas Cavalcanti & Edward Wible @lucascs lucas@nubank.com.br edward@nubank.com.br
  • Radioative

    Jan. 7, 2020
  • tiongks

    Oct. 6, 2019
  • JrnGersdorf

    Mar. 28, 2019
  • tariqhamid127

    May. 13, 2018
  • jocrau

    Sep. 1, 2016
  • neverfox

    Apr. 6, 2016
  • saikrishnagollapudi

    Oct. 20, 2015
  • sinesiojunor

    Jul. 24, 2015
  • marioamaral52

    Jan. 21, 2015
  • tf0054

    Jan. 13, 2015
  • jeroenvandijk

    Dec. 2, 2014
  • asolovyov

    Nov. 28, 2014
  • austinpmaher

    Nov. 28, 2014
  • cemsbr

    Nov. 26, 2014
  • adolfont

    Nov. 26, 2014

https://www.youtube.com/watch?v=7lm3K8zVOdY This session will explore four common problems, and the unique and surprising tools Datomic provides to solve them elegantly: HTTP caching - How to generically generate and validate Last-Modified and If-Modified-Since headers Audit trail - how to extend Datomic’s immutable transaction log to include arbitrary audit related metadata Mobile database sync - trivial implementation of an incremental update API for high latency/low bandwidth clients Authorization - easily determine resource ownership, and centrally isolate users from data they are not allowed to see These problems have certainly been solved before using other databases, but Datomic provides features that make the proposed implementations concise, generic, and purely functional

Views

Total views

5,004

On Slideshare

0

From embeds

0

Number of embeds

438

Actions

Downloads

77

Shares

0

Comments

0

Likes

15

×