1. Exploring Four
Datomic
Superpowers
Lucas Cavalcanti & Edward Wible

2. 2
Why build a bank
from scratch
in Brazil
using Clojure and
Datomic?

3. 3

4. 3

5. 4
1 Audit Trail

6. 5
event stream
01 NOV 10:00 Customer joins waiting list for a card

7. 5
event stream
01 NOV 11:00 Robot 437aae3 approves R$3K limit
01 NOV 10:00 Customer joins waiting list for a card

8. 5
event stream
09 NOV 08:00 Mastercard purchase, Starbucks, R$100
01 NOV 11:00 Robot 437aae3 approves R$3K limit
01 NOV 10:00 Customer joins waiting list for a card

9. 5
event stream
15 NOV 15:00 Support agent increases limit to R$5K
09 NOV 08:00 Mastercard purchase, Starbucks, R$100
01 NOV 11:00 Robot 437aae3 approves R$3K limit
01 NOV 10:00 Customer joins waiting list for a card

10. 5
event stream
15 NOV 17:05 Customer blocks card
15 NOV 15:00 Support agent increases limit to R$5K
09 NOV 08:00 Mastercard purchase, Starbucks, R$100
01 NOV 11:00 Robot 437aae3 approves R$3K limit
01 NOV 10:00 Customer joins waiting list for a card

11. facts over time
6
15 NOV 17:05
15 NOV 15:00
09 NOV 08:00
01 NOV 11:00
01 NOV 10:00
[<card> :card/status :card.status/blocked]
[<card> :card/status :card.status/active]
[<account> :account/limit 5000]
[<account> :account/limit 3000]
[<purchase> :purchase/card <card>]
[<purchase> :purchase/amount 100]
[<purchase> :purchase/merchant “Starbucks”]
[<account> :account/customer <customer>]
[<account> :account/limit 3000]
[<card> :card/account <account>]
[<card> :card/status :card.status/active]
[<customer> :customer/id #uuid “b2c90…”]

12. facts over time
6
15 NOV 17:05
15 NOV 15:00
09 NOV 08:00
01 NOV 11:00
01 NOV 10:00
[<card> :card/status :card.status/blocked]
[<card> :card/status :card.status/active]
[<account> :account/limit 5000]
[<account> :account/limit 3000]
[<purchase> :purchase/card <card>]
[<purchase> :purchase/amount 100]
[<purchase> :purchase/merchant “Starbucks”]
[<account> :account/customer <customer>]
[<account> :account/limit 3000]
[<card> :card/account <account>]
[<card> :card/status :card.status/active]
[<customer> :customer/id #uuid “b2c90…”]
entity attribute value

13. 7

14. 7
don’t
lose data

15. 7
don’t
lose data
what
changed
when

16. 7
don’t
lose data
what
changed
when
as-of
since

17. 7
don’t
lose data
what
changed
when
as-of
since
fork
merge

18. 8
What was the initial limit for the card?

19. 8
What was the initial limit for the card?
At the time the Starbucks transaction occurred, which
fraud triggers would have activated?

20. 8
What was the initial limit for the card?
At the time the Starbucks transaction occurred, which
fraud triggers would have activated?
How long did the customer spend on each stage of the
acquisition funnel?

21. 8
What was the initial limit for the card?
At the time the Starbucks transaction occurred, which
fraud triggers would have activated?
How long did the customer spend on each stage of the
acquisition funnel?
How frequently do we see amount changes on
Starbucks transactions?

22. 9
What sequence of
events resulted in the
Starbucks transaction
being persisted?

23. 9
What sequence of
events resulted in the
Starbucks transaction
being persisted?
iOS
http-in
kafka-out
kafka-in
iZb
iZb.jnA
iZb.jnA.9Cd
iZb.jnA.9Cd.l9A

24. 9
iOS
http-in
kafka-out
kafka-in
iZb
iZb.jnA
iZb.jnA.9Cd
iZb.jnA.9Cd.l9A
What sequence of
events resulted in the
Starbucks transaction
being persisted? correlation-id

25. Who was responsible
for the credit limit
increase?
9
iOS
http-in
kafka-out
kafka-in
iZb
iZb.jnA
iZb.jnA.9Cd
iZb.jnA.9Cd.l9A
What sequence of
events resulted in the
Starbucks transaction
being persisted? correlation-id

26. Who was responsible
for the credit limit
increase?
9
iOS
http-in
kafka-out
kafka-in
iZb
iZb.jnA
iZb.jnA.9Cd
iZb.jnA.9Cd.l9A
What sequence of
events resulted in the
Starbucks transaction
being persisted? correlation-id
#uuid “b2c90…”
“lucas@nubank.com.br”
:kafka-LIMIT-CHANGED
:robot-437aae3

27. Who was responsible
for the credit limit
increase?
9
iOS
http-in
kafka-out
kafka-in
iZb
iZb.jnA
iZb.jnA.9Cd
iZb.jnA.9Cd.l9A
What sequence of
events resulted in the
Starbucks transaction
being persisted? correlation-id
user
#uuid “b2c90…”
“lucas@nubank.com.br”
:kafka-LIMIT-CHANGED
:robot-437aae3

28. Who was responsible
for the credit limit
increase?
9
What sequence of
events resulted in the
Starbucks transaction
being persisted?
Why was the
customer’s card
blocked?
correlation-id
iOS
http-in
kafka-out
kafka-in
iZb
iZb.jnA
iZb.jnA.9Cd
iZb.jnA.9Cd.l9A
user
#uuid “b2c90…”
“lucas@nubank.com.br”
:kafka-LIMIT-CHANGED
:robot-437aae3

29. Who was responsible
for the credit limit
increase?
9
What sequence of
events resulted in the
Starbucks transaction
being persisted?
Why was the
customer’s card
blocked?
correlation-id
iOS
http-in
kafka-out
kafka-in
iZb
iZb.jnA
iZb.jnA.9Cd
iZb.jnA.9Cd.l9A
:fraud-preventative
:recurring-scheduled
:late-payment
:data-migration
user
#uuid “b2c90…”
“lucas@nubank.com.br”
:kafka-LIMIT-CHANGED
:robot-437aae3

30. Who was responsible
for the credit limit
increase?
9
What sequence of
events resulted in the
Starbucks transaction
being persisted?
Why was the
customer’s card
blocked?
correlation-id
iOS
http-in
kafka-out
kafka-in
iZb
iZb.jnA
iZb.jnA.9Cd
iZb.jnA.9Cd.l9A
user
#uuid “b2c90…”
“lucas@nubank.com.br”
:kafka-LIMIT-CHANGED
:robot-437aae3
:fraud-preventative reason
:recurring-scheduled
:late-payment
:data-migration

31. (defn block-card [card reason]
(d/transact
(conn)
[[:db/add card :card/status :card.status/blocked]
]))
10
storing transaction metadata

32. (defn block-card [card reason]
(d/transact
(conn)
[[:db/add card :card/status :card.status/blocked]
{:db/id (d/tempid :db.part/tx)
:audit/user "lucas@nubank.com.br"
:audit/cid "iZb.jnA.9Cd.l9A"
:audit/tags :fraud-preventative}
]))
10
storing transaction metadata

33. 11
transactions over time with metadata
[<card> :card/status :card.status/blocked]
[<card> :card/status :card.status/active] 15 NOV 17:05
[<account> :account/limit 5000M]
15 NOV 15:00 [<account> :account/limit 3000M]
[<purchase> :purchase/card <card>]
[<purchase> :purchase/amount 100M]
[<purchase> :purchase/merchant “Starbucks”] 09 NOV 08:00
[<account> :account/customer <customer>]
[<account> :account/limit 3000M]
[<card> :card/account <account>]
[<card> :card/status :card.status/active]
01 NOV 11:00
01 NOV 10:00 [<customer> :customer/id #uuid “b2c90…”]

34. 11
transactions over time with metadata
[<card> :card/status :card.status/blocked]
[<card> :card/status :card.status/active] 15 NOV 17:05
[<account> :account/limit 5000M]
15 NOV 15:00 [<account> :account/limit 3000M]
[<purchase> :purchase/card <card>]
[<purchase> :purchase/amount 100M]
[<purchase> :purchase/merchant “Starbucks”] 09 NOV 08:00
[<account> :account/customer <customer>]
[<account> :account/limit 3000M]
[<card> :card/account <account>]
[<card> :card/status :card.status/active]
01 NOV 11:00
01 NOV 10:00 [<customer> :customer/id #uuid “b2c90…”]

35. 12
querying transaction metadata
(defn who-blocked-the-card? [card]
(d/q '{:find [?user ?cid ?tags]
:in [$ ?status ?card]
:where [[?card :card/status ?status ?tx]
[?tx :audit/user ?user]
[?tx :audit/cid ?cid]
[?tx :audit/tags ?tags]]}
db :card.status/blocked card))

36. 12
querying transaction metadata
(defn who-blocked-the-card? [card]
(d/q '{:find [?user ?cid ?tags]
:in [$ ?status ?card]
:where [[?entity attribute value txn
[?card :card/status ?status ?tx]
[?tx :audit/user ?user]
[?tx :audit/cid ?cid]
[?tx :audit/tags ?tags]]}
db :card.status/blocked card))

37. 12
querying transaction metadata
(defn who-blocked-the-card? [card]
(d/q '{:find [?user ?cid ?tags]
:in [$ ?status ?card]
:where [[?card :card/status ?status ?tx]
[?tx :audit/user ?user]
[?tx :audit/cid ?cid]
[?tx :audit/tags ?tags]]}
db :card.status/blocked card))

38. 12
querying transaction metadata
(defn who-blocked-the-card? [card]
(d/q '{:find [?user ?cid ?tags]
:in [$ ?status ?card]
:where [[?card :card/status ?status ?tx]
[?tx :audit/user ?user]
[?tx :audit/cid ?cid]
[?tx :audit/tags ?tags]]}
db :card.status/blocked card))

39. 12
querying transaction metadata
(defn who-blocked-the-card? [card]
(d/q '{:find [?user ?cid ?tags]
:in [$ ?status ?card]
:where [[?card :card/status ?status ?tx]
[?tx :audit/user ?user]
[?tx :audit/cid ?cid]
[?tx :audit/tags ?tags]]}
db :card.status/blocked card))

40. 13
2 Authorization

41. should we return data?
GET /purchases/1337/comments

42. should we return data?
GET /purchases/1337/comments
200 OK 401 Unauthorized

43. customer
account
payment
card purchase
comment
should we return data?
GET /purchases/1337/comments
200 OK 401 Unauthorized

44. customer
account
payment
card purchase
comment
should we return data?
GET /purchases/1337/comments
200 OK 401 Unauthorized

45. customer
account
payment
card purchase
comment
should we return data?
GET /purchases/1337/comments
200 OK 401 Unauthorized

46. 15
explicit relationship traversal
(defn owns? [customer-id purchase-id db]
(d/q '{:find [?pur .]
:in [$ ?customer-id ?purchase-id]
:where [[?pur :purchase/id ?purchase-id]
[?pur :purchase/account ?acc]
[?acc :account/customer ?cus]
[?cus :customer/id ?customer-id]]}
db customer-id purchase-id))

47. 15
explicit relationship traversal
(defn owns? [customer-id purchase-id db]
(d/q '{:find [?pur .]
:in [$ ?customer-id ?purchase-id]
:where [[?pur :purchase/id ?purchase-id]
[?pur :purchase/account ?acc]
[?acc :account/customer ?cus]
[?cus :customer/id ?customer-id]]}
db customer-id purchase-id))

48. generic relationship traversal
16
recursive rule
(logical OR)

49. generic relationship traversal
16
recursive rule
(logical OR)
(def owner-rules
'[[(owns? ?cus-id ?e)
[?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
[?e ?ref-attr ?r]
(owns? ?cus-id ?r)]])

50. generic relationship traversal
16
recursive rule
(logical OR)
(def owner-rules
'[[(owns? ?cus-id ?e)
[?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
[?e ?ref-attr ?r]
(owns? ?cus-id ?r)]])

51. generic relationship traversal
16
recursive rule
(logical OR)
(def owner-rules
'[[(owns? ?cus-id ?e)
[?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
[?e ?ref-attr ?r]
(owns? ?cus-id ?r)]])

52. generic relationship traversal
16
recursive rule
(logical OR)
(def owner-rules
'[[(owns? ?cus-id ?e)
[?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
[?e ?ref-attr ?r]
(owns? ?cus-id ?r)]])

53. generic relationship traversal
16
recursive rule
(logical OR)
(def owner-rules
'[[(owns? ?cus-id ?e)
[?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
[?e ?ref-attr ?r]
(owns? ?cus-id ?r)]])

54. generic relationship traversal
16
recursive rule
(logical OR)
(def owner-rules
'[[(owns? ?cus-id ?e)
[?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
[?e ?ref-attr ?r]
(owns? ?cus-id ?r)]])

55. generic relationship traversal
16
recursive rule
(logical OR)
query
using rule
(def owner-rules
'[[(owns? ?cus-id ?e)
[?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
[?e ?ref-attr ?r]
(owns? ?cus-id ?r)]])

56. generic relationship traversal
:in [$ ?cus-id ?e %]
:where [(owns? ?cus-id ?e)]}
db customer-id entity
owner-rules))
16
recursive rule
(logical OR)
query
using rule
(def owner-rules
'[[(owns? ?cus-id ?e)
[?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
[?e ?ref-attr ?r]
(owns? ?cus-id ?r)]])
(defn owns? [customer-id entity db]
(d/q '{:find [?e .]
(owns? customer-id [:purchase/id id] db)

57. generic relationship traversal
:in [$ ?cus-id ?e %]
:where [(owns? ?cus-id ?e)]}
db customer-id entity
owner-rules))
16
recursive rule
(logical OR)
query
using rule
(def owner-rules
'[[(owns? ?cus-id ?e)
[?e :customer/id ?cus-id]]
[(owns? ?cus-id ?e)
[?e ?ref-attr ?r]
(owns? ?cus-id ?r)]])
(defn owns? [customer-id entity db]
(d/q '{:find [?e .]
(owns? customer-id [:purchase/id id] db)

58. 17
Recursion rules!
But can we go further?
Can we prevent queries from
returning entities owned by
other customers?

59. all owned entities
18
query binding
attribute

60. all owned entities
18
query binding
attribute
(defn owns? [customer-id entity db]
(d/q '{:find [?e .]
:in [$ ?cus-id ?e %]
:where [(owns? ?cus-id ?e)]}
db customer-id entity]
owner-rules))

61. all owned entities
18
query binding
attribute
(defn owns? [customer-id entity db]
(d/q '{:find [?e .]
:in [$ ?cus-id ?e %]
:where [(owns? ?cus-id ?e)]}
db customer-id entity]
owner-rules))

62. all owned entities
18
query binding
attribute
attribute
unbound
(defn owns? [customer-id entity db]
(d/q '{:find [?e .]
:in [$ ?cus-id ?e %]
:where [(owns? ?cus-id ?e)]}
db customer-id entity]
owner-rules))

63. all owned entities
18
query binding
attribute
attribute
unbound
(defn owns? [customer-id entity db]
(d/q '{:find [?e .]
:in [$ ?cus-id ?e %]
:where [(owns? ?cus-id ?e)]}
db customer-id entity]
owner-rules))
(defn owned-entities [customer-id db]
(d/q '{:find [[?e ...]]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)]}
db customer-id owner-rules))

64. all owned entities
18
query binding
attribute
attribute
unbound
(defn owns? [customer-id entity db]
(d/q '{:find [?e .]
:in [$ ?cus-id ?e %]
:where [(owns? ?cus-id ?e)]}
db customer-id entity]
owner-rules))
(defn owned-entities [customer-id db]
(d/q '{:find [[?e ...]]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)]}
db customer-id owner-rules))

65. all owned entities
18
query binding
attribute
attribute
unbound
(defn owns? [customer-id entity db]
(d/q '{:find [?e .]
:in [$ ?cus-id ?e %]
:where [(owns? ?cus-id ?e)]}
db customer-id entity]
owner-rules))
(defn owned-entities [customer-id db]
(d/q '{:find [[?e ...]]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)]}
db customer-id owner-rules))

66. filtered db (as value)
19
(defn owned-db [customer-id db]
(let [owned (set (owned-entities customer-id db))]
(d/filter db
(fn [_ [e a v t :as datom]]
(when (or (= e t)
(contains? owned e))
datom)))))

67. filtered db (as value)
19
(defn owned-db [customer-id db]
(let [owned (set (owned-entities customer-id db))]
(d/filter db
(fn [_ [e a v t :as datom]]
(when (or (= e t)
(contains? owned e))
datom)))))

68. filtered db (as value)
19
(defn owned-db [customer-id db]
(let [owned (set (owned-entities customer-id db))]
(d/filter db
(fn [_ [e a v t :as datom]]
(when (or (= e t)
(contains? owned e))
datom)))))

69. filtered db (as value)
19
(defn owned-db [customer-id db]
(let [owned (set (owned-entities customer-id db))]
(d/filter db
(fn [_ [e a v t :as datom]]
(when (or (= e t)
(contains? owned e))
datom)))))

70. filtered db (as value)
19
(defn owned-db [customer-id db]
(let [owned (set (owned-entities customer-id db))]
(d/filter db
(fn [_ [e a v t :as datom]]
(when (or (= e t)
(contains? owned e))
datom)))))

71. filtered db (as value)
(let [owned (set (owned-entities customer-id db))]
(d/filter db
19
(defn owned-db [customer-id db]
(fn [_ [e a v t :as datom]]
(when (or (= e t)
(contains? owned e))
datom)))))
filtered db will not contain datoms
from other owners!

72. passing a filtered db is transparent for queries
20
(defn all-purchases [account-id db]
(d/q '{:find [[(pull ?p [*]) ...]]
:in [$ ?acc]
:where [[?p :purchase/account ?acc]]}
db [:account/id account-id]))
(all-purchases account-id db)
(all-purchases account-id (owned-db customer-id db))

73. passing a filtered db is transparent for queries
20
(defn all-purchases [account-id db]
(d/q '{:find [[(pull ?p [*]) ...]]
:in [$ ?acc]
:where [[?p :purchase/account ?acc]]}
db [:account/id account-id]))
(all-purchases account-id db)
(all-purchases account-id (owned-db customer-id db))

74. 21
We’re Mobile!
Bandwidth is costly

75. 22
3 HTTP Cache

76. 23
http last modified header
GET /accounts/1234/purchases
200 OK
Last-Modified: Fri, 14 Nov 2014 14:28:50 UTC
{"purchases": [...]}

77. http last modified header
If-Modified-Since: Fri, 14 Nov 2014 14:28:50 UTC
GET /accounts/1234/purchases
304 Not Modified
23
GET /accounts/1234/purchases
200 OK
Last-Modified: Fri, 14 Nov 2014 14:28:50 UTC
{"purchases": [...]}

78. 24
How to keep track of a
good last-modified date
for a URL?

79. 24
How to keep track of a
good last-modified date
for a URL?
The last time any owned
entity changed!

80. If no customer-owned data changed, 304
25
(defn last-modified [customer-id db]
(d/q '{:find [(max ?time) .]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)
[?e ?a ?v ?tx]
[?tx :db/txInstant ?time]]}
db customer-id owner-rules))

81. If no customer-owned data changed, 304
25
(defn last-modified [customer-id db]
(d/q '{:find [(max ?time) .]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)
[?e ?a ?v ?tx]
[?tx :db/txInstant ?time]]}
db customer-id owner-rules))

82. If no customer-owned data changed, 304
25
(defn last-modified [customer-id db]
(d/q '{:find [(max ?time) .]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)
[?e ?a ?v ?tx]
[?tx :db/txInstant ?time]]}
db customer-id owner-rules))

83. If no customer-owned data changed, 304
25
(defn last-modified [customer-id db]
(d/q '{:find [(max ?time) .]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)
[?e ?a ?v ?tx]
[?tx :db/txInstant ?time]]}
db customer-id owner-rules))

84. If no customer-owned data changed, 304
25
(defn last-modified [customer-id db]
(d/q '{:find [(max ?time) .]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)
[?e ?a ?v ?tx]
[?tx :db/txInstant ?time]]}
db customer-id owner-rules))

85. If no customer-owned data changed, 304
25
(defn last-modified [customer-id db]
(d/q '{:find [(max ?time) .]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)
[?e ?a ?v ?tx]
[?tx :db/txInstant ?time]]}
db customer-id owner-rules))

86. If no customer-owned data changed, 304
25
(defn last-modified [customer-id db]
(d/q '{:find [(max ?time) .]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)
[?e ?a ?v ?tx]
[?tx :db/txInstant ?time]]}
db customer-id owner-rules))

87. If no customer-owned data changed, 304
25
(defn last-modified [customer-id db]
(d/q '{:find [(max ?time) .]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)
[?e ?a ?v ?tx]
[?tx :db/txInstant ?time]]}
db customer-id owner-rules))

88. If no customer-owned data changed, 304
25
(defn last-modified [customer-id db]
(d/q '{:find [(max ?time) .]
:in [$ ?cus-id %]
:where [(owns? ?cus-id ?e)
[?e ?a ?v ?tx]
[?tx :db/txInstant ?time]]}
db customer-id owner-rules))

89. 26
Cache hits are awesome,
but what about cache
misses?

90. 27
4 Mobile Sync

91. desired hypermedia from the API
28
GET /accounts/1223/purchases
200 OK
{"purchases": [...]
"_links": {
"updates": "https://...?since=2014-11-10T11:12:13Z"
}
}

92. serve purchases added or modified since our last sync
29
(defn updated-purchases [account-id db since]
(d/q '{:find [[(pull ?pur [*]) ...]]
:in [$ $since ?acc]
:where [[?pur :purchase/account ?acc]
[$since ?pur]]}
db
(d/history (d/since db since))
[:account/id account-id]))
(updated-purchases account-id db #inst "2014-11-14"))

93. serve purchases added or modified since our last sync
29
(defn updated-purchases [account-id db since]
(d/q '{:find [[(pull ?pur [*]) ...]]
:in [$ $since ?acc]
:where [[?pur :purchase/account ?acc]
[$since ?pur]]}
db
(d/history (d/since db since))
[:account/id account-id]))
(updated-purchases account-id db #inst "2014-11-14"))

94. serve purchases added or modified since our last sync
29
(defn updated-purchases [account-id db since]
(d/q '{:find [[(pull ?pur [*]) ...]]
:in [$ $since ?acc]
:where [[?pur :purchase/account ?acc]
[$since ?pur]]}
db
(d/history (d/since db since))
[:account/id account-id]))
(updated-purchases account-id db #inst "2014-11-14"))

95. serve purchases added or modified since our last sync
29
(defn updated-purchases [account-id db since]
(d/q '{:find [[(pull ?pur [*]) ...]]
:in [$ $since ?acc]
:where [[?pur :purchase/account ?acc]
[$since ?pur]]}
db
(d/history (d/since db since))
[:account/id account-id]))
(updated-purchases account-id db #inst "2014-11-14"))

96. serve purchases added or modified since our last sync
29
(defn updated-purchases [account-id db since]
(d/q '{:find [[(pull ?pur [*]) ...]]
:in [$ $since ?acc]
:where [[?pur :purchase/account ?acc]
[$since ?pur]]}
db
(d/history (d/since db since))
[:account/id account-id]))
(updated-purchases account-id db #inst "2014-11-14"))

97. serve purchases added or modified since our last sync
29
(defn updated-purchases [account-id db since]
(d/q '{:find [[(pull ?pur [*]) ...]]
:in [$ $since ?acc]
:where [[?pur :purchase/account ?acc]
[$since ?pur]]}
db
(d/history (d/since db since))
[:account/id account-id]))
(updated-purchases account-id db #inst "2014-11-14"))

98. serve purchases added or modified since our last sync
29
(defn updated-purchases [account-id db since]
(d/q '{:find [[(pull ?pur [*]) ...]]
:in [$ $since ?acc]
:where [[?pur :purchase/account ?acc]
[$since ?pur]]}
db
(d/history (d/since db since))
[:account/id account-id]))
(updated-purchases account-id db #inst "2014-11-14"))

99. 30
Bonus

100. 31
5 Future DBs

101. generating a virtual db
(defn db-with-virtual-charges [purchases db]
(:db-after
(d/with db
(mapcat purchase->virtual-charges purchases))))
32

102. generating a virtual db
(defn db-with-virtual-charges [purchases db]
(:db-after
(d/with db
(mapcat purchase->virtual-charges purchases))))
32

103. generating a virtual db
(defn db-with-virtual-charges [purchases db]
(:db-after
(d/with db
(mapcat purchase->virtual-charges purchases))))
32

104. generating a virtual db
(defn db-with-virtual-charges [purchases db]
(:db-after
(d/with db
(mapcat purchase->virtual-charges purchases))))
32

105. generating a virtual db
(defn db-with-virtual-charges [purchases db]
(:db-after
(d/with db
(mapcat purchase->virtual-charges purchases))))
32

106. 33
6 Testing

107. db testing - locally scoped
34
(defn virtual-db [updates db]
(:db-after
(d/with db updates)))
(let [one-account {...}
one-card {...}
one-purchase {...}
wrong-purchase {...}
db (virtual-db [one-account one-card
one-purchase wrong-purchase] db)]
(my-weird-db-query db) => [one-purchase])

108. db testing - locally scoped
34
(defn virtual-db [updates db]
(:db-after
(d/with db updates)))
(let [one-account {...}
one-card {...}
one-purchase {...}
wrong-purchase {...}
db (virtual-db [one-account one-card
one-purchase wrong-purchase] db)]
(my-weird-db-query db) => [one-purchase])

109. db testing - locally scoped
34
(defn virtual-db [updates db]
(:db-after
(d/with db updates)))
(let [one-account {...}
one-card {...}
one-purchase {...}
wrong-purchase {...}
db (virtual-db [one-account one-card
one-purchase wrong-purchase] db)]
(my-weird-db-query db) => [one-purchase])

110. db testing - locally scoped
34
(defn virtual-db [updates db]
(:db-after
(d/with db updates)))
(let [one-account {...}
one-card {...}
one-purchase {...}
wrong-purchase {...}
db (virtual-db [one-account one-card
one-purchase wrong-purchase] db)]
(my-weird-db-query db) => [one-purchase])

111. db testing - locally scoped
34
(defn virtual-db [updates db]
(:db-after
(d/with db updates)))
(let [one-account {...}
one-card {...}
one-purchase {...}
wrong-purchase {...}
db (virtual-db [one-account one-card
one-purchase wrong-purchase] db)]
(my-weird-db-query db) => [one-purchase])

112. db testing - locally scoped
34
(defn virtual-db [updates db]
(:db-after
(d/with db updates)))
(let [one-account {...}
one-card {...}
one-purchase {...}
wrong-purchase {...}
db (virtual-db [one-account one-card
one-purchase wrong-purchase] db)]
(my-weird-db-query db) => [one-purchase])

113. db testing - vector of datoms
35
(fact "on last-modification-for"
(let [customer-id (uuid)
db [[42 :account/customer-id customer-id 99]
[99 :db/txInstant #inst "2014-10-10T12:00:00.909Z" 99]
[1337 :purchase/account 42 100]
[1337 :purchase/id (uuid) 100]
[1337 :purchase/merchant-name "Coiso" 100]
[100 :db/txInstant #inst "2014-10-11T12:00:00.888Z" 100]
[9001 :line-item/account 42 101]
[9001 :charge/id (uuid) 101]
[9001 :line-item/precise-amount 100.00M 101]
[101 :db/txInstant #inst "2014-10-12T13:00:00.777Z" 101]
[45 :account/customer-id (uuid) 102]
[102 :db/txInstant #inst "2014-10-15T12:00:00.909Z" 102]]]
(last-modification-for customer-id db) => #nu/time "2014-10-12T13:00:00.777Z"))

114. db testing - vector of datoms
35
(fact "on last-modification-for"
(let [customer-id (uuid)
db [[42 :account/customer-id customer-id 99]
[99 :db/txInstant #inst "2014-10-10T12:00:00.909Z" 99]
[1337 :purchase/account 42 100]
[1337 :purchase/id (uuid) 100]
[1337 :purchase/merchant-name "Coiso" 100]
[100 :db/txInstant #inst "2014-10-11T12:00:00.888Z" 100]
[9001 :line-item/account 42 101]
[9001 :charge/id (uuid) 101]
[9001 :line-item/precise-amount 100.00M 101]
[101 :db/txInstant #inst "2014-10-12T13:00:00.777Z" 101]
[45 :account/customer-id (uuid) 102]
[102 :db/txInstant #inst "2014-10-15T12:00:00.909Z" 102]]]
(last-modification-for customer-id db) => #nu/time "2014-10-12T13:00:00.777Z"))

115. 36
7 Schema Extension

116. schema is data!
37
(d/transact conn [{:db/valueType :db.type/string,
:db.install/_attribute :db.part/db,
:db/id (d/tempid :db.part/db),
:db/cardinality :db.cardinality/one,
:db/ident :customer/name
}])

117. :nubank/transform :pii
schema is data!
37
(d/transact conn [{:db/valueType :db.type/string,
:db.install/_attribute :db.part/db,
:db/id (d/tempid :db.part/db),
:db/cardinality :db.cardinality/one,
:db/ident :customer/name
}])

118. 38
8 Sharding Reads

119. 39
sharding to the transactor
customers
accounts
notification
acquisition
processor
auth

120. 39
sharding to the transactor
customers
accounts
notification
acquisition
processor
auth

121. customers
1 2 …
39
sharding to the transactor
customers
accounts
notification
acquisition
processor
auth

122. customers
1
2 …
39
sharding to the transactor
customers
accounts
notification
acquisition
processor
auth
shard-specific peer cache

123. customers
1
2
…
39
sharding to the transactor
customers
accounts
notification
acquisition
processor
auth
shard-specific peer cache
inter-shard ACID transactions

124. 40
9 Db Aggregation

125. 41
data science aggregation
customers
accounts
notification
acquisition
processor
auth
data
science

126. 42
querying multiple databases
(defn multi-join [customer-id cus-db acc-db acq-db ntf-db]
(d/q '{:find [...]
:in [$cus $acc $acq $ntf ?cus-id]
:where [[$cus ?cus :customer/id ?cus-id]
[$acc ?acc :account/customer-id ?cus-id]
[$acq ?ar :request/customer-id ?cus-id]
[$ntf ?evt :event/customer-id ?cus-id]
[...]]}
cus-db acc-db acq-db ntf-db customer-id))

127. 42
querying multiple databases
(defn multi-join [customer-id cus-db acc-db acq-db ntf-db]
(d/q '{:find [...]
:in [$cus $acc $acq $ntf ?cus-id]
:where [[$cus ?cus :customer/id ?cus-id]
[$acc ?acc :account/customer-id ?cus-id]
[$acq ?ar :request/customer-id ?cus-id]
[$ntf ?evt :event/customer-id ?cus-id]
[...]]}
cus-db acc-db acq-db ntf-db customer-id))

128. 43
1 Audit Trail
2 Authorization
3 HTTP Cache
4 Mobile Sync
5 Future DBs
6 Testing
7 Schema Extension
8 Sharding Reads
9 Db Aggregation

129. 44
We’re hiring

130. 45
We’re hiring

131. Thanks!
Lucas Cavalcanti & Edward Wible
@lucascs
lucas@nubank.com.br
edward@nubank.com.br