Datomic is a distributed database that uses a Datalog query language and ACID transactions. It stores data as triples of entity-attribute-value and supports various data types and cardinality. Transactions are serialized and update indices in the background. The architecture includes Peer, Transactor, and Storage components. Queries are performed directly on indices using the query and entity APIs.

1. Venturing into
Datomic

2. Kirill Salykin
Former .NET, Ruby developer
Now happy clojurist at AdGoji :)

3. Adgoji is a media agency.
We help advertisers find value on mobile.
From our offices in Amsterdam, we connect
advertisers with the right* mobile users
worldwide. To do so, we have built a Clojure
based real-time bidding platform that buys
targeted mobile advertisement from the largest
market places in the world.
• and by “right”, we mean users willing to see
and/or interact with (click, install, purchase)

4. Kudos to Carlo
For organising meetup every month

5. What is Datomic?
According to Wikipedia: Datomic is a distributed
database and implementation of Datalog.It
has ACID transactions, joins, and a logical query language,
Datalog.
Designed by Rich Hickey

6. Architecture
Datomic (on-prem):
- Peer
- Transactor
- Storage

8. Peer
- Lives in the app
- Does query
- Persistent index + Live index
- Receives updates from transactor

9. Transactor
- Handles transactions
- Commits results to the Storage
- Single threaded
- Transmit changes to connected peers
- Live index
- Index in background, place index to the storage

10. Storage
- Stores the data :)
- Supported storages: DynamoDB, SQL,
Cassandra

11. Information model
Datomic uses Triple to describe data - Datom
Entity Attribute Value
1 :likes “pizza”
1 :likes “lasagna”
1 :first-name “Kirill”

12. Information model 2
Datomic extends triple to include operation
Add or Retract
Entity Attribute Value Op (Add?)
1 :likes “pizza” True
1 :likes “lasagna” True
1 :first-name “Kirill” True
1 :likes “lasagna” False

13. Information model 3
Every added/retracted Datum also references
transaction
Entity Attribute Value Op (Add?) Transaction
1 :likes “pizza” True 2
1 :likes “lasagna” True 2
1 :first-name “Kirill” True 2
1 :likes “lasagna” False 3

14. Information model 4
Schema
{:db/ident :person/name
:db/valueType :db.type/string
:db/cardinality :db.cardinality/one}
valueType:
- keyword
- string
- boolean
- long
- bigint
- float
- double
- bigdec
- ref
- instant
- uuid
- uri
- bytes
cardinality: one or many

15. Transactions
- Serialised
- No `begin transaction` `commit/rollback`
- Updates indices in the background
- Sends updates to the connected peers
- String tempids (references)
- Returns db-before, db-after and tempids
resolution

16. Transactions 2
- Indexes are B-tree-like structures: sorted, immutable, persistent,
1,000+ branching factor, custom sort order, fast lookup and range
scans, able to be efficiently merged (details unknown).
- Index trees are shallow, no more than three level deep: root node,
directories and segments as leafs.

17. Transactions 3
- Constraints via transaction functions
- Runs during transaction
- On exception - transaction rollbacks

18. Query
- Directly via indices
- Query API + Entity API
- Pull API

19. Query 2
- Indicies:
- Covering (contains data, not reference)
- Element of index is not datom, but segment
- Segment is from 1,000 to 20,000 datoms (~50 Kb)
- EAVT
- AEVT
- VAET
- AVET
- Log index
- Keeps fetched segments in the memory

20. Features
- No query planner (you responsible for optimal query)
- Query result must feet in the memory (no dump to file,
as sql does)
- Schema alteration can be tricky (even not possible sometimes)
- No compound indices
- Sort done manually
- Pagination done manually (no offset/limit in query defined)
- Cant add `nil` attribute value - retract only
- RetractEntity to retract all attributes of entity
- Stu’s 10 billion datoms (soft limit)
- Huge heap - may influence GC?
- GC is a manual operation (garbage segments may still be used?
run rarely, e.g. once a week)
- Memcache support out-of-the-box
- Segments easy to cache - immutable
- Excision - not to correct, but because forced (for instance GDPR)

21. Resources
- Rich Hickey: Deconstructing the Database
https://www.youtube.com/watch?v=Cym4TZwTCNU
- Nitika Prokopov: Unofficial guide to Datomic internals
http://tonsky.me/blog/unofficial-guide-to-datomic-internals
- Stu Halloway: Day of Datomic
https://docs.datomic.com/on-prem/day-of-datomic.html
- http://www.learndatalogtoday.org
- Official docs: https://docs.datomic.com/on-prem/index.html