Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,881
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
17
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Vertex a graph database
  • 2. disclaimer
    • v0.1 make it work
    • ~v0.5 <- we are here*
    • v1.0 make it right
    • v2.0 make it efficient
    • *in use at stylous.com
  • 3. overview
    • filesystem inspired
    • small, simple, fast
    • garbage collected
  • 4. details
    • HTTP protocol
    • JSON data format
    • transactions
    • queues
  • 5. data model
    • nodes are collections
    • of lexically ordered key/value pairs
    • a value can contain data
    • or a pointer to another node
    • keys for data begin with underscore
  • 6. API
    • mkdir, rm, size, link
    • read, write, select
    • queuePopTo, queueExpireTo
    • transaction
  • 7. sample requests
    • /users/joe?action=mkdir
    • /users/joe/zip?action=write&
    • value=94114
    • /users?action=select&
    • whereKey=zip&
    • whereValue=94114
  • 8. sample response
    • [
    • {&quot;name&quot;=&quot;joe&quot;, &quot;zip&quot;=&quot;94114&quot;},
    • {&quot;name&quot;=&quot;jill&quot;, &quot;zip&quot;=&quot;94114&quot;}
    • ]
  • 9. implementation
    • in C
    • TokyoCabinet - storage
    • libevent - async networking
    • YAJL - json generation
    • basic unit test coverage
  • 10. how it works
    • async i/o
    • requests handled serially
    • use small requests
    • periodic backups &
    • garbage collection
  • 11. storage format
    • values
    • nodeId/s/key = value
    • meta values
    • nodeId/m/key = value
    • could easily be distributed
  • 12. performance*
    • OSX
    • ~1,000s requests/sec
    • Linux
    • ~10,000s requests/sec?
    • *not durable until periodic backup
  • 13. multiple indexes?
    • client code
    • establishes conventions &
    • maintains indexes
    • path/all
    • path/byDate
    • path/byProductId
  • 14. lessons
    • flexible data model is powerful
    • rapid development
    • dealing with indexes is a hassle
  • 15. to-do
    • AUTH & SSL
    • rewrite in Lua
    • Lua as query & processing language
    • replace TC with BDB or skipdb
    • native FUSE support
    • incremental gc
  • 16. would-like-to-do
    • distributed
    • in-memory durable commits
    • persistent connections
    • value versioning
  • 17. folks
    • Steve Dekorte
    • Rich Collins
    • helpful patches from:
    • Hasan Veldstra (PyFUSE)
    • BooBSD
  • 18. source github.com/stevedekorte/vertexdb