Vertex a graph database
disclaimer <ul><li>v0.1 make it work </li></ul><ul><li>~v0.5 <- we are here* </li></ul><ul><li>v1.0 make it right </li></u...
overview <ul><li>filesystem inspired </li></ul><ul><li>small, simple, fast </li></ul><ul><li>garbage collected </li></ul>
details <ul><li>HTTP protocol </li></ul><ul><li>JSON data format </li></ul><ul><li>transactions </li></ul><ul><li>queues <...
data model <ul><li>nodes are collections  </li></ul><ul><li>of lexically ordered key/value pairs </li></ul><ul><li>a value...
API <ul><li>mkdir, rm, size, link </li></ul><ul><li>read, write, select </li></ul><ul><li>queuePopTo, queueExpireTo </li><...
sample requests <ul><li>/users/joe?action=mkdir </li></ul><ul><li>/users/joe/zip?action=write& </li></ul><ul><li>value=941...
sample response <ul><li>[ </li></ul><ul><li>{&quot;name&quot;=&quot;joe&quot;, &quot;zip&quot;=&quot;94114&quot;}, </li></...
implementation <ul><li>in C </li></ul><ul><li>TokyoCabinet - storage </li></ul><ul><li>libevent - async networking </li></...
how it works <ul><li>async i/o </li></ul><ul><li>requests handled serially </li></ul><ul><li>use small requests </li></ul>...
storage format <ul><li>values </li></ul><ul><li>nodeId/s/key = value </li></ul><ul><li>meta values </li></ul><ul><li>nodeI...
performance* <ul><li>OSX </li></ul><ul><li>~1,000s requests/sec </li></ul><ul><li>Linux </li></ul><ul><li>~10,000s request...
multiple indexes? <ul><li>client code  </li></ul><ul><li>establishes conventions & </li></ul><ul><li>maintains indexes </l...
lessons <ul><li>flexible data model is powerful  </li></ul><ul><li>rapid development </li></ul><ul><li>dealing with indexe...
to-do  <ul><li>AUTH & SSL </li></ul><ul><li>rewrite in Lua </li></ul><ul><li>Lua as query & processing language </li></ul>...
would-like-to-do <ul><li>distributed </li></ul><ul><li>in-memory durable commits </li></ul><ul><li>persistent connections ...
folks <ul><li>Steve Dekorte  </li></ul><ul><li>Rich Collins </li></ul><ul><li>helpful patches from: </li></ul><ul><li>Hasa...
source github.com/stevedekorte/vertexdb
Upcoming SlideShare
Loading in …5
×

vertexdb

2,663 views

Published on

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,663
On SlideShare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
19
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

vertexdb

  1. 1. Vertex a graph database
  2. 2. disclaimer <ul><li>v0.1 make it work </li></ul><ul><li>~v0.5 <- we are here* </li></ul><ul><li>v1.0 make it right </li></ul><ul><li>v2.0 make it efficient </li></ul><ul><li>*in use at stylous.com </li></ul>
  3. 3. overview <ul><li>filesystem inspired </li></ul><ul><li>small, simple, fast </li></ul><ul><li>garbage collected </li></ul>
  4. 4. details <ul><li>HTTP protocol </li></ul><ul><li>JSON data format </li></ul><ul><li>transactions </li></ul><ul><li>queues </li></ul>
  5. 5. data model <ul><li>nodes are collections </li></ul><ul><li>of lexically ordered key/value pairs </li></ul><ul><li>a value can contain data </li></ul><ul><li>or a pointer to another node </li></ul><ul><li>keys for data begin with underscore </li></ul>
  6. 6. API <ul><li>mkdir, rm, size, link </li></ul><ul><li>read, write, select </li></ul><ul><li>queuePopTo, queueExpireTo </li></ul><ul><li>transaction </li></ul>
  7. 7. sample requests <ul><li>/users/joe?action=mkdir </li></ul><ul><li>/users/joe/zip?action=write& </li></ul><ul><li>value=94114 </li></ul><ul><li>/users?action=select& </li></ul><ul><li>whereKey=zip& </li></ul><ul><li>whereValue=94114 </li></ul>
  8. 8. sample response <ul><li>[ </li></ul><ul><li>{&quot;name&quot;=&quot;joe&quot;, &quot;zip&quot;=&quot;94114&quot;}, </li></ul><ul><li>{&quot;name&quot;=&quot;jill&quot;, &quot;zip&quot;=&quot;94114&quot;} </li></ul><ul><li>] </li></ul>
  9. 9. implementation <ul><li>in C </li></ul><ul><li>TokyoCabinet - storage </li></ul><ul><li>libevent - async networking </li></ul><ul><li>YAJL - json generation </li></ul><ul><li>basic unit test coverage </li></ul>
  10. 10. how it works <ul><li>async i/o </li></ul><ul><li>requests handled serially </li></ul><ul><li>use small requests </li></ul><ul><li>periodic backups & </li></ul><ul><li>garbage collection </li></ul>
  11. 11. storage format <ul><li>values </li></ul><ul><li>nodeId/s/key = value </li></ul><ul><li>meta values </li></ul><ul><li>nodeId/m/key = value </li></ul><ul><li>could easily be distributed </li></ul>
  12. 12. performance* <ul><li>OSX </li></ul><ul><li>~1,000s requests/sec </li></ul><ul><li>Linux </li></ul><ul><li>~10,000s requests/sec? </li></ul><ul><li>*not durable until periodic backup </li></ul>
  13. 13. multiple indexes? <ul><li>client code </li></ul><ul><li>establishes conventions & </li></ul><ul><li>maintains indexes </li></ul><ul><li>path/all </li></ul><ul><li>path/byDate </li></ul><ul><li>path/byProductId </li></ul>
  14. 14. lessons <ul><li>flexible data model is powerful </li></ul><ul><li>rapid development </li></ul><ul><li>dealing with indexes is a hassle </li></ul>
  15. 15. to-do <ul><li>AUTH & SSL </li></ul><ul><li>rewrite in Lua </li></ul><ul><li>Lua as query & processing language </li></ul><ul><li>replace TC with BDB or skipdb </li></ul><ul><li>native FUSE support </li></ul><ul><li>incremental gc </li></ul>
  16. 16. would-like-to-do <ul><li>distributed </li></ul><ul><li>in-memory durable commits </li></ul><ul><li>persistent connections </li></ul><ul><li>value versioning </li></ul>
  17. 17. folks <ul><li>Steve Dekorte </li></ul><ul><li>Rich Collins </li></ul><ul><li>helpful patches from: </li></ul><ul><li>Hasan Veldstra (PyFUSE) </li></ul><ul><li>BooBSD </li></ul>
  18. 18. source github.com/stevedekorte/vertexdb

×