Btree Nosql Oak

2,805
-1

Published on

An introduction to CouchDB's append-only storage model.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,805
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
65
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Btree Nosql Oak

  1. 1. Apache CouchDB Monday, November 2, 2009 how couchdb treats the disks append only btree
  2. 2. Replication JSON Map Reduce HTTP Monday, November 2, 2009 intro: offline apps simple development easy to deploy
  3. 3. Stuart Langridge - Canonical ! ! Monday, November 2, 2009 15 million desktops by the end of the year. solve the data island problem
  4. 4. Monday, November 2, 2009 raindrop - next gen messaging platform name: your own personal piece of the cloud
  5. 5. Monday, November 2, 2009
  6. 6. Monday, November 2, 2009 Couchapps -- easiest way to deploy an Ajax application that needs to store data. -- capable of scaling up with CouchDB. (etags etc)
  7. 7. “Ground Computing” @jhuggins http://www.flickr.com/photos/mcpig/872293700/ Monday, November 2, 2009 app runs near the user. lower latency, lower cost.
  8. 8. Monday, November 2, 2009
  9. 9. Offline by Default http://www.flickr.com/photos/shane-h/280084650 Monday, November 2, 2009 reliable telephone poles / like akamai but for dynamic database backed applications. - same application on the client and server or the cloud
  10. 10. http://apod.nasa.gov/apod/ap050930.html “Of the Web” http://jacobian.org/writing/of-the-web/ Monday, November 2, 2009 Let me tell you something: Django may be built for the Web, but CouchDB is built of the Web. I've never seen software that so completely embraces the philosophies behind HTTP. ... this is what the software of the future looks like. Jacob Kaplan-Moss -- October, 2007 perfect spot - discovered not invented When you can move data via replication, it changes the physics of the web so data is useful independent of location.
  11. 11. Lockless Storage Monday, November 2, 2009 html5 story js ruby transactions? -- getting CouchDB right involves going all the way to core lockless storage engine. performance, map reduce, and deployment
  12. 12. B-Tree Monday, November 2, 2009 data structure sorted by keys minimize disk seeks
  13. 13. Monday, November 2, 2009 logical btree
  14. 14. Monday, November 2, 2009 indexes - efficient key lookups range scans
  15. 15. Append Only Monday, November 2, 2009 robust / reliable never touch bytes that are on disk can’t be inconsistent
  16. 16. Monday, November 2, 2009 append only - in action update J to a new value (rewrite j’s leaf node)
  17. 17. Monday, November 2, 2009 rewrite the path along the tree to J up the parent path until we write the root.
  18. 18. Monday, November 2, 2009 all pointers point backwards into the file
  19. 19. Monday, November 2, 2009 update cost? only has O(log n) cost where n is the # of keys
  20. 20. Monday, November 2, 2009 another update - add 5. rewrite path to Z node to add 5
  21. 21. Monday, November 2, 2009 time follows the blue arrow. top half is actually an accumulation of updates like the bottom. note: batch efficiency -- save block updates
  22. 22. Monday, November 2, 2009 2 day test run couchdb over long haul 4 writers, 16 readers, random access, small keyspace - lots of 409s etc.
  23. 23. Incremental Reduce Monday, November 2, 2009 takes advantage of the write pattern we’ve seen setup main storage doc count example
  24. 24. Monday, November 2, 2009 rereduce = false count # of keys in the block
  25. 25. Monday, November 2, 2009 rereduce = true sum # of docs in child nodes flexible range querying
  26. 26. Monday, November 2, 2009 intermediate reductions stored during a view calculation updating J doesn’t change any counts, adding the 5 key does
  27. 27. Monday, November 2, 2009 we are rewriting the parent nodes anyway, so it makes sense to run the reduce calculation
  28. 28. Concurrent Access Monday, November 2, 2009 readers see a consistent view we can support many readers reads don’t block writes writes don’t block reads
  29. 29. Monday, November 2, 2009 concurrent readers list all docs, then a write comes in, you don’t want to be surprised by an extra doc readers can be as long lived as they want without blocking writers or seeing inconsistencies
  30. 30. Local Web Platform Monday, November 2, 2009 user interviews
  31. 31. http://www.longnow.org/projects/clock/orrery/ Monday, November 2, 2009 append-only no fixup phase reliable and concurrent sequential write pattern (few writer seeks) hot backups compaction

×