• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Dirty - How simple is your database?
 

Dirty - How simple is your database?

on

  • 9,475 views

A tiny & fast key value store with append-only disk log. Ideal for apps with < 1 million records.

A tiny & fast key value store with append-only disk log. Ideal for apps with < 1 million records.

Statistics

Views

Total Views
9,475
Views on SlideShare
9,444
Embed Views
31

Actions

Likes
4
Downloads
40
Comments
0

2 Embeds 31

http://lanyrd.com 16
http://rg443blog.wordpress.com 15

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Dirty - How simple is your database? Dirty - How simple is your database? Presentation Transcript

    • Dirty NoSQL How simple is your database? 25.09.2010
    • About Contributor Co-founder Felix Geisendörfer node.js driver node.js driver
    • Dirty
    • Dirty JavaScript views In-memory Dirty
    • Design choices
    • Non-blocking I/O FTW
    • 150 lines of code Lines of code Dirty CouchDB Redis 0 5.000 10.000 15.000 20.000
    • Append-only JSON log $ cat dirty.db {"key": "first", "val": {"foo": "bar"}} {"key": "second", "val": "A string"} {"key": "a-number", "val": 23}
    • No Networking var db = require('dirty')('test.db'); db.set('foo', 'bar'); db.get('foo'); // => bar
    • No Networking var db = require('dirty')('languages.db'); db.set('javascript', {age: 15}); db.set('python', {age: 19}); db.set('perl', {age: 23}); db.forEach(function(key, doc) { console.log('%s is %d years old.', key, doc.age); });
    • Performance
    • Benchmarks Do your own!
    • dirty.get() 50 MHz (50 million / s) v8: 160 MHz
    • dirty.set() 5 MHz (5 million / s) v8: 12 MHz
    • dirty.set() With flushing to disk Numbers 256 byte string 200 kHz 70 kHz (200-thousand / s) (70-thousand / s)
    • dirty.forEach() 4.5 MHz (4.5 million / s) v8: 33 MHz
    • The Wall • Dirty is a wonderful database as long as you have < 1 million records • After that, you hit “The Wall”
    • Scaling beyond Possibilities for future node.js based databases
    • 22.000 2010 The Internet 20.000 18.000 16.000 2009 14.000 12.000 PB / month 2008 10.000 8000 2007 6000 2006 4000 2005 2000 2004 2003 2002 2000 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Year
    • 22 exabyte / month (that is 22 billion gigabytes)
    • Flexible guarantees db.set('my-key', 'my-value', function(err) { if (err) throw err; console.log('Record written to disk.'); }); console.log('Record written to memory.');
    • Memory / Disk Hybrids • “Memcached” built into your database • Better knowledge about your data than any general purpose algorithm
    • Replication • Node.js = perfect for streaming between instances • Node could “hold” the connection if a not- yet replicated key is being requested
    • Web Services • Node.js could act as a proxy for different database backends (written in node or not) • One could query 3rd party services for information
    • Questions? ✎ @felixge / felix@debuggable.com
    • Get it $ npm install dirty or http://github.com/felixge/node-dirty
    • Don’t hit the wall
    • Questions? ✎ @felixge / felix@debuggable.com
    • Sources • Internet traffic numbers: http://en.wikipedia.org/wiki/Internet_traffic • Wonderful dino-rider vs. volcano image: Jesse Star / http://www.geekologie.com/ 2009/10/dinorider_geekologie_writer_vs.php
    • Memory overhead (for setting numeric key / values) • 20mb overhead / 1 million records • 20 bytes / key