Couch db and_the_web

784 views
734 views

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
784
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Couch db and_the_web

  1. 1. CouchDB and the web
  2. 2. Knut O. Hellan twitter.com/knuthellan knuthellan.com
  3. 3. NoSQL
  4. 4. NoSQL <ul><li>Freedom of choice </li></ul><ul><li>Right tool for the job </li></ul><ul><li>Use relational databases when appropriate </li></ul>
  5. 5. Flavor of the week <ul><li>Lightweight key-value: </li></ul><ul><li>Graph: </li></ul><ul><li>Document stores: </li></ul><ul><li>Server spanning: </li></ul><ul><li>  </li></ul><ul><li>More </li></ul>
  6. 6. CouchDB Overview
  7. 7. Pros and cons <ul><li>RESTful JSON API </li></ul><ul><li>Schema free </li></ul><ul><li>Incremental map reduce views </li></ul><ul><li>Incremental bi-directional replication </li></ul>
  8. 8. Pros and cons <ul><li>Robust append-only </li></ul><ul><li>File backup with cp and tar </li></ul><ul><li>Scalable? </li></ul>
  9. 9. CouchDB views
  10. 10. Our database
  11. 11. Our database <ul><li>{ &quot;_id&quot;: &quot;123bc00725fc9f831209440793001bb6&quot;, &quot;_rev&quot;: &quot;3-f30fca1f7fb4f387207e4756795aac74&quot;, &quot;author&quot;: &quot;jchris&quot;, &quot;tweet&quot;: &quot;This weekend just happened to be a good time for me to hack. Node.js CouchDB and Twitter, makes me feel like I'm part of #nodeko but I'm not&quot;, &quot;type&quot;: &quot;tweet&quot; } </li></ul>
  12. 12. View Design
  13. 13. Default mapper <ul><li>function(doc) {   emit(null, doc); } </li></ul>
  14. 14. Indexer <ul><li>function(doc) { </li></ul><ul><li>  if (!doc.tweet) return; </li></ul><ul><li>  var clean_text = doc.tweet.replace(/W/g, ' ').replace(/  /g, ' ');   var lower_text = clean_text.toLowerCase();   var text_vector = lower_text.split(/ /);   for (var i in text_vector) {     var word = text_vector[i];     if (word.length > 2) emit(word, doc._id);   } } </li></ul>
  15. 15. The mapper output <ul><li>http://localhost:5984/news/_design/parser/_view/index </li></ul><ul><li>{&quot;total_rows&quot;:50,&quot;offset&quot;:0,&quot;rows&quot;:[ {&quot;id&quot;:&quot;123bc00725fc9f83120944079300093e&quot;,&quot;key&quot;:&quot;9xmyxw&quot;,&quot;value&quot;:1}, {&quot;id&quot;:&quot;123bc00725fc9f831209440793000f46&quot;,&quot;key&quot;:&quot;a6exod&quot;,&quot;value&quot;:1}, {&quot;id&quot;:&quot;123bc00725fc9f831209440793000f46&quot;,&quot;key&quot;:&quot;and&quot;,&quot;value&quot;:1}, {&quot;id&quot;:&quot;123bc00725fc9f831209440793001bb6&quot;,&quot;key&quot;:&quot;and&quot;,&quot;value&quot;:1}, {&quot;id&quot;:&quot;123bc00725fc9f83120944079300093e&quot;,&quot;key&quot;:&quot;are&quot;,&quot;value&quot;:1}, </li></ul><ul><li>http://localhost:5984/news/_design/parser/_view/index?key=%22couchdb%22 </li></ul><ul><li>{&quot;total_rows&quot;:50,&quot;offset&quot;:11,&quot;rows&quot;:[ {&quot;id&quot;:&quot;123bc00725fc9f83120944079300093e&quot;,&quot;key&quot;:&quot;couchdb&quot;,&quot;value&quot;:1}, {&quot;id&quot;:&quot;123bc00725fc9f831209440793000f46&quot;,&quot;key&quot;:&quot;couchdb&quot;,&quot;value&quot;:1}, {&quot;id&quot;:&quot;123bc00725fc9f831209440793001bb6&quot;,&quot;key&quot;:&quot;couchdb&quot;,&quot;value&quot;:1} ]} </li></ul>
  16. 16. Add reducer <ul><li>function(keys, values, rereduce) {   return sum(values); } </li></ul><ul><li>  </li></ul><ul><li>http://localhost:5984/news/_design/parser/_view/index </li></ul><ul><li>{&quot;rows&quot;:[ {&quot;key&quot;:null,&quot;value&quot;:50} ]}  http://localhost:5984/news/_design/parser/_view/index?key=%22couchdb%22{&quot;rows&quot;:[ {&quot;key&quot;:null,&quot;value&quot;:3} ]} </li></ul>
  17. 17. Grouping <ul><li>http://localhost:5984/news/_design/parser/_view/index?group_level=1 </li></ul><ul><li>{&quot;rows&quot;:[ {&quot;key&quot;:&quot;9xmyxw&quot;,&quot;value&quot;:1}, {&quot;key&quot;:&quot;a6exod&quot;,&quot;value&quot;:1}, {&quot;key&quot;:&quot;and&quot;,&quot;value&quot;:2}, {&quot;key&quot;:&quot;are&quot;,&quot;value&quot;:1}, </li></ul>
  18. 18. CouchDB replication
  19. 19. Futon Replication
  20. 20. Replication Modes <ul><li>- Copy or replicate once </li></ul><ul><li>  </li></ul><ul><li>- Continuous replication </li></ul><ul><li>- Filtered replication </li></ul>
  21. 21. Multi-Couch
  22. 22. Offline replication <ul><li>- Mobile phone </li></ul><ul><li>- Notebook or tablet and the cloud </li></ul>
  23. 23. Conflicts <ul><li>Multi-master => possible conflicts </li></ul><ul><li>All masters will agree </li></ul><ul><li>Conflict information kept in revision history </li></ul>
  24. 24. Filtered Replication <ul><li>function(doc, req) { if (doc.type && doc.type == &quot;foo&quot;) { return true; } else { return false; } } </li></ul>
  25. 25. Filtered Replication <ul><li>{ &quot;_id&quot;:&quot;myddoc&quot;, &quot;filters&quot;: { &quot;myfilter&quot;: &quot;function goes here&quot; } } </li></ul>
  26. 26. Filtered Replication <ul><li>{  &quot;source&quot;:&quot;http://example.org/example-database&quot;,  &quot;target&quot;:&quot;http://admin:password@127.0.0.1:5984/example-database&quot;,   &quot;filter&quot;:&quot;myddoc/myfilter&quot;} </li></ul>
  27. 27. CouchDB scaling
  28. 28. Alternatives <ul><li>- CouchDB lounge  </li></ul><ul><li>  </li></ul><ul><li>- </li></ul><ul><li>  </li></ul><ul><li>- Pillow </li></ul>
  29. 29. Pillow <ul><li>- Reuses CouchDB functionality </li></ul><ul><li>  </li></ul><ul><li>- Transparent for users </li></ul><ul><li>  </li></ul><ul><li>- Automatic resharding </li></ul>
  30. 30. Pillow
  31. 31. Pillow
  32. 32. Pillow
  33. 33. Pillow
  34. 34. Pillow
  35. 35. CouchDB lessons learned
  36. 36. CouchDB Experience <ul><li>Map Reduce based views are powerful </li></ul><ul><li>View experimentation in Futon </li></ul><ul><li>Three-way multi-master replication </li></ul><ul><li>Replication monitor </li></ul>
  37. 37. CouchDB Experience <ul><li>EC2 </li></ul><ul><li>External middleware still useful </li></ul><ul><li>Duck typing matches CouchDB's JSON </li></ul>
  38. 38. Now what? <ul><li>Install and play with CouchDB </li></ul><ul><li>Visit couchdb.apache.org </li></ul><ul><li>Visit knuthellan.com </li></ul>
  39. 39. relax

×