CouchDB on Rails
An Introduction




Jonathan Weiss
31.05.2010
Who am I?

Working for Peritor in Berlin, Germany

Written, maintain, or involved in
   Webistrano
   Capistrano
   Sim...
Scalarium


EC2 Cluster Management
   Auto-config
   Self-Healing
   Auto-Scaling
   One-click-deployment




www.scala...
Database Requirements


High Availability

Easy Replication

Clustering

Robustness

Short Recovery Time




             ...
5
CouchDB


Build for the Web

Scales

Replication built-in

Embracing offline

Flexible schema – document DB




           ...
”CouchDB is built of the Web“
              Jacob Kaplan-Moss




                                  7
Web Technologies

HTTP
 the access protocol



JavaScript
  the query language



JSON
  the storage format




          ...
JSON Document

 {
     "_id": "BCCD12CBB",
     "_rev": "1-AB764C",
     "type": "person",
     "name": "Darth Vader",
   ...
JSON Document

 {
     "_id": "BCCD12CBB",
     "_rev": "1-AB764C",
     "type": "person",
     "name": "Darth Vader",
   ...
JSON Document

 {
     "_id": "BCCD12CBB",
     "_rev": "1-AB764C",
     "type": "person",
     "name": "Darth Vader",
   ...
No Tables or Namespaces




                          12
No Tables or Namespaces




                          13
Manual Namespacing

 {
     "_id": "BCCD12CBB",
     "_rev": "1-AB764C",
     "type": "person",
     "name": "Darth Vader"...
Interacting with CouchDB




                   JSON


HTTP Client

              PUT /dbname/ID



                      ...
Interacting with CouchDB



              GET /dbname/ID



HTTP Client



                   JSON


                     ...
Interacting with CouchDB




              DELETE /dbname/ID
HTTP Client




                                  17
Views




        18
Design Document
 {
     "id": "_design/hats”,
     "_rev": "431212AB4”,
     "language": "javascript”,
     "views": {
   ...
Design Document
 {                                               Document ID
     "id": "_design/hats”,                   ...
Design Document
 {
     "id": "_design/hats”,
     "_rev": "431212AB4”,
     "language": "javascript”,
     "views": {
   ...
Design Document
 {
     "id": "_design/hats”,
     "_rev": "431212AB4”,
     "language": "javascript”,
     "views": {
   ...
Map



function(doc) {
    if (doc.headware) {
      for (var hat in doc.headware) {
        emit(hat, 1);
      }
    }
}...
Map
                                  Passed every
                               document in the DB

function(doc) {
    ...
Map



function(doc) {
    if (doc.headware) {
                                        Inspects
      for (var hat in doc....
Map



function(doc) {
    if (doc.headware) {
      for (var hat in doc.headware) {
        emit(hat, 1);
      }
    }  ...
Reduce



 function(keys, values, rereduce) {
     return sum(values);
 }




                                      27
Reduce
                                     Passed map result
                                  (or partial reduce result)...
Reduce



 function(keys, values, rereduce) {
     return sum(values);
 }




           Aggregates
     (count, sum, aver...
Example Map Result

Map functions are similar to SQL indices

                ID               KEY       VALUE

         5...
Query a view


        GET /dbname/_design/hats/_view/all




HTTP Client


              {"total_rows":348,"offset":0,"ro...
Query a view


       GET /dbname/_design/hats/_view/all?
        include_docs=true




HTTP Client




                  ...
View Query


Filter by
   key=ABC123
   startkey=123 & endkey=9
   limit=100
   descending=true
   group=true
   red...
SQL vs. JavaScript




                     Vs.




                           34
SQL vs. JavaScript


                            ActiveRecord



                      Vs.



       SimplyStored



     ...
SimplyStored


Convenience Layer for CouchDB
   Models & Associations
   Validations
   Callbacks               BSD-lic...
Setup
Install




Load in environment.rb




Configure




                         37
Setup




        38
39
40
RockingChair


In-memory CouchDB
   Just a big Hash
   Understands all SimplyStored generated views
   Speeds up tests
...
Database Requirements


High Availability

Easy Replication

Clustering

Robustness

Short Recovery Time




             ...
Replication
    B-Tree
XXXXX




              Photo by Mathias Meyer
                                       43
B-Tree

Append only

Concurrency (MVCC)

Crash resistant

Hot backups

Compaction




                     44
Replication




              45
CouchDB Replication



                      POST /_replicate




                      POST /_replicate


Eventually cons...
Load Balancing




                             Replication


 HTTP Client     HTTP Load
                 Balancer




   ...
Caching




 HTTP Client   HTTP Cache
               Varnish
               Apache
               …




                  ...
Multi-Master




               49
Sharding/Partitioning with
CouchDB Lounge




 HTTP Client    CouchDB
                Lounge




                         ...
Sharding with CouchDB Lounge




 HTTP Client   CouchDB
               Lounge




                               51
Various

CouchApps

Validations

Filtered replication

Changes feed

Futon

Geo

Fulltext-Search with embedded Lucene

Exp...
Q&A
Peritor GmbH
Blücherstr. 22, Hof III Aufgang 6
10961 Berlin
Tel.: +49 (0)30 69 20 09 84 0
Fax: +49 (0)30 69 20 09 84 9...
Upcoming SlideShare
Loading in …5
×

CouchDB on Rails - RailsWayCon 2010

3,865 views
3,601 views

Published on

An introduction to CouchDB and how to leverage it from Ruby/Rails presented by Jonathan Weiss at RailsWayCon 2010 in Berlin.

Published in: Technology, Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,865
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
36
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

CouchDB on Rails - RailsWayCon 2010

  1. CouchDB on Rails An Introduction Jonathan Weiss 31.05.2010
  2. Who am I? Working for Peritor in Berlin, Germany Written, maintain, or involved in   Webistrano   Capistrano   SimplyStored   Happening   The great fire of London http://github.com/jweiss @jweiss 2
  3. Scalarium EC2 Cluster Management   Auto-config   Self-Healing   Auto-Scaling   One-click-deployment www.scalarium.com 3
  4. Database Requirements High Availability Easy Replication Clustering Robustness Short Recovery Time 4
  5. 5
  6. CouchDB Build for the Web Scales Replication built-in Embracing offline Flexible schema – document DB 6
  7. ”CouchDB is built of the Web“ Jacob Kaplan-Moss 7
  8. Web Technologies HTTP the access protocol JavaScript the query language JSON the storage format 8
  9. JSON Document { "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } } 9
  10. JSON Document { "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } } 10
  11. JSON Document { "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } } 11
  12. No Tables or Namespaces 12
  13. No Tables or Namespaces 13
  14. Manual Namespacing { "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } } 14
  15. Interacting with CouchDB JSON HTTP Client PUT /dbname/ID 15
  16. Interacting with CouchDB GET /dbname/ID HTTP Client JSON 16
  17. Interacting with CouchDB DELETE /dbname/ID HTTP Client 17
  18. Views 18
  19. Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } } 19
  20. Design Document { Document ID "id": "_design/hats”, – "_rev": "431212AB4”, prefixed by “_design/” "language": "javascript”, "views": { "all": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } } 20
  21. Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { Hash of Views "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } } 21
  22. Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { Hash of Views "map": "function(doc){ .... }”, Every view has map & "reduce": "function(doc){ .... }” reduce function }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } } 22
  23. Map function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } } 23
  24. Map Passed every document in the DB function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } } 24
  25. Map function(doc) { if (doc.headware) { Inspects for (var hat in doc.headware) { & Decides emit(hat, 1); } } } 25
  26. Map function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } Emits result } for index 26
  27. Reduce function(keys, values, rereduce) { return sum(values); } 27
  28. Reduce Passed map result (or partial reduce result) function(keys, values, rereduce) { return sum(values); } 28
  29. Reduce function(keys, values, rereduce) { return sum(values); } Aggregates (count, sum, average, …) 29
  30. Example Map Result Map functions are similar to SQL indices ID KEY VALUE 51ABFA211 Cap 1 ABC123456 Cappy 1 BCCD12CBB Helmet 1 BCCD12CBB Sombrero 1 Sorted by the key Key can also be an array Value can be complex JSON 30
  31. Query a view GET /dbname/_design/hats/_view/all HTTP Client {"total_rows":348,"offset":0,"rows”:[ {"id":"A","key":"A","value":1}, {"id":"B","key":"B","value":1}, ]} 31
  32. Query a view GET /dbname/_design/hats/_view/all? include_docs=true HTTP Client 32
  33. View Query Filter by   key=ABC123   startkey=123 & endkey=9   limit=100   descending=true   group=true   reduce=true   Include_docs=true 33
  34. SQL vs. JavaScript Vs. 34
  35. SQL vs. JavaScript ActiveRecord Vs. SimplyStored 35
  36. SimplyStored Convenience Layer for CouchDB   Models & Associations   Validations   Callbacks BSD-licensed on   Dynamic finder http://github.com/peritor/simply_stored   S3 attachments On top of CouchPotato, CouchRest & RestClient   Paranoid delete   ActiveModel compliant 36
  37. Setup Install Load in environment.rb Configure 37
  38. Setup 38
  39. 39
  40. 40
  41. RockingChair In-memory CouchDB   Just a big Hash   Understands all SimplyStored generated views   Speeds up tests   Tests can run in parallel   Nice for debugging BSD-licensed on http://github.com/jweiss/rocking_chair 41
  42. Database Requirements High Availability Easy Replication Clustering Robustness Short Recovery Time 42
  43. Replication B-Tree XXXXX Photo by Mathias Meyer 43
  44. B-Tree Append only Concurrency (MVCC) Crash resistant Hot backups Compaction 44
  45. Replication 45
  46. CouchDB Replication POST /_replicate POST /_replicate Eventually consistent & conflict resolution 46
  47. Load Balancing Replication HTTP Client HTTP Load Balancer 47
  48. Caching HTTP Client HTTP Cache Varnish Apache … 48
  49. Multi-Master 49
  50. Sharding/Partitioning with CouchDB Lounge HTTP Client CouchDB Lounge 50
  51. Sharding with CouchDB Lounge HTTP Client CouchDB Lounge 51
  52. Various CouchApps Validations Filtered replication Changes feed Futon Geo Fulltext-Search with embedded Lucene Experimental Ruby-View-Server 52
  53. Q&A Peritor GmbH Blücherstr. 22, Hof III Aufgang 6 10961 Berlin Tel.: +49 (0)30 69 20 09 84 0 Fax: +49 (0)30 69 20 09 84 9 Internet: www.peritor.com E-Mail: info@peritor.com © Peritor GmbH - Alle Rechte vorbehalten

×