CouchDB on Rails
An Introduction




FrozenRails
Jonathan Weiss
07.05.2010
Who am I?

Working for Peritor in Berlin, Germany

Written, maintain, or involved in
   Webistrano
   Capistrano
   Sim...
Scalarium


EC2 Cluster Management
   Auto-config
   Self-Healing
   Auto-Scaling
   One-click-deployment




www.scala...
Database Requirements


High Availability

Easy Replication

Clustering

Robustness

Short Recovery Time




             ...
5
CouchDB


Build for the Web

Scales

Replication built-in

Embracing offline

Flexible schema – document DB




           ...
”CouchDB is built of the Web“
              Jacob Kaplan-Moss




                                  7
Web Technologies

HTTP
 the access protocol



JavaScript
  the query language



JSON
  the storage format




          ...
JSON Document

 {
     "_id": "BCCD12CBB",
     "_rev": "1-AB764C",
     "type": "person",
     "name": "Darth Vader",
   ...
JSON Document

 {
     "_id": "BCCD12CBB",
     "_rev": "1-AB764C",
     "type": "person",
     "name": "Darth Vader",
   ...
JSON Document

 {
     "_id": "BCCD12CBB",
     "_rev": "1-AB764C",
     "type": "person",
     "name": "Darth Vader",
   ...
No Tables or Namespaces




                          12
No Tables or Namespaces




                          13
Manual Namespacing

 {
     "_id": "BCCD12CBB",
     "_rev": "1-AB764C",
     "type": "person",
     "name": "Darth Vader"...
Interacting with CouchDB




                   JSON


HTTP Client

              PUT /dbname/ID



                      ...
Interacting with CouchDB



              GET /dbname/ID



HTTP Client



                   JSON


                     ...
Interacting with CouchDB




              DELETE /dbname/ID
HTTP Client




                                  17
Views




        18
Design Document
 {
     "id": "_design/hats”,
     "_rev": "431212AB4”,
     "language": "javascript”,
     "views": {
   ...
Design Document
 {                                               Document ID
     "id": "_design/hats”,                   ...
Design Document
 {
     "id": "_design/hats”,
     "_rev": "431212AB4”,
     "language": "javascript”,
     "views": {
   ...
Design Document
 {
     "id": "_design/hats”,
     "_rev": "431212AB4”,
     "language": "javascript”,
     "views": {
   ...
Map



function(doc) {
    if (doc.headware) {
      for (var hat in doc.headware) {
        emit(hat, 1);
      }
    }
}...
Map
                                  Passed every
                               document in the DB

function(doc) {
    ...
Map



function(doc) {
    if (doc.headware) {
                                        Inspects
      for (var hat in doc....
Map



function(doc) {
    if (doc.headware) {
      for (var hat in doc.headware) {
        emit(hat, 1);
      }
    }  ...
Reduce



 function(keys, values, rereduce) {
     return sum(values);
 }




                                      27
Reduce
                                     Passed map result
                                  (or partial reduce result)...
Reduce



 function(keys, values, rereduce) {
     return sum(values);
 }




           Aggregates
     (count, sum, aver...
Example Map Result

Map functions are similar to SQL indices

                ID               KEY       VALUE

         5...
Query a view


        GET /dbname/_design/hats/_view/all




HTTP Client


              {"total_rows":348,"offset":0,"ro...
Query a view


       GET /dbname/_design/hats/_view/all?
        include_docs=true




HTTP Client




                  ...
View Query


Filter by
   key=ABC123
   startkey=123 & endkey=9
   limit=100
   descending=true
   group=true
   red...
SQL vs. JavaScript




                     Vs.




                           34
SQL vs. JavaScript


                            ActiveRecord



                      Vs.



       SimplyStored



     ...
SimplyStored


Convenience Layer for CouchDB
   Models & Associations
   Validations
   Callbacks               BSD-lic...
Setup
Install




Load in environment.rb




Configure




                         37
Setup




        38
39
40
RockingChair


In-memory CouchDB
   Just a big Hash
   Understands all SimplyStored generated views
   Speeds up tests
...
Database Requirements


High Availability

Easy Replication

Clustering

Robustness

Short Recovery Time




             ...
Replication
    B-Tree
XXXXX




              Photo by Mathias Meyer
                                       43
B-Tree

Append only

Concurrency (MVCC)

Crash resistant

Hot backups

Compaction




                     44
Replication




              45
CouchDB Replication



                      POST /_replicate




                      POST /_replicate


Eventually cons...
Load Balancing




                             Replication


 HTTP Client     HTTP Load
                 Balancer




   ...
Caching




 HTTP Client   HTTP Cache
               Varnish
               Apache
               …




                  ...
Multi-Master




               49
Sharding/Partitioning with
CouchDB Lounge




 HTTP Client    CouchDB
                Lounge




                         ...
Sharding with CouchDB Lounge




 HTTP Client   CouchDB
               Lounge




                               51
Various

CouchApps

Validations

Filtered replication

Changes feed

Futon

Geo

Fulltext-Search with embedded Lucene

Exp...
Q&A
Peritor GmbH
Blücherstr. 22, Hof III Aufgang 6
10961 Berlin
Tel.: +49 (0)30 69 20 09 84 0
Fax: +49 (0)30 69 20 09 84 9...
Upcoming SlideShare
Loading in...5
×

CouchDB on Rails - FrozenRails 2010

7,218

Published on

0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
7,218
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
102
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

CouchDB on Rails - FrozenRails 2010

  1. 1. CouchDB on Rails An Introduction FrozenRails Jonathan Weiss 07.05.2010
  2. 2. Who am I? Working for Peritor in Berlin, Germany Written, maintain, or involved in   Webistrano   Capistrano   SimplyStored   Happening   The great fire of London http://github.com/jweiss @jweiss 2
  3. 3. Scalarium EC2 Cluster Management   Auto-config   Self-Healing   Auto-Scaling   One-click-deployment www.scalarium.com 3
  4. 4. Database Requirements High Availability Easy Replication Clustering Robustness Short Recovery Time 4
  5. 5. 5
  6. 6. CouchDB Build for the Web Scales Replication built-in Embracing offline Flexible schema – document DB 6
  7. 7. ”CouchDB is built of the Web“ Jacob Kaplan-Moss 7
  8. 8. Web Technologies HTTP the access protocol JavaScript the query language JSON the storage format 8
  9. 9. JSON Document { "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } } 9
  10. 10. JSON Document { "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } } 10
  11. 11. JSON Document { "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } } 11
  12. 12. No Tables or Namespaces 12
  13. 13. No Tables or Namespaces 13
  14. 14. Manual Namespacing { "_id": "BCCD12CBB", "_rev": "1-AB764C", "type": "person", "name": "Darth Vader", "age": 63, "headware": ["Helmet", "Sombrero"], "dark_side": true, "weapons": { "right_arm": "light_saber", "left_arm": null } } 14
  15. 15. Interacting with CouchDB JSON HTTP Client PUT /dbname/ID 15
  16. 16. Interacting with CouchDB GET /dbname/ID HTTP Client JSON 16
  17. 17. Interacting with CouchDB DELETE /dbname/ID HTTP Client 17
  18. 18. Views 18
  19. 19. Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } } 19
  20. 20. Design Document { Document ID "id": "_design/hats”, – "_rev": "431212AB4”, prefixed by “_design/” "language": "javascript”, "views": { "all": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } } 20
  21. 21. Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { Hash of Views "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } } 21
  22. 22. Design Document { "id": "_design/hats”, "_rev": "431212AB4”, "language": "javascript”, "views": { "all": { Hash of Views "map": "function(doc){ .... }”, Every view has map & "reduce": "function(doc){ .... }” reduce function }, "by_manufacturer": { "map": "function(doc){ .... }”, "reduce": "function(doc){ .... }” } } } 22
  23. 23. Map function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } } 23
  24. 24. Map Passed every document in the DB function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } } 24
  25. 25. Map function(doc) { if (doc.headware) { Inspects for (var hat in doc.headware) { & Decides emit(hat, 1); } } } 25
  26. 26. Map function(doc) { if (doc.headware) { for (var hat in doc.headware) { emit(hat, 1); } } Emits result } for index 26
  27. 27. Reduce function(keys, values, rereduce) { return sum(values); } 27
  28. 28. Reduce Passed map result (or partial reduce result) function(keys, values, rereduce) { return sum(values); } 28
  29. 29. Reduce function(keys, values, rereduce) { return sum(values); } Aggregates (count, sum, average, …) 29
  30. 30. Example Map Result Map functions are similar to SQL indices ID KEY VALUE 51ABFA211 Cap 1 ABC123456 Cappy 1 BCCD12CBB Helmet 1 BCCD12CBB Sombrero 1 Sorted by the key Key can also be an array Value can be complex JSON 30
  31. 31. Query a view GET /dbname/_design/hats/_view/all HTTP Client {"total_rows":348,"offset":0,"rows”:[ {"id":"A","key":"A","value":1}, {"id":"B","key":"B","value":1}, ]} 31
  32. 32. Query a view GET /dbname/_design/hats/_view/all? include_docs=true HTTP Client 32
  33. 33. View Query Filter by   key=ABC123   startkey=123 & endkey=9   limit=100   descending=true   group=true   reduce=true   Include_docs=true 33
  34. 34. SQL vs. JavaScript Vs. 34
  35. 35. SQL vs. JavaScript ActiveRecord Vs. SimplyStored 35
  36. 36. SimplyStored Convenience Layer for CouchDB   Models & Associations   Validations   Callbacks BSD-licensed on   Dynamic finder http://github.com/peritor/simply_stored   S3 attachments On top of CouchPotato, CouchRest & RestClient   Paranoid delete   ActiveModel compliant 36
  37. 37. Setup Install Load in environment.rb Configure 37
  38. 38. Setup 38
  39. 39. 39
  40. 40. 40
  41. 41. RockingChair In-memory CouchDB   Just a big Hash   Understands all SimplyStored generated views   Speeds up tests   Tests can run in parallel   Nice for debugging BSD-licensed on http://github.com/jweiss/rocking_chair 41
  42. 42. Database Requirements High Availability Easy Replication Clustering Robustness Short Recovery Time 42
  43. 43. Replication B-Tree XXXXX Photo by Mathias Meyer 43
  44. 44. B-Tree Append only Concurrency (MVCC) Crash resistant Hot backups Compaction 44
  45. 45. Replication 45
  46. 46. CouchDB Replication POST /_replicate POST /_replicate Eventually consistent & conflict resolution 46
  47. 47. Load Balancing Replication HTTP Client HTTP Load Balancer 47
  48. 48. Caching HTTP Client HTTP Cache Varnish Apache … 48
  49. 49. Multi-Master 49
  50. 50. Sharding/Partitioning with CouchDB Lounge HTTP Client CouchDB Lounge 50
  51. 51. Sharding with CouchDB Lounge HTTP Client CouchDB Lounge 51
  52. 52. Various CouchApps Validations Filtered replication Changes feed Futon Geo Fulltext-Search with embedded Lucene Experimental Ruby-View-Server 52
  53. 53. Q&A Peritor GmbH Blücherstr. 22, Hof III Aufgang 6 10961 Berlin Tel.: +49 (0)30 69 20 09 84 0 Fax: +49 (0)30 69 20 09 84 9 Internet: www.peritor.com E-Mail: info@peritor.com © Peritor GmbH - Alle Rechte vorbehalten
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×