relax...
Topics

• Background
• Replication
• Views
• Implementation
• Questions
Background
RDBMS
RDBMS
Document Based
Database Management
       System
DB DBMS
Nutshell

• Schema free
• JSON API
• Optimized for speed
• HTTP server
• Robust
• Distributed
Landscape
Distributed vs non distributed

       Distributed       Non distributed

•   Amazon Dynamo
•   Amazon S3
•   Scalaris
•   Voldemort
                     •   Redis
•   CouchDb
                     •   Tokyo Tyrant
•   Riak
                     •   MemcacheDb
•   MongoDb
                     •   Amazon SimpleDb
•   BigTable
•   Cassandra
•   HyperTable
•   HBase
Datamodel store
Key-Value store     Document Store            Column Store


•   Amazon Dynamo
                    •   Amazon SimpleDb   •   Cassandra
•   Amazon S3
                    •   Couchdb           •   Google BigTable
•   Redis
                    •   MongoDb           •   HBase
•   Scalaris
                    •   Riak              •   Hyperbase
•   Voldemort
Disk vs Memory
    Memory       Configurable           Disk



             •   BigTable      •   CouchDb
• Scalaris   •   Cassandra     •   MongoDb
• Redis      •   Hbase         •   Riak
             •   HyperTable    •   Voldemort
mysqlrepair
Replication
IDG US    IDG AUSTRALIA




 IDG NL        IDG DE
IDG DE       IDG AUSTRALIA   IDG Poland     IDG Hungary




IDG NL       IDG US          IDG Romania    IDG Sweden




IDG Brazil   IDG UK          IDG Spania     IDG Portugal




IDG Ghana    IDG Japan       IDG Thailand   IDG Vietnam
Implication
Every single bit of IDG content published
worldwide, is available anywhere, anytime.
Collisions
128-bit uuid
128-bit >
0000d65ed67a702aaacf53d59d9d386e
Views
JSON
{

   "_id":"c1447cd2eefb39dc38b1b5c66c294b50",


   "_rev":"2319215746",

   "id":"6993",


   "date":{

   
    "epoch":1245841200,

   
    "date":"2009/06/24 07:00:00 +0100"


   },

   "title":"Help! Ik moet met vakantie!",


   "product":"nl.techworld",

   "language":"nl",

   "permalink":"http://techworld.nl/article/6993/index.html",

}
JSON
{


   "_id":"773fd35de33ec4272c4e864f4c39ed71",

   "_rev":"3410695860",


   "id":"7001",

   "date":"2009/06/24 07:00:00 +0100"

   "title":"Help! Ik moet met vakantie!",


   "product":"nl.techworld",

   "language":"nl",

   "permalink":"http://techworld.nl/article/6993/index.html",


   "tags":[

   
      "zzp",


   
      "vakantie",

   ],

   "author":{


   
      "email":"mblommestein@idg.nl",

   
      "name":"Michiel van Blommestein"

   }

}
MapReduce
~GROUP BY
Javascript
View method
function(doc) {
    var keywords = ['apple', 'iphone', ..., 'mac', 'ipod', 'steve jobs','islate'];
    if (doc.tags)
    {
        for (tag in doc.tags)
        {
            for (keyword in keywords)
            {
                if (doc.tags[tag].toLowerCase().match(keywords[keyword]) != null)
                {
                    emit([keywords[keyword], doc.date.epoch], doc._id)
                }
            }
        }
    }
}
http://couchdb.idg.nl/foo/_design/all

{
    "_id": "_design/all",
    "_rev": "1-5fe9f441bf902e10307d13a1991d9d9c",
    "language": "javascript",
    "views": {
        "tag": {
            "map": "function(doc) {nif (doc.tags)..., doc._id)n}n}n}n"
        }
    }
}
http://couchdb.idg.nl/foo/_design/all/_view/all?limit=20

{"total_rows":1459,"offset":0,"rows":[
{"id":"a7eccbf5619b96cad116633a5b96ab76","key":875782800,"value":"a7eccbf5619b96cad116633a5b96ab76"},
{"id":"5b141eeef9346451f0a1a92d5044cc7a","key":884174640,"value":"5b141eeef9346451f0a1a92d5044cc7a"},
{"id":"831491235bb98c4bf4c7ab64b6fbecba","key":889615980,"value":"831491235bb98c4bf4c7ab64b6fbecba"},
{"id":"361e243e0687f8cb7af42e4b28072e0d","key":890908320,"value":"361e243e0687f8cb7af42e4b28072e0d"},
{"id":"0476368ea53989af3849320ebdf80c3c","key":899979840,"value":"0476368ea53989af3849320ebdf80c3c"},
{"id":"b41391d910225a46ce8022f68a7c4ef2","key":905944560,"value":"b41391d910225a46ce8022f68a7c4ef2"},
{"id":"071ba1886596a96687437423bb625182","key":907946340,"value":"071ba1886596a96687437423bb625182"},
{"id":"797c4b16427d4eb7a1a2aea18a4e67b3","key":908529660,"value":"797c4b16427d4eb7a1a2aea18a4e67b3"},
{"id":"a0bdfe900b886cbdafda05e7b68c2712","key":910091940,"value":"a0bdfe900b886cbdafda05e7b68c2712"},
{"id":"eb6762eac0febf836172b1edebf8f867","key":912778380,"value":"eb6762eac0febf836172b1edebf8f867"},
{"id":"7ee32de77674e77da68cc8eaa33d4461","key":921669480,"value":"7ee32de77674e77da68cc8eaa33d4461"},
{"id":"a7e8db1dfd85d89d30d81c7880a25fee","key":933940080,"value":"a7e8db1dfd85d89d30d81c7880a25fee"},
{"id":"81db2e2769a56c1758b340a9f96b7845","key":936198120,"value":"81db2e2769a56c1758b340a9f96b7845"},
{"id":"cc8464377aca67f4deb5aa7d33974291","key":938439240,"value":"cc8464377aca67f4deb5aa7d33974291"},
{"id":"48ea6978aacd2f228bb66a68c45500bd","key":940252320,"value":"48ea6978aacd2f228bb66a68c45500bd"},
{"id":"223ddc8563ae501fc81bfeab87eb2b0c","key":941183640,"value":"223ddc8563ae501fc81bfeab87eb2b0c"},
{"id":"ae65829fe1ed418da984c13de59762bd","key":944036700,"value":"ae65829fe1ed418da984c13de59762bd"},
{"id":"b8b5251b7573ae07e0b2f27d9e0a9eb1","key":945414000,"value":"b8b5251b7573ae07e0b2f27d9e0a9eb1"},
{"id":"ed80064e6689f38a8597b2e6f8ac11e3","key":946882800,"value":"ed80064e6689f38a8597b2e6f8ac11e3"},
{"id":"7f4217e4a659a3e37f419a5c13b75df9","key":948365040,"value":"7f4217e4a659a3e37f419a5c13b75df9"}
]}
http://couchdb.idg.nl/foo/_design/all/_view/all?limit=20

{"total_rows":1459,"offset":0,"rows":[
{"id":"a7eccbf5619b96cad116633a5b96ab76","key":875782800,"value":"a7eccbf5619b96cad116633a5b96ab76"},
{"id":"5b141eeef9346451f0a1a92d5044cc7a","key":884174640,"value":"5b141eeef9346451f0a1a92d5044cc7a"},
{"id":"831491235bb98c4bf4c7ab64b6fbecba","key":889615980,"value":"831491235bb98c4bf4c7ab64b6fbecba"},
{"id":"361e243e0687f8cb7af42e4b28072e0d","key":890908320,"value":"361e243e0687f8cb7af42e4b28072e0d"},
{"id":"0476368ea53989af3849320ebdf80c3c","key":899979840,"value":"0476368ea53989af3849320ebdf80c3c"},
{"id":"b41391d910225a46ce8022f68a7c4ef2","key":905944560,"value":"b41391d910225a46ce8022f68a7c4ef2"},
{"id":"071ba1886596a96687437423bb625182","key":907946340,"value":"071ba1886596a96687437423bb625182"},
{"id":"797c4b16427d4eb7a1a2aea18a4e67b3","key":908529660,"value":"797c4b16427d4eb7a1a2aea18a4e67b3"},
{"id":"a0bdfe900b886cbdafda05e7b68c2712","key":910091940,"value":"a0bdfe900b886cbdafda05e7b68c2712"},
{"id":"eb6762eac0febf836172b1edebf8f867","key":912778380,"value":"eb6762eac0febf836172b1edebf8f867"},
{"id":"7ee32de77674e77da68cc8eaa33d4461","key":921669480,"value":"7ee32de77674e77da68cc8eaa33d4461"},
{"id":"a7e8db1dfd85d89d30d81c7880a25fee","key":933940080,"value":"a7e8db1dfd85d89d30d81c7880a25fee"},
{"id":"81db2e2769a56c1758b340a9f96b7845","key":936198120,"value":"81db2e2769a56c1758b340a9f96b7845"},
{"id":"cc8464377aca67f4deb5aa7d33974291","key":938439240,"value":"cc8464377aca67f4deb5aa7d33974291"},
{"id":"48ea6978aacd2f228bb66a68c45500bd","key":940252320,"value":"48ea6978aacd2f228bb66a68c45500bd"},
{"id":"223ddc8563ae501fc81bfeab87eb2b0c","key":941183640,"value":"223ddc8563ae501fc81bfeab87eb2b0c"},
{"id":"ae65829fe1ed418da984c13de59762bd","key":944036700,"value":"ae65829fe1ed418da984c13de59762bd"},
{"id":"b8b5251b7573ae07e0b2f27d9e0a9eb1","key":945414000,"value":"b8b5251b7573ae07e0b2f27d9e0a9eb1"},
{"id":"ed80064e6689f38a8597b2e6f8ac11e3","key":946882800,"value":"ed80064e6689f38a8597b2e6f8ac11e3"},
{"id":"7f4217e4a659a3e37f419a5c13b75df9","key":948365040,"value":"7f4217e4a659a3e37f419a5c13b75df9"}
]}
http://couchdb.idg.nl/foo/a7eccbf5619b96cad116633a5b96ab76

{
    "_id": "a7eccbf5619b96cad116633a5b96ab76",
    "_rev": "1-267169778",
    "title": "Microsoft laat oog vallen op Mac-markt",
    "intro": "Uitgeverijen en drukkerijen moeten aan de Windows NT. Die boodschap heeft Microsoft-
topman Bill Gates geopenbaard in Los Angeles voor een gehoor van uitgevers op de Seybold-
conferentie, u00e9u00e9n van de grootste vakbeurzen voor de grafische industrie.",
    "permalink": "http://webwereld.nl//nieuws/25413/microsoft-laat-oog-vallen-op-mac-markt.html",
    "body": "Volgens Gates is NT uitermate geschikt om te dienen als server in een omgeving waar
met verschillende besturingssystemen wordt gewerkt...geleden heeft Bill Gates 150 miljoen dollar
gestoken in het noodlijdende Apple, dat het grafische vak, naast het onderwijs, als zijn
belangrijkste markt beschouwd. ",
    "date": {
        "epoch": 875782800,
        "publish_date": "1997-10-02T11:00:00+02:00",
        "created_on": "1997-10-02T11:00:30+02:00",
        "updated_on": "2009-08-04T15:26:16+02:00"
    },
    "id": "25413",
    "language": "nl",
    "product": "nl.webwereld",
    "categories": ["E-commerce"],
    "tags": ["microsoft", "apple", "mac", "markt"],
    "images": {
        "thumbnail": "http://static.webwereld.nl/uploads/images/gates3.jpg"
    }
}
REST
HTTP
Tim Berners Lee
HTTP


$ telnet webwereld.nl 80
GET / HTTP1.0
REST

• Create: HTTP PUT      /db/
• Read: HTTP GET        /db/docid
• Update: HTTP POST     /db/docid
• Delete: HTTP DELETE   /db/docid
Implementation
Installation

• Ubuntu/Debian
 apt-get install couchdb
• OSX
 Download CouchDBX
• Windows
 Oops!
Libraries
• PHP: PHPPillow, PHP Object_Freezer, PHP
  On Couchdb, etc.
• Ruby: CouchREST, CouchFoot,
  CouchObject, Active Couch, etc.
• Java: JRelax, jcouchdb, couchdb4j, etc.
• .net: SharpCouch
• etc.
Classic model
     Client



     HTTP



    MySQL
Possible model (1)
           Client



    HTTP      CouchDB



   MySQL
Possible model (2)
           Client


              Proxy
    HTTP              CouchDB



   MySQL
Proxy
<Location /api>
    <LimitExcept GET>
        require valid-user
    </LimitExcept>
    RewriteEngine On
    RewriteRule api/search/(.*) http://couchdb.idg.nl/nl_zoom_gallery/_fti/$1 [P,L]
    RewriteRule api/image/(.*) http://couchdb.idg.nl/nl_zoom_gallery/$1 [P,L]
    RewriteRule api/(.*)        http://couchdb.idg.nl/nl_zoom_gallery/_design/images/$1 [P,L]
</Location>
IDG NL

External clients            IDG Publications
     (eg: nu.nl)              (eg: webwereld.nl)


       (ro)                       (ro)
           authentication

           IDG API                                 IDG atk
             (api.idg.nl)                            (admin.*.nl)


            (ro)                                   (rw)

                                                                    IDG (intl)
                                                      (rw)
Want more?

• NoSQL movement
• Distributed web apps
• couchdb-lucene
• O’Reilly: CouchDB: The definitive guide
• NoSQL landscape
Questions?
relax

CouchDB introduction

Editor's Notes

  • #20 - Stored in Binary trees - Append only file structure
  • #21 Append Only, always consistent, no need for mysqlrepair stuff
  • #26 Master &lt;-&gt; Master replication is usually difficult, not with couchdb
  • #46 Couchdb only syncs latest versions
  • #70 Only GET allowed (other will give 500) fti == Full Text Index (Lucene)