CouchDB is a document-oriented database that uses JSON documents, has a RESTful HTTP API, and is queried using map/reduce views. Each of these properties alone, especially MapReduce views, may seem foreign to developers more familiar with relational databases. This tutorial will teach web developers the concepts they need to get started using CouchDB in their projects. CouchDB’s RESTful HTTP API makes it suitable for interfacing with any programming language. CouchDB libraries are available for many programming languages and we will take a look at some of the more popular ones.
19. Bene ts of JSON
Human-readable and simple data interchange format
Data structures from many programming languages can be easily converted
to and from JSON
Lightweight—doesn’t consume much bandwidth
Easily integrated into JavaScript applications (e.g. CouchApps)
20. JSON Objects
A JSON Object is a collection of name/value pairs.
The name is a String.
The value can be a:
• Boolean (false or true)
• JSON Array (e.g. ["a", "b", "c"])
• JSON Object
• Number
• String
• JSON NULL (null)
21. Example JSON Object
{
"_id":"978-0-596-15589-6",
"title":"CouchDB: The De nitive Guide",
"subtitle":"Time to Relax",
"authors":[
"J. Chris Anderson",
"Jan Lehnardt",
"Noah Slater"
],
"publisher":"O'Reilly Media",
"released":"2010-01-19",
"pages":272
}
22. Indexed Queries
MapReduce “views” generate B+ tree indexes
Indexes are incrementally updated by Map functions
Views are queried for result sets:
{ // …
"rows": [
{
"id": "fe8a4a1ca998daafaaa04d7d9000187f",
"key":"Learning CouchDB",
"value": null
}
]
}
23. Multi-Version
Concurrency Control (MVCC)
Each document stores its current revision number and hash:
{
"_id": "fe8a4a1ca998daafaaa04d7d9000187f",
"_rev": "1-bd6ed00168af92895217953f69504bb4",
"title":"Learning CouchDB"
}
Optimistic concurrency ensures that a document has not been modi ed since
it was last retrieved
Old revisions are saved for replication purposes, but are eventually pruned
24. Replication
Peer-based, bi-directional replication of databases
In multi-master scenarios, replicate database A → B
and database B → A
Replication can happen continuously or after having being disconnected for long
periods of time (or never connected)
Document update con icts are agged and can then be processed by your
application or by the end-user
26. HTTP API
No custom binary protocol (well, there is Hovercraft)
Everyone speaks HTTP—trivially easy to interface with CouchDB
HTTP is:
• distributed
• scalable
• cacheable
27. Under The Hood
Atomic, Consistent, Isolated, and Durable (ACID)
“Crash-only” design—no shutdown process
Append-only B-tree les
Runs on the Erlang Open Telecom Platform (OTP)
Erlang is a highly-concurrent functional programming language
Erlang OTP is well suited for distributed systems
29. When You Might
Consider CouchDB
You’ve found yourself denormalizing your SQL database for better performance
Your domain model is a “ t” for documents (e.g. a CMS)
Your application is read-heavy
You need a high level of concurrency and can give up consistency in exchange
You need horizontal scalability
You want your database to be able to run anywhere, even on mobile devices,
and even when disconnected from the cluster
30. Trade-Offs
No ad hoc queries. You need to know what you’re going to want to query
ahead of time
No concept of “joins”. You can relate data, but watch out for consistency issues
Transactions are limited to document boundaries
CouchDB trades storage space for performance
Reads scale well, writes scale well with third-party software (open source)
31. Other Alternatives to SQL
MongoDB
http://www.mongodb.org/
Redis
http://redis.io/
Cassandra
http://cassandra.apache.org/
Riak
http://www.basho.com/
HBase (a database for Hadoop)
http://hbase.apache.org/
32. Don’t be so quick to get rid of SQL! There are
many problems for which an SQL database is a
good t. SQL is a very powerful and exible
query language.
38. Ruby
CouchRest (“close to the metal”)
https://github.com/couchrest/couchrest
CouchRest Model (“close to shiny metal with rounded edges”)
https://github.com/couchrest/couchrest_model
Couch Potato
https://github.com/langalex/couch_potato
CouchFoo (ActiveRecord API)
https://github.com/georgepalmer/couch_foo
CouchObject
http://couchobject.rubyforge.org/
45. REST and Relaxation
Messages are self-described via HTTP headers and HTTP status codes
URIs identify resources
HTTP methods de ne operations on resources
Hypermedia controls are possible through show and list functions
(if you don’t consider JSON to be hypermedia)
62. Create a Document
$ curl -iX POST 'http://localhost:5984/mydb'
-H "Content-Type: application/json"
-d '{"_id":"mydoc"}'
HTTP/1.1 201 Created
Location: http://localhost:5984/mydb/mydoc
{
"ok":true,
"id":"mydoc",
"rev":"1-967a00dff5e02add41819138abb3284d"
}
63. Read a Document
$ curl -iX GET 'http://localhost:5984/mydb/mydoc'
HTTP/1.1 200 OK
Etag: "1-967a00dff5e02add41819138abb3284d"
{
"_id":"mydoc",
"_rev":"1-967a00dff5e02add41819138abb3284d"
}
64. Update a Document
$ curl -iX PUT 'http://localhost:5984/mydb/mydoc'
-H "Content-Type: application/json"
-d '{
"_id":"mydoc",
"_rev":"1-967a00dff5e02add41819138abb3284d",
"title":"CouchDB: The De nitive Guide"
}'
HTTP/1.1 201 Created
{
"ok":true,
"id":"mydoc",
"rev":"2-bbd27429fd1a0daa2b946cbacb22dc3e"
}
65. CouchDB allows for conditional requests,
saving bandwidth and processing.
66. Conditional GET
$ curl -iX GET 'http://localhost:5984/mydb/mydoc'
-H 'If-None-Match: "2-bbd27429fd1a0daa2b946cbacb22dc3e"'
HTTP/1.1 304 Not Modi ed
Etag: "2-bbd27429fd1a0daa2b946cbacb22dc3e"
Content-Length: 0
67. Delete a Document
$ curl -iX DELETE 'http://localhost:5984/mydb/mydoc'
-H 'If-Match: "2-bbd27429fd1a0daa2b946cbacb22dc3e"'
HTTP/1.1 200 OK
{
"ok":true,
"id":"mydoc",
"rev":"3-29d2ef6e0d3558a3547a92dac51f3231"
}
68. Read a Deleted Document
$ curl -iX GET 'http://localhost:5984/mydb/mydoc'
HTTP/1.1 404 Object Not Found
{
"error":"not_found",
"reason":"deleted"
}
69. Read a Deleted Document
$ curl -iX GET 'http://localhost:5984/mydb/mydoc
?rev=3-29d2ef6e0d3558a3547a92dac51f3231'
HTTP/1.1 200 OK
Etag: "3-29d2ef6e0d3558a3547a92dac51f3231"
{
"_id":"mydoc",
"_rev":"3-29d2ef6e0d3558a3547a92dac51f3231",
"_deleted":true
}
71. Do not rely on older revisions! Revisions
are used for concurrency control and for
replication. Old revisions are removed
during database compaction.
74. When POSTing a document, CouchDB will
assign a UUID as the document’s ID if one
is not speci ed.
75. POST My Business Card
$ curl -iX POST 'http://localhost:5984/cards'
-H "Content-Type: application/json"
-d @business_card.json
HTTP/1.1 201 Created
Location: http://localhost:5984/cards/fe8a4a1ca998daafaaa04d7d90000574
{
"ok":true,
"id":"fe8a4a1ca998daafaaa04d7d90000574",
"rev":"1-2526abc0e10aa39167ff7b16a77b4dad"
}
83. Create Your Business Card
(business_card.json)
{
"fn":"Your Name Here",
"url":"http://yoururl.com/",
"role":"Your Role",
"org":"Your Organization",
"email":"youremail@yoururl.com",
"adr":{
"street-address":"Your Street",
"locality":"Your City",
"region":"Your State",
"postal-code":"Your Postal Code",
"country-name":"Your Country"
},
"tel":"Your Phone Number"
}
84. POST Your Business Card
$ curl -iX POST 'http://localhost:5984/cards'
-H "Content-Type: application/json"
-d @business_card.json
HTTP/1.1 201 Created
94. MapReduce consists of Map and Reduce steps
which can be distributed in a way that takes
advantage of the multiple processor cores
found in modern hardware.
99. The First Document
(978-0-596-15589-6.json)
{
"_id":"978-0-596-15589-6",
"title":"CouchDB: The De nitive Guide",
"subtitle":"Time to Relax",
"authors":[
"J. Chris Anderson",
"Jan Lehnardt",
"Noah Slater"
],
"publisher":"O'Reilly Media",
"formats":[
"Print",
"Ebook",
"Safari Books Online"
],
"released":"2010-01-19",
"pages":272
}
100. POST the First Document
$ curl -iX POST 'http://localhost:5984/books'
-H "Content-Type: application/json"
-d @978-0-596-15589-6.json
HTTP/1.1 201 Created
Location: http://localhost:5984/books/978-0-596-15589-6
{
"ok":true,
"id":"978-0-596-15589-6",
"rev":"1-8c1d735475a0401d984c4aff2fe796a5"
}
101. The Second Document
(978-0-596-52926-0.json)
{
"_id":"978-0-596-52926-0",
"title":"RESTful Web Services",
"subtitle":"Web services for the real world",
"authors":[
"Leonard Richardson",
"Sam Ruby"
],
"publisher":"O'Reilly Media",
"formats":[
"Print",
"Ebook",
"Safari Books Online"
],
"released":"2007-05-08",
"pages":448
}
102. POST the Second Document
$ curl -iX POST 'http://localhost:5984/books'
-H "Content-Type: application/json"
-d @978-0-596-52926-0.json
HTTP/1.1 201 Created
Location: http://localhost:5984/books/978-0-596-52926-0
{
"ok":true,
"id":"978-0-596-52926-0",
"rev":"1-12538b07773519133043157220a63d8b"
}
103. The Third Document
(978-1-565-92580-9.json)
{
"_id":"978-1-565-92580-9",
"title":"DocBook: The De nitive Guide",
"authors":[
"Norman Walsh",
"Leonard Muellner"
],
"publisher":"O'Reilly Media",
"formats":[
"Print"
],
"released":"1999-10-28",
"pages":648
}
104. POST the Third Document
$ curl -iX POST 'http://localhost:5984/books'
-H "Content-Type: application/json"
-d @978-1-565-92580-9.json
HTTP/1.1 201 Created
Location: http://localhost:5984/books/978-1-565-92580-9
{
"ok":true,
"id":"978-1-565-92580-9",
"rev":"1-b945cb4799a1ccdd1689eae0e44124f1"
}
105. The Fourth Document
(978-0-596-80579-1.json)
{
"_id":"978-0-596-80579-1",
"title":"Building iPhone Apps with HTML, CSS, and JavaScript",
"subtitle":"Making App Store Apps Without Objective-C or Cocoa",
"authors":[
"Jonathan Stark"
],
"publisher":"O'Reilly Media",
"formats":[
"Print",
"Ebook",
"Safari Books Online"
],
"released":"2010-01-08",
"pages":192
}
106. POST the Fourth Document
$ curl -iX POST 'http://localhost:5984/books'
-H "Content-Type: application/json"
-d @978-0-596-80579-1.json
HTTP/1.1 201 Created
Location: http://localhost:5984/books/978-0-596-80579-1
{
"ok":true,
"id":"978-0-596-80579-1",
"rev":"1-09ce09fef75068834da99957c7b14cf2"
}
107. …or Just Replicate
$ curl -iX POST 'http://localhost:5984/_replicate'
-H "Content-Type: application/json"
-d '{
"source": "https://oscon-tutorial.iriscouch.com/books",
"target": "books"
}'
HTTP/1.1 200 OK
{
"ok":true,
//…
}
109. Map and Reduce are written as JavaScript
functions that are de ned within views.
110. emit(key, value)
Accepts two arguments—both are optional and default to null
First argument is the key to be indexed
Second argument is the value to be associated with the key
Arbitrary JSON values are allowed for both the key and value
The ID of the document being mapped is implicitly included with each
call to the emit function
112. Map Book Titles
function(doc) { // JSON object representing a doc to be mapped
if (doc.title) { // make sure this doc has a title
emit(doc.title); // emit the doc’s title as the key
}
}
113. Temporary views can be used during
development but should be saved
permanently to design documents
for production.
115. POST Titles Temporary View
$ curl -iX POST 'http://localhost:5984/books/_temp_view'
-H "Content-Type: application/json"
-d @titles_view.json
HTTP/1.1 200 OK
{
"total_rows": 4,
"offset": 0,
"rows": [
// …
]
}
116. “Titles” View Rows
key id value
"Building iPhone Apps with
"978-0-596-80579-1" null
HTML, CSS, and JavaScript"
"CouchDB: The Definitive
"978-0-596-15589-6" null
Guide"
"DocBook: The Definitive
"978-1-565-92580-9" null
Guide"
"RESTful Web Services" "978-0-596-52926-0" null
119. Map Book Formats
function(doc) { // JSON object representing a doc to be mapped
if (doc.formats) { // make sure this doc has a formats eld
for (var i in doc.formats) {
emit(doc.formats[i]); // emit each format as the key
}
}
}
120. Formats View
(formats_view.json)
{
"map": "function(doc) { if (doc.formats) { for (var i in doc.formats)
{ emit(doc.formats[i]); } } }"
}
121. POST Formats
Temporary View
$ curl -iX POST 'http://localhost:5984/books/_temp_view'
-H "Content-Type: application/json"
-d @formats_view.json
HTTP/1.1 200 OK
{
"total_rows": 10,
"offset": 0,
"rows": [
// …
]
}
124. Map Book Authors
function(doc) { // JSON object representing a doc to be mapped
if (doc.authors) { // make sure this doc has an authors eld
for (var i in doc.authors) {
emit(doc.authors[i]); // emit each author as the key
}
}
}
125. Authors View
(authors_view.json)
{
"map": "function(doc) { if (doc.authors) { for (var i in doc.authors)
{ emit(doc.authors[i]); } } }"
}
126. POST Authors
Temporary View
$ curl -iX POST 'http://localhost:5984/books/_temp_view'
-H "Content-Type: application/json"
-d @authors_view.json
HTTP/1.1 200 OK
{
"total_rows": 8,
"offset": 0,
"rows": [
// …
]
}
131. Built-in Reduce Functions
Function Output
_count Returns the number of mapped values in the set
_sum Returns the sum of the set of mapped values
Returns numerical statistics of the mapped values in the set
_stats
including the sum, count, min, and max
141. The _count function can count arbitrary values,
including NULLs. However, the _sum and _stats
functions require numbers as values.
142. Updated Book Formats Map
function(doc) {
if (doc.formats) {
for (var i in doc.formats) {
emit(doc.formats[i], doc.pages); // now emit pages as value
}
}
}
143. Updated Formats View
(formats_view.json)
{
"map": "function(doc) { if (doc.formats) { for (var i in doc.formats)
{ emit(doc.formats[i], doc.pages); } } }",
"reduce": "_sum"
}
144. POST Updated Formats
Temporary View, Grouped
$ curl -iX POST 'http://localhost:5984/books/_temp_view?group=true'
-H "Content-Type: application/json"
-d @formats_view.json
HTTP/1.1 200 OK
{
"rows": [
// …
]
}
145. Sum of Pages by Format
key value
"Ebook" 912
"Print" 1560
"Safari Books Online" 912
148. Updated Formats View
(formats_view.json)
{
"map": "function(doc) { if (doc.formats) { for (var i in doc.formats)
{ emit(doc.formats[i], doc.pages); } } }",
"reduce": "_stats"
}
149. POST Updated Formats
Temporary View, Grouped
$ curl -iX POST 'http://localhost:5984/books/_temp_view?group=true'
-H "Content-Type: application/json"
-d @formats_view.json
HTTP/1.1 200 OK
{
"rows": [
// …
]
}
150. Stats of Pages by Format
key value
{"sum":912,"count":3,"min":192,
"Ebook"
"max":448,"sumsqr":311552}
{"sum":1560,"count":4,"min":192,
"Print"
"max":648,"sumsqr":731456}
{"sum":912,"count":3,"min":192,
"Safari Books Online"
"max":448,"sumsqr":311552}
154. Parameters
Keys: An array of mapped key and document IDs in the form of [key,id] where id is
the document ID.
Values: An array of mapped values.
Rereduce: Whether or not the Reduce function is being called recursively on its
own output.
183. The skip and limit parameters can be used
to implement pagination. When skipping a
larger number of rows, it is more efficient to
set the skip parameter’s value to 1 and use
the previous page’s last key and document
ID as the startkey and startkey_docid.
184. Reversing Output
$ curl -iGX GET 'http://localhost:5984/books/_design/default/_view/titles'
-d reduce=false
-d descending=true
HTTP/1.1 200 OK
{"total_rows":4,"offset":0,"rows":[
{"id":"978-0-596-52926-0","key":"RESTful Web Services","value":448},
{"id":"978-1-565-92580-9","key":"DocBook: The De nitive Guide","value":648},
{"id":"978-0-596-15589-6","key":"CouchDB: The De nitive Guide","value":272},
{"id":"978-0-596-80579-1","key":"Building iPhone Apps with HTML, CSS, and
JavaScript","value":192}
]}
185. When reversing output, the values for startkey
and endkey must be swapped as well as the
values for startkey_docid and endkey_docid, if
speci ed. This is because output is reversed
before rows are ltered.
190. Get the View
$ curl -iGX GET 'http://localhost:5984/books/_design/default/_view/titles'
-d reduce=false
-d limit=1
HTTP/1.1 200 OK
{"total_rows":4,"offset":0,"rows":[
{"id":"978-0-596-80579-1","key":"Building iPhone Apps with HTML, CSS, and
JavaScript","value":192}
]}
191. Get a Document
From the View
$ curl -iX GET 'http://localhost:5984/books/978-0-596-80579-1'
HTTP/1.1 200 OK
{
"_id": "978-0-596-80579-1",
"_rev": "1-09ce09fef75068834da99957c7b14cf2",
"title": "Building iPhone Apps with HTML, CSS, and JavaScript",
"subtitle": "Making App Store Apps Without Objective-C or Cocoa",
"authors": ["Jonathan Stark"],
"publisher": "O'Reilly Media",
"formats": ["Print", "Ebook", "Safari Books Online"],
"released":" 2010-01-08",
"pages":192
}
192. Get a Cached Document From the
View
$ curl -iX GET 'http://localhost:5984/books/978-0-596-80579-1'
-H 'If-None-Match: "1-09ce09fef75068834da99957c7b14cf2"'
HTTP/1.1 304 Not Modi ed
Etag: "1-09ce09fef75068834da99957c7b14cf2"
Content-Length: 0
200. GET Con g Section
$ curl -iX GET 'http://localhost:5984/_con g/uuids'
HTTP/1.1 200 OK
{
"algorithm": "sequential"
}
201. GET Con g Section Key
$ curl -iX GET 'http://localhost:5984/_con g/uuids/algorithm'
HTTP/1.1 200 OK
"sequential"
202. PUT Con g Section Key
$ curl -iX PUT 'http://localhost:5984/_con g/uuids/algorithm'
-H "Content-Type: application/json"
-d '"utc_random"'
HTTP/1.1 200 OK
"sequential"
205. Admin Party
Default settings after install are that everyone is admin
By default, CouchDB will only listen on the loopback address
You’ll want to change the default settings if you let CouchDB listen on a public IP
address!
207. SSL Con guration
[daemons]
;…
httpsd = {couch_httpd, start_link, [https]}
;…
[ssl]
cert_ le = /full/path/to/server_cert.pem
key_ le = /full/path/to/server_key.pem
;…
208. Connect to Port 6984 for SSL
$ curl -iX GET 'https://localhost:6984/'
HTTP/1.1 200 OK
Server: CouchDB/1.1.0 (Erlang OTP/R14B)
{
"couchdb": "Welcome",
"version": "1.1.0"
}
234. Validation Functions
De ned within design documents
Function passed:
• New document
• Old document
• User context:
• Database name (db)
• User name (name)
• Roles (roles)
235. Read-Only for Non-Admins
function(newDoc, oldDoc, userCtx) {
if (-1 === userCtx.roles.indexOf('_admin')) {
throw({forbidden: 'Only admins may update documents.'});
}
}
238. Con ict Resolution
Replication will eventually lead to con icting revisions
Both con icted revisions will be preserved, but one “wins”
Con icted documents will get a special "_con icts" ag
Create a view to nd con icted documents:
if (doc._con icts) { for (var i in doc._con icts) { emit(doc._con icts[i]); } }
Have end-user or application logic handle a merge
Clear the "_con icts" ag with a document update
239. Changes Feed
$ curl -iX GET 'http://localhost:5984/books/_changes'
HTTP/1.1 200 OK
{"results":[
{"seq":1,"id":"978-0-596-52926-0","changes":[{"rev":"1-12538b0…"}]},
{"seq":2,"id":"978-1-565-92580-9","changes":[{"rev":"1-b945cb4…"}]},
{"seq":3,"id":"978-0-596-80579-1","changes":[{"rev":"1-09ce09fe…"}]},
{"seq":4,"id":"978-0-596-15589-6","changes":[{"rev":"1-8c1d7354…"}]}
],
"last_seq":4}
240. Partial Replicas:
Filter Functions
Filter functions are de ned within design documents under the " lters" key.
Example of an "authors" lter function:
function(doc, req) { return 'author' == doc.collection }
The lter function (and its containing design document) can then be referenced
during replication:
" lter":"default/authors"
Filter functions can be parameterized:
function(doc, req) { return req.query.collection == doc.collection }
Parameters values can be speci ed during replication:
"query_params": { "collection":"publisher" }
241. Partial Replicas:
Specifying IDs
Provide an array of IDs when replicating:
"doc_ids": [
"be231efa93502b3286aae0ed7b000aed",
"be231efa93502b3286aae0ed7b001a6a",
"be231efa93502b3286aae0ed7b002916",
"be231efa93502b3286aae0ed7b003458",
"be231efa93502b3286aae0ed7b00375e"
]
245. Rewriting to an Attachment
{
"from": "",
"to": "index.html",
"method": "GET",
"query": {}
}
From:
GET /db/_design/app/_rewrite
To:
GET /db/_design/app/index.html
246. Rewriting to a List
{
"from": "books",
"to": "_list/books/all",
"query": {
"limit": 10
}
}
From:
GET /db/_design/app/_rewrite/books
To:
GET /db/_design/app/_list/books/all?limit=10
247. Rewriting to a Show
{
"from": "books/:id",
"to": "_show/books/:id",
"query": {}
}
From:
GET /db/_design/app/_rewrite/books/978-0-596-15589-6
To:
GET /db/_design/app/_show/books/978-0-596-15589-6
249. “vhosts” Con g Section
{
"example.com": "/db/_design/app/_rewrite",
"www.example.com": "/db/_design/app/_rewrite"
}
All requests with a Host HTTP header of example.com or www.example.com will
now be handled by the rewrite rules de ned in the app design document.
256. CouchApp
Applications built using CouchDB, JavaScript and HTML5
CouchDB is a database, web server and application server
No more middle-tier between your data and presentation layers:
• CouchDB becomes both the data and application tiers
• User agent becomes the presentation tier
Data and application can be replicated together
CouchApp Tool:
http://couchapp.org/
257. Mobile Couchbase for Android & iOS
http://www.couchbase.com/products-and-services/mobile-couchbase
259. CouchDB Resources
CouchDB: The De nitive Guide CouchDB Wiki
by J. Chris Anderson, Jan Lehnardt, and http://wiki.apache.org/couchdb/
Noah Slater (O’Reilly)
978-0-596-15589-6 Beginning CouchDB
by Joe Lennon (Apress)
Writing and Querying MapReduce Views in 978-1-430-27237-3
CouchDB
by Bradley Holt (O’Reilly)
978-1-449-30312-9
Scaling CouchDB
by Bradley Holt (O’Reilly)
063-6-920-01840-7