Next Generation Data Storage with CouchDb

1. Next Generation Data
Storage with CouchDb
Jan Lehnardt <jan@php.net>
1

2. About Me
• Freelancing Developer with strong focus on
the Web for ~6 Years
• Studies in Computer Science and
Linguistics (not yet finished :-)
• Co-founder of FREISATZ
• Love Rock music and kittens
2

3. About You?
•What do you work on?
• What are you interested in?
• Why are you here?
3

4. Target Audience
• (Let’s see how well this matches the responses to the last slide)
• Webdevelopers, interested in new
technology
• System Architects that look for scalable and
fault tolerant software solutions
• Anybody else is welcome, too
4

5. Forget SQL
• Just for a moment, imagine you know
nothing about relational data, normalization
and other fancy terminology
• You only want to store data
5

6. The Four Pillars of Data
Management
6

7. Four Pillars (1): Save
• Save data securely,
permanent etc...
• i.e. ACID compliant
• Save data efficient
7

8. Four Pillars (II): See
• Make data available for easy
retrieval
• Integrate simple reporting
methods
• Provide (fulltext) search
8

9. Four Pillars (III): Secure
• Enable strong
compartmentalization of data
• Assign users, groups and roles
to data
• Allow connections over SSL
9

10. Four Pillars (IV): Share
• To make it complete:
• Do all that in a distributed
way
• On- and Offline(!)
10

11. No Relations
• Enter CouchDb
• New implementation, no legacy
• Designed to…
• …implement the four Pillars
• …leverage recent paradigm shifts
11

12. CouchDb for Database
Experts
Now, remember SQL and all you
know about traditional database
systems
12

13. Databases
Pool for related data
13

14. Documents
• Records of data
• Individual storage elements
• Can have an arbitrary (JSON)
structure…
• …but don’t have to!
• Attachments
• Have IDs & revisions
14

15. Views
• Similar to Views
• Compiled, pre-defined subset of stored
data
• External JavaScript Daemon(s)
15

16. CouchDb Features
Roubust Data Storage Repliaction
REST API User Authentication
JavaScript Views Built on Erlang/OTP
Your Feature Here, it’s
Fulltext Search Open Source!
16

17. Documents
• Individual storage elements
• Can have an arbitrary (JSON)
structure…
• …but don’t have to!
• Attachments
• Have IDs & revisions
17

18. Example Document
{
_id:”098F6BCD4621D373CADE4E832627B4F6”,
_rev: 4134847,
“BugId”: “098F6BCD4621D373CADE4E832627B4F6”,
“Counter”:1,
“Name”:”test”,
“Type”:”Project”
}
18

19. Data Storage
• State-of-the-art storage model
• MVCC with optimistic locking
• Sacles magnitudes better than
pessimistic locking
• Append-only writes
19

20. REST API
• Easy retrieval using a method proven to be
scalable: HTTP
• Exchange in industry-standard formats
(JSON)
• Simple and intuitive interface
20

21. Example URIs
• http://server/company/_all_docs
• http://server/company/765432
• http://server/company/_design/people
• http://server/company/_design/people?
count=7&startkey=”L”&endkey=”M”
21

22. JavaScript Views
• JS Functions to perform pattern matching
on documents
• Map-Reduce to define document subsets
• (Actual language changeable)
22

23. Example View
function(doc) {
map(doc.time, {
'teaser':doc.text.substr(0,10),
'_id':doc.id
});
}
23

24. Example View
function(doc) {
if(“Article” == doc.type) {
map(doc.time, {
'teaser':doc.text.substr(0,10),
'_id':doc.id
});
}}
24

25. Fulltext Search
• Leverages Lucene
• Reference implementation included
• Simple API to plug in other search
technology
25

26. Security and
Authentication
• Not yet implemented
• Will have
• Standard user authentication for
applications
• Document ownership
26

27. Replication
• Bi-directional replication (or
synchronization) between any number of
instances.
• Online, and:
• Offline(!)
• Including consistent conflict resolution
for disconnected systems. Wow!
27

28. Erlang Core
• Awkward syntax, but hey :-)
• Multi-everything
• Lightweight threading
• Awesome performance
• Die-on-error design
28

29. Demo
29

30. Cool things to do with CouchDb
• Distributed Anything-Application
• The Wiki
• Version Control System
• FUSE plugin
• MySQL Storage Engine
30

31. About Damien
• Damien Katz
• Ex-Iris/Lotus/IBM
• Helped creating the Lotus Notes
Database / Domino
31

32. Links
• http://couchdb.com/
• http://couchdbwiki.com/
• http://damienkatz.net/
• http://jan.prima.de/
• The Book
32

33. Cheers!
You’ve been great!
33