NoSQL @ Qbranch -2010-04-15

  • 7,992 views
Uploaded on

NoSQL overview presentation with details on Riak and CouchDB. …

NoSQL overview presentation with details on Riak and CouchDB.

Presented at Qbranch CODE Night 2010-04-15.


Thanks to @frli01 for arranging and @xlson for invitation.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
7,992
On Slideshare
0
From Embeds
0
Number of Embeds
9

Actions

Shares
Downloads
182
Comments
0
Likes
22

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide



  • * Relational not always most suitable model
    * Schema-less gives freedom
    * Non-relational gives interesting scalability capabilities (which most provides)
    * Most provides REST/JSON API
    ** Very suitable for web dev’t
    ** Easy peasy to use, regardless of environment





















































  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff
  • * Hinted handoff


































  • collation - assembling in proper numerical or logical sequence
  • Simplified view explanation










































Transcript

  • 1. Not only SQL Mårten Gustafson Qbranch CODE tech-meet @ 2010-04-15
  • 2. What? “NoSQL is a movement promoting a loosely defined class of non-relational data stores that break with a long history of relational databases” - Wikipedia
  • 3. What? “NoSQL is a movement promoting a loosely defined class of non-relational data stores that break with a long history of relational databases” - Wikipedia Not a single technique Not a single type of data Not a single type of use case
  • 4. Why? • Non-relational • Schema-less • “Easily” scalable • REST/JSON API = web friendly
  • 5. What’s out there? Storage type License Implemented in Amazon Dynamo Key/Value n/a ? Cassandra Columnfamily ASL 2.0 Java CouchDB Document ASL 2.0 Erlang Dynomite Key/Value BSD/MIT-style Erlang HBase Columnfamily ASL 2.0 Java MongoDB Document AGPL v3.0 C++ Neo4J Graph AGPL v3.0 / Comm Java Riak Key/Value ASL 2.0 Erlang Redis Key/Value BSD/MIT-style C Scalaris Key/Value ASL 2.0 Erlang Tokyo Cabinet Key/Value LGPL C Voldemort Key/Value ASL 2.0 Java
  • 6. Distribution • Master / Slave • Master / Slave(s) • Masterless (Master / Master)
  • 7. Distribution Masterless Master/Slave Hot standby Amazon Dynamo X Cassandra X CouchDB X Dynomite X HBase ? MongoDB X X Neo4J* Riak X Redis X Scalaris X Tokyo Cabinet Voldemort X * Neo4J HA coming “soon”
  • 8. Distribution Masterless Master/Slave ie Hot standbyw Amazon Dynamo X d v Cassandra X ifie l CouchDB X Dynomite X m p i HBase ? MongoDB y s X X Neo4J* e r v Riak X a Redis X is Scalaris X h i s Tokyo Cabinet Voldemort X T * Neo4J HA coming “soon”
  • 9. Common factor “...of the web...” Of the who?!
  • 10. Of the web “...Django may be built for the Web, but CouchDB is built of the Web. I’ve never seen software that so completely embraces the philosophies behind HTTP. CouchDB makes Django look old-school in the same way that Django makes ASP look outdated” - http://jacobian.org/writing/of-the-web/
  • 11. Of the web “...CouchDB may succeeded, and it may fail; who knows. I’m sure of one thing, though — this is what the software of the future looks like” - http://jacobian.org/writing/of-the-web/
  • 12. So freakin’ what?! All your webish skillz and tools apply...
  • 13. So freakin’ what?! language-, platform- and OS-neutral load balancers proxies MIME / Content-Type All your webish skillz and tools apply... HTTP client libs (etag, if-modified-since, etc) caches
  • 14. These guys can just suck it HTTP/REST is integration that works (YMMV)
  • 15. Buckle Up Dorothy. Cause' Kansas, Is Going Bye-Bye
  • 16. Key/Value Store I got keys but no locks
  • 17. Riak Decentralized key-value store A flexible map/reduce engine HTTP/JSON API A database ideally suited for Web applications
  • 18. The Ring
  • 19. The Ring 12 1 11 2 10 3 9 4 8 5 7 6 ring size = 12
  • 20. The Ring One Ring size to rule them all, One Ring size to find them, One Ring size to bring them all and in the cluster bind them...
  • 21. Consistent Hashing Store/Save (PUT)
  • 22. Consistent Hashing Store/Save (PUT)
  • 23. Consistent Hashing Read (GET) “I want “ is answered by: where is on the ring?
  • 24. Consistent Hashing Read (GET) “I want “ is answered by: where is on the ring?
  • 25. Cluster Instance A Instance B Instance C ring size = 12 instances = 3 ring size / nodes = ~slices per instances
  • 26. Cluster Instance A Instance B Instance C ring size = 12 instances = 3 ring size / nodes = ~slices per instances
  • 27. Cluster - Read (GET) Instance A Instance B Instance C
  • 28. Cluster - Read (GET) I can haz ? Instance A Instance B Instance C Hm, lives in a slice of the ring owned by instance C.
  • 29. Cluster - Read (GET) I can haz ? Instance A Instance B Instance C Okidoki, now Hey C! I need where’s he...a yeah in my fourth slice
  • 30. Cluster - Read (GET) I can haz ? Here ya go Instance A Instance B Instance C Cheers!
  • 31. Riak “stuff”
  • 32. Riak “stuff” Bucket Container/keyspace. Determines number of replicas for its contents
  • 33. Riak “stuff” Consistent Hashing Key hashing technique used to distribute keys on the ring Bucket Container/keyspace. Determines number of replicas for its contents
  • 34. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Gossiping Consistent Hashing Key hashing technique used to distribute keys on the ring Bucket Container/keyspace. Determines number of replicas for its contents
  • 35. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Gossiping Consistent Hashing Key hashing technique used to distribute keys on the ring Hinted Handoff Covering for a Bucket failed “neighbor” node while gone Container/keyspace. Determines number of replicas for its contents
  • 36. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Allows retrieval of “weakly” linked objects Gossiping Links Consistent Hashing Key hashing technique used to distribute keys on the ring Hinted Handoff Covering for a Bucket failed “neighbor” node while gone Container/keyspace. Determines number of replicas for its contents
  • 37. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Allows retrieval of “weakly” linked objects Gossiping Links Consistent Hashing Key hashing technique used to distribute keys on the ring Hinted Handoff Merkle Tree Covering for a Bucket Data structure for failed “neighbor” node while gone Container/keyspace. efficient summary about Determines number of objects. Gossiped. replicas for its contents
  • 38. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Allows retrieval of “weakly” linked objects Gossiping Links Node Consistent Hashing One server. Runs vnodes which claims partitions. Key hashing technique used to distribute keys on the ring Hinted Handoff Merkle Tree Covering for a Bucket Data structure for failed “neighbor” node while gone Container/keyspace. efficient summary about Determines number of objects. Gossiped. replicas for its contents
  • 39. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Allows retrieval of “weakly” linked objects Partition Gossiping Links One slice (part) of the ring. Node Consistent Hashing One server. Runs vnodes which claims partitions. Key hashing technique used to distribute keys on the ring Hinted Handoff Merkle Tree Covering for a Bucket Data structure for failed “neighbor” node while gone Container/keyspace. efficient summary about Determines number of objects. Gossiped. replicas for its contents
  • 40. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Allows retrieval of “weakly” linked objects Partition Gossiping Links One slice (part) of the ring. Node Auto correction of out-of-date objects Consistent Hashing One server. Runs vnodes which claims partitions. Read Repair Key hashing technique used to distribute keys on the ring Hinted Handoff Merkle Tree Covering for a Bucket Data structure for failed “neighbor” node while gone Container/keyspace. efficient summary about Determines number of objects. Gossiped. replicas for its contents
  • 41. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Allows retrieval of “weakly” linked objects Partition Gossiping Links One slice (part) of the ring. Node Auto correction of out-of-date objects Consistent Hashing One server. Runs vnodes which claims partitions. Read Repair Key hashing technique used to distribute keys on the ring Hinted Handoff Merkle Tree Covering for a Bucket Data structure for failed “neighbor” node while gone Number of copies Container/keyspace. efficient summary about of the same object Determines number of replicas for its contents objects. Gossiped. Replica in the cluster
  • 42. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Allows retrieval of “weakly” linked objects Partition Gossiping Links The complete “space”, One slice (part) of the ring. divided into partitions which are claimed by vnodes Ring Node Auto correction of out-of-date objects Consistent Hashing One server. Runs vnodes which claims partitions. Read Repair Key hashing technique used to distribute keys on the ring Hinted Handoff Merkle Tree Covering for a Bucket Data structure for failed “neighbor” node while gone Number of copies Container/keyspace. efficient summary about of the same object Determines number of replicas for its contents objects. Gossiped. Replica in the cluster
  • 43. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Allows retrieval of “weakly” linked objects Partition Gossiping Links The complete “space”, One slice (part) of the ring. divided into partitions which Vector Clock are claimed by vnodes Conflic detection technique for objects. Ring Node Auto correction of out-of-date objects Consistent Hashing One server. Runs vnodes which claims partitions. Read Repair Key hashing technique used to distribute keys on the ring Hinted Handoff Merkle Tree Covering for a Bucket Data structure for failed “neighbor” node while gone Number of copies Container/keyspace. efficient summary about of the same object Determines number of replicas for its contents objects. Gossiped. Replica in the cluster
  • 44. Shares state, bucket and ring knowledge in the cluster Riak “stuff” Allows retrieval of “weakly” linked objects Partition Gossiping Links The complete “space”, One slice (part) of the ring. Vnode divided into partitions which Vector Clock are claimed by vnodes Conflic detection technique for objects. Ring Node Runs in a node and claims one Auto correction of out-of-date objects Consistent Hashing One server. Runs vnodes which claims partition on the ring partitions. Read Repair Key hashing technique used to distribute keys on the ring Hinted Handoff Merkle Tree Covering for a Bucket Data structure for failed “neighbor” node while gone Number of copies Container/keyspace. efficient summary about of the same object Determines number of replicas for its contents objects. Gossiped. Replica in the cluster
  • 45. Riak - Takeaways • No single point of failure • Choose your levels for: • availability • consistency • partition tolerance
  • 46. But wait, there’s more... • Binary data + Content-Type = whatever • MP3’s, Images, Text, ... • Map/Reduce • Local data, parallel
  • 47. This slide intentionally left blank
  • 48. Document Store Relax
  • 49. CouchDB Document oriented databased Kick ass replication HTTP/JSON API Map/reduce view (index) definitions
  • 50. World view One document == JSON One document == One record Many documents == One database Many databases == One instance No schema
  • 51. World view Documents can have attachments (binary + mime type) be rendered differently (HTML, XML)
  • 52. A document Key, either you choose it or CouchDB does it for you { "_id": "b098445d587b1f347e48e1a79301de02", "_rev": "1-80bfd8302e0f08eec2396c8107cafc19", "platform": { "browser": "mozilla", "version": "1.9.1.8" }, "timestamp": 1270131033337 Revision } number
  • 53. Views Filter Collate Aggregate
  • 54. Views { "_id": "b098445d587b1f347e48e1a79301de02", "_rev": "1-80bfd8302e0f08eec2396c8107cafc19", "platform": { "browser": "mozilla", "version": "1.9.1.8" }, "timestamp": 1270131033337 } + function(doc) { emit(doc.platform.browser, doc.platform.version); } = { "total_rows": 1, "offset": 0, "rows": [ "id": "b098445d587b1f347e48e1a79301de02", "key": "mozilla", "value": "1.9.1.8" ]
  • 55. Views Views are stored as an accessible web resource on disk and incrementally updated as well as replicated with the database
  • 56. Replication Peer to peer Online/Offline Conflict detection and resolution Any number of nodes Local Remote
  • 57. Replication
  • 58. Replication
  • 59. Replication
  • 60. Replication
  • 61. Replication
  • 62. CouchDB “stuff”
  • 63. CouchDB “stuff” Append only Hence, won’t corrupt its data files
  • 64. CouchDB “stuff” MVCC Multi version concurrency control. Writers do not block readers. Readers do not block writers. Append only Hence, won’t corrupt its data files
  • 65. CouchDB “stuff” BDCRR MVCC Bi-directional, conflict resolving, replication Multi version concurrency control. Writers do not block readers. Readers do not block writers. Append only Hence, won’t corrupt its data files
  • 66. CouchDB “stuff” BDCRR MVCC Bi-directional, conflict resolving, replication Multi version concurrency control. Writers do not block readers. Readers do not block writers. Append only Compaction Hence, won’t corrupt its data files Append only will cause data files to grow. Compaction to the rescue, in the background - for your pleasure.
  • 67. CouchDB “stuff” BDCRR MVCC Bi-directional, conflict resolving, replication Multi version concurrency control. Writers do not block readers. Readers do not block writers. Append only Compaction Hence, won’t corrupt its data files Append only will cause data files to grow. Compaction to the rescue, in ACID the background - for your pleasure. Awesome, Cool, Impressive, Dope
  • 68. CouchDB - Takeaways • Kick ass replication • Views are fast • Can host and serve complete webapps
  • 69. Outro • Test one or more NoSQL thingys • Get familiar with Brewers CAP theorem • Get familiar with the Dynamo paper
  • 70. Over and out. Mårten Gustafson @martengustafson http://marten.gustafson.pp.se/ marten.gustafson@gmail.com