CouchConf London: Developing II Advanced Document Design
 

CouchConf London: Developing II Advanced Document Design

on

  • 815 views

 

Statistics

Views

Total Views
815
Views on SlideShare
815
Embed Views
0

Actions

Likes
0
Downloads
7
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • Modeling data from the real world to software is nothing new. In the case of Couchbase, as a document oriented system, data modeling is pretty easy. We aren’t constrained by schemas and needing to fit things into relational algebra.\n\nInstead, we only need to think about \n
  • \n
  • \n
  • \n
  • \n
  • No easy way to query:\nHow often do you need to run complex join query? When data is denormalized for speed, how much complex query are you really running?\n\n"Stop thinking in terms of joins and queries is ticket to speed"\n\nNot handling bank transactions:\nWe can live with small percentage of concurrency issue. Err on the side of making player happy. \n
  • \n
  • \n
  • \n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • If you’re expecting a very large number of comments, or want to display them threaded, you can easily imagine doing so by extending the list technique discussed earlier. This allows us to very easily build very complex arrangements of the data across various keys. Since they distribute throughout the cluster, we spread load out among the cluster nodes.\n\nIn contrast, with a typical relational model, you may have to have the comments and blogs colocated on a single shard system so you can use join queries. This creates hotspots in the system, and resharding to redistribute the data becomes a manual process.\n\nBecause of the active cache management in Couchbase Server, the hottest data will be in memory and served very quickly, so the data items may be served very quickly if they’re popular.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Using a view over all transactions, say they’re in a separate bucket or have type information on them, we can easily query for individual balances.\n
  • Using a view over all transactions, say they’re in a separate bucket or have type information on them, we can easily query for individual balances.\n
  • Using a view over all transactions, say they’re in a separate bucket or have type information on them, we can easily query for individual balances.\n
  • \n
  • \n

CouchConf London: Developing II Advanced Document Design CouchConf London: Developing II Advanced Document Design Presentation Transcript

  • Advanced Document Design Jan Lehnardt jan@couchbase.com
  • First Principles
  • First Principles All Hail JSON Model your data for your app, not for the databaseGroup data together in types of documents
  • Example of A User Profile
  • Example: Data Profile for a User Start with a { username "_id": "user7778_profile", "user7778" as "user_id": 7778 the key "password": "a3191b7", "name": "Alan User", Use this as a "nicknames": ["Al", "Fred"], common prefix "sign_up": 1224612317, for related data "last_login": 1245613101 } When building user’s session information, fetch these items
  • Example: Data Profile for a User Create { related "_id": "user7778_friends", documents "friends": [ "joe", by prefixing "ingenthr", with the "toru" ] same } username { "_id": "user7778_history", Can extend "history": [ this concept {"action":"post", further "timestamp":1245614000}, {"action":"comment", "timestamp":1245623563} Other data records for this ] user, building out }
  • The Famous Blog Example
  • Modeling a Blog Post Blog posts to Comment comments: a 1:N Comment relationship Comment Comment 6
  • Blog Data in CouchbaseStart out easy: - Make model objects as JSON documents - use previously learned _id - add a unique part- add rest of your data{ "_id": "user7778_post_e11eadea" "title": "It’s a dogs life in the British Army", "body": "Well, well well…", "posted": 1234533356} 9
  • The Front Page A list document with references { "_id": "user7778_blog", "posts": [ "user7778_post_e11eadea", "user7778_post_44ff337c", "user7778_post_037ababa" ] } 9
  • Comments as Lists of DocumentsTwo simple steps: - List items as documents - A list document with references{"_id": "user7778_comment_f4ea7b17", "text":"First"}{"_id": "user7778_comment_f00db44d", "text":"No!"}{"_id": "user7778_comment_deadbeef", "text":"I agree."}{ "_id": "user7778_post_e11eadea_comments", "comments": [ "user7778_comment_f4ea7b17", "user7778_comment_f00db44d", "user7778_comment_deadbeef" ]} 9
  • Threaded Comments
  • Threaded CommentsYou can imagine how to take this to a threaded list
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog First comment
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog First comment Reply to comment
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog First comment Reply to comment More Comment s
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog Li st First comment Reply to comment More Comment s
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog Li st First comment Reply to comment More Comment s
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog Li st First comment Reply to comment More Comment s
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog Li st First comment Reply to comment More Comment s
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog Li st First comment Reply to comment List More Comment s
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog Li st First comment Reply to comment List More Comment s
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog Li st First comment Reply to comment List More Comment s
  • Threaded CommentsYou can imagine how to take this to a threaded list Blog Li st First comment Reply to comment List More Comment sAdvantages:• Only fetch the data when you need it• For example, rendering part of the page, jQuery style• Spread the data (and thus the load) across the entire cluster
  • Schema-less Modeling of Data● No ALTER TABLE● Add new "fields" to any object any time ● improves development speed● JSON data is interoperable ● 2.0 Features ● dynamic queries with views ● Exchange with other systems ● Web-native 10
  • Couchbase Server 2.0:Querying and Aggregation with Views
  • The Bank
  • The BankDun, dun, dun duuuhn.
  • Exchanging Virtual Currency• Simulate the way { "from": "matt", federated systems "to": "jchris", "coins": 10 } work: { "from": "matt", "to": "perry", "coins": 20 }• Examples: checking accounts, credit card { "from": "jchris", "to": "perry", "coins": 30 } transactions• Create a record per { "from": "jchris", "to": "matt", "coins": 10 } transaction { "from": "perry",• Use views to reconcile "to": "matt", "coins": 30 } the results of the transaction { "from": "jchris", "to": "perry", "coins": 20 }• If results don’t reconcile, there is a missing transaction or
  • Exchanging Virtual Currency { "from": "matt", "to": "jchris", "coins": 10 } { "from": "matt", "to": "perry", "coins": 20 } { "from": "jchris", "to": "perry", "coins": 30 } Mapper function(transaction) { emit(transaction.from, transaction.amount *-1); emit(transaction.to, transaction.amount); } Reduce _sum
  • The View Result {"key":"jchris", "value": 10} {"key":"jchris", "value":-30} {"key":"jchris", "value":-10} {"key":"jchris", "value":-20} {"key":"matt", "value":-10} {"key":"matt", "value":-20} {"key":"matt", "value": 10} {"key":"matt", "value": 30} {"key":"perry", "value": 20} {"key":"perry", "value": 30} {"key":"perry", "value":-30} {"key":"perry", "value": 20}
  • Group! {"key":"jchris", "value":-50} {"key":"matt", "value":10} {"key":"perry", "value":40}
  • Questions?
  • Thanks! Jan Lehnardt jan@couchbase.com