Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CouchConf Tokyo Developing with Couchbase Part II

1,825 views

Published on

Published in: Technology
  • Be the first to comment

CouchConf Tokyo Developing with Couchbase Part II

  1. 1. Advanced Document Design J Chris Anderson Mobile
  2. 2. <ul><li>When considering how to model data for a given application, you should… </li></ul><ul><li>Think of a logical container for the data </li></ul><ul><li>Think of how data groups together </li></ul><ul><li>You may notice that… </li></ul><ul><li>From a software development standpoint, this maps better to the way you structure the data </li></ul><ul><li>It’s a more natural way of representing the entities the application will be handling </li></ul>Modeling Data
  3. 3. Example of A User Profile
  4. 4. <ul><li>Key selection: start with the username </li></ul><ul><ul><li>Use this as a common prefix for related data </li></ul></ul><ul><ul><li>When building user’s session information, fetch these items </li></ul></ul><ul><li>Create related documents by prefixing with the same username </li></ul><ul><ul><li>Can extend this concept further </li></ul></ul><ul><ul><ul><li>Other data records for this user, building out object graph, etc. </li></ul></ul></ul>Example: Data Profile for a User { “ _id”: “ auser_profile ” , “ user_id ” : 7778 “ password”: “ a1004cdcaa3191b7 ” , ” common_name ” : ” Alan User ” , ” nicknames ” : [ ” Al ” , ” Buddy ” ], &quot;sign_up_timestamp&quot;: 1224612317, &quot;last_login_timestamp&quot;: 1245613101 } { “ _id”: “ auser_friends ” , “ friends”: [ “ joe ” , “ ingenthr ” , “ toru ” ] }
  5. 5. Social Game Example
  6. 6. <ul><li>As they prepped their new game in 2011, the developers at Tribal Crossing recognized they needed to plan for this game a bit differently: </li></ul><ul><li>Planned to scale </li></ul><ul><li>Deploys to cloud servers </li></ul><ul><li>Be able to easily scale and monitor the overall system </li></ul><ul><li>Previous experience: </li></ul><ul><li>My polls went from 100 users to 1M users in a very long weekend of resharding! </li></ul>Tribal Crossing
  7. 7. <ul><li>But, there are some differences in using Couchbase to handle the game data: </li></ul><ul><ul><ul><li>Direct key or _id queries only </li></ul></ul></ul><ul><ul><ul><li>Transactions do not span objects </li></ul></ul></ul><ul><li>add speed and distribution </li></ul><ul><li>Tribal Questioned: Can this work for an online game? </li></ul><ul><ul><ul><li>Decided they could, just needed to tackle the modeling issues </li></ul></ul></ul><ul><ul><ul><li>Decided that for their deployment, they could relax some constraints to allow for higher concurrency and “repair” issues when encountered by erring to player happyness </li></ul></ul></ul>Modeling Data
  8. 8. <ul><li>Representing Objects </li></ul><ul><li>Simply treat an object as a JSON document </li></ul><ul><ul><li>Could also represent it with serialized objects from the higher level language </li></ul></ul><ul><li>Determine the key for an object using the class name (or type) of the object and an unique ID </li></ul><ul><li>Representing Object Lists </li></ul><ul><li>Denormalization </li></ul><ul><li>Save an array of object IDs in this list </li></ul>Tribal Crossing: Representing Game Data in Couchbase
  9. 9. <ul><li>Three entities (for this simple example) </li></ul><ul><ul><li>Players </li></ul></ul><ul><ul><ul><li>Hold the player’s profile, enough to instantiate the needed data for a session </li></ul></ul></ul><ul><ul><li>Plant lists </li></ul></ul><ul><ul><ul><li>A simple list of plants owned by a given player– semi normalized </li></ul></ul></ul><ul><ul><li>Plants </li></ul></ul><ul><ul><ul><li>The plants owned by a given player </li></ul></ul></ul>Modeling the Example Game Data
  10. 10. <ul><li>Player Object </li></ul><ul><li>Key : 'Player1' </li></ul><ul><li>JSON </li></ul><ul><li>{ </li></ul><ul><li>“ _id ” : “ Player1 ” , </li></ul><ul><li>“ nid ” : 1, </li></ul><ul><li>“ name ” : “ Shawn ” </li></ul><ul><li>} </li></ul>Social Game Data in Couchbase Plant Object Key : 'Plant201' JSON { “ _id ” : “ Plant201 ” , “ nid ” : 201, “ player_id ” : 1 “ name ” : “ Starflower ” } PlayerPlant List Key : 'Player1_PlantList' JSON { “ _id ” : “ Player1_Plantlist ” , “ plants ” : [201, 202, 204] }
  11. 11. <ul><li>No need to “ ALTER TABLE ” </li></ul><ul><li>Add new “ fields ” all objects at any time </li></ul><ul><ul><ul><li>Specify default value for missing fields </li></ul></ul></ul><ul><ul><ul><li>Increased development speed </li></ul></ul></ul><ul><li>Using JSON for data objects </li></ul><ul><ul><li>This will allow future capabilities with Couchbase Server 2.0 </li></ul></ul><ul><ul><li>Offers the ability to query and analyze arbitrary fields with views </li></ul></ul>Tribal Crossing: Schema-less Modeling of Data
  12. 12. <ul><li>Give a player a new plant </li></ul><ul><li>// Create the new plant </li></ul><ul><li>Plant givenPlant = new Plant(100, &quot;Mushroom&quot;); </li></ul><ul><li>cbclient.set(&quot;Plant100&quot;, givenPlant); </li></ul><ul><li>// Update the player plant list </li></ul><ul><li>Player thePlayer = player.fetch(cbclient.get(&quot;Player1&quot;); </li></ul><ul><li>// Add the plant to the player </li></ul><ul><li>thePlayer.receivePlant(givenPlant); </li></ul><ul><li>// Store the player's new plant list </li></ul><ul><li>cbclient.set(&quot;Player1_PlantList&quot;, </li></ul><ul><li>thePlayer.getPlantsArray()); </li></ul>Modifying Game Data in Couchbase
  13. 13. When to break up a document: blog example
  14. 14. <ul><li>User profile </li></ul><ul><ul><li>Main pointer into the user data: </li></ul></ul><ul><ul><ul><li>Blog entries </li></ul></ul></ul><ul><ul><ul><li>Badge settings, like a twitter badge </li></ul></ul></ul><ul><li>Blog posts </li></ul><ul><ul><li>Contains the blogs themselves </li></ul></ul><ul><li>Blog comments </li></ul><ul><ul><li>Comments from other users </li></ul></ul>Entities for a Blog
  15. 15. <ul><li>{ </li></ul><ul><li>“ _id”: “ jchris_Hello_World ” , </li></ul><ul><li>“ author”: “ jchris ” , </li></ul><ul><li>“ type”: “post” </li></ul><ul><li>“ title”: “Hello World”, </li></ul><ul><li>“ format”: “markdown”, </li></ul><ul><li>“ body”: “Hello from [Couchbase]( http://couchbase.com ).”, </li></ul><ul><li>“ html”: “<p>Hello from <a href=“http: … </li></ul><ul><li>“ comments”: [ </li></ul><ul><li>[ “format”: “markdown”, “ body ” : ” Awesome post! ” ], </li></ul><ul><li>[ “format”: “markdown”, “ body ” : ” Like it. ” ] </li></ul><ul><li>] </li></ul><ul><li>} </li></ul>Blog Document Sample
  16. 16. Blog Document Sample, Broken up { “ _id”: “ jchris_Hello_World ” , “ author”: “ jchris ” , “ type”: “post” “ title”: “Hello World”, “ format”: “markdown”, “ body”: “Hello from [Couchbase]( http://couchbase.com ).”, “ html”: “<p>Hello from <a href=“http: … “ comments”: [ “ comment1_jchris_Hello_world” ] } { “ _id”: “comment1_jchris_Hello_World”, “ format”: “markdown”, “ body ” : ” Awesome post! ” }
  17. 17. <ul><li>You can imagine how to take this to a threaded list </li></ul>Threaded Comments Blog First comment Reply to comment More Comments List List <ul><li>Advantages: </li></ul><ul><li>Only fetch the data when you need it </li></ul><ul><ul><li>For example, rendering part of the page, jQuery style </li></ul></ul><ul><li>Spread the data (and thus the load) across the entire cluster </li></ul>
  18. 18. Couchbase Server 2.0: Querying and Aggregation with Views
  19. 19. <ul><li>Player items </li></ul><ul><ul><li>Find all the weapons owned by an individual player </li></ul></ul><ul><li>Leader board view </li></ul><ul><ul><li>Showing who has the highest level in the system </li></ul></ul>Demo: Game Simulator Views
  20. 20. <ul><li>Q: How can I implement transactions across multiple documents? </li></ul><ul><li>A: Transactions in this distributed system can be re-imagined with views . (example to follow…) </li></ul><ul><li>Q: Can I write more complex aggregation logic? For example apply a transform or a filter when reducing? </li></ul><ul><li>A: While the built in reduce functions are often enough and highly tuned, Couchbase Server can execute arbitrary javascript functions. </li></ul>Common Questions
  21. 21. <ul><li>Simulate the way federated systems work: </li></ul><ul><ul><li>Examples: checking accounts, credit card transactions </li></ul></ul><ul><li>Create a record per transaction </li></ul><ul><ul><li>Leverage views to reconcile the results of the transaction </li></ul></ul><ul><ul><li>If results don’t reconcile, there is a missing transaction or a flaw in the business logic </li></ul></ul>Exchanging Virtual Currency { “From”: “matt”, “ to”: “ jchris ” , “ coins ” : 30 } { “From”: “matt”, “ to”: “ perry ” , “ coins ” : 30 } { “From”: “ jchris ” , “ to”: “ perry ” , “ coins ” : 30 } { “From”: “matt”, “ to”: “ jchris ” , “ coins ” : 30 } { “From”: “matt”, “ to”: “ perry ” , “ coins ” : 30 } { “From”: “ jchris ” , “ to”: “ perry ” , “ coins ” : 30 }
  22. 22. Exchanging Virtual Currency { “From”: “matt”, “to”: “ jchris ” , “ coins ” : 30 } { “From”: “matt”, “to”: “ perry ” , “ coins ” : 30 } { “From”: “ jchris ” , “ to ” : “ perry ” , “ coins ” : 30 } Mapper function(transaction) { emit(transaction.from, transaction.amount *-1); emit (transaction.to, transaction.amount); } Reduce function(keys, values) { return sum(values); } Query balance view with key == user, get the balance for the user. Query the sum of the entire view, value should be 0.
  23. 23. Questions?

×