Schema Design for Riak
Upcoming SlideShare
Loading in...5
×
 

Schema Design for Riak

on

  • 13,290 views

Moving to Riak involves a number of changes from the status quo of RDBMS systems, one of which is taking greater control over your schema design. ...

Moving to Riak involves a number of changes from the status quo of RDBMS systems, one of which is taking greater control over your schema design.

* How do you structure data when you don't have tables and foreign keys?
* When should you denormalize, add links, or create map-reduce queries?
* Where will Riak be a natural fit and where will it be challenging?

Statistics

Views

Total Views
13,290
Views on SlideShare
11,582
Embed Views
1,708

Actions

Likes
18
Downloads
236
Comments
0

12 Embeds 1,708

http://basho.com 1407
http://nosql.mypopescu.com 179
http://basho.co.jp 52
http://coderwall.com 36
http://crowdczar.com 15
http://localhost 5
http://localhost:3000 4
http://localhost:3000 4
http://www.slideshare.net 2
http://translate.googleusercontent.com 2
http://twitter.com 1
http://www-new.basho.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Schema Design for Riak Schema Design for Riak Presentation Transcript

  • Schema Design for Riak Sean Cribbs Developer Advocate basho
  • Thinking Non-Relationally basho
  • “There is no spoon schema.” basho
  • There is an application. basho
  • What you lose basho
  • What you lose •Tables basho
  • What you lose •Tables •Foreign keys and constraints basho
  • What you lose •Tables •Foreign keys and constraints •ACID basho
  • What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners basho
  • What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners •Declarative query language (SQL) basho
  • What you gain basho
  • What you gain •More flexible, fluid designs basho
  • What you gain •More flexible, fluid designs •More natural data representations basho
  • What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain basho
  • What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain •Reduced operational complexity basho
  • Lose the crutches basho
  • Lose the crutches •Sparse data (optional/multi-value fields) basho
  • Lose the crutches •Sparse data (optional/multi-value fields) •Richer data structures basho
  • Lose the crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers basho
  • Lose the crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers •Innovative access patterns basho
  • Eventual Consistency basho
  • Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) basho
  • Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) •Plan for EC instead of triaging basho
  • Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) •Plan for EC instead of triaging •Determine resolution processes basho
  • Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) •Plan for EC instead of triaging •Determine resolution processes •Weigh risk of conflicts and emergent issues basho
  • Know your data basho
  • Make your Top 10 list basho
  • Make your Top 10 list •Frequently requested pages/screens basho
  • Make your Top 10 list •Frequently requested pages/screens •Slow queries basho
  • Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes basho
  • Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes •Complicated joins or aggregations basho
  • Analyze basho
  • Analyze •Interdependencies, coupling basho
  • Analyze •Interdependencies, coupling •Cardinalities of relationships basho
  • Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern basho
  • Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern •Shoehorned structures basho
  • Example basho
  • basho
  • basho
  • Top queries basho
  • List Features basho
  • Reorder Features basho
  • Regroup Features basho
  • Edit Feature basho
  • List Folders basho
  • Project Statistics basho
  • projects users folders features milestones project_items Current Schema basho
  • projects users folders artificial ordering features milestones project_items Current Schema basho
  • projects users additional key for stats folders artificial ordering features milestones project_items Current Schema basho
  • projects users additional key for stats BLOB folders columns artificial serialized ordering features milestones parse tree formatted project_items plain text Current Schema basho
  • collaborator projects users additional key for stats BLOB folders columns artificial serialized ordering features milestones parse tree formatted project_items plain text Current Schema basho
  • collaborator projects users additional key for stats owner BLOB folders columns artificial serialized ordering features milestones parse tree formatted project_items plain text Current Schema basho
  • Observations basho
  • Observations •Features and milestones are disjunct, except in list participation basho
  • Observations •Features and milestones are disjunct, except in list participation •Opacity of serialized BLOB data basho
  • Observations •Features and milestones are disjunct, except in list participation •Opacity of serialized BLOB data •Duplication of relationships - collaborator-owner, project-folder-item basho
  • List/Reorder Features basho
  • List/Reorder Features • AJAX postback was already “total” ordering basho
  • List/Reorder Features • AJAX postback was already “total” ordering • List is really a property of the parent “folder” basho
  • List/Reorder Features • AJAX postback was already “total” ordering • List is really a property of the parent “folder” • Membership, order are coupled concepts basho
  • Links in Riak basho
  • Links in Riak •Lightweight pointers, like HTML <link> tags. basho
  • Links in Riak •Lightweight pointers, like HTML <link> tags. •Composed of bucket, key, tag basho
  • Links in Riak •Lightweight pointers, like HTML <link> tags. •Composed of bucket, key, tag •Traversable through “link-walking” basho
  • Riak: List/Reorder Features basho
  • Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag basho
  • Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag •Query: Follow links to items, map values (with index), reduce sort basho
  • Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag •Query: Follow links to items, map values (with index), reduce sort •One write to order list, easy to link-walk basho
  • Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag •Query: Follow links to items, map values (with index), reduce sort •One write to order list, easy to link-walk • Concurrent writes may conflict basho
  • Regroup Features basho
  • Regroup Features • Changing foreign key to folder basho
  • Regroup Features • Changing foreign key to folder • Recalculate order for both lists (multiple UPDATE statements) basho
  • Riak: Regroup Features basho
  • Riak: Regroup Features •Already solved for us by folder owning list as links basho
  • Riak: Regroup Features •Already solved for us by folder owning list as links •List order still recalculated basho
  • Riak: Regroup Features •Already solved for us by folder owning list as links •List order still recalculated •Two writes (vs. N for relational) basho
  • Riak: Regroup Features •Already solved for us by folder owning list as links •List order still recalculated •Two writes (vs. N for relational) • No transactions, need extra info for conflict resolution basho
  • Edit Feature basho
  • Edit Feature • Required two representations (text, tree), both stored as BLOBs basho
  • Edit Feature • Required two representations (text, tree), both stored as BLOBs • Application deserializes tree to present web form, recreates text on save basho
  • Composition basho
  • Composition •Sometimes called “denormalization” basho
  • Composition •Sometimes called “denormalization” •Store dependent data in the same object as the parent basho
  • Composition •Sometimes called “denormalization” •Store dependent data in the same object as the parent •JSON, XML, Erlang terms basho
  • Riak: Edit Feature basho
  • Riak: Edit Feature •Already denormalized! basho
  • Riak: Edit Feature •Already denormalized! •Now parse tree can have the same format (JSON, XML), exposing it to other types of processing basho
  • Riak: Edit Feature •Already denormalized! •Now parse tree can have the same format (JSON, XML), exposing it to other types of processing •Individual objects larger basho
  • List Folders basho
  • List Folders • Number of folders is small per project basho
  • List Folders • Number of folders is small per project • Essentially unordered within project basho
  • Riak: List Folders basho
  • Riak: List Folders • Store links to folder objects (or entire list) in project basho
  • Riak: List Folders • Store links to folder objects (or entire list) in project • Simple link-walking or map-reduce to get folders basho
  • Riak: List Folders • Store links to folder objects (or entire list) in project • Simple link-walking or map-reduce to get folders • Two writes required, or move feature links/ order to project basho
  • Riak: List Folders • Store links to folder • Parameterized bucket objects (or entire list) in based on project key project • Simple link-walking or map-reduce to get folders • Two writes required, or move feature links/ order to project basho
  • Riak: List Folders • Store links to folder • Parameterized bucket objects (or entire list) in based on project key project • One write to add folder • Simple link-walking or map-reduce to get folders • Two writes required, or move feature links/ order to project basho
  • Riak: List Folders • Store links to folder • Parameterized bucket objects (or entire list) in based on project key project • One write to add folder • Simple link-walking or map-reduce to get • List keys is expensive folders • Two writes required, or move feature links/ order to project basho
  • Project Statistics basho
  • Project Statistics • Relational design had per-feature counters (maintained by app) basho
  • Project Statistics • Relational design had per-feature counters (maintained by app) • Extra foreign key sped up aggregation by reducing joins basho
  • Riak: Project Statistics basho
  • Riak: Project Statistics •Use scatter-gather: Map-reduce was made for this type of problem! basho
  • Riak: Project Statistics •Use scatter-gather: Map-reduce was made for this type of problem! •Take advantage of now-transparent internal format of feature objects basho
  • Riak: Project Statistics •Use scatter-gather: Map-reduce was made for this type of problem! •Take advantage of now-transparent internal format of feature objects •Query: Follow links to features, map per- feature counters, reduce summarize the counters basho
  • Awkward Fits basho
  • Awkward Fits •User uniqueness constraint basho
  • Awkward Fits •User uniqueness constraint •“double-click” problem basho
  • Awkward Fits •User uniqueness constraint •“double-click” problem •Existing authentication solutions basho
  • Awkward Fits •User uniqueness constraint •“double-click” problem •Existing authentication solutions •Project membership basho
  • Awkward Fits •User uniqueness constraint •“double-click” problem •Existing authentication solutions •Project membership •M-to-N is hard basho
  • Review basho
  • Review •Analyze your relational model basho
  • Review •Analyze your relational model •Identify pain points, take stats basho
  • Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives basho
  • Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives •Test, Measure, Repeat! basho
  • Plug Interested in learning about support, consulting, or Enterprise features?   Email info@basho.com or go to http://www.basho.com/contact.html to talk with us. www.basho.com basho
  • Plug Interested in learning about support, consulting, or Enterprise features?   Email info@basho.com or go to http://www.basho.com/contact.html to talk with us. www.basho.com sean@basho.com @seancribbs basho