Schema Design for Riak
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Schema Design for Riak

  • 13,720 views
Uploaded on

Moving to Riak involves a number of changes from the status quo of RDBMS systems, one of which is taking greater control over your schema design. ...

Moving to Riak involves a number of changes from the status quo of RDBMS systems, one of which is taking greater control over your schema design.

* How do you structure data when you don't have tables and foreign keys?
* When should you denormalize, add links, or create map-reduce queries?
* Where will Riak be a natural fit and where will it be challenging?

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
13,720
On Slideshare
12,012
From Embeds
1,708
Number of Embeds
12

Actions

Shares
Downloads
241
Comments
0
Likes
18

Embeds 1,708

http://basho.com 1,407
http://nosql.mypopescu.com 179
http://basho.co.jp 52
http://coderwall.com 36
http://crowdczar.com 15
http://localhost 5
http://localhost:3000 4
http://localhost:3000 4
http://www.slideshare.net 2
http://translate.googleusercontent.com 2
http://twitter.com 1
http://www-new.basho.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Transcript

  • 1. Schema Design for Riak Sean Cribbs Developer Advocate basho
  • 2. Thinking Non-Relationally basho
  • 3. “There is no spoon schema.” basho
  • 4. There is an application. basho
  • 5. What you lose basho
  • 6. What you lose •Tables basho
  • 7. What you lose •Tables •Foreign keys and constraints basho
  • 8. What you lose •Tables •Foreign keys and constraints •ACID basho
  • 9. What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners basho
  • 10. What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners •Declarative query language (SQL) basho
  • 11. What you gain basho
  • 12. What you gain •More flexible, fluid designs basho
  • 13. What you gain •More flexible, fluid designs •More natural data representations basho
  • 14. What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain basho
  • 15. What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain •Reduced operational complexity basho
  • 16. Lose the crutches basho
  • 17. Lose the crutches •Sparse data (optional/multi-value fields) basho
  • 18. Lose the crutches •Sparse data (optional/multi-value fields) •Richer data structures basho
  • 19. Lose the crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers basho
  • 20. Lose the crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers •Innovative access patterns basho
  • 21. Eventual Consistency basho
  • 22. Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) basho
  • 23. Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) •Plan for EC instead of triaging basho
  • 24. Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) •Plan for EC instead of triaging •Determine resolution processes basho
  • 25. Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) •Plan for EC instead of triaging •Determine resolution processes •Weigh risk of conflicts and emergent issues basho
  • 26. Know your data basho
  • 27. Make your Top 10 list basho
  • 28. Make your Top 10 list •Frequently requested pages/screens basho
  • 29. Make your Top 10 list •Frequently requested pages/screens •Slow queries basho
  • 30. Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes basho
  • 31. Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes •Complicated joins or aggregations basho
  • 32. Analyze basho
  • 33. Analyze •Interdependencies, coupling basho
  • 34. Analyze •Interdependencies, coupling •Cardinalities of relationships basho
  • 35. Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern basho
  • 36. Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern •Shoehorned structures basho
  • 37. Example basho
  • 38. basho
  • 39. basho
  • 40. Top queries basho
  • 41. List Features basho
  • 42. Reorder Features basho
  • 43. Regroup Features basho
  • 44. Edit Feature basho
  • 45. List Folders basho
  • 46. Project Statistics basho
  • 47. projects users folders features milestones project_items Current Schema basho
  • 48. projects users folders artificial ordering features milestones project_items Current Schema basho
  • 49. projects users additional key for stats folders artificial ordering features milestones project_items Current Schema basho
  • 50. projects users additional key for stats BLOB folders columns artificial serialized ordering features milestones parse tree formatted project_items plain text Current Schema basho
  • 51. collaborator projects users additional key for stats BLOB folders columns artificial serialized ordering features milestones parse tree formatted project_items plain text Current Schema basho
  • 52. collaborator projects users additional key for stats owner BLOB folders columns artificial serialized ordering features milestones parse tree formatted project_items plain text Current Schema basho
  • 53. Observations basho
  • 54. Observations •Features and milestones are disjunct, except in list participation basho
  • 55. Observations •Features and milestones are disjunct, except in list participation •Opacity of serialized BLOB data basho
  • 56. Observations •Features and milestones are disjunct, except in list participation •Opacity of serialized BLOB data •Duplication of relationships - collaborator-owner, project-folder-item basho
  • 57. List/Reorder Features basho
  • 58. List/Reorder Features • AJAX postback was already “total” ordering basho
  • 59. List/Reorder Features • AJAX postback was already “total” ordering • List is really a property of the parent “folder” basho
  • 60. List/Reorder Features • AJAX postback was already “total” ordering • List is really a property of the parent “folder” • Membership, order are coupled concepts basho
  • 61. Links in Riak basho
  • 62. Links in Riak •Lightweight pointers, like HTML <link> tags. basho
  • 63. Links in Riak •Lightweight pointers, like HTML <link> tags. •Composed of bucket, key, tag basho
  • 64. Links in Riak •Lightweight pointers, like HTML <link> tags. •Composed of bucket, key, tag •Traversable through “link-walking” basho
  • 65. Riak: List/Reorder Features basho
  • 66. Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag basho
  • 67. Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag •Query: Follow links to items, map values (with index), reduce sort basho
  • 68. Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag •Query: Follow links to items, map values (with index), reduce sort •One write to order list, easy to link-walk basho
  • 69. Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag •Query: Follow links to items, map values (with index), reduce sort •One write to order list, easy to link-walk • Concurrent writes may conflict basho
  • 70. Regroup Features basho
  • 71. Regroup Features • Changing foreign key to folder basho
  • 72. Regroup Features • Changing foreign key to folder • Recalculate order for both lists (multiple UPDATE statements) basho
  • 73. Riak: Regroup Features basho
  • 74. Riak: Regroup Features •Already solved for us by folder owning list as links basho
  • 75. Riak: Regroup Features •Already solved for us by folder owning list as links •List order still recalculated basho
  • 76. Riak: Regroup Features •Already solved for us by folder owning list as links •List order still recalculated •Two writes (vs. N for relational) basho
  • 77. Riak: Regroup Features •Already solved for us by folder owning list as links •List order still recalculated •Two writes (vs. N for relational) • No transactions, need extra info for conflict resolution basho
  • 78. Edit Feature basho
  • 79. Edit Feature • Required two representations (text, tree), both stored as BLOBs basho
  • 80. Edit Feature • Required two representations (text, tree), both stored as BLOBs • Application deserializes tree to present web form, recreates text on save basho
  • 81. Composition basho
  • 82. Composition •Sometimes called “denormalization” basho
  • 83. Composition •Sometimes called “denormalization” •Store dependent data in the same object as the parent basho
  • 84. Composition •Sometimes called “denormalization” •Store dependent data in the same object as the parent •JSON, XML, Erlang terms basho
  • 85. Riak: Edit Feature basho
  • 86. Riak: Edit Feature •Already denormalized! basho
  • 87. Riak: Edit Feature •Already denormalized! •Now parse tree can have the same format (JSON, XML), exposing it to other types of processing basho
  • 88. Riak: Edit Feature •Already denormalized! •Now parse tree can have the same format (JSON, XML), exposing it to other types of processing •Individual objects larger basho
  • 89. List Folders basho
  • 90. List Folders • Number of folders is small per project basho
  • 91. List Folders • Number of folders is small per project • Essentially unordered within project basho
  • 92. Riak: List Folders basho
  • 93. Riak: List Folders • Store links to folder objects (or entire list) in project basho
  • 94. Riak: List Folders • Store links to folder objects (or entire list) in project • Simple link-walking or map-reduce to get folders basho
  • 95. Riak: List Folders • Store links to folder objects (or entire list) in project • Simple link-walking or map-reduce to get folders • Two writes required, or move feature links/ order to project basho
  • 96. Riak: List Folders • Store links to folder • Parameterized bucket objects (or entire list) in based on project key project • Simple link-walking or map-reduce to get folders • Two writes required, or move feature links/ order to project basho
  • 97. Riak: List Folders • Store links to folder • Parameterized bucket objects (or entire list) in based on project key project • One write to add folder • Simple link-walking or map-reduce to get folders • Two writes required, or move feature links/ order to project basho
  • 98. Riak: List Folders • Store links to folder • Parameterized bucket objects (or entire list) in based on project key project • One write to add folder • Simple link-walking or map-reduce to get • List keys is expensive folders • Two writes required, or move feature links/ order to project basho
  • 99. Project Statistics basho
  • 100. Project Statistics • Relational design had per-feature counters (maintained by app) basho
  • 101. Project Statistics • Relational design had per-feature counters (maintained by app) • Extra foreign key sped up aggregation by reducing joins basho
  • 102. Riak: Project Statistics basho
  • 103. Riak: Project Statistics •Use scatter-gather: Map-reduce was made for this type of problem! basho
  • 104. Riak: Project Statistics •Use scatter-gather: Map-reduce was made for this type of problem! •Take advantage of now-transparent internal format of feature objects basho
  • 105. Riak: Project Statistics •Use scatter-gather: Map-reduce was made for this type of problem! •Take advantage of now-transparent internal format of feature objects •Query: Follow links to features, map per- feature counters, reduce summarize the counters basho
  • 106. Awkward Fits basho
  • 107. Awkward Fits •User uniqueness constraint basho
  • 108. Awkward Fits •User uniqueness constraint •“double-click” problem basho
  • 109. Awkward Fits •User uniqueness constraint •“double-click” problem •Existing authentication solutions basho
  • 110. Awkward Fits •User uniqueness constraint •“double-click” problem •Existing authentication solutions •Project membership basho
  • 111. Awkward Fits •User uniqueness constraint •“double-click” problem •Existing authentication solutions •Project membership •M-to-N is hard basho
  • 112. Review basho
  • 113. Review •Analyze your relational model basho
  • 114. Review •Analyze your relational model •Identify pain points, take stats basho
  • 115. Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives basho
  • 116. Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives •Test, Measure, Repeat! basho
  • 117. Plug Interested in learning about support, consulting, or Enterprise features?   Email info@basho.com or go to http://www.basho.com/contact.html to talk with us. www.basho.com basho
  • 118. Plug Interested in learning about support, consulting, or Enterprise features?   Email info@basho.com or go to http://www.basho.com/contact.html to talk with us. www.basho.com sean@basho.com @seancribbs basho