Your SlideShare is downloading. ×
Schema Design for Riak
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Schema Design for Riak

12,358
views

Published on

Moving to Riak involves a number of changes from the status quo of RDBMS systems, one of which is taking greater control over your schema design. …

Moving to Riak involves a number of changes from the status quo of RDBMS systems, one of which is taking greater control over your schema design.

* How do you structure data when you don't have tables and foreign keys?
* When should you denormalize, add links, or create map-reduce queries?
* Where will Riak be a natural fit and where will it be challenging?

Published in: Technology

0 Comments
19 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,358
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
256
Comments
0
Likes
19
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide















































































































  • Transcript

    • 1. Schema Design for Riak Sean Cribbs Developer Advocate basho
    • 2. Thinking Non-Relationally basho
    • 3. “There is no spoon schema.” basho
    • 4. There is an application. basho
    • 5. What you lose basho
    • 6. What you lose •Tables basho
    • 7. What you lose •Tables •Foreign keys and constraints basho
    • 8. What you lose •Tables •Foreign keys and constraints •ACID basho
    • 9. What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners basho
    • 10. What you lose •Tables •Foreign keys and constraints •ACID •Sophisticated query planners •Declarative query language (SQL) basho
    • 11. What you gain basho
    • 12. What you gain •More flexible, fluid designs basho
    • 13. What you gain •More flexible, fluid designs •More natural data representations basho
    • 14. What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain basho
    • 15. What you gain •More flexible, fluid designs •More natural data representations •Scaling without pain •Reduced operational complexity basho
    • 16. Lose the crutches basho
    • 17. Lose the crutches •Sparse data (optional/multi-value fields) basho
    • 18. Lose the crutches •Sparse data (optional/multi-value fields) •Richer data structures basho
    • 19. Lose the crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers basho
    • 20. Lose the crutches •Sparse data (optional/multi-value fields) •Richer data structures •Meaningful identifiers •Innovative access patterns basho
    • 21. Eventual Consistency basho
    • 22. Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) basho
    • 23. Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) •Plan for EC instead of triaging basho
    • 24. Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) •Plan for EC instead of triaging •Determine resolution processes basho
    • 25. Eventual Consistency •You are already dealing with inconsistency (e.g. caching, read slave) •Plan for EC instead of triaging •Determine resolution processes •Weigh risk of conflicts and emergent issues basho
    • 26. Know your data basho
    • 27. Make your Top 10 list basho
    • 28. Make your Top 10 list •Frequently requested pages/screens basho
    • 29. Make your Top 10 list •Frequently requested pages/screens •Slow queries basho
    • 30. Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes basho
    • 31. Make your Top 10 list •Frequently requested pages/screens •Slow queries •Secondary indexes •Complicated joins or aggregations basho
    • 32. Analyze basho
    • 33. Analyze •Interdependencies, coupling basho
    • 34. Analyze •Interdependencies, coupling •Cardinalities of relationships basho
    • 35. Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern basho
    • 36. Analyze •Interdependencies, coupling •Cardinalities of relationships •Access pattern •Shoehorned structures basho
    • 37. Example basho
    • 38. basho
    • 39. basho
    • 40. Top queries basho
    • 41. List Features basho
    • 42. Reorder Features basho
    • 43. Regroup Features basho
    • 44. Edit Feature basho
    • 45. List Folders basho
    • 46. Project Statistics basho
    • 47. projects users folders features milestones project_items Current Schema basho
    • 48. projects users folders artificial ordering features milestones project_items Current Schema basho
    • 49. projects users additional key for stats folders artificial ordering features milestones project_items Current Schema basho
    • 50. projects users additional key for stats BLOB folders columns artificial serialized ordering features milestones parse tree formatted project_items plain text Current Schema basho
    • 51. collaborator projects users additional key for stats BLOB folders columns artificial serialized ordering features milestones parse tree formatted project_items plain text Current Schema basho
    • 52. collaborator projects users additional key for stats owner BLOB folders columns artificial serialized ordering features milestones parse tree formatted project_items plain text Current Schema basho
    • 53. Observations basho
    • 54. Observations •Features and milestones are disjunct, except in list participation basho
    • 55. Observations •Features and milestones are disjunct, except in list participation •Opacity of serialized BLOB data basho
    • 56. Observations •Features and milestones are disjunct, except in list participation •Opacity of serialized BLOB data •Duplication of relationships - collaborator-owner, project-folder-item basho
    • 57. List/Reorder Features basho
    • 58. List/Reorder Features • AJAX postback was already “total” ordering basho
    • 59. List/Reorder Features • AJAX postback was already “total” ordering • List is really a property of the parent “folder” basho
    • 60. List/Reorder Features • AJAX postback was already “total” ordering • List is really a property of the parent “folder” • Membership, order are coupled concepts basho
    • 61. Links in Riak basho
    • 62. Links in Riak •Lightweight pointers, like HTML <link> tags. basho
    • 63. Links in Riak •Lightweight pointers, like HTML <link> tags. •Composed of bucket, key, tag basho
    • 64. Links in Riak •Lightweight pointers, like HTML <link> tags. •Composed of bucket, key, tag •Traversable through “link-walking” basho
    • 65. Riak: List/Reorder Features basho
    • 66. Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag basho
    • 67. Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag •Query: Follow links to items, map values (with index), reduce sort basho
    • 68. Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag •Query: Follow links to items, map values (with index), reduce sort •One write to order list, easy to link-walk basho
    • 69. Riak: List/Reorder Features •Store list of features and milestones as links on folder, with index as tag •Query: Follow links to items, map values (with index), reduce sort •One write to order list, easy to link-walk • Concurrent writes may conflict basho
    • 70. Regroup Features basho
    • 71. Regroup Features • Changing foreign key to folder basho
    • 72. Regroup Features • Changing foreign key to folder • Recalculate order for both lists (multiple UPDATE statements) basho
    • 73. Riak: Regroup Features basho
    • 74. Riak: Regroup Features •Already solved for us by folder owning list as links basho
    • 75. Riak: Regroup Features •Already solved for us by folder owning list as links •List order still recalculated basho
    • 76. Riak: Regroup Features •Already solved for us by folder owning list as links •List order still recalculated •Two writes (vs. N for relational) basho
    • 77. Riak: Regroup Features •Already solved for us by folder owning list as links •List order still recalculated •Two writes (vs. N for relational) • No transactions, need extra info for conflict resolution basho
    • 78. Edit Feature basho
    • 79. Edit Feature • Required two representations (text, tree), both stored as BLOBs basho
    • 80. Edit Feature • Required two representations (text, tree), both stored as BLOBs • Application deserializes tree to present web form, recreates text on save basho
    • 81. Composition basho
    • 82. Composition •Sometimes called “denormalization” basho
    • 83. Composition •Sometimes called “denormalization” •Store dependent data in the same object as the parent basho
    • 84. Composition •Sometimes called “denormalization” •Store dependent data in the same object as the parent •JSON, XML, Erlang terms basho
    • 85. Riak: Edit Feature basho
    • 86. Riak: Edit Feature •Already denormalized! basho
    • 87. Riak: Edit Feature •Already denormalized! •Now parse tree can have the same format (JSON, XML), exposing it to other types of processing basho
    • 88. Riak: Edit Feature •Already denormalized! •Now parse tree can have the same format (JSON, XML), exposing it to other types of processing •Individual objects larger basho
    • 89. List Folders basho
    • 90. List Folders • Number of folders is small per project basho
    • 91. List Folders • Number of folders is small per project • Essentially unordered within project basho
    • 92. Riak: List Folders basho
    • 93. Riak: List Folders • Store links to folder objects (or entire list) in project basho
    • 94. Riak: List Folders • Store links to folder objects (or entire list) in project • Simple link-walking or map-reduce to get folders basho
    • 95. Riak: List Folders • Store links to folder objects (or entire list) in project • Simple link-walking or map-reduce to get folders • Two writes required, or move feature links/ order to project basho
    • 96. Riak: List Folders • Store links to folder • Parameterized bucket objects (or entire list) in based on project key project • Simple link-walking or map-reduce to get folders • Two writes required, or move feature links/ order to project basho
    • 97. Riak: List Folders • Store links to folder • Parameterized bucket objects (or entire list) in based on project key project • One write to add folder • Simple link-walking or map-reduce to get folders • Two writes required, or move feature links/ order to project basho
    • 98. Riak: List Folders • Store links to folder • Parameterized bucket objects (or entire list) in based on project key project • One write to add folder • Simple link-walking or map-reduce to get • List keys is expensive folders • Two writes required, or move feature links/ order to project basho
    • 99. Project Statistics basho
    • 100. Project Statistics • Relational design had per-feature counters (maintained by app) basho
    • 101. Project Statistics • Relational design had per-feature counters (maintained by app) • Extra foreign key sped up aggregation by reducing joins basho
    • 102. Riak: Project Statistics basho
    • 103. Riak: Project Statistics •Use scatter-gather: Map-reduce was made for this type of problem! basho
    • 104. Riak: Project Statistics •Use scatter-gather: Map-reduce was made for this type of problem! •Take advantage of now-transparent internal format of feature objects basho
    • 105. Riak: Project Statistics •Use scatter-gather: Map-reduce was made for this type of problem! •Take advantage of now-transparent internal format of feature objects •Query: Follow links to features, map per- feature counters, reduce summarize the counters basho
    • 106. Awkward Fits basho
    • 107. Awkward Fits •User uniqueness constraint basho
    • 108. Awkward Fits •User uniqueness constraint •“double-click” problem basho
    • 109. Awkward Fits •User uniqueness constraint •“double-click” problem •Existing authentication solutions basho
    • 110. Awkward Fits •User uniqueness constraint •“double-click” problem •Existing authentication solutions •Project membership basho
    • 111. Awkward Fits •User uniqueness constraint •“double-click” problem •Existing authentication solutions •Project membership •M-to-N is hard basho
    • 112. Review basho
    • 113. Review •Analyze your relational model basho
    • 114. Review •Analyze your relational model •Identify pain points, take stats basho
    • 115. Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives basho
    • 116. Review •Analyze your relational model •Identify pain points, take stats •Design some alternatives •Test, Measure, Repeat! basho
    • 117. Plug Interested in learning about support, consulting, or Enterprise features?   Email info@basho.com or go to http://www.basho.com/contact.html to talk with us. www.basho.com basho
    • 118. Plug Interested in learning about support, consulting, or Enterprise features?   Email info@basho.com or go to http://www.basho.com/contact.html to talk with us. www.basho.com sean@basho.com @seancribbs basho