Moving to Riak involves a number of changes from the status quo of RDBMS systems, one of which is taking greater control over your schema design.
* How do you structure data when you don't have tables and foreign keys?
* When should you denormalize, add links, or create map-reduce queries?
* Where will Riak be a natural fit and where will it be challenging?
22. Eventual Consistency
•You are already dealing with
inconsistency (e.g. caching, read slave)
basho
23. Eventual Consistency
•You are already dealing with
inconsistency (e.g. caching, read slave)
•Plan for EC instead of triaging
basho
24. Eventual Consistency
•You are already dealing with
inconsistency (e.g. caching, read slave)
•Plan for EC instead of triaging
•Determine resolution processes
basho
25. Eventual Consistency
•You are already dealing with
inconsistency (e.g. caching, read slave)
•Plan for EC instead of triaging
•Determine resolution processes
•Weigh risk of conflicts and emergent
issues
basho
47. projects users
folders
features milestones
project_items
Current Schema
basho
48. projects users
folders
artificial
ordering features milestones
project_items
Current Schema
basho
49. projects users
additional
key for stats
folders
artificial
ordering features milestones
project_items
Current Schema
basho
50. projects users
additional
key for stats
BLOB
folders columns
artificial serialized
ordering features milestones parse tree
formatted
project_items plain text
Current Schema
basho
51. collaborator
projects users
additional
key for stats
BLOB
folders columns
artificial serialized
ordering features milestones parse tree
formatted
project_items plain text
Current Schema
basho
52. collaborator
projects users
additional
key for stats owner
BLOB
folders columns
artificial serialized
ordering features milestones parse tree
formatted
project_items plain text
Current Schema
basho
54. Observations
•Features and milestones are disjunct,
except in list participation
basho
55. Observations
•Features and milestones are disjunct,
except in list participation
•Opacity of serialized BLOB data
basho
56. Observations
•Features and milestones are disjunct,
except in list participation
•Opacity of serialized BLOB data
•Duplication of relationships -
collaborator-owner, project-folder-item
basho
59. List/Reorder Features
• AJAX postback was
already “total” ordering
• List is really a property
of the parent “folder”
basho
60. List/Reorder Features
• AJAX postback was
already “total” ordering
• List is really a property
of the parent “folder”
• Membership, order are
coupled concepts
basho
66. Riak: List/Reorder
Features
•Store list of features and milestones as
links on folder, with index as tag
basho
67. Riak: List/Reorder
Features
•Store list of features and milestones as
links on folder, with index as tag
•Query: Follow links to items, map values
(with index), reduce sort
basho
68. Riak: List/Reorder
Features
•Store list of features and milestones as
links on folder, with index as tag
•Query: Follow links to items, map values
(with index), reduce sort
•One write to order list, easy to link-walk
basho
69. Riak: List/Reorder
Features
•Store list of features and milestones as
links on folder, with index as tag
•Query: Follow links to items, map values
(with index), reduce sort
•One write to order list, easy to link-walk
• Concurrent writes may conflict
basho
74. Riak: Regroup
Features
•Already solved for us by folder owning
list as links
basho
75. Riak: Regroup
Features
•Already solved for us by folder owning
list as links
•List order still recalculated
basho
76. Riak: Regroup
Features
•Already solved for us by folder owning
list as links
•List order still recalculated
•Two writes (vs. N for relational)
basho
77. Riak: Regroup
Features
•Already solved for us by folder owning
list as links
•List order still recalculated
•Two writes (vs. N for relational)
• No transactions, need extra info for
conflict resolution
basho
79. Edit Feature
• Required two
representations (text,
tree), both stored as
BLOBs
basho
80. Edit Feature
• Required two
representations (text,
tree), both stored as
BLOBs
• Application
deserializes tree to
present web form,
recreates text on save
basho
87. Riak: Edit Feature
•Already denormalized!
•Now parse tree can have the same
format (JSON, XML), exposing it to other
types of processing
basho
88. Riak: Edit Feature
•Already denormalized!
•Now parse tree can have the same
format (JSON, XML), exposing it to other
types of processing
•Individual objects larger
basho
93. Riak: List Folders
• Store links to folder
objects (or entire list) in
project
basho
94. Riak: List Folders
• Store links to folder
objects (or entire list) in
project
• Simple link-walking or
map-reduce to get
folders
basho
95. Riak: List Folders
• Store links to folder
objects (or entire list) in
project
• Simple link-walking or
map-reduce to get
folders
• Two writes required, or
move feature links/
order to project
basho
96. Riak: List Folders
• Store links to folder • Parameterized bucket
objects (or entire list) in based on project key
project
• Simple link-walking or
map-reduce to get
folders
• Two writes required, or
move feature links/
order to project
basho
97. Riak: List Folders
• Store links to folder • Parameterized bucket
objects (or entire list) in based on project key
project
• One write to add folder
• Simple link-walking or
map-reduce to get
folders
• Two writes required, or
move feature links/
order to project
basho
98. Riak: List Folders
• Store links to folder • Parameterized bucket
objects (or entire list) in based on project key
project
• One write to add folder
• Simple link-walking or
map-reduce to get • List keys is expensive
folders
• Two writes required, or
move feature links/
order to project
basho
100. Project Statistics
• Relational design had
per-feature counters
(maintained by app)
basho
101. Project Statistics
• Relational design had
per-feature counters
(maintained by app)
• Extra foreign key sped
up aggregation by
reducing joins
basho
103. Riak: Project Statistics
•Use scatter-gather: Map-reduce was
made for this type of problem!
basho
104. Riak: Project Statistics
•Use scatter-gather: Map-reduce was
made for this type of problem!
•Take advantage of now-transparent
internal format of feature objects
basho
105. Riak: Project Statistics
•Use scatter-gather: Map-reduce was
made for this type of problem!
•Take advantage of now-transparent
internal format of feature objects
•Query: Follow links to features, map per-
feature counters, reduce summarize the
counters
basho
114. Review
•Analyze your relational model
•Identify pain points, take stats
basho
115. Review
•Analyze your relational model
•Identify pain points, take stats
•Design some alternatives
basho
116. Review
•Analyze your relational model
•Identify pain points, take stats
•Design some alternatives
•Test, Measure, Repeat!
basho
117. Plug
Interested in learning about support,
consulting, or Enterprise features?
Email info@basho.com or go to
http://www.basho.com/contact.html to talk
with us.
www.basho.com
basho
118. Plug
Interested in learning about support,
consulting, or Enterprise features?
Email info@basho.com or go to
http://www.basho.com/contact.html to talk
with us.
www.basho.com
sean@basho.com
@seancribbs
basho