2011 03-31 Riak Stockholm Meetup
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,136
On Slideshare
763
From Embeds
373
Number of Embeds
2

Actions

Shares
Downloads
13
Comments
0
Likes
1

Embeds 373

http://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com 358
https://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com 15

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. OverviewMårten Gustafson Stockholm Riak Meetup #1 2011-03-31
  • 2. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 3. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 4. http://www.flickr.com/photos/linneberg/4481309196/
  • 5. Operations HTTP Client PUT /bucket/key GET /bucket/key DELETE /bucket/key Item ItemItem Item Item Item Item Item Bucket Bucket Bucket Riak Key/Value Store
  • 6. An entry• lives in a bucket• has a key• has a value
  • 7. An entry arbitrary name• lives in a bucket bucket• has a key• has a value
  • 8. An entry arbitrary name• lives in a bucket bucket• has a key arbitrary name key• has a value
  • 9. An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value
  • 10. An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value value a binary blob and mime type
  • 11. An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value value a binary blob and mime type = Store anything, yay!
  • 12. Store anything bucket meetup
  • 13. Store anything bucket meetupkey foo.html mime text/html <html><body> <h1> <a href=”bar.html”>foo</a> </h1> </body></html>
  • 14. Store anything bucket meetupkey foo.html mime text/html key bar.html <html><body> mime text/html <h1> <a href=”bar.html”>foo</a> <html><body> </h1> <h1>bar!</h1> </body></html> </body></html>
  • 15. Store anything bucket meetupkey foo.html mime text/html key bar.html <html><body> mime text/html <h1> <a href=”bar.html”>foo</a> <html><body> </h1> <h1>bar!</h1> </body></html> </body></html> http://127.0.0.1:8088/riak/meetup/foo.html
  • 16. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 17. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 18. StorageRiak Key/Value Store
  • 19. Storage Riak Key/Value StoreBitcask InnoDB DETS File system Balanced trees LRU ETS
  • 20. Storage Riak Key/Value StoreBitcask InnoDB DETS File system Balanced trees LRU ETS disk based, ram based, durable not durable
  • 21. Storagedefault Riak Key/Value Store Bitcask InnoDB DETS File system Balanced trees LRU ETS commo disk based, ram based, durable not durable
  • 22. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 23. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 24. The Ring
  • 25. The Ring
  • 26. The Ring 12 1 11 210 39 4 8 5 7 6 ring size = 12
  • 27. Consistent Hashing 12 1 11 2 10 3 9 4 8 5 7 6
  • 28. Consistent Hashing 12 1 11 2 10 3 9 4 8 5 7 6
  • 29. Read“I want “requires us to know:where is on the ring?
  • 30. Read“I want “requires us to know:where is on the ring?
  • 31. ClusterOne Ring size to rule them all, One Ring size tofind them, One Ring size to bring them all and inthe cluster bind them...
  • 32. Cluster node A node B node Cring size = 12instances = 3ring size / nodes = ~slices per instances
  • 33. Cluster node A node B node Cring size = 12instances = 3ring size / nodes = ~slices per instances
  • 34. Cluster - Readnode A node B node C
  • 35. Cluster - ReadI can haz ? node A node B node CHm, hashes to aslice of the ringowned by node C.
  • 36. Cluster - ReadI can haz ? node A node B node C Okidoki, now Hey C! I need where’s he...a yeah in my fourth slice
  • 37. Cluster - ReadI can haz ? node A node B node C
  • 38. So what about...
  • 39. ...network partitions? node A node B node C
  • 40. ...network partitions? node A node B X node C
  • 41. ...failed nodes?node A node B node C
  • 42. ...failed nodes?node A node B
  • 43. ...concurrent writes? node A node B node C client 1 client 2
  • 44. ...concurrent writes? node A ? node B node C hey, A! save for me client 1 client 2 hey, C! save for me
  • 45. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 46. Livin’ on the web• concurrent users• unreliable networks• node failures
  • 47. The N to the R to the W to the DW and the RW
  • 48. N_VALClient BucketNode A Node B Node C
  • 49. N_VAL 1Client BucketNode A Node B Node C
  • 50. N_VAL 2Client BucketNode A Node B Node C
  • 51. N_VAL 3Client BucketNode A Node B Node C
  • 52. RClient GET /bucket/key?R=1 BucketNode A Node B Node C
  • 53. RClient GET /bucket/key?R=1 BucketNode A Node B Node C
  • 54. RClient GET /bucket/key?R=2 BucketNode A Node B Node C
  • 55. RClient GET /bucket/key?R=2 BucketNode A Node B Node C
  • 56. RClient GET /bucket/key?R=2 Agree? BucketNode A Node B Node C
  • 57. RClient GET /bucket/key?R=2 BucketNode A Node B Node C
  • 58. The N to the R to the W to the DW and the RW
  • 59. The N to the R to the W to the DW and the RWNumber of copiesie. distribute to N nodes
  • 60. Read ie. have R nodes agree The N to the R to the W to the DW and the RWNumber of copiesie. distribute to N nodes
  • 61. Read ie. have R nodes agree The N to the R to the W to the DW and the RW WriteNumber of copies ie. ack’d byie. distribute to N nodes W nodes
  • 62. Durable write Read ie. persistently written by ie. have R nodes agree DW nodes The N to the R to the W to the DW and the RW WriteNumber of copies ie. ack’d byie. distribute to N nodes W nodes
  • 63. Durable write Read ie. persistently written by ie. have R nodes agree DW nodes The N to the R to the W to the DW and the RW Write Read-writeNumber of copies ie. ack’d by ie. persistently deleted byie. distribute to N nodes W nodes RW nodes
  • 64. The Quorum([node count] / 2) + 1
  • 65. Bucket propertieshttp://127.0.0.1:8088/riak/meetup/
  • 66. Conflicts
  • 67. When worlds collide
  • 68. I don’t care! allow_mult = falselast_write_wins = true
  • 69. I do care!allow_mult = true
  • 70. I do care!• Resolve conflicts in application logic• Conflicts exposed as siblings beneath a key• Response is HTTP 300 Multiple Choice• Served as mime/multipart
  • 71. ExampleHTTP/1.1 300 Multiple ChoicesX-Riak-Vclock: a85hYGBgzWDKBVIsrLnh3BlMiYx5rAymfeeO8EGFWRLl30G==Content-Type: multipart/mixed; boundary=ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Length: 368--ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Type: text/plainTuesday--ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Type: text/plainThursday--ZZ3eyjUllBi7GXRRMJsUublFxjn--
  • 72. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 73. Map / Reduce• Javascript or Erlang• Exposed in the HTTP API
  • 74. Map / Reduce count words function(v) { var words = v.values[0].data.toLowerCase().match(w*,g); var counts = []; for(var word in words) if (words[word] != ) { var count = {}; count[words[word]] = 1; counts.push(count); } return counts;}
  • 75. Map / Reduce count wordsfunction(values) { var result = {}; for (var value in values) { for(var word in values[value]) { if (word in result) result[word] += values[value][word]; else result[word] = values[value][word]; } } return [result];}
  • 76. Map & Reduce count words Put this in your POST request and let Riak smoke it{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
  • 77. Map & Reduce count wordsfunction(v) { var words = v.values[0].data.toLowerCase().match(w*,g); var counts = []; for(var word in words) if (words[word] != ) { var count = {}; count[words[word]] = 1; counts.push(count); } return counts;}{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
  • 78. Map & Reduce count wordsfunction(values) { var result = {}; for (var value in values) { for(var word in values[value]) { if (word in result) result[word] += values[value][word]; else result[word] = values[value][word]; } } return [result];}{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
  • 79. Map / Reduce Demo
  • 80. Links• Non-enforced• Traversable
  • 81. LinksDemo
  • 82. The whole enchilada HTTP API Riak Search Luwak Riak Key/Value Store Bitcask InnoDB DETS File system Balanced trees LRU ETS Riak Core Membership management Work distribution leave/join Cluster state Partitioning gossip protocol (consistent hashing, hinted handoff) Erlang / OTP
  • 83. Try it downloads.basho.com brew install riakWeb admin @ github.com/gmaurice/Riaktive
  • 84. ResourcesRiak Fast Track @ wiki.basho.com #riak @ freenode github.com/basho/
  • 85. Thanks for listeningMårten Gustafson@martengustafsonhttp://marten.gustafson.pp.se/marten.gustafson@gmail.com