Your SlideShare is downloading. ×
  • Like
2011 03-31 Riak Stockholm Meetup
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

2011 03-31 Riak Stockholm Meetup

  • 941 views
Published

 

Published in Sports
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
941
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
13
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. OverviewMårten Gustafson Stockholm Riak Meetup #1 2011-03-31
  • 2. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 3. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 4. http://www.flickr.com/photos/linneberg/4481309196/
  • 5. Operations HTTP Client PUT /bucket/key GET /bucket/key DELETE /bucket/key Item ItemItem Item Item Item Item Item Bucket Bucket Bucket Riak Key/Value Store
  • 6. An entry• lives in a bucket• has a key• has a value
  • 7. An entry arbitrary name• lives in a bucket bucket• has a key• has a value
  • 8. An entry arbitrary name• lives in a bucket bucket• has a key arbitrary name key• has a value
  • 9. An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value
  • 10. An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value value a binary blob and mime type
  • 11. An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value value a binary blob and mime type = Store anything, yay!
  • 12. Store anything bucket meetup
  • 13. Store anything bucket meetupkey foo.html mime text/html <html><body> <h1> <a href=”bar.html”>foo</a> </h1> </body></html>
  • 14. Store anything bucket meetupkey foo.html mime text/html key bar.html <html><body> mime text/html <h1> <a href=”bar.html”>foo</a> <html><body> </h1> <h1>bar!</h1> </body></html> </body></html>
  • 15. Store anything bucket meetupkey foo.html mime text/html key bar.html <html><body> mime text/html <h1> <a href=”bar.html”>foo</a> <html><body> </h1> <h1>bar!</h1> </body></html> </body></html> http://127.0.0.1:8088/riak/meetup/foo.html
  • 16. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 17. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 18. StorageRiak Key/Value Store
  • 19. Storage Riak Key/Value StoreBitcask InnoDB DETS File system Balanced trees LRU ETS
  • 20. Storage Riak Key/Value StoreBitcask InnoDB DETS File system Balanced trees LRU ETS disk based, ram based, durable not durable
  • 21. Storagedefault Riak Key/Value Store Bitcask InnoDB DETS File system Balanced trees LRU ETS commo disk based, ram based, durable not durable
  • 22. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 23. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 24. The Ring
  • 25. The Ring
  • 26. The Ring 12 1 11 210 39 4 8 5 7 6 ring size = 12
  • 27. Consistent Hashing 12 1 11 2 10 3 9 4 8 5 7 6
  • 28. Consistent Hashing 12 1 11 2 10 3 9 4 8 5 7 6
  • 29. Read“I want “requires us to know:where is on the ring?
  • 30. Read“I want “requires us to know:where is on the ring?
  • 31. ClusterOne Ring size to rule them all, One Ring size tofind them, One Ring size to bring them all and inthe cluster bind them...
  • 32. Cluster node A node B node Cring size = 12instances = 3ring size / nodes = ~slices per instances
  • 33. Cluster node A node B node Cring size = 12instances = 3ring size / nodes = ~slices per instances
  • 34. Cluster - Readnode A node B node C
  • 35. Cluster - ReadI can haz ? node A node B node CHm, hashes to aslice of the ringowned by node C.
  • 36. Cluster - ReadI can haz ? node A node B node C Okidoki, now Hey C! I need where’s he...a yeah in my fourth slice
  • 37. Cluster - ReadI can haz ? node A node B node C
  • 38. So what about...
  • 39. ...network partitions? node A node B node C
  • 40. ...network partitions? node A node B X node C
  • 41. ...failed nodes?node A node B node C
  • 42. ...failed nodes?node A node B
  • 43. ...concurrent writes? node A node B node C client 1 client 2
  • 44. ...concurrent writes? node A ? node B node C hey, A! save for me client 1 client 2 hey, C! save for me
  • 45. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 46. Livin’ on the web• concurrent users• unreliable networks• node failures
  • 47. The N to the R to the W to the DW and the RW
  • 48. N_VALClient BucketNode A Node B Node C
  • 49. N_VAL 1Client BucketNode A Node B Node C
  • 50. N_VAL 2Client BucketNode A Node B Node C
  • 51. N_VAL 3Client BucketNode A Node B Node C
  • 52. RClient GET /bucket/key?R=1 BucketNode A Node B Node C
  • 53. RClient GET /bucket/key?R=1 BucketNode A Node B Node C
  • 54. RClient GET /bucket/key?R=2 BucketNode A Node B Node C
  • 55. RClient GET /bucket/key?R=2 BucketNode A Node B Node C
  • 56. RClient GET /bucket/key?R=2 Agree? BucketNode A Node B Node C
  • 57. RClient GET /bucket/key?R=2 BucketNode A Node B Node C
  • 58. The N to the R to the W to the DW and the RW
  • 59. The N to the R to the W to the DW and the RWNumber of copiesie. distribute to N nodes
  • 60. Read ie. have R nodes agree The N to the R to the W to the DW and the RWNumber of copiesie. distribute to N nodes
  • 61. Read ie. have R nodes agree The N to the R to the W to the DW and the RW WriteNumber of copies ie. ack’d byie. distribute to N nodes W nodes
  • 62. Durable write Read ie. persistently written by ie. have R nodes agree DW nodes The N to the R to the W to the DW and the RW WriteNumber of copies ie. ack’d byie. distribute to N nodes W nodes
  • 63. Durable write Read ie. persistently written by ie. have R nodes agree DW nodes The N to the R to the W to the DW and the RW Write Read-writeNumber of copies ie. ack’d by ie. persistently deleted byie. distribute to N nodes W nodes RW nodes
  • 64. The Quorum([node count] / 2) + 1
  • 65. Bucket propertieshttp://127.0.0.1:8088/riak/meetup/
  • 66. Conflicts
  • 67. When worlds collide
  • 68. I don’t care! allow_mult = falselast_write_wins = true
  • 69. I do care!allow_mult = true
  • 70. I do care!• Resolve conflicts in application logic• Conflicts exposed as siblings beneath a key• Response is HTTP 300 Multiple Choice• Served as mime/multipart
  • 71. ExampleHTTP/1.1 300 Multiple ChoicesX-Riak-Vclock: a85hYGBgzWDKBVIsrLnh3BlMiYx5rAymfeeO8EGFWRLl30G==Content-Type: multipart/mixed; boundary=ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Length: 368--ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Type: text/plainTuesday--ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Type: text/plainThursday--ZZ3eyjUllBi7GXRRMJsUublFxjn--
  • 72. What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • 73. Map / Reduce• Javascript or Erlang• Exposed in the HTTP API
  • 74. Map / Reduce count words function(v) { var words = v.values[0].data.toLowerCase().match(w*,g); var counts = []; for(var word in words) if (words[word] != ) { var count = {}; count[words[word]] = 1; counts.push(count); } return counts;}
  • 75. Map / Reduce count wordsfunction(values) { var result = {}; for (var value in values) { for(var word in values[value]) { if (word in result) result[word] += values[value][word]; else result[word] = values[value][word]; } } return [result];}
  • 76. Map & Reduce count words Put this in your POST request and let Riak smoke it{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
  • 77. Map & Reduce count wordsfunction(v) { var words = v.values[0].data.toLowerCase().match(w*,g); var counts = []; for(var word in words) if (words[word] != ) { var count = {}; count[words[word]] = 1; counts.push(count); } return counts;}{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
  • 78. Map & Reduce count wordsfunction(values) { var result = {}; for (var value in values) { for(var word in values[value]) { if (word in result) result[word] += values[value][word]; else result[word] = values[value][word]; } } return [result];}{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
  • 79. Map / Reduce Demo
  • 80. Links• Non-enforced• Traversable
  • 81. LinksDemo
  • 82. The whole enchilada HTTP API Riak Search Luwak Riak Key/Value Store Bitcask InnoDB DETS File system Balanced trees LRU ETS Riak Core Membership management Work distribution leave/join Cluster state Partitioning gossip protocol (consistent hashing, hinted handoff) Erlang / OTP
  • 83. Try it downloads.basho.com brew install riakWeb admin @ github.com/gmaurice/Riaktive
  • 84. ResourcesRiak Fast Track @ wiki.basho.com #riak @ freenode github.com/basho/
  • 85. Thanks for listeningMårten Gustafson@martengustafsonhttp://marten.gustafson.pp.se/marten.gustafson@gmail.com