2011 03-31 Riak Stockholm Meetup
Upcoming SlideShare
Loading in...5
×
 

2011 03-31 Riak Stockholm Meetup

on

  • 1,078 views

 

Statistics

Views

Total Views
1,078
Views on SlideShare
705
Embed Views
373

Actions

Likes
1
Downloads
9
Comments
0

2 Embeds 373

http://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com 358
https://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com 15

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

2011 03-31 Riak Stockholm Meetup 2011 03-31 Riak Stockholm Meetup Presentation Transcript

  • OverviewMårten Gustafson Stockholm Riak Meetup #1 2011-03-31
  • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • http://www.flickr.com/photos/linneberg/4481309196/
  • Operations HTTP Client PUT /bucket/key GET /bucket/key DELETE /bucket/key Item ItemItem Item Item Item Item Item Bucket Bucket Bucket Riak Key/Value Store
  • An entry• lives in a bucket• has a key• has a value
  • An entry arbitrary name• lives in a bucket bucket• has a key• has a value
  • An entry arbitrary name• lives in a bucket bucket• has a key arbitrary name key• has a value
  • An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value
  • An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value value a binary blob and mime type
  • An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value value a binary blob and mime type = Store anything, yay!
  • Store anything bucket meetup
  • Store anything bucket meetupkey foo.html mime text/html <html><body> <h1> <a href=”bar.html”>foo</a> </h1> </body></html>
  • Store anything bucket meetupkey foo.html mime text/html key bar.html <html><body> mime text/html <h1> <a href=”bar.html”>foo</a> <html><body> </h1> <h1>bar!</h1> </body></html> </body></html>
  • Store anything bucket meetupkey foo.html mime text/html key bar.html <html><body> mime text/html <h1> <a href=”bar.html”>foo</a> <html><body> </h1> <h1>bar!</h1> </body></html> </body></html> http://127.0.0.1:8088/riak/meetup/foo.html
  • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • StorageRiak Key/Value Store
  • Storage Riak Key/Value StoreBitcask InnoDB DETS File system Balanced trees LRU ETS
  • Storage Riak Key/Value StoreBitcask InnoDB DETS File system Balanced trees LRU ETS disk based, ram based, durable not durable
  • Storagedefault Riak Key/Value Store Bitcask InnoDB DETS File system Balanced trees LRU ETS commo disk based, ram based, durable not durable
  • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • The Ring
  • The Ring
  • The Ring 12 1 11 210 39 4 8 5 7 6 ring size = 12
  • Consistent Hashing 12 1 11 2 10 3 9 4 8 5 7 6
  • Consistent Hashing 12 1 11 2 10 3 9 4 8 5 7 6
  • Read“I want “requires us to know:where is on the ring?
  • Read“I want “requires us to know:where is on the ring?
  • ClusterOne Ring size to rule them all, One Ring size tofind them, One Ring size to bring them all and inthe cluster bind them...
  • Cluster node A node B node Cring size = 12instances = 3ring size / nodes = ~slices per instances
  • Cluster node A node B node Cring size = 12instances = 3ring size / nodes = ~slices per instances
  • Cluster - Readnode A node B node C
  • Cluster - ReadI can haz ? node A node B node CHm, hashes to aslice of the ringowned by node C.
  • Cluster - ReadI can haz ? node A node B node C Okidoki, now Hey C! I need where’s he...a yeah in my fourth slice
  • Cluster - ReadI can haz ? node A node B node C
  • So what about...
  • ...network partitions? node A node B node C
  • ...network partitions? node A node B X node C
  • ...failed nodes?node A node B node C
  • ...failed nodes?node A node B
  • ...concurrent writes? node A node B node C client 1 client 2
  • ...concurrent writes? node A ? node B node C hey, A! save for me client 1 client 2 hey, C! save for me
  • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • Livin’ on the web• concurrent users• unreliable networks• node failures
  • The N to the R to the W to the DW and the RW
  • N_VALClient BucketNode A Node B Node C
  • N_VAL 1Client BucketNode A Node B Node C
  • N_VAL 2Client BucketNode A Node B Node C
  • N_VAL 3Client BucketNode A Node B Node C
  • RClient GET /bucket/key?R=1 BucketNode A Node B Node C
  • RClient GET /bucket/key?R=1 BucketNode A Node B Node C
  • RClient GET /bucket/key?R=2 BucketNode A Node B Node C
  • RClient GET /bucket/key?R=2 BucketNode A Node B Node C
  • RClient GET /bucket/key?R=2 Agree? BucketNode A Node B Node C
  • RClient GET /bucket/key?R=2 BucketNode A Node B Node C
  • The N to the R to the W to the DW and the RW
  • The N to the R to the W to the DW and the RWNumber of copiesie. distribute to N nodes
  • Read ie. have R nodes agree The N to the R to the W to the DW and the RWNumber of copiesie. distribute to N nodes
  • Read ie. have R nodes agree The N to the R to the W to the DW and the RW WriteNumber of copies ie. ack’d byie. distribute to N nodes W nodes
  • Durable write Read ie. persistently written by ie. have R nodes agree DW nodes The N to the R to the W to the DW and the RW WriteNumber of copies ie. ack’d byie. distribute to N nodes W nodes
  • Durable write Read ie. persistently written by ie. have R nodes agree DW nodes The N to the R to the W to the DW and the RW Write Read-writeNumber of copies ie. ack’d by ie. persistently deleted byie. distribute to N nodes W nodes RW nodes
  • The Quorum([node count] / 2) + 1
  • Bucket propertieshttp://127.0.0.1:8088/riak/meetup/
  • Conflicts
  • When worlds collide
  • I don’t care! allow_mult = falselast_write_wins = true
  • I do care!allow_mult = true
  • I do care!• Resolve conflicts in application logic• Conflicts exposed as siblings beneath a key• Response is HTTP 300 Multiple Choice• Served as mime/multipart
  • ExampleHTTP/1.1 300 Multiple ChoicesX-Riak-Vclock: a85hYGBgzWDKBVIsrLnh3BlMiYx5rAymfeeO8EGFWRLl30G==Content-Type: multipart/mixed; boundary=ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Length: 368--ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Type: text/plainTuesday--ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Type: text/plainThursday--ZZ3eyjUllBi7GXRRMJsUublFxjn--
  • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
  • Map / Reduce• Javascript or Erlang• Exposed in the HTTP API
  • Map / Reduce count words function(v) { var words = v.values[0].data.toLowerCase().match(w*,g); var counts = []; for(var word in words) if (words[word] != ) { var count = {}; count[words[word]] = 1; counts.push(count); } return counts;}
  • Map / Reduce count wordsfunction(values) { var result = {}; for (var value in values) { for(var word in values[value]) { if (word in result) result[word] += values[value][word]; else result[word] = values[value][word]; } } return [result];}
  • Map & Reduce count words Put this in your POST request and let Riak smoke it{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
  • Map & Reduce count wordsfunction(v) { var words = v.values[0].data.toLowerCase().match(w*,g); var counts = []; for(var word in words) if (words[word] != ) { var count = {}; count[words[word]] = 1; counts.push(count); } return counts;}{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
  • Map & Reduce count wordsfunction(values) { var result = {}; for (var value in values) { for(var word in values[value]) { if (word in result) result[word] += values[value][word]; else result[word] = values[value][word]; } } return [result];}{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
  • Map / Reduce Demo
  • Links• Non-enforced• Traversable
  • LinksDemo
  • The whole enchilada HTTP API Riak Search Luwak Riak Key/Value Store Bitcask InnoDB DETS File system Balanced trees LRU ETS Riak Core Membership management Work distribution leave/join Cluster state Partitioning gossip protocol (consistent hashing, hinted handoff) Erlang / OTP
  • Try it downloads.basho.com brew install riakWeb admin @ github.com/gmaurice/Riaktive
  • ResourcesRiak Fast Track @ wiki.basho.com #riak @ freenode github.com/basho/
  • Thanks for listeningMårten Gustafson@martengustafsonhttp://marten.gustafson.pp.se/marten.gustafson@gmail.com