• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
2011 03-31 Riak Stockholm Meetup
 

2011 03-31 Riak Stockholm Meetup

on

  • 1,051 views

 

Statistics

Views

Total Views
1,051
Views on SlideShare
678
Embed Views
373

Actions

Likes
1
Downloads
9
Comments
0

2 Embeds 373

http://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com 358
https://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com 15

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    2011 03-31 Riak Stockholm Meetup 2011 03-31 Riak Stockholm Meetup Presentation Transcript

    • OverviewMårten Gustafson Stockholm Riak Meetup #1 2011-03-31
    • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
    • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
    • http://www.flickr.com/photos/linneberg/4481309196/
    • Operations HTTP Client PUT /bucket/key GET /bucket/key DELETE /bucket/key Item ItemItem Item Item Item Item Item Bucket Bucket Bucket Riak Key/Value Store
    • An entry• lives in a bucket• has a key• has a value
    • An entry arbitrary name• lives in a bucket bucket• has a key• has a value
    • An entry arbitrary name• lives in a bucket bucket• has a key arbitrary name key• has a value
    • An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value
    • An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value value a binary blob and mime type
    • An entry arbitrary name forms the path• lives in a bucket bucket to the value• has a key arbitrary name key• has a value value a binary blob and mime type = Store anything, yay!
    • Store anything bucket meetup
    • Store anything bucket meetupkey foo.html mime text/html <html><body> <h1> <a href=”bar.html”>foo</a> </h1> </body></html>
    • Store anything bucket meetupkey foo.html mime text/html key bar.html <html><body> mime text/html <h1> <a href=”bar.html”>foo</a> <html><body> </h1> <h1>bar!</h1> </body></html> </body></html>
    • Store anything bucket meetupkey foo.html mime text/html key bar.html <html><body> mime text/html <h1> <a href=”bar.html”>foo</a> <html><body> </h1> <h1>bar!</h1> </body></html> </body></html> http://127.0.0.1:8088/riak/meetup/foo.html
    • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
    • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
    • StorageRiak Key/Value Store
    • Storage Riak Key/Value StoreBitcask InnoDB DETS File system Balanced trees LRU ETS
    • Storage Riak Key/Value StoreBitcask InnoDB DETS File system Balanced trees LRU ETS disk based, ram based, durable not durable
    • Storagedefault Riak Key/Value Store Bitcask InnoDB DETS File system Balanced trees LRU ETS commo disk based, ram based, durable not durable
    • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
    • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
    • The Ring
    • The Ring
    • The Ring 12 1 11 210 39 4 8 5 7 6 ring size = 12
    • Consistent Hashing 12 1 11 2 10 3 9 4 8 5 7 6
    • Consistent Hashing 12 1 11 2 10 3 9 4 8 5 7 6
    • Read“I want “requires us to know:where is on the ring?
    • Read“I want “requires us to know:where is on the ring?
    • ClusterOne Ring size to rule them all, One Ring size tofind them, One Ring size to bring them all and inthe cluster bind them...
    • Cluster node A node B node Cring size = 12instances = 3ring size / nodes = ~slices per instances
    • Cluster node A node B node Cring size = 12instances = 3ring size / nodes = ~slices per instances
    • Cluster - Readnode A node B node C
    • Cluster - ReadI can haz ? node A node B node CHm, hashes to aslice of the ringowned by node C.
    • Cluster - ReadI can haz ? node A node B node C Okidoki, now Hey C! I need where’s he...a yeah in my fourth slice
    • Cluster - ReadI can haz ? node A node B node C
    • So what about...
    • ...network partitions? node A node B node C
    • ...network partitions? node A node B X node C
    • ...failed nodes?node A node B node C
    • ...failed nodes?node A node B
    • ...concurrent writes? node A node B node C client 1 client 2
    • ...concurrent writes? node A ? node B node C hey, A! save for me client 1 client 2 hey, C! save for me
    • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
    • Livin’ on the web• concurrent users• unreliable networks• node failures
    • The N to the R to the W to the DW and the RW
    • N_VALClient BucketNode A Node B Node C
    • N_VAL 1Client BucketNode A Node B Node C
    • N_VAL 2Client BucketNode A Node B Node C
    • N_VAL 3Client BucketNode A Node B Node C
    • RClient GET /bucket/key?R=1 BucketNode A Node B Node C
    • RClient GET /bucket/key?R=1 BucketNode A Node B Node C
    • RClient GET /bucket/key?R=2 BucketNode A Node B Node C
    • RClient GET /bucket/key?R=2 BucketNode A Node B Node C
    • RClient GET /bucket/key?R=2 Agree? BucketNode A Node B Node C
    • RClient GET /bucket/key?R=2 BucketNode A Node B Node C
    • The N to the R to the W to the DW and the RW
    • The N to the R to the W to the DW and the RWNumber of copiesie. distribute to N nodes
    • Read ie. have R nodes agree The N to the R to the W to the DW and the RWNumber of copiesie. distribute to N nodes
    • Read ie. have R nodes agree The N to the R to the W to the DW and the RW WriteNumber of copies ie. ack’d byie. distribute to N nodes W nodes
    • Durable write Read ie. persistently written by ie. have R nodes agree DW nodes The N to the R to the W to the DW and the RW WriteNumber of copies ie. ack’d byie. distribute to N nodes W nodes
    • Durable write Read ie. persistently written by ie. have R nodes agree DW nodes The N to the R to the W to the DW and the RW Write Read-writeNumber of copies ie. ack’d by ie. persistently deleted byie. distribute to N nodes W nodes RW nodes
    • The Quorum([node count] / 2) + 1
    • Bucket propertieshttp://127.0.0.1:8088/riak/meetup/
    • Conflicts
    • When worlds collide
    • I don’t care! allow_mult = falselast_write_wins = true
    • I do care!allow_mult = true
    • I do care!• Resolve conflicts in application logic• Conflicts exposed as siblings beneath a key• Response is HTTP 300 Multiple Choice• Served as mime/multipart
    • ExampleHTTP/1.1 300 Multiple ChoicesX-Riak-Vclock: a85hYGBgzWDKBVIsrLnh3BlMiYx5rAymfeeO8EGFWRLl30G==Content-Type: multipart/mixed; boundary=ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Length: 368--ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Type: text/plainTuesday--ZZ3eyjUllBi7GXRRMJsUublFxjnContent-Type: text/plainThursday--ZZ3eyjUllBi7GXRRMJsUublFxjn--
    • What is Riak and what’s the agenda?Decentralized key-value storeA database ideally suited for web applicationsA flexible map/reduce engine
    • Map / Reduce• Javascript or Erlang• Exposed in the HTTP API
    • Map / Reduce count words function(v) { var words = v.values[0].data.toLowerCase().match(w*,g); var counts = []; for(var word in words) if (words[word] != ) { var count = {}; count[words[word]] = 1; counts.push(count); } return counts;}
    • Map / Reduce count wordsfunction(values) { var result = {}; for (var value in values) { for(var word in values[value]) { if (word in result) result[word] += values[value][word]; else result[word] = values[value][word]; } } return [result];}
    • Map & Reduce count words Put this in your POST request and let Riak smoke it{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
    • Map & Reduce count wordsfunction(v) { var words = v.values[0].data.toLowerCase().match(w*,g); var counts = []; for(var word in words) if (words[word] != ) { var count = {}; count[words[word]] = 1; counts.push(count); } return counts;}{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
    • Map & Reduce count wordsfunction(values) { var result = {}; for (var value in values) { for(var word in values[value]) { if (word in result) result[word] += values[value][word]; else result[word] = values[value][word]; } } return [result];}{"inputs":"bucket", "query":[{"map":{"language":"javascript","source":"function(v) { var words = v.values[0].data.toLowerCase().match(/w*/g); var counts = []; for(var word in words) if (words[word] != ){ var count = {}; count[words[word]] = 1; counts.push(count); } returncounts; }"}},{"reduce":{"language":"javascript", "source":"function(values){ var result = {}; for (var value in values) { for(var word invalues[value]) { if (word in result) result[word] += values[value][word];else result[word] = values[value][word]; } } return [result]; }"}}]}
    • Map / Reduce Demo
    • Links• Non-enforced• Traversable
    • LinksDemo
    • The whole enchilada HTTP API Riak Search Luwak Riak Key/Value Store Bitcask InnoDB DETS File system Balanced trees LRU ETS Riak Core Membership management Work distribution leave/join Cluster state Partitioning gossip protocol (consistent hashing, hinted handoff) Erlang / OTP
    • Try it downloads.basho.com brew install riakWeb admin @ github.com/gmaurice/Riaktive
    • ResourcesRiak Fast Track @ wiki.basho.com #riak @ freenode github.com/basho/
    • Thanks for listeningMårten Gustafson@martengustafsonhttp://marten.gustafson.pp.se/marten.gustafson@gmail.com