Riak in Ten Minutes
Upcoming SlideShare
Loading in...5
×
 

Riak in Ten Minutes

on

  • 9,164 views

Here's my presentation from the NoSQL vendor track at GlueCon.

Here's my presentation from the NoSQL vendor track at GlueCon.

Statistics

Views

Total Views
9,164
Slideshare-icon Views on SlideShare
7,725
Embed Views
1,439

Actions

Likes
8
Downloads
153
Comments
1

3 Embeds 1,439

http://nosql.mypopescu.com 1353
http://www.slideshare.net 82
http://webcache.googleusercontent.com 4

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • This rules
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • Inspired by the paper Amazon wrote about developing their Dynamo system. <br />
  • One common encoding is JSON, but you can use what you&#x2019;d like. <br />
  • <br />
  • Applications can use the content type to decide how to decode the value. <br /> We call the combination of key/bucket/value/metadata a Riak Object. <br /> One of the things we use the metadata for is to make a nice restful HTTP interface. <br /> <br />
  • Here&#x2019;s an example - if we stored an HTML fragment under they key greeting, in the bucket gluecon <br /> <br />
  • The additional metadata lets us generate a nice HTTP response. <br /> Riak plays very well with the web. <br />
  • The additional metadata lets us generate a nice HTTP response. <br /> Riak plays very well with the web. <br />
  • Riak is a decentralized key/value store. <br /> It uses distributed hashing to spread the data across all available machines. <br /> All nodes are the same and we have no single points of failure. <br />
  • From an operations point of view, adding a node is as simple as configuring the base operating system, <br /> installing riak and joining the new node to the cluster. <br />
  • There are no replication topologies to update. <br /> Data is automatically migrated to new nodes. <br />
  • Add more nodes for extra capacity or lowering latency. <br />
  • <br />
  • And when your spike is over, Riak scales back down again. <br />
  • Once you have multiple nodes, you get reliability. <br /> Riak can store multiple copies of your data. <br /> get/put/delete operations request each of the replicas and make sure they are all up to date. <br />
  • The N value stores how many replicas of each key are stored. <br /> As nodes may come and go, there is a chance of the cluster containing stale data or conflicts. <br /> Riak handles this by versioning objects with vector clocks and requesting all N objects. <br /> Using the vector clock we can tell if the data is just stale or in conflict. <br /> Riak can handle this with a last timestamp wins strategy or provide the conflicts back to the application. <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • Dealing with clusters increases your chance of something failing. <br /> In Riak when a node becomes unavailable, another node takes over for it. <br />
  • <br />
  • If the failure is temporary and the node comes back, the data it missed is returned to it. <br /> Or it can be removed from the cluster and the other nodes will take over for it permanently. <br />
  • If the failure is temporary and the node comes back, the data it missed is returned to it. <br /> Or it can be removed from the cluster and the other nodes will take over for it permanently. <br />
  • The get/put/delete operations in Riak access data by primary key. <br /> Most applications need more. <br /> Riak has an implementation of map/reduce that allows you to visit each key in a bucket <br /> in a map phase extract any information you&#x2019;d like from it (possibly none) and then combine the results <br /> in a reduce phase. <br />
  • map and reduce functions can be written in Javascript or Erlang. <br /> The work of mapping is distributed to all nodes in the cluster. <br /> The results of the map are streamed to one node for the reduce(). <br />
  • map and reduce functions can be written in Javascript or Erlang. <br /> The work of mapping is distributed to all nodes in the cluster. <br /> The results of the map are streamed to one node for the reduce(). <br />
  • map and reduce functions can be written in Javascript or Erlang. <br /> The work of mapping is distributed to all nodes in the cluster. <br /> The results of the map are streamed to one node for the reduce(). <br />
  • map and reduce functions can be written in Javascript or Erlang. <br /> The work of mapping is distributed to all nodes in the cluster. <br /> The results of the map are streamed to one node for the reduce(). <br />
  • map and reduce functions can be written in Javascript or Erlang. <br /> The work of mapping is distributed to all nodes in the cluster. <br /> The results of the map are streamed to one node for the reduce(). <br />
  • map and reduce functions can be written in Javascript or Erlang. <br /> The work of mapping is distributed to all nodes in the cluster. <br /> The results of the map are streamed to one node for the reduce(). <br />
  • map and reduce functions can be written in Javascript or Erlang. <br /> The work of mapping is distributed to all nodes in the cluster. <br /> The results of the map are streamed to one node for the reduce(). <br />
  • map and reduce functions can be written in Javascript or Erlang. <br /> The work of mapping is distributed to all nodes in the cluster. <br /> The results of the map are streamed to one node for the reduce(). <br />
  • map and reduce functions can be written in Javascript or Erlang. <br /> The work of mapping is distributed to all nodes in the cluster. <br /> The results of the map are streamed to one node for the reduce(). <br />
  • <br />
  • <br />
  • <br />
  • As they say, no riak object is an island. Application care about relationships in their data. <br /> Riak can store one way &#x2018;links&#x2019; in the object metadata, pointing to another bucket/key. <br /> Each link has a tag and there can be many objects. <br /> Riak can combine map/reduce with object metadata to handle relationships between them and we have a special link walking interface included in our HTTP interace. <br /> <br />
  • Here&#x2019;s an example using biological relationships. <br /> The first set of links store children. <br />
  • but links are one way, so if you want to get back you have to add more links. <br />
  • and more links - however it makes sense to use them <br />
  • We expose links through map/reduce <br /> and directly over our HTTP interface <br /> We can <br />
  • <br />
  • You can also return intermediate steps - after the trailing comma for the observant of you. <br />
  • Riak has clients in many languages that are supported by the Basho team and there are also <br /> community provided ones springing up too. <br />
  • That&#x2019;s about all I could show you in 10 minutes - please visit our website and check us out. <br /> We have a FastTrack section on our wiki to go through all you need to get started. <br /> Thank you for listening. <br />

Riak in Ten Minutes Riak in Ten Minutes Presentation Transcript

  • Jon Meredith Basho Technologies
  • Riak is a scalable, highly-available, networked key/value store. basho
  • Inspired by Amazon’s Dynamo ...with some enhancements. Open Source basho
  • • Riak stores values against keys. • Encode your data how you like it. • Keys are organized into buckets. basho
  • BASIC OPERATIONS get put delete basho
  • • Riak also stores metadata against keys • Content Type, Charset, Encoding, ... • Custom application metadata basho
  • bucket: gluecon key: greeting value: “<h1>Hi GlueCon<h1>” metadata: content-type=text/html charset=utf8 basho
  • > PUT /riak/gluecon/greeting HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: 127.0.0.1:8098 > Accept: */* > content-type: text/html > charset: utf8 > Content-Length: 19 > > <h1>Hi GlueCon<h1> < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgzWDKBVIsTNPiOTOYEhnzWBm+5uUf5YMIszUnMdcLv4dKsOTDJVhYPx1JwiLMeH/ bU6iwWj6yMUwtt6yQJbIA < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger) < Link: </riak/gluecon>; rel="up" < Date: Mon, 24 May 2010 21:44:06 GMT < Content-Type: text/html < Content-Length: 19 < basho
  • > GET /riak/gluecon/greeting HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: localhost:8098 > Accept: */* > < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgymDKBVIsrDk3LTOYEhnzWBmUmtOP8kGE2ZqTmNf8Xg6VCAJJZAEA < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger) < Link: </riak/gluecon>; rel="up" < Last-Modified: Tue, 18 May 2010 21:32:02 GMT < ETag: 1VW7QmCuyofVCuQbqBNVYh < Date: Tue, 18 May 2010 21:32:06 GMT < Content-Type: text/html; charset=utf8 < Content-Length: 19 < * Connection #0 to host localhost left intact * Closing connection #0 <h1>Hi GlueCon</h1> basho
  • Riak was designed to scale horizontally. basho
  • Riak is highly available basho
  • #Replicas - N R+W > N #Get response - R #Put responses - W for quorum
  • Query your data with map/reduce basho
  • basho
  • function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho
  • map() function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho
  • map() map() map() map() map() map() map() map() basho
  • R1 R8 R2 R7 R3 R6 R4 R5 basho
  • R2 reduce() 6 5 8 7 4 3 1 function(v) { var r = {}; for (var i in v) { for(var w in v[i]) { if (w in r) r[w] += v[i][w]; else r[w] = v[i][w]; } } return [r]; } basho
  • Store relationships with links basho
  • ch ild people/dean child chi ld people/bob people/alice people/claire basho
  • he r m ot ch ild father people/dean child mo the r chi ld people/bob people/alice people/claire basho
  • he r hild m ot gra ndc ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght grandc er hild people/claire basho
  • he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_ key link link bucket tag people/claire basho
  • he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_/_,child,_ key link1 link1 link2 link2 bucket tag bucket tag people/claire basho
  • he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,1/people,child,_ people/claire basho
  • CLIENT LIBRARIES Ruby Javascript Java .NET linq Twisted Python basho
  • http://wiki.basho.com follow twitter.com/basho/team riak-users@lists.basho.com #riak on Freenode basho