Riak in Ten Minutes
May. 26, 2010•0 likes•3,643 views
Download to read offline
Report
Technology
Here's my presentation from the NoSQL vendor track at GlueCon.
Jon MeredithFollow
More Related Content
Recently uploaded
Nymity Framework: Privacy & Data Protection Update in 7 StatesTrustArc
Supplier Sourcing_Cathy.pptxCatarinaTorrenuevaMa
Elevate Your Enterprise with FME 23.1Safe Software
Webinar: Discover the Power of SpiraTeam - A Jira Alternative To Revolutioniz...Inflectra
Google Cloud Study Jams Info SessionGDSCPCCE
Future of SkillsAlison B. Lowndes
Featured
Introduction to Data ScienceChristy Abraham Joy
Time Management & Productivity - Best PracticesVit Horky
The six step guide to practical project managementMindGenius
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...RachelPearson36
Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...Applitools
12 Ways to Increase Your Influence at WorkGetSmarter
Riak in Ten Minutes
- 1. Jon Meredith Basho Technologies
- 2. Riak is a scalable, highly-available, networked key/value store. basho
- 3. Inspired by Amazon’s Dynamo ...with some enhancements. Open Source basho
- 4. • Riak stores values against keys. • Encode your data how you like it. • Keys are organized into buckets. basho
- 5. BASIC OPERATIONS get put delete basho
- 6. • Riak also stores metadata against keys • Content Type, Charset, Encoding, ... • Custom application metadata basho
- 7. bucket: gluecon key: greeting value: “<h1>Hi GlueCon<h1>” metadata: content-type=text/html charset=utf8 basho
- 8. > PUT /riak/gluecon/greeting HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: 127.0.0.1:8098 > Accept: */* > content-type: text/html > charset: utf8 > Content-Length: 19 > > <h1>Hi GlueCon<h1> < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgzWDKBVIsTNPiOTOYEhnzWBm+5uUf5YMIszUnMdcLv4dKsOTDJVhYPx1JwiLMeH/ bU6iwWj6yMUwtt6yQJbIA < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger) < Link: </riak/gluecon>; rel="up" < Date: Mon, 24 May 2010 21:44:06 GMT < Content-Type: text/html < Content-Length: 19 < basho
- 9. > GET /riak/gluecon/greeting HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: localhost:8098 > Accept: */* > < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgymDKBVIsrDk3LTOYEhnzWBmUmtOP8kGE2ZqTmNf8Xg6VCAJJZAEA < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger) < Link: </riak/gluecon>; rel="up" < Last-Modified: Tue, 18 May 2010 21:32:02 GMT < ETag: 1VW7QmCuyofVCuQbqBNVYh < Date: Tue, 18 May 2010 21:32:06 GMT < Content-Type: text/html; charset=utf8 < Content-Length: 19 < * Connection #0 to host localhost left intact * Closing connection #0 <h1>Hi GlueCon</h1> basho
- 10. Riak was designed to scale horizontally. basho
- 16. Riak is highly available basho
- 20. #Replicas - N R+W > N #Get response - R #Put responses - W for quorum
- 26. Query your data with map/reduce basho
- 27. basho
- 28. function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho
- 29. map() function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho
- 30. map() map() map() map() map() map() map() map() basho
- 31. R1 R8 R2 R7 R3 R6 R4 R5 basho
- 32. R2 reduce() 6 5 8 7 4 3 1 function(v) { var r = {}; for (var i in v) { for(var w in v[i]) { if (w in r) r[w] += v[i][w]; else r[w] = v[i][w]; } } return [r]; } basho
- 33. Store relationships with links basho
- 34. ch ild people/dean child chi ld people/bob people/alice people/claire basho
- 35. he r m ot ch ild father people/dean child mo the r chi ld people/bob people/alice people/claire basho
- 36. he r hild m ot gra ndc ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght grandc er hild people/claire basho
- 37. he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_ key link link bucket tag people/claire basho
- 38. he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_/_,child,_ key link1 link1 link2 link2 bucket tag bucket tag people/claire basho
- 39. he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,1/people,child,_ people/claire basho
- 40. CLIENT LIBRARIES Ruby Javascript Java .NET linq Twisted Python basho
- 41. http://wiki.basho.com follow twitter.com/basho/team riak-users@lists.basho.com #riak on Freenode basho
Editor's Notes
- Inspired by the paper Amazon wrote about developing their Dynamo system.
- One common encoding is JSON, but you can use what you&#x2019;d like.
- Applications can use the content type to decide how to decode the value. We call the combination of key/bucket/value/metadata a Riak Object. One of the things we use the metadata for is to make a nice restful HTTP interface.
- Here&#x2019;s an example - if we stored an HTML fragment under they key greeting, in the bucket gluecon
- The additional metadata lets us generate a nice HTTP response. Riak plays very well with the web.
- The additional metadata lets us generate a nice HTTP response. Riak plays very well with the web.
- Riak is a decentralized key/value store. It uses distributed hashing to spread the data across all available machines. All nodes are the same and we have no single points of failure.
- From an operations point of view, adding a node is as simple as configuring the base operating system, installing riak and joining the new node to the cluster.
- There are no replication topologies to update. Data is automatically migrated to new nodes.
- Add more nodes for extra capacity or lowering latency.
- And when your spike is over, Riak scales back down again.
- Once you have multiple nodes, you get reliability. Riak can store multiple copies of your data. get/put/delete operations request each of the replicas and make sure they are all up to date.
- The N value stores how many replicas of each key are stored. As nodes may come and go, there is a chance of the cluster containing stale data or conflicts. Riak handles this by versioning objects with vector clocks and requesting all N objects. Using the vector clock we can tell if the data is just stale or in conflict. Riak can handle this with a last timestamp wins strategy or provide the conflicts back to the application.
- Dealing with clusters increases your chance of something failing. In Riak when a node becomes unavailable, another node takes over for it.
- If the failure is temporary and the node comes back, the data it missed is returned to it. Or it can be removed from the cluster and the other nodes will take over for it permanently.
- If the failure is temporary and the node comes back, the data it missed is returned to it. Or it can be removed from the cluster and the other nodes will take over for it permanently.
- The get/put/delete operations in Riak access data by primary key. Most applications need more. Riak has an implementation of map/reduce that allows you to visit each key in a bucket in a map phase extract any information you&#x2019;d like from it (possibly none) and then combine the results in a reduce phase.
- map and reduce functions can be written in Javascript or Erlang. The work of mapping is distributed to all nodes in the cluster. The results of the map are streamed to one node for the reduce().
- map and reduce functions can be written in Javascript or Erlang. The work of mapping is distributed to all nodes in the cluster. The results of the map are streamed to one node for the reduce().
- map and reduce functions can be written in Javascript or Erlang. The work of mapping is distributed to all nodes in the cluster. The results of the map are streamed to one node for the reduce().
- map and reduce functions can be written in Javascript or Erlang. The work of mapping is distributed to all nodes in the cluster. The results of the map are streamed to one node for the reduce().
- map and reduce functions can be written in Javascript or Erlang. The work of mapping is distributed to all nodes in the cluster. The results of the map are streamed to one node for the reduce().
- map and reduce functions can be written in Javascript or Erlang. The work of mapping is distributed to all nodes in the cluster. The results of the map are streamed to one node for the reduce().
- map and reduce functions can be written in Javascript or Erlang. The work of mapping is distributed to all nodes in the cluster. The results of the map are streamed to one node for the reduce().
- map and reduce functions can be written in Javascript or Erlang. The work of mapping is distributed to all nodes in the cluster. The results of the map are streamed to one node for the reduce().
- map and reduce functions can be written in Javascript or Erlang. The work of mapping is distributed to all nodes in the cluster. The results of the map are streamed to one node for the reduce().
- As they say, no riak object is an island. Application care about relationships in their data. Riak can store one way &#x2018;links&#x2019; in the object metadata, pointing to another bucket/key. Each link has a tag and there can be many objects. Riak can combine map/reduce with object metadata to handle relationships between them and we have a special link walking interface included in our HTTP interace.
- Here&#x2019;s an example using biological relationships. The first set of links store children.
- but links are one way, so if you want to get back you have to add more links.
- and more links - however it makes sense to use them
- We expose links through map/reduce and directly over our HTTP interface We can
- You can also return intermediate steps - after the trailing comma for the observant of you.
- Riak has clients in many languages that are supported by the Basho team and there are also community provided ones springing up too.
- That&#x2019;s about all I could show you in 10 minutes - please visit our website and check us out. We have a FastTrack section on our wiki to go through all you need to get started. Thank you for listening.