Your SlideShare is downloading. ×
0
Jon Meredith   Basho Technologies
Riak is
        a scalable, highly-available, networked
                    key/value store.



basho
Inspired by Amazon’s Dynamo
          ...with some enhancements.

               Open Source


basho
• Riak   stores values against keys.

        • Encode    your data how you like it.

        • Keys   are organized into ...
BASIC OPERATIONS

              get
              put
             delete


basho
• Riak   also stores metadata against keys

         • Content Type, Charset, Encoding, ...

         • Custom     applica...
bucket: gluecon
        key: greeting
        value: “<h1>Hi GlueCon<h1>”
        metadata:
          content-type=text/ht...
>   PUT /riak/gluecon/greeting HTTP/1.1
        >   User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 Op...
> GET /riak/gluecon/greeting HTTP/1.1
        > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSS...
Riak was designed to scale horizontally.




basho
Riak is highly available


basho
#Replicas - N        R+W > N
#Get response - R
#Put responses - W   for quorum
Query your data with map/reduce


basho
basho
function(v) {
           var m = v.values[0].data.
                   match('w*','g');
           var r = [];
           f...
map()
                function(v) {
                   var m = v.values[0].data.
                           match('w*','g'...
map()

                map()           map()




        map()                           map()




                map()  ...
R1

             R8        R2




        R7                  R3




             R6        R4

                  R5
basho
R2
        reduce()
            6
            5
            8
            7
            4
            3
            1
    ...
Store relationships with links


basho
ch ild
                                                       people/dean
                     child
                     ...
he r
                                             m ot
                                                ch ild
            ...
he r
                                          hild      m ot
                                  gra ndc
                  ...
he r
                                                 m ot
                                                    ch ild
    ...
he r
                                                   m ot
                                                       ch ild...
he r
                                                   m ot
                                                       ch ild...
CLIENT LIBRARIES


                             Ruby


                           Javascript
        Java

          .NET ...
http://wiki.basho.com
           follow twitter.com/basho/team
             riak-users@lists.basho.com
                 #r...
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Riak in Ten Minutes
Upcoming SlideShare
Loading in...5
×

Riak in Ten Minutes

7,753

Published on

Here's my presentation from the NoSQL vendor track at GlueCon.

Published in: Technology
1 Comment
10 Likes
Statistics
Notes
No Downloads
Views
Total Views
7,753
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
163
Comments
1
Likes
10
Embeds 0
No embeds

No notes for slide


  • Inspired by the paper Amazon wrote about developing their Dynamo system.
  • One common encoding is JSON, but you can use what you&amp;#x2019;d like.

  • Applications can use the content type to decide how to decode the value.
    We call the combination of key/bucket/value/metadata a Riak Object.
    One of the things we use the metadata for is to make a nice restful HTTP interface.

  • Here&amp;#x2019;s an example - if we stored an HTML fragment under they key greeting, in the bucket gluecon

  • The additional metadata lets us generate a nice HTTP response.
    Riak plays very well with the web.
  • The additional metadata lets us generate a nice HTTP response.
    Riak plays very well with the web.
  • Riak is a decentralized key/value store.
    It uses distributed hashing to spread the data across all available machines.
    All nodes are the same and we have no single points of failure.
  • From an operations point of view, adding a node is as simple as configuring the base operating system,
    installing riak and joining the new node to the cluster.
  • There are no replication topologies to update.
    Data is automatically migrated to new nodes.
  • Add more nodes for extra capacity or lowering latency.

  • And when your spike is over, Riak scales back down again.
  • Once you have multiple nodes, you get reliability.
    Riak can store multiple copies of your data.
    get/put/delete operations request each of the replicas and make sure they are all up to date.
  • The N value stores how many replicas of each key are stored.
    As nodes may come and go, there is a chance of the cluster containing stale data or conflicts.
    Riak handles this by versioning objects with vector clocks and requesting all N objects.
    Using the vector clock we can tell if the data is just stale or in conflict.
    Riak can handle this with a last timestamp wins strategy or provide the conflicts back to the application.





  • Dealing with clusters increases your chance of something failing.
    In Riak when a node becomes unavailable, another node takes over for it.

  • If the failure is temporary and the node comes back, the data it missed is returned to it.
    Or it can be removed from the cluster and the other nodes will take over for it permanently.
  • If the failure is temporary and the node comes back, the data it missed is returned to it.
    Or it can be removed from the cluster and the other nodes will take over for it permanently.
  • The get/put/delete operations in Riak access data by primary key.
    Most applications need more.
    Riak has an implementation of map/reduce that allows you to visit each key in a bucket
    in a map phase extract any information you&amp;#x2019;d like from it (possibly none) and then combine the results
    in a reduce phase.
  • map and reduce functions can be written in Javascript or Erlang.
    The work of mapping is distributed to all nodes in the cluster.
    The results of the map are streamed to one node for the reduce().
  • map and reduce functions can be written in Javascript or Erlang.
    The work of mapping is distributed to all nodes in the cluster.
    The results of the map are streamed to one node for the reduce().
  • map and reduce functions can be written in Javascript or Erlang.
    The work of mapping is distributed to all nodes in the cluster.
    The results of the map are streamed to one node for the reduce().
  • map and reduce functions can be written in Javascript or Erlang.
    The work of mapping is distributed to all nodes in the cluster.
    The results of the map are streamed to one node for the reduce().
  • map and reduce functions can be written in Javascript or Erlang.
    The work of mapping is distributed to all nodes in the cluster.
    The results of the map are streamed to one node for the reduce().
  • map and reduce functions can be written in Javascript or Erlang.
    The work of mapping is distributed to all nodes in the cluster.
    The results of the map are streamed to one node for the reduce().
  • map and reduce functions can be written in Javascript or Erlang.
    The work of mapping is distributed to all nodes in the cluster.
    The results of the map are streamed to one node for the reduce().
  • map and reduce functions can be written in Javascript or Erlang.
    The work of mapping is distributed to all nodes in the cluster.
    The results of the map are streamed to one node for the reduce().
  • map and reduce functions can be written in Javascript or Erlang.
    The work of mapping is distributed to all nodes in the cluster.
    The results of the map are streamed to one node for the reduce().



  • As they say, no riak object is an island. Application care about relationships in their data.
    Riak can store one way &amp;#x2018;links&amp;#x2019; in the object metadata, pointing to another bucket/key.
    Each link has a tag and there can be many objects.
    Riak can combine map/reduce with object metadata to handle relationships between them and we have a special link walking interface included in our HTTP interace.

  • Here&amp;#x2019;s an example using biological relationships.
    The first set of links store children.
  • but links are one way, so if you want to get back you have to add more links.
  • and more links - however it makes sense to use them
  • We expose links through map/reduce
    and directly over our HTTP interface
    We can

  • You can also return intermediate steps - after the trailing comma for the observant of you.
  • Riak has clients in many languages that are supported by the Basho team and there are also
    community provided ones springing up too.
  • That&amp;#x2019;s about all I could show you in 10 minutes - please visit our website and check us out.
    We have a FastTrack section on our wiki to go through all you need to get started.
    Thank you for listening.
  • Transcript of "Riak in Ten Minutes"

    1. 1. Jon Meredith Basho Technologies
    2. 2. Riak is a scalable, highly-available, networked key/value store. basho
    3. 3. Inspired by Amazon’s Dynamo ...with some enhancements. Open Source basho
    4. 4. • Riak stores values against keys. • Encode your data how you like it. • Keys are organized into buckets. basho
    5. 5. BASIC OPERATIONS get put delete basho
    6. 6. • Riak also stores metadata against keys • Content Type, Charset, Encoding, ... • Custom application metadata basho
    7. 7. bucket: gluecon key: greeting value: “<h1>Hi GlueCon<h1>” metadata: content-type=text/html charset=utf8 basho
    8. 8. > PUT /riak/gluecon/greeting HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: 127.0.0.1:8098 > Accept: */* > content-type: text/html > charset: utf8 > Content-Length: 19 > > <h1>Hi GlueCon<h1> < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgzWDKBVIsTNPiOTOYEhnzWBm+5uUf5YMIszUnMdcLv4dKsOTDJVhYPx1JwiLMeH/ bU6iwWj6yMUwtt6yQJbIA < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger) < Link: </riak/gluecon>; rel="up" < Date: Mon, 24 May 2010 21:44:06 GMT < Content-Type: text/html < Content-Length: 19 < basho
    9. 9. > GET /riak/gluecon/greeting HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: localhost:8098 > Accept: */* > < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgymDKBVIsrDk3LTOYEhnzWBmUmtOP8kGE2ZqTmNf8Xg6VCAJJZAEA < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger) < Link: </riak/gluecon>; rel="up" < Last-Modified: Tue, 18 May 2010 21:32:02 GMT < ETag: 1VW7QmCuyofVCuQbqBNVYh < Date: Tue, 18 May 2010 21:32:06 GMT < Content-Type: text/html; charset=utf8 < Content-Length: 19 < * Connection #0 to host localhost left intact * Closing connection #0 <h1>Hi GlueCon</h1> basho
    10. 10. Riak was designed to scale horizontally. basho
    11. 11. Riak is highly available basho
    12. 12. #Replicas - N R+W > N #Get response - R #Put responses - W for quorum
    13. 13. Query your data with map/reduce basho
    14. 14. basho
    15. 15. function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho
    16. 16. map() function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho
    17. 17. map() map() map() map() map() map() map() map() basho
    18. 18. R1 R8 R2 R7 R3 R6 R4 R5 basho
    19. 19. R2 reduce() 6 5 8 7 4 3 1 function(v) { var r = {}; for (var i in v) { for(var w in v[i]) { if (w in r) r[w] += v[i][w]; else r[w] = v[i][w]; } } return [r]; } basho
    20. 20. Store relationships with links basho
    21. 21. ch ild people/dean child chi ld people/bob people/alice people/claire basho
    22. 22. he r m ot ch ild father people/dean child mo the r chi ld people/bob people/alice people/claire basho
    23. 23. he r hild m ot gra ndc ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght grandc er hild people/claire basho
    24. 24. he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_ key link link bucket tag people/claire basho
    25. 25. he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_/_,child,_ key link1 link1 link2 link2 bucket tag bucket tag people/claire basho
    26. 26. he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,1/people,child,_ people/claire basho
    27. 27. CLIENT LIBRARIES Ruby Javascript Java .NET linq Twisted Python basho
    28. 28. http://wiki.basho.com follow twitter.com/basho/team riak-users@lists.basho.com #riak on Freenode basho
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×