Riak (Øredev nosql day)

Sean Cribbs Basho Technologies

Riak is a scalable, highly-available, networked key/value store. basho

Inspired by Amazon’s Dynamo ...with some enhancements. Open Source basho

• Riak stores values against keys. • Encode your data how you like it. • Keys are organized into buckets. basho

BASIC OPERATIONS get put delete

• Riak also stores metadata against keys • Content-Type, Last-Modiﬁed, ... • Custom application metadata basho

bucket: oredev key: greeting value: “<h1>Hi oredev<h1>” metadata: content-type=text/html basho

> PUT /riak/oredev/greeting?returnbody=true HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: localhost:8098 > Accept: */* > Content-Type: text/html > Content-Length: 18 > < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgzmDKBVIsTDb6JzOYEhnzWBmit8gf44MJv9t5FipcgBBma05inekYiiyRBQA= < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.7.2 (participate in the frantic) < Link: </riak/oredev>; rel="up" < Date: Tue, 05 Oct 2010 14:38:40 GMT < Content-Type: text/html < Content-Length: 18 < basho

> GET /riak/oredev/greeting HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: localhost:8098 > Accept: */* > < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgzmDKBVIsTDb6JzOYEhnzWBmit8gf44MJv9t5FipcgBBma05inekYiiyRBQA= < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.7.2 (participate in the frantic) < Link: </riak/oredev>; rel="up" < Last-Modified: Tue, 05 Oct 2010 14:38:40 GMT < ETag: 71hTllswGrWbI2O3WEkZYE < Date: Tue, 05 Oct 2010 14:40:04 GMT < Content-Type: text/html < Content-Length: 18 < <h1>Hi oredev</h1> basho

Riak was designed to scale horizontally. basho

Riak is highly available basho

#Replicas - N R+W > N #Get response - R for “strong #Put responses - W consistency”

Query your data with map/reduce basho

function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho

map() function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho

map() map() map() map() map() map() map() map() basho

R1 R8 R2 R7 R3 R6 R4 R5 basho

R2 reduce() 6 5 8 7 4 3 1 function(v) { var r = {}; for (var i in v) { for(var w in v[i]) { if (w in r) r[w] += v[i][w]; else r[w] = v[i][w]; } } return [r]; } basho

Store relationships with links basho

ch ild people/dean child chi ld people/bob people/alice people/claire basho

he r m ot ch ild father people/dean child mo the r chi ld people/bob people/alice people/claire basho

he r hild m ot gra ndc ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght grandc er hild people/claire basho

he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_ key link link bucket tag people/claire basho

he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_/_,child,_ key link1 link1 link2 link2 bucket tag bucket tag people/claire basho

he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,1/people,child,_ people/claire basho

CLIENT LIBRARIES Ruby Javascript Java .NET linq Twisted Python basho

RIAK IN PRODUCTION basho

MOCHI MEDIA www.mochimedia.com • Multiple clusters in production • Sessions (web and Flash) - 5 nodes, ~20MM keys - highest trafﬁc Developer • User data storage - 3 nodes, bucket-per-game • URL Shortener - 2 nodes, bucket-per-game • Social graph - 3 nodes, OAuth credentials, relationships Text and link here but basho

MOZILLA TEST PILOT http://bit.ly/aC0i4f • Data collector for anonymized Firefox usage statistics • High peak throughput - pure writes • Large MapReduce queries for analytics “Riak meets or exceeds all the write performance requirements...it was nice to see predictable performance and stability under load.” basho

WEGEO • Mobile Productivity / Collaboration App - Task lists, Shopping lists, Calendar • Event and Check-in logs • Monthly reports archive “Scalability I could understand, on a platform I could understand.” • Third-party coupon and deal aggregation basho

MAP-REDUCE Not always so rosy basho

RIAK SEARCH (VERY NEW) • Distributed, fault-tolerant, scalable full-text indexing search engine • Supports most Lucene syntax, Solr-like interface • Riak KV integration • automatic indexing of your objects • feed MapReduce with search results basho

RIAK CORE Dynamo Abstracted basho

RIAK CORE Dynamo Abstracted Riak KV Riak Core Riak KV basho

RIAK CORE Dynamo Abstracted clients Riak KV Riak Core Riak KV backend basho

RIAK CORE Dynamo Abstracted clients Riak KV cluster Riak Core Riak KV backend basho

RIAK CORE Dynamo Abstracted clients Riak KV cluster Riak Core Riak KV backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho

RIAK CORE Dynamo Abstracted clients Riak KV Riak Search cluster Riak Core Riak Core Riak KV Riak Search backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho

RIAK CORE Dynamo Abstracted clients Riak KV Riak Search cluster Riak Core Riak Core Riak Core Riak KV Riak Search backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho

RIAK CORE Dynamo Abstracted clients Riak KV Riak Search BigTable? cluster Riak Core Riak Core Riak Core Riak KV Riak Search BigTable? backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho

RIAK CORE Dynamo Abstracted clients Riak KV Riak Search BigTable? Graph? cluster Riak Core Riak Core Riak Core Riak KV Riak Search BigTable? Graph? backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho

RIAK CORE Dynamo Abstracted clients Riak KV Riak Search BigTable? ? Graph? cluster Riak Core Riak Core Riak Core Riak KV Riak Search BigTable? ? Graph? backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho

http://wiki.basho.com follow twitter.com/basho/team riak-users@lists.basho.com #riak on Freenode basho

SOAPBOX() -> basho

http://www.ﬂickr.com/photos/mullica/2769082212/ NOSQL IS ABOUT CHOICE Including the choice to fail spectacularly. basho

http://www.ﬂickr.com/photos/bike/236125877/ DON’T CARGO-CULT You are not Digg, Twitter, Google, Amazon, or Yahoo. Except Kyle. basho

http://www.ﬂickr.com/photos/library_of_virginia/2898496299/ IGNORE THE SALESPERSON Do your own evaluation. Don’t parrot the hype. basho

http://bit.ly/cGVPv7 UNDERSTAND YOUR DATA One size does not ﬁt all. Give up your crutches. basho

PLAN FOR FAILURE Your code is not perfect. Your server will go down. basho

http://www.solarnavigator.net/animal_kingdom/humans/simon_hall.htm TAKE RESPONSIBILITY Own your choice. Understand the consequences. basho

END. basho

NOSQL IS DEAD LONG LIVE NOSQL basho

http://basho.com	1101
http://www.nosqldatabases.com	186
http://basho.co.jp	42
http://coderwall.com	34
http://crowdczar.com	12
http://localhost:3000	5
http://localhost:8000	2
http://translate.googleusercontent.com	2
http://www-new.basho.com	1
http://nosqldatabases.squarespace.com	1

Riak (Øredev nosql day)

by Sean Cribbs

Accessibility

Categories

Upload Details

Usage Rights

10 Embeds 1,386

Statistics

Riak (Øredev nosql day) Presentation Transcript