• Like
  • Save

Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Riak (Øredev nosql day)

  • 3,980 views
Uploaded on

 

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,980
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
0
Comments
0
Likes
11

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Sean Cribbs Basho Technologies
  • 2. Riak is a scalable, highly-available, networked key/value store. basho
  • 3. Inspired by Amazon’s Dynamo ...with some enhancements. Open Source basho
  • 4. • Riak stores values against keys. • Encode your data how you like it. • Keys are organized into buckets. basho
  • 5. BASIC OPERATIONS get put delete
  • 6. • Riak also stores metadata against keys • Content-Type, Last-Modified, ... • Custom application metadata basho
  • 7. bucket: oredev key: greeting value: “<h1>Hi oredev<h1>” metadata: content-type=text/html basho
  • 8. > PUT /riak/oredev/greeting?returnbody=true HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: localhost:8098 > Accept: */* > Content-Type: text/html > Content-Length: 18 > < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgzmDKBVIsTDb6JzOYEhnzWBmit8gf44MJv9t5FipcgBBma05inekYiiyRBQA= < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.7.2 (participate in the frantic) < Link: </riak/oredev>; rel="up" < Date: Tue, 05 Oct 2010 14:38:40 GMT < Content-Type: text/html < Content-Length: 18 < basho
  • 9. > GET /riak/oredev/greeting HTTP/1.1 > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 > Host: localhost:8098 > Accept: */* > < HTTP/1.1 200 OK < X-Riak-Vclock: a85hYGBgzmDKBVIsTDb6JzOYEhnzWBmit8gf44MJv9t5FipcgBBma05inekYiiyRBQA= < Vary: Accept-Encoding < Server: MochiWeb/1.1 WebMachine/1.7.2 (participate in the frantic) < Link: </riak/oredev>; rel="up" < Last-Modified: Tue, 05 Oct 2010 14:38:40 GMT < ETag: 71hTllswGrWbI2O3WEkZYE < Date: Tue, 05 Oct 2010 14:40:04 GMT < Content-Type: text/html < Content-Length: 18 < <h1>Hi oredev</h1> basho
  • 10. Riak was designed to scale horizontally. basho
  • 11. Riak is highly available basho
  • 12. #Replicas - N R+W > N #Get response - R for “strong #Put responses - W consistency”
  • 13. Query your data with map/reduce basho
  • 14. basho
  • 15. function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho
  • 16. map() function(v) { var m = v.values[0].data. match('w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r; } basho
  • 17. map() map() map() map() map() map() map() map() basho
  • 18. R1 R8 R2 R7 R3 R6 R4 R5 basho
  • 19. R2 reduce() 6 5 8 7 4 3 1 function(v) { var r = {}; for (var i in v) { for(var w in v[i]) { if (w in r) r[w] += v[i][w]; else r[w] = v[i][w]; } } return [r]; } basho
  • 20. Store relationships with links basho
  • 21. ch ild people/dean child chi ld people/bob people/alice people/claire basho
  • 22. he r m ot ch ild father people/dean child mo the r chi ld people/bob people/alice people/claire basho
  • 23. he r hild m ot gra ndc ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght grandc er hild people/claire basho
  • 24. he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_ key link link bucket tag people/claire basho
  • 25. he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,_/_,child,_ key link1 link1 link2 link2 bucket tag bucket tag people/claire basho
  • 26. he r m ot ch ild father so n people/dean child mo the daughter r chi dau ld people/bob people/alice ght er GET /riak/person/bob/people,child,1/people,child,_ people/claire basho
  • 27. CLIENT LIBRARIES Ruby Javascript Java .NET linq Twisted Python basho
  • 28. RIAK IN PRODUCTION basho
  • 29. MOCHI MEDIA www.mochimedia.com • Multiple clusters in production • Sessions (web and Flash) - 5 nodes, ~20MM keys - highest traffic Developer • User data storage - 3 nodes, bucket-per-game • URL Shortener - 2 nodes, bucket-per-game • Social graph - 3 nodes, OAuth credentials, relationships Text and link here but basho
  • 30. MOZILLA TEST PILOT http://bit.ly/aC0i4f • Data collector for anonymized Firefox usage statistics • High peak throughput - pure writes • Large MapReduce queries for analytics “Riak meets or exceeds all the write performance requirements...it was nice to see predictable performance and stability under load.” basho
  • 31. WEGEO • Mobile Productivity / Collaboration App - Task lists, Shopping lists, Calendar • Event and Check-in logs • Monthly reports archive “Scalability I could understand, on a platform I could understand.” • Third-party coupon and deal aggregation basho
  • 32. MAP-REDUCE Not always so rosy basho
  • 33. RIAK SEARCH (VERY NEW) • Distributed, fault-tolerant, scalable full-text indexing search engine • Supports most Lucene syntax, Solr-like interface • Riak KV integration • automatic indexing of your objects • feed MapReduce with search results basho
  • 34. RIAK CORE Dynamo Abstracted basho
  • 35. RIAK CORE Dynamo Abstracted Riak KV Riak Core Riak KV basho
  • 36. RIAK CORE Dynamo Abstracted clients Riak KV Riak Core Riak KV backend basho
  • 37. RIAK CORE Dynamo Abstracted clients Riak KV cluster Riak Core Riak KV backend basho
  • 38. RIAK CORE Dynamo Abstracted clients Riak KV cluster Riak Core Riak KV backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho
  • 39. RIAK CORE Dynamo Abstracted clients Riak KV Riak Search cluster Riak Core Riak Core Riak KV Riak Search backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho
  • 40. RIAK CORE Dynamo Abstracted clients Riak KV Riak Search cluster Riak Core Riak Core Riak Core Riak KV Riak Search backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho
  • 41. RIAK CORE Dynamo Abstracted clients Riak KV Riak Search BigTable? cluster Riak Core Riak Core Riak Core Riak KV Riak Search BigTable? backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho
  • 42. RIAK CORE Dynamo Abstracted clients Riak KV Riak Search BigTable? Graph? cluster Riak Core Riak Core Riak Core Riak KV Riak Search BigTable? Graph? backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho
  • 43. RIAK CORE Dynamo Abstracted clients Riak KV Riak Search BigTable? ? Graph? cluster Riak Core Riak Core Riak Core Riak KV Riak Search BigTable? ? Graph? backend consistent hashing, membership, node-liveness, handoff, gossip, buckets basho
  • 44. http://wiki.basho.com follow twitter.com/basho/team riak-users@lists.basho.com #riak on Freenode basho
  • 45. SOAPBOX() -> basho
  • 46. http://www.flickr.com/photos/mullica/2769082212/ NOSQL IS ABOUT CHOICE Including the choice to fail spectacularly. basho
  • 47. http://www.flickr.com/photos/bike/236125877/ DON’T CARGO-CULT You are not Digg, Twitter, Google, Amazon, or Yahoo. Except Kyle. basho
  • 48. http://www.flickr.com/photos/library_of_virginia/2898496299/ IGNORE THE SALESPERSON Do your own evaluation. Don’t parrot the hype. basho
  • 49. http://bit.ly/cGVPv7 UNDERSTAND YOUR DATA One size does not fit all. Give up your crutches. basho
  • 50. PLAN FOR FAILURE Your code is not perfect. Your server will go down. basho
  • 51. http://www.solarnavigator.net/animal_kingdom/humans/simon_hall.htm TAKE RESPONSIBILITY Own your choice. Understand the consequences. basho
  • 52. END. basho
  • 53. NOSQL IS DEAD LONG LIVE NOSQL basho