Your SlideShare is downloading. ×
Introduction to Riak and Ripple (KC.rb)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Introduction to Riak and Ripple (KC.rb)

3,698
views

Published on

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,698
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
57
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
































































































































  • Transcript

    • 1. Introduction to Riak & Ripple Sean Cribbs basho Developer Advocate
    • 2. Introduction to Riak & Ripple Sean Cribbs basho Developer Advocate Rubyist, Evangelist, Support Monkey
    • 3. Riak @10,000 ft.
    • 4. Riak @10,000 ft. • Amazon Dynamo-inspired replicated, distributed, fault-tolerant, masterless, scalable, operations-friendly
    • 5. Riak @10,000 ft. • Amazon Dynamo-inspired replicated, distributed, fault-tolerant, masterless, scalable, operations-friendly • Key-Value / Document
    • 6. Riak @10,000 ft. • Amazon Dynamo-inspired replicated, distributed, fault-tolerant, masterless, scalable, operations-friendly • Key-Value / Document • Schema-less, content-agnostic
    • 7. Riak @10,000 ft. • Amazon Dynamo-inspired replicated, distributed, fault-tolerant, masterless, scalable, operations-friendly • Key-Value / Document • Schema-less, content-agnostic • Web-friendly HTTP, JSON, Javascript
    • 8. Dynamo-like Scalability
    • 9. Dynamo-like Scalability • To get...
    • 10. Dynamo-like Scalability • To get... • more storage,
    • 11. Dynamo-like Scalability • To get... • more storage, • more throughput,
    • 12. Dynamo-like Scalability • To get... • more storage, • more throughput, • lower latency...
    • 13. Dynamo-like Scalability • To get... • more storage, • more throughput, • lower latency... • ...add more machines. aka horizontal & linear
    • 14. Key-Value++
    • 15. Key-Value++ • Data objects are identified by keys, and have metadata
    • 16. Key-Value++ • Data objects are identified by keys, and have metadata • Keys are grouped in buckets
    • 17. Key-Value++ • Data objects are identified by keys, and have metadata • Keys are grouped in buckets • Links enable lightweight relationships
    • 18. Key-Value++ • Data objects are identified by keys, and have metadata • Keys are grouped in buckets • Links enable lightweight relationships • Query with MapReduce
    • 19. Sans-Schema
    • 20. Sans-Schema • Buckets created on the fly
    • 21. Sans-Schema • Buckets created on the fly • Values are opaque
    • 22. Sans-Schema • Buckets created on the fly • Values are opaque •Content-Type matters
    • 23. Sans-Schema • Buckets created on the fly • Values are opaque •Content-Type matters •The application defines the semantics: more flexibility, more responsibility
    • 24. Web-Friendly
    • 25. Web-Friendly •HTTP is primary interface
    • 26. Web-Friendly •HTTP is primary interface •JSON is used for structured data
    • 27. Web-Friendly •HTTP is primary interface •JSON is used for structured data •Javascript is used for MapReduce functions
    • 28. Web-Friendly •HTTP is primary interface •JSON is used for structured data •Javascript is used for MapReduce functions •Plays well with Varnish, Squid, HAProxy, F5, etc.
    • 29. Web-Friendly •HTTP is primary interface •JSON is used for structured data •Javascript is used for MapReduce functions •Plays well with Varnish, Squid, HAProxy, F5, etc. [see also Webmachine]
    • 30. Use Cases
    • 31. Use Cases •Document storage (sparse structure)
    • 32. Use Cases •Document storage (sparse structure) •Distributed file storage (small size, large soon)
    • 33. Use Cases •Document storage (sparse structure) •Distributed file storage (small size, large soon) •Session storage
    • 34. Use Cases •Document storage (sparse structure) •Distributed file storage (small size, large soon) •Session storage •Distributed cache
    • 35. Use Cases •Document storage (sparse structure) •Distributed file storage (small size, large soon) •Session storage •Distributed cache • Browser-only/Mobile Apps
    • 36. Getting Started
    • 37. Getting Started • Download a package from http://downloads.basho.com
    • 38. Getting Started • Download a package from http://downloads.basho.com • Set the node name, IPs
    • 39. Getting Started • Download a package from http://downloads.basho.com • Set the node name, IPs • Start up the node
    • 40. Getting Started • Download a package from http://downloads.basho.com • Set the node name, IPs • Start up the node • Join a cluster
    • 41. Getting Started • Download a package from http://downloads.basho.com • Set the node name, IPs • Start up the node • Join a cluster • Start storing/retrieving values
    • 42. Riak in Ruby
    • 43. Riak in Ruby • Three gems
    • 44. Riak in Ruby • Three gems • riak-client (basic ops)
    • 45. Riak in Ruby • Three gems • riak-client (basic ops) • ripple (ODM)
    • 46. Riak in Ruby • Three gems • riak-client (basic ops) • ripple (ODM) • riak-sessions (Rack/Rails session stores)
    • 47. Riak in Ruby • Three gems • riak-client (basic ops) • ripple (ODM) • riak-sessions (Rack/Rails session stores) • All HTTP - Protobuffs coming
    • 48. require ‘riak’
    • 49. require ‘riak’ • Make a client object client = Riak::Client.new
    • 50. require ‘riak’ • Make a client object client = Riak::Client.new • Get a bucket bucket = client.bucket(‘foo’) # Riak::Bucket
    • 51. require ‘riak’ • Make a client object client = Riak::Client.new • Get a bucket bucket = client.bucket(‘foo’) # Riak::Bucket • Get an object from the bucket obj = bucket.get(‘bar’) # Riak::RObject
    • 52. require ‘riak’ • Make a client object client = Riak::Client.new • Get a bucket bucket = client.bucket(‘foo’) # Riak::Bucket • Get an object from the bucket obj = bucket.get(‘bar’) # Riak::RObject • Initialize a new object obj = bucket.new(‘baz’)
    • 53. Riak::RObject
    • 54. Riak::RObject •Get/set object key obj.key = “bar”
    • 55. Riak::RObject •Get/set object key obj.key = “bar” •Get/set content-type obj.content_type = ‘application/json’
    • 56. Riak::RObject •Get/set object key obj.key = “bar” •Get/set content-type obj.content_type = ‘application/json’ •Get/set the object body data obj.data = {“name” => “Sean”}
    • 57. Riak::RObject •Get/set object key obj.key = “bar” •Get/set content-type obj.content_type = ‘application/json’ •Get/set the object body data obj.data = {“name” => “Sean”} •Store the object obj.store
    • 58. More RObject
    • 59. More RObject •Get object’s bucket obj.bucket
    • 60. More RObject •Get object’s bucket obj.bucket •Delete the object obj.delete # freezes the object
    • 61. More RObject •Get object’s bucket obj.bucket •Delete the object obj.delete # freezes the object •Detect/extract siblings (more later) obj.conflict? && obj.siblings # Array<RObject>
    • 62. More RObject •Get object’s bucket obj.bucket •Delete the object obj.delete # freezes the object •Detect/extract siblings (more later) obj.conflict? && obj.siblings # Array<RObject> •Follow/traverse links (more later) obj.walk(:tag => “friend”)
    • 63. Riak::Bucket
    • 64. Riak::Bucket •Set the replication factor bucket.n_val = 5
    • 65. Riak::Bucket •Set the replication factor bucket.n_val = 5 •Set default request quorums bucket.r = 3 # w,dw,rw # number or one/all/quorum
    • 66. Riak::Bucket •Set the replication factor bucket.n_val = 5 •Set default request quorums bucket.r = 3 # w,dw,rw # number or one/all/quorum •List keys bucket.keys # pass block to stream
    • 67. Riak::Bucket •Set the replication factor bucket.n_val = 5 •Set default request quorums bucket.r = 3 # w,dw,rw # number or one/all/quorum •List keys bucket.keys # pass block to stream •Set consistency flag (allow sibling generation) obj.allow_mult = true
    • 68. Links: Lightweight Relationships
    • 69. Link Header Link: </riak/demo/test1>; riaktag=”userinfo”
    • 70. Link Header bucket Link: </riak/demo/test1>; riaktag=”userinfo”
    • 71. Link Header bucket Link: </riak/demo/test1>; riaktag=”userinfo” key
    • 72. Link Header bucket tag Link: </riak/demo/test1>; riaktag=”userinfo” key
    • 73. Links in Ruby API
    • 74. Links in Ruby API • Create a link Riak::Link.new(“/riak/bucket/key”, “tag”)
    • 75. Links in Ruby API • Create a link Riak::Link.new(“/riak/bucket/key”, “tag”) • Read an#object’s links obj.links Set<Riak::Link>
    • 76. Links in Ruby API • Create a link Riak::Link.new(“/riak/bucket/key”, “tag”) • Read an#object’s links obj.links Set<Riak::Link> • Convert<< obj2.to_link(“next”) obj.links an object to a link obj.store
    • 77. Link-Walking
    • 78. Link-Walking • Asks Riak to traverse a sequence of links via special URL
    • 79. Link-Walking • Asks Riak to traverse a sequence of links via special URL • Filter by bucket/tag
    • 80. Link-Walking • Asks Riak to traverse a sequence of links via special URL • Filter by bucket/tag • Can return results from intermediate traversals
    • 81. Link-Walking • Asks Riak to traverse a sequence of links via special URL • Filter by bucket/tag • Can return results from intermediate traversals • Response is nested multipart/mixed (riak-client handles this for you)
    • 82. L/W Example
    • 83. L/W Example GET /riak/demo/test1/_,friend,1 Start at demo/test1, follow all links tagged “friend” and return the objects obj = client[‘demo’][‘test1’] obj.walk(:tag => “friend”,:keep => true)
    • 84. L/W Example GET /riak/demo/test1/_,friend,1 Start at demo/test1, follow all links tagged “friend” and return the objects obj = client[‘demo’][‘test1’] obj.walk(:tag => “friend”,:keep => true)
    • 85. L/W Example
    • 86. L/W Example GET /riak/demo/test1/_,_,_/_,_,1 Start at demo/test1, follow all links, follow all links from those, return everything from the last set obj.walk({},{:keep => true})
    • 87. Map-Reduce
    • 88. list of keys Map-Reduce
    • 89. list of keys map map map map map Map-Reduce
    • 90. list of keys map map map map map reduce Map-Reduce
    • 91. list of keys map map map map map reduce results Map-Reduce
    • 92. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 93. Map in Javascript Riak object function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 94. Map in Javascript Riak object function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) { {         if(object[field] != arg[field]) {             return []; bucket:”foo”,         } key:”bar”,     } vclock:”...”,     return [object]; } values:[ {metadata:{...}, data:”....”} ] }
    • 95. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 96. Map in Javascript Key-specific data function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 97. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 98. Map in Javascript Phase- specific data function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 99. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 100. Map in Javascript function(value, keyData, arg) { Parse JSON data     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 101. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 102. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     } Check field equality     return [object]; }
    • 103. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 104. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return []; Return nothing if unequal         }     }     return [object]; }
    • 105. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 106. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; Return the object if all equal }
    • 107. Map in Javascript function(value, keyData, arg) {     var object = Riak.mapValuesJson(value)[0];     for(field in arg) {         if(object[field] != arg[field]) {             return [];         }     }     return [object]; }
    • 108. Reduce in Javascript function(values,arg) {     return values.sort(function(a,b){ return a[arg] - b[arg]; }); }
    • 109. Reduce in Javascript Map results function(values,arg) {     return values.sort(function(a,b){ return a[arg] - b[arg]; }); }
    • 110. Reduce in Javascript function(values,arg) {     return values.sort(function(a,b){ return a[arg] - b[arg]; }); }
    • 111. Reduce in Javascript Phase- specific data function(values,arg) {     return values.sort(function(a,b){ return a[arg] - b[arg]; }); }
    • 112. Reduce in Javascript function(values,arg) {     return values.sort(function(a,b){ return a[arg] - b[arg]; }); }
    • 113. Reduce in Javascript function(values,arg) {     return values.sort(function(a,b){ return a[arg] - b[arg]; }); Custom sort on }
    • 114. Reduce in Javascript function(values,arg) {     return values.sort(function(a,b){ return a[arg] - b[arg]; }); }
    • 115. Reduce in Javascript function(values,arg) {     return values.sort(function(a,b){ return a[arg] - b[arg]; }); } Reduce potentially called multiple times!!
    • 116. Example Query
    • 117. Example Query {“inputs”: “goog”, “query”: [{“map”:{“language”:”javascript”, “name”: “App.findHighGreater”, “arg”: 600.0, “keep”: false}, {“reduce”:{“language”:”javascript”, “name”: “Riak.reduceMax”, “keep”: true}]}
    • 118. Example Query {“inputs”: “goog”, “query”: [{“map”:{“language”:”javascript”, “name”: “App.findHighGreater”, “arg”: 600.0, “keep”: false}, {“reduce”:{“language”:”javascript”, “name”: “Riak.reduceMax”, “keep”: true}]} Riak::MapReduce.new(c).add(‘goog’). map(‘App.findHighGreater’, :arg => 600.0). reduce(“Riak.reduceMax”, :keep => true).run
    • 119. Built-in Functions
    • 120. Built-in Functions • Riak.mapValues
    • 121. Built-in Functions • Riak.mapValues • Riak.mapValuesJson
    • 122. Built-in Functions • Riak.mapValues • Riak.mapValuesJson • Riak.mapByFields
    • 123. Built-in Functions • Riak.mapValues • Riak.mapValuesJson • Riak.mapByFields • Riak.reduceSum
    • 124. Built-in Functions • Riak.mapValues • Riak.mapValuesJson • Riak.mapByFields • Riak.reduceSum • Riak.reduceSort
    • 125. Built-in Functions • Riak.mapValues • Riak.mapValuesJson • Riak.mapByFields • Riak.reduceSum • Riak.reduceSort • Riak.reduceMin/reduceMax
    • 126. Ripple - ODM
    • 127. Document Models class Person include Ripple::Document property :name, String, :presence => true many :addresses many :friends, :class => Person end class Address include Ripple::EmbeddedDocument property :street, String end
    • 128. Rails Setup
    • 129. Rails Setup # Gemfile gem 'curb' # Faster HTTP gem 'yajl-ruby' # Faster JSON gem 'ripple'
    • 130. Rails Setup # Gemfile gem 'curb' # Faster HTTP gem 'yajl-ruby' # Faster JSON gem 'ripple'
    • 131. Rails Setup # Gemfile gem 'curb' # Faster HTTP gem 'yajl-ruby' # Faster JSON gem 'ripple' $ gem install curb yajl-ruby ripple
    • 132. Rails Setup (cont.)
    • 133. Rails Setup (cont.) # config/ripple.yml development: host: 127.0.0.1 port: 8098
    • 134. Rails Setup (cont.) # config/ripple.yml development: host: 127.0.0.1 port: 8098 # config/application.rb require 'ripple/railtie'
    • 135. Ripple Roadmap
    • 136. Ripple Roadmap • Testing server (edge on Github)
    • 137. Ripple Roadmap • Testing server (edge on Github) • Protocol Buffers support
    • 138. Ripple Roadmap • Testing server (edge on Github) • Protocol Buffers support • Streaming MapReduce
    • 139. Ripple Roadmap • Testing server (edge on Github) • Protocol Buffers support • Streaming MapReduce • Ripple-specific built-ins
    • 140. Ripple Roadmap • Testing server (edge on Github) • Protocol Buffers support • Streaming MapReduce • Ripple-specific built-ins • Better ActionView support (form_for)
    • 141. Happy Fun Demo Time
    • 142. Plug basho
    • 143. Plug Interested in learning about support, consulting, or Enterprise features? basho
    • 144. Plug Interested in learning about support, consulting, or Enterprise features? Email info@basho.com or go to http://www.basho.com/ contact.html to talk with us. basho
    • 145. Plug Interested in learning about support, consulting, or Enterprise features? Email info@basho.com or go to http://www.basho.com/ contact.html to talk with us. basho
    • 146. Plug Interested in learning about support, consulting, or Enterprise features? Email info@basho.com or go to http://www.basho.com/ contact.html to talk with us. www.basho.com sean@basho.com @seancribbs basho
    • 147. Questions