0
Introducing Riak and Ripple
          Sean Cribbs




Wednesday, April 14, 2010
system “whoami”




Wednesday, April 14, 2010
system “whoami”

               BCS, three years as a musician




Wednesday, April 14, 2010
system “whoami”

               BCS, three years as a musician
               Started with Ruby in early 2006




Wednesda...
system “whoami”

               BCS, three years as a musician
               Started with Ruby in early 2006
            ...
system “whoami”

               BCS, three years as a musician
               Started with Ruby in early 2006
            ...
system “whoami”

               BCS, three years as a musician
               Started with Ruby in early 2006
            ...
Wednesday, April 14, 2010
Wednesday, April 14, 2010
Rails Can’t Scale




Wednesday, April 14, 2010
Rails Can’t Scale
                              what does it mean?




Wednesday, April 14, 2010
Ruby Rails Can’t Scale
                            what does it mean?




Wednesday, April 14, 2010
MySQL
         Ruby Rails Can’t Scale
                            what does it mean?




Wednesday, April 14, 2010
insert straw man here


           MySQL
         Ruby Rails Can’t Scale
                            what does it mean?


...
Mythbusting “scalability”




Wednesday, April 14, 2010
Mythbusting “scalability”

               Scalability is not a yes/no question




Wednesday, April 14, 2010
Mythbusting “scalability”

               Scalability is not a yes/no question
               It’s a ratio of benefit to co...
Mythbusting “scalability”

               Scalability is not a yes/no question
               It’s a ratio of benefit to co...
Mythbusting “scalability”

               Scalability is not a yes/no question
               It’s a ratio of benefit to co...
Mythbusting “scalability”

               Scalability is not a yes/no question
               It’s a ratio of benefit to co...
Scaling goes both ways




Wednesday, April 14, 2010
Scaling goes both ways


               Up - high traffic, “Big Data”, multi-datacenter




Wednesday, April 14, 2010
Scaling goes both ways


               Up - high traffic, “Big Data”, multi-datacenter
               Down - laptop, set-t...
How do you scale?




Wednesday, April 14, 2010
How do you scale?


               Vertically - get bigger, expensive machines




Wednesday, April 14, 2010
How do you scale?


               Vertically - get bigger, expensive machines
               Horizontally - get more comm...
What is Riak?
                            (the horizontal scaling bits)




Wednesday, April 14, 2010
What is Riak?
                            (the horizontal scaling bits)

               Based on Amazon’s Dynamo (2007)


...
What is Riak?
                            (the horizontal scaling bits)

               Based on Amazon’s Dynamo (2007)
  ...
What is Riak?
                            (the horizontal scaling bits)

               Based on Amazon’s Dynamo (2007)
  ...
What is Riak?
                            (the horizontal scaling bits)

               Based on Amazon’s Dynamo (2007)
  ...
What is Riak?
                            (the horizontal scaling bits)

               Based on Amazon’s Dynamo (2007)
  ...
What is Riak?
                            (the horizontal scaling bits)

               Based on Amazon’s Dynamo (2007)
  ...
“O/RM is the Vietnam of Computer Science.”
                                 -- Ted Neward, 2004/6




Wednesday, April 14,...
“O/RM is the Vietnam of Computer Science.”
                                 -- Ted Neward, 2004/6
                        ...
“O/RM is the Vietnam of Computer Science.”
                                 -- Ted Neward, 2004/6
                        ...
Impedance mismatch




Wednesday, April 14, 2010
Impedance mismatch


               Relational = predicate logic, truth statements




Wednesday, April 14, 2010
Impedance mismatch


               Relational = predicate logic, truth statements
               Object = identity, state...
Pick your poison




Wednesday, April 14, 2010
Pick your poison

               Make the objects fit the database




Wednesday, April 14, 2010
Pick your poison

               Make the objects fit the database
                     Active Record




Wednesday, April ...
Pick your poison

               Make the objects fit the database
                     Active Record
               Make t...
Pick your poison

               Make the objects fit the database
                     Active Record
               Make t...
Middle ground




Wednesday, April 14, 2010
Middle ground

               Document databases (identity and state)




Wednesday, April 14, 2010
Middle ground

               Document databases (identity and state)
                     No JOINs - denormalized / compo...
Middle ground

               Document databases (identity and state)
                     No JOINs - denormalized / compo...
Middle ground

               Document databases (identity and state)
                     No JOINs - denormalized / compo...
What is Riak?
                            (the document database bits)




Wednesday, April 14, 2010
What is Riak?
                            (the document database bits)



               Store your objects as JSON (or an...
What is Riak?
                            (the document database bits)



               Store your objects as JSON (or an...
What is Riak?
                            (the document database bits)



               Store your objects as JSON (or an...
“DevOps” - agile infrastructure




Wednesday, April 14, 2010
“DevOps” - agile infrastructure
                            programming your infrastructure




Wednesday, April 14, 2010
“DevOps” - agile infrastructure
                            programming your infrastructure
                              ...
Befriend your sysadmin




Wednesday, April 14, 2010
Befriend your sysadmin


               Integrate with existing infrastructure




Wednesday, April 14, 2010
Befriend your sysadmin


               Integrate with existing infrastructure
               Reduce the amount of hands-o...
Befriend your sysadmin


               Integrate with existing infrastructure
               Reduce the amount of hands-o...
What is Riak?
                            (the ops-friendly bits)




Wednesday, April 14, 2010
What is Riak?
                            (the ops-friendly bits)

               Web-shaped storage




Wednesday, April ...
What is Riak?
                               (the ops-friendly bits)

               Web-shaped storage
                  ...
What is Riak?
                               (the ops-friendly bits)

               Web-shaped storage
                  ...
What is Riak?
                                   (the ops-friendly bits)

               Web-shaped storage
              ...
What is Riak?
                                   (the ops-friendly bits)

               Web-shaped storage
              ...
What is Riak?
                                   (the ops-friendly bits)

               Web-shaped storage
              ...
To review, Riak is...




Wednesday, April 14, 2010
To review, Riak is...

               Magical Horizontally-
               Scaling Unicorns




Wednesday, April 14, 2010
To review, Riak is...

               Magical Horizontally-
               Scaling Unicorns
               Application-Fri...
To review, Riak is...

               Magical Horizontally-
               Scaling Unicorns
               Application-Fri...
http://downloads.basho.com/riak/
                            hg clone http://hg.basho.com/riak/




Wednesday, April 14, 2...
gem install ripple
               git clone git://github.com/seancribbs/ripple.git




Wednesday, April 14, 2010
CRUD in Riak




Wednesday, April 14, 2010
CRUD in Riak


               PUT /riak/bucket/key (C,U) (also POST)




Wednesday, April 14, 2010
CRUD in Riak


               PUT /riak/bucket/key (C,U) (also POST)
               GET /riak/bucket/key (R)




Wednesday...
CRUD in Riak


               PUT /riak/bucket/key (C,U) (also POST)
               GET /riak/bucket/key (R)
             ...
CRUD Demo




Wednesday, April 14, 2010
Links




Wednesday, April 14, 2010
Links


               One-way, qualified “pointers” to other objects




Wednesday, April 14, 2010
Links


               One-way, qualified “pointers” to other objects
               “Link” header in HTTP




Wednesday, A...
Links


               One-way, qualified “pointers” to other objects
               “Link” header in HTTP
               <...
Link-walking




Wednesday, April 14, 2010
Link-walking
               GET /riak/bucket/key/[bucket,tag,keep]*




Wednesday, April 14, 2010
Link-walking
               GET /riak/bucket/key/[bucket,tag,keep]*
                     bucket = scope to a bucket




We...
Link-walking
               GET /riak/bucket/key/[bucket,tag,keep]*
                     bucket = scope to a bucket
      ...
Link-walking
               GET /riak/bucket/key/[bucket,tag,keep]*
                     bucket = scope to a bucket
      ...
Link-walking
               GET /riak/bucket/key/[bucket,tag,keep]*
                     bucket = scope to a bucket
      ...
Link-walking
               GET /riak/bucket/key/[bucket,tag,keep]*
                     bucket = scope to a bucket
      ...
Link-walking
               GET /riak/bucket/key/[bucket,tag,keep]*
                     bucket = scope to a bucket
      ...
Links Demo




Wednesday, April 14, 2010
Riak Map-Reduce
          I believe I did, Bob.

Wednesday, April 14, 2010
Riak Map-Reduce




Wednesday, April 14, 2010
Riak Map-Reduce

               Based on Google’s Map-Reduce idea




Wednesday, April 14, 2010
Riak Map-Reduce

               Based on Google’s Map-Reduce idea
               Some similarities to Hadoop and CouchDB

...
Riak Map-Reduce

               Based on Google’s Map-Reduce idea
               Some similarities to Hadoop and CouchDB
 ...
Riak Map-Reduce

               Based on Google’s Map-Reduce idea
               Some similarities to Hadoop and CouchDB
 ...
Map-Reduce Terminology




Wednesday, April 14, 2010
Map-Reduce Terminology

               Job: a query, composed of inputs and phases




Wednesday, April 14, 2010
Map-Reduce Terminology

               Job: a query, composed of inputs and phases
               Phase: a single map or r...
Map-Reduce Terminology

               Job: a query, composed of inputs and phases
               Phase: a single map or r...
Map-Reduce Terminology

               Job: a query, composed of inputs and phases
               Phase: a single map or r...
Map Phases




Wednesday, April 14, 2010
Map Phases


               Inputs are bucket-key pairs (with optional key-specific
               data)




Wednesday, Apr...
Map Phases


               Inputs are bucket-key pairs (with optional key-specific
               data)
               Mus...
Map Phases


               Inputs are bucket-key pairs (with optional key-specific
               data)
               Mus...
Map Built-ins




Wednesday, April 14, 2010
Map Built-ins


               Riak.mapValues




Wednesday, April 14, 2010
Map Built-ins


               Riak.mapValues
               Riak.mapValuesJson




Wednesday, April 14, 2010
Reduce Phases




Wednesday, April 14, 2010
Reduce Phases


               Performed on a single node




Wednesday, April 14, 2010
Reduce Phases


               Performed on a single node
               Two processes per reduce phase increase paralleli...
Reduce Phases


               Performed on a single node
               Two processes per reduce phase increase paralleli...
Reduce Built-ins




Wednesday, April 14, 2010
Reduce Built-ins


               Riak.reduceMin




Wednesday, April 14, 2010
Reduce Built-ins


               Riak.reduceMin
               Riak.reduceMax




Wednesday, April 14, 2010
Reduce Built-ins


               Riak.reduceMin
               Riak.reduceMax
               Riak.reduceSort




Wednesda...
Build a M/R Job




Wednesday, April 14, 2010
Build a M/R Job

               Specify inputs




Wednesday, April 14, 2010
Build a M/R Job

               Specify inputs
                     bucket/key, bucket/key/key-specific-arg, bucket




Wed...
Build a M/R Job

               Specify inputs
                     bucket/key, bucket/key/key-specific-arg, bucket
       ...
Build a M/R Job

               Specify inputs
                     bucket/key, bucket/key/key-specific-arg, bucket
       ...
Build a M/R Job

               Specify inputs
                     bucket/key, bucket/key/key-specific-arg, bucket
       ...
Map-Reduce on HTTP




Wednesday, April 14, 2010
Map-Reduce on HTTP


               Job is a JSON object




Wednesday, April 14, 2010
Map-Reduce on HTTP


               Job is a JSON object
               POST /mapred




Wednesday, April 14, 2010
A simple M/R job

           {“inputs”: “goog”,

              “query”: [{“map”:

                            {“language”:...
A simple M/R job

           {“inputs”: “goog”,

              “query”: [{“map”:

                            {“language”:...
Another M/R Job
                 {“inputs”: “stocks”,

                   “query”: [{“map”:{“language”:”javascript”,

    ...
Another M/R Job
                 {“inputs”: “stocks”,

                   “query”: [{“map”:{“language”:”javascript”,

    ...
Map-Reduce Demo




Wednesday, April 14, 2010
Ripple




Wednesday, April 14, 2010
Ripple

               Document-style persistence (like MongoMapper)




Wednesday, April 14, 2010
Ripple

               Document-style persistence (like MongoMapper)
               ActiveModel niceties




Wednesday, Ap...
Ripple

               Document-style persistence (like MongoMapper)
               ActiveModel niceties
                 ...
Ripple

               Document-style persistence (like MongoMapper)
               ActiveModel niceties
                 ...
Ripple

               Document-style persistence (like MongoMapper)
               ActiveModel niceties
                 ...
Ripple TODO




Wednesday, April 14, 2010
Ripple TODO

               Associations




Wednesday, April 14, 2010
Ripple TODO

               Associations
               Indexes




Wednesday, April 14, 2010
Ripple TODO

               Associations
               Indexes
               Use a property/method as the key




Wednes...
Ripple TODO

               Associations
               Indexes
               Use a property/method as the key
          ...
Ripple TODO

               Associations
               Indexes
               Use a property/method as the key
          ...
Ripple Demo




Wednesday, April 14, 2010
sean@basho.com
                                                 @seancribbs
                            “seancribbs” on Fr...
Upcoming SlideShare
Loading in...5
×

Introducing Riak and Ripple

11,930

Published on

An introduction to Basho's Riak distributed data store, and the Ripple client in Ruby. Code samples from the demos are here: http://gist.github.com/365791

Published in: Technology
0 Comments
25 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
11,930
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
278
Comments
0
Likes
25
Embeds 0
No embeds

No notes for slide

Transcript of "Introducing Riak and Ripple"

  1. 1. Introducing Riak and Ripple Sean Cribbs Wednesday, April 14, 2010
  2. 2. system “whoami” Wednesday, April 14, 2010
  3. 3. system “whoami” BCS, three years as a musician Wednesday, April 14, 2010
  4. 4. system “whoami” BCS, three years as a musician Started with Ruby in early 2006 Wednesday, April 14, 2010
  5. 5. system “whoami” BCS, three years as a musician Started with Ruby in early 2006 Rails freelancer 2007-2010 Wednesday, April 14, 2010
  6. 6. system “whoami” BCS, three years as a musician Started with Ruby in early 2006 Rails freelancer 2007-2010 Radiant CMS Wednesday, April 14, 2010
  7. 7. system “whoami” BCS, three years as a musician Started with Ruby in early 2006 Rails freelancer 2007-2010 Radiant CMS March 2010 - Developer Advocate basho Wednesday, April 14, 2010
  8. 8. Wednesday, April 14, 2010
  9. 9. Wednesday, April 14, 2010
  10. 10. Rails Can’t Scale Wednesday, April 14, 2010
  11. 11. Rails Can’t Scale what does it mean? Wednesday, April 14, 2010
  12. 12. Ruby Rails Can’t Scale what does it mean? Wednesday, April 14, 2010
  13. 13. MySQL Ruby Rails Can’t Scale what does it mean? Wednesday, April 14, 2010
  14. 14. insert straw man here MySQL Ruby Rails Can’t Scale what does it mean? Wednesday, April 14, 2010
  15. 15. Mythbusting “scalability” Wednesday, April 14, 2010
  16. 16. Mythbusting “scalability” Scalability is not a yes/no question Wednesday, April 14, 2010
  17. 17. Mythbusting “scalability” Scalability is not a yes/no question It’s a ratio of benefit to cost Wednesday, April 14, 2010
  18. 18. Mythbusting “scalability” Scalability is not a yes/no question It’s a ratio of benefit to cost Benefits: low latency, throughput, uptime, concurrency, reliability Wednesday, April 14, 2010
  19. 19. Mythbusting “scalability” Scalability is not a yes/no question It’s a ratio of benefit to cost Benefits: low latency, throughput, uptime, concurrency, reliability Costs: CPU, RAM, disk, bandwidth, power, hw/sw Wednesday, April 14, 2010
  20. 20. Mythbusting “scalability” Scalability is not a yes/no question It’s a ratio of benefit to cost Benefits: low latency, throughput, uptime, concurrency, reliability Costs: CPU, RAM, disk, bandwidth, power, hw/sw scalability = bang for your buck (ROI) Wednesday, April 14, 2010
  21. 21. Scaling goes both ways Wednesday, April 14, 2010
  22. 22. Scaling goes both ways Up - high traffic, “Big Data”, multi-datacenter Wednesday, April 14, 2010
  23. 23. Scaling goes both ways Up - high traffic, “Big Data”, multi-datacenter Down - laptop, set-top, mobile Wednesday, April 14, 2010
  24. 24. How do you scale? Wednesday, April 14, 2010
  25. 25. How do you scale? Vertically - get bigger, expensive machines Wednesday, April 14, 2010
  26. 26. How do you scale? Vertically - get bigger, expensive machines Horizontally - get more commodity machines Wednesday, April 14, 2010
  27. 27. What is Riak? (the horizontal scaling bits) Wednesday, April 14, 2010
  28. 28. What is Riak? (the horizontal scaling bits) Based on Amazon’s Dynamo (2007) Wednesday, April 14, 2010
  29. 29. What is Riak? (the horizontal scaling bits) Based on Amazon’s Dynamo (2007) key-value storage Wednesday, April 14, 2010
  30. 30. What is Riak? (the horizontal scaling bits) Based on Amazon’s Dynamo (2007) key-value storage masterless, peer-to-peer replication Wednesday, April 14, 2010
  31. 31. What is Riak? (the horizontal scaling bits) Based on Amazon’s Dynamo (2007) key-value storage masterless, peer-to-peer replication consistent hashing Wednesday, April 14, 2010
  32. 32. What is Riak? (the horizontal scaling bits) Based on Amazon’s Dynamo (2007) key-value storage masterless, peer-to-peer replication consistent hashing eventual consistency Wednesday, April 14, 2010
  33. 33. What is Riak? (the horizontal scaling bits) Based on Amazon’s Dynamo (2007) key-value storage masterless, peer-to-peer replication consistent hashing eventual consistency failover - quorums, hinted handoff Wednesday, April 14, 2010
  34. 34. “O/RM is the Vietnam of Computer Science.” -- Ted Neward, 2004/6 Wednesday, April 14, 2010
  35. 35. “O/RM is the Vietnam of Computer Science.” -- Ted Neward, 2004/6 slippery slope Wednesday, April 14, 2010
  36. 36. “O/RM is the Vietnam of Computer Science.” -- Ted Neward, 2004/6 slippery slope diminishing returns Wednesday, April 14, 2010
  37. 37. Impedance mismatch Wednesday, April 14, 2010
  38. 38. Impedance mismatch Relational = predicate logic, truth statements Wednesday, April 14, 2010
  39. 39. Impedance mismatch Relational = predicate logic, truth statements Object = identity, state, behavior, encapsulation Wednesday, April 14, 2010
  40. 40. Pick your poison Wednesday, April 14, 2010
  41. 41. Pick your poison Make the objects fit the database Wednesday, April 14, 2010
  42. 42. Pick your poison Make the objects fit the database Active Record Wednesday, April 14, 2010
  43. 43. Pick your poison Make the objects fit the database Active Record Make the database fit the objects Wednesday, April 14, 2010
  44. 44. Pick your poison Make the objects fit the database Active Record Make the database fit the objects OODBMS, Graph DBs Wednesday, April 14, 2010
  45. 45. Middle ground Wednesday, April 14, 2010
  46. 46. Middle ground Document databases (identity and state) Wednesday, April 14, 2010
  47. 47. Middle ground Document databases (identity and state) No JOINs - denormalized / composed Wednesday, April 14, 2010
  48. 48. Middle ground Document databases (identity and state) No JOINs - denormalized / composed Loosely structured Wednesday, April 14, 2010
  49. 49. Middle ground Document databases (identity and state) No JOINs - denormalized / composed Loosely structured Friendly transport formats, e.g. JSON Wednesday, April 14, 2010
  50. 50. What is Riak? (the document database bits) Wednesday, April 14, 2010
  51. 51. What is Riak? (the document database bits) Store your objects as JSON (or any format) Wednesday, April 14, 2010
  52. 52. What is Riak? (the document database bits) Store your objects as JSON (or any format) Link between objects (like hypertext) Wednesday, April 14, 2010
  53. 53. What is Riak? (the document database bits) Store your objects as JSON (or any format) Link between objects (like hypertext) No SQL - query with Javascript Map-Reduce Wednesday, April 14, 2010
  54. 54. “DevOps” - agile infrastructure Wednesday, April 14, 2010
  55. 55. “DevOps” - agile infrastructure programming your infrastructure Wednesday, April 14, 2010
  56. 56. “DevOps” - agile infrastructure programming your infrastructure rapid “cloud” deployments Wednesday, April 14, 2010
  57. 57. Befriend your sysadmin Wednesday, April 14, 2010
  58. 58. Befriend your sysadmin Integrate with existing infrastructure Wednesday, April 14, 2010
  59. 59. Befriend your sysadmin Integrate with existing infrastructure Reduce the amount of hands-on work Wednesday, April 14, 2010
  60. 60. Befriend your sysadmin Integrate with existing infrastructure Reduce the amount of hands-on work Have a growth plan Wednesday, April 14, 2010
  61. 61. What is Riak? (the ops-friendly bits) Wednesday, April 14, 2010
  62. 62. What is Riak? (the ops-friendly bits) Web-shaped storage Wednesday, April 14, 2010
  63. 63. What is Riak? (the ops-friendly bits) Web-shaped storage Store data in its original format Wednesday, April 14, 2010
  64. 64. What is Riak? (the ops-friendly bits) Web-shaped storage Store data in its original format It’s just HTTP - same techniques apply Wednesday, April 14, 2010
  65. 65. What is Riak? (the ops-friendly bits) Web-shaped storage Store data in its original format It’s just HTTP - same techniques apply load balancing, proxy caches, round-robin DNS Wednesday, April 14, 2010
  66. 66. What is Riak? (the ops-friendly bits) Web-shaped storage Store data in its original format It’s just HTTP - same techniques apply load balancing, proxy caches, round-robin DNS No node is special - grow horizontally Wednesday, April 14, 2010
  67. 67. What is Riak? (the ops-friendly bits) Web-shaped storage Store data in its original format It’s just HTTP - same techniques apply load balancing, proxy caches, round-robin DNS No node is special - grow horizontally Get some sleep, fix it in the morning Wednesday, April 14, 2010
  68. 68. To review, Riak is... Wednesday, April 14, 2010
  69. 69. To review, Riak is... Magical Horizontally- Scaling Unicorns Wednesday, April 14, 2010
  70. 70. To review, Riak is... Magical Horizontally- Scaling Unicorns Application-Friendly Rainbows in the Cloud Wednesday, April 14, 2010
  71. 71. To review, Riak is... Magical Horizontally- Scaling Unicorns Application-Friendly Rainbows in the Cloud Data-Shredding MapReduce Ninjas Wednesday, April 14, 2010
  72. 72. http://downloads.basho.com/riak/ hg clone http://hg.basho.com/riak/ Wednesday, April 14, 2010
  73. 73. gem install ripple git clone git://github.com/seancribbs/ripple.git Wednesday, April 14, 2010
  74. 74. CRUD in Riak Wednesday, April 14, 2010
  75. 75. CRUD in Riak PUT /riak/bucket/key (C,U) (also POST) Wednesday, April 14, 2010
  76. 76. CRUD in Riak PUT /riak/bucket/key (C,U) (also POST) GET /riak/bucket/key (R) Wednesday, April 14, 2010
  77. 77. CRUD in Riak PUT /riak/bucket/key (C,U) (also POST) GET /riak/bucket/key (R) DELETE /riak/bucket/key (D) Wednesday, April 14, 2010
  78. 78. CRUD Demo Wednesday, April 14, 2010
  79. 79. Links Wednesday, April 14, 2010
  80. 80. Links One-way, qualified “pointers” to other objects Wednesday, April 14, 2010
  81. 81. Links One-way, qualified “pointers” to other objects “Link” header in HTTP Wednesday, April 14, 2010
  82. 82. Links One-way, qualified “pointers” to other objects “Link” header in HTTP </riak/bucket/key>; riaktag=”tag” Wednesday, April 14, 2010
  83. 83. Link-walking Wednesday, April 14, 2010
  84. 84. Link-walking GET /riak/bucket/key/[bucket,tag,keep]* Wednesday, April 14, 2010
  85. 85. Link-walking GET /riak/bucket/key/[bucket,tag,keep]* bucket = scope to a bucket Wednesday, April 14, 2010
  86. 86. Link-walking GET /riak/bucket/key/[bucket,tag,keep]* bucket = scope to a bucket tag = scope to a tag Wednesday, April 14, 2010
  87. 87. Link-walking GET /riak/bucket/key/[bucket,tag,keep]* bucket = scope to a bucket tag = scope to a tag keep = return results (last one always) Wednesday, April 14, 2010
  88. 88. Link-walking GET /riak/bucket/key/[bucket,tag,keep]* bucket = scope to a bucket tag = scope to a tag keep = return results (last one always) “_” means match anything Wednesday, April 14, 2010
  89. 89. Link-walking GET /riak/bucket/key/[bucket,tag,keep]* bucket = scope to a bucket tag = scope to a tag keep = return results (last one always) “_” means match anything Example: Wednesday, April 14, 2010
  90. 90. Link-walking GET /riak/bucket/key/[bucket,tag,keep]* bucket = scope to a bucket tag = scope to a tag keep = return results (last one always) “_” means match anything Example: GET /riak/artists/TheBeatles/albums,_,1/tracks,_,1 Wednesday, April 14, 2010
  91. 91. Links Demo Wednesday, April 14, 2010
  92. 92. Riak Map-Reduce I believe I did, Bob. Wednesday, April 14, 2010
  93. 93. Riak Map-Reduce Wednesday, April 14, 2010
  94. 94. Riak Map-Reduce Based on Google’s Map-Reduce idea Wednesday, April 14, 2010
  95. 95. Riak Map-Reduce Based on Google’s Map-Reduce idea Some similarities to Hadoop and CouchDB Wednesday, April 14, 2010
  96. 96. Riak Map-Reduce Based on Google’s Map-Reduce idea Some similarities to Hadoop and CouchDB High data-locality Wednesday, April 14, 2010
  97. 97. Riak Map-Reduce Based on Google’s Map-Reduce idea Some similarities to Hadoop and CouchDB High data-locality Phases can be Javascript or Erlang Wednesday, April 14, 2010
  98. 98. Map-Reduce Terminology Wednesday, April 14, 2010
  99. 99. Map-Reduce Terminology Job: a query, composed of inputs and phases Wednesday, April 14, 2010
  100. 100. Map-Reduce Terminology Job: a query, composed of inputs and phases Phase: a single map or reduce function application Wednesday, April 14, 2010
  101. 101. Map-Reduce Terminology Job: a query, composed of inputs and phases Phase: a single map or reduce function application Map: phase applied to each item - filter, transform Wednesday, April 14, 2010
  102. 102. Map-Reduce Terminology Job: a query, composed of inputs and phases Phase: a single map or reduce function application Map: phase applied to each item - filter, transform Reduce: phase applied to the collection - collate, aggregate, compute Wednesday, April 14, 2010
  103. 103. Map Phases Wednesday, April 14, 2010
  104. 104. Map Phases Inputs are bucket-key pairs (with optional key-specific data) Wednesday, April 14, 2010
  105. 105. Map Phases Inputs are bucket-key pairs (with optional key-specific data) Must return a list Wednesday, April 14, 2010
  106. 106. Map Phases Inputs are bucket-key pairs (with optional key-specific data) Must return a list Parallel results are aggregated Wednesday, April 14, 2010
  107. 107. Map Built-ins Wednesday, April 14, 2010
  108. 108. Map Built-ins Riak.mapValues Wednesday, April 14, 2010
  109. 109. Map Built-ins Riak.mapValues Riak.mapValuesJson Wednesday, April 14, 2010
  110. 110. Reduce Phases Wednesday, April 14, 2010
  111. 111. Reduce Phases Performed on a single node Wednesday, April 14, 2010
  112. 112. Reduce Phases Performed on a single node Two processes per reduce phase increase parallelism Wednesday, April 14, 2010
  113. 113. Reduce Phases Performed on a single node Two processes per reduce phase increase parallelism Must return a list Wednesday, April 14, 2010
  114. 114. Reduce Built-ins Wednesday, April 14, 2010
  115. 115. Reduce Built-ins Riak.reduceMin Wednesday, April 14, 2010
  116. 116. Reduce Built-ins Riak.reduceMin Riak.reduceMax Wednesday, April 14, 2010
  117. 117. Reduce Built-ins Riak.reduceMin Riak.reduceMax Riak.reduceSort Wednesday, April 14, 2010
  118. 118. Build a M/R Job Wednesday, April 14, 2010
  119. 119. Build a M/R Job Specify inputs Wednesday, April 14, 2010
  120. 120. Build a M/R Job Specify inputs bucket/key, bucket/key/key-specific-arg, bucket Wednesday, April 14, 2010
  121. 121. Build a M/R Job Specify inputs bucket/key, bucket/key/key-specific-arg, bucket Specify phases Wednesday, April 14, 2010
  122. 122. Build a M/R Job Specify inputs bucket/key, bucket/key/key-specific-arg, bucket Specify phases Each can receive a static argument Wednesday, April 14, 2010
  123. 123. Build a M/R Job Specify inputs bucket/key, bucket/key/key-specific-arg, bucket Specify phases Each can receive a static argument Each can return intermediate results Wednesday, April 14, 2010
  124. 124. Map-Reduce on HTTP Wednesday, April 14, 2010
  125. 125. Map-Reduce on HTTP Job is a JSON object Wednesday, April 14, 2010
  126. 126. Map-Reduce on HTTP Job is a JSON object POST /mapred Wednesday, April 14, 2010
  127. 127. A simple M/R job {“inputs”: “goog”, “query”: [{“map”: {“language”:”javascript”, “name”: “Riak.mapValuesJson”, “keep”: true}]} Wednesday, April 14, 2010
  128. 128. A simple M/R job {“inputs”: “goog”, “query”: [{“map”: {“language”:”javascript”, “name”: “Riak.mapValuesJson”, “keep”: true}]} Riak::MapReduce.new(c).add(‘goog’). map(‘Riak.mapValuesJson’, :keep => true).run Wednesday, April 14, 2010
  129. 129. Another M/R Job {“inputs”: “stocks”, “query”: [{“map”:{“language”:”javascript”, “name”: “App.extractTickers”, “arg”: “GOOG”, “keep”: false}, {“reduce”:{“language”:”javascript, “name”: “Riak.reduceMin”, “keep”: true}]} Wednesday, April 14, 2010
  130. 130. Another M/R Job {“inputs”: “stocks”, “query”: [{“map”:{“language”:”javascript”, “name”: “App.extractTickers”, “arg”: “GOOG”, “keep”: false}, {“reduce”:{“language”:”javascript, “name”: “Riak.reduceMin”, “keep”: true}]} Riak::MapReduce.new(c).add(‘stocks’). map(‘App.extractTickers’, :arg =>”GOOG”). reduce(“Riak.reduceMin”, :keep => true).run Wednesday, April 14, 2010
  131. 131. Map-Reduce Demo Wednesday, April 14, 2010
  132. 132. Ripple Wednesday, April 14, 2010
  133. 133. Ripple Document-style persistence (like MongoMapper) Wednesday, April 14, 2010
  134. 134. Ripple Document-style persistence (like MongoMapper) ActiveModel niceties Wednesday, April 14, 2010
  135. 135. Ripple Document-style persistence (like MongoMapper) ActiveModel niceties Callbacks Wednesday, April 14, 2010
  136. 136. Ripple Document-style persistence (like MongoMapper) ActiveModel niceties Callbacks Validations Wednesday, April 14, 2010
  137. 137. Ripple Document-style persistence (like MongoMapper) ActiveModel niceties Callbacks Validations ActionPack compatibility Wednesday, April 14, 2010
  138. 138. Ripple TODO Wednesday, April 14, 2010
  139. 139. Ripple TODO Associations Wednesday, April 14, 2010
  140. 140. Ripple TODO Associations Indexes Wednesday, April 14, 2010
  141. 141. Ripple TODO Associations Indexes Use a property/method as the key Wednesday, April 14, 2010
  142. 142. Ripple TODO Associations Indexes Use a property/method as the key Dynamic finders Wednesday, April 14, 2010
  143. 143. Ripple TODO Associations Indexes Use a property/method as the key Dynamic finders Session/cache stores Wednesday, April 14, 2010
  144. 144. Ripple Demo Wednesday, April 14, 2010
  145. 145. sean@basho.com @seancribbs “seancribbs” on Freenode IRC #riak Questions? http://wiki.basho.com riak-users@lists.basho.com Wednesday, April 14, 2010
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×