Riak from Small to Large
Strange Loop · October 2010

    Rusty Klophaus (@rklophaus)
         Basho Technologies
Riak is a Dynamo-inspired,
open-sourced, key/value datastore
built to scale predictably and easily.




                  ...
Riak is a Dynamo-inspired,
open-sourced, key/value datastore
built to scale predictably and easily.
 Your ops guy: calm, r...
NoSQL




  4
5
http://www.flickr.com/photos/ghewgill/129950124
NoSQL has changed like Keanu hasn’t.




                           6
        http://www.youtube.com/watch?v=nEubt6HpGhs
Companies, Funding, Customers, Awareness...




                                 7
         http://www.flickr.com/photos/fr...
“NoSQL? Yeah, it’s this really cool technology,
   you probably haven’t heard of it yet.”




                            ...
“Johnson! Get me a memo on
    this ‘No sequal’ thing!”




                          9
    http://www.flickr.com/photos/pr...
Why?



 10
Because NoSQL is driven by
 narcissism, voyeurism, and
         materialism.
The catalyst for NoSQL
      is the data
generated or consumed
      by humans.


          12
Photos, music, and video...




            13
Constantly connected devices...




              14
Social Networks...




                       15
http://www.flickr.com/photos/hanspoldoja/5001818922
Your Grandmama...




                   16
 http://www.flickr.com/photos/mega/7358278
Requirements That Confound an RDBMS
Large Data Objects                  Changing Schemas
 BLOBs?                          ...
The roots of NoSQL are practical,
         not academic.




               18
Turning Point


Scratch an Itch ➮ Build a Company



                19
Four Important Things
1. Broad Knowledge
 Learn from the past, both DB *and* non-DB worlds.

 •   VP Technical Services, A...
Four Important Things
  2. Strong Community
    Stay in touch with real problems.




          Mark Phillips
Basho Commun...
Four Important Things
3. Agility
 Evolve to solve new problems.




                         22
Four Important Things
4. Restraint
 No solution can do it all well.




                            23
Back to Riak




     24
What characteristics of Riak become
important at different cluster sizes?




                 25
Single Box Riak



          Simplicity


                  26
Single Box Riak
Simple Key/Value Store, Flexible Schema

client = RiakClient(host, port)
bucket = client.bucket('mybucket'...
Single Box Riak
Clients in Many Languages




                      28
Single Box Riak
Predictable Development Interface


                                Riak        Riak
     Riak       ===  ...
Single Box Riak
Configurable Buckets



                      !=

       Profile               Audio
       Bucket          ...
Single Box Riak
Links (Lightweight Data Relations)




     Profile         Projects         Comments



                  ...
Small Riak Cluster



 Parallelism and Power


                32
Small Riak Cluster
Expanding the Cluster



bin/riak-admin join riak@hostname




                        33
Small Riak Cluster
Expanding the Cluster



bin/riak-admin join riak@hostname




                        34
Small Riak Cluster
Expanding the Cluster



bin/riak-admin join riak@hostname

                               Hello ladies...
Small Riak Cluster
Distributed Queries




                      35
Small Riak Cluster
Distributed Queries




                      36
Small Riak Cluster
 Javascript-Based Map/Reduce

          Map
          Map
                     Reduce
          Map
Key...
Small Riak Cluster
Well-Behaved HTTP
                                              Client
     Client

                   ...
Small Riak Cluster
Riak Full-Text Search
bin/search-cmd install mybucket

"converse AND category:shoes"


                ...
Large Riak Cluster



         Operations


                40
Large Riak Cluster
No Special Nodes



              Special
              Nodes

                   41
Large Riak Cluster
Laugh at Machine Failure


                  Riak        Riak
                Riak
                    ...
Large Riak Cluster
Scale by Adding Nodes
                   Riak              Riak
          Riak
                  Riak  ...
Large Riak Cluster
Packages or Self-Contained Installation


sudo dpkg -i riak_0.13.0-2_amd64.deb

-or-

unzip riak.zip
ri...
$0.00
Order now, and     $0.00
  we’ll include an
extra Riak for free!*




*Offer limited to open-
source editions of Riak.
Enterprise ($$$ / ~10’s of boxes)




•   On-Call Support 24x7x365
•   Management Tools
•   SNMP Monitoring
•   Multi-site...
Enterprise ($$$ / ~10’s of boxes)




•   On-Call Support 24x7x365
•   Management Tools
•   SNMP Monitoring
•   Multi-site...
Thanks! Questions?
Next Steps
 Read more at http://wiki.basho.com
 Download binaries from http://downloads.basho.com
 Get ...
Additional Slides
Tutorial
Get Riak
Connect with a Client
Store Data
Store an Object with Links
Linkwalking
Map/Reduce


                   ...
Get Riak
Download
 http://downloads.basho.com/riak

Start Riak
 cd riak
 bin/riak start




                     52
Connect with Python
# Code is at http://hg.basho.com
import riak

# Connect
client = riak.RiakClient('127.0.0.1', 8098)


...
Store Data
mybucket = client.bucket('mybucket')

# Create an object...
obj = mybucket.new('myobject')
obj.set_data({ 'foo'...
Store an Object with Links
bands = client.bucket('bands')
albums = client.bucket('albums')
members = client.bucket('member...
Linkwalking
# Get the albums...
albums = obj.link('albums').run()

# Get the songs (assumes data is present)...
songs = ob...
Map/Reduce
# Count the number of sales...
result = obj 
    .link('albums') 
    .map("function(v) { return [v.values[0].d...
Upcoming SlideShare
Loading in...5
×

Riak - From Small to Large

5,265

Published on

Riak ( http://wiki.basho.com ), a Dynamo-inspired, open-source key/value datastore, was built to scale from a single machine to a 100+ server cluster without driving you or your operations team crazy. This presentation discusses the characteristics of Riak that become important in small, medium, and large clusters.

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,265
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
41
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Riak - From Small to Large

  1. 1. Riak from Small to Large Strange Loop · October 2010 Rusty Klophaus (@rklophaus) Basho Technologies
  2. 2. Riak is a Dynamo-inspired, open-sourced, key/value datastore built to scale predictably and easily. 2
  3. 3. Riak is a Dynamo-inspired, open-sourced, key/value datastore built to scale predictably and easily. Your ops guy: calm, relaxed, and wearing a party hat. 3
  4. 4. NoSQL 4
  5. 5. 5 http://www.flickr.com/photos/ghewgill/129950124
  6. 6. NoSQL has changed like Keanu hasn’t. 6 http://www.youtube.com/watch?v=nEubt6HpGhs
  7. 7. Companies, Funding, Customers, Awareness... 7 http://www.flickr.com/photos/free-stock/4791385567
  8. 8. “NoSQL? Yeah, it’s this really cool technology, you probably haven’t heard of it yet.” 8 http://www.flickr.com/photos/slava/285199203
  9. 9. “Johnson! Get me a memo on this ‘No sequal’ thing!” 9 http://www.flickr.com/photos/proimos/4045973322
  10. 10. Why? 10
  11. 11. Because NoSQL is driven by narcissism, voyeurism, and materialism.
  12. 12. The catalyst for NoSQL is the data generated or consumed by humans. 12
  13. 13. Photos, music, and video... 13
  14. 14. Constantly connected devices... 14
  15. 15. Social Networks... 15 http://www.flickr.com/photos/hanspoldoja/5001818922
  16. 16. Your Grandmama... 16 http://www.flickr.com/photos/mega/7358278
  17. 17. Requirements That Confound an RDBMS Large Data Objects Changing Schemas BLOBs? Auto-generated DDL? Generic Columns? High-Availability Systems Non-Standard Access Bi-Directional Replication? Recursive SQL? Log shipping? High-Capacity Systems Shards? Vertical Partitioning? 17
  18. 18. The roots of NoSQL are practical, not academic. 18
  19. 19. Turning Point Scratch an Itch ➮ Build a Company 19
  20. 20. Four Important Things 1. Broad Knowledge Learn from the past, both DB *and* non-DB worlds. • VP Technical Services, Akamai • Senior Architect, Akamai • Chief of Operations, Snapfish • Release Manager, Engine Yard • Bob Ippolito (CTO/CoFounder MochiMedia) • Dr. Eric Brewer (CAP Theorum) 20
  21. 21. Four Important Things 2. Strong Community Stay in touch with real problems. Mark Phillips Basho Community Manager 21
  22. 22. Four Important Things 3. Agility Evolve to solve new problems. 22
  23. 23. Four Important Things 4. Restraint No solution can do it all well. 23
  24. 24. Back to Riak 24
  25. 25. What characteristics of Riak become important at different cluster sizes? 25
  26. 26. Single Box Riak Simplicity 26
  27. 27. Single Box Riak Simple Key/Value Store, Flexible Schema client = RiakClient(host, port) bucket = client.bucket('mybucket') obj = bucket.new('foo', 'bar') obj.store() obj2 = bucket.get(‘foo’) 27
  28. 28. Single Box Riak Clients in Many Languages 28
  29. 29. Single Box Riak Predictable Development Interface Riak Riak Riak === Riak Riak Riak Development Production 29
  30. 30. Single Box Riak Configurable Buckets != Profile Audio Bucket Bucket 30
  31. 31. Single Box Riak Links (Lightweight Data Relations) Profile Projects Comments 31
  32. 32. Small Riak Cluster Parallelism and Power 32
  33. 33. Small Riak Cluster Expanding the Cluster bin/riak-admin join riak@hostname 33
  34. 34. Small Riak Cluster Expanding the Cluster bin/riak-admin join riak@hostname 34
  35. 35. Small Riak Cluster Expanding the Cluster bin/riak-admin join riak@hostname Hello ladies, can your datastore grow like mine? *monocle smile* 34
  36. 36. Small Riak Cluster Distributed Queries 35
  37. 37. Small Riak Cluster Distributed Queries 36
  38. 38. Small Riak Cluster Javascript-Based Map/Reduce Map Map Reduce Map Keys Reduce Output Map Reduce ... Map 37
  39. 39. Small Riak Cluster Well-Behaved HTTP Client Client nginx Browser Client Riak Riak Squid Riak Riak Riak 38 nginx proxy config - http://gist.github.com/323048
  40. 40. Small Riak Cluster Riak Full-Text Search bin/search-cmd install mybucket "converse AND category:shoes" Riak Riak Riak Riak Riak 39 nginx proxy config - http://gist.github.com/323048
  41. 41. Large Riak Cluster Operations 40
  42. 42. Large Riak Cluster No Special Nodes Special Nodes 41
  43. 43. Large Riak Cluster Laugh at Machine Failure Riak Riak Riak Riak X Riak 42
  44. 44. Large Riak Cluster Scale by Adding Nodes Riak Riak Riak Riak Riak Riak Riak Riak Riak Riak Riak Riak 43
  45. 45. Large Riak Cluster Packages or Self-Contained Installation sudo dpkg -i riak_0.13.0-2_amd64.deb -or- unzip riak.zip riak/bin/riak start 44
  46. 46. $0.00
  47. 47. Order now, and $0.00 we’ll include an extra Riak for free!* *Offer limited to open- source editions of Riak.
  48. 48. Enterprise ($$$ / ~10’s of boxes) • On-Call Support 24x7x365 • Management Tools • SNMP Monitoring • Multi-site Replication 47
  49. 49. Enterprise ($$$ / ~10’s of boxes) • On-Call Support 24x7x365 • Management Tools • SNMP Monitoring • Multi-site Replication 48
  50. 50. Thanks! Questions? Next Steps Read more at http://wiki.basho.com Download binaries from http://downloads.basho.com Get source code from http://hg.basho.com Join the public mailing list at riak-users@lists.basho.com Mailing list archives at http://riak.markmail.org Thanks! Rusty Klophaus (@rklophaus) Mark Phillips (@pharkmillups) 49
  51. 51. Additional Slides
  52. 52. Tutorial Get Riak Connect with a Client Store Data Store an Object with Links Linkwalking Map/Reduce 51
  53. 53. Get Riak Download http://downloads.basho.com/riak Start Riak cd riak bin/riak start 52
  54. 54. Connect with Python # Code is at http://hg.basho.com import riak # Connect client = riak.RiakClient('127.0.0.1', 8098) 53
  55. 55. Store Data mybucket = client.bucket('mybucket') # Create an object... obj = mybucket.new('myobject') obj.set_data({ 'foo' : 1, 'bar' : 2 }) obj.store() # Read the object... obj = mybucket.get('myobject') print obj.get_data() # Or, open a web browser... http://127.0.0.1:8098/riak/mybucket/myobject 54
  56. 56. Store an Object with Links bands = client.bucket('bands') albums = client.bucket('albums') members = client.bucket('members') # Store a band, link to album and members... obj = bands.new('Winger') .add_link(albums.new('Pull', 1275922).store()) .add_link(albums.new('IV', 542731).store()) .add_link(albums.new('Karma', 200170).store()) .add_link(members.new('Kip Winger').store()) .add_link(members.new('Reb Beach').store()) .add_link(members.new('John Roth').store()) .add_link(members.new('Rod M.').store()) .store() 55
  57. 57. Linkwalking # Get the albums... albums = obj.link('albums').run() # Get the songs (assumes data is present)... songs = obj.link('albums').link('songs').run() # Get the members... members = riak.MapReduce(client) .add('bands', 'Winger') .link('members') .run() 56
  58. 58. Map/Reduce # Count the number of sales... result = obj .link('albums') .map("function(v) { return [v.values[0].data]; }") .reduce("Riak.reduceSum") .run() 57
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×