Riak - From Small to Large

5,393
-1

Published on

Riak ( http://wiki.basho.com ), a Dynamo-inspired, open-source key/value datastore, was built to scale from a single machine to a 100+ server cluster without driving you or your operations team crazy. This presentation discusses the characteristics of Riak that become important in small, medium, and large clusters.

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,393
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
41
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Riak - From Small to Large

  1. 1. Riak from Small to Large Strange Loop · October 2010 Rusty Klophaus (@rklophaus) Basho Technologies
  2. 2. Riak is a Dynamo-inspired, open-sourced, key/value datastore built to scale predictably and easily. 2
  3. 3. Riak is a Dynamo-inspired, open-sourced, key/value datastore built to scale predictably and easily. Your ops guy: calm, relaxed, and wearing a party hat. 3
  4. 4. NoSQL 4
  5. 5. 5 http://www.flickr.com/photos/ghewgill/129950124
  6. 6. NoSQL has changed like Keanu hasn’t. 6 http://www.youtube.com/watch?v=nEubt6HpGhs
  7. 7. Companies, Funding, Customers, Awareness... 7 http://www.flickr.com/photos/free-stock/4791385567
  8. 8. “NoSQL? Yeah, it’s this really cool technology, you probably haven’t heard of it yet.” 8 http://www.flickr.com/photos/slava/285199203
  9. 9. “Johnson! Get me a memo on this ‘No sequal’ thing!” 9 http://www.flickr.com/photos/proimos/4045973322
  10. 10. Why? 10
  11. 11. Because NoSQL is driven by narcissism, voyeurism, and materialism.
  12. 12. The catalyst for NoSQL is the data generated or consumed by humans. 12
  13. 13. Photos, music, and video... 13
  14. 14. Constantly connected devices... 14
  15. 15. Social Networks... 15 http://www.flickr.com/photos/hanspoldoja/5001818922
  16. 16. Your Grandmama... 16 http://www.flickr.com/photos/mega/7358278
  17. 17. Requirements That Confound an RDBMS Large Data Objects Changing Schemas BLOBs? Auto-generated DDL? Generic Columns? High-Availability Systems Non-Standard Access Bi-Directional Replication? Recursive SQL? Log shipping? High-Capacity Systems Shards? Vertical Partitioning? 17
  18. 18. The roots of NoSQL are practical, not academic. 18
  19. 19. Turning Point Scratch an Itch ➮ Build a Company 19
  20. 20. Four Important Things 1. Broad Knowledge Learn from the past, both DB *and* non-DB worlds. • VP Technical Services, Akamai • Senior Architect, Akamai • Chief of Operations, Snapfish • Release Manager, Engine Yard • Bob Ippolito (CTO/CoFounder MochiMedia) • Dr. Eric Brewer (CAP Theorum) 20
  21. 21. Four Important Things 2. Strong Community Stay in touch with real problems. Mark Phillips Basho Community Manager 21
  22. 22. Four Important Things 3. Agility Evolve to solve new problems. 22
  23. 23. Four Important Things 4. Restraint No solution can do it all well. 23
  24. 24. Back to Riak 24
  25. 25. What characteristics of Riak become important at different cluster sizes? 25
  26. 26. Single Box Riak Simplicity 26
  27. 27. Single Box Riak Simple Key/Value Store, Flexible Schema client = RiakClient(host, port) bucket = client.bucket('mybucket') obj = bucket.new('foo', 'bar') obj.store() obj2 = bucket.get(‘foo’) 27
  28. 28. Single Box Riak Clients in Many Languages 28
  29. 29. Single Box Riak Predictable Development Interface Riak Riak Riak === Riak Riak Riak Development Production 29
  30. 30. Single Box Riak Configurable Buckets != Profile Audio Bucket Bucket 30
  31. 31. Single Box Riak Links (Lightweight Data Relations) Profile Projects Comments 31
  32. 32. Small Riak Cluster Parallelism and Power 32
  33. 33. Small Riak Cluster Expanding the Cluster bin/riak-admin join riak@hostname 33
  34. 34. Small Riak Cluster Expanding the Cluster bin/riak-admin join riak@hostname 34
  35. 35. Small Riak Cluster Expanding the Cluster bin/riak-admin join riak@hostname Hello ladies, can your datastore grow like mine? *monocle smile* 34
  36. 36. Small Riak Cluster Distributed Queries 35
  37. 37. Small Riak Cluster Distributed Queries 36
  38. 38. Small Riak Cluster Javascript-Based Map/Reduce Map Map Reduce Map Keys Reduce Output Map Reduce ... Map 37
  39. 39. Small Riak Cluster Well-Behaved HTTP Client Client nginx Browser Client Riak Riak Squid Riak Riak Riak 38 nginx proxy config - http://gist.github.com/323048
  40. 40. Small Riak Cluster Riak Full-Text Search bin/search-cmd install mybucket "converse AND category:shoes" Riak Riak Riak Riak Riak 39 nginx proxy config - http://gist.github.com/323048
  41. 41. Large Riak Cluster Operations 40
  42. 42. Large Riak Cluster No Special Nodes Special Nodes 41
  43. 43. Large Riak Cluster Laugh at Machine Failure Riak Riak Riak Riak X Riak 42
  44. 44. Large Riak Cluster Scale by Adding Nodes Riak Riak Riak Riak Riak Riak Riak Riak Riak Riak Riak Riak 43
  45. 45. Large Riak Cluster Packages or Self-Contained Installation sudo dpkg -i riak_0.13.0-2_amd64.deb -or- unzip riak.zip riak/bin/riak start 44
  46. 46. $0.00
  47. 47. Order now, and $0.00 we’ll include an extra Riak for free!* *Offer limited to open- source editions of Riak.
  48. 48. Enterprise ($$$ / ~10’s of boxes) • On-Call Support 24x7x365 • Management Tools • SNMP Monitoring • Multi-site Replication 47
  49. 49. Enterprise ($$$ / ~10’s of boxes) • On-Call Support 24x7x365 • Management Tools • SNMP Monitoring • Multi-site Replication 48
  50. 50. Thanks! Questions? Next Steps Read more at http://wiki.basho.com Download binaries from http://downloads.basho.com Get source code from http://hg.basho.com Join the public mailing list at riak-users@lists.basho.com Mailing list archives at http://riak.markmail.org Thanks! Rusty Klophaus (@rklophaus) Mark Phillips (@pharkmillups) 49
  51. 51. Additional Slides
  52. 52. Tutorial Get Riak Connect with a Client Store Data Store an Object with Links Linkwalking Map/Reduce 51
  53. 53. Get Riak Download http://downloads.basho.com/riak Start Riak cd riak bin/riak start 52
  54. 54. Connect with Python # Code is at http://hg.basho.com import riak # Connect client = riak.RiakClient('127.0.0.1', 8098) 53
  55. 55. Store Data mybucket = client.bucket('mybucket') # Create an object... obj = mybucket.new('myobject') obj.set_data({ 'foo' : 1, 'bar' : 2 }) obj.store() # Read the object... obj = mybucket.get('myobject') print obj.get_data() # Or, open a web browser... http://127.0.0.1:8098/riak/mybucket/myobject 54
  56. 56. Store an Object with Links bands = client.bucket('bands') albums = client.bucket('albums') members = client.bucket('members') # Store a band, link to album and members... obj = bands.new('Winger') .add_link(albums.new('Pull', 1275922).store()) .add_link(albums.new('IV', 542731).store()) .add_link(albums.new('Karma', 200170).store()) .add_link(members.new('Kip Winger').store()) .add_link(members.new('Reb Beach').store()) .add_link(members.new('John Roth').store()) .add_link(members.new('Rod M.').store()) .store() 55
  57. 57. Linkwalking # Get the albums... albums = obj.link('albums').run() # Get the songs (assumes data is present)... songs = obj.link('albums').link('songs').run() # Get the members... members = riak.MapReduce(client) .add('bands', 'Winger') .link('members') .run() 56
  58. 58. Map/Reduce # Count the number of sales... result = obj .link('albums') .map("function(v) { return [v.values[0].data]; }") .reduce("Riak.reduceSum") .run() 57

×