Riak - From Small to Large

Riak from Small to Large Strange Loop · October 2010 Rusty Klophaus (@rklophaus) Basho Technologies

Riak is a Dynamo-inspired, open-sourced, key/value datastore built to scale predictably and easily. 2

Riak is a Dynamo-inspired, open-sourced, key/value datastore built to scale predictably and easily. Your ops guy: calm, relaxed, and wearing a party hat. 3

NoSQL 4

5 http://www.ﬂickr.com/photos/ghewgill/129950124

NoSQL has changed like Keanu hasn’t. 6 http://www.youtube.com/watch?v=nEubt6HpGhs

Companies, Funding, Customers, Awareness... 7 http://www.ﬂickr.com/photos/free-stock/4791385567

“NoSQL? Yeah, it’s this really cool technology, you probably haven’t heard of it yet.” 8 http://www.ﬂickr.com/photos/slava/285199203

“Johnson! Get me a memo on this ‘No sequal’ thing!” 9 http://www.ﬂickr.com/photos/proimos/4045973322

Why? 10

Because NoSQL is driven by narcissism, voyeurism, and materialism.

The catalyst for NoSQL is the data generated or consumed by humans. 12

Photos, music, and video... 13

Constantly connected devices... 14

Social Networks... 15 http://www.ﬂickr.com/photos/hanspoldoja/5001818922

Your Grandmama... 16 http://www.ﬂickr.com/photos/mega/7358278

Requirements That Confound an RDBMS Large Data Objects Changing Schemas BLOBs? Auto-generated DDL? Generic Columns? High-Availability Systems Non-Standard Access Bi-Directional Replication? Recursive SQL? Log shipping? High-Capacity Systems Shards? Vertical Partitioning? 17

The roots of NoSQL are practical, not academic. 18

Turning Point Scratch an Itch ➮ Build a Company 19

Four Important Things 1. Broad Knowledge Learn from the past, both DB *and* non-DB worlds. • VP Technical Services, Akamai • Senior Architect, Akamai • Chief of Operations, Snapﬁsh • Release Manager, Engine Yard • Bob Ippolito (CTO/CoFounder MochiMedia) • Dr. Eric Brewer (CAP Theorum) 20

Four Important Things 2. Strong Community Stay in touch with real problems. Mark Phillips Basho Community Manager 21

Four Important Things 3. Agility Evolve to solve new problems. 22

Four Important Things 4. Restraint No solution can do it all well. 23

Back to Riak 24

What characteristics of Riak become important at different cluster sizes? 25

Single Box Riak Simplicity 26

Single Box Riak Simple Key/Value Store, Flexible Schema client = RiakClient(host, port) bucket = client.bucket('mybucket') obj = bucket.new('foo', 'bar') obj.store() obj2 = bucket.get(‘foo’) 27

Single Box Riak Clients in Many Languages 28

Single Box Riak Predictable Development Interface Riak Riak Riak === Riak Riak Riak Development Production 29

Single Box Riak Conﬁgurable Buckets != Proﬁle Audio Bucket Bucket 30

Single Box Riak Links (Lightweight Data Relations) Proﬁle Projects Comments 31

Small Riak Cluster Parallelism and Power 32

Small Riak Cluster Expanding the Cluster bin/riak-admin join riak@hostname 33

Small Riak Cluster Expanding the Cluster bin/riak-admin join riak@hostname 34

Small Riak Cluster Expanding the Cluster bin/riak-admin join riak@hostname Hello ladies, can your datastore grow like mine? *monocle smile* 34

Small Riak Cluster Distributed Queries 35

Small Riak Cluster Distributed Queries 36

Small Riak Cluster Javascript-Based Map/Reduce Map Map Reduce Map Keys Reduce Output Map Reduce ... Map 37

Small Riak Cluster Well-Behaved HTTP Client Client nginx Browser Client Riak Riak Squid Riak Riak Riak 38 nginx proxy conﬁg - http://gist.github.com/323048

Small Riak Cluster Riak Full-Text Search bin/search-cmd install mybucket "converse AND category:shoes" Riak Riak Riak Riak Riak 39 nginx proxy conﬁg - http://gist.github.com/323048

Large Riak Cluster Operations 40

Large Riak Cluster No Special Nodes Special Nodes 41

Large Riak Cluster Laugh at Machine Failure Riak Riak Riak Riak X Riak 42

Large Riak Cluster Scale by Adding Nodes Riak Riak Riak Riak Riak Riak Riak Riak Riak Riak Riak Riak 43

Large Riak Cluster Packages or Self-Contained Installation sudo dpkg -i riak_0.13.0-2_amd64.deb -or- unzip riak.zip riak/bin/riak start 44

Order now, and $0.00 we’ll include an extra Riak for free!* *Offer limited to open- source editions of Riak.

Enterprise ($$$ / ~10’s of boxes) • On-Call Support 24x7x365 • Management Tools • SNMP Monitoring • Multi-site Replication 47

Enterprise ($$$ / ~10’s of boxes) • On-Call Support 24x7x365 • Management Tools • SNMP Monitoring • Multi-site Replication 48

Thanks! Questions? Next Steps Read more at http://wiki.basho.com Download binaries from http://downloads.basho.com Get source code from http://hg.basho.com Join the public mailing list at riak-users@lists.basho.com Mailing list archives at http://riak.markmail.org Thanks! Rusty Klophaus (@rklophaus) Mark Phillips (@pharkmillups) 49

Additional Slides

Tutorial Get Riak Connect with a Client Store Data Store an Object with Links Linkwalking Map/Reduce 51

Get Riak Download http://downloads.basho.com/riak Start Riak cd riak bin/riak start 52

Connect with Python # Code is at http://hg.basho.com import riak # Connect client = riak.RiakClient('127.0.0.1', 8098) 53

Store Data mybucket = client.bucket('mybucket') # Create an object... obj = mybucket.new('myobject') obj.set_data({ 'foo' : 1, 'bar' : 2 }) obj.store() # Read the object... obj = mybucket.get('myobject') print obj.get_data() # Or, open a web browser... http://127.0.0.1:8098/riak/mybucket/myobject 54

Store an Object with Links bands = client.bucket('bands') albums = client.bucket('albums') members = client.bucket('members') # Store a band, link to album and members... obj = bands.new('Winger') .add_link(albums.new('Pull', 1275922).store()) .add_link(albums.new('IV', 542731).store()) .add_link(albums.new('Karma', 200170).store()) .add_link(members.new('Kip Winger').store()) .add_link(members.new('Reb Beach').store()) .add_link(members.new('John Roth').store()) .add_link(members.new('Rod M.').store()) .store() 55

Linkwalking # Get the albums... albums = obj.link('albums').run() # Get the songs (assumes data is present)... songs = obj.link('albums').link('songs').run() # Get the members... members = riak.MapReduce(client) .add('bands', 'Winger') .link('members') .run() 56

Map/Reduce # Count the number of sales... result = obj .link('albums') .map("function(v) { return [v.values[0].data]; }") .reduce("Riak.reduceSum") .run() 57

Riak - From Small to Large

by Rusty Klophaus

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Riak - From Small to Large Presentation Transcript