Rolling With Riak

(Yet-Another-NoSQL-DB) Presented By John Lynch [email_address]

Web App Developers ORM Focus on the App Speed of Development DB Agnostic(ish) Fixed Schema Limits design choices Migration Hell Scaling at DB layer Rails!

Shiny New Toys Decades of research and best practices Awesome ad-hoc query capability Zillions of vendors/tools/libraries/code Flexibility of schema-less design Ability to scale…. Web-scale

Web Scale
Changing App Types
Social
Games
Marketing / Advertising
Freemium Business Models
1M free => 10K paying customers

NoSQL Landscape
Pure Key/Value (Redis/Tokyo Cabinet/etc)
Key/Value+ (CouchDB/MongoDB/Riak)
BigTable Type (Hbase, HyperTable)
Choose wisely! No standard API.
(Good general overview can be found here: http://cattell.net/datastores/Datastores.pdf)

MongoDB
Popular with Ruby community
Combines Key/Value with ability to do Indexed Queries

Scaling MongoDB Master, Slave, Replica Set, Replica Pair, Shard Server, Connection Pool, ack!

Scaling MongoDB 

If all you want is NoSQL… NoSQL on MySQL Leverages all MySQL skills, tools, techniques, stability, dependability If you want NoSQL + Scalability… … not so much.

Riak
Developed by Basho.com
Used on several large production sites
Written in Erlang
Distributed – Fault Tolerant
Buckets – Keys – Values
Values can be anything (json,binary,etc)
Ruby & Rails Client (Ripple project @ Github)

Riak speaks HTTP > curl –i http://host:8098/riak/bucket1/key1 HTTP/1.1 200 OK X-Riak-Vclock: awpcFAA== Content-Type: text/plain Content-Length: 9 Last-Modified: Wed, 01 Se… Etag: 45364657 I am a value Leverage existing HTTP infrastructure, tools, etc

Scaling Riak Riak Riak Riak Riak Riak Http Load Balancer Varnish (cache) Standard HTTP Protocol Rails Rails Rails Rails Rails

Scaling Riak (alt) Riak Rails Nginx Riak Rails Nginx Riak Rails Nginx Riak Rails Nginx Riak Rails Nginx Http Load Balancer Varnish (cache) Standard HTTP Protocol

Key Differentiator - Distributed
Inspired by Amazon’s Dynamo
Uses consistent hashing algorithim
No “Master Node”
No single point of failure
Any node can service any request
Automatically rebalances as nodes join
Tunable CAP Properties
Consistency, Availability, Partition Tolerance

N R W
N = # of copies of the data
R = # of nodes necessary to read
W = # of nodes necessary to write
Tunable by the application, on a per-bucket and per-query basis

Low Value Data (N=2 R=1 W=1)
Logging
Web Content (N=4 R=1 W=4)
Maximum availability and consistency
Financial Data (N=4 R=1 W=4 DW=4)
DW is “Durable Write”
Riak cluster of 4 Physical Computers

Network Split

Map/Reduce
Map steps run on each node
Final reduce runs on single node
results = Riak::MapReduce.new(client). add(“albums”). map("function(v){ return [JSON.parse(v.values[0].data).title]; }", :keep => true).run

Links
Riak documents can have links to other documents, each link can be “tagged”
Link data is separate from doc data
Easy URL access to walk these links
GET /riak/artists/TheBeatles/albums,_,_/tracks,_,1

When NOT to use Riak
Single machine
Small scale or bog-standard apps
Need rich ad-hoc indexed queries
Need mature tools and libraries

Any questions? (First round at Rock Bottom generously sponsored by Basho.com)

Rolling With Riak

by John Lynch on Jun 01, 2010

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 146

Statistics

Rolling With Riak Presentation Transcript