This document provides an overview of Riak, an open source distributed database. It discusses Riak's key features like fault tolerance, horizontal scalability, and high availability. It also summarizes Riak's data model, interfaces, client libraries, ways to store and query data including CRUD, MapReduce, secondary indexing, and search. The document outlines Riak's architecture including consistent hashing, virtual nodes, vector clocks, and append-only storage. It previews upcoming Riak 1.4 features and discusses commercial Riak products and the growing Riak community.

1. Wednesday, March 13, 13

2. #WHOIS
Adron Hall | @adron | Coder, Messenger, Recon
Wednesday, March 13, 13

3. 芭蕉
Wednesday, March 13, 13

4. Distributed, masterless, highly-available key/value store
Wednesday, March 13, 13

5. DESIGN GOALS
Fault-Tolerance Ops Friendliness
Horizontal Scalability Predictability
Low-latency High-Availability
Wednesday, March 13, 13

6. Use, when & what to use Riak for...
Users/Profiles Logging Systems
Metadata Sensor Data
Session Storage Notification Systems
Object Storage Record Systems
Wednesday, March 13, 13

7. IN PRODUCTION AT
And 1000s more...
Wednesday, March 13, 13

8. DATA MODEL
Wednesday, March 13, 13

9. {“Key”:“Value”}
• Values are stored against keys
• Key/Value + Metadata = Object
• Fundamental Unit of Replication
• Any Datatype will work
• Record to disk in binary format
Wednesday, March 13, 13

10. <<BUCKET>>/<<KEY>>
• Virtual Namespace
• Bucket + Keys = Object Address
• Buckets have properties
• Objects in bucket inherit properties
• No relationships between buckets
Wednesday, March 13, 13

11. DATA ACCESS
Wednesday, March 13, 13

12. INTERFACES
HTTP API - Via a little piece of magic called Webmachine
Largely-faithful REST implementation
Protocol Buffers API - Thanks, Google!
Compact, binary protocol
Wednesday, March 13, 13

13. CLIENT LIBS
Python Java Clojure
Erlang PHP Haskell
...and more via Basho
Ruby C/C++ OCaml
or our community.
Perl .NET Scala
Dart Go Node.js
Wednesday, March 13, 13

14. RIAK GIVES YOU
[FOUR] WAYS TO STORE,
RETRIEVE, AND QUERY
DATA
Wednesday, March 13, 13

15. CRUD
// GET
1
2
3
4 GET /buckets/bucket/keys/key
5
6
// PUT
7
8
9
10 POST /buckets/bucket/keys/key // Riak-­‐defined key
11
PUT /buckets/bucket/keys/key // User-­‐defined key
12
13
// DELETE
14
15
16
17 DELETE /buckets/bucket/keys/key
18
Wednesday, March 13, 13

16. MapReduce
Distributed processing system using Riak Pipe
Efficient for targeted queries over known key range
Write jobs in Erlang or JS. (Erlang more performant)
Wednesday, March 13, 13

17. Secondary Indexing (2i)
riak_object X-Riak-Index-email_bin
“mark@basho.com”
riak_object X-Riak-Index-value_int
“42”
Tag objects with custom metadata on PUT...
Exact match and range queries...
No multi-index queries yet...
Pagination is on its way...
Wednesday, March 13, 13

18. Riak Search
Store and index documents (JSON, text, XML, etc)
Current Riak Search supports subset of Solr API
Next iteration (Yokozuna; in beta)will implement distributed
Solr on Riak. It will be sexy.
Looking for beta testers to help harden Yokozuna
Wednesday, March 13, 13

19. ARCHITECTURE
Consistent Hashing
The scaleability and
Virtual Nodes
ease of operation
goals inform Handoff/Rebalancing
architectural decisions.
Vector Clocks
These come with tradeoffs.
Append-only storage
Active Anti-Entropy*
Wednesday, March 13, 13

20. Consistent Hashing
Location of data in the Riak ring is determined based on hash of
bucket + key.
Provides even distribution of storage and query load
Trades off advantages gained from locality
- e.g. Range queries and aggregates
Wednesday, March 13, 13

21. Consistent Hashing
Wednesday, March 13, 13

22. Virtual Nodes
Unit of addressing and concurrency in Riak
Each physical host manage many vnodes
Partition count / physical machines = vnodes/machine*
Decouples physical assets from data distribution. This provides:
- simplicity in cluster sizing
- failure isolation
Wednesday, March 13, 13

23. Handoff/Rebalancing
Mechanisms for data rebalancing
When nodes join/leave cluster, handoff and rebalancing
manage the date shuffling dynamically
Trades off speed of convergence vs. effects on cluster
performance
- causes disk & network load
Wednesday, March 13, 13

24. Vector Clocks
VCs used to rectify object consistency at READ time.
Lots of knobs to turn; well-documented
Trades off space, speed, and complexity for safety
- will store all sibling objects until resolved
- can lead to object size issues
Wednesday, March 13, 13

25. Append-Only Storage
Riak provides a pluggable backend interface. (Write your own;
we’ll probably hire you...)
Bitcask, LevelDB are most-heavily used. Both are
append - only
Provides crash safety and speed.
Trade off: periodic compaction/merge ops
Wednesday, March 13, 13

26. RIAK 1.3
(AKA “new hotness”)
Active Anti Entropy Riaknostic included by default
MapReduce Improvements Riak Control improvements
IPv6 Support Much more
Full release notes: https://github.com/basho/riak/blob/1.3/RELEASE-NOTES.md
Wednesday, March 13, 13

27. FUTURE WORK*
(1.4 and beyond)
Dynamic Ring Size Consistency
Yokozuna 2i Improvements
CRDTs/Data Types Riak Pipe work
Riak Object Much more
(* all code subject to ship early, late, or not at all)
Wednesday, March 13, 13

28. Multi-tenant cloud storage software for
public and private clouds.
Designed to provide simple, available, distributed
cloud storage at any scale.
S3-API compatible and supports per-tenant
reporting for billing and metering use cases.
Additional APIs on the way.
Stores files of arbitrary size. Under the hood
stores 1MB chunks along side a manifest.
Stateless proxy (CS) does chunking. Riak does
distribution, storage, etc.
Wednesday, March 13, 13

29. Extends Riak's capabilities with:
- multi-datacenter replication
- SNMP Configuration
- JMX-Monitoring
- 24x7 support from Basho Engineers
One cluster acts as a "source cluster". The source
cluster replicates its data to one or more
"sink clusters" using either real-time or full sync.
Data transfer is unidirectional (source -> sink).
Bidirectional synchronization can be
achieved by configuring a pair of connections
between clusters.
Wednesday, March 13, 13

30. RIAK COMMUNITY
Mailing List - 1300 developers
IRC - 200+ people every day yelling about software
GitHub - 1000s of watchers; 200+ contributors to all projects
Meetups - 10 Countries, 23 Cities, 3700+ Members & growing fast!
Deployments - 1000s in production.
Wednesday, March 13, 13

31. ricon.io/east.html
May 13-14th in New York City
Talks, hacking, parties
Dedicated to the future of Riak and
distributed systems in production
REGISTER https://ricon-east-2013.eventbrite.com/?discount=lovevnodes
NOW!
Wednesday, March 13, 13

32. GETTING STARTED
Downloads - http://docs.basho.com/riak/latest/downloads/
Docs - http://docs.basho.com
Riak Source Code - github.com/basho/riak
All Basho source Code - github.com/basho/
Riak Mailing List - http://bit.ly/riak-list
Email or Tweet me @adron or adron@basho.com
Wednesday, March 13, 13

33. Let’s Talk UI & CLI - Demo Things
Wednesday, March 13, 13

34. #WHOIS
Adron Hall | @adron | Coder, Messenger, Recon
Wednesday, March 13, 13