The document discusses using Riak, an open source NoSQL database, in the cloud to provide highly available and fault tolerant storage for applications. It describes how the authors' application struggled with availability when hosted on AWS, and their research led them to choose Riak as a solution. Riak provides features like consistent hashing, hinted handoff, and active anti-entropy that allow it to scale linearly and remain available even during failures or added/reduced nodes. The document provides guidance on deploying a Riak cluster in the cloud, including choosing instance types, sizing the cluster, disabling swap, using the right filesystem and scheduler, and monitoring and scaling the cluster over time.

1. Getting Started with
Riak in the
Cloud
Big Nerd Ranch

2. About Me
Ines
Sombra
@RandomMood

3. Our Agenda
About Us
Our Customers
The Problem
The Solution

4. Engine Yard

5. About US Clouds
Regions
100% Cloud
Based Stacks
Support
Critical services DR
MUST be up HA
24/7/365

6. Our Customers
http://www.flickr.com/photos/meetrajesh/6972883401

7. An APPLICATION
App
App DB
Master
us-east-1a
DB
Replica
App us-east-1b

8. Then AWS Crashed
http://www.flickr.com/photos/brandongrasley/8227882239/

9. Our App AFTER
App
App
XDB
Master
us-east-1a
DB
Replica
App us-east-1b

10. This sucked, A LOT
! Time to strengthen
our HA/DR Story
$ Customers and us
need to be up all
the time!

11. Our Research
Data Models Consistency Partitions
Relational Are ACID Consistency
properties
Document Availability
really
Key/Value needed?
Column

12. Impedance Mismatch
Relational Model

13. Impedance Mismatch
Memory data-structure

14. We needed a fault
tolerant and highly
available (clustered)
datastore

15. OSS, Dynamo-based,
distributed K/V store
Hello RIAK!
Linearly scalable
Fault-tolerant
Finally provide
HA/DR solution

16. Riak’s Use Cases
Application Type Key Value
Session User/Session id Data
Advertising Campaign id Data
Logs Date Log file
text, json,
Content title, integer
xml, media..

17. When is Riak a good fit
When availability is more
important than consistency
When data can be
modeled as keys/values
When the problem fits

18. Riak is OPS FRIENDLY
X
node
1.2.1
Growth
node node
1.2.1 1.2.1
Reduction
node
1.2.1
node
1.2.1
Upgrades
node
1.3.0 Failures

19. World’s most famous slide

20. Riak is SO MUCH MORE
Consistent Hashing
Hinted Handoff
Active Anti Entropy
Tuneable consistency
N|W|R

21. OUR APP AFTER
App
node
1.3.0
App node node
1.3.0 1.3.0
node node
App 1.3.0 1.3.0

22. We need a Riak
Cluster but we have
no hardware

23. RIAK In the Wild
18th and Harrison, San Francisco, CA

24. Let’s build A PRODUCT
http://www.flickr.com/photos/worldbank/8261699153/

25. Your CHOICES
You are trying
to create a Riak
cluster. Do you
need any help?
1 Number of Nodes
2 Instance Type
3 Data Location
4 Backend Type

26. What we GIVE YOU

27. Our SUPPORT
X2

28. HOWTO
What you
OPS to know

29. Prepare
Yourself
More instances to
compensate for the
performance variability of
virtualized resources

30. Choosing INSTANCES
^ m1.large & m1.xlarge
EBS-optimized or PIOPs
High I/O Quadruple XL
(hi1.4xlarge) - SSDs!

31. Cluster Sizes
5 nodes or more
Best performance
& growth
Scales linearly
Don’t be stingy

32. Ring SIZE
Change Default
CANNOT BE RESET
Ring Size at 256
(8-64 vnodes per node)
DONT GO OVER 512

33. Our PROVISIONING
Riak Cluster
Riak Cluster
App MyRiakCluster: node-0
haproxy
HTTP
or MyRiakCluster: node-1
Protobuff
App
haproxy
MyRiakCluster: node-2
App
MyRiakCluster: node-3
haproxy
MyRiakCluster: node-4

34. Disable SWAP
^ Riak process pages
should not hit swap
Swap usage can result
in unresponsive server
Let the kernel kill it

35. Mount & SCHEDULER
^ Mount with noatime flag
Use deadline scheduler
on EBS
cat /sys/block/xvdj1/queue/scheduler
noop [deadline] cfq
echo deadline > /sys/block/xvdj1/queue/
scheduler

36. Linux FILESYSTEMS
^ xfs, ext3
ext4
barrier = 0
data = writeback
ZFS not recommended

37. Backup STRATEGY
^ Backup both the ring
and data directories
scp or rsync data dir
EBS snapshots

38. Monitoring STRATEGY
Stats Interface
riak-admin status
Collectd
Nagios

39. Scaling RIAK
^ Vertical
riak-admin cluster replace <old>
<new>
^ Horizontal
riak-admin cluster add <new>

40. How we SCALE RIAK

41. Hybrid Solutions FTW
RIAK
+ PostgreSQL
RIAK
+ FTS Engine*
RIAK
+ Redis

42. Try OUR RIAK

43. How difficult IS IT?

44. Finally!
A few
thoughts

45. Polyglot PERSISTENCE
Pick the right tool
for the job
Apps & Services
can have multiple
databases

46. On SOA
My super nice e-commerce site
Shopping Cart & Completed Inventory and Recommendations
Session Data Orders Item Pricing Engine
Session Order Inventory & Nodes and
Storage Persistence Price service relations
service service service
RDBMS
K/V Store Document Graph Store
Store
NoSQL Distilled: Fowler & Sadalage

47. We RECOMMEND
Don’t optimize too quickly
Take time to understand
your problem
Complexity increases with
more databases

48. Do KEEP IN MIND
Different paradigms
are different
Document
Relational != Store

49. Whatever
Give me
MOAR!

50. Great RESOURCES
Little Riak Book
(https://github.com/coderoshi/little_riak_book/)
docs.basho.com
blog.engineyard.com
nosql.mypopescu.com

51. Questions?