Your SlideShare is downloading. ×
Riak at Engine Yard Cloud
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Riak at Engine Yard Cloud


Published on

Given at Big Nerd Ranch in Atlanta.

Given at Big Nerd Ranch in Atlanta.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Getting Started with Riak in the Cloud Big Nerd Ranch
  • 2. About Me InesSombra@RandomMood
  • 3. Our Agenda About Us Our Customers The Problem The Solution
  • 4. Engine Yard
  • 5. About US Clouds Regions100% CloudBased Stacks SupportCritical services DRMUST be up HA24/7/365
  • 6. Our Customers
  • 7. An APPLICATIONAppApp DB Master us-east-1a DB ReplicaApp us-east-1b
  • 8. Then AWS Crashed
  • 9. Our App AFTERAppApp XDB Master us-east-1a DB ReplicaApp us-east-1b
  • 10. This sucked, A LOT! Time to strengthen our HA/DR Story$ Customers and us need to be up all the time!
  • 11. Our ResearchData Models Consistency PartitionsRelational Are ACID Consistency propertiesDocument Availability reallyKey/Value needed?Column
  • 12. Impedance Mismatch Relational Model
  • 13. Impedance MismatchMemory data-structure
  • 14. We needed a faulttolerant and highlyavailable (clustered) datastore
  • 15. OSS, Dynamo-based,distributed K/V store Hello RIAK! Linearly scalable Fault-tolerant Finally provide HA/DR solution
  • 16. Riak’s Use CasesApplication Type Key ValueSession User/Session id DataAdvertising Campaign id DataLogs Date Log file text, json,Content title, integer xml, media..
  • 17. When is Riak a good fitWhen availability is moreimportant than consistencyWhen data can bemodeled as keys/valuesWhen the problem fits
  • 18. Riak is OPS FRIENDLY X node 1.2.1 Growth node node 1.2.1 1.2.1 Reduction node 1.2.1 node 1.2.1 Upgrades node 1.3.0 Failures
  • 19. World’s most famous slide
  • 20. Riak is SO MUCH MORE Consistent Hashing Hinted Handoff Active Anti Entropy Tuneable consistency N|W|R
  • 21. OUR APP AFTER App node 1.3.0 App node node 1.3.0 1.3.0 node node App 1.3.0 1.3.0
  • 22. We need a RiakCluster but we have no hardware
  • 23. RIAK In the Wild 18th and Harrison, San Francisco, CA
  • 24. Let’s build A PRODUCT
  • 25. Your CHOICES You are trying to create a Riak cluster. Do you need any help? 1 Number of Nodes 2 Instance Type 3 Data Location 4 Backend Type
  • 26. What we GIVE YOU
  • 27. Our SUPPORT X2
  • 28. HOWTO What youOPS to know
  • 29. Prepare Yourself More instances to compensate for theperformance variability of virtualized resources
  • 30. Choosing INSTANCES ^ m1.large & m1.xlarge EBS-optimized or PIOPs High I/O Quadruple XL (hi1.4xlarge) - SSDs!
  • 31. Cluster Sizes 5 nodes or more Best performance & growth Scales linearly Don’t be stingy
  • 32. Ring SIZE Change Default CANNOT BE RESET Ring Size at 256 (8-64 vnodes per node) DONT GO OVER 512
  • 33. Our PROVISIONING Riak Cluster Riak Cluster App MyRiakCluster: node-0 haproxy HTTP or MyRiakCluster: node-1 Protobuff App haproxy MyRiakCluster: node-2 App MyRiakCluster: node-3 haproxy MyRiakCluster: node-4
  • 34. Disable SWAP ^ Riak process pages should not hit swap Swap usage can result in unresponsive server Let the kernel kill it
  • 35. Mount & SCHEDULER ^ Mount with noatime flag Use deadline scheduler on EBS cat /sys/block/xvdj1/queue/scheduler noop [deadline] cfq echo deadline > /sys/block/xvdj1/queue/ scheduler
  • 36. Linux FILESYSTEMS ^ xfs, ext3 ext4 barrier = 0 data = writeback ZFS not recommended
  • 37. Backup STRATEGY ^ Backup both the ring and data directories scp or rsync data dir EBS snapshots
  • 38. Monitoring STRATEGY Stats Interface riak-admin status Collectd Nagios
  • 39. Scaling RIAK^ Vertical riak-admin cluster replace <old> <new>^ Horizontal riak-admin cluster add <new>
  • 40. How we SCALE RIAK
  • 41. Hybrid Solutions FTWRIAK + PostgreSQLRIAK + FTS Engine*RIAK + Redis
  • 42. Try OUR RIAK
  • 43. How difficult IS IT?
  • 44. Finally! A fewthoughts
  • 45. Polyglot PERSISTENCE Pick the right tool for the job Apps & Services can have multiple databases
  • 46. On SOA My super nice e-commerce siteShopping Cart & Completed Inventory and Recommendations Session Data Orders Item Pricing EngineSession Order Inventory & Nodes andStorage Persistence Price service relationsservice service service RDBMSK/V Store Document Graph Store Store NoSQL Distilled: Fowler & Sadalage
  • 47. We RECOMMEND Don’t optimize too quickly Take time to understand your problem Complexity increases with more databases
  • 48. Do KEEP IN MIND Different paradigms are different Document Relational != Store
  • 49. WhateverGive meMOAR!
  • 50. Great RESOURCES Little Riak Book (
  • 51. Questions?