Your SlideShare is downloading. ×
0
Getting Started with   Riak in the     Cloud                       Big Nerd Ranch
About Me InesSombra@RandomMood
Our Agenda    About Us    Our Customers    The Problem    The Solution
Engine Yard
About US            Clouds                             Regions100% CloudBased                             Stacks          ...
Our Customershttp://www.flickr.com/photos/meetrajesh/6972883401
An APPLICATIONAppApp      DB        Master         us-east-1a                        DB                      ReplicaApp   ...
Then AWS Crashed http://www.flickr.com/photos/brandongrasley/8227882239/
Our App AFTERAppApp       XDB       Master        us-east-1a                       DB                     ReplicaApp      ...
This sucked, A LOT!   Time to strengthen    our HA/DR Story$   Customers and us    need to be up all    the time!
Our ResearchData Models   Consistency   PartitionsRelational    Are ACID Consistency              propertiesDocument      ...
Impedance Mismatch  Relational Model
Impedance MismatchMemory data-structure
We needed a faulttolerant and highlyavailable (clustered)     datastore
OSS, Dynamo-based,distributed K/V store                        Hello RIAK!                        Linearly scalable       ...
Riak’s Use CasesApplication Type           Key          ValueSession            User/Session id DataAdvertising        Cam...
When is Riak a good fitWhen availability is moreimportant than consistencyWhen data can bemodeled as keys/valuesWhen the p...
Riak is OPS FRIENDLY                     X            node             1.2.1                               Growth node    ...
World’s most famous slide
Riak is SO MUCH MORE Consistent Hashing Hinted Handoff Active Anti Entropy Tuneable consistency N|W|R
OUR APP AFTER  App                         node                          1.3.0 App    node                               n...
We need a RiakCluster but we have   no hardware
RIAK In the Wild          18th and Harrison, San Francisco, CA
Let’s build A PRODUCT        http://www.flickr.com/photos/worldbank/8261699153/
Your CHOICES     You are trying    to create a Riak     cluster. Do you     need any help?                       1   Numbe...
What we GIVE YOU
Our SUPPORT       X2
HOWTO What youOPS to know
Prepare                 Yourself    More instances to   compensate for theperformance variability of  virtualized resources
Choosing INSTANCES ^   m1.large & m1.xlarge     EBS-optimized or PIOPs     High I/O Quadruple XL     (hi1.4xlarge) - SSDs!
Cluster Sizes        5 nodes or more        Best performance        & growth        Scales linearly        Don’t be stingy
Ring SIZE      Change Default      CANNOT BE RESET      Ring Size at 256       (8-64 vnodes per node)      DONT GO OVER 512
Our PROVISIONING                            Riak Cluster                         Riak Cluster    App                  MyRi...
Disable SWAP ^   Riak process pages     should not hit swap     Swap usage can result     in unresponsive server     Let t...
Mount & SCHEDULER ^   Mount with noatime flag     Use deadline scheduler     on EBS     cat /sys/block/xvdj1/queue/schedul...
Linux FILESYSTEMS ^   xfs, ext3     ext4       barrier = 0       data = writeback     ZFS not recommended
Backup STRATEGY ^   Backup both the ring     and data directories     scp or rsync data dir     EBS snapshots
Monitoring STRATEGY        Stats Interface        riak-admin status        Collectd        Nagios
Scaling RIAK^   Vertical    riak-admin cluster replace <old>    <new>^   Horizontal    riak-admin cluster add <new>
How we SCALE RIAK
Hybrid Solutions FTWRIAK       + PostgreSQLRIAK     + FTS Engine*RIAK     + Redis
Try OUR RIAK
How difficult IS IT?
Finally! A fewthoughts
Polyglot PERSISTENCE  Pick the right tool  for the job  Apps & Services  can have multiple  databases
On SOA         My super nice e-commerce siteShopping Cart &    Completed    Inventory and       Recommendations Session Da...
We RECOMMEND Don’t optimize too quickly Take time to understand your problem Complexity increases with more databases
Do KEEP IN MIND Different paradigms are different                    Document  Relational   !=     Store
WhateverGive meMOAR!
Great RESOURCES Little Riak Book (https://github.com/coderoshi/little_riak_book/) docs.basho.com blog.engineyard.com nosql...
Questions?
Upcoming SlideShare
Loading in...5
×

Riak at Engine Yard Cloud

2,220

Published on

Given at Big Nerd Ranch in Atlanta.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,220
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Riak at Engine Yard Cloud"

  1. 1. Getting Started with Riak in the Cloud Big Nerd Ranch
  2. 2. About Me InesSombra@RandomMood
  3. 3. Our Agenda About Us Our Customers The Problem The Solution
  4. 4. Engine Yard
  5. 5. About US Clouds Regions100% CloudBased Stacks SupportCritical services DRMUST be up HA24/7/365
  6. 6. Our Customershttp://www.flickr.com/photos/meetrajesh/6972883401
  7. 7. An APPLICATIONAppApp DB Master us-east-1a DB ReplicaApp us-east-1b
  8. 8. Then AWS Crashed http://www.flickr.com/photos/brandongrasley/8227882239/
  9. 9. Our App AFTERAppApp XDB Master us-east-1a DB ReplicaApp us-east-1b
  10. 10. This sucked, A LOT! Time to strengthen our HA/DR Story$ Customers and us need to be up all the time!
  11. 11. Our ResearchData Models Consistency PartitionsRelational Are ACID Consistency propertiesDocument Availability reallyKey/Value needed?Column
  12. 12. Impedance Mismatch Relational Model
  13. 13. Impedance MismatchMemory data-structure
  14. 14. We needed a faulttolerant and highlyavailable (clustered) datastore
  15. 15. OSS, Dynamo-based,distributed K/V store Hello RIAK! Linearly scalable Fault-tolerant Finally provide HA/DR solution
  16. 16. Riak’s Use CasesApplication Type Key ValueSession User/Session id DataAdvertising Campaign id DataLogs Date Log file text, json,Content title, integer xml, media..
  17. 17. When is Riak a good fitWhen availability is moreimportant than consistencyWhen data can bemodeled as keys/valuesWhen the problem fits
  18. 18. Riak is OPS FRIENDLY X node 1.2.1 Growth node node 1.2.1 1.2.1 Reduction node 1.2.1 node 1.2.1 Upgrades node 1.3.0 Failures
  19. 19. World’s most famous slide
  20. 20. Riak is SO MUCH MORE Consistent Hashing Hinted Handoff Active Anti Entropy Tuneable consistency N|W|R
  21. 21. OUR APP AFTER App node 1.3.0 App node node 1.3.0 1.3.0 node node App 1.3.0 1.3.0
  22. 22. We need a RiakCluster but we have no hardware
  23. 23. RIAK In the Wild 18th and Harrison, San Francisco, CA
  24. 24. Let’s build A PRODUCT http://www.flickr.com/photos/worldbank/8261699153/
  25. 25. Your CHOICES You are trying to create a Riak cluster. Do you need any help? 1 Number of Nodes 2 Instance Type 3 Data Location 4 Backend Type
  26. 26. What we GIVE YOU
  27. 27. Our SUPPORT X2
  28. 28. HOWTO What youOPS to know
  29. 29. Prepare Yourself More instances to compensate for theperformance variability of virtualized resources
  30. 30. Choosing INSTANCES ^ m1.large & m1.xlarge EBS-optimized or PIOPs High I/O Quadruple XL (hi1.4xlarge) - SSDs!
  31. 31. Cluster Sizes 5 nodes or more Best performance & growth Scales linearly Don’t be stingy
  32. 32. Ring SIZE Change Default CANNOT BE RESET Ring Size at 256 (8-64 vnodes per node) DONT GO OVER 512
  33. 33. Our PROVISIONING Riak Cluster Riak Cluster App MyRiakCluster: node-0 haproxy HTTP or MyRiakCluster: node-1 Protobuff App haproxy MyRiakCluster: node-2 App MyRiakCluster: node-3 haproxy MyRiakCluster: node-4
  34. 34. Disable SWAP ^ Riak process pages should not hit swap Swap usage can result in unresponsive server Let the kernel kill it
  35. 35. Mount & SCHEDULER ^ Mount with noatime flag Use deadline scheduler on EBS cat /sys/block/xvdj1/queue/scheduler noop [deadline] cfq echo deadline > /sys/block/xvdj1/queue/ scheduler
  36. 36. Linux FILESYSTEMS ^ xfs, ext3 ext4 barrier = 0 data = writeback ZFS not recommended
  37. 37. Backup STRATEGY ^ Backup both the ring and data directories scp or rsync data dir EBS snapshots
  38. 38. Monitoring STRATEGY Stats Interface riak-admin status Collectd Nagios
  39. 39. Scaling RIAK^ Vertical riak-admin cluster replace <old> <new>^ Horizontal riak-admin cluster add <new>
  40. 40. How we SCALE RIAK
  41. 41. Hybrid Solutions FTWRIAK + PostgreSQLRIAK + FTS Engine*RIAK + Redis
  42. 42. Try OUR RIAK
  43. 43. How difficult IS IT?
  44. 44. Finally! A fewthoughts
  45. 45. Polyglot PERSISTENCE Pick the right tool for the job Apps & Services can have multiple databases
  46. 46. On SOA My super nice e-commerce siteShopping Cart & Completed Inventory and Recommendations Session Data Orders Item Pricing EngineSession Order Inventory & Nodes andStorage Persistence Price service relationsservice service service RDBMSK/V Store Document Graph Store Store NoSQL Distilled: Fowler & Sadalage
  47. 47. We RECOMMEND Don’t optimize too quickly Take time to understand your problem Complexity increases with more databases
  48. 48. Do KEEP IN MIND Different paradigms are different Document Relational != Store
  49. 49. WhateverGive meMOAR!
  50. 50. Great RESOURCES Little Riak Book (https://github.com/coderoshi/little_riak_book/) docs.basho.com blog.engineyard.com nosql.mypopescu.com
  51. 51. Questions?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×