Getting started with Riak in the Cloud

960 views
789 views

Published on

Given at RICON 2012

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
960
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Getting started with Riak in the Cloud

  1. 1. Getting started withRiak in the Cloud
  2. 2. I work with I work with Code! Data!Michael Brodhead Ines Sombra @mojinations @RandomMood
  3. 3. AgendaApplication layer Database layer
  4. 4. A tale of 2 apps• One brand new (no code written)• One in production running MySQL
  5. 5. MultipleRegions
  6. 6. MultipleClouds
  7. 7. 11
  8. 8. The Librariesriak-client• simple Ruby access to Riaks HTTP & protocol buffer APIs
  9. 9. The LibrariesRipple• Higher level, ORM interface• ActiveModel based buffer APIs
  10. 10. Simplicity
  11. 11. The Librariesdm-riak• Already using DataMapper• Hoping for trivial switchover
  12. 12. The LibrariesRiak Shim• database.yml• Bucket naming• Hides JSON• 2i
  13. 13. Challenges • Unit tests • Data migration • Deployment
  14. 14. Testing
  15. 15. Testing
  16. 16. Deployment
  17. 17. Use your app logic
  18. 18. How do we get here?
  19. 19. “We need aRiak cluster” (@mojinations circa 3 months ago) ... but we have no hardware
  20. 20. Let’s do it!
  21. 21. Why Riak Product?• Linearly Scalable• Fault-tolerant• Ops friendly• Product lacks HA options
  22. 22. Riak @ Engine Yard • We’ll use it first • No custom Chef • Provisioning, configuration, monitoring, & backups
  23. 23. Your Choices• Number of nodes• Instance type You are trying to create a Riak cluster. Do you need any help?• Data location• Backend type
  24. 24. AWS OpsThe Riak on AWS slide
  25. 25. Instances & I/O• Riak’s I/O patterns• Small blobs from many places on disk• EBS best at bulk reads &writes
  26. 26. Instance types• m1.large & m1.xlarge• EBS-optimized NEW!• Consider PIOPS NEW!• High I/O Quadruple XL (hi1.4xlarge) - SSDs! NEW!
  27. 27. Prepare YourselfMore instances to compensatefor the performance variabilityof virtualized resources
  28. 28. Cluster Sizes• 5 nodes or more Best performance & growth• Scales linearly• Don’t be stingy
  29. 29. Ring Size• Change default value!• Cannot be reset• Ring size: 256 8/64 vnodes per node• Don’t go above 512
  30. 30. Provisioning Riak Cluster App MyRiakCluster: node-0haproxy HTTP or MyRiakCluster: node-1 Protobuff Apphaproxy MyRiakCluster: node-2 App MyRiakCluster: node-3haproxy MyRiakCluster: node-4
  31. 31. Disable Swap• Riak process pages should not hit swap• Swap usage can result in unresponsive server• Let the kernel kill it
  32. 32. Mount & Scheduler• mount with noatime flag• Use deadline scheduler on EBS cat /sys/block/xvdj1/queue/scheduler noop [deadline] cfq echo deadline > /sys/block/xvdj1/ queue/scheduler
  33. 33. Linux Filesystems• xfs, ext3• ext4 • barrier = 0 • data = writeback• ZFS not recommended
  34. 34. Backend choices• Bitcask* • help mitigate EBS• LevelDB Can be changed• Memory and optimized!
  35. 35. Kernel Optimizations• /etc/sysctl.conf vm.swappiness = 0 net.core.somaxconn= 4000 net.ipv4.tcp_timestamps = 0 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_max_syn_backlog = 40000 net.ipv4.tcp_sack = 1 net.ipv4.tcp_fin_timeout = 15 net.ipv4.tcp_keepalive_intvl = 30 net.ipv4.tcp_tw_reuse = 1 http://wiki.basho.com/Linux-Performance-Tuning.html
  36. 36. Monitoring• stats or riak-admin status• Collectd & Nagios• Riaknostic• Riak Control (maybe soon)
  37. 37. Backups• Backup both the ring and data directories• Integrate with EY tools• scp or rsync data directory• EBS snapshots
  38. 38. Scaling Riak• Verticalriak-admin cluster replace <old> <new>• Horizontal • reduces node load • batch is better
  39. 39. Benchmarks
  40. 40. Tests• 5-node cluster / 5 hours• 50 concurrent connections• gets 10 /puts 5 /deletes 1• Bitcask backend• Protobuffs protocol
  41. 41. Medium - PIOPS 500
  42. 42. Medium - PIOPS 1K
  43. 43. Medium - EphemeralFAIL!
  44. 44. XL - EBS
  45. 45. XL -PIOPS 500
  46. 46. 4XL - EBS
  47. 47. 4XL - PIOPS 1k
  48. 48. What’s next?

×