Getting started with Riak in the Cloud

  • 553 views
Uploaded on

Given at RICON 2012

Given at RICON 2012

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
553
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Getting started withRiak in the Cloud
  • 2. I work with I work with Code! Data!Michael Brodhead Ines Sombra @mojinations @RandomMood
  • 3. AgendaApplication layer Database layer
  • 4. A tale of 2 apps• One brand new (no code written)• One in production running MySQL
  • 5. MultipleRegions
  • 6. MultipleClouds
  • 7. 11
  • 8. The Librariesriak-client• simple Ruby access to Riaks HTTP & protocol buffer APIs
  • 9. The LibrariesRipple• Higher level, ORM interface• ActiveModel based buffer APIs
  • 10. Simplicity
  • 11. The Librariesdm-riak• Already using DataMapper• Hoping for trivial switchover
  • 12. The LibrariesRiak Shim• database.yml• Bucket naming• Hides JSON• 2i
  • 13. Challenges • Unit tests • Data migration • Deployment
  • 14. Testing
  • 15. Testing
  • 16. Deployment
  • 17. Use your app logic
  • 18. How do we get here?
  • 19. “We need aRiak cluster” (@mojinations circa 3 months ago) ... but we have no hardware
  • 20. Let’s do it!
  • 21. Why Riak Product?• Linearly Scalable• Fault-tolerant• Ops friendly• Product lacks HA options
  • 22. Riak @ Engine Yard • We’ll use it first • No custom Chef • Provisioning, configuration, monitoring, & backups
  • 23. Your Choices• Number of nodes• Instance type You are trying to create a Riak cluster. Do you need any help?• Data location• Backend type
  • 24. AWS OpsThe Riak on AWS slide
  • 25. Instances & I/O• Riak’s I/O patterns• Small blobs from many places on disk• EBS best at bulk reads &writes
  • 26. Instance types• m1.large & m1.xlarge• EBS-optimized NEW!• Consider PIOPS NEW!• High I/O Quadruple XL (hi1.4xlarge) - SSDs! NEW!
  • 27. Prepare YourselfMore instances to compensatefor the performance variabilityof virtualized resources
  • 28. Cluster Sizes• 5 nodes or more Best performance & growth• Scales linearly• Don’t be stingy
  • 29. Ring Size• Change default value!• Cannot be reset• Ring size: 256 8/64 vnodes per node• Don’t go above 512
  • 30. Provisioning Riak Cluster App MyRiakCluster: node-0haproxy HTTP or MyRiakCluster: node-1 Protobuff Apphaproxy MyRiakCluster: node-2 App MyRiakCluster: node-3haproxy MyRiakCluster: node-4
  • 31. Disable Swap• Riak process pages should not hit swap• Swap usage can result in unresponsive server• Let the kernel kill it
  • 32. Mount & Scheduler• mount with noatime flag• Use deadline scheduler on EBS cat /sys/block/xvdj1/queue/scheduler noop [deadline] cfq echo deadline > /sys/block/xvdj1/ queue/scheduler
  • 33. Linux Filesystems• xfs, ext3• ext4 • barrier = 0 • data = writeback• ZFS not recommended
  • 34. Backend choices• Bitcask* • help mitigate EBS• LevelDB Can be changed• Memory and optimized!
  • 35. Kernel Optimizations• /etc/sysctl.conf vm.swappiness = 0 net.core.somaxconn= 4000 net.ipv4.tcp_timestamps = 0 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_max_syn_backlog = 40000 net.ipv4.tcp_sack = 1 net.ipv4.tcp_fin_timeout = 15 net.ipv4.tcp_keepalive_intvl = 30 net.ipv4.tcp_tw_reuse = 1 http://wiki.basho.com/Linux-Performance-Tuning.html
  • 36. Monitoring• stats or riak-admin status• Collectd & Nagios• Riaknostic• Riak Control (maybe soon)
  • 37. Backups• Backup both the ring and data directories• Integrate with EY tools• scp or rsync data directory• EBS snapshots
  • 38. Scaling Riak• Verticalriak-admin cluster replace <old> <new>• Horizontal • reduces node load • batch is better
  • 39. Benchmarks
  • 40. Tests• 5-node cluster / 5 hours• 50 concurrent connections• gets 10 /puts 5 /deletes 1• Bitcask backend• Protobuffs protocol
  • 41. Medium - PIOPS 500
  • 42. Medium - PIOPS 1K
  • 43. Medium - EphemeralFAIL!
  • 44. XL - EBS
  • 45. XL -PIOPS 500
  • 46. 4XL - EBS
  • 47. 4XL - PIOPS 1k
  • 48. What’s next?