Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Real-time HBase: Lessons from the Cloud

3,655 views

Published on

Speaker: Bryan Beaudreault (HubSpot)

Running HBase in real time in the cloud provides an interesting and ever-changing set of challenges -- instance types are not ideal, neighbors can degrade your performance, and instances can randomly die in unanticipated ways. This talk will cover what HubSpot has learned about running in production on Amazon EC2, how it handle DR and redundancy, and the tooling the team has found to be the most helpful.

  • Be the first to comment

Real-time HBase: Lessons from the Cloud

  1. 1. Real-time HBase: Lessons from the Cloud Bryan Beaudreault, @HubSpotDev
  2. 2. You’re doing it WRONG!
  3. 3. Instance types Network, Neighbors,
  4. 4. PERSPECTIVE: Consumer/ Prospect
  5. 5. PERSPECTIVE: Consumer/ Prospect
  6. 6. PERSPECTIVE: Consumer/ Prospect
  7. 7. PERSPECTIVE: Consumer/ Prospect
  8. 8. Improve reads, limit impact
  9. 9. PERSPECTIVE: Consumer/ Prospect
  10. 10. PERSPECTIVE: Consumer/ Prospect
  11. 11. PERSPECTIVE: Consumer/ Prospect
  12. 12. Over-provision, fail fast
  13. 13. PERSPECTIVE: Consumer/ Prospect
  14. 14. PERSPECTIVE: Consumer/ Prospect
  15. 15. PERSPECTIVE: Consumer/ Prospect
  16. 16. PERSPECTIVE: Consumer/ Prospect
  17. 17. PERSPECTIVE: Consumer/ Prospect
  18. 18. PERSPECTIVE: Consumer/ Prospect
  19. 19. PERSPECTIVE: Consumer/ Prospect
  20. 20. c1.xlarge i2.4xlarge CPU heavy workloads Reduce memory footprint Add more servers Excellent, but expensive Use data encoding to reduce disk Use Java7 and G1 to reduce GCs m1.xlarge Memory heavy workloads Beware compactions More regions
  21. 21. PERSPECTIVE: Consumer/ Prospect
  22. 22. cpu fsWriteLatency fsReadLatency load callQueueLen compactionQueueSize iowait steal heap locality requests memstoreSizeMB blockCacheHitCachingRatio flushQueueSize
  23. 23. PERSPECTIVE: Consumer/ Prospect
  24. 24. Master HBase with us. dev.hubspot.com
  25. 25. Bryan Beaudreault @HubSpotDev

×