Downloaded 80 times
Uploaded byTony Hillerson
Rails on HBase
This document summarizes a presentation about using the HBase database with Ruby on Rails applications. HBase is a distributed, column-oriented database that is well-suited for large datasets and integrates with Hadoop. The presentation covers what HBase is, its tradeoffs compared to relational databases, when it may be a good or bad fit, and examples of using HBase with Rails through libraries like hbase-stargate and MassiveRecord. It also includes a demo of running JRuby scripts against HBase and building a Rails frontend.
More Related Content
Rails on HBase
- 1. Rails on HBase ZacharyPinter and Tony Hillerson RailsConf 2011
- 2. What we willcover
- 3.
- 4. What are the tradeoffsthat HBase makes?
- 5. Why HBase is probablythe wrong choice for your app
- 6. Why HBase mightbe the right choice for your app
- 7. + How to useHBase with Rails
- 8. What we won’tcover
- 9. This is nota deep dive
- 10. This is nota guide to setting up a production HBase cluster *See http://cloudera.com
- 11.
- 12. HBase is basedon Bigtable
- 13. HBase Grew outof Hadoop
- 14.
- 15.
- 16. Tables column key core:link row llb0ga http://google.com http://en.oreilly.com/rails2011/public/ llb260 schedule/detail/17886
- 17. Column Families core column family stats column family key core:link stats:hits llb0ga http://google.com 100504 http://en.oreilly.com/rails2011/ llb260 2 public/schedule/detail/17886
- 18. Column Families Must be defined at creation or added to schema later Should group data accessed and sized similarly
- 19. Columns Can be dynamically added given that the family exists Stored as a byte array - do what you need with it
- 20. Column Families: DataLocality
- 21. Regions master llb0ga lld92q ... ... llbee9 lq1rro Regionserver Regionserver
- 22.
- 23.
- 24. hbase shell ruby-1.9.2-p180 :002> status 1 servers, 0 dead, 1.0000 average load ruby-1.9.2-p180 :003 > (1..10).each{ status } 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load => 1..10 ruby-1.9.2-p180 :004 >
- 25.
- 26. What are thetradeoffs that HBase makes? NoSQL = Tradeoffs
- 27. Highly Distributed Pros Cons • Built-in sharding • Makes joins difficult/impossible • No single server holds all the data • Limited real-time queries/sorting • Different servers operate on different slices of the data
- 28. Sorted Row Keys Bytearraykeys Any kind of data you want Sorted in byte order All row access through the key
- 29. Sorted Row Keys Pros Cons • Real-time random lookups by key • Conflates unique identifier with default • Real-time range queries sort • Keys are an integral part of schema design, requiring extra consideration
- 30. Map / Reduce Pros Cons • Scales linearly: more servers = faster • Scans every row per query * response times • Not real-time (batch operation) • Operates on huge datasets (billions of rows) • Works well with dynamic/non-uniform schemas
- 31. HBase vs. X http://nosql-database.org http://www.sevenweeks.org/ SevenDatabases in Seven Weeks Eric Redmond Don’t Miss His Talk!!
- 32. Why HBase is probablythe wrong choice for your app
- 33.
- 34. HBase Limitations Limited real-timequeries (key lookup, range lookup) Limited sorting (one default sort per table) Limited transactions Manual indexing Joins/Normalization Storage of large binary data *
- 35. HBase Requirements • Atleast 5 Servers (Zookeeper, Region Server, Name Nodes, Data Nodes, etc) • Memory Intensive (2-4gb bar minimum per server, depending on server role) • CPU Intensive • Rough EC2 Cost: 5 c1.xlarge reserved for 1 year = over $850/month (not counting bandwidth!)
- 36. Why HBase mightbe the right choice for your app
- 37. Large data sets Abilityto aggregate and analyze billions of rows Starts to shine when relational databases break down Is your db already sharded? Are you using SQL queries that take hours to run?
- 38. Hadoop Integration HDFS forstoring files Native Map/Reduce Supports data locality Doesn’t require data to be transferred
- 39. Flexible schema Supports non-heterogeneousdata through column families with mixed key/value pairs Ability to refactor the data store via map/reduce (beats a multi-day ALTER TABLE command)
- 40.
- 41.
- 42.
- 43.
- 44. Kwisatz: An URL Shortener
- 45.
- 46.
- 47.
- 48. Getting Set Up (Mac)brew install hbase Cloudera VM http://bit.ly/dSfPa9
- 49.
- 50. Where Would IUse HBase?
- 51. Where is HBase’s Sweet Spot
- 52. Where Would IUse Rails With HBase?
- 53.
- 54. Thank You! Zachary: @zpinter Tony:@thillerson EffectiveUI.com