HBase is a distributed, column-oriented database that is modeled after Google's Bigtable. It runs on top of HDFS and provides real-time read/write access to large datasets. HBase tables are split into regions that can be distributed across multiple servers. It uses a log-structured merge tree to store data on disk for efficient read/write operations. HBase is well suited for handling large volumes of randomly accessible data.

1. Kim yong-duk
Feb 21, 2016
Google's mission is to organize the world's information and make it universally accessible and
useful
HBASE

2. Contents
■ Intro
□ HBasic
□ Tips
□ Conclusion
□ Q/A

3. HBase
• Google’s Big Table
• Bigtable: A Distributed Storage System for Structed Data
• Lars George
• Facebook, Twitter, Salesforce.com, Adobe, Line,..
• No-SQL
• Java

4. Contents
□ Intro
■ HBasic
□ Tips
□ Conclusion
□ Q/A

5. Positionning
• Distributed Column oriented
• Flexible
• Automatic sharding and load balancing
• Scalability
• Key-Value
• Consistent Read & Write
• HDFS
• Automatic failover between RegionServers
• Various clients such as Java API, REST, Thrift, Hive, Pig, MapReduce,..

6. Architecture
< NexR 자료에서 발췌 >

7. Master
• Monitoring RegionServers
• Manage Metadata
• Interface for Table (create,modify,..), ColumnFamily (add,modify,..), Region (move, assign,..)
• Multi-Master by Zookeeper
• Background Thread such as LoadBalancer for Region

8. RegionServers
• A RegionServer manages N – Region
• Interface for data (get,put,delete,..), region (split,compaction,..)
• Background Threads
- CompactSplit : split chck, minor compaction
- MajorCompactionChecker : major compaction
- MemstoreFlusher : flush memstore to storefile
- LogRoller : hlog check

9. Data Model (1/2)
• Table = ∑ Row
• Row = ∑ ColumnFamily
• ColumnFamily = ∑ Column
• Column
- unlimited
- dynamic
• Cell
- a {row, column, version} tuple
- uninterpreted bytes

10. Data Model (2/2)
< Bigtable: A Distributed Storage System for Structed Data >

11. Region
• A Region is managed by a RegionServer
• Table = ∑Region
• Region = ∑Store
• Region Size as performance-inducer
• Region name = table name + start row of region + region creation timestamp
< example: regions >

12. Store
• A Store must manage 1- Column Family only
• Store = Memstore + ∑StoreFile
• StoreFile = ∑Block (not equals block of HDFS)
• MemStore (arranged ,flushed to StoreFile if the size exceeds)
• StoreFile=HFile (From memstore, No modification, Minor/Major Compaction)
• Log-Structured Merge tree
- Write => Memstore(sorted) –flush–> Hfile
- Read <= Memstore + Hfile
- Bloom Filter
< key:value >

13. Key/Value
• Low-level byte array
• Zero-copy access
• Key length, Value length ,..
• Small key
< key/value format >

14. Logical vs. on-disk layout of a table
• Main unit of separation within a table is the column family
• Although cells are stored logically in a table format, rows are stored as linear sets of
the cells
• Cells contain all the vital information inside them

15. Write (1/2)
• put(), delete(), increment()
• Key/value Instance
• RPC call
• Send Key/value instance to HRegion
• WAL -> MemStore (A RegionServer manages 1- Hlog)
• MemStore Flush

16. Write (2/2)

17. Read
< Bigtable: A Distributed Storage System for Structed Data >
• Scan all store (memstore, hfile) of the region

18. Compression
• Column Family Level
• Ct + Nt < nCt + Nt, so must be sended after compressed!!
• SNAPPY, GZIP, LZO

19. Client - WEB
• Master UI : http://${HBASE_MASTER_HOST}:60010
• RegionServer UI : http://${HBASE_REGION_SERVER_HOST}:60030

20. Client – SHELL
• Command Line Interface
• Jruby

21. Hive Hbase StorageHandler
• Client ––<jdbc>––<haproxy>––hiveservers––hbase
• Web can access to Hbase by HQL
• Components of HiveStorageHandler
- input formats
- output formats
- SerDe (Serialize/Deserialize library)

22. hbase-site.xml
• hbase.rootdir
- The directory shared by region servers and into which HBase persists. [hdfs://hadoop1:9000/hbase]
• hbase.zookeeper.quorum
- zookeeper servers [ehadoop22,ehadoop23]
• hbase.regionservers.codecs
- Compression codecs [snappy]
• hbsae.hregion.max.filesize
- Maximum HStoreFile size [17179869184]
• hbase.hregion.memstore.flush.size
- Memstore will be flushed to disk if size of the memstore exceeds this number of bytes [134217728]
• hbase.hregion.majorcompaction
- The time (in miliseconds) between 'major' compactions of all HStoreFiles in a region [0 -> Disable!!]

23. Contents
□ Intro
□ HBasic
■ Tips
□ Conclusion
□ Q/A

24. RowKey Design
• Monotonically Increasing Row Keys/TimeSeries problems!!
• Try to minimize row and column sizes
• Reverse timestamp
- Long.MAX_VALUE – timestamp
• Composite key 0 padding (10 -> 2, 02 -> 10)
• Binary < ascii (number)

25. Monotonically increasing values are bad.
• http://ikaisays.com/2011/01/25/app-engine-datastore-tip-monotonically-increasing-
values-are-bad/

26. Contents
□ Intro
□ HBasic
□ Tips
■ Conclusion
□ Q/A

27. Conclusion
• NoSQL (main query is API, not SQL), C + P
• Key/value based on Columns
• Ad-hoc query (Neary real-time)
• Compressed by level of Column-Family
• HDFS based
• Mapreduce connectivity
• Access to Hbase by jdbc
• CouchBase, MongoDB, Cassandra,..

28. Contents
□ Intro
□ HBasic
□ Tips
□ Conclusion
■ Q/A

29. Q/A
• Why BigTable use column-family?