© Hortonworks Inc. 2011
Apache HBase
For Architects
Nick Dimiduk
Member of Technical Staff, HBase
Seattle Technical Forum, 2013-05-15
Page 1

© Hortonworks Inc. 2011
Page 2
Architecting the Future of Big Data

© Hortonworks Inc. 2011
Agenda
• Background
– (how did we get here?)
• High-level Architecture
– (where are we?)
• Anatomy of a RegionServer
– (how does this thing work?)
• TL;DR
– (what did we learn?)
• Resources
– (where do we go from here?)
Page 3
Architecting the Future of Big Data

© Hortonworks Inc. 2011
Background
Architecting the Future of Big Data
Page 4

© Hortonworks Inc. 2011
Apache Hadoop in Review
• Apache Hadoop Distributed Filesystem (HDFS)
– Distributed, fault-tolerant, throughput-optimized data storage
– Uses a filesystem analogy, not structured tables
– The Google File System, 2003, Ghemawat et al.
– http://research.google.com/archive/gfs.html
• Apache Hadoop MapReduce (MR)
– Distributed, fault-tolerant, batch-oriented data processing
– Line- or record-oriented processing of the entire dataset *
– “[Application] schema on read”
– MapReduce: Simplified Data Processing on Large Clusters, 2004, Dean and
Ghemawat
– http://research.google.com/archive/mapreduce.html
Page 5
Architecting the Future of Big Data
* For more on writing MapReduce applications, see “MapReduce
Patterns, Algorithms, and Use Cases”
http://highlyscalable.wordpress.com/2012/02/01/mapreduce-patterns/

© Hortonworks Inc. 2011
So what is HBase anyway?
• BigTable paper from Google, 2006, Dean et al.
– “Bigtable is a sparse, distributed, persistent multi-dimensional sorted map.”
– http://research.google.com/archive/bigtable.html
• Key Features:
– Distributed storage across cluster of machines
– Random, online read and write data access
– Schemaless data model (“NoSQL”)
– Self-managed data partitions
Page 6
Architecting the Future of Big Data

© Hortonworks Inc. 2011
High-level Architecture
Architecting the Future of Big Data
Page 7

© Hortonworks Inc. 2011
Page 9
Architecting the Future of Big Data
Logical Architecture
Distributed, persistent partitions of a BigTable
a
b
d
c
e
f
h
g
i
j
l
k
m
n
p
o
Table A
Region 1
Region 2
Region 3
Region 4
Region Server 7
Table A, Region 1
Table A, Region 2
Table G, Region 1070
Table L, Region 25
Region Server 86
Table A, Region 3
Table C, Region 30
Table F, Region 160
Table F, Region 776
Region Server 367
Table A, Region 4
Table C, Region 17
Table E, Region 52
Table P, Region 1116
Legend:
- A single table is partitioned into Regions of roughly equal size.
- Regions are assigned to Region Servers across the cluster.
- Region Servers host roughly the same number of regions.

© Hortonworks Inc. 2011
Page 11
Architecting the Future of Big Data
Physical Architecture
Distribution and Data Path
...
Zoo
Keeper
Zoo
Keeper
Zoo
Keeper
HBase
Client
JavaApp
HBase
Client
JavaApp
HBase
Client
HBase Shell
HBase
Client
REST/Thrift
Gateway
HBase
Client
JavaApp
HBase
Client
JavaApp
Region
Server
Data
Node
Region
Server
Data
Node
...
Region
Server
Data
Node
Region
Server
Data
Node
HBase
Master
Name
Node
Legend:
- An HBase RegionServer is collocated with an HDFS DataNode.
- HBase clients communicate directly with Region Servers for sending and receiving data.
- HMaster manages Region assignment and handles DDL operations.
- Online configuration state is maintained in ZooKeeper.
- HMaster and ZooKeeper are NOT involved in data path.

© Hortonworks Inc. 2011
Page 13
Architecting the Future of Big Data
Logical Data Model
A sparse, multi-dimensional, sorted map
Legend:
- Rows are sorted by rowkey.
- Within a row, values are located by column family and qualifier.
- Values also carry a timestamp; there can me multiple versions of a value.
- Within a column family, data is schemaless. Qualifiers and values are treated as arbitrary bytes.
1368387247 [3.6 kb png data]"thumb"cf2b
a
cf1
1368394583 7
1368394261 "hello"
"bar"
1368394583 22
1368394925 13.6
1368393847 "world"
"foo"
cf2
1368387684 "almost the loneliest number"1.0001
1368396302 "fourth of July""2011-07-04"
Table A
rowkey
column
family
column
qualifier
timestamp value

© Hortonworks Inc. 2011
Anatomy of a
RegionServer
Architecting the Future of Big Data
Page 14

© Hortonworks Inc. 2011
Page 16
Architecting the Future of Big Data
RegionServer
HDFS
HLog
(WAL)
HRegion
HStore
StoreFile
HFile
StoreFile
HFile
MemStore
...
...
HStore
BlockCache
HRegion
...
HStoreHStore
...
Legend:
- A RegionServer contains a single WAL, single BlockCache, and multiple Regions.
- A Region contains multiple Stores, one for each Column Family.
- A Store consists of multiple StoreFiles and a MemStore.
- A StoreFile corresponds to a single HFile.
- HFiles and WAL are persisted on HDFS.
Storage Machinery
Implementing the data model

© Hortonworks Inc. 2011
TL;DR
Architecting the Future of Big Data
Page 21

© Hortonworks Inc. 2011
For what kinds of workloads is it well suited?
• It depends on how you tune it, but…
• HBase is good for:
– Large datasets
– Sparse datasets
– Loosely coupled (denormalized) records
– Lots of concurrent clients
• Try to avoid:
– Small datasets (unless you have lots of them)
– Highly relational records
– Schema designs requiring transactions *
Page 22
Architecting the Future of Big Data
* Transactions might not be as necessary as you think, see “Eric
Brewer on why banks are BASE not ACID”
http://highscalability.com/blog/2013/5/1/myth-eric-brewer-on-why-
banks-are-base-not-acid-availability.html
** Or maybe not, “We believe it is better to have application
programmers deal with performance problems due to overuse of
transactions as bottlenecks arise, rather than always coding around
the lack of transactions.” – Google Spanner paper, http://
research.google.com/archive/spanner.html

© Hortonworks Inc. 2011
How does it integrate with my infrastructure?
• Horizontally scale application data
– Highly concurrent, read/write access
– Consistent, persisted shared state
– Distributed online data processing via Coprocessors (experimental)
• Gateway between online services and offline storage/analysis
– Staging area to receive new data
– Serve online, indexed “views” on datasets from HDFS
– Glue between batch (HDFS, MR1) and online (CEP, Storm) systems
Page 23
Architecting the Future of Big Data

© Hortonworks Inc. 2011
What data semantics does it provide?
• GET, PUT, DELETE key-value operations
• SCAN for queries
• INCREMENT, CAS server-side atomic operations
• Row-level write atomicity
• MapReduce integration
– Online API (today)
– Bulkload (today)
– Snapshots (coming)
Page 24
Architecting the Future of Big Data

© Hortonworks Inc. 2011
What about operational concerns?
• Provision hardware with more spindles/TB
• Balance memory and IO for reads
– Contention between random and sequential access
– Configure Block size, BlockCache, compression, codecs based on access patterns
– Additional resources
– “HBase: Performance Tuners,” http://labs.ericsson.com/blog/hbase-performance-tuners
– “Scanning in HBase,” http://hadoop-hbase.blogspot.com/2012/01/scanning-in-
hbase.html
• Balance IO for writes
– Configure C1 (compactions, region size, compression, pre-splits, &c.) based on
write pattern
– Balance IO contention between maintaining C1 and serving reads
– Additional resources
– “Configuring HBase Memstore: what you should know,” http://blog.sematext.com/
2012/07/16/hbase-memstore-what-you-should-know/
– “Visualizing HBase Flushes And Compactions,” http://www.ngdata.com/visualizing-
hbase-flushes-and-compactions/
Page 25
Architecting the Future of Big Data

© Hortonworks Inc. 2011
Resources
Architecting the Future of Big Data
Page 26

© Hortonworks Inc. 2011
Join the Community!
• hbase.apache.org
– blogs.apache.org/hbase/
• Mailing lists
– hbase.apache.org/mail-lists.html
– user@hbase.apache.org
• IRC
– irc.freenode.net#hbase
• JIRA
– issues.apache.org/jira/browse/HBASE
• Source
– git clone git://git.apache.org/hbase.git
– svn checkout http://svn.apache.org/repos/asf/hbase/trunk hbase
• Conference Season
– HBaseCon 2013, June 13, hbasecon.com
– Hadoop Summit, June 26-27, hadoopsummit.org
Page 27
Architecting the Future of Big Data

© Hortonworks Inc. 2011
HBase@Hortonworks
• Mean Time To Recovery (MTTR)
– HDFS improvements, faster recovery of META, log replay instead of log splitting,
improving failure detection
• Testing
– Integration test suite, system tests, destructive testing, ChaosMonkey, load tests,
Namenode HA, test coverage and consistency
• Compaction Improvements
– Pluggable compaction, tier based compaction, stripe / leveldb compactions, etc
• IPC / Wire compatibility
– Migration to Google’s Protocol Buffers
• HBase MapReduce improvements (Import / Export, etc)
– Performance improvements, API uniformity/usability
• Hardening 0.94
– Assignment Manager, Log splitting, Region splits, Replication
• Not to mention:
– Windows support, Security, Snapshots, Hadoop2, 0.96, LOTS of bug fixes and
community reviews
Page 28
Architecting the Future of Big Data

© Hortonworks Inc. 2011
Thanks!
Architecting the Future of Big Data
Page 29
M A N N I N G
Nick Dimiduk
Amandeep Khurana
FOREWORD BY
Michael Stack
hbaseinaction.com
Nick Dimiduk
github.com/ndimiduk
@xefyr
n10k.com