HBase talk @ OSDC.TW 2009

1. HBase
Distributed, column-oriented database on Hadoop
Rong-En Fan
rafan@infor.org
OSDC.TW 2009
Í7Ån2 CC-SA 2.5 0J
Rong-En Fan (rafan) HBase Apr 19, 2009 1 / 13

2. 7‚Šý HBase¨
.
@Î Open Source. . .
‡1¨3œ9߇u32…«Õ‹—«D‹—
×Ëb½y RDBMS Ý;DÙ
K 2009 Ýý FreeBSD ports. . .
×OG$§„àÝ HBase ¡Æÿœb¶
Rong-En Fan (rafan) HBase Apr 19, 2009 2 / 13

3. $Î RDBMS
HBase
^b×À RDBMS bÝ‹—§relational, joins,
types, SQL, query engine, . . .
bװ RDBMS bݧtransaction, secondary
indices (K^œW!)
X|...µ§à MySQL Ï! (0)
Rong-En Fan (rafan) HBase Apr 19, 2009 3 / 13

4. Έ{¨
HBase
distributed database modeled on Google’s Bigtable1
tables of column-oriented rows
scalable data store (scales horizontally !!!)
billions of rows X millions of columns X thousands of
versions
running on top of “commodity” hardware
Apache Hadoop subproject since 2008
1
Bigtable: A Distributed Storage System for Structured Data
Rong-En Fan (rafan) HBase Apr 19, 2009 4 / 13

5. §‚D‹—¨
simple key-value
(table, row, family:column, timestamp) → value
table: rows are sorted
column family: any number of columns, columns are
sorted and stored together
timestamp: one column can have multiple versions
value: just byte array!
Rong-En Fan (rafan) HBase Apr 19, 2009 5 / 13

6. §‚¢¡¨
Ts! (table → regions)
×Íxߐ.Sg .ø°‹— (one master
w/ many region servers)
xß,ݵIà# (ì×̺bߊ›#W©xß)
bß,ÝÝ•¡Í€ßºŠ›#W
Rong-En Fan (rafan) HBase Apr 19, 2009 6 / 13

7. §‚=¨
Java API
Thrift for PHP, Ruby, Python, C, . . .
REST (JSON or XML)
No ACL, encryption for now
Rong-En Fan (rafan) HBase Apr 19, 2009 7 / 13

8. §‚Dã¨
ÃÍÝDãÿP§Create, Read, Update, Delete
(CRUD) « (table) Scan
à primary key Ý^DãTïÎà scan Ý1g
Dã
ÓÝ‹—KŠ3ŠÝP…«ŠX¡ô|
Õ?DR¼
AŒŠà web server à#ÆÝ•¡˜ÈÎG«
×· memcached (vÝ‹—
Rong-En Fan (rafan) HBase Apr 19, 2009 8 / 13

9. b7‚Tà¨
ÎlÝàHΩ† crawler Ý;DÙ
Ý¥Õ(©Î=”¢7n(vÝ)
(webtable)¡…«wÝÎ8¼Ýç°«×ÀbÝ^
1gDã§5—ç°/¢† index
^Dã§?±ç°/¢èºquot;ãç°
J¼D¡Ý log Q¡†S—§©Î Google
Analytics
Rong-En Fan (rafan) HBase Apr 19, 2009 9 / 13

10. 1Ý9‚9¡Õ9b 3à¨
Powerset, a Microsoft company
Trend Micro: Advanced Threats Research
HBase PoweredBy list
Mahalo (the first human-powered search engine)
Streamy (realtime social news site)
WorldLingo (multilingual archive)
Rong-En Fan (rafan) HBase Apr 19, 2009 10 / 13

11. ݨµ
HBase
êGb4Íf´ðŒ^Ý committers¡b°¢
4Íf´þ$Ý contributors
JÍŒiÎã Jim Kellerman (@Powerset) ñr
ì×Ì (0.20) ji[«%—;Ÿð9
Rong-En Fan (rafan) HBase Apr 19, 2009 11 / 13

12. Õ9$à«...
¡VsÝËO (2007 OÞ–Õ¨3)¡ŒÝèÌ
(êGÎ 0.19.1)
(ŠÝŽÆ)
Ê)Ø°ËvÝTà
g)Ê©ÝÚx|J¼†aîDã
%—“«ì×Ì (0.20) º;Ÿœ9
Rong-En Fan (rafan) HBase Apr 19, 2009 12 / 13

13. װ HBase ݣ]
“ç°
HBase
IRC channel #hbase on freenode
Rong-En Fan (rafan) HBase Apr 19, 2009 13 / 13