SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
4.
What
is
a
block?
HFile
v2
Format
HFile
v2
format
figure
reproduced
from
MaMeo
Bertozzi,
"Apache
HBase
I/O
–
HFile”,
hMp://blog.cloudera.com/blog/2012/06/hbase-‐io-‐
hfile-‐input-‐output/
5.
Why
cache
blocks?
Storage hierarchy: a different view
Dean/2009
Memory
is
100000x
faster
than
disk!
Disk
seek
=
10ms
6.
LruBlockCache!
• On-‐heap
ConcurrentHashMap!
• 3
block
prioriFes,
based
on
access
frequency
– Single
– MulF
– In-‐memory
• Background
evicFon
thread
• HBASE-‐1460,
0.20
10.
BucketCache!
• L2
cache;
fronted
by
LruBlockCache
(L1)
• CombinedBlockCache
caches
data
blocks
in
L2,
everything
else
in
L1
• Blocks
evicted
from
L1
are
promoted
to
L2
• Allocates
porFons
of
space
with
target
block
size
• Three
operaFng
modes:
– onheap
–
HeapByteBuffers
manage
on-‐heap
memory
space
– offheap
–
DirectByteBuffers
manage
off-‐heap
memory
space
– file
–
RandomAccessFile
manages
file-‐backed
space
• Blocks
copied
on
and
off
heap
• HBASE-‐7404,
0.96
11.
BucketCache
ConfiguraFons
• OperaFng
mode
– hbase.bucketcache.ioengine!
• Total
cache
size
as
pct
of
heap
OR
size
in
MB:
– hbase.bucketcache.size!
• Size
of
L1
as
pct
of
total
cache
size:
– hbase.bucketcache.percentage.in.combinedcache!
– default:
0.9
• Target
Block
size
– hbase.offheapcache.minblocksize!
– default:
64k
• Bucket
target
block
sizes
– hbase.bucketcache.bucket.sizes!
– default:
14
sizes,
clustered
around
64k
12.
BlockCache
Showdown
• Compare
performance:
– Total
cache
size
– Cache
size
vs.
Data
size
n10k.com/blog/
blockcache-‐showdown/
L2
off-‐heap
BucketCache
makes
a
strong
showing
13.
Thanks!
M A N N I N G
Nick Dimiduk
Amandeep Khurana
FOREWORD BY
Michael Stack
hbaseinacFon.com
Nick
Dimiduk
github.com/ndimiduk
@xefyr
n10k.com
n10k.com/blog/blockcache-‐101