BigBase is a read-optimized version of HBase NoSQL data store and is FULLY, 100% HBase compatible. 100% compatibility means that the upgrade from HBase to BigBase and other way around does not involve data migration and even can be made without stopping the cluster (via rolling restart).
3. About myself
• Principal Platform Engineer @Carrier IQ, Sunnyvale, CA
• Prior to Carrier IQ, I worked @ GE, EBay, Plumtree/BEA.
• HBase user since 2009.
• HBase hacker since 2013.
• Areas of expertise include (but not limited to) Java,
HBase, Hadoop, Hive, large-scale OLAP/Analytics, and in-
memory data processing.
• Founder of BigBase.org
11. Real Agenda
• Why BigBase?
• Brief history of BigBase.org project
• BigBase MLC high level architecture (L1/L2/L3)
• Level 1 - Row Cache.
• Level 2/3 - Block Cache RAM/SSD.
• YCSB benchmark results
• Upcoming features in R1.5, 2.0, 3.0.
• Q&A
12.
13. HBase
• Still lacks some original BigTable’s features.
• Still not able to utilize efficiently all RAM.
• No good mixed storage (SSD/HDD) support.
• Single Level Caching only. Simple.
• HBase + Large JVM Heap (MemStore) = ?
14. BigBase
• Adds Row Cache and block cache compression.
• Utilizes efficiently all RAM (TBs).
• Supports mixed storage (SSD/HDD).
• Has Multi Level Caching. Not that simple.
• Will move MemStore off heap in R2.
16. Koda (2010)
• Koda - Java off heap object cache, similar to
Terracotta’s BigMemory.
• Delivers 4x times more transactions …
• 10x times better latencies than BigMemory 4.
• Compression (Snappy, LZ4, LZ4HC, Deflate).
• Disk persistence and periodic cache snapshots.
• Tested up to 240GB.
17. Karma (2011-12)
• Karma - Java off heap BTree implementation to
support fast in memory queries.
• Supports extra large heaps, 100s millions – billions
objects.
• Stores 300M objects in less than 10G of RAM.
• Block Compression.
• Tested up to 240GB.
• Off Heap MemStore in R2.
18. Yamm (2013)
• Yet Another Memory Manager.
– Pure 100% Java memory allocator.
– Replaced jemalloc in Koda.
– Now Koda is 100% Java.
– Karma is the next (still on jemalloc).
– Similar to memcached slab allocator.
• BigBase project started (Summer 2013).
28. Where is BigTable’s Scan Cache?
• Scan Cache caches hot rows data.
• Complimentary to Block Cache.
• Still missing in HBase (as of 0.98).
• It’s very hard to implement in Java (off heap).
• Max GC pause is ~ 0.5-2 sec per 1GB of heap
• G1 GC in Java 7 does not resolve the problem.
• We call it Row Cache in BigBase.
34. BigBase Row Cache
• Off Heap Scan Cache for HBase.
• Cache size: 100’s of GBs to TBs.
• Eviction policies: LRU, LFU, FIFO,
Random.
• Pure 100% - compatible Java.
• Sub-millisecond latencies, zero GC.
• Implemented as RegionObserver
coprocessor.
Row Cache
YAMM Codecs
Kryo
SerDe
KODA
35. BigBase Row Cache
• Read through cache.
• It caches rowkey:CF.
• Invalidates key on every mutation.
• Can be enabled/disabled per table
and per table:CF.
• New ROWCACHE attribute.
• Best for small rows (< block size)
Row Cache
YAMM Codecs
Kryo
SerDe
KODA
36. Performance-Scalability
• GET (small rows < 100 bytes): 175K operations per sec
per one Region Server (from cache).
• MULTI-GET (small rows < 100 bytes): > 1M records per
second (network limited) per one Region Server.
• LATENCY : 99% < 1ms (for GETs) with 100K ops.
• Vertical scalability: tested up to 240GB (the maximum
available in Amazon EC2).
• Horizontal scalability: limited by HBase scalability.
• No more memcached farms in front of HBase clusters.
38. What is wrong with Bucket Cache?
Scalability LIMITED
Multi-Level Caching (MLC) NOT SUPPORTED
Persistence (‘offheap’ mode) NOT SUPPORTED
Low latency apps NOT SUPPORTED
SSD friendliness (‘file’ mode) NOT FRIENDLY
Compression NOT SUPPORTED
39. What is wrong with Bucket Cache?
Scalability LIMITED
Multi-Level Caching (MLC) NOT SUPPORTED
Persistence (‘offheap’ mode) NOT SUPPORTED
Low latency apps NOT SUPPORTED
SSD friendliness (‘file’ mode) NOT FRIENDLY
Compression NOT SUPPORTED
40. What is wrong with Bucket Cache?
Scalability LIMITED
Multi-Level Caching (MLC) NOT SUPPORTED
Persistence (‘offheap’ mode) NOT SUPPORTED
Low latency apps NOT SUPPORTED
SSD friendliness (‘file’ mode) NOT FRIENDLY
Compression NOT SUPPORTED
41. What is wrong with Bucket Cache?
Scalability LIMITED
Multi-Level Caching (MLC) NOT SUPPORTED
Persistence (‘offheap’ mode) NOT SUPPORTED
Low latency apps ?
SSD friendliness (‘file’ mode) NOT FRIENDLY
Compression NOT SUPPORTED
42. What is wrong with Bucket Cache?
Scalability LIMITED
Multi-Level Caching (MLC) NOT SUPPORTED
Persistence (‘offheap’ mode) NOT SUPPORTED
Low latency apps NOT SUPPORTED
SSD friendliness (‘file’ mode) NOT FRIENDLY
Compression NOT SUPPORTED
43. What is wrong with Bucket Cache?
Scalability LIMITED
Multi-Level Caching (MLC) NOT SUPPORTED
Persistence (‘offheap’ mode) NOT SUPPORTED
Low latency apps NOT SUPPORTED
SSD friendliness (‘file’ mode) NOT FRIENDLY
Compression NOT SUPPORTED
44. Here comes BigBase
Scalability HIGH
Multi-Level Caching (MLC) SUPPORTED
Persistence (‘offheap’ mode) SUPPORTED
Low latency apps SUPPORTED
SSD friendliness (‘file’ mode) SSD-FRIENDLY
Compression SNAPPY, LZ4, LZHC, DEFLATE
45. Here comes BigBase
Scalability HIGH
Multi-Level Caching (MLC) SUPPORTED
Persistence (‘offheap’ mode) SUPPORTED
Low latency apps SUPPORTED
SSD friendliness (‘file’ mode) SSD-FRIENDLY
Compression SNAPPY, LZ4, LZHC, DEFLATE
46. Here comes BigBase
Scalability HIGH
Multi-Level Caching (MLC) SUPPORTED
Persistence (‘offheap’ mode) SUPPORTED
Low latency apps SUPPORTED
SSD friendliness (‘file’ mode) SSD-FRIENDLY
Compression SNAPPY, LZ4, LZHC, DEFLATE
47. Here comes BigBase
Scalability HIGH
Multi-Level Caching (MLC) SUPPORTED
Persistence (‘offheap’ mode) SUPPORTED
Low latency apps SUPPORTED
SSD friendliness (‘file’ mode) SSD-FRIENDLY
Compression SNAPPY, LZ4, LZHC, DEFLATE
48. Here comes BigBase
Scalability HIGH
Multi-Level Caching (MLC) SUPPORTED
Persistence (‘offheap’ mode) SUPPORTED
Low latency apps SUPPORTED
SSD friendliness (‘file’ mode) SSD-FRIENDLY
Compression SNAPPY, LZ4, LZHC, DEFLATE
49. Here comes BigBase
Scalability HIGH
Multi-Level Caching (MLC) SUPPORTED
Persistence (‘offheap’ mode) SUPPORTED
Low latency apps SUPPORTED
SSD friendliness (‘file’ mode) SSD-FRIENDLY
Compression SNAPPY, LZ4, LZHC, DEFLATE
50. Wait, there are more …
Scalability HIGH
Multi-Level Caching (MLC) SUPPORTED
Persistence (‘offheap’ mode) SUPPORTED
Low latency apps SUPPORTED
SSD friendliness (‘file’ mode) SSD-FRIENDLY
Compression SNAPPY, LZ4, LZHC, DEFLATE
Non disk–based L3 cache SUPPORTED
RAM Cache optimization IBCO
51. Wait, there are more …
Scalability HIGH
Multi-Level Caching (MLC) SUPPORTED
Persistence (‘offheap’ mode) SUPPORTED
Low latency apps SUPPORTED
SSD friendliness (‘file’ mode) SSD-FRIENDLY
Compression SNAPPY, LZ4, LZHC, DEFLATE
Non disk–based L3 cache SUPPORTED
RAM Cache optimization IBCO
52. BigBase 1.0 vs. HBase 0.98
BigBase HBase 0.98
Row Cache (L1) YES NO
Block Cache RAM (L2) YES (fully off heap) YES (partially off heap)
Block Cache (L3) DISK YES (SSD- friendly) YES (not SSD – friendly)
Block Cache (L3) NON DISK YES NO
Compression YES NO
RAM Cache persistence YES (both L1 and L2) NO
Low Latency optimized YES NO
MLC support YES (L1, L2, L3) NO (either L2 or L3)
Scalability HIGH MEDIUM (limited by JVM heap)
64. What is next?
• Release 1.1 (2014 Q2)
– Support HBase 0.96, 0.98, trunk
– Fully tested L3 cache (SSD)
• Release 1.5 (2014 Q3)
– YAMM: memory allocator compacting mode .
– Integration with Hadoop metrics.
– Row Cache: merge rows on update (good for counters).
– Block Cache: new eviction policy (LRU-2Q).
– File read posix_fadvise ( bypass OS page cache).
– Row Cache: make it available for server-side apps
65. What is next?
• Release 2.0 (2014 Q3)
– HBASE-5263: Preserving cache data on compaction
– Cache data blocks on memstore flush (configurable).
– HBASE-10648: Pluggable Memstore. Off heap
implementation, based on Karma (off heap BTree lib).
• Release 3.0 (2014 Q4)
– Real Scan Cache – caches results of Scan operations on
immutable store files.
– Scan Cache integration with Phoenix and with other 3rd
party libs provided rich query API for HBase.
66. Download/Install/Uninstall
• Download BigBase 1.0 from www.bigbase.org
• Installation/upgrade takes 10-20 minutes
• Beatification operator EM(*) is invertible:
HBase = EM-1(BigBase) (the same 10-20 min)
67. Q & A
Vladimir Rodionov
Hadoop/HBase architect
Founder of BigBase.org
HBase: Extreme makeover
Features & Internal Track