This talk will give an overview of two exciting releases for Apache HBase 2.0 and Phoenix 5.0. HBase provides a NoSQL column store on Hadoop for random, real-time read/write workloads. Phoenix provides SQL on top of HBase. HBase 2.0 contains a large number of features that were a long time in development, including rewritten region assignment, performance improvements (RPC, rewritten write pipeline, etc), async clients and WAL, a C++ client, offheaping memstore and other buffers, shading of dependencies, as well as a lot of other fixes and stability improvements. We will go into details on some of the most important improvements in the release, as well as what are the implications for the users in terms of API and upgrade paths. Phoenix 5.0 is the next big Phoenix release because of its integration with HBase 2.0 and a lot of performance improvements in support of secondary Indexes. It has many important new features such as encoded columns, Kafka and Hive integration, and many other performance improvements. This session will also describe the uses cases that HBase and Phoenix are a good architectural fit for.

1. 1 © Hortonworks Inc. 2011–2018. All rights reserved
Apache HBase and Apache Phoenix
Alan Gates
Co-founder Hortonworks
@alanfgates

2. 2 © Hortonworks Inc. 2011–2018. All rights reserved
Phoenix and HBase Provides OLTP Solution for HDP
Flexible Schema
Columnar Storage
Optimal for Reads by Key
Insert, Update, Delete
Millisecond Latency
Strong Consistency
SQL and NoSQL Interfaces
Flexible Schema
Extreme Low Latency
Directly Integrated with Hadoop
SQL and NoSQL Interfaces
YARN : Data Operating System
HBase
RegionServer
1 ° ° ° ° ° ° ° ° ° °
° ° ° ° ° ° ° ° ° ° N
HDFS
(Permanent Data Storage)
HBase
RegionServer
HBase
RegionServer

3. 3 © Hortonworks Inc. 2011–2018. All rights reserved
HBase: Fast Facts
Largest Database
5 Petabytes
(Flurry)
Best Known App
Facebook Messages
(Facebook)
Fastest Ingestion
25 Million Events/s
(Alibaba)
Biggest SQL App
Real-Time SQL on 200m+ Records
(PubMatic)

4. 4 © Hortonworks Inc. 2011–2018. All rights reserved
Partition
Tolerant
Consistent Available
CP AP
CA
MongoDB
HBase
Redis
Cassandra
CouchBase
Dynamo
Riak
MySQL
Postgres
CAP Theorem

5. 5 © Hortonworks Inc. 2011–2018. All rights reserved

6. 6 © Hortonworks Inc. 2011–2018. All rights reserved
New Features in HBase 2

7. 7 © Hortonworks Inc. 2011–2018. All rights reserved
HBase 2.0 off-heap read
4-7x decrease in data access latency
Up to 2x increase in throughput

8. 8 © Hortonworks Inc. 2011–2018. All rights reserved
HBase 2: Compacting Memstore (aka. Accordion)
• In memory compaction
• In memory index reduction
• Reduce flushes to disk
• 30% reduction of write amplification
• 20% increase of write throughput
• 22% reduction of GC
https://blogs.apache.org/hbase/entry/accordion-hbase-breathes-with-in

9. 9 © Hortonworks Inc. 2011–2018. All rights reserved
Apache Phoenix:
• Originally from Salesforce to offload Oracle
workloads
• ANSI SQL (SQL-92+)
• Upsert for easy ingestion from streams
• Millisecond Latency
• Secondary Indexes
• Develop in JDBC, ODBC, .NET, Python and more
• Standalone or on top of existing HBase apps
Apache Phoenix Overview

10. 10 © Hortonworks Inc. 2011–2018. All rights reserved
HBase/Phoenix as the data store for mutable data
• HBase provides low-level, key-value store semantics
• Phoenix provides usability and accessibility
• Use the right tool and optimally-structured data
• Dynamic schema benefits on read and write
Flexible Schemas for Evolving Use-cases

11. 11 © Hortonworks Inc. 2011–2018. All rights reserved
⬢ Plugs into any ODBC-compatible BI tool
⬢ Phoenix layers natively on existing
HBase applications for ad-hoc SQL
analytics on Phoenix or HBase
applications
⬢ Self-service analytics and insight for The
Hadoop Database
Ad-Hoc Analytics for Phoenix or HBase
Phoenix or HBase Applications
BI Tools Connectivity

12. 12 © Hortonworks Inc. 2011–2018. All rights reserved
Native Interfaces (DB-API, DBI, etc.)
Applications within the Datacenter
Phoenix Query
Server
HBase
RegionServer
HBase
RegionServer
HBase
RegionServer
Phoenix Query
Server
Phoenix Query
Server
Protobuf over HTTP
Phoenix Query
Server
Edge Servers
Business Analysts +
Standard SQL Tools
Phoenix Query Server

13. 13 © Hortonworks Inc. 2011–2018. All rights reserved
New Features in Phoenix

14. 14 © Hortonworks Inc. 2011–2018. All rights reserved
HBase and Phoenix Connectors
GA
✗
Tech-preview
GA (RDD and DF)Tech-preview
GA in 3.0.x (DF only)

15. 15 © Hortonworks Inc. 2011–2018. All rights reserved
Phoenix Kafka Consumer
• Users often connect Kafka to HBase
• As of Phoenix 4.10, can now do it in SQL using Phoenix
serializer.columns=c1,c2,c3
jdbcUrl=jdbc:phoenix:localhost
table=SAMPLE2
ddl=CREATE TABLE IF NOT EXISTS SAMPLE2(uid VARCHAR NOT NULL,c1 VARCHAR,c2 VARCHAR,c3 VARCHAR
CONSTRAINT pk PRIMARY KEY(uid))
topics=topic1,topic2
...
Kafka-consumer-json.properties:
hadoop jar phoenix-kafka-minimal.jar org.apache.phoenix.kafka.consumer.PhoenixConsumerTool
--file /data/kafka-consumer-json.properties
Start the consumer:

16. 16 © Hortonworks Inc. 2011–2018. All rights reserved
SQL Cursors
PreparedStatement statement = conn.prepareStatement(
"DECLARE empCursor CURSOR FOR SELECT * FROM EMP_TABLE");
statement.execute();
statement = con.prepareStatement("OPEN empCursor");
statement.execute();
statement = con.prepareStatement("FETCH NEXT 10 ROWS FROM empCursor");
ResultSet rset = statement.execute();
while (rset.next() != null){
...
}

17. 17 © Hortonworks Inc. 2011–2018. All rights reserved
Repeatable Bernoulli Sampling
• TABLESAMPLE function allows only a portion of table to be read
• Stats include marker rows at intervals in the table
• TABLESAMPLE argument indicates probably of any given marker being chosen, all rows
from chosen marker to next marker read
• Deterministic function used to select markers, so select returns consistent results across
reads
SELECT * from PERSON TABLESAMPLE(10);

18. 18 © Hortonworks Inc. 2011–2018. All rights reserved
Plus More
• Optimize DISTINCT queries by pushing down seek logic to server
• Atomic UPSERT through new ON DUPLICATE KEY syntax
• Support Apache Spark 2.0 in Phoenix/Spark integration
• SELECT APPROX_COUNT_DISTINCT(x) – Uses hyperloglog, much faster than regular count
distinct

19. 19 © Hortonworks Inc. 2011–2018. All rights reserved
Thank You