SlideShare a Scribd company logo
Apache Phoenix
James Taylor @JamesPlusPlus
Maryann Xue @MaryannXue
Eli Levine @teleturn
We put the SQL back in NoSQL
http://phoenix.incubator.apache.org
About James
Completed
o Engineer at Salesforce.com in BigData group
o Started Phoenix as internal project ~2.5 years ago
o Open-source on Github ~1.5 years ago
o Apache incubator for past 5 months
o Engineer at BEA Systems
o XQuery-based federated query engine
o SQL-based complex event processing engine
Agenda
Completed
o What is Apache Phoenix?
o Why is it so fast?
o How does it help HBase scale?
o Roadmap
o Q&A
What is Apache Phoenix?
Completed
What is Apache Phoenix?
Completed
1. Turns HBase into a SQL database
o Query Engine
o MetaData Repository
o Embedded JDBC driver
o Only for HBase data
What is Apache Phoenix?
Completed
2. Fastest way to access HBase data
o HBase-specific push down
o Compiles queries into native
HBase calls (no map-reduce)
o Executes scans in parallel
Completed
SELECT * FROM t WHERE k IN (?,?,?)
Phoenix Stinger (Hive 0.13)
0.04 sec 280 sec
* 110M row table
7,000x faster
What is Apache Phoenix?
Completed
3. Lightweight
o No additional servers required
o Bundled with Hortonworks 2.1
o 100% Java
HBase Cluster Architecture
HBase Cluster Architecture
Phoenix
HBase Cluster Architecture
Phoenix
Phoenix
What is Apache Phoenix?
Completed
4. Integration-friendly
o Map to existing HBase table
o Integrate with Apache Pig
o Integrate with Apache Flume
o Integrate with Apache Sqoop (wip)
What is Apache Phoenix?
Completed
1. Turns HBase into a SQL database
2. Fastest way to access HBase data
3. Lightweight
4. Integration-friendly
Why is Phoenix so fast?
Completed
Why is Phoenix so fast?
Completed
1. HBase
o Fast, but “dumb” (on purpose)
2. Data model
o Support for composite primary key
o Binary data sorts naturally
3. Client-side parallelization
4. Push down
o Custom filters and coprocessors
Phoenix Data Model
HBase Table
Phoenix maps HBase data model to the relational world
Phoenix Data Model
HBase Table
Column Family A Column Family B
Phoenix maps HBase data model to the relational world
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Phoenix maps HBase data model to the relational world
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 KeyValue
Phoenix maps HBase data model to the relational world
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 KeyValue
Row Key 2 KeyValue KeyValue
Phoenix maps HBase data model to the relational world
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 KeyValue
Row Key 2 KeyValue KeyValue
Row Key 3 KeyValue
Phoenix maps HBase data model to the relational world
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 Value
Row Key 2 Value Value
Row Key 3 Value
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 KeyValue
Row Key 2 KeyValue KeyValue
Row Key 3 KeyValue
Phoenix maps HBase data model to the relational world
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 Value
Row Key 2 Value Value
Row Key 3 Value
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 Value
Row Key 2 Value Value
Row Key 3 Value
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 KeyValue
Row Key 2 KeyValue KeyValue
Row Key 3 KeyValue
Phoenix maps HBase data model to the relational world
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 Value
Row Key 2 Value Value
Row Key 3 Value
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 Value
Row Key 2 Value Value
Row Key 3 Value
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 KeyValue
Row Key 2 KeyValue KeyValue
Row Key 3 KeyValue
Phoenix maps HBase data model to the relational world
Multiple Versions
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 KeyValue
Row Key 2 KeyValue KeyValue
Row Key 3 KeyValue
Phoenix maps HBase data model to the relational world
Phoenix Table
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 KeyValue
Row Key 2 KeyValue KeyValue
Row Key 3 KeyValue
Phoenix maps HBase data model to the relational world
Phoenix Table
Key Value Columns
Phoenix Data Model
HBase Table
Column Family A Column Family B
Qualifier 1 Qualifier 2 Qualifier 3
Row Key 1 KeyValue
Row Key 2 KeyValue KeyValue
Row Key 3 KeyValue
Phoenix maps HBase data model to the relational world
Phoenix Table
Key Value ColumnsPrimary Key Constraint
Example
Row Key
SERVER METRICS
HOST VARCHAR
DATE DATE
RESPONSE_TIME INTEGER
GC_TIME INTEGER
CPU_TIME INTEGER
IO_TIME INTEGER
Over metrics data for servers with a schema like this:
Example
Over metrics data for servers with a schema like this:
Key Values
SERVER METRICS
HOST VARCHAR
DATE DATE
RESPONSE_TIME INTEGER
GC_TIME INTEGER
CPU_TIME INTEGER
IO_TIME INTEGER
Example
CREATE TABLE SERVER_METRICS (
HOST VARCHAR,
DATE DATE,
RESPONSE_TIME INTEGER,
GC_TIME INTEGER,
CPU_TIME INTEGER,
IO_TIME INTEGER,
CONSTRAINT pk PRIMARY KEY (HOST, DATE))
DDL command looks like this:
With data that looks like this:
SERVER METRICS
HOST + DATE RESPONSE_TIME GC_TIME
SF1 1396743589 1234
SF1 1396743589 8012
…
SF3 1396002345 2345
SF3 1396002345 2340
SF7 1396552341 5002 1234
…
Example
Row Key
With data that looks like this:
SERVER METRICS
HOST + DATE RESPONSE_TIME GC_TIME
SF1 1396743589 1234
SF1 1396743589 8012
…
SF3 1396002345 2345
SF3 1396002345 2340
SF7 1396552341 5002 1234
…
Example
Key Values
Phoenix Push Down: Example
Completed
SELECT host, avg(response_time)
FROM server_metrics
WHERE date > CURRENT_DATE() – 7
AND host LIKE ‘SF%’
GROUP BY host
Phoenix Push Down: Example
Completed
SELECT host, avg(response_time)
FROM server_metrics
WHERE date > CURRENT_DATE() – 7
AND host LIKE ‘SF%’
GROUP BY host
Phoenix Push Down: Example
Completed
SELECT host, avg(response_time)
FROM server_metrics
WHERE date > CURRENT_DATE() – 7
AND host LIKE ‘SF%’
GROUP BY host
Phoenix Push Down: Example
Completed
SELECT host, avg(response_time)
FROM server_metrics
WHERE date > CURRENT_DATE() – 7
AND host LIKE ‘SF%’
GROUP BY host
Phoenix Push Down: Example
Completed
SELECT host, avg(response_time)
FROM server_metrics
WHERE date > CURRENT_DATE() – 7
AND host LIKE ‘SF%’
GROUP BY host
Phoenix Push Down
1. Skip scan filter
2. Aggregation
3. TopN
4. Hash Join
Phoenix Push Down: Skip scan
SELECT host, avg(response_time)
FROM server_metrics
WHERE date > CURRENT_DATE() – 7
AND host LIKE ‘SF%’
GROUP BY host
Phoenix Push Down: Skip scan
Completed
R1
R2
R3
R4
Phoenix Push Down: Skip scan
Client-side parallel scans
Completed
R1
R2
R3
R4
scan1
scan3
scan2
Phoenix Push Down: Skip scan
Server-side filter
Completed
SKIP
Phoenix Push Down: Skip scan
Server-side filter
Completed
INCLUDE
Phoenix Push Down: Skip scan
Server-side filter
Completed
SKIP
Phoenix Push Down: Skip scan
Server-side filter
Completed
INCLUDE
Phoenix Push Down: Skip scan
Server-side filter
SKIP
Phoenix Push Down: Skip scan
Server-side filter
INCLUDE
Phoenix Push Down: Skip scan
Server-side filter
INCLUDE
INCLUDE
INCLUDE
Phoenix Push Down: Aggregation
SELECT host, avg(response_time)
FROM server_metrics
WHERE date > CURRENT_DATE() – 7
AND host LIKE ‘SF%’
GROUP BY host
SERVER METRICS
HOST DATE KV1 KV2 KV3
SF1 Jun 2 10:10:10.234 239 234 674
SF1 Jun 3 23:05:44.975 23 234
SF1 Jun 9 08:10:32.147 256 314 341
SF1 Jun 9 08:10:32.147 235 256
SF1 Jun 1 11:18:28.456 235 23
SF1 Jun 3 22:03:22.142 234 314
SF1 Jun 3 22:03:22.142 432 234 256
SF2 Jun 1 10:29:58.950 23 432
SF2 Jun 2 14:55:34.104 314 876 23
SF2 Jun 3 12:46:19.123 256 234 314
SF2 Jun 3 12:46:19.123 432
SF2 Jun 8 08:23:23.456 876 876 235
SF2 Jun 1 10:31:10.234 234 234 876
SF3 Jun 1 10:31:10.234 432 432 234
SF3 Jun 3 10:31:10.234 890
SF3 Jun 8 10:31:10.234 314 314 235
SF3 Jun 1 10:31:10.234 256 256 876
SF3 Jun 1 10:31:10.234 235 234
SF3 Jun 8 10:31:10.234 876 876 432
SF3 Jun 9 10:31:10.234 234 234
SF3 Jun 3 10:31:10.234 432 276
… … … … …
Phoenix Push Down: Aggregation
Aggregate on server-side
SERVER METRICS
HOST AGGREGATE VALUES
SF1 3421
SF2 2145
SF3 9823
Phoenix Push Down: TopN
Completed
SELECT host, date, gc_time
FROM server_metrics
WHERE date > CURRENT_DATE() – 7
AND host LIKE ‘SF%’
ORDER BY gc_time DESC
LIMIT 5
Phoenix Push Down: TopN
Client-side parallel scans
Completed
R1
R2
R3
R4
scan1
scan3
scan2
Phoenix Push Down: TopN
Each region holds N rows
Completed
R1
R2
R3
R4
scan1
Phoenix Push Down: TopN
Each region holds N rows
Completed
R1
R2
R3
R4
scan2
Phoenix Push Down: TopN
Each region holds N rows
Completed
R1
R2
R3
R4
scan3
SERVER METRICS
HOST DATE GC_TIME
SF3 Jun 2 10:10:10.234 22123
SF5 Jun 3 23:05:44.975 19876
SF2 Jun 9 08:10:32.147 11345
SF2 Jun 1 11:18:28.456 10234
SF1 Jun 3 22:03:22.142 10111
Phoenix Push Down: TopN
Client-side final merge sort
Scan1
Scan2
Scan3
Phoenix Push Down: TopN
Secondary Index
Completed
CREATE INDEX gc_time_index
ON server_metrics (gc_time DESC, date DESC)
INCLUDE (response_time)
Row Key
GC_TIME_INDEX
GC_TIME INTEGER
DATE DATE
HOST VARCHAR
RESPONSE_TIME INTEGER
Phoenix Push Down: TopN
Secondary Index
Completed
CREATE INDEX gc_time_index
ON server_metrics (gc_time DESC, date DESC)
INCLUDE (response_time)
Key Value
GC_TIME_INDEX
GC_TIME INTEGER
DATE DATE
HOST VARCHAR
RESPONSE_TIME INTEGER
Phoenix Push Down: TopN
Secondary Index
Completed
o Original query doesn’t change
o Phoenix rewrites query to use index table
o All referenced columns must exist in index
table for it to be considered
o Local Indexing coming soon!
o Stats coming soon!
Phoenix Push Down: Hash Join
Completed
SELECT m.*, i.location
FROM server_metrics m
JOIN host_info i ON m.host = i.host
WHERE m.date > CURRENT_DATE() – 7
AND i.location = ‘SF’
ORDER BY m.gc_time DESC
LIMIT 5
Phoenix Push Down: Hash Join
Separate LHS and RHS
Completed
SELECT m.*, i.location
FROM server_metrics m
JOIN host_info i ON m.host = i.host
WHERE m.date > CURRENT_DATE() – 7
AND i.location = ‘SF’
ORDER BY m.gc_time DESC
LIMIT 5
Phoenix Push Down: Hash Join
Separate LHS and RHS
Completed
SELECT m.*, i.location
FROM server_metrics m
JOIN host_info i ON m.host = i.host
WHERE m.date > CURRENT_DATE() – 7
AND i.location = ‘SF’
ORDER BY m.gc_time DESC
LIMIT 5
Phoenix Push Down: Hash Join
Separate LHS and RHS
Completed
LHS
SELECT *
FROM server_metrics
WHERE date >
CURRENT_DATE() – 7
ORDER BY gc_time DESC
LIMIT 5
RHS
SELECT host, location
FROM host_info
WHERE location = ‘SF’
Phoenix Push Down: Hash Join
Execute & broadcast RHS to each RS
Completed
RS1
RS2
RHS
Phoenix Push Down: Hash Join
Server-side map lookup during scan
Completed
R1
R2
R3
R4
RHS
RHS
LHS
scan1
scan2
scan3
scan4
Phoenix Push Down: Hash Join
Derived Tables and Filter Rewrite
Completed
SELECT i.host, i.location, m.res, m.gc
FROM host_info i
JOIN (SELECT host, avg(response_time) res,
avg(gc_time) gc FROM server_metrics GROUP
BY host) AS m
ON i.host = m.host
WHERE i.location != ‘NJ’
AND (m.res >= 10000 OR m.gc >= 2000)
Phoenix Push Down: Hash Join
Filters Pushed Down and Rewritten for RHS
Completed
RHS
SELECT host, avg(response_time),
avg(gc_time)
FROM server_metrics
GROUP BY host
HAVING
avg(response_time)>=10000
OR avg(gc_time)>=2000
LHS
SELECT host,
location
FROM host_info
WHERE
location != ‘NJ’
Phoenix Push Down: Hash Join
Multiple Joins and Sub-joins
Completed
SELECT *
FROM server_metrics m
JOIN (host_info h JOIN location_info l
ON h.location = l.location)
ON m.host = h.host
WHERE m.date > CURRENT_DATE() – 7
AND l.user_count >= 200000
Phoenix Push Down: Hash Join
Recursively Separate LHS and RHS
Completed
Outer RHSOuter LHS
SELECT *
FROM server_metrics
WHERE date >
CURRENT_DATE() – 7
Inner LHS
SELECT *
FROM host_info
Inner RHS
SELECT *
FROM location_info
WHERE
user_count >= 200000
Phoenix Push Down: Hash Join
Iterative Execution of Multiple Joins
CompletedRS1
RS2
Inner
RHS
Broadcast
RS1
RS1
RS2
RS3
RS4
Scan
Inner LHS
Outer
RHS
Broadcast
Scan
Outer LHS
Scan
Inner RHS
Phoenix Push Down: Hash Join
Star-join Optimization
Completed
SELECT *
FROM server_metrics m
JOIN host_info h
ON m.host = h.host
JOIN location_info l
ON h.location = l.location
WHERE m.date > CURRENT_DATE() – 7
AND l.user_count >= 200000
Phoenix Push Down: Hash Join
Separate LHS and Multiple Parallel RHS
Completed
LHS
SELECT *
FROM server_metrics
WHERE date >
CURRENT_DATE() – 7
RHS 1
SELECT *
FROM host_info
RHS 2
SELECT *
FROM location_info
WHERE
user_count >= 200000
Phoenix Push Down: Hash Join
Star-join Execution of Multiple Joins
Completed
RS1
RS2
RHS 1
RS1 RS1
RS2
RS3
RS4
Scan RHS 2
Broadcast
Scan LHSScan RHS 1
RHS 2
New in Phoenix 3:
Shared Tables
• HBase wants small # of big tables instead
of large # of small tables
• Two types of shared tables in Phoenix 3:
• Views
• Tenant-specific Tables
Views
• Multiple Phoenix tables share same
physical HBase table
• Inherit parent table’s PK, KV columns
• Updateable Views
• Secondary Indexes on Views
Completed
CREATE TABLE event (
type CHAR(1),
event_id BIGINT,
created_date DATE,
created_by VARCHAR,
CONSTRAINT pk PRIMARY KEY (type, event_id));
CREATE VIEW web_event (
referrer VARCHAR) AS
SELECT * FROM event
WHERE type=‘w’;
• Includes columns from TABLE
• Cannot define PK
• Updateable if only equality
expressions separated by AND
Views
Completed
type = ‘c’
type = ‘m’
type = ‘p’
type = ‘w’
EVENT
CHAT_EVENT
MOBILE_EVENT
PHONE_EVENT
WEB_EVENT
Views
Tenant-Specific Tables
• Tenant data isolation and co-location
• Built using Views
• Uses tenant-specific Connections
Completed
CREATE TABLE event (
tenant_id VARCHAR,
type CHAR(1),
event_id BIGINT,
created_date DATE,
created_by VARCHAR,
CONSTRAINT pk PRIMARY KEY (tenant_id, type, event_id))
MULTI_TENANT=true;
First PK column identifies tenant ID
Tenant-Specific Tables
Step 1: Create multi-tenant base table
Completed
CREATE VIEW web_event (
referrer VARCHAR) AS
SELECT * FROM event
WHERE type=‘w’;
DriverManager.connect(“jdbc:phoenix:localhost;TenantId=me”);
CREATE VIEW my_web_event AS
SELECT * FROM web_event;
Tenant-specific view
Tenant-specific
connection
Tenant-Specific Tables
Step 2: Create tenant-specific tables
tenant_id = ‘me’
…
EVENT
tenant_id = ‘you’
tenant_id = ‘them’
Tenant-Specific Tables
type = ‘c’
type = ‘m’
type = ‘p’
type = ‘w’
EVENT
CHAT_EVENT
MOBILE_EVENT
PHONE_EVENT
WEB_EVENT
PER
tenant_id
Tenant-Specific Tables
• Tenant-specific connection may only see
and operate on their data
• Inherit parent table’s PK, KV columns
• Tenant-specific secondary indexes
• Restrictions:
• No ALTER base table
• No DROP columns used in PK and where clause
• PK columns same as parent
Tenant-Specific Tables
• Allow shared tables to extend parent’s PK
• Support more complex WHERE clauses for
updatable views
• Support projecting subset of columns to
View
Shared Tables Future Work
Phoenix Roadmap
Completed
o Local Indexes
o Transactions
o More Join strategies
o Cost-based query optimizer
o OLAP extensions
Thank you!
Questions/comments?

More Related Content

What's hot

An Introduction to time series with Team Apache
An Introduction to time series with Team ApacheAn Introduction to time series with Team Apache
An Introduction to time series with Team Apache
Patrick McFadin
 
Extending Apache Spark – Beyond Spark Session Extensions
Extending Apache Spark – Beyond Spark Session ExtensionsExtending Apache Spark – Beyond Spark Session Extensions
Extending Apache Spark – Beyond Spark Session Extensions
Databricks
 
Apache Calcite overview
Apache Calcite overviewApache Calcite overview
Apache Calcite overview
Julian Hyde
 
SQL Performance Improvements At a Glance in Apache Spark 3.0
SQL Performance Improvements At a Glance in Apache Spark 3.0SQL Performance Improvements At a Glance in Apache Spark 3.0
SQL Performance Improvements At a Glance in Apache Spark 3.0
Kazuaki Ishizaki
 
What's New in Apache Spark 3.0 !!
What's New in Apache Spark 3.0 !!What's New in Apache Spark 3.0 !!
What's New in Apache Spark 3.0 !!
Aparup Chatterjee
 
Spark+flume seattle
Spark+flume seattleSpark+flume seattle
Spark+flume seattle
Hari Shreedharan
 
Hadoop, Hive, Spark and Object Stores
Hadoop, Hive, Spark and Object StoresHadoop, Hive, Spark and Object Stores
Hadoop, Hive, Spark and Object Stores
Steve Loughran
 
Data Science Lab Meetup: Cassandra and Spark
Data Science Lab Meetup: Cassandra and SparkData Science Lab Meetup: Cassandra and Spark
Data Science Lab Meetup: Cassandra and Spark
Christopher Batey
 
Monitor Apache Spark 3 on Kubernetes using Metrics and Plugins
Monitor Apache Spark 3 on Kubernetes using Metrics and PluginsMonitor Apache Spark 3 on Kubernetes using Metrics and Plugins
Monitor Apache Spark 3 on Kubernetes using Metrics and Plugins
Databricks
 
Have your cake and eat it too
Have your cake and eat it tooHave your cake and eat it too
Have your cake and eat it too
Gwen (Chen) Shapira
 
Analyzing Log Data With Apache Spark
Analyzing Log Data With Apache SparkAnalyzing Log Data With Apache Spark
Analyzing Log Data With Apache Spark
Spark Summit
 
Xldb2011 tue 0940_facebook_realtimeanalytics
Xldb2011 tue 0940_facebook_realtimeanalyticsXldb2011 tue 0940_facebook_realtimeanalytics
Xldb2011 tue 0940_facebook_realtimeanalytics
liqiang xu
 
Calcite meetup-2016-04-20
Calcite meetup-2016-04-20Calcite meetup-2016-04-20
Calcite meetup-2016-04-20
Josh Elser
 
Performant Streaming in Production: Preventing Common Pitfalls when Productio...
Performant Streaming in Production: Preventing Common Pitfalls when Productio...Performant Streaming in Production: Preventing Common Pitfalls when Productio...
Performant Streaming in Production: Preventing Common Pitfalls when Productio...
Databricks
 
How to Automate Performance Tuning for Apache Spark
How to Automate Performance Tuning for Apache SparkHow to Automate Performance Tuning for Apache Spark
How to Automate Performance Tuning for Apache Spark
Databricks
 
Cost-Based Optimizer in Apache Spark 2.2 Ron Hu, Sameer Agarwal, Wenchen Fan ...
Cost-Based Optimizer in Apache Spark 2.2 Ron Hu, Sameer Agarwal, Wenchen Fan ...Cost-Based Optimizer in Apache Spark 2.2 Ron Hu, Sameer Agarwal, Wenchen Fan ...
Cost-Based Optimizer in Apache Spark 2.2 Ron Hu, Sameer Agarwal, Wenchen Fan ...
Databricks
 
Maximilian Michels – Google Cloud Dataflow on Top of Apache Flink
Maximilian Michels – Google Cloud Dataflow on Top of Apache FlinkMaximilian Michels – Google Cloud Dataflow on Top of Apache Flink
Maximilian Michels – Google Cloud Dataflow on Top of Apache Flink
Flink Forward
 
Presto on Apache Spark: A Tale of Two Computation Engines
Presto on Apache Spark: A Tale of Two Computation EnginesPresto on Apache Spark: A Tale of Two Computation Engines
Presto on Apache Spark: A Tale of Two Computation Engines
Databricks
 
Large scale near real-time log indexing with Flume and SolrCloud
Large scale near real-time log indexing with Flume and SolrCloudLarge scale near real-time log indexing with Flume and SolrCloud
Large scale near real-time log indexing with Flume and SolrCloud
DataWorks Summit
 
Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Algebird : Abstract Algebra for big data analytics. Devoxx 2014Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Samir Bessalah
 

What's hot (20)

An Introduction to time series with Team Apache
An Introduction to time series with Team ApacheAn Introduction to time series with Team Apache
An Introduction to time series with Team Apache
 
Extending Apache Spark – Beyond Spark Session Extensions
Extending Apache Spark – Beyond Spark Session ExtensionsExtending Apache Spark – Beyond Spark Session Extensions
Extending Apache Spark – Beyond Spark Session Extensions
 
Apache Calcite overview
Apache Calcite overviewApache Calcite overview
Apache Calcite overview
 
SQL Performance Improvements At a Glance in Apache Spark 3.0
SQL Performance Improvements At a Glance in Apache Spark 3.0SQL Performance Improvements At a Glance in Apache Spark 3.0
SQL Performance Improvements At a Glance in Apache Spark 3.0
 
What's New in Apache Spark 3.0 !!
What's New in Apache Spark 3.0 !!What's New in Apache Spark 3.0 !!
What's New in Apache Spark 3.0 !!
 
Spark+flume seattle
Spark+flume seattleSpark+flume seattle
Spark+flume seattle
 
Hadoop, Hive, Spark and Object Stores
Hadoop, Hive, Spark and Object StoresHadoop, Hive, Spark and Object Stores
Hadoop, Hive, Spark and Object Stores
 
Data Science Lab Meetup: Cassandra and Spark
Data Science Lab Meetup: Cassandra and SparkData Science Lab Meetup: Cassandra and Spark
Data Science Lab Meetup: Cassandra and Spark
 
Monitor Apache Spark 3 on Kubernetes using Metrics and Plugins
Monitor Apache Spark 3 on Kubernetes using Metrics and PluginsMonitor Apache Spark 3 on Kubernetes using Metrics and Plugins
Monitor Apache Spark 3 on Kubernetes using Metrics and Plugins
 
Have your cake and eat it too
Have your cake and eat it tooHave your cake and eat it too
Have your cake and eat it too
 
Analyzing Log Data With Apache Spark
Analyzing Log Data With Apache SparkAnalyzing Log Data With Apache Spark
Analyzing Log Data With Apache Spark
 
Xldb2011 tue 0940_facebook_realtimeanalytics
Xldb2011 tue 0940_facebook_realtimeanalyticsXldb2011 tue 0940_facebook_realtimeanalytics
Xldb2011 tue 0940_facebook_realtimeanalytics
 
Calcite meetup-2016-04-20
Calcite meetup-2016-04-20Calcite meetup-2016-04-20
Calcite meetup-2016-04-20
 
Performant Streaming in Production: Preventing Common Pitfalls when Productio...
Performant Streaming in Production: Preventing Common Pitfalls when Productio...Performant Streaming in Production: Preventing Common Pitfalls when Productio...
Performant Streaming in Production: Preventing Common Pitfalls when Productio...
 
How to Automate Performance Tuning for Apache Spark
How to Automate Performance Tuning for Apache SparkHow to Automate Performance Tuning for Apache Spark
How to Automate Performance Tuning for Apache Spark
 
Cost-Based Optimizer in Apache Spark 2.2 Ron Hu, Sameer Agarwal, Wenchen Fan ...
Cost-Based Optimizer in Apache Spark 2.2 Ron Hu, Sameer Agarwal, Wenchen Fan ...Cost-Based Optimizer in Apache Spark 2.2 Ron Hu, Sameer Agarwal, Wenchen Fan ...
Cost-Based Optimizer in Apache Spark 2.2 Ron Hu, Sameer Agarwal, Wenchen Fan ...
 
Maximilian Michels – Google Cloud Dataflow on Top of Apache Flink
Maximilian Michels – Google Cloud Dataflow on Top of Apache FlinkMaximilian Michels – Google Cloud Dataflow on Top of Apache Flink
Maximilian Michels – Google Cloud Dataflow on Top of Apache Flink
 
Presto on Apache Spark: A Tale of Two Computation Engines
Presto on Apache Spark: A Tale of Two Computation EnginesPresto on Apache Spark: A Tale of Two Computation Engines
Presto on Apache Spark: A Tale of Two Computation Engines
 
Large scale near real-time log indexing with Flume and SolrCloud
Large scale near real-time log indexing with Flume and SolrCloudLarge scale near real-time log indexing with Flume and SolrCloud
Large scale near real-time log indexing with Flume and SolrCloud
 
Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Algebird : Abstract Algebra for big data analytics. Devoxx 2014Algebird : Abstract Algebra for big data analytics. Devoxx 2014
Algebird : Abstract Algebra for big data analytics. Devoxx 2014
 

Viewers also liked

Apache Phoenix: Transforming HBase into a SQL Database
Apache Phoenix: Transforming HBase into a SQL DatabaseApache Phoenix: Transforming HBase into a SQL Database
Apache Phoenix: Transforming HBase into a SQL Database
DataWorks Summit
 
Hortonworks Technical Workshop: HBase and Apache Phoenix
Hortonworks Technical Workshop: HBase and Apache Phoenix Hortonworks Technical Workshop: HBase and Apache Phoenix
Hortonworks Technical Workshop: HBase and Apache Phoenix
Hortonworks
 
Apache Phoenix and Apache HBase: An Enterprise Grade Data Warehouse
Apache Phoenix and Apache HBase: An Enterprise Grade Data WarehouseApache Phoenix and Apache HBase: An Enterprise Grade Data Warehouse
Apache Phoenix and Apache HBase: An Enterprise Grade Data Warehouse
Josh Elser
 
Apache HBase Performance Tuning
Apache HBase Performance TuningApache HBase Performance Tuning
Apache HBase Performance Tuning
Lars Hofhansl
 
April 2014 HUG : Apache Phoenix
April 2014 HUG : Apache PhoenixApril 2014 HUG : Apache Phoenix
April 2014 HUG : Apache Phoenix
Yahoo Developer Network
 
HBaseCon 2013: Full-Text Indexing for Apache HBase
HBaseCon 2013: Full-Text Indexing for Apache HBaseHBaseCon 2013: Full-Text Indexing for Apache HBase
HBaseCon 2013: Full-Text Indexing for Apache HBase
Cloudera, Inc.
 
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
Cloudera, Inc.
 
HBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQL
HBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQLHBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQL
HBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQL
Cloudera, Inc.
 
HBaseCon 2013: Near Real Time Indexing for eBay Search
HBaseCon 2013: Near Real Time Indexing for eBay SearchHBaseCon 2013: Near Real Time Indexing for eBay Search
HBaseCon 2013: Near Real Time Indexing for eBay Search
Cloudera, Inc.
 
A Survey of HBase Application Archetypes
A Survey of HBase Application ArchetypesA Survey of HBase Application Archetypes
A Survey of HBase Application Archetypes
HBaseCon
 
OpenTSDB 2.0
OpenTSDB 2.0OpenTSDB 2.0
OpenTSDB 2.0
HBaseCon
 
HBase for Architects
HBase for ArchitectsHBase for Architects
HBase for Architects
Nick Dimiduk
 
Intro to HBase
Intro to HBaseIntro to HBase
Intro to HBase
alexbaranau
 
Five major tips to maximize performance on a 200+ SQL HBase/Phoenix cluster
Five major tips to maximize performance on a 200+ SQL HBase/Phoenix clusterFive major tips to maximize performance on a 200+ SQL HBase/Phoenix cluster
Five major tips to maximize performance on a 200+ SQL HBase/Phoenix cluster
mas4share
 
Techdays2015 big data-realtime
Techdays2015 big data-realtimeTechdays2015 big data-realtime
Techdays2015 big data-realtime
Romain Casteres
 
Stats web avec Hive chez Scoop.it
Stats web avec Hive chez Scoop.itStats web avec Hive chez Scoop.it
Stats web avec Hive chez Scoop.it
hibnico
 
De-Mystifying the Apache Phoenix QueryServer
De-Mystifying the Apache Phoenix QueryServerDe-Mystifying the Apache Phoenix QueryServer
De-Mystifying the Apache Phoenix QueryServer
Josh Elser
 
HBase杂谈
HBase杂谈HBase杂谈
HBase杂谈
Joseph Pan
 
Transactions Over Apache HBase
Transactions Over Apache HBaseTransactions Over Apache HBase
Transactions Over Apache HBase
alexbaranau
 
Quand utiliser MongoDB … Et quand vous en passer…
Quand utiliser MongoDB	… Et quand vous en passer…Quand utiliser MongoDB	… Et quand vous en passer…
Quand utiliser MongoDB … Et quand vous en passer…
MongoDB
 

Viewers also liked (20)

Apache Phoenix: Transforming HBase into a SQL Database
Apache Phoenix: Transforming HBase into a SQL DatabaseApache Phoenix: Transforming HBase into a SQL Database
Apache Phoenix: Transforming HBase into a SQL Database
 
Hortonworks Technical Workshop: HBase and Apache Phoenix
Hortonworks Technical Workshop: HBase and Apache Phoenix Hortonworks Technical Workshop: HBase and Apache Phoenix
Hortonworks Technical Workshop: HBase and Apache Phoenix
 
Apache Phoenix and Apache HBase: An Enterprise Grade Data Warehouse
Apache Phoenix and Apache HBase: An Enterprise Grade Data WarehouseApache Phoenix and Apache HBase: An Enterprise Grade Data Warehouse
Apache Phoenix and Apache HBase: An Enterprise Grade Data Warehouse
 
Apache HBase Performance Tuning
Apache HBase Performance TuningApache HBase Performance Tuning
Apache HBase Performance Tuning
 
April 2014 HUG : Apache Phoenix
April 2014 HUG : Apache PhoenixApril 2014 HUG : Apache Phoenix
April 2014 HUG : Apache Phoenix
 
HBaseCon 2013: Full-Text Indexing for Apache HBase
HBaseCon 2013: Full-Text Indexing for Apache HBaseHBaseCon 2013: Full-Text Indexing for Apache HBase
HBaseCon 2013: Full-Text Indexing for Apache HBase
 
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
HBaseCon 2013: Streaming Data into Apache HBase using Apache Flume: Experienc...
 
HBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQL
HBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQLHBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQL
HBaseCon 2013: How (and Why) Phoenix Puts the SQL Back into NoSQL
 
HBaseCon 2013: Near Real Time Indexing for eBay Search
HBaseCon 2013: Near Real Time Indexing for eBay SearchHBaseCon 2013: Near Real Time Indexing for eBay Search
HBaseCon 2013: Near Real Time Indexing for eBay Search
 
A Survey of HBase Application Archetypes
A Survey of HBase Application ArchetypesA Survey of HBase Application Archetypes
A Survey of HBase Application Archetypes
 
OpenTSDB 2.0
OpenTSDB 2.0OpenTSDB 2.0
OpenTSDB 2.0
 
HBase for Architects
HBase for ArchitectsHBase for Architects
HBase for Architects
 
Intro to HBase
Intro to HBaseIntro to HBase
Intro to HBase
 
Five major tips to maximize performance on a 200+ SQL HBase/Phoenix cluster
Five major tips to maximize performance on a 200+ SQL HBase/Phoenix clusterFive major tips to maximize performance on a 200+ SQL HBase/Phoenix cluster
Five major tips to maximize performance on a 200+ SQL HBase/Phoenix cluster
 
Techdays2015 big data-realtime
Techdays2015 big data-realtimeTechdays2015 big data-realtime
Techdays2015 big data-realtime
 
Stats web avec Hive chez Scoop.it
Stats web avec Hive chez Scoop.itStats web avec Hive chez Scoop.it
Stats web avec Hive chez Scoop.it
 
De-Mystifying the Apache Phoenix QueryServer
De-Mystifying the Apache Phoenix QueryServerDe-Mystifying the Apache Phoenix QueryServer
De-Mystifying the Apache Phoenix QueryServer
 
HBase杂谈
HBase杂谈HBase杂谈
HBase杂谈
 
Transactions Over Apache HBase
Transactions Over Apache HBaseTransactions Over Apache HBase
Transactions Over Apache HBase
 
Quand utiliser MongoDB … Et quand vous en passer…
Quand utiliser MongoDB	… Et quand vous en passer…Quand utiliser MongoDB	… Et quand vous en passer…
Quand utiliser MongoDB … Et quand vous en passer…
 

Similar to Taming HBase with Apache Phoenix and SQL

Apache HAWQ Architecture
Apache HAWQ ArchitectureApache HAWQ Architecture
Apache HAWQ Architecture
Alexey Grishchenko
 
Flex Tables Guide Software V. 7.0.x
Flex Tables Guide Software V. 7.0.xFlex Tables Guide Software V. 7.0.x
Flex Tables Guide Software V. 7.0.x
Andrey Karpov
 
Deploying your Data Warehouse on AWS
Deploying your Data Warehouse on AWSDeploying your Data Warehouse on AWS
Deploying your Data Warehouse on AWS
Amazon Web Services
 
How Rackspace Cloud Monitoring uses Cassandra
How Rackspace Cloud Monitoring uses CassandraHow Rackspace Cloud Monitoring uses Cassandra
How Rackspace Cloud Monitoring uses Cassandra
gdusbabek
 
Apache Big Data EU 2015 - Phoenix
Apache Big Data EU 2015 - PhoenixApache Big Data EU 2015 - Phoenix
Apache Big Data EU 2015 - Phoenix
Nick Dimiduk
 
2008 Ur Tech Talk Zshao
2008 Ur Tech Talk Zshao2008 Ur Tech Talk Zshao
2008 Ur Tech Talk Zshao
Jeff Hammerbacher
 
Hadoop and Hive
Hadoop and HiveHadoop and Hive
Hadoop and Hive
Zheng Shao
 
What is in All of Those SSTable Files Not Just the Data One but All the Rest ...
What is in All of Those SSTable Files Not Just the Data One but All the Rest ...What is in All of Those SSTable Files Not Just the Data One but All the Rest ...
What is in All of Those SSTable Files Not Just the Data One but All the Rest ...
DataStax
 
Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!
Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!
Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!
confluent
 
Stream Analytics with SQL on Apache Flink
 Stream Analytics with SQL on Apache Flink Stream Analytics with SQL on Apache Flink
Stream Analytics with SQL on Apache Flink
Fabian Hueske
 
Flink Forward SF 2017: Timo Walther - Table & SQL API – unified APIs for bat...
Flink Forward SF 2017: Timo Walther -  Table & SQL API – unified APIs for bat...Flink Forward SF 2017: Timo Walther -  Table & SQL API – unified APIs for bat...
Flink Forward SF 2017: Timo Walther - Table & SQL API – unified APIs for bat...
Flink Forward
 
Amazon DynamoDB Workshop
Amazon DynamoDB WorkshopAmazon DynamoDB Workshop
Amazon DynamoDB Workshop
Amazon Web Services
 
How Pony ORM translates Python generators to SQL queries
How Pony ORM translates Python generators to SQL queriesHow Pony ORM translates Python generators to SQL queries
How Pony ORM translates Python generators to SQL queries
ponyorm
 
Patterns of Streaming Applications
Patterns of Streaming ApplicationsPatterns of Streaming Applications
Patterns of Streaming Applications
C4Media
 
GraphConnect 2014 SF: From Zero to Graph in 120: Scale
GraphConnect 2014 SF: From Zero to Graph in 120: ScaleGraphConnect 2014 SF: From Zero to Graph in 120: Scale
GraphConnect 2014 SF: From Zero to Graph in 120: Scale
Neo4j
 
Hyperspace for Delta Lake
Hyperspace for Delta LakeHyperspace for Delta Lake
Hyperspace for Delta Lake
Databricks
 
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWSAWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
Cobus Bernard
 
phoenix-on-calcite-nyc-meetup
phoenix-on-calcite-nyc-meetupphoenix-on-calcite-nyc-meetup
phoenix-on-calcite-nyc-meetup
Maryann Xue
 
WCM Transfer Services
WCM Transfer Services WCM Transfer Services
WCM Transfer Services
Alfresco Software
 
Fabian Hueske - Stream Analytics with SQL on Apache Flink
Fabian Hueske - Stream Analytics with SQL on Apache FlinkFabian Hueske - Stream Analytics with SQL on Apache Flink
Fabian Hueske - Stream Analytics with SQL on Apache Flink
Ververica
 

Similar to Taming HBase with Apache Phoenix and SQL (20)

Apache HAWQ Architecture
Apache HAWQ ArchitectureApache HAWQ Architecture
Apache HAWQ Architecture
 
Flex Tables Guide Software V. 7.0.x
Flex Tables Guide Software V. 7.0.xFlex Tables Guide Software V. 7.0.x
Flex Tables Guide Software V. 7.0.x
 
Deploying your Data Warehouse on AWS
Deploying your Data Warehouse on AWSDeploying your Data Warehouse on AWS
Deploying your Data Warehouse on AWS
 
How Rackspace Cloud Monitoring uses Cassandra
How Rackspace Cloud Monitoring uses CassandraHow Rackspace Cloud Monitoring uses Cassandra
How Rackspace Cloud Monitoring uses Cassandra
 
Apache Big Data EU 2015 - Phoenix
Apache Big Data EU 2015 - PhoenixApache Big Data EU 2015 - Phoenix
Apache Big Data EU 2015 - Phoenix
 
2008 Ur Tech Talk Zshao
2008 Ur Tech Talk Zshao2008 Ur Tech Talk Zshao
2008 Ur Tech Talk Zshao
 
Hadoop and Hive
Hadoop and HiveHadoop and Hive
Hadoop and Hive
 
What is in All of Those SSTable Files Not Just the Data One but All the Rest ...
What is in All of Those SSTable Files Not Just the Data One but All the Rest ...What is in All of Those SSTable Files Not Just the Data One but All the Rest ...
What is in All of Those SSTable Files Not Just the Data One but All the Rest ...
 
Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!
Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!
Apache Kafka and KSQL in Action: Let's Build a Streaming Data Pipeline!
 
Stream Analytics with SQL on Apache Flink
 Stream Analytics with SQL on Apache Flink Stream Analytics with SQL on Apache Flink
Stream Analytics with SQL on Apache Flink
 
Flink Forward SF 2017: Timo Walther - Table & SQL API – unified APIs for bat...
Flink Forward SF 2017: Timo Walther -  Table & SQL API – unified APIs for bat...Flink Forward SF 2017: Timo Walther -  Table & SQL API – unified APIs for bat...
Flink Forward SF 2017: Timo Walther - Table & SQL API – unified APIs for bat...
 
Amazon DynamoDB Workshop
Amazon DynamoDB WorkshopAmazon DynamoDB Workshop
Amazon DynamoDB Workshop
 
How Pony ORM translates Python generators to SQL queries
How Pony ORM translates Python generators to SQL queriesHow Pony ORM translates Python generators to SQL queries
How Pony ORM translates Python generators to SQL queries
 
Patterns of Streaming Applications
Patterns of Streaming ApplicationsPatterns of Streaming Applications
Patterns of Streaming Applications
 
GraphConnect 2014 SF: From Zero to Graph in 120: Scale
GraphConnect 2014 SF: From Zero to Graph in 120: ScaleGraphConnect 2014 SF: From Zero to Graph in 120: Scale
GraphConnect 2014 SF: From Zero to Graph in 120: Scale
 
Hyperspace for Delta Lake
Hyperspace for Delta LakeHyperspace for Delta Lake
Hyperspace for Delta Lake
 
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWSAWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
AWS SSA Webinar 20 - Getting Started with Data Warehouses on AWS
 
phoenix-on-calcite-nyc-meetup
phoenix-on-calcite-nyc-meetupphoenix-on-calcite-nyc-meetup
phoenix-on-calcite-nyc-meetup
 
WCM Transfer Services
WCM Transfer Services WCM Transfer Services
WCM Transfer Services
 
Fabian Hueske - Stream Analytics with SQL on Apache Flink
Fabian Hueske - Stream Analytics with SQL on Apache FlinkFabian Hueske - Stream Analytics with SQL on Apache Flink
Fabian Hueske - Stream Analytics with SQL on Apache Flink
 

More from HBaseCon

hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kuberneteshbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
HBaseCon
 
hbaseconasia2017: HBase on Beam
hbaseconasia2017: HBase on Beamhbaseconasia2017: HBase on Beam
hbaseconasia2017: HBase on Beam
HBaseCon
 
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huaweihbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
HBaseCon
 
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinteresthbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon
 
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
HBaseCon
 
hbaseconasia2017: Apache HBase at Netease
hbaseconasia2017: Apache HBase at Neteasehbaseconasia2017: Apache HBase at Netease
hbaseconasia2017: Apache HBase at Netease
HBaseCon
 
hbaseconasia2017: HBase在Hulu的使用和实践
hbaseconasia2017: HBase在Hulu的使用和实践hbaseconasia2017: HBase在Hulu的使用和实践
hbaseconasia2017: HBase在Hulu的使用和实践
HBaseCon
 
hbaseconasia2017: 基于HBase的企业级大数据平台
hbaseconasia2017: 基于HBase的企业级大数据平台hbaseconasia2017: 基于HBase的企业级大数据平台
hbaseconasia2017: 基于HBase的企业级大数据平台
HBaseCon
 
hbaseconasia2017: HBase at JD.com
hbaseconasia2017: HBase at JD.comhbaseconasia2017: HBase at JD.com
hbaseconasia2017: HBase at JD.com
HBaseCon
 
hbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecturehbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecture
HBaseCon
 
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huaweihbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
HBaseCon
 
hbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMihbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMi
HBaseCon
 
hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0
HBaseCon
 
HBaseCon2017 Democratizing HBase
HBaseCon2017 Democratizing HBaseHBaseCon2017 Democratizing HBase
HBaseCon2017 Democratizing HBase
HBaseCon
 
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in PinterestHBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon
 
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBaseHBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
HBaseCon
 
HBaseCon2017 Transactions in HBase
HBaseCon2017 Transactions in HBaseHBaseCon2017 Transactions in HBase
HBaseCon2017 Transactions in HBase
HBaseCon
 
HBaseCon2017 Highly-Available HBase
HBaseCon2017 Highly-Available HBaseHBaseCon2017 Highly-Available HBase
HBaseCon2017 Highly-Available HBase
HBaseCon
 
HBaseCon2017 Apache HBase at Didi
HBaseCon2017 Apache HBase at DidiHBaseCon2017 Apache HBase at Didi
HBaseCon2017 Apache HBase at Didi
HBaseCon
 
HBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase ClientHBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon
 

More from HBaseCon (20)

hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kuberneteshbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
hbaseconasia2017: Building online HBase cluster of Zhihu based on Kubernetes
 
hbaseconasia2017: HBase on Beam
hbaseconasia2017: HBase on Beamhbaseconasia2017: HBase on Beam
hbaseconasia2017: HBase on Beam
 
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huaweihbaseconasia2017: HBase Disaster Recovery Solution at Huawei
hbaseconasia2017: HBase Disaster Recovery Solution at Huawei
 
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinteresthbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
hbaseconasia2017: Removable singularity: a story of HBase upgrade in Pinterest
 
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
hbaseconasia2017: HareQL:快速HBase查詢工具的發展過程
 
hbaseconasia2017: Apache HBase at Netease
hbaseconasia2017: Apache HBase at Neteasehbaseconasia2017: Apache HBase at Netease
hbaseconasia2017: Apache HBase at Netease
 
hbaseconasia2017: HBase在Hulu的使用和实践
hbaseconasia2017: HBase在Hulu的使用和实践hbaseconasia2017: HBase在Hulu的使用和实践
hbaseconasia2017: HBase在Hulu的使用和实践
 
hbaseconasia2017: 基于HBase的企业级大数据平台
hbaseconasia2017: 基于HBase的企业级大数据平台hbaseconasia2017: 基于HBase的企业级大数据平台
hbaseconasia2017: 基于HBase的企业级大数据平台
 
hbaseconasia2017: HBase at JD.com
hbaseconasia2017: HBase at JD.comhbaseconasia2017: HBase at JD.com
hbaseconasia2017: HBase at JD.com
 
hbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecturehbaseconasia2017: Large scale data near-line loading method and architecture
hbaseconasia2017: Large scale data near-line loading method and architecture
 
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huaweihbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
hbaseconasia2017: Ecosystems with HBase and CloudTable service at Huawei
 
hbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMihbaseconasia2017: HBase Practice At XiaoMi
hbaseconasia2017: HBase Practice At XiaoMi
 
hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0hbaseconasia2017: hbase-2.0.0
hbaseconasia2017: hbase-2.0.0
 
HBaseCon2017 Democratizing HBase
HBaseCon2017 Democratizing HBaseHBaseCon2017 Democratizing HBase
HBaseCon2017 Democratizing HBase
 
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in PinterestHBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
HBaseCon2017 Removable singularity: a story of HBase upgrade in Pinterest
 
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBaseHBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
HBaseCon2017 Quanta: Quora's hierarchical counting system on HBase
 
HBaseCon2017 Transactions in HBase
HBaseCon2017 Transactions in HBaseHBaseCon2017 Transactions in HBase
HBaseCon2017 Transactions in HBase
 
HBaseCon2017 Highly-Available HBase
HBaseCon2017 Highly-Available HBaseHBaseCon2017 Highly-Available HBase
HBaseCon2017 Highly-Available HBase
 
HBaseCon2017 Apache HBase at Didi
HBaseCon2017 Apache HBase at DidiHBaseCon2017 Apache HBase at Didi
HBaseCon2017 Apache HBase at Didi
 
HBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase ClientHBaseCon2017 gohbase: Pure Go HBase Client
HBaseCon2017 gohbase: Pure Go HBase Client
 

Recently uploaded

Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Octavian Nadolu
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
Rakesh Kumar R
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemUI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
Peter Muessig
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
Alberto Brandolini
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
Top 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptxTop 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptx
devvsandy
 
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative AnalysisOdoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Envertis Software Solutions
 
Mobile app Development Services | Drona Infotech
Mobile app Development Services  | Drona InfotechMobile app Development Services  | Drona Infotech
Mobile app Development Services | Drona Infotech
Drona Infotech
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
Łukasz Chruściel
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
Peter Muessig
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Łukasz Chruściel
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
Remote DBA Services
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
dakas1
 
zOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL DifferenceszOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL Differences
YousufSait3
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
sjcobrien
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
rodomar2
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
ICS
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
Alina Yurenko
 

Recently uploaded (20)

Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemUI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
Top 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptxTop 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptx
 
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative AnalysisOdoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
 
Mobile app Development Services | Drona Infotech
Mobile app Development Services  | Drona InfotechMobile app Development Services  | Drona Infotech
Mobile app Development Services | Drona Infotech
 
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf2024 eCommerceDays Toulouse - Sylius 2.0.pdf
2024 eCommerceDays Toulouse - Sylius 2.0.pdf
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
 
zOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL DifferenceszOS Mainframe JES2-JES3 JCL-JECL Differences
zOS Mainframe JES2-JES3 JCL-JECL Differences
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
 

Taming HBase with Apache Phoenix and SQL

  • 1. Apache Phoenix James Taylor @JamesPlusPlus Maryann Xue @MaryannXue Eli Levine @teleturn We put the SQL back in NoSQL http://phoenix.incubator.apache.org
  • 2. About James Completed o Engineer at Salesforce.com in BigData group o Started Phoenix as internal project ~2.5 years ago o Open-source on Github ~1.5 years ago o Apache incubator for past 5 months o Engineer at BEA Systems o XQuery-based federated query engine o SQL-based complex event processing engine
  • 3. Agenda Completed o What is Apache Phoenix? o Why is it so fast? o How does it help HBase scale? o Roadmap o Q&A
  • 4. What is Apache Phoenix? Completed
  • 5. What is Apache Phoenix? Completed 1. Turns HBase into a SQL database o Query Engine o MetaData Repository o Embedded JDBC driver o Only for HBase data
  • 6. What is Apache Phoenix? Completed 2. Fastest way to access HBase data o HBase-specific push down o Compiles queries into native HBase calls (no map-reduce) o Executes scans in parallel
  • 7. Completed SELECT * FROM t WHERE k IN (?,?,?) Phoenix Stinger (Hive 0.13) 0.04 sec 280 sec * 110M row table 7,000x faster
  • 8. What is Apache Phoenix? Completed 3. Lightweight o No additional servers required o Bundled with Hortonworks 2.1 o 100% Java
  • 12. What is Apache Phoenix? Completed 4. Integration-friendly o Map to existing HBase table o Integrate with Apache Pig o Integrate with Apache Flume o Integrate with Apache Sqoop (wip)
  • 13. What is Apache Phoenix? Completed 1. Turns HBase into a SQL database 2. Fastest way to access HBase data 3. Lightweight 4. Integration-friendly
  • 14. Why is Phoenix so fast? Completed
  • 15. Why is Phoenix so fast? Completed 1. HBase o Fast, but “dumb” (on purpose) 2. Data model o Support for composite primary key o Binary data sorts naturally 3. Client-side parallelization 4. Push down o Custom filters and coprocessors
  • 16. Phoenix Data Model HBase Table Phoenix maps HBase data model to the relational world
  • 17. Phoenix Data Model HBase Table Column Family A Column Family B Phoenix maps HBase data model to the relational world
  • 18. Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Phoenix maps HBase data model to the relational world
  • 19. Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 KeyValue Phoenix maps HBase data model to the relational world
  • 20. Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 KeyValue Row Key 2 KeyValue KeyValue Phoenix maps HBase data model to the relational world
  • 21. Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 KeyValue Row Key 2 KeyValue KeyValue Row Key 3 KeyValue Phoenix maps HBase data model to the relational world
  • 22. HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 Value Row Key 2 Value Value Row Key 3 Value Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 KeyValue Row Key 2 KeyValue KeyValue Row Key 3 KeyValue Phoenix maps HBase data model to the relational world
  • 23. HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 Value Row Key 2 Value Value Row Key 3 Value HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 Value Row Key 2 Value Value Row Key 3 Value Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 KeyValue Row Key 2 KeyValue KeyValue Row Key 3 KeyValue Phoenix maps HBase data model to the relational world
  • 24. HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 Value Row Key 2 Value Value Row Key 3 Value HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 Value Row Key 2 Value Value Row Key 3 Value Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 KeyValue Row Key 2 KeyValue KeyValue Row Key 3 KeyValue Phoenix maps HBase data model to the relational world Multiple Versions
  • 25. Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 KeyValue Row Key 2 KeyValue KeyValue Row Key 3 KeyValue Phoenix maps HBase data model to the relational world Phoenix Table
  • 26. Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 KeyValue Row Key 2 KeyValue KeyValue Row Key 3 KeyValue Phoenix maps HBase data model to the relational world Phoenix Table Key Value Columns
  • 27. Phoenix Data Model HBase Table Column Family A Column Family B Qualifier 1 Qualifier 2 Qualifier 3 Row Key 1 KeyValue Row Key 2 KeyValue KeyValue Row Key 3 KeyValue Phoenix maps HBase data model to the relational world Phoenix Table Key Value ColumnsPrimary Key Constraint
  • 28. Example Row Key SERVER METRICS HOST VARCHAR DATE DATE RESPONSE_TIME INTEGER GC_TIME INTEGER CPU_TIME INTEGER IO_TIME INTEGER Over metrics data for servers with a schema like this:
  • 29. Example Over metrics data for servers with a schema like this: Key Values SERVER METRICS HOST VARCHAR DATE DATE RESPONSE_TIME INTEGER GC_TIME INTEGER CPU_TIME INTEGER IO_TIME INTEGER
  • 30. Example CREATE TABLE SERVER_METRICS ( HOST VARCHAR, DATE DATE, RESPONSE_TIME INTEGER, GC_TIME INTEGER, CPU_TIME INTEGER, IO_TIME INTEGER, CONSTRAINT pk PRIMARY KEY (HOST, DATE)) DDL command looks like this:
  • 31. With data that looks like this: SERVER METRICS HOST + DATE RESPONSE_TIME GC_TIME SF1 1396743589 1234 SF1 1396743589 8012 … SF3 1396002345 2345 SF3 1396002345 2340 SF7 1396552341 5002 1234 … Example Row Key
  • 32. With data that looks like this: SERVER METRICS HOST + DATE RESPONSE_TIME GC_TIME SF1 1396743589 1234 SF1 1396743589 8012 … SF3 1396002345 2345 SF3 1396002345 2340 SF7 1396552341 5002 1234 … Example Key Values
  • 33. Phoenix Push Down: Example Completed SELECT host, avg(response_time) FROM server_metrics WHERE date > CURRENT_DATE() – 7 AND host LIKE ‘SF%’ GROUP BY host
  • 34. Phoenix Push Down: Example Completed SELECT host, avg(response_time) FROM server_metrics WHERE date > CURRENT_DATE() – 7 AND host LIKE ‘SF%’ GROUP BY host
  • 35. Phoenix Push Down: Example Completed SELECT host, avg(response_time) FROM server_metrics WHERE date > CURRENT_DATE() – 7 AND host LIKE ‘SF%’ GROUP BY host
  • 36. Phoenix Push Down: Example Completed SELECT host, avg(response_time) FROM server_metrics WHERE date > CURRENT_DATE() – 7 AND host LIKE ‘SF%’ GROUP BY host
  • 37. Phoenix Push Down: Example Completed SELECT host, avg(response_time) FROM server_metrics WHERE date > CURRENT_DATE() – 7 AND host LIKE ‘SF%’ GROUP BY host
  • 38. Phoenix Push Down 1. Skip scan filter 2. Aggregation 3. TopN 4. Hash Join
  • 39. Phoenix Push Down: Skip scan SELECT host, avg(response_time) FROM server_metrics WHERE date > CURRENT_DATE() – 7 AND host LIKE ‘SF%’ GROUP BY host
  • 40. Phoenix Push Down: Skip scan Completed R1 R2 R3 R4
  • 41. Phoenix Push Down: Skip scan Client-side parallel scans Completed R1 R2 R3 R4 scan1 scan3 scan2
  • 42. Phoenix Push Down: Skip scan Server-side filter Completed SKIP
  • 43. Phoenix Push Down: Skip scan Server-side filter Completed INCLUDE
  • 44. Phoenix Push Down: Skip scan Server-side filter Completed SKIP
  • 45. Phoenix Push Down: Skip scan Server-side filter Completed INCLUDE
  • 46. Phoenix Push Down: Skip scan Server-side filter SKIP
  • 47. Phoenix Push Down: Skip scan Server-side filter INCLUDE
  • 48. Phoenix Push Down: Skip scan Server-side filter INCLUDE INCLUDE INCLUDE
  • 49. Phoenix Push Down: Aggregation SELECT host, avg(response_time) FROM server_metrics WHERE date > CURRENT_DATE() – 7 AND host LIKE ‘SF%’ GROUP BY host
  • 50. SERVER METRICS HOST DATE KV1 KV2 KV3 SF1 Jun 2 10:10:10.234 239 234 674 SF1 Jun 3 23:05:44.975 23 234 SF1 Jun 9 08:10:32.147 256 314 341 SF1 Jun 9 08:10:32.147 235 256 SF1 Jun 1 11:18:28.456 235 23 SF1 Jun 3 22:03:22.142 234 314 SF1 Jun 3 22:03:22.142 432 234 256 SF2 Jun 1 10:29:58.950 23 432 SF2 Jun 2 14:55:34.104 314 876 23 SF2 Jun 3 12:46:19.123 256 234 314 SF2 Jun 3 12:46:19.123 432 SF2 Jun 8 08:23:23.456 876 876 235 SF2 Jun 1 10:31:10.234 234 234 876 SF3 Jun 1 10:31:10.234 432 432 234 SF3 Jun 3 10:31:10.234 890 SF3 Jun 8 10:31:10.234 314 314 235 SF3 Jun 1 10:31:10.234 256 256 876 SF3 Jun 1 10:31:10.234 235 234 SF3 Jun 8 10:31:10.234 876 876 432 SF3 Jun 9 10:31:10.234 234 234 SF3 Jun 3 10:31:10.234 432 276 … … … … … Phoenix Push Down: Aggregation Aggregate on server-side SERVER METRICS HOST AGGREGATE VALUES SF1 3421 SF2 2145 SF3 9823
  • 51. Phoenix Push Down: TopN Completed SELECT host, date, gc_time FROM server_metrics WHERE date > CURRENT_DATE() – 7 AND host LIKE ‘SF%’ ORDER BY gc_time DESC LIMIT 5
  • 52. Phoenix Push Down: TopN Client-side parallel scans Completed R1 R2 R3 R4 scan1 scan3 scan2
  • 53. Phoenix Push Down: TopN Each region holds N rows Completed R1 R2 R3 R4 scan1
  • 54. Phoenix Push Down: TopN Each region holds N rows Completed R1 R2 R3 R4 scan2
  • 55. Phoenix Push Down: TopN Each region holds N rows Completed R1 R2 R3 R4 scan3
  • 56. SERVER METRICS HOST DATE GC_TIME SF3 Jun 2 10:10:10.234 22123 SF5 Jun 3 23:05:44.975 19876 SF2 Jun 9 08:10:32.147 11345 SF2 Jun 1 11:18:28.456 10234 SF1 Jun 3 22:03:22.142 10111 Phoenix Push Down: TopN Client-side final merge sort Scan1 Scan2 Scan3
  • 57. Phoenix Push Down: TopN Secondary Index Completed CREATE INDEX gc_time_index ON server_metrics (gc_time DESC, date DESC) INCLUDE (response_time) Row Key GC_TIME_INDEX GC_TIME INTEGER DATE DATE HOST VARCHAR RESPONSE_TIME INTEGER
  • 58. Phoenix Push Down: TopN Secondary Index Completed CREATE INDEX gc_time_index ON server_metrics (gc_time DESC, date DESC) INCLUDE (response_time) Key Value GC_TIME_INDEX GC_TIME INTEGER DATE DATE HOST VARCHAR RESPONSE_TIME INTEGER
  • 59. Phoenix Push Down: TopN Secondary Index Completed o Original query doesn’t change o Phoenix rewrites query to use index table o All referenced columns must exist in index table for it to be considered o Local Indexing coming soon! o Stats coming soon!
  • 60. Phoenix Push Down: Hash Join Completed SELECT m.*, i.location FROM server_metrics m JOIN host_info i ON m.host = i.host WHERE m.date > CURRENT_DATE() – 7 AND i.location = ‘SF’ ORDER BY m.gc_time DESC LIMIT 5
  • 61. Phoenix Push Down: Hash Join Separate LHS and RHS Completed SELECT m.*, i.location FROM server_metrics m JOIN host_info i ON m.host = i.host WHERE m.date > CURRENT_DATE() – 7 AND i.location = ‘SF’ ORDER BY m.gc_time DESC LIMIT 5
  • 62. Phoenix Push Down: Hash Join Separate LHS and RHS Completed SELECT m.*, i.location FROM server_metrics m JOIN host_info i ON m.host = i.host WHERE m.date > CURRENT_DATE() – 7 AND i.location = ‘SF’ ORDER BY m.gc_time DESC LIMIT 5
  • 63. Phoenix Push Down: Hash Join Separate LHS and RHS Completed LHS SELECT * FROM server_metrics WHERE date > CURRENT_DATE() – 7 ORDER BY gc_time DESC LIMIT 5 RHS SELECT host, location FROM host_info WHERE location = ‘SF’
  • 64. Phoenix Push Down: Hash Join Execute & broadcast RHS to each RS Completed RS1 RS2 RHS
  • 65. Phoenix Push Down: Hash Join Server-side map lookup during scan Completed R1 R2 R3 R4 RHS RHS LHS scan1 scan2 scan3 scan4
  • 66. Phoenix Push Down: Hash Join Derived Tables and Filter Rewrite Completed SELECT i.host, i.location, m.res, m.gc FROM host_info i JOIN (SELECT host, avg(response_time) res, avg(gc_time) gc FROM server_metrics GROUP BY host) AS m ON i.host = m.host WHERE i.location != ‘NJ’ AND (m.res >= 10000 OR m.gc >= 2000)
  • 67. Phoenix Push Down: Hash Join Filters Pushed Down and Rewritten for RHS Completed RHS SELECT host, avg(response_time), avg(gc_time) FROM server_metrics GROUP BY host HAVING avg(response_time)>=10000 OR avg(gc_time)>=2000 LHS SELECT host, location FROM host_info WHERE location != ‘NJ’
  • 68. Phoenix Push Down: Hash Join Multiple Joins and Sub-joins Completed SELECT * FROM server_metrics m JOIN (host_info h JOIN location_info l ON h.location = l.location) ON m.host = h.host WHERE m.date > CURRENT_DATE() – 7 AND l.user_count >= 200000
  • 69. Phoenix Push Down: Hash Join Recursively Separate LHS and RHS Completed Outer RHSOuter LHS SELECT * FROM server_metrics WHERE date > CURRENT_DATE() – 7 Inner LHS SELECT * FROM host_info Inner RHS SELECT * FROM location_info WHERE user_count >= 200000
  • 70. Phoenix Push Down: Hash Join Iterative Execution of Multiple Joins CompletedRS1 RS2 Inner RHS Broadcast RS1 RS1 RS2 RS3 RS4 Scan Inner LHS Outer RHS Broadcast Scan Outer LHS Scan Inner RHS
  • 71. Phoenix Push Down: Hash Join Star-join Optimization Completed SELECT * FROM server_metrics m JOIN host_info h ON m.host = h.host JOIN location_info l ON h.location = l.location WHERE m.date > CURRENT_DATE() – 7 AND l.user_count >= 200000
  • 72. Phoenix Push Down: Hash Join Separate LHS and Multiple Parallel RHS Completed LHS SELECT * FROM server_metrics WHERE date > CURRENT_DATE() – 7 RHS 1 SELECT * FROM host_info RHS 2 SELECT * FROM location_info WHERE user_count >= 200000
  • 73. Phoenix Push Down: Hash Join Star-join Execution of Multiple Joins Completed RS1 RS2 RHS 1 RS1 RS1 RS2 RS3 RS4 Scan RHS 2 Broadcast Scan LHSScan RHS 1 RHS 2
  • 74. New in Phoenix 3: Shared Tables • HBase wants small # of big tables instead of large # of small tables • Two types of shared tables in Phoenix 3: • Views • Tenant-specific Tables
  • 75. Views • Multiple Phoenix tables share same physical HBase table • Inherit parent table’s PK, KV columns • Updateable Views • Secondary Indexes on Views
  • 76. Completed CREATE TABLE event ( type CHAR(1), event_id BIGINT, created_date DATE, created_by VARCHAR, CONSTRAINT pk PRIMARY KEY (type, event_id)); CREATE VIEW web_event ( referrer VARCHAR) AS SELECT * FROM event WHERE type=‘w’; • Includes columns from TABLE • Cannot define PK • Updateable if only equality expressions separated by AND Views
  • 77. Completed type = ‘c’ type = ‘m’ type = ‘p’ type = ‘w’ EVENT CHAT_EVENT MOBILE_EVENT PHONE_EVENT WEB_EVENT Views
  • 78. Tenant-Specific Tables • Tenant data isolation and co-location • Built using Views • Uses tenant-specific Connections
  • 79. Completed CREATE TABLE event ( tenant_id VARCHAR, type CHAR(1), event_id BIGINT, created_date DATE, created_by VARCHAR, CONSTRAINT pk PRIMARY KEY (tenant_id, type, event_id)) MULTI_TENANT=true; First PK column identifies tenant ID Tenant-Specific Tables Step 1: Create multi-tenant base table
  • 80. Completed CREATE VIEW web_event ( referrer VARCHAR) AS SELECT * FROM event WHERE type=‘w’; DriverManager.connect(“jdbc:phoenix:localhost;TenantId=me”); CREATE VIEW my_web_event AS SELECT * FROM web_event; Tenant-specific view Tenant-specific connection Tenant-Specific Tables Step 2: Create tenant-specific tables
  • 81. tenant_id = ‘me’ … EVENT tenant_id = ‘you’ tenant_id = ‘them’ Tenant-Specific Tables
  • 82. type = ‘c’ type = ‘m’ type = ‘p’ type = ‘w’ EVENT CHAT_EVENT MOBILE_EVENT PHONE_EVENT WEB_EVENT PER tenant_id Tenant-Specific Tables
  • 83. • Tenant-specific connection may only see and operate on their data • Inherit parent table’s PK, KV columns • Tenant-specific secondary indexes • Restrictions: • No ALTER base table • No DROP columns used in PK and where clause • PK columns same as parent Tenant-Specific Tables
  • 84. • Allow shared tables to extend parent’s PK • Support more complex WHERE clauses for updatable views • Support projecting subset of columns to View Shared Tables Future Work
  • 85. Phoenix Roadmap Completed o Local Indexes o Transactions o More Join strategies o Cost-based query optimizer o OLAP extensions