Sadayuki Furuhashi
Founder & Software Architect
ODBC & JDBC connectivity for Presto
Treasure Data, inc.
A little about me...
> Sadayuki Furuhashi
github/twitter: @frsyuki
> Treasure Data, Inc.
Founder & Software Architect
> Op...
Background + Intro:
Background
Pig
• Tableau
• Pentaho
• Web apps
RDB, HTTP, etc.
“Plazma”
Columnar

Cloud Storage
This is us

(Treasure Data)
Pig
• Tableau
• Pentaho
• Web apps
RDB, HTTP, etc.
“Plazma”
Columnar

Cloud Storage
Data collection
> “Fluentd”streaming d...
Pig
• Tableau
• Pentaho
• Web apps
RDB, HTTP, etc.
“Plazma”
Columnar

Cloud Storage
Hadoop as a service
> “BigData”process...
Pig
• Tableau
• Pentaho
• Web apps
RDB, HTTP, etc.
“Plazma”
Columnar

Cloud Storage
Presto as a service
> Interactive quer...
Pig
• Tableau
• Pentaho
• Web apps
RDB, HTTP, etc.
“Plazma”
Columnar

Cloud Storage
Here is the problem…
ODBC/JDBC
Missing!
The problem to solve
• Providing open-source ODBC/JDBC connectivity 
for Presto quickly
• Tableau
• Pentaho
• Web apps
ODB...
A solution
•Using PostgreSQL ODBC/JDBC drivers
•Creating PostgreSQL protocol gateway
A solution
•Using PostgreSQL ODBC/JDBC drivers
•Creating PostgreSQL protocol gateway
PostgreSQL protocol gateway for Prest...
Architecture
Architecture
Tableau
Pentaho

Web apps

…
PostgreSQL protocol
PostgreSQL ODBC/JDBC driver,

Other PostgreSQL clients
pgpool-II
(patched)
Internal Architecture
Tableau…
select count(*) from x;
run_presto_as_temp_table(

…, ’select count(*) ...
SELECT from system catalogs
pgpool-II
(patched)
Tableau…
get table list
PostgreSQL
run CREATE TABLE

for each actual table...
Demo
Limitations
• Server-side prepare is not supported
• Cursor (DECLARE/FETCH) is not supported
• JDBC driver needs ?protocol...
We’re hiring!
www.treasuredata.com/careers
Upcoming SlideShare
Loading in …5
×

Prestogres, ODBC & JDBC connectivity for Presto

3,842 views
3,466 views

Published on

Prestogres provides ODBC & JDBC connectivity to Presto, a distributed SQL query engine.
Presto meetup @ Facebook (2014-05-14)

Published in: Software, Technology

Prestogres, ODBC & JDBC connectivity for Presto

  1. 1. Sadayuki Furuhashi Founder & Software Architect ODBC & JDBC connectivity for Presto Treasure Data, inc.
  2. 2. A little about me... > Sadayuki Furuhashi github/twitter: @frsyuki > Treasure Data, Inc. Founder & Software Architect > Open source projects MessagePack - efficient object serializer Fluentd - data collection tool ServerEngine - ruby framework to build multiprocess servers LS4 - distributed object storage system (suspended) kumofs - distributed key-value data store (suspended)
  3. 3. Background + Intro:
  4. 4. Background Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage This is us
 (Treasure Data)
  5. 5. Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage Data collection > “Fluentd”streaming data collection tool > Plugin architecture > github.com/fluent/fluentd
  6. 6. Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage Hadoop as a service > “BigData”processing • Funnel analysis for
 web services • Correlation analysis for
 ad-tech (DSP/SSP/DMP) • Creating OLAP cube > Multi-tenant scheduling • utilize idling resources
 purchased by other users
  7. 7. Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage Presto as a service > Interactive queries > Multi-tenant scheduling
 (in progress)
  8. 8. Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage Here is the problem… ODBC/JDBC Missing!
  9. 9. The problem to solve • Providing open-source ODBC/JDBC connectivity  for Presto quickly • Tableau • Pentaho • Web apps ODBC/JDBC • ODBC/JDBC are VERY complicated API > PostgreSQL ODBC driver: 60,000 lines > PostgreSQL JDBC driver: 43,000 lines
  10. 10. A solution •Using PostgreSQL ODBC/JDBC drivers •Creating PostgreSQL protocol gateway
  11. 11. A solution •Using PostgreSQL ODBC/JDBC drivers •Creating PostgreSQL protocol gateway PostgreSQL protocol gateway for Presto feature-complete &
 matured for many years some middleware
 already implemented
  12. 12. Architecture
  13. 13. Architecture Tableau Pentaho
 Web apps
 … PostgreSQL protocol PostgreSQL ODBC/JDBC driver,
 Other PostgreSQL clients
  14. 14. pgpool-II (patched) Internal Architecture Tableau… select count(*) from x; run_presto_as_temp_table(
 …, ’select count(*) from x’); patched pgpool-II wraps
 the SQL in a function call PostgreSQL the function sends the
 original sql to Presto select count(*) from x;
  15. 15. SELECT from system catalogs pgpool-II (patched) Tableau… get table list PostgreSQL run CREATE TABLE
 for each actual table run the original query to get metadata of tables
  16. 16. Demo
  17. 17. Limitations • Server-side prepare is not supported • Cursor (DECLARE/FETCH) is not supported • JDBC driver needs ?protocolVersion=2 option
  18. 18. We’re hiring! www.treasuredata.com/careers

×