Prestogres, ODBC & JDBC connectivity for Presto
Upcoming SlideShare
Loading in...5
×
 

Prestogres, ODBC & JDBC connectivity for Presto

on

  • 1,475 views

Prestogres provides ODBC & JDBC connectivity to Presto, a distributed SQL query engine.

Prestogres provides ODBC & JDBC connectivity to Presto, a distributed SQL query engine.
Presto meetup @ Facebook (2014-05-14)

Statistics

Views

Total Views
1,475
Views on SlideShare
1,426
Embed Views
49

Actions

Likes
7
Downloads
17
Comments
0

2 Embeds 49

http://www.slideee.com 27
https://twitter.com 22

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Prestogres, ODBC & JDBC connectivity for Presto Prestogres, ODBC & JDBC connectivity for Presto Presentation Transcript

  • Sadayuki Furuhashi Founder & Software Architect ODBC & JDBC connectivity for Presto Treasure Data, inc.
  • A little about me... > Sadayuki Furuhashi github/twitter: @frsyuki > Treasure Data, Inc. Founder & Software Architect > Open source projects MessagePack - efficient object serializer Fluentd - data collection tool ServerEngine - ruby framework to build multiprocess servers LS4 - distributed object storage system (suspended) kumofs - distributed key-value data store (suspended)
  • Background + Intro:
  • Background Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage This is us
 (Treasure Data)
  • Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage Data collection > “Fluentd”streaming data collection tool > Plugin architecture > github.com/fluent/fluentd
  • Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage Hadoop as a service > “BigData”processing • Funnel analysis for
 web services • Correlation analysis for
 ad-tech (DSP/SSP/DMP) • Creating OLAP cube > Multi-tenant scheduling • utilize idling resources
 purchased by other users
  • Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage Presto as a service > Interactive queries > Multi-tenant scheduling
 (in progress)
  • Pig • Tableau • Pentaho • Web apps RDB, HTTP, etc. “Plazma” Columnar
 Cloud Storage Here is the problem… ODBC/JDBC Missing!
  • The problem to solve • Providing open-source ODBC/JDBC connectivity  for Presto quickly • Tableau • Pentaho • Web apps ODBC/JDBC • ODBC/JDBC are VERY complicated API > PostgreSQL ODBC driver: 60,000 lines > PostgreSQL JDBC driver: 43,000 lines
  • A solution •Using PostgreSQL ODBC/JDBC drivers •Creating PostgreSQL protocol gateway
  • A solution •Using PostgreSQL ODBC/JDBC drivers •Creating PostgreSQL protocol gateway PostgreSQL protocol gateway for Presto feature-complete &
 matured for many years some middleware
 already implemented
  • Architecture
  • Architecture Tableau Pentaho
 Web apps
 … PostgreSQL protocol PostgreSQL ODBC/JDBC driver,
 Other PostgreSQL clients
  • pgpool-II (patched) Internal Architecture Tableau… select count(*) from x; run_presto_as_temp_table(
 …, ’select count(*) from x’); patched pgpool-II wraps
 the SQL in a function call PostgreSQL the function sends the
 original sql to Presto select count(*) from x;
  • SELECT from system catalogs pgpool-II (patched) Tableau… get table list PostgreSQL run CREATE TABLE
 for each actual table run the original query to get metadata of tables
  • Demo
  • Limitations • Server-side prepare is not supported • Cursor (DECLARE/FETCH) is not supported • JDBC driver needs ?protocolVersion=2 option
  • We’re hiring! www.treasuredata.com/careers