Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Apache Calcite: One planner fits all

4,364 views

Published on

Lightning talk at XLDB 2015 (Stanford, California).

Apache Calcite: One planner fits all

  1. 1. One planner fits all 
 (How Apache Calcite makes it easier to write a DBMS) Lightning talk at XLDB 2015 Stanford, California Julian Hyde (Hortonworks)
  2. 2. –Mike Stonebraker (2005) “One size fits all” is an idea whose time has come and gone
  3. 3. • Hadoop and other open source technologies have deconstructed the DBMS • Query parser/API + catalog + authorization + algorithms + scheduler + engine + data format + storage
  4. 4. image credit: http://oliviaobryon.com
  5. 5. image credit: http://oliviaobryon.com Interesting Boring
  6. 6. Conventional
 DB Parser Algebra 
 Catalog 
 Data Algorithms
  7. 7. Apache Calcite
 DB framework Parser Algebra Engine 
 Data Engine 
 Data Engine 
 Data Schema SPI Operators,
 Rules,
 Statistics, Cost model
  8. 8. SELECT products.name, COUNT(*)
 FROM sales
 JOIN products USING (productId)
 WHERE sales.discount IS NOT NULL
 GROUP BY products.name
 ORDER BY COUNT(*) DESC scan [products] scan [sales] join filter aggregate sort scan [products] scan [sales] filter’ join’ aggregate sort FilterIntoJoinRule translate SQL to relational algebra
  9. 9. Relational algebra • Robust • Allows re-use • Complex cost-based optimization • Multiple front-ends & back- ends • Not just for “flat” relations SQL Other
 QL API Algebra Rules Engine A Engine B
  10. 10. Thank you! Download: http://calcite.incubator.apache.org Use Calcite to build your next database! Calcite powers Apache Hive, Drill, Phoenix, Kylin An Apache Incubator project
 since May 2014 @julianhyde What’s in the box? • SQL parser & AST • JDBC/ODBC framework • Built-in operators (project, filter, …) • In-memory engine • 100+ rules • Planning engines • Adapters (CSV, JDBC, Mongo, …) • Streaming SQL • Materialized views Apache
 Calcite Apache Calcite

×