Calcite meetup-2016-04-20

Introduction to
Apache Calcite
Josh Elser
MTS
2016-04-20

2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
About me
Apache Calcite is a project at the Apache Software Foundation.
This name is a trademark of the Foundation.
Apache Calcite Committer and PMC
(Slowly) Re-learning SQL
Distributed systems nerd

Users
Apache
Kylin
Apache Samza
Quark
SQL-Gremlin/Apache TinkerPop
See the respective project pages at the ASF

Brief History
 Originally known as “Optiq” (https://github.com/julianhyde/optiq): 2012-05-07
 Entered Apache Software Fundation’s Incubator: 2014-05-25
 Renamed to Apache Calcite (incubating): 2014-09-30
 Graduates to top-level project (TLP): 2015-10-21
 2 major releases since graduation: 2016-03-XX
 Currently comprised of 16 committers and 14 PMC members
“The foundation for your next high-performance database.”

Agenda
SQL Parser

SQL Parser
SELECT d.name, COUNT(*) as c
FROM Emps as e
JOIN Depts as d ON e.deptno = d.deptno
WHERE e.age < 30
GROUP BY d.deptno
HAVING COUNT(*) > 5
ORDER BY c DESC
Scan
Join
Filter
Aggregate
Filter
Project
Sort
Scanhttps://calcite.apache.org/docs/reference.html

Agenda
SQL Parser
Cost-Based Optimizer

 Extensible
 Java API
– Parser output
– Inline Java code
AKA Relational Algebra
RelBuilder builder = RelBuilder.create(config);
RelNode node = builder
.scan("EMP")
.project(builder.field(“DEPTNO”),
builder.field(“ENAME”))
.build();
SELECT ename, deptno FROM emp;
LogicalProject(DEPTNO, ENAME)
LogicalTableScan(EMP)

SELECT d.name,
COUNT(*) as c
FROM depts AS d
JOIN emp AS e
on d.deptno = e.deptno
GROUP BY d.name;
Scan Emp[deptno]
Join
Aggregate
Scan Depts[deptno,
name]
Join
Aggregate
Project[name, c]
Scan Emp[*] Scan Depts[*]

Agenda
SQL Parser
Pluggable Data Sources

 User-implemented
– Yes, you.
 Custom optimizations
– Predicate pushdown
– Projections
 Sources of Sources
– Federation
Everything but the data
Join
Aggregate
Project
Scan Emp[*] Scan Depts[*]

Agenda
SQL Parser
Avatica

Avatica
 Calcite sub-project
 Wire protocol
– Protocol Buffers
– JSON
 Metrics
 Authentication
 Clients
– JDBC client
– Python and Go (in-progress)
JDBC over HTTP – SQL for Everyone

Thank You!
Email: elserj@apache.org
Twitter: @josh_elser
Mailing lists:
dev@calcite.apache.org
Project info:
https://calcite.apache.org/

Calcite meetup-2016-04-20

More Related Content

What's hot

Viewers also liked

Similar to Calcite meetup-2016-04-20

Recently uploaded

Calcite meetup-2016-04-20