Talk at Apache Drill Meetup (November 2018) describing how to accelerate SQL queries in a NoSQL database using Apache Drill and Secondary Indexes. Drill (in conjunction with Apache Calcite) provides a comprehensive cost-based index planning and execution framework. Queries with indexed columns in the WHERE clause, ORDER BY, GROUP BY and Joins can be sped up substantially. A reference implementation with MapR-DB JSON database is described.

1. Apache Drill Meetup Nov’18
Accelerating SQL queries in NoSQL Databases using Apache
Drill and Secondary Indexes
Aman Sinha
Apache Drill Meetup, November 2018

2. Apache Drill Meetup Nov’18
About me
• Apache Drill PMC and Apache Calcite PMC. Past PMC chair of Drill.
• Engineering @ MapR
• Main areas of interest: SQL query processing for RDBMS, NoSQL, Hadoop
• Contact: amansinha@apache.org, Github: https://github.com/amansinha100

3. Apache Drill Meetup Nov’18
Talk Outline
• Background
– Overview of Apache Drill
– Overview of Secondary Index in NoSQL databases
• Leveraging secondary index via Drill
• Index interfaces exposed to plugins
• Demo

4. Apache Drill Meetup Nov’18
Architecture Summary
• Schema-on-read
• No centralized metastore
• Supports wide range of data sources via ‘plugins’
• Fully Java based
• In-memory columnar processing
• Mostly off-heap memory management (negligible GC overhead)
• Code generation for run-time operators
• Optimistic, pipelined execution model
• Spill to disk for blocking operations under memory pressure
• Integrated with YARN for resource management
• Provides an extensible framework for UDFs

5. Apache Drill Meetup Nov’18
Background
● Drill overview
● Secondary Index overview

6. Apache Drill Meetup Nov’18
HDFS HBase
MapR-
DB
HDFS HBase
MapR-
DB
DRILLBIT (any
drillbit can be
Foreman)
HDFS HBase
MapR-
DB
DRILLBIT DRILLBIT
JDBC/ODBC
Client
Web
Console
Data Sources (exposed to Drill as
Storage/Format Plugins)
Zookeeper (could be
co-resident with
Drillbit nodes)
Drill: Distributed SQL Query Engine
BI Tools

7. Apache Drill Meetup Nov’18
NoSQL DB: Primary Index
• HBase and MapR-DB tables have primary key (row key) column
– Column values are sorted
– Efficient range pruning is done for rowkey predicates:
WHERE rowkey BETWEEN ‘user003’ AND ‘user007’
• Secondary columns (e.g ‘State’) values are not sorted
– Predicate WHERE state = ‘CA’ need full table scan !
user009 WA
user010 OR
user011 CA
user012 TX
user005 WA
user006 OR
user007 CA
user008 TX
user001 WA
user002 OR
user003 CA
user004 TX
Rowkey Rowkey Rowkey
Primary table regions

8. Apache Drill Meetup Nov’18
Solution: Secondary Index ‘Tables’
user001 WA
user002 OR
user003 CA
user004 TX
user005 WA
user006 OR
user007 CA
user008 TX
user009 WA
user010 OR
user011 CA
user012 TX
CA_user003
CA_user007
CA_user011
OR_user002
OR_user006
OR_user010
TX_user004
TX_user008
TX_user012
WA_user001
WA_user005
WA_user009
Primary table regions
Index table regions
Rowkey Rowkey Rowkey
Rowkey

9. Apache Drill Meetup Nov’18
Secondary Index
• NoSQL DBs supporting secondary index
– MongoDB
– MapR-DB JSON
– HBase + Phoenix
– Couchbase
– Cassandra
• What’s missing ?
– Other than Hbase + Phoenix, others don’t have an ANSI SQL interface
– There’s a need for a generalized cost-based index planning and execution framework
– A key requirement:
• Framework must be able to support ‘global’ non-covering indexes, not just
covering index

10. Apache Drill Meetup Nov’18
Terminology and concepts
• Indexed fields, Included fields
• Simple Index, Composite Index
• Range Index, Hash Index
• Covering Index: All columns referenced in the query are available in the
index
– Easier to handle by the optimizer. Generate an index-only plan.
• Non-Covering Index: Only a subset of the columns referenced in the query
are available in the index
– Needs more supporting infrastructure from optimizer and executor

11. Apache Drill Meetup Nov’18
Leveraging Secondary Index via Drill

12. Apache Drill Meetup Nov’18
Feature summary
• Designed for storage/format plugin whose backend supports secondary
indexing
– Reference implementation is with MapR-DB JSON
– NOTE: Drill does not create indexes; it uses them
• Index metadata is exposed to Drill planner through well defined interfaces
• Statistics (if available) are also exposed
• New run-time operators added for executing index plans
• Drill Planner extends Apache Calcite’s planner and does cost-based index
selection
• Feature will be available in upcoming Drill 1.15 release

13. Apache Drill Meetup Nov’18
Types of Queries Eligible for Index Planning
• WHERE clause with local filters
– <, >, =, BETWEEN
– IN, LIKE
– Eligible ANDed conditions
– Eligible ORed conditions
– Certain types of functions, e.g CAST(zipcode as BIGINT) = 12345 (only if data
source supports functional indexes)
• ORDER BY
• GROUP BY (using StreamingAggregate)
• JOIN (using MergeJoin)

14. Apache Drill Meetup Nov’18
Covering index plan
Index Scan
• SELECT zipcode FROM T WHERE state = ‘CA’ AND age < 30
• Composite key index on {state, age}, included field: {zipcode}
Filter
state = ‘CA’ AND age <
30
Project
UnionExchange
These are subsequently pushed
down by plugin-specific filter
pushdown rule and project
pushdown rule

15. Apache Drill Meetup Nov’18
Non-Covering Index Plan
• SELECT * FROM T WHERE state = ‘CA’ AND age < 30
• Composite key index on {state, age}
• How to produce the remaining (‘star’) columns ?
Index Scan
state = ‘CA’ AND
age < 30
RowKey Join
Range Partition
Exchange
Restricted (a.k.a
‘skip’ ) scan
Return rows
Row keys
Supply row
keys
Do ‘bucketing’ of row keys belonging
to the same region/tablet (this needs
knowledge of the tablet map)
Project all
columns

16. Apache Drill Meetup Nov’18
Example: ORDER BY queries
state county zipcode
SELECT zipcode
FROM T
WHERE state = ‘CA’
AND county = ‘Santa Clara’
ORDER BY zipcode
● Composite Index on {state, county,
zipcode}
● Planner will use this index and avoid sort
Index keys

17. Apache Drill Meetup Nov’18
Index Intersection
• SELECT * FROM T WHERE state IN (‘TX’, ‘CA’) AND county like ‘San%’
• Suppose single key index exists on ‘state’ and ‘county’
Index Scan
state in (‘TX’,
‘CA’)
Index Scan
county LIKE
‘San%’
Intersect HashJoin
Broadcast Exchange
RowKey Join
Range Partition
Exchange
Restricted Scan

18. Apache Drill Meetup Nov’18
Index Selection
● Drill planner in conjunction with Calcite’s Volcano planner provides
cost-based index selection. In addition, Drill planner employs a heuristic to
reduce the overall search space
○ Ranks the indexes based on few criteria: leading prefix selectivity, covering property,
collation property
○ Picks top 5 indexes (configurable) for further plan generation
○ Intersection of indexes is factored into selection
● Volcano planner compares cost of index plans with each other and the full
table scan plan and picks cheapest

19. Apache Drill Meetup Nov’18
Index Interfaces Exposed to Plugins

20. Apache Drill Meetup Nov’18
Sample interfaces to be implemented by plugin
● DbGroupScan
○ IndexCollection getSecondaryIndexCollection(RelNode scan)
○ DbGroupScan getRestrictedScan(List<SchemaPath> columns);
○ PartitionFunction getRangePartitionFunction(List<FieldReference> refList)
○ PluginCost getPluginCostModel()
● PluginCost
○ int getSequentialBlockReadCost(GroupScan scan)
○ int getRandomBlockReadCost(GroupScan scan)
● IndexDiscover
○ IndexCollection getTableIndex(String tableName)
● IndexDefinition
○ List<LogicalExpression> getRowKeyColumns()
○ List<LogicalExpression> getIndexColumns()
○ List<LogicalExpression> getNonIndexColumns()
○ Map<LogicalExpression, RelFieldCollation> getCollationMap()

21. Apache Drill Meetup Nov’18
Demo
• Sample queries on TPC-H
• Examine query profile : identify which index is picked
• Show the index definitions: indexed fields, included fields

22. Apache Drill Meetup Nov’18
Additional Resources
Apache Drill:
• http://drill.apache.org
• Mailing lists:
– user@drill.apache.org
– dev@drill.apache.org
Secondary Indexes in MapR-DB:
https://mapr.com/docs/61/MapR-DB/Indexes/Indexes.html
Thank you !