The document discusses improvements that could be made to the Apache Derby query optimizer. It notes that the Derby optimizer currently uses an exhaustive search approach that executes serially. This leads to poor performance on complex queries. Some proposed improvements include using parallel processing to optimize queries concurrently, exploring bushy tree plans with more join order flexibility, incorporating randomized algorithms, and using heuristics and genetic algorithms to optimize queries within a timeout threshold.

1. Apache Derby
Query Optimizer
Improvements
CS 4420
Group 13
Presented by: Nufail, Nadeeshani, Amila, Malith

2. Query optimization
• Evaluates the least cost execution plan to be
sent to the evaluation engine
• A key factor in deciding DBMS performance
• Cost-based vs. Heuristic

3. Derby Optimizer
• Considers left-deep trees
• Represent the tables in an array
• Goes through the search space in depth-first
manner
• Exhaustive search of query plans
• Cost based search space pruning

5. Performance Statistics
H2 PostgreSQL Derby
Query
embedded (ms) (ms) (ms)
1 28 43 1416
2 438 26 151733
3 420 35 147261
4 84256 31 125356
5 312 52 2026
6 63456 68 142458
Complex join queries for 8 relations, each with 400 records

6. Concurrency
• Derby optimizer's poor design is its main drawback,
which executes serially
• Uses two while loops to iterate through each join order,
and access path per each order
o getNextPermutation()
o getNextDecoratedPermutation()
o costPermutation()
• As easy approach: Use loop-parellel programming
pattern.
• Make each iteration independent and execute each
iteration in new threads.

7. Bushy Trees
• Left deep trees & Bushy trees
• More flexibility in query plan generation
• Has a large search space
• Best plan may be bushy

8. Bushy Trees contd.
More than half of the queries have better
solutions in the bushy tree solution space
M. Steinbrunn, et. al. 1997. Heuristic and randomized optimization for the join ordering problem

9. Randomized Algorithms
• Deterministic - Start from base relations
and build plans by adding one relation at
each step
• Randomized - Search for optimal solutions
around a particular starting point
• Trade optimization time for execution time
• No guarantee of the best solution
• Useful for joins with a high number of
relations

10. Heuristics For Timeout
nImprovements
• Consequence of a miserable timeout
value
• Time wasted for generation of numerous
plans + estimating their costs
• Applicability of optimal solution over sub-
optimal
• Heuristic based values for timeout
• Improvement over time

11. Genetic Algorithms
• Genetic?
• Used by PostgreSQL
• Evolution
o Removal of least fit individuals
o Recombination of individuals of high fitness
• Initial population: query plans with
possible join orders
• Fitness function: to minimize cost
• Lower cost join sequence has higher
fitness

12. Questions?

13. Thank you!