MariaDB's
New-Generation Optimizer Hints
Fine Optimizer Control and compatibility
Sergei Petrunia
MariaDB Foundation DeepDive
August 2015
Traditional ways to control the optimizer
●
Index Hints
– select * from tbl force index(idx1) where ...
●
Join Order Hints
– select straight_join col1 from t1, t2 ….
– select ... from t1 straight join t2 …
●
Session variables
– set optimizer_switch='materialization=off';
set join_cache_level=...;
set ...
select ...
New-style Hints
●
Special /*+ */ comment after main keyword
●
A lot of hints for access methods, optimizations, etc
– About 20 in total
– Covers what index and join order hints did
– Adds much more
●
Hints apply to query block, table, or index
select
/*+ JOIN_ORDER(t1 t2) INDEX(t1 idx1) */
col1, col2
from t1, t2
where ...
Origins of new-style Hints
●
Hints in Oracle Database
●
Introduced starting from MySQL 5.7
– Inherit the syntax and some concepts
– Hints themselves are different due to different optimizer
●
(what about the hypergraph optimizer? Do all hints apply?)
●
Hints in TiDB
select
/*+ JOIN_ORDER(t1 t2) INDEX(t1 idx1) */
...
MariaDB’s needs for hints
=> MariaDB is getting hints
Need fine
optimizer
control
Want
Compatibility
with MySQL
But not
”bug-for-bug”
MariaDB is getting New-style hints
●
MariaDB 12.0 (March 2025, rolling release, now stable)
– Initial support: Join Order, Subquery, some Access Method hints.
●
MariaDB 12.1 (June 2025, rolling release, now RC)
– More hints: Index use, control for most optimizations.
●
MariaDB 12.2 (no releases yet)
– Remaining optimizations: index_merge, Rowid Filter, etc.
●
MariaDB 12.3?
– ?
Hint walkthrough
Index hints
Similar to old-style:
(FORCE|IGNORE) INDEX [FOR (GROUP|ORDER|JOIN)}
select /*+ INDEX(tbl1 index1, index2, ...) */ ... from tbl1 ... ;
MySQL MariaDB
[NO_]INDEX Yes Yes
[NO_]GROUP_INDEX Yes Yes
[NO_]JOIN_INDEX Yes Yes
[NO_]ORDER_INDEX Yes Yes
Join order hints
MySQL MariaDB
JOIN_FIXED_ORDER Yes Yes
JOIN_ORDER Yes Yes
JOIN_PREFIX Yes Yes
JOIN_SUFFIX Yes Yes
select
/*+ JOIN_ORDER(TBL3, TBL2, t1) */ select_columns
from
t2 as TBL2
join t3 as TBL3 on ...
join t1 on ...
STRAIGHT_JOIN
…,T1,…,T2,…,T3,…
T1,T2,T3,……
……,T1,T2,T3
Easier to write than STRAIGHT JOIN
select
/*+ JOIN_ORDER(t1, TBL3, TBL2) */ select_columns
from
t2 as TBL2
join t3 as TBL3 on ...
join t1 on ...
●
“JOIN_ORDER(A,B,C)” is much easier than rewriting a query to
“A STRAIGHT JOIN B STRAIGHT JOIN C”
– ON expressions
– Outer joins (converted to inner)
– Etc
●
Hints use table aliases.
Gotcha: subquery merges
●
Semi-join WHERE IN (SELECT …) are processed in a similar way
●
Does merging make aliases ambiguous?
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
| 1 | SIMPLE | TBL1 | ALL | NULL | NULL | NULL | NULL | 10 | |
| 1 | SIMPLE | TBL1 | ALL | NULL | NULL | NULL | NULL | 1000 | Using join buffer (flat, BNL join) |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
| 1 | SIMPLE | TBL1 | ALL | NULL | NULL | NULL | NULL | 10 | |
| 1 | SIMPLE | TBL2 | ALL | NULL | NULL | NULL | NULL | 1000 | Using join buffer (flat, BNL join) |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
explain select * from t1 as TBL1, t2 as TBL2;
explain select * from t1 as TBL1, (select * from t2 as TBL1) as TBL2;
Ambiguous aliases
create view v1 as select * from t1 ... ;
select /*+ NO_INDEX(t1 a) */ * from v1,t1 where ...;
select /*+ JOIN_PREFIX(t1) */ * from v1,t1 where ... ;
●
This will apply to t1 outside v1:
●
This will apply to t1-inside-v1:
●
Hint “resolution” has different code paths
●
Causes counter-intuitive rules.
– Avoid this for now
Cost-based optimizations
MySQL MariaDB
[NO_]BKA Yes Yes
[NO_]BNL Yes Yes
[NO_]INDEX_MERGE Yes In progress
[NO_]MRR Yes Yes
[NO_]SKIP_SCAN Yes -
[NO_]ROWID_FILTER - In progress
[NO_]SPLIT_MATERIALIZED - Yes
●
NO_optname: disables use of the optimization
●
Optname: Try using(*) the optimization, even if it’s more expensive
(*)- or Encourage to use :-)
Subquery optimizations
●
Special hint syntax due to multiple strategies
●
Same meaning as cost-based.
MySQL MariaDB
SUBQUERY(strategy) Yes Yes
[NO_]SEMIJOIN(strategies) Yes Yes
/*+ [NO_]SEMIJOIN([query_block] sj_strategy,...) */
sj_strategy: FIRSTMATCH|LOOSESCAN|MATERIALIZATION|DUPSWEEDOUT
/*+ [NO_]SUBQUERY([query_block] strategy,...) */
strategy: MATERIALIZATION|INTOEXISTS
Does the Optimizer take hints?
●
Hints are “Best effort”, “Dumb interpretation”
TRAVEL /*+ USE_TRAM() */ FROM Helsinki TO Tallinn;
TRAVEL /*+ HAVE_BEER_BEFORE_ARRIVAL() */
FROM Helsinki TO Tallinn;
Non-cost-based optimizations
●
NO_optname: disable the optimization
●
optname: enable if disabled
– Not much point?
MySQL MariaDB
[NO_]MERGE Yes Yes
[NO_]DERIVED_CONDITION_PUSHDOWN Yes Yes
NO_ICP Yes Yes
NO_RANGE_OPTIMIZATION Yes Yes
Miscellaneous hints
MariaDB already has
●
●
Misc MySQL MariaDB
SET_VAR Yes Yes
MAX_EXECUTION_TIME Yes Yes
RESOURCE_GROUP Yes No
SET STATEMENT variable=value FOR ...;
SET STATEMENT max_statement_time=... FOR ...;
MariaDB doesn’t support resource groups.
Query Block hint
MySQL MariaDB
QB_NAME Yes Yes
Query blocks
Query block addressing
●
QB_NAME declares Query Block Name.
●
Hints control behavior in different query blocks.
with TOP_CUSTOMER as (
select /*+ QB_NAME(subq1) */ *
from customer C
order by C.balance desc limit 10
)
select /*+ INDEX(C@subq1 idx_balance) */ *
from TOP_CUSTOMER TC, orders ORD
where ...
with TOP_CUSTOMER as (
select /*+ QB_NAME(subq1) */ *
from customer C
order by C.balance desc limit 10
)
select /*+ INDEX(C@subq1 idx_balance) */ *
from TOP_CUSTOMER TC, orders ORD
where ...
Implicit Query Block names
● Implicit select#n name for every select
with TOP_CUSTOMER as (
select *
from customer C
order by C.balance desc limit 10
)
select /*+ INDEX(C@`select#2` idx_balance) */ *
from TOP_CUSTOMER TC, orders ORD
where ...
QB_NAME use cases
●
Does’t work in MySQL:
– Hints can’t control anything inside VIEWs
– QB_NAME has global visibilty
●
Conflicts?
●
CTEs that are used multiple times?
– Implicit names do not scale
select
/*+ hint1(arg1) hint2(arg2) hint(arg3, arg4) */ *
Col1,col2,somef_function_col(3(
from
Tbl1 join tbl2 on t1b.col1=tbl2.cond2 left join tbl3 ...
●
Can tune everything in the query from one location!
TiDB’s way – a possible solution?
●
QB_NAME assigns a name to Query Block elsewhere
●
Uses “path” through views
●
Uses SEL_n to find selects inside VIEWs.
●
=> We’re looking into this.
create view v1 as select ... from t where ...;
create view v2 as select ... from v1 where .... ;
select
/*+
QB_NAME(select1_in_v1, v2@SEL_1 .v1@SEL_1 .@SEL_1)
NO_ORDER_INDEX(t@select1_in_v1, idx1)
*/
col1
from v2;
From hints to saved query plans?
From Hints to Query Plans
●
Can we save the query plan as a set of hints?
●
In Oracle DB this was “stored outlines”
– Deprecated since Oracle 11g
●
TiDB doesn’t have it documented, but
TiDB> explain format='hint' select * from t10, t11 where t10.b=t11.a;
+--------------------------------------------------------------------------------------------------------+
| hint |
+--------------------------------------------------------------------------------------------------------+
| hash_join(@`sel_1` `test`.`t10`), use_index(@`sel_1` `test`.`t10` ), use_index(@`sel_1` `test`.`t11` ) |
+--------------------------------------------------------------------------------------------------------+
●
Is this worth doing?
Summary
Summary (1)
●
MariaDB is getting new-style hints.
– Introduced in MariaDB 12.0
– Reasonaby complete set in MariaDB 12.2
●
Hints are compatible with MySQL
– which were inspired by OracleDB’s
●
MariaDB hints cover
– MySQL’s hints, where applicable
– MariaDB’s optimizations
Summary (2)
●
Query block addressing can be improved
●
Coudl use hints for saving query plans in the
future?
Credits
●
Hints in MariaDB are brought to you by
– Oleg Smirnov
– Alexander Barkov
– Dave Gosselin
– Lena Startseva
– Sergei Petrunia
Thanks!

MariaDB's New-Generation Optimizer Hints

  • 1.
    MariaDB's New-Generation Optimizer Hints FineOptimizer Control and compatibility Sergei Petrunia MariaDB Foundation DeepDive August 2015
  • 2.
    Traditional ways tocontrol the optimizer ● Index Hints – select * from tbl force index(idx1) where ... ● Join Order Hints – select straight_join col1 from t1, t2 …. – select ... from t1 straight join t2 … ● Session variables – set optimizer_switch='materialization=off'; set join_cache_level=...; set ... select ...
  • 3.
    New-style Hints ● Special /*+*/ comment after main keyword ● A lot of hints for access methods, optimizations, etc – About 20 in total – Covers what index and join order hints did – Adds much more ● Hints apply to query block, table, or index select /*+ JOIN_ORDER(t1 t2) INDEX(t1 idx1) */ col1, col2 from t1, t2 where ...
  • 4.
    Origins of new-styleHints ● Hints in Oracle Database ● Introduced starting from MySQL 5.7 – Inherit the syntax and some concepts – Hints themselves are different due to different optimizer ● (what about the hypergraph optimizer? Do all hints apply?) ● Hints in TiDB select /*+ JOIN_ORDER(t1 t2) INDEX(t1 idx1) */ ...
  • 5.
    MariaDB’s needs forhints => MariaDB is getting hints Need fine optimizer control Want Compatibility with MySQL But not ”bug-for-bug”
  • 6.
    MariaDB is gettingNew-style hints ● MariaDB 12.0 (March 2025, rolling release, now stable) – Initial support: Join Order, Subquery, some Access Method hints. ● MariaDB 12.1 (June 2025, rolling release, now RC) – More hints: Index use, control for most optimizations. ● MariaDB 12.2 (no releases yet) – Remaining optimizations: index_merge, Rowid Filter, etc. ● MariaDB 12.3? – ?
  • 7.
  • 8.
    Index hints Similar toold-style: (FORCE|IGNORE) INDEX [FOR (GROUP|ORDER|JOIN)} select /*+ INDEX(tbl1 index1, index2, ...) */ ... from tbl1 ... ; MySQL MariaDB [NO_]INDEX Yes Yes [NO_]GROUP_INDEX Yes Yes [NO_]JOIN_INDEX Yes Yes [NO_]ORDER_INDEX Yes Yes
  • 9.
    Join order hints MySQLMariaDB JOIN_FIXED_ORDER Yes Yes JOIN_ORDER Yes Yes JOIN_PREFIX Yes Yes JOIN_SUFFIX Yes Yes select /*+ JOIN_ORDER(TBL3, TBL2, t1) */ select_columns from t2 as TBL2 join t3 as TBL3 on ... join t1 on ... STRAIGHT_JOIN …,T1,…,T2,…,T3,… T1,T2,T3,…… ……,T1,T2,T3
  • 10.
    Easier to writethan STRAIGHT JOIN select /*+ JOIN_ORDER(t1, TBL3, TBL2) */ select_columns from t2 as TBL2 join t3 as TBL3 on ... join t1 on ... ● “JOIN_ORDER(A,B,C)” is much easier than rewriting a query to “A STRAIGHT JOIN B STRAIGHT JOIN C” – ON expressions – Outer joins (converted to inner) – Etc ● Hints use table aliases.
  • 11.
    Gotcha: subquery merges ● Semi-joinWHERE IN (SELECT …) are processed in a similar way ● Does merging make aliases ambiguous? +------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+ | 1 | SIMPLE | TBL1 | ALL | NULL | NULL | NULL | NULL | 10 | | | 1 | SIMPLE | TBL1 | ALL | NULL | NULL | NULL | NULL | 1000 | Using join buffer (flat, BNL join) | +------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+ +------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+ | id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | +------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+ | 1 | SIMPLE | TBL1 | ALL | NULL | NULL | NULL | NULL | 10 | | | 1 | SIMPLE | TBL2 | ALL | NULL | NULL | NULL | NULL | 1000 | Using join buffer (flat, BNL join) | +------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+ explain select * from t1 as TBL1, t2 as TBL2; explain select * from t1 as TBL1, (select * from t2 as TBL1) as TBL2;
  • 12.
    Ambiguous aliases create viewv1 as select * from t1 ... ; select /*+ NO_INDEX(t1 a) */ * from v1,t1 where ...; select /*+ JOIN_PREFIX(t1) */ * from v1,t1 where ... ; ● This will apply to t1 outside v1: ● This will apply to t1-inside-v1: ● Hint “resolution” has different code paths ● Causes counter-intuitive rules. – Avoid this for now
  • 13.
    Cost-based optimizations MySQL MariaDB [NO_]BKAYes Yes [NO_]BNL Yes Yes [NO_]INDEX_MERGE Yes In progress [NO_]MRR Yes Yes [NO_]SKIP_SCAN Yes - [NO_]ROWID_FILTER - In progress [NO_]SPLIT_MATERIALIZED - Yes ● NO_optname: disables use of the optimization ● Optname: Try using(*) the optimization, even if it’s more expensive (*)- or Encourage to use :-)
  • 14.
    Subquery optimizations ● Special hintsyntax due to multiple strategies ● Same meaning as cost-based. MySQL MariaDB SUBQUERY(strategy) Yes Yes [NO_]SEMIJOIN(strategies) Yes Yes /*+ [NO_]SEMIJOIN([query_block] sj_strategy,...) */ sj_strategy: FIRSTMATCH|LOOSESCAN|MATERIALIZATION|DUPSWEEDOUT /*+ [NO_]SUBQUERY([query_block] strategy,...) */ strategy: MATERIALIZATION|INTOEXISTS
  • 15.
    Does the Optimizertake hints? ● Hints are “Best effort”, “Dumb interpretation” TRAVEL /*+ USE_TRAM() */ FROM Helsinki TO Tallinn; TRAVEL /*+ HAVE_BEER_BEFORE_ARRIVAL() */ FROM Helsinki TO Tallinn;
  • 16.
    Non-cost-based optimizations ● NO_optname: disablethe optimization ● optname: enable if disabled – Not much point? MySQL MariaDB [NO_]MERGE Yes Yes [NO_]DERIVED_CONDITION_PUSHDOWN Yes Yes NO_ICP Yes Yes NO_RANGE_OPTIMIZATION Yes Yes
  • 17.
    Miscellaneous hints MariaDB alreadyhas ● ● Misc MySQL MariaDB SET_VAR Yes Yes MAX_EXECUTION_TIME Yes Yes RESOURCE_GROUP Yes No SET STATEMENT variable=value FOR ...; SET STATEMENT max_statement_time=... FOR ...; MariaDB doesn’t support resource groups.
  • 18.
    Query Block hint MySQLMariaDB QB_NAME Yes Yes
  • 19.
  • 20.
    Query block addressing ● QB_NAMEdeclares Query Block Name. ● Hints control behavior in different query blocks. with TOP_CUSTOMER as ( select /*+ QB_NAME(subq1) */ * from customer C order by C.balance desc limit 10 ) select /*+ INDEX(C@subq1 idx_balance) */ * from TOP_CUSTOMER TC, orders ORD where ... with TOP_CUSTOMER as ( select /*+ QB_NAME(subq1) */ * from customer C order by C.balance desc limit 10 ) select /*+ INDEX(C@subq1 idx_balance) */ * from TOP_CUSTOMER TC, orders ORD where ...
  • 21.
    Implicit Query Blocknames ● Implicit select#n name for every select with TOP_CUSTOMER as ( select * from customer C order by C.balance desc limit 10 ) select /*+ INDEX(C@`select#2` idx_balance) */ * from TOP_CUSTOMER TC, orders ORD where ...
  • 22.
    QB_NAME use cases ● Does’twork in MySQL: – Hints can’t control anything inside VIEWs – QB_NAME has global visibilty ● Conflicts? ● CTEs that are used multiple times? – Implicit names do not scale select /*+ hint1(arg1) hint2(arg2) hint(arg3, arg4) */ * Col1,col2,somef_function_col(3( from Tbl1 join tbl2 on t1b.col1=tbl2.cond2 left join tbl3 ... ● Can tune everything in the query from one location!
  • 23.
    TiDB’s way –a possible solution? ● QB_NAME assigns a name to Query Block elsewhere ● Uses “path” through views ● Uses SEL_n to find selects inside VIEWs. ● => We’re looking into this. create view v1 as select ... from t where ...; create view v2 as select ... from v1 where .... ; select /*+ QB_NAME(select1_in_v1, v2@SEL_1 .v1@SEL_1 .@SEL_1) NO_ORDER_INDEX(t@select1_in_v1, idx1) */ col1 from v2;
  • 24.
    From hints tosaved query plans?
  • 25.
    From Hints toQuery Plans ● Can we save the query plan as a set of hints? ● In Oracle DB this was “stored outlines” – Deprecated since Oracle 11g ● TiDB doesn’t have it documented, but TiDB> explain format='hint' select * from t10, t11 where t10.b=t11.a; +--------------------------------------------------------------------------------------------------------+ | hint | +--------------------------------------------------------------------------------------------------------+ | hash_join(@`sel_1` `test`.`t10`), use_index(@`sel_1` `test`.`t10` ), use_index(@`sel_1` `test`.`t11` ) | +--------------------------------------------------------------------------------------------------------+ ● Is this worth doing?
  • 26.
  • 27.
    Summary (1) ● MariaDB isgetting new-style hints. – Introduced in MariaDB 12.0 – Reasonaby complete set in MariaDB 12.2 ● Hints are compatible with MySQL – which were inspired by OracleDB’s ● MariaDB hints cover – MySQL’s hints, where applicable – MariaDB’s optimizations
  • 28.
    Summary (2) ● Query blockaddressing can be improved ● Coudl use hints for saving query plans in the future?
  • 29.
    Credits ● Hints in MariaDBare brought to you by – Oleg Smirnov – Alexander Barkov – Dave Gosselin – Lena Startseva – Sergei Petrunia
  • 30.