MDEV-36092 New-style hint: [NO_]SPLIT_MATERIALIZED

[DaveGosselin-MariaDB]

Support for optimizer hints NO_SPLIT_MATERIALIZED and SPLIT_MATERIALIZED. These hints allow fine-grained control of the "lateral derived" optimization within a query.

[CLAassistant]

All committers have signed the CLA.

[spetrunia]

@DaveGosselin-MariaDB , example where hint doesn't force it:

create table ten(a int);
insert into ten select seq from seq_1_to_10;

create table one_k(a int);
insert into one_k select seq from seq_1_to_1000;

CREATE TABLE `t1000` (
  `grp` int(11) DEFAULT NULL,
  `val` int(11) DEFAULT NULL,
  KEY `grp` (`grp`)
);

insert into t1000 select A.seq, B.seq from seq_1_to_100 A, seq_1_to_10 B;
analyze table t1000;

This works:

explain 
select /*+ SPLIT_MATERIALIZED(TBL) */ * 
from 
  ten T1, (select grp, count(*) from t1000 group by grp) TBL where TBL.grp=T1.a;

+------+-----------------+------------+------+---------------+------+---------+---------+------+-------------+
| id   | select_type     | table      | type | possible_keys | key  | key_len | ref     | rows | Extra       |
+------+-----------------+------------+------+---------------+------+---------+---------+------+-------------+
|    1 | PRIMARY         | T1         | ALL  | NULL          | NULL | NULL    | NULL    | 10   | Using where |
|    1 | PRIMARY         | <derived2> | ref  | key0          | key0 | 5       | j4.T1.a | 1    |             |
|    2 | LATERAL DERIVED | t1000      | ref  | grp           | grp  | 5       | j4.T1.a | 10   | Using index |
+------+-----------------+------------+------+---------------+------+---------+---------+------+-------------+

This doesn't (the query is the same except table ten is changed to one_k)

explain 
select /*+ SPLIT_MATERIALIZED(TBL) */ * 
from 
  one_k T1, (select grp, count(*) from t1000 group by grp) TBL where TBL.grp=T1.a;
+------+-------------+------------+-------+---------------+------+---------+---------+------+----------------------------------------------+
| id   | select_type | table      | type  | possible_keys | key  | key_len | ref     | rows | Extra                                        |
+------+-------------+------------+-------+---------------+------+---------+---------+------+----------------------------------------------+
|    1 | PRIMARY     | T1         | ALL   | NULL          | NULL | NULL    | NULL    | 1000 | Using where                                  |
|    1 | PRIMARY     | <derived2> | ref   | key0          | key0 | 5       | j4.T1.a | 10   |                                              |
|    2 | DERIVED     | t1000      | index | grp           | grp  | 5       | NULL    | 1000 | Using index; Using temporary; Using filesort |
+------+-------------+------------+-------+---------------+------+---------+---------+------+----------------------------------------------+

[DaveGosselin-MariaDB]

@DaveGosselin-MariaDB , example where hint doesn't force it:

Fixed in latest patch, thank you for this great example!

[spetrunia]

Ok, retrying the example above on the latest patch and indeed one can use a hint to force split-materialized...

analyze format=json select  * from one_k T1, (select grp, count(*) from t1000 group by grp) TBL where TBL.grp=T1.a\G
analyze format=json select /*+ SPLIT_MATERIALIZED(TBL) */ * from one_k T1, (select grp, count(*) from t1000 group by grp) TBL where TBL.grp=T1.a\G

It's interesting that the cost becomes lower:

• without hint: query_block.cost= 2.047789395
• with hint: query_block.cost= 0.4491132

Looks like two things are wrong: if the cost is lower with split-materialized why didn't we pick it by default, second, why is the cost lower if we're going to access all GROUP BY groups?

Also filed https://jira.mariadb.org/browse/MDEV-36913.
Well, all of this seems to be outside of the scope of this MDEV.

[spetrunia]

Please check and apply this patch https://gist.github.com/spetrunia/efdf4394e386a367e441a75d5c27ea22

it does this:

• print "forced_by_hint" only if it's true (which is rare)
• move hint_table_state() declaration to be together with others.
• fix coding style in Table_levelhint_table_state_hint::resolve and
• make is_split_materialized_allowed use pointer for OUT parameter

Ok to push after this is done