[FEA] Look into adding an AST Filter, turning on AST project by default, and partial AST projection???

[revans2]

Is your feature request related to a problem? Please describe.
Right now AST is really only used in joins. We added in an AST Project as a way to easily test AST versions of various expressions, but no one has really done any serious benchmark to find out how good it might be. Some recent experimental work has indicated that for some operations a filter that could do AST operations might be very highly performant compared to a non-AST version. We should take the time to explore what cases the AST operations are better than non-AST operations, and which cases they are not. If we see some big gains we should work with CUDF to add in AST filter support, and either turn on AST Project in cases where it would be best, or possibly update tiered project so that it could select some tiers to be executed using AST and not others.

[mattahrens]

We can start by experimenting with AST project config turned on.

[revans2]

As a note on why this might be a big win. #11810 needed to do a special case where multiple logical OR operators needed to be nested together. Using AST sped up the processing of this significantly. The AST or processing dropped the end to end time almost as much as the multi-contains work did.

If this works out we could eventually rewrite the complex rlike expression into a set of contains with OR expressions, which then we could automatically combine the into a multi-contains and an ast OR for the results. This would let us, in theory combine multiple complex rlike expressions or similar processing into a single multi-contains with some AST post processing.