Lambda/var expression to impl HoF #7334
joseph-isaacs
started this conversation in
Feature Requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
We want to introduce lambda and variable expressions to support higher-order functions in the Vortex expression tree.
Static Lambda Expressions
Vortex expressions today form a tree of scalar function nodes, where Root references the input array and every other node computes a value from its children. There is no way to name an intermediate result or express a parameterized subcomputation within the expression tree. Lambda and Variable expressions introduce these capabilities: lambda("x", checked_mul(var("x"), var("x"))) defines a function over x, and var("x") references the bound parameter.
A Lambda is a scalar function with a FieldName option (the parameter name) and exactly one child (the body). A Variable is a zero-arity leaf node with options { name: FieldName, dtype: DType } that references a named parameter. Both are non-serializable and non-executable — they are purely structural constructs that must be eliminated before reaching the execution layer.
Elimination happens statically in ArrayRef::apply. Before the existing Root/Literal/general logic runs, a top-down tree rewrite removes all lambda and variable nodes by substituting variable references with the appropriate bound expressions. This produces a plain lambda-free expression tree that the rest of apply handles normally. Lambda::simplify_untyped performs the same substitution for expression-level optimization passes that operate before arrays exist.
Execution
Lambda and Variable expressions cannot be executed directly today — execute will bail if either survives to the execution layer. In the future, we could support execution by threading a scope map (HashMap<FieldName, ArrayRef>) through apply, allowing a lambda to evaluate its bound value once and resolve variable references to the precomputed array.
Beta Was this translation helpful? Give feedback.
All reactions