[SPARK-52497][DOCS] Add docs for SQL user-defined functions #51281
Conversation
|
|
||
| When `TEMPORARY` is specified, the function is only available for the current session. Otherwise, it is persisted in the catalog and available across sessions. The `OR REPLACE` option allows updating an existing function definition, while `IF NOT EXISTS` prevents errors when creating a function that already exists. | ||
|
|
||
| The function parameters must be specified with their data types. The return type can be either a scalar data type or a table with an optional schema definition. |
There was a problem hiding this comment.
oh, the return table schema definition is optional?
There was a problem hiding this comment.
actually, the entire RETURN clause is optional, right?
There was a problem hiding this comment.
The RETURNS is optional, RETURN is not
There was a problem hiding this comment.
Yes the RETURNS clause is optional for scalar UDFs, and the RETURNS TABLE schema is optional for TVFs.
There was a problem hiding this comment.
So it should be The return type can be either a scalar data type or a table with an schema definition. If not specified the return type will be inferred from the function body?
|
|
||
| ### Syntax | ||
|
|
||
| ```sql |
There was a problem hiding this comment.
I know in Spark we use sql, but it makes no sense to do that for the syntax.. I wish there were a BNF ....
Co-authored-by: Wenchen Fan <[email protected]>
|
LGTM! |
What changes were proposed in this pull request?
This PR adds docs for SQL UDFs.
Why are the changes needed?
Add documentation for a new Spark 4 feature.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Manually verify the documentation build
Was this patch authored or co-authored using generative AI tooling?
No