feat(workflow-operator): add Python UDF UI parameter injection model by carloea2 · Pull Request #5141 · apache/texera

carloea2 · 2026-05-21T06:42:07Z

What changes were proposed in this PR?

This PR adds the Scala backend foundation for Python UDF UI parameters.

It introduces:

Area	Change
UI parameter model	Adds `UiUDFParameter`, containing backend-compatible `attribute` metadata and an editable `value`.
Python UDF injector	Adds `PythonUdfUiParameterInjector`, which validates UI parameters and injects a reserved hook method into supported Python UDF classes.
Safe string encoding	Marks `Attribute.getName()` as encodable so UI parameter names are safely rendered through the Python template builder.
Test coverage	Adds Scala tests for hook injection, validation, unsupported types, reserved method conflicts, and unchanged behavior when no UI parameters exist.

This PR is stacked after the merged frontend foundation PR #5043. It does not yet wire the injector into operator execution; that wiring is handled by later PRs in the stack.

Existing Python UDF workflow execution remains unchanged in this PR because PythonUDFOpDescV2, PythonUDFSourceOpDescV2, and DualInputPortsPythonUDFOpDescV2 are not modified here.

Any related issues, documentation, discussions?

Part of the Python UDF UI parameter feature split from feat/ui-parameter.

Related tracking issue / stack: #5044

Stack order:

Frontend UI parameter building blocks: feat(frontend): add Python UDF UI parameter form support #5043
Scala backend injection model: this PR
Python runtime support
End-to-end integration

How was this PR tested?

Commands run:

sbt "WorkflowOperator / Test / testOnly org.apache.texera.amber.operator.udf.python.PythonUdfUiParameterInjectorSpec"
sbt scalafmtAll
sbt scalafmtCheckAll "scalafixAll --check"

Results:

PythonUdfUiParameterInjectorSpec: 10 tests passed.
scalafmtAll: no file changes.
scalafmtCheckAll and scalafixAll --check: passed.

Was this PR authored or co-authored using generative AI tooling?

No

codecov-commenter · 2026-05-21T06:44:38Z

Codecov Report

❌ Patch coverage is 90.36145% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.66%. Comparing base (d8c254c) to head (75fe048).

Files with missing lines	Patch %	Lines
...ator/udf/python/PythonUdfUiParameterInjector.scala	90.00%	0 Missing and 8 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #5141      +/-   ##
============================================
- Coverage     48.95%   46.66%   -2.30%     
+ Complexity     2377     2376       -1     
============================================
  Files          1048     1048              
  Lines         40270    40098     -172     
  Branches       4272     4267       -5     
============================================
- Hits          19714    18711    -1003     
- Misses        19402    20256     +854     
+ Partials       1154     1131      -23

Flag	Coverage Δ		*Carryforward flag
access-control-service	`39.53% <ø> (ø)`
agent-service	`33.74% <ø> (-0.03%)`	⬇️	Carriedforward from 0a42fcd
amber	`51.74% <90.36%> (+0.17%)`	⬆️
computing-unit-managing-service	`0.00% <ø> (ø)`
config-service	`0.00% <ø> (ø)`
file-service	`37.99% <ø> (ø)`
frontend	`35.21% <ø> (-5.44%)`	⬇️	Carriedforward from 0a42fcd
python	`90.50% <ø> (-0.30%)`	⬇️	Carriedforward from 0a42fcd
workflow-compiling-service	`56.81% <ø> (ø)`

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

carloea2 · 2026-05-24T23:18:35Z

@mengw15 can you review this PR? Thanks

Xiao-zhen-Liu

A few correctness notes from me — I've left them as inline comments on the relevant lines below. They're mostly about the generated Python surviving the handoff to the runtime PR; the happy path itself looks right.

Xiao-zhen-Liu · 2026-05-27T18:50:19Z

+
+  private val InjectedUiParametersHookMethodName = "_texera_injected_ui_parameters"
+  private val InjectedUiParametersHookMethodHeader =
+    s"def $InjectedUiParametersHookMethodName(self) -> Dict[str, Any]:"


Dict and Any aren't actually exported by from pytexera import * — its __all__ only re-exports Iterator, Optional, and Union from typing. So in the user's module these names are undefined, and since we run Python 3.10–3.13 (where return annotations are evaluated at definition time), the class raises NameError: name 'Dict' is not defined the moment it loads. The test that asserts the output shouldn't contain import typing quietly locks this assumption in, so it'll stay green while generating code that can't run. We could add Dict/Any to pytexera's __all__, inject a from typing import Dict, Any, or just drop the return annotation — whichever we pick, it'd be good to settle it before the runtime PR starts depending on it.

This is covered in next PR actually but I did not include it here since this is for scala changes. Next PR will be about Python side.

Xiao-zhen-Liu · 2026-05-27T18:50:19Z

+  private def buildInjectedHookMethod(uiParameters: List[UiUDFParameter]): String = {
+    val injectedParametersMap = buildInjectedParametersMap(uiParameters)
+
+    (pyb"""|@overrides


There's no _texera_injected_ui_parameters in any base class yet, so @overrides will fail at class-definition time — it checks that the method really overrides a parent, and its default signature check calls get_type_hints, which also runs into the Dict/Any issue above. That's a perfectly reasonable thing to leave for the runtime PR; I'd just call it out in the description (or in a comment right here) so it's clear the generated code isn't runnable on its own yet.

Xiao-zhen-Liu · 2026-05-27T18:50:19Z

+    if (lineEnd < 0) text.length else lineEnd
+  }
+
+  private def detectClassBlockEnd(code: String, classHeaderStart: Int, classIndent: String): Int = {


This decides the class body ends at the first non-blank line indented at or below the class header. That's right for ordinary code, but it doesn't account for string literals, so a triple-quoted block inside a method whose contents start at column 0 looks like the end of the class — and we'd splice the hook into the middle of the string:

class ProcessTupleOperator(UDFOperatorV2): def process_tuple(self, tuple_, port): sql = """ SELECT * FROM t """ yield tuple_

pybuilder already has PythonLexerUtils/BoundaryValidator for exactly this kind of lexical case, so it might be worth reusing them instead of scanning by indentation. If that's more than you want to take on here, a test that documents the limitation would at least make it a known boundary.

I can take a look about this one, thanks

Xiao-zhen-Liu · 2026-05-27T18:50:19Z

+  private def injectHookIntoUserClass(encodedUserCode: String, hookMethod: String): String = {
+    val classHeaderMatch =
+      SupportedPythonUdfClassHeaderRegex.findFirstMatchIn(encodedUserCode).getOrElse {
+        return encodedUserCode


When the code doesn't contain one of the four supported class names, we hand it back unchanged even if there are parameters to inject. So if a user renames their class, their configured parameters just disappear at runtime with no signal. Since validate already raises for bad input, it might be friendlier to raise here too when uiParameters is non-empty but nothing matched — otherwise it's a confusing "my parameters did nothing" situation.

Got it, I will update it

Xiao-zhen-Liu · 2026-05-27T18:50:19Z

+    val parameters = Option(uiParameters).getOrElse(List.empty)
+    validate(parameters)
+
+    val encodedUserCode = pyb"$code".encode


Minor: running the whole program through pyb"$code".encode also sends it through stripMargin (the builder's render calls it), so any user line that's whitespace followed by | would get rewritten — even on the no-parameter passthrough path. It's an unlikely shape in a real UDF, but since this is the first place arbitrary user source flows through pyb, it's worth being aware of.

I'll take care of it.

carloea2 · 2026-05-27T19:19:16Z

Thanks @Xiao-zhen-Liu, I will ping you when it is ready for the next review.

feat(workflow-operator): add Python UDF UI parameter injection model

4ec7c7a

github-actions Bot added the common label May 21, 2026

github-actions Bot assigned carloea2 May 21, 2026

carloea2 added 2 commits May 21, 2026 02:58

refactor(workflow-operator): tighten UI parameter injector

f386522

fix(workflow-operator): restore attribute name encoding

4652aac

Merge branch 'main' into ui-parameter-backend-scala

962a52c

carloea2 mentioned this pull request May 24, 2026

Add Python UDF UI parameter support #5044

Open

6 tasks

Merge branch 'main' into ui-parameter-backend-scala

b18d708

mengw15 requested a review from Xiao-zhen-Liu May 26, 2026 03:36

Xiao-zhen-Liu added 2 commits May 26, 2026 12:16

Merge branch 'main' into ui-parameter-backend-scala

16373a4

Merge branch 'main' into ui-parameter-backend-scala

75fe048

Xiao-zhen-Liu reviewed May 27, 2026

View reviewed changes

Conversation

carloea2 commented May 21, 2026

What changes were proposed in this PR?

Any related issues, documentation, discussions?

How was this PR tested?

Was this PR authored or co-authored using generative AI tooling?

Uh oh!

codecov-commenter commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

carloea2 commented May 24, 2026

Uh oh!

Xiao-zhen-Liu left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

carloea2 commented May 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov-commenter commented May 21, 2026 •

edited

Loading

Xiao-zhen-Liu left a comment •

edited

Loading