Skip to content

feat(plugin): document streaming function frames in OpenAPI [ASTD-350] - #1024

Draft
marcusds wants to merge 1 commit into
mainfrom
astd-350-streaming-function-response-schema/mschwab
Draft

feat(plugin): document streaming function frames in OpenAPI [ASTD-350]#1024
marcusds wants to merge 1 commit into
mainfrom
astd-350-streaming-function-response-schema/mschwab

Conversation

@marcusds

Copy link
Copy Markdown
Contributor

Streaming function routes were registered with response_model=None, so FastAPI emitted an empty 200 schema under the wrong media type:

responses:
  '200':
    content:
      application/json:   # the route returns application/x-ndjson
        schema: {}

Generated clients therefore returned unknown, leaving every consumer to re-declare the frame union and its type guards by hand. ASTD-332 hand-rolled ~60 lines of exactly that. Same class of bug as ASTD-349 — the spec not describing what the service actually returns.

Closes ASTD-350.

What changed

A NemoFunction can now declare frame_schema — the frame type it yields — and the route factory turns that into an application/x-ndjson response. The anonymizer preview function declares its union:

frame_schema: ClassVar[Any] = PreviewFrame

The spec's 200 becomes the full kind-discriminated union, and the generated TypeScript client's return type goes from unknown to:

LogFrame | PreviewDatasetFrame | TraceDatasetFrame | FailedRecordsFrame | Heartbeat | Done | Error

with all seven emitted as SDK schemas.

Two FastAPI details worth knowing

Both cost me an experiment, so they're documented in the code for the next streaming function.

NdjsonFrameResponse subclasses JSONResponse despite never being instantiated. fastapi/openapi/utils.py:393 hardcodes the 200 schema to {"type": "string"} for any response class that isn't a JSONResponse subclass, silently discarding the declared model. Subclassing it is what gets the union documented, while the overridden media_type still files it under application/x-ndjson. Nothing is ever constructed from it — streaming handlers build their own StreamingResponse, and FastAPI returns a Response untouched.

The schema rides on response_model, not responses[200]["model"]. The latter deep-merges onto that same {"type": "string"} default, yielding a schema that claims to be both a string and a union. Runtime is unaffected either way: FastAPI skips response validation when a handler returns a Response, which the streaming branch always does.

Blast radius

Opt-in. Functions that declare no frame_schema keep their exact previous registration. Regenerating all nine plugin specs changed only the anonymizer's — data designer's streaming preview is untouched until it opts in.

web/packages/sdk/generated/ is gitignored, so the SDK changes here are reproduced by pnpm --filter @nemo/sdk gen:all-force rather than committed. The Python SDK regen via Stainless is a separate step and isn't included.

Testing

  • 5 new route tests: the media type, the union and its discriminator, frame models landing in components, functions without a frame_schema being left alone, and streamed output being unchanged
  • 1244 Python tests pass across nemo_platform_plugin, anonymizer and data designer
  • ty and ruff clean; Studio typechecks against the regenerated SDK

Follow-on

Once this lands, #1005 can drop its hand-written PreviewFrame union and guards from previewApi.ts and import the generated types. The streaming reader stays hand-written either way — orval doesn't generate incremental readers.

Streaming function routes registered with `response_model=None`, so FastAPI
emitted an empty 200 schema under `application/json` — untyped, and the wrong
media type for a route that returns `application/x-ndjson`. Generated clients
got back `unknown`, leaving every consumer to re-declare the frame union and
its type guards by hand.

A `NemoFunction` can now declare `frame_schema`, the frame type it yields, and
the route factory turns that into an `application/x-ndjson` response. The
anonymizer `preview` function declares its union, so the SDK now generates
`LogFrame`, `PreviewDatasetFrame`, `TraceDatasetFrame`, `FailedRecordsFrame`,
`Heartbeat`, `Done` and `Error`, and the client returns their discriminated
union instead of `unknown`.

Two details worth knowing for the next streaming function:

`NdjsonFrameResponse` subclasses `JSONResponse` despite never being
instantiated. FastAPI hardcodes the 200 schema to `{"type": "string"}` for any
response class that isn't a `JSONResponse` subclass, which silently discards
the declared model — subclassing it is what gets the union documented while
`media_type` still files it under `application/x-ndjson`.

The schema rides on `response_model` rather than `responses[200]["model"]`.
The latter deep-merges onto that same `{"type": "string"}` default, leaving a
schema that claims to be both a string and a union. Runtime is unchanged
either way: FastAPI skips response validation when a handler returns a
`Response`, which the streaming branch always does.

Functions that declare no `frame_schema` keep their existing registration, so
this is inert for every other function today.

Signed-off-by: mschwab <mschwab@nvidia.com>
@github-actions github-actions Bot added the feat label Jul 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 29446/37454 78.6% 63.2%
Integration Tests 17384/36172 48.1% 20.5%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant