feat(plugin): document streaming function frames in OpenAPI [ASTD-350] - #1024
Draft
marcusds wants to merge 1 commit into
Draft
feat(plugin): document streaming function frames in OpenAPI [ASTD-350]#1024marcusds wants to merge 1 commit into
marcusds wants to merge 1 commit into
Conversation
Streaming function routes registered with `response_model=None`, so FastAPI
emitted an empty 200 schema under `application/json` — untyped, and the wrong
media type for a route that returns `application/x-ndjson`. Generated clients
got back `unknown`, leaving every consumer to re-declare the frame union and
its type guards by hand.
A `NemoFunction` can now declare `frame_schema`, the frame type it yields, and
the route factory turns that into an `application/x-ndjson` response. The
anonymizer `preview` function declares its union, so the SDK now generates
`LogFrame`, `PreviewDatasetFrame`, `TraceDatasetFrame`, `FailedRecordsFrame`,
`Heartbeat`, `Done` and `Error`, and the client returns their discriminated
union instead of `unknown`.
Two details worth knowing for the next streaming function:
`NdjsonFrameResponse` subclasses `JSONResponse` despite never being
instantiated. FastAPI hardcodes the 200 schema to `{"type": "string"}` for any
response class that isn't a `JSONResponse` subclass, which silently discards
the declared model — subclassing it is what gets the union documented while
`media_type` still files it under `application/x-ndjson`.
The schema rides on `response_model` rather than `responses[200]["model"]`.
The latter deep-merges onto that same `{"type": "string"}` default, leaving a
schema that claims to be both a string and a union. Runtime is unchanged
either way: FastAPI skips response validation when a handler returns a
`Response`, which the streaming branch always does.
Functions that declare no `frame_schema` keep their existing registration, so
this is inert for every other function today.
Signed-off-by: mschwab <mschwab@nvidia.com>
Contributor
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Streaming function routes were registered with
response_model=None, so FastAPI emitted an empty 200 schema under the wrong media type:Generated clients therefore returned
unknown, leaving every consumer to re-declare the frame union and its type guards by hand. ASTD-332 hand-rolled ~60 lines of exactly that. Same class of bug as ASTD-349 — the spec not describing what the service actually returns.Closes ASTD-350.
What changed
A
NemoFunctioncan now declareframe_schema— the frame type it yields — and the route factory turns that into anapplication/x-ndjsonresponse. The anonymizerpreviewfunction declares its union:The spec's 200 becomes the full
kind-discriminated union, and the generated TypeScript client's return type goes fromunknownto:with all seven emitted as SDK schemas.
Two FastAPI details worth knowing
Both cost me an experiment, so they're documented in the code for the next streaming function.
NdjsonFrameResponsesubclassesJSONResponsedespite never being instantiated.fastapi/openapi/utils.py:393hardcodes the 200 schema to{"type": "string"}for any response class that isn't aJSONResponsesubclass, silently discarding the declared model. Subclassing it is what gets the union documented, while the overriddenmedia_typestill files it underapplication/x-ndjson. Nothing is ever constructed from it — streaming handlers build their ownStreamingResponse, and FastAPI returns aResponseuntouched.The schema rides on
response_model, notresponses[200]["model"]. The latter deep-merges onto that same{"type": "string"}default, yielding a schema that claims to be both a string and a union. Runtime is unaffected either way: FastAPI skips response validation when a handler returns aResponse, which the streaming branch always does.Blast radius
Opt-in. Functions that declare no
frame_schemakeep their exact previous registration. Regenerating all nine plugin specs changed only the anonymizer's — data designer's streaming preview is untouched until it opts in.web/packages/sdk/generated/is gitignored, so the SDK changes here are reproduced bypnpm --filter @nemo/sdk gen:all-forcerather than committed. The Python SDK regen via Stainless is a separate step and isn't included.Testing
components, functions without aframe_schemabeing left alone, and streamed output being unchangednemo_platform_plugin, anonymizer and data designertyandruffclean; Studio typechecks against the regenerated SDKFollow-on
Once this lands, #1005 can drop its hand-written
PreviewFrameunion and guards frompreviewApi.tsand import the generated types. The streaming reader stays hand-written either way — orval doesn't generate incremental readers.