feat: move the OpenAI Responses API into the dwctl edge (COR-519 + COR-536 + COR-522)

[JoshC8C7]

feat: move the OpenAI Responses API into the dwctl edge (COR-519 + COR-536 + COR-522)

Moves OpenAI Responses translation into the dwctl edge, placed so the whole
Responses lifecycle keeps working, and retires the dwctl-side machinery this makes
dead. onwards is left alone: nothing is removed there.

1. Translate Responses at the dwctl edge (COR-519)

/v1/responses is now handled by dwctl's edge translation framework (the same
shell as the Anthropic Messages ingress): the Responses request is translated to
Chat Completions, forwarded through the unchanged proxy path, and the reply
translated back into a Responses object.

The translated request still arrives at onwards on the /responses route (axum
cannot re-trigger route matching from a URI rewrite through a nest), so onwards
needs to recognise it - that is the one-line-ish dispatch in onwards
#274, already released.

2. Place the translation correctly (the important part)

The first cut put the translator at the OUTERMOST layer, copying the stateless
Anthropic pattern. That is wrong for Responses, which is a stored-object API with
an async control plane (background, flex, GET /v1/responses/{id}):

• GET 404'd: the outlet writes the row GET reads, but it sat inner to the
  translator, so on the response path it captured the chat body, and the id the
  client got (resp_<chat id>) never matched the row's tracking id.
• background stopped engaging: the path rewrite hid the flag from the router.

Fix: reorder to inference_mw -> outlet -> translation -> cache -> ... -> onwards.
The router now sees the raw /responses request (so background/service_tier
routing and previous_response_id hydration work), the outlet captures the
translated Responses object (so GET returns the right shape), and the translator
becomes a pure converter that stamps the tracking id. This restores exactly what
the outlet saw in the old onwards-side placement.

3. Retire the dwctl server-side tool loop, own ResponseStore (COR-536)

The server-side multi-step tool loop is retired (COR-517: unpublicised, unused).
Removed from dwctl: the loop dispatch, the engine loop-driver
(transition/assembly/loop_http_client/processor), the MultiStepStore
impl + loop storage helpers, the multi-step processor wiring, and
HttpToolExecutor. Kept: flex, background, GET, the storage substrate, and the
tool-injection primitives.

ResponseStore now lives in dwctl (inference/response_store.rs), which is its
only remaining user. onwards keeps its own copy of the trait; dwctl simply no
longer depends on it.

Server-side tool execution on /responses stops (the model's tool_calls go back
to the client); client-side tool calling, flex, background, and GET are unaffected.

4. Scrub client-supplied request ids at the edge (COR-522)

dwctl's inference_middleware now strips client-supplied
id/completion_id/completionId/response_id/responseId from the request
body, matching what onwards PR #240 does. Exact-key removal, so
previous_response_id and every other extension survive.

Note this is additive, not a relocation: onwards still performs its own scrub, so
the keys are now removed on both sides. That is harmless (idempotent removal of
the same five keys) and means the protection holds whichever path a request takes.

Review fixes

• The translation middleware now skips non-POST requests, so a GET/DELETE to a
  body-translating path falls through to real routing (and its 404/405) instead of
  being rejected by the body parse. The gate is per-translator, behind
  ProtocolTranslator::translates_request_body(): AnthropicModels deliberately
  claims GET /models only to normalise auth, and a blanket POST check silently
  401'd Anthropic model discovery. Both cases are pinned by tests.
• Corrected stale docs in translation/mod.rs (it described removed
  pre_request/post_response hooks and still called this the outermost layer).
• Removed the temporary [patch.crates-io]; onwards is a plain versioned
  dependency resolved from the registry.
• convert_tools still applies additionalProperties: false regardless of
  strict. That is a verbatim port of onwards' adapter; narrowing it would change
  the schema sent upstream for explicitly non-strict tools, so it is deliberately
  left alone and the misleading comment fixed instead.

Testing

• just lint rust clean (fmt, clippy, ZDR guard, SQLx).
• cargo test -p dwctl --lib: 1816 passed. The residual failures are sqlx
  test-harness DB contention under the parallel suite - the failing set differs
  between runs and every one passes in isolation (verified individually).
• test_responses_post_then_get_by_client_id: POST /ai/v1/responses then GET by
  the client's own id - verified it FAILS at the broken id (404) and PASSES with
  the placement fix.
• New translation-middleware tests: non_post_request_is_not_intercepted and
  get_is_still_translated_for_header_only_translator.

Known gaps

Flex + background POST-then-GET aren't covered end-to-end: they complete on the
fusillade daemon, which dispatches back through the dwctl loopback (.../ai), and
the in-memory axum_test transport has no real port for that loopback. The daemon
is exercised by sla.rs but only against a direct upstream endpoint. Covering the
/responses loopback needs a real-port e2e harness - logged as a follow-up; the
invariant it relies on is documented at the flex enqueue site.

Separately, previous_response_id hydration is broken by a pre-existing defect in
detail_to_response_object (it rebuilds a schema-invalid Responses object, so the
strict deserialize fails). That predates this PR - the same lossy path existed on
main - and is tracked as COR-549 rather than fixed here.

Cross-repo note

The onwards side is already merged and released: onwards #274 shipped in
v0.35.5, which is the version this PR depends on. No unreleased dependency and
no merge-ordering constraint. That change is small and additive - it teaches
onwards' /responses route to recognise an already-translated request and hand it
to the chat handler. Nothing was removed from onwards: the Open Responses adapter,
schemas, passthrough mode and the tool loop all remain.

Usage type unification + Anthropic billing fix

This part rides along because the Responses-edge move surfaced a broader
fragmentation in how responses were typed for logging and billing. The captured
response was parsed twice, through three different type systems: logging via
AiResponse on async-openai types, billing via a separate UsageExtractor ->
ProtocolUsage, and translation via the onwards chat schema plus dwctl's
responses/anthropic types. The most visible symptom was Anthropic: it had no
deserialize-able response type, so its usage degraded to AiResponse::Other and it
was billed zero tokens.

What this part does:

• One canonical response type per endpoint, shared by translation, request logging
  and billing. Logging and billing now derive from a single parse_ai_response -> AiResponse (billing via TokenMetrics::from) instead of two divergent parses.
• Ownership follows who produces the captured body (the outlet sits outside the
  translation layer): responses and anthropic bodies are produced by dwctl's own
  edge translators, so they parse back with dwctl's own types (responses/types.rs,
  anthropic/model.rs, the latter now Deserialize). chat/completions/embeddings
  stay on async-openai - they are native passthrough, and async-openai keeps typed
  reasoning tokens and tolerates raw provider bodies that strict schemas would
  reject.
• Removes the parallel usage layer entirely: the UsageExtractor trait,
  ProtocolUsage, UsageRegistry, the per-translator extract_usage impls, and
  NativeOpenAiUsage.

Anthropic is now charged correctly: /v1/messages responses parse into
AiResponse::Anthropic (blocking) or AiResponse::AnthropicStream (streaming), and
TokenMetrics::from charges input_tokens + output_tokens (Anthropic reports no
total and folds thinking into output). Covered by new unit tests: blocking usage,
streaming usage, stream-error reclassification to 500, and the /messages parse.

Caching is unaffected. The cache-token split (cache_read_input_tokens,
cache_creation.*) is still read from the raw response body by
extract_cache_tokens, independent of the typed parse, so it behaves identically.

Onwards stops manipulating request bodies

Continues the "consolidate manipulation into dwctl" theme. onwards' strict handlers
used to deserialise each request into a typed schema and re-serialise it before
forwarding (which reordered keys and silently dropped unknown nested fields), and
onwards owned the caller id-scrub. Two things it did to the request body have moved:

• The chat/completions, completions and embeddings handlers now validate the
  shape (deserialise only to confirm it parses) and forward the ORIGINAL request
  bytes untouched. Malformed-request statuses are preserved via a shared parse
  helper (422 for a schema-invalid body, 400 for bad JSON).
• The caller id-scrub (id, completion_id, response_id, ...) and the streaming
  usage-flag injection (stream_options.include_usage, x-fusillade-stream
  force-stream) move into a new dwctl outbound_request middleware, wired innermost
  on the onwards router (inner to the cache layer, which must hash the original
  body). This retires dwctl's stream_usage_transform and the onwards
  BodyTransformFn wiring.

The /responses background strip the old transform did is dropped: by the time a
request reaches this layer the translation middleware has already flattened
/responses to a chat body and background was consumed by the (outer) inference
middleware, so it never arrives.

Behaviour is preserved end-to-end - the id-scrub and stream flags happen one layer
earlier, in dwctl - but onwards no longer re-serialises or drops fields from edge
request bodies. (The additionalProperties injection for Responses tool schemas was
already migrated to dwctl's convert_tools.)

Not in this change (tracked as a separate follow-up): moving fusillade's
convert-to-streaming + reassemble into a dwctl response-side reframer, which also
retires fusillade's transitional ZDR ResponseTransformer hook.

[cloudflare-workers-and-pages]

Deploying control-layer with    Cloudflare Pages

Latest commit:	a7a0e15
Status:	✅  Deploy successful!
Preview URL:	https://cf61862c.control-layer.pages.dev
Branch Preview URL:	https://preview-responses-edge-in-dw.control-layer.pages.dev

View logs

[Copilot]

Pull request overview

This PR fixes OpenAI Responses API edge-translation behavior in dwctl by reordering middleware so the inference middleware and outlet see the raw /responses request and persist the translated Responses-shaped object, restoring correct GET /v1/responses/{id} retrieval and background handling.

Changes:

• Reorders the onwards middleware stack to inference_mw → outlet → translation → … → onwards, so control-plane fields are handled before translation and persisted responses match client IDs.
• Ports Responses request/response/streaming translation logic into dwctl/src/inference/translation/responses/* and threads the fusillade tracking ID through translation.
• Adds an end-to-end POST-then-GET test to ensure the client-facing Responses ID resolves via GET.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
dwctl/src/lib.rs	Reorders middleware layering and wires in OpenResponses translator in strict mode.
dwctl/src/inference/middleware.rs	Moves previous_response_id hydration into inference middleware and expands intercept paths to include /messages.
dwctl/src/inference/translation/mod.rs	Updates translation trait signatures to accept original request bytes + tracking ID (docs need correction).
dwctl/src/inference/translation/middleware.rs	Captures original request bytes and x-fusillade-request-id for response/stream translation.
dwctl/src/inference/translation/responses/mod.rs	Adds OpenResponses translator (detect should be header-gated).
dwctl/src/inference/translation/responses/request.rs	Converts Responses requests into Chat Completions requests (pure conversion).
dwctl/src/inference/translation/responses/response.rs	Converts Chat Completions responses into Responses objects and stamps tracking IDs.
dwctl/src/inference/translation/responses/streaming.rs	Reframes Chat Completions SSE into Responses streaming events.
dwctl/src/inference/translation/responses/types.rs	Adds Open Responses schema types copied from onwards (strict schemas + helpers).
dwctl/src/inference/translation/responses/util.rs	Adds shared utility helpers (usage mapping, reasoning merge, scrub ID fields).
dwctl/src/inference/translation/responses/hydrate.rs	Adds previous_response_id hydration helper using ResponseStore.
dwctl/src/inference/translation/anthropic/mod.rs	Adapts Anthropic translator to widened trait signatures.
dwctl/src/inference/translation/anthropic/models.rs	Adapts Anthropic models translator to widened trait signatures.
dwctl/src/test/responses.rs	Adds POST-then-GET integration test for client-facing Responses ID resolution.
Cargo.toml	Adds a local [patch.crates-io] override for onwards (must not ship).
Cargo.lock	Reflects the local onwards path patch (missing registry source/checksum).

[JoshC8C7]

Fixed. That paragraph described a design that got replaced: the pre_request/post_response hooks were removed when the layer was repositioned, and the trait is now pure and synchronous.

The module doc now states that explicitly and says where the stateful work actually lives - control-plane routing, tracking-id minting and previous_response_id hydration in the inference middleware, persistence in the outlet, both outer to this layer - and why the translator staying pure is what allows it to sit inside them.

Also corrected the opening line in the same file, which still claimed this middleware is the OUTERMOST Tower layer. It is not: it sits inside the inference middleware and the outlet.

[Copilot]

Pull request overview

Copilot reviewed 31 out of 32 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

Cargo.toml:9

• The workspace uses a [patch.crates-io] onwards = { path = "../onwards" } override, but the parent directory in this repo checkout does not contain an onwards/ crate. This will break CI/builds for anyone who doesn't have a sibling repo checked out locally. Prefer landing this PR with a normal crates.io dependency (or keep the patch in a local-only workflow, not committed).

# TEMPORARY (COR-519): test the local onwards edge-Responses changes together
# with dwctl before either PR lands. Remove before opening the dwctl PR; the
# onwards change ships as its own release and dwctl bumps the version dep.
[patch.crates-io]
onwards = { path = "../onwards" }

dwctl/src/inference/translation/mod.rs:33

• This module-level doc comment says translators can opt into async/stateful work via ProtocolTranslator::pre_request / post_response hooks, but the ProtocolTranslator trait in this file doesn't define those methods, and the Responses control-plane work is implemented outside the translator (inference middleware + outlet). The comment should be updated to reflect the actual API and current implementation.

//! Anthropic Messages (`/v1/messages`) is the first implementation and is purely
//! synchronous. The core transform stays sync and pure; a translator that needs
//! async, stateful work opts in via the [`ProtocolTranslator::pre_request`] /
//! [`ProtocolTranslator::post_response`] hooks, which the middleware awaits
//! around the pure translation. These brackets are how the planned OpenAI

[JoshC8C7]

Correct that the code and the comment disagreed, but I have deliberately kept the behaviour and fixed the comment instead.

This function is a verbatim port of onwards' OpenResponsesAdapter::convert_tools (adapter.rs:684-723) - the unconditional insert and the misleading "in strict mode" comment both came across as-is. Narrowing it to strict == true here would make the dwctl edge send a different tool schema than the onwards adapter does today for explicitly non-strict tools, which is a real behaviour change for anyone relying on the current shape. The point of this PR is to move Responses translation without changing what goes upstream, so I would rather not smuggle that in.

Scope note: strict defaults to true in the Responses schema (default_strict), so this only differs when a caller explicitly sends "strict": false.

The comment now says what the code actually does and why it is left alone. Happy to change the behaviour as a separate, visible change if you think the current shape is wrong.

[Copilot]

Pull request overview

Copilot reviewed 43 out of 43 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (3)

dwctl/src/lib.rs:1642

• This comment says the outbound_request layer “does the id-scrub”, but outbound_request_middleware only injects streaming usage flags (it explicitly does NOT scrub IDs). The id-scrub now happens in the inference middleware, so this is misleading for future maintainers reading the router stack.

    // Last-mile request-body prep, applied innermost so it runs right before onwards
    // (inner to cache, which must hash the original body). Does the id-scrub and the
    // streaming usage-flag injection that onwards / the BodyTransformFn hook used to
    // do, so onwards can forward the body untouched.

dwctl/src/inference/response_store.rs:60

• NoOpResponseStore is documented as returning a generated id, but it currently always returns the constant string "noop". Even in tests this can cause accidental collisions if the id is used as a key more than once. Either adjust the docs or generate a unique id per call.

/// No-op store: returns a generated id and never resolves context. Used where a
/// store is structurally required but no persistence is wanted (e.g. tests).
#[derive(Debug, Clone, Default)]
pub struct NoOpResponseStore;

dwctl/src/inference/translation/responses/util.rs:11

• The module docs mention scrub_request_id_fields_from_extra, but this helper is not actually defined in this file. The follow-up comment also references it. This is misleading (it looks like a missing function or incomplete copy) and should be corrected.

//! `ensure_field` / `scrub_request_id_fields_from_extra` are copied from onwards'
//! `strict::schemas::utils`; `merge_reasoning_text` / `chat_usage_to_response_usage`
//! from onwards' `strict` module (both `pub(crate)` there, so not importable).
//! They are duplicated here as part of moving Responses ownership into dwctl; the
//! onwards copies retire with the rest of its Responses code (COR-536).