Preview/responses edge in dwctl by JoshC8C7 · Pull Request #1418 · doublewordai/control-layer

JoshC8C7 · 2026-08-03T11:42:01Z

No description provided.

… from onwards yet.

Move the Responses edge translation from the outermost layer to inner of the outlet, and make the inference middleware outermost, so GET /v1/responses/{id} and background mode work. The outlet now persists the translated Responses object (what GET reads); the control plane (id minting, previous_response_id hydration, background/flex routing) owns the Responses lifecycle in the inference middleware; and the translator becomes a pure converter that stamps the platform tracking id so client and stored ids match. Also adds /messages to should_intercept and a POST-then-GET test for the /responses path.

…wctl (COR-536) With Responses translation placed correctly at the dwctl edge, onwards' Responses adapter is dead and the server-side multi-step tool loop is retired (COR-517). Removes the warm-path loop dispatch, the engine loop-driver, the MultiStepStore impl + loop storage helpers, the multi-step processor wiring, and HttpToolExecutor - keeping flex/background/GET, the storage substrate, and the tool-injection primitives. Relocates ResponseStore/NoOpResponseStore/StoreError from onwards into dwctl (its only remaining user). Pairs with the onwards deletion PR.

dwctl's inference_middleware now strips client-supplied id/completion_id/response_id keys from the request body before it re-serialises, so the scrub (onwards PR #240) happens where dwctl owns the single parse-and-shape. Exact-key removal preserves previous_response_id and every other extension. Removes the dead ported scrub from responses/util.rs + types.rs. Lets onwards forward the bytes verbatim.

# Conflicts: # Cargo.lock # dwctl/Cargo.toml # dwctl/src/inference/middleware.rs

# Conflicts: # Cargo.lock # dwctl/Cargo.toml

The blanket POST gate added for the /responses review comment also blocked AnthropicModels, which deliberately claims GET /models to normalise x-api-key into Authorization. That 401'd Anthropic model discovery. Move the gate behind translator.translates_request_body(), so it only guards translators that actually deserialise a body. Defaults to true; AnthropicModels overrides it to false.

# Conflicts: # Cargo.lock # dwctl/Cargo.toml

The multi-step feature on onwards pulls in the optional fusillade dep, which the merged lockfile was missing.

…billing onto one parse Fixes Anthropic under-charging: Anthropic responses had no deserialize-able type so they fell to AiResponse::Other and billed zero tokens. Logging and billing now derive from a single parse_ai_response -> AiResponse (via TokenMetrics::from) instead of two parses through divergent types. Responses parse with dwctl's own types.rs; Anthropic with model.rs (now Deserialize + Clone); OpenAI endpoints keep async-openai (typed reasoning tokens, tolerant of raw passthrough bodies). Removes the UsageExtractor / ProtocolUsage / UsageRegistry layer and the per-translator extract_usage impls. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

Formatting-only fixup for test helpers added in the previous commit before the final fmt pass ran. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

onwards' strict chat/completions/embeddings handlers now validate the request shape and forward the ORIGINAL bytes instead of deserialising and re-serialising (which reordered keys and silently dropped unknown nested fields). The caller id-scrub and the streaming usage-flag injection (stream_options.include_usage, x-fusillade-stream force-stream) move into a new dwctl outbound_request middleware, wired innermost on the onwards router (inner to cache). This retires dwctl's stream_usage_transform + the onwards BodyTransformFn wiring, so onwards no longer manipulates request bodies for edge traffic. The /responses background strip is dropped: translation has already flattened /responses to chat by that layer. Malformed-request status mapping (422 for schema-invalid, 400 for bad JSON) is preserved via a shared parse helper. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

# Conflicts: # dwctl/src/inference/engine/assembly.rs # dwctl/src/inference/engine/loop_http_client.rs # dwctl/src/inference/engine/processor.rs # dwctl/src/inference/engine/transition.rs # dwctl/src/inference/middleware.rs # dwctl/src/inference/store.rs # dwctl/src/inference/streaming.rs # dwctl/src/lib.rs # dwctl/src/request_logging/serializers.rs # dwctl/src/test/multi_step_executor.rs # dwctl/src/test/responses.rs

…the body When the streaming usage flags are injected the body changes size, so the inbound Content-Length no longer matches. onwards forwards headers verbatim to the upstream, so a stale length could truncate or hang the upstream read. Remove the header (as the Anthropic translator already does) so it is recomputed. Addresses a Copilot review comment on PR #1317. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

After the multi-step machinery was removed on this branch, dwctl no longer references onwards-fusillade (main still declares it because main uses the multi-step loop). Removing the unused path dependency. Addresses a Copilot review comment on PR #1317. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

This reverts commit 5d11e27.

# Conflicts: # dwctl/src/inference/store.rs # dwctl/src/request_logging/analytics_handler.rs

Copilot

Pull request overview

This PR shifts “Responses” and request/response body ownership from onwards into dwctl: onwards strict-mode handlers now validate schema but forward original request bytes, while dwctl adds an edge translation layer for /responses (and improved logging/billing parsing for translated Responses + Anthropic bodies).

Changes:

Move strict-mode request-body manipulation out of onwards: onwards validates then forwards original bytes unchanged for chat/completions/embeddings/completions.
Add a dwctl-owned OpenAI Responses edge translator (request/response + streaming + previous_response_id hydration helpers) and adjust middleware interfaces to carry the original request + tracking id into translators.
Update request logging/analytics to parse dwctl-owned Responses/Anthropic shapes; remove legacy multi-step loop/tool-executor plumbing and associated tests/modules.

Reviewed changes

Copilot reviewed 43 out of 44 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
onwards/src/strict/schemas/utils.rs	Removes extra-field request-id scrubbing helper from onwards strict schemas.
onwards/src/strict/schemas/responses.rs	Drops scrubbing API from Responses request schema and removes unused import.
onwards/src/strict/schemas/chat_completions.rs	Drops scrubbing API from ChatCompletion request schema and removes unused import.
onwards/src/strict/handlers.rs	Strict handlers now validate then forward original bytes; adds shared strict JSON parse helper; updates tests accordingly.
dwctl/src/test/strict_mode.rs	Makes strict-mode chat test resilient to onwards cache sync timing by retrying requests.
dwctl/src/test/responses.rs	Replaces multi-step store-driven test with POST-then-GET integration coverage for Responses id correctness.
dwctl/src/test/multi_step_executor.rs	Removes end-to-end loop executor integration test module.
dwctl/src/test/mod.rs	Stops exporting the removed multi_step_executor test module.
dwctl/src/request_logging/utils.rs	Switches Responses parsing to dwctl-owned response/event types; adds Anthropic response parsing helpers.
dwctl/src/request_logging/stream_usage.rs	Removes old body transform used to inject streaming usage flags (moved to new middleware).
dwctl/src/request_logging/models.rs	Updates `AiResponse` variants to use dwctl-owned Responses/Anthropic types (incl. streaming).
dwctl/src/request_logging/mod.rs	Removes `stream_usage` module export.
dwctl/src/request_logging/analytics_handler.rs	Simplifies parse/usage alarms and adds path-based “usage-bearing” gating.
dwctl/src/metrics/errors.rs	Removes `ZDR_DISPATCH` component constant.
dwctl/src/inference/translation/responses/util.rs	Adds Responses translator utility functions (ported from onwards).
dwctl/src/inference/translation/responses/response.rs	Adds Chat Completions → Responses response conversion (ported).
dwctl/src/inference/translation/responses/request.rs	Adds Responses → Chat Completions request conversion (ported).
dwctl/src/inference/translation/responses/mod.rs	Adds full OpenAI Responses translator (detect, request/response translation, SSE reframing).
dwctl/src/inference/translation/responses/hydrate.rs	Adds `previous_response_id` hydration helper that inlines prior output items.
dwctl/src/inference/translation/mod.rs	Extends translator trait to accept original request + tracking id; adds POST gating hook.
dwctl/src/inference/translation/middleware.rs	Caps buffered body size for translation; threads original request + tracking id into response translation and SSE reframing.
dwctl/src/inference/translation/anthropic/models.rs	Updates translator impl for new trait signature and opts out of POST gating (GET-only translator).
dwctl/src/inference/translation/anthropic/model.rs	Makes Anthropic Messages response types `Deserialize` to support parsing for analytics/billing.
dwctl/src/inference/translation/anthropic/mod.rs	Updates Anthropic Messages translator impl for new trait signature.
dwctl/src/inference/tools/mod.rs	Updates module docs/exports to reflect removal of server-side executor from this surface.
dwctl/src/inference/tools/executor.rs	Removes `HttpToolExecutor` and related tests; keeps only tool resolution/rendering helpers.
dwctl/src/inference/streaming.rs	Removes inline multi-step loop streaming/blocking helpers; keeps flex replay streaming utility.
dwctl/src/inference/store.rs	Removes multi-step pending-input side-channel + MultiStepStore impl; keeps response store retrieval path.
dwctl/src/inference/response_store.rs	Introduces dwctl-owned `ResponseStore` trait and `StoreError` for Responses lifecycle.
dwctl/src/inference/outbound_request.rs	Adds new outbound request middleware to inject streaming usage flags pre-onwards.
dwctl/src/inference/mod.rs	Registers new outbound_request + response_store modules.
dwctl/src/inference/handler.rs	Switches to dwctl’s `StoreError` type for response retrieval handler.
dwctl/src/inference/engine/mod.rs	Refocuses engine module to persistence/writer; removes multi-step loop modules.
dwctl/src/inference/engine/processor.rs	Removes daemon processor that routed `/v1/responses` into the retired multi-step loop.
dwctl/src/inference/engine/loop_http_client.rs	Removes fusillade HttpClient implementation that ran the retired multi-step loop.
dwctl/src/inference/engine/assembly.rs	Removes chain-to-Responses assembly logic (tied to retired multi-step loop).
dwctl/src/api/models/requests.rs	Extends API-side response enum to include Anthropic (blocking + streaming) variants.
.sqlx/query-c29f0d9ae97f622eb24178d88c93fff72be81635a89ee33e0a8a5385d5655fed.json	Removes SQLx metadata for deleted tool_call_analytics insert query.

Files not reviewed (1)

.sqlx/query-c29f0d9ae97f622eb24178d88c93fff72be81635a89ee33e0a8a5385d5655fed.json: Generated file

+//! Small helpers backing the Responses translator.
+//!
+//! `ensure_field` / `scrub_request_id_fields_from_extra` are copied from onwards'
+//! `strict::schemas::utils`; `merge_reasoning_text` / `chat_usage_to_response_usage`
+//! from onwards' `strict` module (both `pub(crate)` there, so not importable).
+//! They are duplicated here as part of moving Responses ownership into dwctl; the
+//! onwards copies retire with the rest of its Responses code (COR-536).
+
+// `ensure_field` / `scrub_request_id_fields_from_extra` back the copied response
+// normaliser, which dwctl doesn't call yet; allow dead code until it's wired.
+#![allow(dead_code)]


+    // Outer layers (onwards body limit, cache) already bound the body, so buffering
+    // with no extra limit here can't widen the exposure.
+    let bytes = match axum::body::to_bytes(body, usize::MAX).await {
+        Ok(b) => b,
+        Err(_) => return (StatusCode::BAD_REQUEST, "failed to read request body").into_response(),
+    };


+    let routing_deadline = std::time::Instant::now();
+    let mut chat_response = send_chat(&chat_request).await;
+    while chat_response.status_code() == 404 && routing_deadline.elapsed() < std::time::Duration::from_secs(5) {
+        tokio::time::sleep(tokio::time::Duration::from_millis(1)).await;
+        chat_response = send_chat(&chat_request).await;
+    }


@@ -314,25 +311,6 @@ mod tests {
    use std::time::{Duration, SystemTime};
    use tokio::sync::mpsc;


+    // Last-mile request-body prep, applied innermost so it runs right before onwards
+    // (inner to cache, which must hash the original body). Does the id-scrub and the
+    // streaming usage-flag injection that onwards / the BodyTransformFn hook used to
+    // do, so onwards can forward the body untouched.


JoshC8C7 and others added 27 commits July 16, 2026 15:13

v1 - copy response handling to dwctl and switch over, but dont delete…

147050e

… from onwards yet.

Merge branch 'main' into josh/responses-edge-in-dwctl

f141e51

# Conflicts: # Cargo.lock # dwctl/Cargo.toml # dwctl/src/inference/middleware.rs

lockfile

f18d283

comments

8e16ccc

Merge branch 'main' into josh/responses-edge-in-dwctl

efa5bb9

# Conflicts: # Cargo.lock # dwctl/Cargo.toml

onwards

4d851ca

comments

04239d2

Merge branch 'main' into josh/responses-edge-in-dwctl

0f7f3c5

# Conflicts: # Cargo.lock # dwctl/Cargo.toml

chore: sync Cargo.lock after merging main

b7d6a47

The multi-step feature on onwards pulls in the optional fusillade dep, which the merged lockfile was missing.

Merge branch 'main' into josh/responses-edge-in-dwctl

b7782f8

comments

3bd892e

Merge branch 'main' into josh/responses-edge-in-dwctl

b564c65

Merge branch 'main' into josh/responses-edge-in-dwctl

fd8db2b

translate in non-strict mode also

e77c29e

style: rustfmt anthropic billing test helpers

76dbde4

Formatting-only fixup for test helpers added in the previous commit before the final fmt pass ran. Claude-Session: https://claude.ai/code/session_018UmxkSpbKjjYCNcgb9bZwp

sqlx

0b372d2

Revert "chore(dwctl): drop unused onwards-fusillade dependency"

fc8555b

This reverts commit 5d11e27.

Merge branch 'main' into josh/responses-edge-in-dwctl

368a12b

# Conflicts: # dwctl/src/inference/store.rs # dwctl/src/request_logging/analytics_handler.rs

Copilot AI review requested due to automatic review settings August 3, 2026 11:42

Copilot started reviewing on behalf of JoshC8C7 August 3, 2026 11:42 View session

Copilot AI reviewed Aug 3, 2026

View reviewed changes

Merge branch 'main' into josh/responses-edge-in-dwctl

624481a

Navigation Menu

Uh oh!

Preview/responses edge in dwctl - #1418

Preview/responses edge in dwctl#1418
JoshC8C7 wants to merge 28 commits into
mainfrom
preview/responses-edge-in-dwctl

JoshC8C7 commented Aug 3, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@@ -314,25 +311,6 @@ mod tests {
		use std::time::{Duration, SystemTime};
		use tokio::sync::mpsc;

Uh oh!

Conversation

JoshC8C7 commented Aug 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants