[codex] Support raw image offload in v1 train client

[eligotts]

Design update — inline/offload image storage

This PR now follows the prime-rl multimodal image storage policy:

• offload: current behavior, rewrite base64 data images to file:// run assets and require file-backed image URLs.
• inline: keep data:image/...;base64,... URLs in the message payload and validate them without rewriting.

TrainClient now calls the policy-aware image preparation helper, so prime-rl can be the single source of truth via environment/config propagation.

Validation after latest push: uv run pytest tests/v1/test_train_client_multimodal.py -q passed (5 passed). Commit/push hooks also passed (ruff check, ruff format, generated AGENTS/CLAUDE check, ty).

Design update — dropped the None/cache-only image path

This PR and its companions (prime-rl #2836 / verifiers #1746 / renderers #89) no longer use the "send None for already-cached images" mechanism. Every image carries its raw descriptor ref at every slot (current and prior turns); /inference/v1/generate rematerializes each ref from disk every request.

Why: the None path coupled correctness to deployment (LRU cache present, single replica / DP-affinity, no eviction) and surfaced a miss as a hard vLLM EngineDeadError (qwen3-vl mrope dereferences a None image_grid_thw) that the retry net couldn't catch across the engine→API IPC. Dropping it is deployment-agnostic (a miss is impossible) and non-hacky. vLLM's mm_hash encoder cache still skips the expensive GPU re-encode for free — we only forgo the cheap IPC/CPU-reprocess dedup.

Validated: color-codeword (Qwen3-VL-4B) under DP=2, no affinity / no cache reliance: 0 crashes, 0 data=None, multi-turn accumulation correct, reward ~0.84. Also confirmed under TP.

This repo: with every image carrying its ref, no cache miss can occur — removed the retry subsystem (_generate_with_image_ref_retry, _has_descriptor_only_images, _retryable_mm_error_type, _json_error_type, _RETRYABLE_MM_ERROR_TYPES). Rollouts call renderers.client.generate directly. Obsolete retry tests removed.

---

Original description

Summary

• tighten v1 multimodal graph serialization around strict raw descriptor sidecars
• reject processed multimodal payload keys recursively, including nested pixel_values, image_embeds, and image_features
• update v1 multimodal tests to use strict prime_raw_mm_item envelopes instead of descriptor-only Qwen payloads
• keep raw image offload and retry behavior aligned with the companion Renderers and Prime-RL PRs

Companion PRs

• Prime-RL: Support v1 raw multimodal image offload prime-rl#2836
• Renderers: [codex] Support raw image refs for multimodal rendering renderers#89

Notes

• Draft/WIP: this depends on the renderer generic raw multimodal ref contract in the companion PR.
• v1 multimodal sidecars intentionally carry raw descriptors only, not processed image tensors or image-processor payloads.
• Prime/vLLM materialization happens from raw image refs rather than Verifiers-held processor outputs.

Validation

• Commit hooks: ruff check, ruff format, generated AGENTS/CLAUDE check passed.
• Push hook: ty (ci parity) passed.
• End-to-end hosted-style smoke through Prime-RL with /home/ubuntu/verifiers, /home/ubuntu/renderers, and /home/ubuntu/prime-rl-v1-raw-mm-offload completed inference, env rollouts, train batch creation, trainer step 0, and decoded strict trainer-bound raw image refs.

[!NOTE]

Support raw image offload in the v1 train client for multimodal rollouts

• Adds prepare_images_inplace in verifiers/utils/multimodal.py to normalize HF-style and OpenAI-style image parts to a canonical image_url shape and offload non-file:// URLs to run assets.
• Introduces a prepare_request_body async hook on Client (no-op by default); TrainClient overrides it to run image preparation before rendering when the dialect is ChatDialect.
• The interception server now calls prepare_request_body before streaming decisions, surfacing failures as structured InterceptionError or RolloutError.
• Extends MessageNode, _attribute_mm, and Branch.spread in verifiers/v1/graph.py and verifiers/v1/trace.py to carry mm_placeholders alongside items and hashes, and validates that multimodal sidecars use raw raw_image_uri descriptors rather than tensors.
• Bridging in TrainClient.get_response no longer blocks on multimodal content; multimodal renderers now receive previous_multi_modal_data from prior turns.
• Risk: legacy (v0) rollouts containing multimodal content now raise RuntimeError rather than silently proceeding.

^{Macroscope summarized 333052c.}

Update: ingress hardening (2c2824ae)

• prepare_images_inplace now covers the full renderer part treaty: nested image_url dicts, direct-string image_url, direct-string image parts, and typed pydantic parts. Non-string sources raise with the part shape named; renderer-side raw mode hard-requires file:// (no second offload layer).
• Interception server labels prepare_messages failures as InterceptionError instead of misattributing them to the user simulator.
• New shape-coverage test (skips until the renderers pin ships mm_store; passes against the sibling renderers checkout). Suite: 839 passed with pre-existing test_envs/test_opencode_rlm_env failures reproduced on the base branch.

Update: merged main (2b1627d03)

Reconciled with verifiers main (111 commits ahead at the merge-base — mostly unrelated v1 harness/multi-agent, taskset/environment, and trace/data-model churn; none of it touched this branch's actual offload files, verifiers/utils/multimodal.py / clients/renderer_client.py / types.py).

• interception/server.py: main's d21100bea ("resume replaces mid-request user injection") independently moved the user-simulator loop out of the interception server entirely, into Harness.launch/resume, and added graph-atomicity retry coalescing to handle_request/_stream. Took main's structure as authoritative; dropped this branch's now-obsolete session.user/session.opening mid-request injection (and its prepare_messages call sites) since injected/resumed messages now re-enter as ordinary request bodies, already covered by prepare_request_body (kept, rewired to the top of the new single-shot handle_request).
• graph.py: kept finish_reason/usage/multi_modal_data/previous_multi_modal_data() (this branch) alongside main's SkipJsonSchema wrapping convention and commit()'s new (tools) -> assistant_node_id signature. Caught and fixed an auto-merge that silently dropped the FinishReason/Usage imports — surfaced as a pydantic "Trace not fully defined" failure at Trace construction, not a textual conflict.
• trace.py: kept this branch's more accurate multi_modal_data docstring; took main's tuple-based bridge-mutation assertion in the test suite (strictly better — reuses already-computed prior_mm/prior_counts instead of recomputing).
• verifiers/v1/ARCHITECTURE.md: main deleted this file (docs moved to a much shorter docs/v1/architecture.md that doesn't cover this depth) — accepted the deletion; the one load-bearing fact (why multi_modal_data is JSON-excluded) is now a docstring on _NODE_DUMP_EXCLUDE instead of a dedicated doc.

Validation: full suite passing with a local renderers checkout override (uv run --with-editable ../renderers pytest tests/, minus PRIME_API_KEY-gated e2e/env tests) — includes test_prepare_images_inplace_offloads_every_image_part_shape, previously skipped pending a renderers pin with mm_store.

Update: dropped the orphaned prepare_messages hook (d9e79d6c)

Main's harness-segment-resume refactor removed the interception server's mid-request user-simulator injection — the only call sites of this PR's prepare_messages hook. Resumed user turns now re-enter the server as ordinary request bodies, already covered by prepare_request_body at request ingress, so the hook (base Client + TrainClient override) is deleted rather than carried as dead code.

Update: synced with main (0e4166608)

Merged verifiers main (35 commits): the v1 structure cleanup (#2146) retiring StrictBaseModel and renaming _NODE_DUMP_EXCLUDE → EXCLUDE_FIELDS, the ruff 0.16 / ty tooling bump (#2147, #2148), runtime-on-agent with the agent config stamped on the trace (#2106), Reward score/weight records (#2119), MCP tools for Codex (#2140).

Resolutions:

• trace.py: main's restructure subsumes this branch's block wholesale — EXCLUDE_FIELDS already carries multi_modal_data, and TRACE_VERSION = 1 is main's deliberate reset (this branch never touched it). Took main.
• graph.py: kept the raw-mm sidecar validators and previous_multi_modal_data; adopted main's plain BaseModel now that StrictBaseModel is gone.
• clients/train.py: kept the is_multimodal import — still used by the bridge path that threads previous_multi_modal_data.

Ruff 0.16 flagged two spots this branch added that main's own cleanup pass never saw: a constant getattr in the ingress offload walker, and the blind except at the prepare_request_body boundary (annotated # noqa: BLE001, matching main's convention at rollout boundaries).

Full suite with the sibling renderers checkout (minus PRIME_API_KEY-gated e2e/envs): all passing.

Update: merged main — renderer pool reconciliation (ebc0e2f26)

Merged verifiers main (25 commits: process-shared renderer pool #2218, serve CLI removal / ServeConfig rename #2237, Harbor separate verifier envs #2152, unscored-reward None placeholders #2235, pydantic adapter reuse #2233, and more).

One conflict, in train.py, against #2218's restructure:

• Adopted main's RendererSlot / ElasticRendererPool and the slot.run call structure wholesale. It subsumes this branch's _maybe_offload thread-hop (main now thread-hops + locks all encode-side work, generalized).
• Dropped main's multimodal bridging gate (not _has_multimodal_content(prompt) in can_bridge). That gate exists because processed-tensor sidecars can't bridge safely; this PR's raw descriptor refs are exactly what makes multimodal bridging safe, so the gate is removed and the previous_multi_modal_data bridge kwargs are re-applied inside the pooled bridge() closure.
• The ingress prepare_request_body image-offload hook auto-merged into the new TrainClient untouched.

Full suite: 913 passed, 74 skipped.

Update: merged main (c9d1f134c)

Merged verifiers main (11 commits: per-token advantages #2245, run info moved trace-to-episode #2244, ACP harnesses #2257, persistent harness runtimes #2249, and more). One conflict, in graph.py: main's advantages field landed adjacent to this branch's finish_reason on MessageNode — union of both. Full suite: 913 passed.

Update: review fixes — ingress hook + image-part parsing (97b6af8bd)

Two confirmed finds from Bugbot/Macroscope; the rest of the review items were assessed and intentionally skipped (impossible-deployment hardening, already-loud failure paths, and the deliberate raw-only trace contract):

• Interception ingress called the hook on the wrong object (Bugbot, High): session.ctx.client is the client config; the live client is session.client. Every intercepted request died at ingress with an AttributeError-turned-rollout-failure, so image offload never ran on that path. One-word fix.
• HF-style {type: "image", image: url} parts were silently dropped (Macroscope, High): the ingress offloader and the renderer part treaty both accept the shape, but content_to_parts kept only text/image_url — the image was offloaded, then lost before rendering. The typing boundary now normalizes it to ImageUrlContentPart.

Full suite: 913 passed.

Update: canonical image-part shape at ingress (49a7dc1a0)

Every image part is rewritten at v1 training ingress to exactly {"type": "image_url", "image_url": {"url": ...}} — HF-style {type: image} normalizes into it, data images offload to file:// run assets, and anything else (http, non-string) is rejected loudly. prepare_images_inplace loses the multi-shape/pydantic tolerance (walks request-body dicts/lists only), already-offloaded file:// URLs pass without importing renderers, and the v0 RendererClient ingress call is dropped — v0 renderer-client multimodal is explicitly not a supported path on this branch. One shape after ingress means dialects, renderers, and trace validation have a single place to look for an image. Full suite: 913 passed.

---

Note

Medium Risk
Changes the v1 multimodal wire format and training ingress path; rollouts fail at prepare time if renderers lacks mm_store, and multimodal v0 legacy envs are explicitly unsupported.

Overview
V1 multimodal training now normalizes every image part at interception ingress to {"type": "image_url", "image_url": {"url": ...}}, offloading data:image/... sources to file:// run assets via renderers.mm_store (prepare_images_inplace + new Client.prepare_request_body, implemented on TrainClient for chat dialects).

Trace multimodal sidecars no longer serialize numpy pixel tensors; they carry raw image descriptors (raw_image_uri, hashes, placeholder ranges) with strict validation on wire and in MessageNode. _attribute_mm assigns placeholders per introducing message; PendingTurn.previous_multi_modal_data() feeds multimodal turn bridging (the old “no bridge if images present” guard is removed).

Boundaries: HF-style {type: "image"} parts are normalized in content_to_parts; the v0 legacy bridge raises if messages or tokens include images/MM data. Lockfile adds optional nemo-gym extra.

^{Reviewed by Cursor Bugbot for commit 333052c. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR introduces new multimodal image offload capability with changes across multiple files affecting the training pipeline's handling of images. There is also an unresolved high-severity comment identifying a potential bug in the legacy bridge's multimodal rejection logic.

^{You can customize Macroscope's approvability policy. Learn more.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 4c6f31e. Configure here.}

[macroscopeapp]

🟠 High

verifiers/verifiers/v1/legacy.py

Line 295 in 35714d3

trace: Trace = Trace[WireTaskData](

The multimodal rejection check added in _to_v1_messages only runs for messages inside trajectory steps. A v0 rollout that fails before producing a step can carry image content in the top-level out["prompt"] with an empty trajectory, in which case rollout_output_to_trace never reaches the per-step check. The top-level prompt is instead passed to _to_wire_task, which silently flattens image content to empty text, so the bridge returns a text-only trace instead of raising _LEGACY_MM_UNSUPPORTED for an unsupported multimodal rollout. Consider validating the top-level out["prompt"] for image content before constructing the trace.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/legacy.py around line 295:

The multimodal rejection check added in `_to_v1_messages` only runs for messages inside `trajectory` steps. A v0 rollout that fails before producing a step can carry image content in the top-level `out["prompt"]` with an empty trajectory, in which case `rollout_output_to_trace` never reaches the per-step check. The top-level prompt is instead passed to `_to_wire_task`, which silently flattens image content to empty text, so the bridge returns a text-only trace instead of raising `_LEGACY_MM_UNSUPPORTED` for an unsupported multimodal rollout. Consider validating the top-level `out["prompt"]` for image content before constructing the trace.