Skip to content

th#1043: joint precision fit for shared-component multi-lane loads#373

Merged
PaulFidika merged 1 commit into
masterfrom
release/th1043-shared-lane-precision
Jul 23, 2026
Merged

th#1043: joint precision fit for shared-component multi-lane loads#373
PaulFidika merged 1 commit into
masterfrom
release/th1043-shared-lane-precision

Conversation

@PaulFidika

Copy link
Copy Markdown
Contributor

Summary

  • Sequential per-lane loading for a gw#479 shared-component group (qwen-image's t2i/edit lanes: shared Qwen2.5-VL text_encoder+VAE, exclusive transformer per lane) let the FIRST lane consume free VRAM at native precision, starving the second lane into an offload placement — which the shared-component invariant then refuses outright with a hard RetryableError ("shared-component lanes require resident placement").
  • Root cause confirmed live (th#1043): a real qwen-image request loaded the edit lane resident at bf16 (~40GB transformer + ~15.5GB shared encoder), leaving too little free VRAM for the t2i lane's transformer, which then required offload and was refused.
  • Fix: decide precision for the WHOLE shared-component group jointly, against its combined footprint (shared components counted once), BEFORE any lane loads — forcing fp8 storage on every lane in the group when (and only when) native precision doesn't fit the group but fp8 does.

Test plan

  • New unit tests (tests/test_shared_lane_precision_th1043.py) cover: starved group forces fp8 when it fits; group that fits natively is untouched; group that can't fit even at fp8 is left to the existing per-lane ladder; single-lane / no-shared-components never force.
  • mypy clean on touched files
  • ruff check clean on touched files
  • Full local pytest suite green (one pre-existing unrelated flake in test_activity_gw601.py, confirmed to pass in isolation — timing-sensitive under shared-box load, not touched by this change)

Sequential per-lane loading let the first lane in a gw#479 shared-
component group (qwen-image's t2i/edit) consume free VRAM at native
precision, starving the second lane into an offload placement the
shared-component invariant refuses outright ("shared-component lanes
require resident placement"). Decide fp8 storage for the whole group
up front, against its combined footprint, before any lane loads.
@PaulFidika
PaulFidika merged commit 5c0689e into master Jul 23, 2026
1 check passed
@PaulFidika
PaulFidika deleted the release/th1043-shared-lane-precision branch July 23, 2026 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant