[Engine] Idle GPU spin: both cards stay at 100% / ~110 W with no request in flight #5372

Summary

With the model loaded and no requests in flight, both GPUs sit at 100% busy and ~110–120 W each (sclk pinned at ~3060 MHz, mclk idle at 96 MHz). Stopping the engine drops them to 13–17 W. It looks like a GPU-side wait kernel (doorbell / hand-off spin) is left resident after a request, so the driver never lets the card idle.

Environment

• Fork: nicejunjie/Strata-rdna3, engine self-built 0.1.49.1 (gfx1100)
• 2× RX 7900 XTX (24 GB), Ubuntu 26.04, kernel 7.x, 94 GB RAM
• Model: Qwen3.8-Flash-Next UD-IQ4_XS (experts in RAM, 55.4 GiB)
• Args: --tensor-parallel 2 --layer-split 24 --max-context 262144 --kv int8 --pipeline-windows 2 --vram-reserve-mib 768 --vram-reserve-later-mib 768 --prefill auto --spec 4 --mtp ...
• STRATA_MEGAKERNEL=1

Measurements (idle, no request, 60 s sample)

mclk is 96 MHz in both states, so it is a compute-side spin, not memory.

Ruled out

• STRATA_MEGAKERNEL=0 → same 110 W
• removing --pipeline-windows 2 → same
• killing strata-vision → same
• requests: 0 connections on :8001, 0 prompt lines in the log, engine threads at 0% CPU
• restart with zero requests still goes to 100% within ~20 s (it is not a leftover from the last request only)

Tried (no effect, reverted)

Adding a bounded spin to doorbell_wait_kernel, wait_flag_ge_kernel, wait_flag_copy_i32_kernel and a 3 s default in wait_handoff_kernel did not change the idle power (correctness and speed were fine).

Request

When the pipeline has no work, could the resident wait kernel be released — e.g. publish the doorbell / exit the wait after a timeout, or host-side park and relaunch on the next request? Even an opt-in flag ("let the card idle between requests", trading ~0.1–0.5 ms of first-token latency) would be enough.

Current workaround

server.py --lazy --idle-unload 1800 — the model is unloaded after 30 min idle and reloads on the next request (idle then 5–11 W/card), at the cost of a ~70 s cold start.