Skip to content

Make ext-image-capture work with damage-driven compositors - #1

Merged
powersemmi merged 7 commits into
masterfrom
fix/ext-capture-damage-driven
Sep 12, 2026
Merged

powersemmi merged 7 commits into
masterfrom
fix/ext-capture-damage-driven

Conversation

@powersemmi

@powersemmi powersemmi commented Sep 12, 2026 •

Copy link
Copy Markdown
Owner

The ext-image-capture backend assumed frames arrive at a steady rate, but compositors implementing the protocol as intended (niri with niri-wm/niri#3942, wlroots) complete a capture only when the source content actually changes. This surfaced three issues when streaming a nested niri:

  • The capture thread died with frame timeout: compositor did not produce a frame within 1s as soon as the screen went quiet. A timeout is "no changes yet", not an error, so keep waiting.
  • The picture arrived upside down: the buffer comes in the source's native orientation with the transform delivered per frame (flipped-180 on niri's nested winit backend), and the event was ignored. The transform is now undone on the captured pixels; damage falls back to the full frame in that case since the reported rects are in transformed buffer coordinates — this also fixes the stripe artifacts the delta encoder produced from mixing transformed data with untransformed rects.
  • The remote cursor was invisible: the session is now created with paint_cursors so the compositor composites it into the frames.

Verified end to end: RemoteWay client/server streaming a nested niri over ext-image-capture, correct orientation, visible cursor, idle screen produces no traffic, and animations stream at full rate.

The rest of the branch brings CI back to green — it had been failing on cargo fmt --check since May, so the later gates never ran and accumulated debt:

  • The tree is reformatted with rustfmt (own commit, no functional changes).
  • Clippy is clean again: the renamed nix MFdFlags type, doc backticks, unnecessary qualifications, dead assignments of zeroed reference structs in the encoder backends; not-yet-used fields of the WIP Vulkan encoders are kept under allow(dead_code).
  • The bicubic upscaler clamped the kernel anchor instead of the sample indices, shifting the whole kernel by one sample at the edges (caught by upscale_preserves_distinct_pixels).
  • idle_sleep_value still expected the old 500µs constant after IDLE_SLEEP was raised to 10ms — flagging this one for review: the test now matches the constant, but if 500µs was the intended value, it's the constant that should change.
  • The coverage gate now excludes the GPU crates (remoteway-vulkan, remoteway-encode): they can only be exercised on a machine with a GPU, so on CI they sat at single-digit coverage and dragged the total to 62%, making the 80% threshold unpassable. Measured over the code the test suite can actually reach, coverage is 82%.

cargo fmt --check, cargo clippy --all -- -D warnings, cargo test --workspace, and scripts/coverage.sh all pass locally.

Tested against niri with its ext-image-copy-capture implementation
(niri-wm/niri#3942), which, like wlroots, completes a capture only when
the source content actually changes:

- A frame timeout is no longer fatal. On a quiet screen the compositor
  legitimately produces no frame for an unbounded amount of time, so
  keep waiting instead of tearing down the capture pipeline.
- Handle the frame transform event. The buffer arrives in the source's
  native orientation (e.g. flipped-180 on niri's nested winit backend),
  so undo the transform on the captured pixels and fall back to full
  damage since the reported rects are in transformed coordinates.
- Request cursor compositing via the paint_cursors option so the remote
  cursor is visible in the stream.
@powersemmi powersemmi self-assigned this Sep 12, 2026
CI has been failing on cargo fmt --check since May; this brings the
whole tree back to standard formatting so the gate is meaningful again.
The clippy gate never ran on the new encoder backends because CI failed
at the fmt step first. Mechanical fixes throughout: the renamed nix
MFdFlags type, doc backticks, unnecessary qualifications, a stray
unreachable match arm, and dead assignments of zeroed reference structs
that are always overwritten before use. Fields and helpers of the
work-in-progress Vulkan encoders that are not read yet are kept under
allow(dead_code) rather than removed.
The kernel anchor was clamped into the image before the four sample
offsets were derived from it, so for edge pixels the whole kernel
shifted by one sample and the outermost pixels blended into their
neighbors. Clamp only the sample indices, keeping the anchor (and the
fractional weight) where the mapping puts it. Fixes the
upscale_preserves_distinct_pixels test.
IDLE_SLEEP was raised to 10ms at some point but the test still expected
500us; CI never reached the test step to notice.
clippy 1.98 on CI flags chunks_exact with a constant size; as_chunks
also gives a fixed-size array, dropping the manual byte indexing.
remoteway-vulkan and remoteway-encode can only be exercised on a
machine with a GPU, so on CI they sat at single-digit coverage and
dragged the total to 62%, making the 80% gate unpassable. The gate now
measures the code the test suite can actually reach: 82% as of this
commit. Both --exclude and --ignore-filename-regex are needed — the
former skips the crates' own test runs, the latter keeps their files
out of the report when built as dependencies.
@powersemmi
powersemmi merged commit 5b1d2a2 into master Sep 12, 2026
2 checks passed
@powersemmi
powersemmi deleted the fix/ext-capture-damage-driven branch September 12, 2026 01:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant