Add video frame format render option#1142
Draft
xuelongmu wants to merge 1 commit into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a first-class render option for source-video frame extraction:
and the matching programmatic config field:
The default remains
auto, preserving the existing behavior: alpha or alpha-capable video sources extract as PNG, and opaque sources extract as JPG.Rationale
In an
apple-devices-piprender, the timeline/browser preview looked close to the original iPhone recording, but the rendered MP4 shifted saturated UI reds. Changing final H.264 color tags/range did not fix it, and pixel checks showed the bad color was already present in HyperFrames' captured browser PNG frames.The root cause was earlier in the pipeline: non-alpha source videos were extracted as JPEG frames before browser capture. That JPEG step visibly changed saturated UI colors before final encoding ever ran. In the reported case, the original iPhone frame sampled
RGB(221,56,46)on the red "Elevated Risk" indicator, while HyperFrames' browser PNG frame sampledRGB(186,51,58). A direct ffmpeg source-video composite preserved the same pixel closely atRGB(220,56,50), showing the final H.264 encoder can preserve the color when the source video does not pass through JPEG extraction.This PR adds an explicit, opt-in PNG extraction path for UI recordings, screen captures, and other color-sensitive source videos. It avoids an RGB lift or color-correction workaround and leaves final encoder defaults unchanged.
What Changed
--video-frame-format auto|jpg|pngtohyperframes render.RenderConfig.videoFrameFormat.resolveFrameFormatso:pngextracts opaque videos as PNGjpgextracts opaque videos as JPGauto/undefined preserves existing behaviorValidation
bun run --filter @hyperframes/engine test -- src/services/videoFrameExtractor.test.tsbun run --filter @hyperframes/cli test -- src/commands/render.test.ts src/utils/dockerRunArgs.test.tsbun run --filter @hyperframes/aws-lambda test -- src/sdk/validateConfig.test.tsbun run --filter @hyperframes/engine typecheckbun run --filter @hyperframes/producer typecheckbun run --filter @hyperframes/cli typecheckbun run --filter @hyperframes/aws-lambda typecheckbunx oxfmt --checkbunx oxlintThe new regression test synthesizes a tiny saturated-red UI fixture on a pale pink background and verifies PNG extraction keeps sampled red pixels within a max channel delta of 5 from the decoded source, while also proving JPG and PNG extraction caches remain separate.