-
Notifications
You must be signed in to change notification settings - Fork 191
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature request: Support unclamped floating point RGB data, and HDR JXR images #1884
Comments
Thank you for the report.
Regarding the API, there is Line 919 in 4865c1c
Unfortunately converting from F16 RGB to YUV is unimplemented.
I believe this is possible with today's API and some sample manipulation:
Regarding
What would be your options as of today to go from JXR to AVIF? From AVIF to JXR? |
Perhaps more widely used unclamped float options are EXR, TIFF/DNG and PFM (the latter requiring no dependency really as it's so simple). And JXL now maybe as well. |
I'm considering both encoding normal HDR image and image with gainmap. Seems you are only considering gainmap encoding? For normal HDR image encoding, the missing part is converting between float RGB and unorm RGB images (of different primaries and transfer function, most of which already implemented in #1861 and #1674). (Optionally we may also support direct conversion from float RGB to YUV).
That's probably a bad idea, and I think we're not planning to do tone mapping in libavif.
https://github.com/joedrago/colorist wrote by Joe can convert float JXR to HDR AVIF, but it haven't been updated for 2 years. I personally have a fork with some necessary updates making it work with the newest libavif. As for AVIF to JXR (with float RGB format) there's no such tool (and probably no such need).
Let me rewrite this to JXR -> Float RGB -> UNorm RGB (HDR)
I wonder if there is reasonable way non-professional user can produce images of these formats, like JXR's situation? If they always come from professional workflow, maybe libavif is not the best place handling them. For JXR support, maybe we can make it optional and only available on Windows. Windows system API provides support for JXR, so no extra dependency is needed in this case. |
Thank you for the explanation.
Indeed.
Assuming this is done both for "normal" HDR (which is PQ or HLG, right?) and for gain maps, there is little benefit in that feature unless
Another entry in |
HLG has more nuance due to it being scene referred. In the first step, I intend to only support PQ.
One usage I can imagine would be screenshot tools. Those JXR screenshots store exactly what Windows API returns, so tools can feed the same data to libavif and get HDR screenshot, but this time in AVIF format. Other cases dealing with HDR content would mostly be in professional workflows, so I expect more advanced tools being used.
For the float RGB support part, I was considering it too complicated to support in libavif. I bring this idea out now because gainmap support have most pieces implemented, and while we are here supporting this shouldn't be too much extra work. Gainmap itself is experimental though, so we may set back and wait gainmap to graduate from expermental state first. For JXR support, I prefer strictly limiting our support to 4 channel interleaved RGBA F16 and F32 pixel formats only, which should drastically reduce the burden to support it. |
If I understand correctly, this would be done using the following: avifRGBImage floatRgb;
floatRgb->depth = 16;
floatRgb->isFloat = true;
// Set other fields accordingly
avifImage * unormRgb = avifImageCreate(floatRgb.width, floatRgb.height, /*depth=*/12, AVIF_PIXEL_FORMAT_YUV444);
unormRgb->matrixCoefficients = AVIF_MATRIX_COEFFICIENTS_IDENTITY;
// Set other fields accordingly, for example AVIF_COLOR_PRIMARIES_SRGB and AVIF_TRANSFER_CHARACTERISTICS_UNSPECIFIED
avifImageAllocatePlanes();
avifImageRGBToYUV(unormRgb, &floatRgb);
Lines 236 to 238 in 03a12f8
So replacing the
This is the same code path as above, unless I missed something.
Some JXR reader in either |
I was considering a separated
I'd like to implement this if we consider it worthwhile.
If we are going to support JXR in |
Performance is not critical yet there. For now I would favor simplicity for review, test coverage, maintenance and debugging easiness over fewer plane copies, which is usually a cheap operation compared to encoding.
Feel free to do so, as you said HDR is quite experimental for now so even if it creates issues, it will just be for a few expert users.
I would prefer a platform-independent solution. Also most of our development and testing happen in Linux environments, so debugging there will be easier. |
With gainmap feature support, most colorspace manipulation functionalities are implemented in libavif. It may worth consider supporting floating point RGB input data with values outside [0, 1] range (Linear TC in H.273 clamps value), which is common for HDR data.
A good example is scRGB: scRGB uses BT709 primaries and a linear transfer characteristics where 1.0 maps to 80 nit. Windows uses scRGB for composition so HDR content captured on Windows usually are in this colorspace.
Xbox Game Bar and GeForce Experience are two common tools used to take HDR screenshot (of games usually), and they both directly save captured float data in JXR format, as Windows natively supports it. As a showcase for floating point RGB support, and providing a tool to convert HDR screenshot to AVIF for sharing, it may also worth consider supporting loading JXR images in
avifenc
.Here are some example JXR files from Microsoft: https://aka.ms/hdr-backgrounds-wip
The text was updated successfully, but these errors were encountered: