Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

header: implement frame_size_with_refs() #2520

Merged
merged 5 commits into from
Sep 2, 2020

Conversation

jamrial
Copy link
Contributor

@jamrial jamrial commented Aug 29, 2020

The spec has a code path in the uncompressed header named frame_size_with_refs(), defined in section 5.9.7, which an inter frame may use to infer its dimensions from one of the seven frames it uses as reference instead of inferring them from the sequence header or ultimately hardcoding them. This path is triggered on non error resilient encodings by signaling frame_size_override_flag == 1, a bit that means that either frame dimensions are coded in the frame, or that an attempt to infer both frame and render dimensions from a reference frame should be done.

For encodings using a SAR other than 1:1, rav1e is currently hardcoding the render sizes on every frame, which is an overhead of about 4 bytes. With this change, for inter frames the encoder will look at each of the seven reference frames and if it finds one where both frame and render sizes are the same as this frame, instead of writing the sizes it will write the bits needed to tell the decoder to infer them from it.

Here's a comparison of the uncompressed header written for inter frames before and after this PR when encoding a 720x480 sample with SAR 2:1.

Before

[trace_headers @ 00000203eb3bfda0] Frame Header
[trace_headers @ 00000203eb3bfda0] 16          show_existing_frame                                         0 = 0
[trace_headers @ 00000203eb3bfda0] 17          frame_type                                                 01 = 1
[trace_headers @ 00000203eb3bfda0] 19          show_frame                                                  1 = 1
[trace_headers @ 00000203eb3bfda0] 20          error_resilient_mode                                        0 = 0
[trace_headers @ 00000203eb3bfda0] 21          disable_cdf_update                                          0 = 0
[trace_headers @ 00000203eb3bfda0] 22          frame_size_override_flag                                    0 = 0
[trace_headers @ 00000203eb3bfda0] 23          order_hint                                             000011 = 3
[trace_headers @ 00000203eb3bfda0] 29          primary_ref_frame                                         010 = 2
[trace_headers @ 00000203eb3bfda0] 32          refresh_frame_flags                                  00100000 = 32
[trace_headers @ 00000203eb3bfda0] 40          frame_refs_short_signaling                                  0 = 0
[trace_headers @ 00000203eb3bfda0] 41          ref_frame_idx[0]                                          100 = 4
[trace_headers @ 00000203eb3bfda0] 44          ref_frame_idx[1]                                          100 = 4
[trace_headers @ 00000203eb3bfda0] 47          ref_frame_idx[2]                                          101 = 5
[trace_headers @ 00000203eb3bfda0] 50          ref_frame_idx[3]                                          100 = 4
[trace_headers @ 00000203eb3bfda0] 53          ref_frame_idx[4]                                          100 = 4
[trace_headers @ 00000203eb3bfda0] 56          ref_frame_idx[5]                                          100 = 4
[trace_headers @ 00000203eb3bfda0] 59          ref_frame_idx[6]                                          001 = 1
[trace_headers @ 00000203eb3bfda0] 62          render_and_frame_size_different                             1 = 1
[trace_headers @ 00000203eb3bfda0] 63          render_width_minus_1                         0000010110011111 = 1439
[trace_headers @ 00000203eb3bfda0] 79          render_height_minus_1                        0000000111011111 = 479

After

[trace_headers @ 000001e8e5d5fda0] Frame Header
[trace_headers @ 000001e8e5d5fda0] 16          show_existing_frame                                         0 = 0
[trace_headers @ 000001e8e5d5fda0] 17          frame_type                                                 01 = 1
[trace_headers @ 000001e8e5d5fda0] 19          show_frame                                                  1 = 1
[trace_headers @ 000001e8e5d5fda0] 20          error_resilient_mode                                        0 = 0
[trace_headers @ 000001e8e5d5fda0] 21          disable_cdf_update                                          0 = 0
[trace_headers @ 000001e8e5d5fda0] 22          frame_size_override_flag                                    1 = 1
[trace_headers @ 000001e8e5d5fda0] 23          order_hint                                             000011 = 3
[trace_headers @ 000001e8e5d5fda0] 29          primary_ref_frame                                         010 = 2
[trace_headers @ 000001e8e5d5fda0] 32          refresh_frame_flags                                  00100000 = 32
[trace_headers @ 000001e8e5d5fda0] 40          frame_refs_short_signaling                                  0 = 0
[trace_headers @ 000001e8e5d5fda0] 41          ref_frame_idx[0]                                          100 = 4
[trace_headers @ 000001e8e5d5fda0] 44          ref_frame_idx[1]                                          100 = 4
[trace_headers @ 000001e8e5d5fda0] 47          ref_frame_idx[2]                                          101 = 5
[trace_headers @ 000001e8e5d5fda0] 50          ref_frame_idx[3]                                          100 = 4
[trace_headers @ 000001e8e5d5fda0] 53          ref_frame_idx[4]                                          100 = 4
[trace_headers @ 000001e8e5d5fda0] 56          ref_frame_idx[5]                                          100 = 4
[trace_headers @ 000001e8e5d5fda0] 59          ref_frame_idx[6]                                          001 = 1
[trace_headers @ 000001e8e5d5fda0] 62          found_ref[0]                                                1 = 1

Currently, rav1e has fixed frame and render sizes for the entire video sequence, so it will always signal to use the very first reference frame, but with this change the frame header writing code is ready to handle both frame and render sizes changing in a per-frame basis, if it ever gets implemented.

@coveralls
Copy link
Collaborator

coveralls commented Aug 30, 2020

Coverage Status

Coverage increased (+0.001%) to 83.093% when pulling f5f8b0a on jamrial:frame_size_with_refs into 47039d9 on xiph:master.

Copy link
Collaborator

@lu-zero lu-zero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks fine to me, thank you.

Rename write_frame_size to write_max_frame_size and
write_frame_size_override to write_frame_size while at it.

Signed-off-by: James Almer <[email protected]>
@jamrial jamrial force-pushed the frame_size_with_refs branch 2 times, most recently from da3e198 to 287c8ec Compare August 30, 2020 14:39
@jamrial jamrial force-pushed the frame_size_with_refs branch from 287c8ec to f5f8b0a Compare August 30, 2020 20:08
@jamrial
Copy link
Contributor Author

jamrial commented Sep 1, 2020

Can this be merged? Among other things, it's required to ensure #2490 works as intended.

@shssoichiro shssoichiro merged commit 0fb9209 into xiph:master Sep 2, 2020
@jamrial jamrial deleted the frame_size_with_refs branch September 4, 2020 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants