libobs/media-io: Avoid allocating frame memory if size is zero

[RytoEX]

Description

It is possible that size can still be zero after the preceding loop. If that happens, and bmalloc is called with 0 as its argument, OBS will crash as intended with OBS Studio 31. Avoid crashing by not allocating memory for a frame with a size of zero.

Motivation and Context

See the crash in:

• OBS High CPU Usage and General Issues #11729

If there's a better place to do this check, I'm open to alternatives.

How Has This Been Tested?

Compiled and ran locally on Windows 11 with a Video Capture Device added.

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[tt2468]

I think my primary worry here would be that video_frame_init is a void return, so we might not actually be fixing any issues, rather pushing the undefined behavior to somewhere else.

[RytoEX]

I think my primary worry here would be that video_frame_init is a void return, so we might not actually be fixing any issues, rather pushing the undefined behavior to somewhere else.

video_frame_init can already early return:

obs-studio/libobs/media-io/video-frame.c

Lines 198 to 199 in 0b2c858

	if (!frame)
	return;

That said, I'm open to alternatives, but we intentionally crash if bmalloc(0) is called now, so we should really try to avoid calling bmalloc(0).