feat(nvenc): add support for explicit cuda streams

[Yozer]

The performance gains probably depend on the specific pipeline.

In my case, I didn’t see any improvement with 1080p HEVC videos, but at 4K and 8K, the speed-up was around 13% and 25%, respectively (e2e wall time).

I haven’t looked into it too deeply, but my guess is that NVENC uses the default stream 0 and requires a full synchronization before encoding. At lower resolutions, that overhead doesn’t make much difference, but with 4K and 8K workloads, waiting for all the processing kernels to finish before encoding becomes more significant.

[WyattBlue]

Found one blocking concern:

Avoid modifying an initialized FFmpeg device context — frame.py lines 89–105 (

PyAV/av/video/frame.py

Lines 89 to 105 in dbef919

	err_check(
	lib.av_hwdevice_ctx_create(
	cython.address(device_ref),
	lib.AV_HWDEVICE_TYPE_CUDA,
	c_device,
	options.ptr,
	0,
	)
	)
	if self.cuda_stream:
	device_ctx: cython.pointer[lib.AVHWDeviceContext] = cython.cast(
	cython.pointer[lib.AVHWDeviceContext], device_ref.data
	)
	cuda_ctx: cython.pointer[AVCUDADeviceContext] = cython.cast(
	cython.pointer[AVCUDADeviceContext], device_ctx.hwctx
	)
	cuda_ctx.stream = cython.cast(CUstream, self.cuda_stream)

).

av_hwdevice_ctx_create() returns an initialized context, after which the PR writes directly to AVCUDADeviceContext.stream. FFmpeg explicitly states that hwctx must not be modified after av_hwdevice_ctx_init(). Although current NVENC happens to read the stream later during setup, this relies on behavior outside FFmpeg’s API contract. The stream should be supplied through a supported pre-initialization/creation path. FFmpeg API documentation (https://www.ffmpeg.org/doxygen/8.0/structAVHWDeviceContext.html)

[Yozer]

You’re right. Sorry, I have little experience with ffmpeg av.
It should look better now. My pipeline still has perf gain.
I also updated the changelog.

[WyattBlue]

The previous post-initialization mutation concern is resolved, but I found one remaining multi-GPU issue:

The secondary CUDA device context created in CudaContext._get_device_ref() is zero-initialized and copies only cuda_ctx and stream before calling av_hwdevice_ctx_init():

PyAV/av/video/frame.py

Lines 116 to 139 in 0b8d386

	device_ref = lib.av_hwdevice_ctx_alloc(lib.AV_HWDEVICE_TYPE_CUDA)
	if device_ref == cython.NULL:
	lib.av_buffer_unref(cython.address(owner_ref))
	raise MemoryError("av_hwdevice_ctx_alloc() failed")
	try:
	owner_device_ctx: cython.pointer[lib.AVHWDeviceContext] = cython.cast(
	cython.pointer[lib.AVHWDeviceContext], owner_ref.data
	)
	owner_cuda_ctx: cython.pointer[AVCUDADeviceContext] = cython.cast(
	cython.pointer[AVCUDADeviceContext], owner_device_ctx.hwctx
	)
	device_ctx: cython.pointer[lib.AVHWDeviceContext] = cython.cast(
	cython.pointer[lib.AVHWDeviceContext], device_ref.data
	)
	cuda_ctx: cython.pointer[AVCUDADeviceContext] = cython.cast(
	cython.pointer[AVCUDADeviceContext], device_ctx.hwctx
	)
	cuda_ctx.cuda_ctx = owner_cuda_ctx.cuda_ctx
	cuda_ctx.stream = cython.cast(CUstream, self.cuda_stream)
	device_ctx.free = _cuda_device_ctx_free
	device_ctx.user_opaque = cython.cast(cython.p_void, owner_ref)
	owner_ref = cython.NULL
	err_check(lib.av_hwdevice_ctx_init(device_ref))

In FFmpeg 8.1.2, av_hwdevice_ctx_init() only loads the CUDA functions; it does not populate the private cuda_device field. The normal creation path sets that field from the requested device, and CUDA frame initialization later uses it to query texture alignment:

• https://github.com/FFmpeg/FFmpeg/blob/n8.1.2/libavutil/hwcontext_cuda.c#L320-L335
• https://github.com/FFmpeg/FFmpeg/blob/n8.1.2/libavutil/hwcontext_cuda.c#L445-L463
• https://github.com/FFmpeg/FFmpeg/blob/n8.1.2/libavutil/hwcontext_cuda.c#L150-L167

As written, CudaContext(device_id > 0, cuda_stream=...) therefore combines the requested GPU context with device-0 metadata, which can produce an incorrect frame-pool alignment.

[Yozer]

Hmm, I wonder what would be best way to fix this one.
From what I can see the cuda device is hold in ffmpeg private struct - we could do hacky modifications it but I would like to avoid it.

The only solution I can see is to add a guard so it works for first gpu only. To be more specific: streams feature can be only used for device_id=0.
People could still in theory use different gpus via cuda visibility env variable.
Let me know @WyattBlue if you want me to add this guard.

The cleanest solution would be to improve ABI ffmpeg contract but I'm not sure if I want to go this route alone.

[Yozer]

In theory I could also push owner CUcontext, call cuCtxGetDevice and compare that device’s CU_DEVICE_ATTRIBUTE_TEXTURE_ALIGNMENT with logical device 0. But I'm not sure if its any better.

//edit
and I don't even see pyav importing cuda libs so that's probably bad idea/too complex to implement

[WyattBlue]

Yes, please add the guard. Reject cuda_stream when device_id != 0, with an error explaining that users can expose the desired physical GPU as logical device 0 through CUDA_VISIBLE_DEVICES.

I agree that modifying FFmpeg’s private struct or loading CUDA solely to reconstruct its private state would be too brittle. Supporting arbitrary device IDs cleanly should wait for an upstream FFmpeg API that accepts an external stream during device creation.

[Yozer]

done