Optimize RF-DETR Triton preprocessing

[aseembits93]

What does this PR do?

This is the middle PR in the RF-DETR optimization stack:

main <- opt-python-postproc <- opt-preprocess <- opt-pipeline-integration

It adds the optimized RF-DETR Triton preprocessing path on top of the Triton RLE postprocess branch. End-to-end numbers for this PR therefore represent the cumulative postproc + preproc gain.

The preprocessing fast path is guarded by:

INFERENCE_MODELS_RFDETR_TRITON_PREPROC_ENABLED=true

The implementation uses a two-pass Triton resize/normalize flow:

• pass 1: PIL-compatible antialias horizontal resize into a uint8 CHW scratch buffer
• pass 2: vertical resize + /255 + normalization into the TensorRT input tensor

The low-level kernels live in triton_preprocess.py. The TensorRT adapter delegates fast-path eligibility, reusable buffer state, warning throttling, and CUDA event handoff to FastPreprocessRuntime in triton_preprocess_runtime.py. That runtime caches resample tables, pinned host input, device input, scratch buffers, and a ring of output tensors so later pipelined execution can reuse buffers safely. The fast path remains opt-in and falls back to the reference path with a RuntimeWarning when the request is outside the supported Triton contract.

Type of Change

• Other: Performance improvement

Testing

• I have tested this change locally
• I have added/updated tests and benchmark coverage for this change

Test details: (Make sure Triton is installed in your environment)

• Cumulative postproc + preproc gains on TensorRT video input

Reference command on main:

env -u RFDETR_NSIGHT_MARKERS \
  INFERENCE_MODELS_RFDETR_TRITON_POSTPROC_ENABLED=false \
  INFERENCE_MODELS_RFDETR_TRITON_PREPROC_ENABLED=false \
  ENABLE_AUTO_CUDA_GRAPHS_FOR_TRT_BACKEND=false \
  RFDETR_PIPELINE_DEPTH=1 \
  PYTHONPATH=<repo>:<repo>/inference_models \
  python development/stream_interface/rfdetr_nano_seg_trt_workflow.py \
    --video_reference <video> \
    --backend trt

Candidate command on opt-preprocess:

env -u RFDETR_NSIGHT_MARKERS \
  INFERENCE_MODELS_RFDETR_TRITON_POSTPROC_ENABLED=true \
  INFERENCE_MODELS_RFDETR_TRITON_PREPROC_ENABLED=true \
  ENABLE_AUTO_CUDA_GRAPHS_FOR_TRT_BACKEND=false \
  RFDETR_PIPELINE_DEPTH=1 \
  PYTHONPATH=<repo>:<repo>/inference_models \
  python development/stream_interface/rfdetr_nano_seg_trt_workflow.py \
    --video_reference <video> \
    --backend trt

vehicles_312px.mp4 (538 frames, src 312x176):

	fps	elapsed	ms/frame
main reference	34.58	15.56 s	28.92
Triton postproc + preproc	50.00	10.76 s	20.00
Delta	+44.6%	-4.80 s	-8.92 ms

vehicles_720p.mp4 (538 frames, src 1280x720):

	fps	elapsed	ms/frame
main reference	18.25	29.48 s	54.79
Triton postproc + preproc	29.97	17.95 s	33.37
Delta	+64.2%	-11.53 s	-21.42 ms

vehicles_1080p.mp4 (538 frames, src 1920x1080):

	fps	elapsed	ms/frame
main reference	10.76	49.98 s	92.94
Triton postproc + preproc	18.99	28.33 s	52.66
Delta	+76.5%	-21.65 s	-40.28 ms

• Correctness on 1000 same-shape COCO val2017 images vs main flags-off

env PARITY_MODEL_PATH=<repo>/rfdetr-seg-nano-orin-trt-package \
  PYTHONPATH=<repo>:<repo>/inference_models \
  python development/stream_interface/rfdetr_coco_same_shape_parity.py \
    --base-ref main \
    --candidate-ref opt-preprocess \
    --coco-dir <repo>/coco/val2017 \
    --height 480 \
    --width 640 \
    --image-count 1000 \
    --warmup-frames 10

	main flags-off	opt-preprocess
Images	1000	1000
Detections	5959	5959
Matched same-class IoU > 0.5	5959 (100.00%)	—
Count-mismatch images	0	—
Class-id disagreements	0	—
Mean / min box IoU	0.999992 / 0.981132	—
Mean / max |Δscore|	0.000e+00 / 0.000e+00	—
Mean / min mask IoU	0.999497 / 0.000000	—
Byte-identical RLEs	5954 / 5959	—

• Preprocess microbench replay from captured e2e inputs

Triton replay exercises the production FastPreprocessRuntime helper used by the TRT adapter; the harness only materializes captured inputs and compares outputs.

env PYTHONPATH=<repo>:<repo>/inference_models \
  python development/stream_interface/rfdetr_preprocess_microbenchmark.py \
    --mode replay \
    --cases-dir <captured-cases-dir> \
    --replay-implementation <reference|triton> \
    --repeats 3 \
    --warmup-repeats 1 \
    --device captured \
    --atol 1e-6

Each row uses 100 captured calls and 300 timed replays per implementation. These numbers were rerun after replay was switched to the production FastPreprocessRuntime helper.

captured case set	reference mean / p50 / p90 / p99	Triton mean / p50 / p90 / p99	mean speedup
vehicles_312px	3.204 / 3.128 / 3.518 / 4.089 ms	0.745 / 0.707 / 0.916 / 0.971 ms	4.30x
vehicles_720p	14.957 / 14.837 / 15.084 / 17.787 ms	1.151 / 1.104 / 1.421 / 1.495 ms	12.99x
vehicles_1080p	30.697 / 29.415 / 34.063 / 35.713 ms	1.884 / 1.837 / 2.098 / 2.180 ms	16.29x

All Triton replay outputs matched captured reference outputs at atol=1e-6.

• Unit tests and compile checks

python -m py_compile \
  development/stream_interface/rfdetr_coco_same_shape_parity.py \
  development/stream_interface/rfdetr_preprocess_microbenchmark.py \
  inference_models/inference_models/configuration.py \
  inference_models/inference_models/models/rfdetr/rfdetr_instance_segmentation_trt.py \
  inference_models/inference_models/models/rfdetr/triton_preprocess.py \
  inference_models/inference_models/models/rfdetr/triton_preprocess_runtime.py \
  inference_models/tests/integration_tests/models/test_rfdetr_seg_predictions_trt.py \
  inference_models/tests/unit_tests/models/rfdetr/test_triton_preprocess.py \
  inference_models/tests/unit_tests/models/rfdetr/test_trt_preprocess_fast_path.py

PYTHONPATH=<repo>:<repo>/inference_models \
  python -m pytest -q \
    inference_models/tests/unit_tests/models/rfdetr/test_pre_processing.py \
    inference_models/tests/unit_tests/models/rfdetr/test_triton_preprocess.py \
    inference_models/tests/unit_tests/models/rfdetr/test_trt_preprocess_fast_path.py

RFDETR_SEG_TRT_PACKAGE_PATH=<repo>/rfdetr-seg-nano-orin-trt-package \
  PYTHONPATH=<repo>:<repo>/inference_models \
  python -m pytest -q \
    inference_models/tests/integration_tests/models/test_rfdetr_seg_predictions_trt.py::test_trt_triton_preprocess_output_matches_reference_preprocess

Results: unit coverage 40 passed; model-level Triton preproc parity 1 passed
with the local Orin TRT package override. The default T4 fixture skips cleanly
on this Orin runtime due TensorRT serialized-engine platform mismatch.

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• My changes generate no new runtime errors in the tested paths above
• I have updated the documentation accordingly (if applicable): changelog updated

Additional Context

Relative to the original Triton preproc template PR, this branch is no longer a single-kernel implementation. It uses a two-pass resize/normalize path because profiling showed better memory behavior and simpler preallocation. The fast-path gate is conservative and rejects custom image-size overrides, dataset-version resize, enabled static crop / contrast / grayscale, non-3-channel inputs, missing normalization, unsupported scaling factors, unsupported resize modes, non-numpy inputs, and batch sizes above 1.

[misrasaurabh1]

add a comment saying that to support these modes, we can write more triton kernels

[aseembits93]

explain with better docstrings