Optimize RF-DETR segmentation pipeline

[aseembits93]

What does this PR do?

This is the top PR in the RF-DETR optimization stack:

main <- opt-python-postproc <- opt-preprocess <- opt-pipeline-integration

It integrates the optimized Triton postprocess and preprocess branches into the streaming workflow path and removes the remaining CPU/GPU scheduling bubbles visible in Nsight. End-to-end numbers for this PR represent the full stack: Triton postproc, Triton preproc, CUDA graph execution, depth-2 pipeline scheduling, sparse RLE-to-polygon conversion, and workflow response-path optimizations.

Key changes:

• Adds depth-2 RF-DETR pipeline scheduling where CPU frame preparation overlaps with GPU inference/postprocess.
• Submits postprocess GPU work eagerly when the RF-DETR future reaches adapter postprocess, before CPU response finalization.
• Carries response finalization through futures so CPU polygon/workflow conversion does not block the inference thread from submitting the next frame.
• Adds sparse RLE-to-polygon conversion that materializes only a tight foreground crop for cv2.findContours.
• Adds faster workflow/supervision conversion for polygon instance-segmentation responses.

Type of Change

• Other: Performance improvement

Testing

• I have tested this change locally
• I have added/updated tests or benchmark coverage for this change

Test details:

• Full-stack gains on TensorRT video input

Reference command on main:

ENABLE_AUTO_CUDA_GRAPHS_FOR_TRT_BACKEND=false \
  INFERENCE_MODELS_RFDETR_TRITON_PREPROC_ENABLED=false \
  INFERENCE_MODELS_RFDETR_TRITON_POSTPROC_ENABLED=false \
  RFDETR_PIPELINE_DEPTH=1 \
  PYTHONPATH=<repo>:<repo>/inference_models \
  python development/stream_interface/rfdetr_nano_seg_trt_workflow.py \
    --video_reference <video> \
    --backend trt

Candidate command on opt-pipeline-integration:

ENABLE_AUTO_CUDA_GRAPHS_FOR_TRT_BACKEND=true \
  INFERENCE_MODELS_RFDETR_TRITON_PREPROC_ENABLED=true \
  INFERENCE_MODELS_RFDETR_TRITON_POSTPROC_ENABLED=true \
  RFDETR_PIPELINE_DEPTH=2 \
  PYTHONPATH=/app/helloworld/inference:/app/helloworld/inference/inference_models \
  python development/stream_interface/rfdetr_nano_seg_trt_workflow.py \
    --video_reference <video> \
    --backend trt

vehicles_312px.mp4 (538 frames, src 312x176):

	fps	elapsed	ms/frame
main reference	34.58	15.56 s	28.92
full stack	90.60	5.94 s	11.04
Delta	+162.0%	-9.62 s	-17.88 ms

vehicles_720p.mp4 (538 frames, src 1280x720):

	fps	elapsed	ms/frame
main reference	18.25	29.48 s	54.79
full stack	86.66	6.21 s	11.54
Delta	+374.8%	-23.27 s	-43.25 ms

vehicles_1080p.mp4 (538 frames, src 1920x1080):

	fps	elapsed	ms/frame
main reference	10.76	49.98 s	92.94
full stack	82.72	6.50 s	12.08
Delta	+668.8%	-43.48 s	-80.86 ms

• Kernel correctness on 1000 same-shape COCO val2017 images vs main flags-off

env PARITY_MODEL_PATH=/app/helloworld/inference/rfdetr-seg-nano-orin-trt-package \
  PYTHONPATH=/app/helloworld/inference:/app/helloworld/inference/inference_models \
  python development/stream_interface/rfdetr_coco_same_shape_parity.py \
    --base-ref main \
    --candidate-ref opt-pipeline-integration \
    --height 480 \
    --width 640 \
    --image-count 1000

	main flags-off	opt-pipeline-integration optimized kernels
Images	1000	1000
Detections	5959	5959
Matched same-class IoU > 0.5	5959 (100.00%)	—
Count-mismatch images	0	—
Class-id disagreements	0	—
Mean / min box IoU	0.999992 / 0.981132	—
Mean / max |Δscore|	0.000e+00 / 0.000e+00	—
Mean / min mask IoU	0.999497 / 0.000000	—
Byte-identical RLEs	5954 / 5959	—

• Full workflow video parity on 1080p workload vs main flags-off

env PARITY_MODEL_PATH=/app/helloworld/inference/rfdetr-seg-nano-orin-trt-package \
  PYTHONPATH=/app/helloworld/inference:/app/helloworld/inference/inference_models \
  python development/stream_interface/rfdetr_workflow_video_parity.py \
    --video_reference vehicles_1080p.mp4 \
    --base-ref main \
    --candidate-ref opt-pipeline-integration

This compares serialized workflow sink outputs frame by frame, ignoring generated detection IDs and matching detections by same class and box IoU > 0.5.

	main flags-off	opt-pipeline-integration full stack
Frames	538	538
Detections	1962	1962
Matched detections	1962 (100.00%)	—
Count-mismatch frames	0	—
Unmatched base / candidate detections	0 / 0	—
Mean / min box IoU	0.993391 / 0.552967	—
Mean / max |Δscore|	1.021e-03 / 2.236e-01	—
Polygon point-count diffs	38	—

• RLE-to-polygon replay microbench from captured e2e inputs

env PYTHONPATH=/app/helloworld/inference:/app/helloworld/inference/inference_models \
  python development/stream_interface/rfdetr_rle_to_poly_microbenchmark.py \
    --mode replay \
    --cases-dir <captured-cases-dir> \
    --repeats 3 \
    --warmup-repeats 1

Each row uses 100 captured calls and 300 timed replays.

captured case set	total masks in capture	mean	p50	p90	p99	correctness
vehicles_312px	525	0.705 ms	0.674 ms	0.950 ms	0.987 ms	matched captured polygons
vehicles_720p	533	1.171 ms	1.170 ms	1.658 ms	1.703 ms	matched captured polygons
vehicles_1080p	534	1.131 ms	0.902 ms	2.335 ms	3.865 ms	matched captured polygons

• Focused unit tests

PYTHONPATH=/app/helloworld/inference:/app/helloworld/inference/inference_models \
  python -m pytest -q \
    tests/inference/unit_tests/core/interfaces/stream/test_workflows.py \
    tests/inference/unit_tests/core/interfaces/stream/test_interface_pipeline.py::test_inference_pipeline_drain_enqueues_flush_results_with_bound_frames \
    tests/inference/unit_tests/core/interfaces/stream/test_interface_pipeline.py::test_resolve_prediction_futures_recursively_resolves_nested_values \
    tests/inference/unit_tests/core/interfaces/stream/test_interface_pipeline.py::test_inference_pipeline_close_calls_handler_close_hook \
    tests/inference/unit_tests/core/models/test_inference_models_adapters.py \
    tests/inference/unit_tests/core/utils/test_rle_to_polygon.py

Result: 17 passed.

PYTHONPATH=/app/helloworld/inference/inference_models:/app/helloworld/inference \
  python -m pytest -q tests/unit_tests/test_configuration.py \
    tests/unit_tests/models/test_instance_segmentation_future.py

Result: 16 passed.

Latest 1080p review-cleanup verification:

ENABLE_AUTO_CUDA_GRAPHS_FOR_TRT_BACKEND=true \
  INFERENCE_MODELS_RFDETR_TRITON_PREPROC_ENABLED=true \
  INFERENCE_MODELS_RFDETR_TRITON_POSTPROC_ENABLED=true \
  RFDETR_PIPELINE_DEPTH=2 \
  PYTHONPATH=/app/helloworld/inference/inference_models:/app/helloworld/inference \
  python development/stream_interface/rfdetr_nano_seg_trt_workflow.py \
    --video_reference vehicles_1080p.mp4 \
    --backend trt

Result: frames=538 elapsed=6.50s fps=82.72.

Note: after rebasing onto newer main, the workflow benchmark explicitly sets enforce_dense_masks_in_inference_models=False so the optimized sparse/RLE postprocess path is measured instead of the newer dense-mask workflow default.

How It Works

Pipeline scheduling

With RFDETR_PIPELINE_DEPTH=2, depth means one CPU stage and one GPU stage in flight. CPU prepares frame N, submits GPU work for frame N, immediately prepares/submits frame N+1, and only then finalizes the older response. The key ordering fix is submitting postprocess GPU work before releasing an older response for CPU finalization.

Sparse RLE to polygon

The old response path decoded each COCO RLE into a full H x W dense mask before calling cv2.findContours. The new path parses uncompressed RLE counts into foreground intervals, materializes only the tight foreground crop, and calls OpenCV with an offset so output polygon coordinates match the legacy dense path.

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• My changes generate no new runtime errors in the tested paths above
• I have updated the documentation accordingly (if applicable)

Additional Context

This PR intentionally does not introduce new Triton preproc/postproc kernels; those are in the two lower PRs. This branch wires them into the stream pipeline and optimizes the remaining CPU response path so the GPU queue stays close to full.

[misrasaurabh1]

is this being used elsewhere? is this a breaking change?

[misrasaurabh1]

this is resolved

[misrasaurabh1]

these changes are unncessary, can be removed to reduce diff. alongside the RoboflowModelHandler class

[misrasaurabh1]

need helpful comments here