[None][WIP-feat] Add NanoJet backend for AutoDeploy prefill-only FP8 inference

[wejoncy]

Description

This PR adds NanoJet as an optional TensorRT-LLM AutoDeploy backend for prefill-only inference with pre-quantized FP8 models.

Five custom ops and five post-load graph transformations route supported AutoDeploy graph patterns to NanoJet kernels:

• FlashAttention-3 with fused FP8 output quantization in the kernel epilogue.
• Fused FP8 QKV projection, Q/K RMSNorm, and RoPE.
• RMSNorm with FP8 quantization in the kernel epilogue.
• Gated SwiGLU FP8 GEMM fusing the gate/up projections, SiLU, multiply, and output quantization.
• FP8 projection fused with an in-place residual add.

NanoJet is loaded lazily and applies tuned-kernel configurations through its integration contract. All NanoJet transformations are disabled by default, and changes to existing files are additive, so existing backends and installations without NanoJet retain their current behavior.

Benchmark

Benchmark setup:

• GPU: NVIDIA H100 NVL.
• Model: Qwen3-Embedding-0.6B FP8.
• Workload: 64 sequences x 1,024 tokens.
• Total tokens: 65,536.
• Sequence-length range: 1,024-1,024.
• Measurement: 1 warmup iteration and 3 measured iterations; per-forward latency.

Backend	Latency
trtllm-nanojet	102.05 ms
trtllm-flashinfer	203.48 ms
trtllm-default	200.37 ms

The NanoJet backend delivers 49.85% lower latency than FlashInfer (1.99x speedup) and 49.07% lower latency than the default backend (1.96x speedup).

Test Coverage

This PR adds unit coverage for:

• NanoJet configuration validation and plain-prefill acceptance.
• Lazy dependency loading and behavior when NanoJet is not requested.
• Runtime rejection of decode, continuation-prefill, and shared-KV paths.
• Fused QKV projection, Q/K norm, and RoPE graph-matching and shape guards.
• Head layout, projection schema, RoPE width, and graph-buffer naming guards.
• Residual-folding correctness, including aliasing, dtype, shape, reader-order, and backend checks.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.