[Model] Qwen3TTS

[ashok-arora]

Detailed description of the requested feature

Support for quantization and deployment of Qwen3-TTS-style models within the NVIDIA optimization stack, ideally including compatibility with TensorRT-LLM or a clearly defined alternative pipeline.

Specifically, the request is for:

Ability to quantize non-Transformer / non-text-generation models (e.g., TTS pipelines) using a unified workflow similar to LLMs
Support for multi-component models, including:
text encoder (Transformer-based)
acoustic model (autoregressive / diffusion / codec-based)
vocoder (CNN-based waveform generator)

End-to-end export pipeline:

PyTorch → Quantization → ONNX → TensorRT engine(s)
Guidance or tooling for:
handling models not implemented in Hugging Face Transformers
exporting models with custom forward passes or generation loops
Optional: partial support for prefill/decode-style optimization where applicable (e.g., transformer submodules)

This would enable efficient deployment of modern TTS systems on NVIDIA GPUs with reduced latency and memory usage.

Describe alternatives you've considered

1. torch AO library

Target hardware/use case

1. NVIDIA GPUs (eg. A5000, etc.)

[ChenhanYu]

Triage Analysis

Field	Value
Classification	Feature Request
Severity	High
Complexity	Complex
Auto-fixable	No

Summary

Request to extend NVIDIA Model Optimizer to support quantization and deployment of Qwen3-TTS and other non-Transformer, multi-component models (text encoders, acoustic models, vocoders). Currently, the toolkit is primarily optimized for LLM/text-generation models and needs unified support for TTS pipelines with PyTorch→Quantization→ONNX→TensorRT end-to-end export.

Root Cause / Approach

Model Optimizer currently assumes Transformer-based text-generation models. Multi-component models like TTS pipelines with diverse architectures (CNN vocoders, autoregressive/diffusion acoustic models, Transformer text encoders) lack: (1) flexible quantization configs beyond LLM patterns, (2) multi-component model handling in export pipelines, (3) custom forward pass/generation loop support, (4) guidance for non-Hugging-Face-Transformers models.

Suggested Fix

1. Extend QuantizeConfig to support per-component quantization profiles and flexible module filtering beyond wildcard patterns. 2) Create a multi-component model abstraction/registry system to handle composite models with separate quantization/export pipelines for each component. 3) Add generic export pipeline in unified_export_hf.py or new unified_export.py to handle arbitrary PyTorch→ONNX→TensorRT without HF Transformers dependency. 4) Implement component-aware calibration that can forward data through heterogeneous model architectures. 5) Add TTS-specific examples and documentation showing quantization of text encoders, acoustic models, and vocoders independently and in-pipeline. 6) Provide helper utilities for exporting models with custom forward passes and generation loops.

Relevant Files

• modelopt/torch/quantization/config.py
• modelopt/torch/quantization/model_quant.py
• modelopt/torch/export/unified_export_hf.py
• modelopt/torch/export/model_utils.py
• modelopt/onnx/export/base_exporter.py
• modelopt/onnx/quantization/quantize.py
• modelopt/torch/quantization/plugins/diffusion/diffusers.py
• examples/diffusers/quantization/quantize.py

---

Auto-triaged by pensieve /magic-triage