Remove Float8Linear from quant_api.py

[danielvegamyhre]

Fixes #3069

Discussed issue with @vkuzo offline, references to Float8Linear in quant_api.py conversion code are technical debt that can be removed.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3085

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 5e88d65 with merge base d2fae7a ():

NEW FAILURES - The following jobs have failed:

• Run 1xH100 Tests / test (H100, linux.aws.h100, --pre torch torchvision torchaudio fbgemm-gpu-genai --index-url https... / linux-job (gh)
  test/float8/test_base.py::TestFloat8Linear::test_quantize
• Run 1xL4 Tests / test (SM-89, linux.g6.4xlarge.experimental.nvidia.gpu, --pre torch --index-url https://download.p... / linux-job (gh)
  test/float8/test_base.py::TestFloat8Linear::test_quantize

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[vkuzo]

lg if CI is green

[jerryzh168]

I don't think it's as simple as just removing it? @jainapurva might have more context on why this is needed IIRC

[jainapurva]

I don't think it's as simple as just removing it? @jainapurva might have more context on why this is needed IIRC

This code was added to enable quantization using quantize_ api of torchao's fp8 trained model, which is a Float8Linear Tensor. To remove this, we need to test the following flow: TorchAO's Fp8 pre-trained model (model will have Float8Linear layer), then quantize the model using quantize_ api.

[danielvegamyhre]

I don't think it's as simple as just removing it? @jainapurva might have more context on why this is needed IIRC

This code was added to enable quantization using quantize_ api of torchao's fp8 trained model, which is a Float8Linear Tensor. To remove this, we need to test the following flow: TorchAO's Fp8 pre-trained model (model will have Float8Linear layer), then quantize the model using quantize_ api.

The code being removed dequantizes the model (swaps Float8Linear -> nn.Linear) using the quantize_ api. Why do we need the quantize_ api to do this?

[jainapurva]

The quantize_ api would only worked for linear layers, hence first we used to convert Float8Linear into Linear layers, and then those linear layers would be quantized

…

On Mon, Sep 29, 2025 at 9:49 PM Daniel Vega-Myhre ***@***.***> wrote: *danielvegamyhre* left a comment (pytorch/ao#3085) <#3085 (comment)> I don't think it's as simple as just removing it? @jainapurva <https://github.com/jainapurva> might have more context on why this is needed IIRC This code was added to enable quantization using quantize_ api of torchao's fp8 trained model, which is a Float8Linear Tensor. To remove this, we need to test the following flow: TorchAO's Fp8 pre-trained model (model will have Float8Linear layer), then quantize the model using quantize_ api. The code being removed *dequantizes* the model (swaps Float8Linear -> nn.Linear) using the quantize_ api. Why do we need the quantize_ api to do this? — Reply to this email directly, view it on GitHub <#3085 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEVCDALVPE2ODJKSYBRXBYD3VIDU7AVCNFSM6AAAAACHUIT6WCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGNBZHE3DMNRVGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[vkuzo]

It does not make sense to me to put workflow-specific logic in a general utility such as _replace_with_custom_fn_if_matches_filter. IMO we should land this PR to remove the technical debt and re-enable the "quantize a float8 trained model for inferen) logic in a follow-up PR, if needed.

[danielvegamyhre]

It does not make sense to me to put workflow-specific logic in a general utility such as _replace_with_custom_fn_if_matches_filter. IMO we should land this PR to remove the technical debt and re-enable the "quantize a float8 trained model for inferen) logic in a follow-up PR, if needed.

I agree with this.... I will look into the test failures and fix before landing