Allow compressed-tensors quantized model to be trained

[horheynm]

What does this PR do?

Using HFQuantizer, models that were quantized using compressed-tensors can be loaded.
Here, we fix the problem where the above models can also be trained.

Who can review?

@SunMarc @younesbelkada

[SunMarc]

Thanks for the PR ! Left a few suggestion. Could you explain a bit more how you are performing training with compressed-tensors models if you are not using peft ? Are you maybe doing qat or just adding custom lora layers by yourself ?

[horheynm]

@SunMarc
Yes we are quantizing the model using oneshot from compressed-tensors, loading that model using AutoModelForCausalLM and HFQuantizer. Once loaded we will be running the training - qat finetuning. The 'quantization' we run is fakequant.

We are not using LoRA adapters