Support for llava_next Architecture in LLM-AWQ (Issue with Quantizing llava-hf/llava-v1.6-mistral-7b-hf)

[ShobhaRajanna]

Hello! I am working on quantizing the llava-hf/llava-v1.6-mistral-7b-hf model, which uses the llava_next architecture. However, I encountered an issue when attempting to integrate it with llm-awq. It seems llava_next isn't directly supported, resulting in the following errors during quantization:
You are using a model of type llava_next to instantiate a model of type llava_llama. This is not supported for all configurations of models and can yield errors.

This issue likely arises from llava_next being a newer architecture with unique vision components. Since llava_next models handle both text and images, it would be great if llm-awq could support its architecture or provide guidance for a workaround.

Steps to Reproduce:

Load the llava-hf/llava-v1.6-mistral-7b-hf model in llm-awq.
Attempt to quantize it using the standard AWQ process.
Encounter the architecture compatibility error.

Thank you in advance for any assistance or guidance!