Llava Models

[choochtech]

Does the Llava part work ?

https://github.com/intel/intel-extension-for-transformers/tree/main/intel_extension_for_transformers/transformers/modeling/llava_models

If so are they optimized for Intel Device and are there any examples ?

Thanks for building this library. I have seen the token generation performance to be very good compared to OpenVino.

Great work !

Thanks

[a32543254]

Thanks for your usage!!!
Unfortunately, we are not support LLaVA for now.
But we may consider to support it in future, please stay tune.

Regrads,
Bo

[kevinintel]

It's for multi-model training, but optimization is WIP.

[choochtech]

@kevinintel how do you do the optimization for the llava model and use it ?

[kevinintel]

Someone tried low-bits for llava: https://arxiv.org/pdf/2306.00978.pdf and we will try to quantize it.

[choochtech]

Thanks @kevinintel

[WeiweiZhang1]

Hi, support for quantization of multimodal models is currently planned, and any updates will be communicated here.

[kevinintel]

we can optimize llava in intel/neural-compressor#1797
will add examples