llama : support LiquidAI LFM2 hybrid model family

[tdakhran]

Add support for LiquidAI LFM2 model family.
For more information about models, please read the blog post.

• Support hybrid LFM2-350M, LFM2-700M, and LFM2-1.2B models.
• Support LFM2-Tokenizer.
• Support ShortConv operator.
• Implement conversion to gguf, quantization.

Important
LFM2 was merged into transformers, but has not yet been released.
To convert into gguf, install transformers from source

pip install "transformers @ git+https://github.com/huggingface/transformers.git@main"

[CISC]

Impressive, very clean PR and a nice model!

[CISC]

Granite4 is ahead of you in the merge queue, so you will have to do a minor rebase after that, will give other reviewers a little time to chime in as well. :)

[tdakhran]

Granite4 is ahead of you in the merge queue, so you will have to do a minor rebase after that, will give other reviewers a little time to chime in as well. :)

Thank you for a quick review @CISC. Rebase is not an issue, I'm happy to address any feedback.

[ggerganov]

The lfm2_rms_norm seems redundant compared to just using ggml_rms_norm. In which cases is it necessary?

[tdakhran]

Thanks for review @ggerganov .

The lfm2_rms_norm seems redundant compared to just using ggml_rms_norm. In which cases is it necessary?

The reason I added upcast to f32 is that we have it in our HF RMSNorm implementation.
Are activations always in f32? I had an impression that for quantized models, e.g., f16, activations also switch (or might be switched in the future) to f16. If activations are guaranteed to be f32, then switching to lfm2_rms_norm will be numerically equivalent.

[ggerganov]

Are activations always in f32? I had an impression that for quantized models, e.g., f16, activations also switch (or might be switched in the future) to f16. If activations are guaranteed to be f32, then switching to lfm2_rms_norm will be numerically equivalent.

Activations in ggml are generally always in F32. Certain operators (such as ggml_mul_mat), depending on the backend, will internally cast the input F32 activations to whatever is needed and supported (F16, Q8, ...) in order to gain performance. But otherwise it is safe to assume that all intermediate results from ggml operations are in F32.

In the future, we might add support for keeping those in lower precision, but it's a long way until then and in any case it would not require special-handling. So no need to keep these checks.

[tdakhran]

Thank you all for your constructive comments and for maintaining the high quality of the codebase.
@ggerganov, @ngxson, @compilade, all comments are addressed, please take one more look.

[tdakhran]

The model architecture naming convention changed in HF from LFM2ForCausalLM to Lfm2ForCausalLM. I added additionally

[email protected]("Lfm2ForCausalLM")
 @ModelBase.register("LFM2ForCausalLM")
 class LFM2Model(TextModel):
     model_arch = gguf.MODEL_ARCH.LFM2

Hope it's not too late.

[paulpak58]

The model architecture naming convention changed in HF from LFM2ForCausalLM to Lfm2ForCausalLM. I added additionally

[email protected]("Lfm2ForCausalLM")
 @ModelBase.register("LFM2ForCausalLM")
 class LFM2Model(TextModel):
     model_arch = gguf.MODEL_ARCH.LFM2

Hope it's not too late.

Yeah apologies, transformers prefers to keep camel casing.

[CISC]

@paulpak58 BTW, I submitted tool call support to your chat template on HF.