You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Instead of a single model file, ramalama will need to mount and pull a directory. It would be really nice to have ramalama pull the/model --format safetensors or something like that
The text was updated successfully, but these errors were encountered:
Great idea, if we're using vllm (or others that support safe tensor) as the runtime we can mount the folder as is. If we are using llama.cpp we can convert the safetensors to a gguf then mount the file as usual!
I agree we should implement this. I would say even drop the "--format safetensors", I think it would not be too hard to automatically detect a certain model is safetensors.
Instead of a single model file, ramalama will need to mount and pull a directory. It would be really nice to have
ramalama pull the/model --format safetensors
or something like thatThe text was updated successfully, but these errors were encountered: