-
Notifications
You must be signed in to change notification settings - Fork 48
Open
Description
When trying to initialize the Gemma 3 4B model (gemma-3-4b-it-Q8_0.gguf) with the mmproj file (e.g., mmproj-BF16.gguf) on a MacBook Pro 16” M1 Max (32GB RAM), the initialization times out with:
TimeoutException (TimeoutException: Operation "model loading" timed out)
Without the mmproj, the model loads fine and can process normal text prompts, but it does not process images, which is expected.
To Reproduce:
final modelParams = ModelParams()..nGpuLayers = -1;
final contextParams = ContextParams()
..nPredict = -1
..nCtx = 4096
..nBatch = 1024;
final samplerParams = SamplerParams()
..temp = 0.25
..topP = 0.90;
final loadCommand = LlamaLoad(
path: "PATH_TO_MODEL",
modelParams: modelParams,
contextParams: contextParams,
samplingParams: samplerParams,
mmprojPath: "PATH_TO_MMPROJ_MODEL",
);
final parent = LlamaParent(loadCommand);
await parent.init(); // <- TimeoutException occurs here
The code looks similar to the given examples, so i don't know what I'm doing wrong.
I got the models here
Expected behavior
The model should load successfully with the mmproj file and be able to process image prompts.
Metadata
Metadata
Assignees
Labels
No labels