Skip to content

TimeoutException when loading Gemma 3 4B with mmproj #85

@GijsWithagen

Description

@GijsWithagen

When trying to initialize the Gemma 3 4B model (gemma-3-4b-it-Q8_0.gguf) with the mmproj file (e.g., mmproj-BF16.gguf) on a MacBook Pro 16” M1 Max (32GB RAM), the initialization times out with:

TimeoutException (TimeoutException: Operation "model loading" timed out)

Without the mmproj, the model loads fine and can process normal text prompts, but it does not process images, which is expected.

To Reproduce:

final modelParams = ModelParams()..nGpuLayers = -1;
    final contextParams = ContextParams()
      ..nPredict = -1
      ..nCtx = 4096
      ..nBatch = 1024;
    final samplerParams = SamplerParams()
      ..temp = 0.25
      ..topP = 0.90;

final loadCommand = LlamaLoad(
    path: "PATH_TO_MODEL",
    modelParams: modelParams,
    contextParams: contextParams,
    samplingParams: samplerParams,
    mmprojPath: "PATH_TO_MMPROJ_MODEL",
 );

final parent = LlamaParent(loadCommand);
await parent.init(); // <- TimeoutException occurs here

The code looks similar to the given examples, so i don't know what I'm doing wrong.

I got the models here

Expected behavior
The model should load successfully with the mmproj file and be able to process image prompts.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions