ggml_new_tensor_impl: not enough space in the context's memory pool

[domxch]

Edit - sorry I guess this should be under Issues - I'm new to GitHub and don't see where to delete this so opening an Issue...

Hi, firstly, great work, this runs acceptably on my non GPU laptop, whereas the standard release wasn't useable at all :-)
I got fairly poor results trying to translate japanese language audio with the base model, so I tried to use the gpt-2-345M model but I got the following, even when I had 8GB+ of currently available RAM. Is there any way to configure this to work or does the larger model require a vast amount of RAM?
The "mem required" to load the model looks fine, but "ggml ctx size" = 17592179909347.84 MB?!

whisper_model_load: loading model from 'ggml-model-gpt-2-345M.bin'
whisper_model_load: n_vocab = 50257
whisper_model_load: n_audio_ctx = 1024
whisper_model_load: n_audio_state = 1024
whisper_model_load: n_audio_head = 16
whisper_model_load: n_audio_layer = 24
whisper_model_load: n_text_ctx = 1
whisper_model_load: n_text_state = 50257
whisper_model_load: n_text_head = 1
whisper_model_load: n_text_layer = 289
whisper_model_load: n_mels = 74240
whisper_model_load: f16 = 19070976
whisper_model_load: type = 4
whisper_model_load: mem_required = 2608.00 MB
whisper_model_load: adding 50257 extra tokens
whisper_model_load: ggml ctx size = 17592179909347.84 MB
ggml_new_tensor_impl: not enough space in the context's memory pool

[ggerganov]

ggml-model-gpt-2-345M.bin is a GPT-2 model. It cannot be used with Whisper.

You need to get a ggml Whisper model from here: https://huggingface.co/datasets/ggerganov/whisper.cpp

[busyyang]

Hi @ggerganov , I am trying to run the talk example, which include a ggml-gpt-2-117M.bin model. But I have a simalar error.

PS C:\GitRepos\whisper.cpp-master\build\bin\Release> .\talk.exe -mw ..\..\..\models\ggml-small.bin -mg ..\..\..\models\ggml-gpt-2-117M.bin -p Santa
whisper_init_from_file_no_state: loading model from '..\..\..\models\ggml-small.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head  = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 768
whisper_model_load: n_text_head   = 12
whisper_model_load: n_text_layer  = 12
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 3
whisper_model_load: mem required  =  743.00 MB (+   16.00 MB per decoder)
whisper_model_load: adding 1608 extra tokens
whisper_model_load: model ctx     =  464.68 MB
whisper_model_load: model size    =  464.44 MB
whisper_init_state: kv self size  =   15.75 MB
whisper_init_state: kv cross size =   52.73 MB
gpt2_model_load: loading model from '..\..\..\models\ggml-gpt-2-117M.bin'
gpt2_model_load: n_vocab = 50257
gpt2_model_load: n_ctx   = 1024
gpt2_model_load: n_embd  = 768
gpt2_model_load: n_head  = 12
gpt2_model_load: n_layer = 12
gpt2_model_load: ftype   = 1
gpt2_model_load: ggml ctx size = 384.74 MB
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 403426048, available 403425792)

Can I get any help from your side?

OS: Windows Visual Studio
with CuBLAS: YES
Memory: 64 GB
GPU Memory: 8 GB