Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ollama创建openbmb/MiniCPM-o-2_6-gguf报错! #768

Open
twythebest opened this issue Jan 20, 2025 · 2 comments
Open

ollama创建openbmb/MiniCPM-o-2_6-gguf报错! #768

twythebest opened this issue Jan 20, 2025 · 2 comments
Assignees

Comments

@twythebest
Copy link

我下载了文件:Model-7.6B-F16.gguf和mmproj-model-f16.gguf,然后我的模型配置文件为:
FROM ./mmproj-model-f16.gguf
FROM ./Model-7.6B-F16.gguf

TEMPLATE """{{- if .Messages }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
<|im_start|>{{ .Role }}
{{ .Content }}
{{- if $last }}
{{- if (ne .Role "assistant") }}<|im_end|>
<|im_start|>assistant
{{ end }}
{{- else }}<|im_end|>
{{ end }}
{{- end }}
{{- else }}
{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ end }}{{ .Response }}{{ if .Response }}<|im_end|>{{ end }}"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

PARAMETER num_ctx 8192

但我使用命令:ollama create MiniCPM-o -f config.txt后,接着运行命令ollama run MiniCPM-o,出现报错:
Error: llama runner process has terminated: exit status 2
怎么解决???

@tc-mb
Copy link
Collaborator

tc-mb commented Jan 21, 2025

需要先合并llama.cpp官方分支,然后ollama官方更新自己的llama.cpp依赖的分支并发布新版本,然后大家才能直接用ollama跑模型。
在此之前,ollama是不能跑minicpm-omni的。大家能方便使用,需要依赖链条都是走通的才行。

@tc-mb tc-mb self-assigned this Jan 21, 2025
@Shiven-saini
Copy link

I have compiled the ollama from source code with the latest version of llama.cpp, which I think supposedly contains the pull request you made for the latest model right ? But I am also getting the same error.

Any way I can help ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants