-
Notifications
You must be signed in to change notification settings - Fork 9.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
support MiniCPM-V-2.6 #8967
support MiniCPM-V-2.6 #8967
Conversation
sync master
sync master
examples/llava/minicpmv-convert/minicpmv2_6-convert-image-encoder-to-gguf.py
Outdated
Show resolved
Hide resolved
waiting for merge |
waiting for merge |
waiting for merge |
I have opened an issue 9066 where I experienced a crash after this pull request was merged. The crash was unrelated to this miniCPM-V-2.6 model. I hope you can reproduce the error |
Hello, I saw that the issue you mentioned was that llava would crash, but my update only involves the part of minicpmv. Although I am not sure about the issue problem, I feel that it may not be the problem with this branch. |
@tc-mb Can we use mini cpm with context cache ? So that we upload image once and ask for multiple question referring to the same image ? |
Yes, it's now storing cache. You can run in interactive mode to ask multiple rounds of questions.
or modify the minicpmv-cli function (which is more like an example) to achieve the functionality you want. |
Eagerly awaiting... |
if args.text_only: | ||
fname_middle = "text-" | ||
has_vision_encoder = False | ||
elif args.minicpmv_projector is not None: | ||
fname_middle = "mmproj-" | ||
has_text_encoder = False | ||
has_minicpmv_projector = True | ||
minicpmv_version = 3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this line necessary? It overrides minicpmv_version
value set in the command line when converting MiniCPM-V2.5 which results in a broken mmproj-model-f16.gguf.
cool, thats a great feature, thanks @tc-mb |
Very cool! Are GPU operations supported at this time? |
I have tested in Ubuntu + Nvidia(4090), it is available and speed looks good. You can use it in the following way.
|
Awesome, thanks! |
@tc-mb |
Sorry, I didn't test the server method when I updated it, I will support this capability in the near future. |
Dear llama.cpp Official,
Hi, I'm writing to address our new PR submission for integrating our model MiniCPM-V 2.6 into llama.cpp. MiniCPM-V 2.6 is the latest and most capable model in the MiniCPM-V series. This model is stronger and supports multi-images understanding and video understanding.
This version of the model supports video understanding, and I have implemented functions such as video frame extraction in my fork version. However, because ffmpeg is introduced, there may be many environment and compilation issues in other devices. Therefore, I think it can be divided into multiple PR submissions.
Best regards,
MiniCPM-V Official ^_^