How does GGML syncing work across ggml / llama.cpp / whisper.cpp repos?

[peter277]

I've noticed there appears to be a process to sync GGML changes across the three repos ggml, llama.cpp, and whisper.cpp. Would anybody be able to please refer me to or provide an explanation of how this works?

I attempted compiling this standalone ggml repo with the same GGML options I was using on llama.cpp but it failed with errors (in my case option GGML_OPENCL=ON was the problem - while recently this commit was merged into the ggml repo it has less changes than its corresponding commit in llama.cpp so it appears the synchronisation may be partial and the specific error is CMake Error at src/CMakeLists.txt:257 (add_subdirectory): add_subdirectory given source "ggml-opencl" which is not an existing directory.).

The broader problem I'm trying to solve is to compile a common GGML library that could be used across llama.cpp and whisper.cpp in my MSYS2 environment. I would potentially also share the solution back into the official MSYS2 package repositories (recently submitted an update to the llama.cpp package, but whisper.cpp is still out of date and the point of difficulty for me is the common ggml dependency). Given the apparent synchronisation of the codebase, is this easily possible if I choose the codebases for ggml/llama.cpp/whisper.cpp as at the last synchronisation point?

[ggerganov]

Would anybody be able to please refer me to or provide an explanation of how this works?

The ggml source code is synced manually across the ggml, whisper.cpp and llama.cpp repositories using the sync Bash scripts in the scripts folder of the repos. Currently, I do this every once in a while - usually once per week.

I attempted compiling this standalone ggml repo with the same GGML options I was using on llama.cpp but it failed with errors (in my case option GGML_OPENCL=ON was the problem - while recently this commit was merged into the ggml repo it has less changes than its corresponding commit in llama.cpp so it appears the synchronisation may be partial and the specific error is CMake Error at src/CMakeLists.txt:257 (add_subdirectory): add_subdirectory given source "ggml-opencl" which is not an existing directory.).

I missed that - the sync scripts need to be updated to sync the new ggml-opencl source. Will do it during the next sync.

Given the apparent synchronisation of the codebase, is this easily possible if I choose the codebases for ggml/llama.cpp/whisper.cpp as at the last synchronisation point?

As you've noticed, the ggml library currently is not versioned, so using specific ggml binaries for different projects will not have any sort of guarantees that it will be compatible, because the different projects can depend on different versions of the library. It's something that we have to fix eventually.

[peter277]

Thanks for your response.

As you've noticed, the ggml library currently is not versioned, so using specific ggml binaries for different projects will not have any sort of guarantees that it will be compatible, because the different projects can depend on different versions of the library. It's something that we have to fix eventually.

Sure, understand that version management doesn't exist presently. However, say I were to develop a joint build process that compiles ggml, whisper.cpp and llama.cpp together by:

• Starting with the last version of ggml that synchronised with llama and whisper (identified by finding the latest 'sync : llama/whisper' commit pair) and checkout llama.cpp and whisper.cpp code using commit references sync-llama.last and sync-whisper.last. Is the GGML code contained within the individual llama.cpp and whisper.cpp repos guaranteed to be exactly the same as ggml master when chosen at these reference points?
• Patching the llama.cpp and whisper.cpp CMake scripts to look for a precompiled shared GGML (which will be as at the last synchronisation point) instead of entering the ggml subdirectory to compile their own. Possibly I would need to pass through the compile defines used for the common GGML into the parent CMake process for llama and whisper in case the rest of the CMake logic depends on them.

Is this approach expected to work, or are there specific reasons it might be prone to failure? If it is expected to work, would it make sense to work towards patching the master llama.cpp and whisper.cpp CMake processes to include an option like GGML_USE_STANDALONE? This option would enable using a shared GGML library instead of compiling their own, as a step towards full modularisation of GGML (with synchronisation and compatibility checks left to the developer at this point in time).

[ggerganov]

Even if this works, it looks to me this process is very error prone, so I cannot really recommend it.

It makes sense as a first step to add option in llama.cpp and whisper.cpp to use an external ggml build. If we have that, then we can add CI workflows that would checkout a release branch of the ggml repo and would run some builds and tests with it. When these workflows fail, it would indicate that the release branch of ggml requires an update and we can go and manually synchronise it with the master branch and update the ggml version. llama.cpp already has an extensive build and test suite that we can exercise for this. However, whisper.cpp lacks any CI tests - I haven't had the time to make those yet. Any help with that is appreciated: ggerganov/whisper.cpp#2454