Update Llama.cpp & Llama 3 Support

[PSchmiedmayer]

Update Llama.cpp & Llama 3 Support

⚙️ Release Notes

• Updates llama.cpp
• Supports Llama 3

📝 Code of Conduct & Contributing Guidelines

By submitting creating this pull request, you agree to follow our Code of Conduct and Contributing Guidelines:

• I agree to follow the Code of Conduct and Contributing Guidelines.

[PSchmiedmayer]

@vishnuravi @philippzagar Feel free to add to this WIP here after lifting llama.cpp. @vishnuravi can you share some of the issues you had with Llama3 so we can document this here in case this gets stale or someone else wants to pick it up?

[vishnuravi]

@vishnuravi @philippzagar Feel free to add to this WIP here after lifting llama.cpp. @vishnuravi can you share some of the issues you had with Llama3 so we can document this here in case this gets stale or someone else wants to pick it up?

Hi @PSchmiedmayer, I'm running into the same issue documented here: ollama/ollama#3759 and here: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/discussions/4. The model does not stop at the stop token and continues generating indefinitely.

[vishnuravi]

@vishnuravi @philippzagar Feel free to add to this WIP here after lifting llama.cpp. @vishnuravi can you share some of the issues you had with Llama3 so we can document this here in case this gets stale or someone else wants to pick it up?

Hi @PSchmiedmayer, I'm running into the same issue documented here: ollama/ollama#3759 and here: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/discussions/4. The model does not stop at the stop token and continues generating indefinitely.

I was able to resolve this issue by using a different GGUF with the end token correctly configured.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 91 lines in your changes are missing coverage. Please review.

Project coverage is 31.18%. Comparing base (cbaf204) to head (98753fc).

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #55      +/-   ##
==========================================
- Coverage   32.03%   31.18%   -0.85%     
==========================================
  Files          67       67              
  Lines        2932     3012      +80     
==========================================
  Hits          939      939              
- Misses       1993     2073      +80     

Files	Coverage Δ
...ces/SpeziLLMLocal/LLMLocalSession+Generation.swift	0.00% <0.00%> (ø)
...s/SpeziLLMLocal/LLMLocalSession+Tokenization.swift	0.00% <0.00%> (ø)
...ocal/Configuration/LLMLocalContextParameters.swift	0.00% <0.00%> (ø)
...Download/LLMLocalDownloadManager+DefaultUrls.swift	0.00% <0.00%> (ø)
...peziLLMLocal/LLMLocalSchema+PromptFormatting.swift	0.00% <0.00%> (ø)

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cbaf204...98753fc. Read the comment docs.