ggerganov llama.cpp · Discussions · GitHub

Categories

Community guidelines

Discussions

You must be logged in to vote

GGML_ASSERT(seq_id < n_tokens && "seq_id cannot be larger than n_tokens with pooling_type == CLS") failed (also MEAN)

Morphus1 asked Aug 12, 2024 in Q&A · Unanswered

0
You must be logged in to vote

Json schema with integer field always returns 0 as value

arpitjjw asked Aug 12, 2024 in Q&A · Closed · Unanswered

0
You must be logged in to vote

[SYCL][Intel GPU] Long Term Features & Issues Tracking
Intel GPU
airMeng started Feb 2, 2024 in General

93
You must be logged in to vote

Exposing the LoRA merge/export as a libary public function

cyanic-selkie started Aug 11, 2024 in Ideas

0
You must be logged in to vote

Why the answer it gives looks like this?

XinyuGroceryStore asked Aug 10, 2024 in Q&A · Unanswered

2
You must be logged in to vote

Can llama-cli let me choose tokens one by one?

42e6 asked Aug 10, 2024 in Q&A · Unanswered

0
You must be logged in to vote

How to get data from one web while deploying LLMs locally?

XinyuGroceryStore asked Aug 8, 2024 in Q&A · Unanswered

1
You must be logged in to vote

KV Cache and Context Shifting

hentaitaku asked Aug 10, 2024 in Q&A · Unanswered

0
You must be logged in to vote

Stop and Continue response like llmstudio

electro199 started Aug 10, 2024 in General

0
You must be logged in to vote

Why when Flash Attention enabled ,the model is loaded to CPU and inferring process run on CPU ?

hzgdeerHo asked Aug 7, 2024 in Q&A · Unanswered

3
You must be logged in to vote

How to cache system prompt?

arpitjjw asked Aug 9, 2024 in Q&A · Answered

5
You must be logged in to vote

Support for OPT-families

daminho asked Aug 10, 2024 in Q&A · Unanswered

0
You must be logged in to vote

How to use the gguf-split / Model sharding demo
split GGUF split model sharding
phymbert started Mar 31, 2024 in Show and tell

23
You must be logged in to vote

how to delete llama.cpp build?

octadion asked Aug 9, 2024 in Q&A · Closed · Unanswered

0
You must be logged in to vote

llama-server RAM consumption

tunglambk asked Aug 7, 2024 in Q&A · Unanswered

1
You must be logged in to vote

Adjust VRAM/RAM split on Apple Silicon

r3muxd started Jul 11, 2023 in General

43
You must be logged in to vote

How to swap LoRA during a post request to llama-server?

Ujjawal-K-Panchal asked Aug 4, 2024 in Q&A · Answered

6
You must be logged in to vote

Can llama.cpp run the same model in parallel?

Zuo-Peng asked Aug 3, 2024 in Q&A · Answered

2
You must be logged in to vote

Parallel Requests support

akhilreddy0703 asked Jul 18, 2024 in Q&A · Unanswered

11
You must be logged in to vote

Does llama.cpp provide quant weights in one column of Matrix A?

ErvinXie asked Aug 7, 2024 in Q&A · Unanswered

1
You must be logged in to vote

Dealing with type conversion and relayout for NPU backends (how to know the data type during buffer creation/rw)?

marty1885 asked Jun 25, 2024 in Q&A · Answered

10
You must be logged in to vote

Is there a way to lower VRAM usage without lowering context size?

curvedinf asked Aug 5, 2024 in Q&A · Answered

2
You must be logged in to vote

Add a no-gpu CLI arg to llama-cli

WilliamTambellini started Aug 6, 2024 in Ideas

0
You must be logged in to vote

Finetuned LLM model conversion to GGUF - performance drop

TomekPro asked Jun 20, 2024 in Q&A · Unanswered

5
You must be logged in to vote

Am I being limited by single core CPU performance when fully offloaded?

reversebias asked Feb 29, 2024 in Q&A · Unanswered

12