RAG chunking scheme #1921
-
|
Does anyone know how documents are chunked when using copilot as a RAG? I'm trying to optimize the context lengths for my local models. It feels like a setting you should be able to adjust. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
You can set "max sources" under QA settings to control your context length. Chunking size is not exposed to the user in the UI but you can change it in the code. |
Beta Was this translation helpful? Give feedback.
-
|
Awesome, thanks. It looks like the default is 6k. It may be worth exposing this in the settings panel. I've had difficulty finding open embedding networks that offer contexts longer than 2K. Many have maximum contexts of 512. |
Beta Was this translation helpful? Give feedback.
You can set "max sources" under QA settings to control your context length. Chunking size is not exposed to the user in the UI but you can change it in the code.