Vault and RAG upper limit?

[40Think]

Can i put to you RAG 1 000 000 markdown (OR 1-10 GB total size)? What tech you use for RAG?
Can i make 300 GB Vault?
For index i use qwen3-embedding-8b, some times indexing crash.

[ichts]

Try

• adjusting the partition number: Go to Settings → Copilot → QA → increase "Number of Partitions" to 8-16+.
• reducing Embedding Batch Size

As for model, you can try the copilot plus large.

Performance-wise:
OpenAI Large < copilot-plus-small < copilot-plus-multilingual < copilot-plus-large

[logancyang]

1GB vault should be fine, 10GB might be at the limit for the Semantic Search route. But you can use the index-free route by turning off Semantic Search.

[40Think]

Copilot plus sepatare index? In click plus in chat selector start new indexing

[40Think]

After introduce MCP please add options external memory MCPs and options turn off you memory.

[40Think]

@logancyang can you get some advice with this?
can symbols like this (#@%)*"!?;|~`{}[] make problems? In filename and inside files?

2025-10-16T21:54:19.615Z ERROR Error saving semantic index database: Failed to save database: Invalid string length\nError: Failed to save database: Invalid string length\n at kge.saveDatabase (plugin:copilot:336:129220)\n at async t.saveDB (plugin:copilot:336:137299)
2025-10-16T21:54:19.615Z ERROR Background save failed: Failed to save database: Invalid string length\nError: Failed to save database: Invalid string length\n at kge.saveDatabase (plugin:copilot:336:129220)\n at async t.saveDB (plugin:copilot:336:137299)
2025-10-16T22:15:12.458Z INFO Files to index: 21\nPreviously indexed: 1461\nEmpty files skipped: 7\nFiles missing embeddings: 0
2025-10-16T22:15:12.491Z INFO safeFetch request

and

2025-10-17T01:37:03.378Z ERROR Failed to load existing semantic index database, creating new one: Failed to load database: Expected ',' or ']' after array element in JSON at position 31238054 (line 1 column 31238055)\nError: Failed to load database: Expected ',' or ']' after array element in JSON at position 31238054 (line 1 column 31238055)\n at kge.loadDatabase (plugin:copilot:336:130429)\n at async t.initializeDB (plugin:copilot:336:136496)\n at async t.initialize (plugin:copilot:544:3337)
2025-10-17T01:37:03.378Z INFO safeFetch request
2025-10-17T01:37:03.404Z INFO Created new semantic index database for /.copilot-index. Embedding model: text-embedding-qwen3-embedding-8b with vector length 4096.