feat(kb): [WIP]overhaul knowledge base workflows and retrieval#8630

Open

lxfight wants to merge 48 commits into

AstrBotDevs:masterfrom

lxfight:enhance/kb-stability-and-features

Commits on Jun 8, 2026

fix(kb): scope chunk_count to current KB in update_kb_stats
lxfight
committed
fix(kb): unify sparse retrieval score direction to lower-is-better
lxfight
committed
fix(kb): rollback orphan vectors when metadata save fails after upload
lxfight
committed
fix(sidebar): prevent memory leak in upload_tasks/upload_progress dicts
lxfight
committed
feat(kb): add batch delete documents API
lxfight
committed
fix(kb): cap BM25 fallback at 10K chunks to prevent OOM
lxfight
committed
fix(kb): offload faiss.write_index to thread via asyncio.to_thread
lxfight
committed
fix(kb): switch FAISS from L2 to IP cosine similarity with write lock
lxfight
committed
fix(kb): correct BM25 top_k_sparse aggregation across multiple KBs
lxfight
committed
perf(kb): add kb_id generated column and parallelize dense retrieval
lxfight
committed
feat(kb): add HNSW index support and thread-safe LRU embedding cache
lxfight
committed
fix(kb): preserve FAISS external IDs during L2→IP index migration
lxfight
committed
fix(kb): return bool from vec_db.delete to signal whether chunk existed
lxfight
committed
fix(kb): serialize FTS5 rebuild, use VIRTUAL generated columns, return None on limit<=0
lxfight
committed
fix(kb): add incremental index_type migration and guard document deletion with kb_id
lxfight
committed
refactor(kb): clarify RRF variable names and simplify BM25 per-KB aggregation
lxfight
committed
fix(kb): add rate-limiter lock, validate chunk/doc existence on delete, track init retries
lxfight
committed
fix(kb): cascade-delete KB records, add instance lock, name index, and init retry
lxfight
committed
fix(kb): improve dashboard error messages, rerank sentinel, and upload validation
lxfight
committed
test(kb): add coverage for kb_id-guarded deletion, vec-delete failure, and migration
lxfight
committed
feat(kb): add document search by name/type and remove default list_kbs limit
lxfight
committed
feat(kb): support kb_ids in retrieve and validate KB options before mutation
lxfight
committed
fix(kb): validate upload parameters, add file type/size checks, support search/kb_ids in API
lxfight
committed
feat(dashboard): add document search, improved upload UX, and validation feedback in KB UI
lxfight
committed
feat(kb): upgrade knowledge base workflows
lxfight
committed
fix: back up faiss index before migration
lxfight
committed
feat: preserve markdown structures during chunking
lxfight
committed
fix: surface failed knowledge base uploads
lxfight
committed
style: improve knowledge base document detail layout
lxfight
committed
style: simplify knowledge base overview layout
lxfight
committed
fix: clean up faiss batch insert failures
lxfight
committed
fix: enforce kb document ownership
lxfight
committed
fix: report partial kb task failures
lxfight
committed
test: consolidate knowledge base regression coverage
lxfight
committed
test: merge knowledge base test files
lxfight
committed
test: restore split knowledge base tests
lxfight
committed
test: align kb import failure status expectation
lxfight
committed
fix(kb): re-raise RuntimeError in single-KB dense retrieval
lxfight
committed
fix(kb): add path validation and atomic migration for FAISS index
lxfight
committed
fix(kb): add timestamp null-safety and concurrency control
lxfight
committed
fix(kb): prevent deadlock in KB instance initialization retry
lxfight
committed
fix(kb): improve error logging for dense retrieval exceptions
lxfight
committed
fix(kb): remove unused imports from kb_db_sqlite.py
lxfight
committed
fix(kb): ensure atomic FAISS index writes to prevent corruption
lxfight
committed
fix(kb): handle concurrent duplicate document uploads gracefully
lxfight
committed
test(kb): add comprehensive stability and concurrency tests
lxfight
committed
fix(test): update tests for atomic FAISS write and correct RetrievalManager init
lxfight
committed
fix(kb): improve error handling and cache consistency
lxfight
committed