Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: unifed old retrieve API #573

Merged
merged 7 commits into from
Jan 8, 2025
Merged

refactor: unifed old retrieve API #573

merged 7 commits into from
Jan 8, 2025

Conversation

Mini256
Copy link
Member

@Mini256 Mini256 commented Jan 7, 2025

Unify /admin/retrieve/documents and /admin/embedding_retrieve API impl

Copy link

vercel bot commented Jan 7, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
tidb-ai-preview ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 8, 2025 3:11am
tidb-ai-storybook ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 8, 2025 3:11am

Mini256 added a commit that referenced this pull request Jan 7, 2025
…ling_factor for retrieve api (#575)

close #570

For  `/admin/knowledge_bases/{kb_id}/graph/search` API, , add parameter:

- `relationship_meta_filters`: pass the filtering conditions.

For `/admin/retrieve/documents` API, add two parameters:
- `similarity_top_k` to control how many nodes should the vector search
return, if not set, using the value of `top_k` by default.
- `oversampling_factor`: This is similar to the `ef_search` parameter of
the HNSW index, the larger the parameter, the higher the recall rate.
Since TiDB does not yet support modifying the value of `ef_search`, the
current implementation uses subquery. The subquery returns the
`similarity_top_k * oversampling_factor` rows, and the outer query
finally returns the `similarity_top_k` rows.
- At this time, if you need to turn on `metadata_filter`, you need to
modify the Chat Engine configuration, this problem will be fixed after
the retrieve API refactor (#573), thr new retrieve API will not
dependant on the ChatEngine configuration.
@Mini256 Mini256 marked this pull request as ready for review January 8, 2025 03:13
@Mini256 Mini256 changed the title refactor: unifed retrieve API refactor: unifed old retrieve API Jan 8, 2025
Copy link

github-actions bot commented Jan 8, 2025

@Mini256 Mini256 merged commit a0cf8ac into main Jan 8, 2025
14 checks passed
@Mini256 Mini256 deleted the refactor-retrieve-api-1 branch January 8, 2025 04:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant