-
Notifications
You must be signed in to change notification settings - Fork 285
feat(qdrant):support qdrant cloud and add index #522
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
## Description <!-- Please include a summary of the changes below; Fill in the issue number that this PR addresses (if applicable); Fill in the related MemOS-Docs repository issue or PR link (if applicable); Mention the person who will review this PR (if you know who it is); Replace (summary), (issue), (docs-issue-or-pr-link), and (reviewer) with the appropriate information. 请在下方填写更改的摘要; 填写此 PR 解决的问题编号(如果适用); 填写相关的 MemOS-Docs 仓库 issue 或 PR 链接(如果适用); 提及将审查此 PR 的人(如果您知道是谁); 替换 (summary)、(issue)、(docs-issue-or-pr-link) 和 (reviewer) 为适当的信息。 --> Summary: (summary) Fix: #(issue) Docs Issue/PR: (docs-issue-or-pr-link) Reviewer: @(reviewer) ## Checklist: - [ ] I have performed a self-review of my own code | 我已自行检查了自己的代码 - [ ] I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释 - [ ] I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常 - [ ] I have created related documentation issue/PR in [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) (if applicable) | 我已在 [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) 中创建了相关的文档 issue/PR(如果适用) - [ ] I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR(如果适用) - [ ] I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人
## Description <!-- Please include a summary of the changes below; Fill in the issue number that this PR addresses (if applicable); Fill in the related MemOS-Docs repository issue or PR link (if applicable); Mention the person who will review this PR (if you know who it is); Replace (summary), (issue), (docs-issue-or-pr-link), and (reviewer) with the appropriate information. 请在下方填写更改的摘要; 填写此 PR 解决的问题编号(如果适用); 填写相关的 MemOS-Docs 仓库 issue 或 PR 链接(如果适用); 提及将审查此 PR 的人(如果您知道是谁); 替换 (summary)、(issue)、(docs-issue-or-pr-link) 和 (reviewer) 为适当的信息。 --> Summary: (summary) Fix: #(issue) Docs Issue/PR: (docs-issue-or-pr-link) Reviewer: @(reviewer) ## Checklist: - [ ] I have performed a self-review of my own code | 我已自行检查了自己的代码 - [ ] I have commented my code in hard-to-understand areas | 我已在难以理解的地方对代码进行了注释 - [ ] I have added tests that prove my fix is effective or that my feature works | 我已添加测试以证明我的修复有效或功能正常 - [ ] I have created related documentation issue/PR in [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) (if applicable) | 我已在 [MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) 中创建了相关的文档 issue/PR(如果适用) - [ ] I have linked the issue to this PR (if applicable) | 我已将 issue 链接到此 PR(如果适用) - [ ] I have mentioned the person who will review this PR | 我已提及将审查此 PR 的人
…emTensor#511) ## Summary Updated [docker/.env.example](cci:7://file:///Users/nowcoder/Desktop/MemOS/docker/.env.example:0:0-0:0) to include all currently used environment variables in the codebase, ensuring users have a complete reference for configuration. ## Source - Related Issue: MemTensor#505 ## Changes - Added missing variables (e.g., `API_SCHEDULER_ON`, `MOS_CUBE_PATH`). - Clarified comments and usage for existing variables. - Verified against [src/memos/api/config.py] and other source files. ## Suggestion To further improve the onboarding experience, I suggest we consider providing multiple preset configuration files (e.g., `.env.simple`, `.env.production`, `.env.dev`) or a `.env.sync` mechanism. This would allow users to quickly choose a configuration that matches their use case without needing to manually toggle every option in a single large file.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds support for Qdrant Cloud configuration, automatic payload indexing, and improved error handling for cloud deployments. It enables the use of cloud-hosted Qdrant instances via URL and API key configuration, while maintaining backward compatibility with local/embedded deployments.
Key Changes:
- Added Qdrant Cloud support with
urlandapi_keyconfiguration options that take priority over local host/port/path settings - Implemented automatic creation of payload indexes for common filter fields (memory_type, status, vector_sync, user_name) to improve query performance
- Added JSON string parsing for reranker
headers_extraconfiguration to support environment variable configuration
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/memos/configs/vec_db.py | Added url and api_key fields to QdrantVecDBConfig for cloud support |
| src/memos/vec_dbs/qdrant.py | Refactored client initialization to support cloud mode, added automatic payload index creation, improved error handling for cloud-specific collection creation scenarios |
| src/memos/api/config.py | Added environment variable support for QDRANT_PATH, QDRANT_URL, and QDRANT_API_KEY |
| src/memos/reranker/factory.py | Added JSON parsing for headers_extra string configuration to support environment variable usage |
| tests/configs/test_vec_db.py | Added test case validating cloud configuration with url and api_key fields |
| tests/vec_dbs/test_qdrant.py | Added test verifying correct QdrantClient initialization with cloud parameters |
| docker/.env.example | Extensively updated with comprehensive environment variable documentation including Qdrant cloud configuration |
| docs/product-api-tests.md | Added smoke test documentation for product APIs including notes on payload index requirements |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| NEO4J_DB_NAME=neo4j # required for shared-db mode | ||
| MOS_NEO4J_SHARED_DB=false | ||
| QDRANT_HOST=localhost | ||
| QDRANT_PORT=6333 |
Copilot
AI
Nov 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing QDRANT_PATH environment variable documentation. This variable is used in src/memos/api/config.py (line 503) but is not documented here. Consider adding it after QDRANT_PORT for completeness:
QDRANT_PATH= # local/embedded path (optional)
| QDRANT_PORT=6333 | |
| QDRANT_PORT=6333 | |
| QDRANT_PATH= # local/embedded path (optional) |
| except Exception: | ||
| # Bubble up other exceptions so callers can observe failures | ||
| raise |
Copilot
AI
Nov 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The except Exception block is redundant as it only re-raises the exception without adding any value. Consider removing lines 101-103 entirely, as the exception will naturally propagate without this catch-and-rethrow pattern.
| except Exception: | |
| # Bubble up other exceptions so callers can observe failures | |
| raise |
| try: | ||
| headers_extra = json.loads(headers_extra) | ||
| except Exception: | ||
| headers_extra = None |
Copilot
AI
Nov 24, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The exception handling silently swallows JSON parsing errors and sets headers_extra to None. This could hide configuration mistakes. Consider adding logging to help users debug invalid configurations. You'll need to import the logger first:
from memos.log import get_logger
logger = get_logger(__name__)Then update the exception handler:
except Exception as e:
logger.warning(f"Failed to parse headers_extra as JSON: {e}. Using None instead.")
headers_extra = None
Description
Summary: Support Qdrant Cloud config (url/api_key), auto-create common payload indexes, parse JSON string headers for the BGE reranker, and add product API smoke-test notes (source: #518).
Fix: Source#518
Docs Issue/PR: N/A
Reviewer: @CaralHsi
Checklist: