Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

embedding的维度数量限制问题 #1850

Open
6 of 7 tasks
travisgu opened this issue Jun 26, 2024 · 1 comment
Open
6 of 7 tasks

embedding的维度数量限制问题 #1850

travisgu opened this issue Jun 26, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@travisgu
Copy link

travisgu commented Jun 26, 2024

例行检查

  • 我已确认目前没有类似 issue
  • 我已完整查看过项目 README,以及项目文档
  • 我使用了自己的 key,并确认我的 key 是可正常使用的
  • 我理解并愿意跟进此 issue,协助测试和提供反馈
  • 我理解并认可上述内容,并理解项目维护者精力有限,不遵循规则的 issue 可能会被无视或直接关闭

你的版本

  • 公有云版本
  • 私有部署版本, 具体版本号: 4.8.1

问题描述, 日志截图
image
看来起embedding的维度不能超过1536,PG Vector的index算法不能超过2000 dimension, 但是通过其他方式进行支持:
You can use half-precision indexing to index up to 4,000 dimensions or binary quantization to index up to 64,000 dimensions. Another option is dimensionality reduction.

不知是否有计划在以后的版本支持更高维度的embedding。

复现步骤
使用Ollama上的Qwen:72b模型进行embedding
预期结果

相关截图

@travisgu travisgu added the bug Something isn't working label Jun 26, 2024
@c121914yu
Copy link
Collaborator

half 模式后续打算增加,不过得看具体情况。可以考虑先用 milvus

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants