To compress the token length，your prompt is too long

[555chy]

In Chinese

我发现你们做BI分析的时候，都是把所有的表放到提示词里面送给GPT，然后再让他来找出跟问题有关的表。
但是实际中，业务表往往可能很多，比如几千张，如果把这个schema全量送给GPT，这个每次的token损耗是巨大的，而且有可能超出长度。
能否在本地通过规则、表的关联关系、表名、表含义，或者字段注释等，先通过向量数据库来筛选出可能涉及的表，再给GPT？以压缩token长度

并且有些简单的问题，可能直接发给gpt，就能返回正确的sql，而无需过量的提示词封装。

另外，我在使用自己的数据库（department + staff 表），我问Wren AI，“哪个部门平均年龄最大？你觉得可能是什么原因导致的”，这种开放性问题，Wren AI 居然报错了，提示无法回答

---

In English

I found that when you do BI analysis, you always put all the tables in prompt words and send them to GPT, and then let him find the tables related to the problem.
However, in reality, there may be many business tables, such as thousands. If all of this schema is sent to GPT, the token loss each time is huge and may exceed the length.
Can we use rules, table associations, table names, table meanings, or field annotations locally to filter out potential tables through a vector database before providing them to GPT? To compress the token length

And some simple questions may be directly sent to GPT, which can return the correct SQL without excessive prompt word encapsulation.

In addition, while using my own database (department+staff table), I asked Wren AI, "Which department has the highest average age? What do you think might be the reason for this?" This open-ended question, Wren AI actually reported an error and was unable to answer

[cyyeh]

In Chinese

我发现你们做BI分析的时候，都是把所有的表放到提示词里面送给GPT，然后再让他来找出跟问题有关的表。
但是实际中，业务表往往可能很多，比如几千张，如果把这个schema全量送给GPT，这个每次的token损耗是巨大的，而且有可能超出长度。
能否在本地通过规则、表的关联关系、表名、表含义，或者字段注释等，先通过向量数据库来筛选出可能涉及的表，再给GPT？以压缩token长度

并且有些简单的问题，可能直接发给gpt，就能返回正确的sql，而无需过量的提示词封装。

另外，我在使用自己的数据库（department + staff 表），我问Wren AI，“哪个部门平均年龄最大？你觉得可能是什么原因导致的”，这种开放性问题，Wren AI 居然报错了，提示无法回答

---

In English

I found that when you do BI analysis, you always put all the tables in prompt words and send them to GPT, and then let him find the tables related to the problem.
However, in reality, there may be many business tables, such as thousands. If all of this schema is sent to GPT, the token loss each time is huge and may exceed the length.
Can we use rules, table associations, table names, table meanings, or field annotations locally to filter out potential tables through a vector database before providing them to GPT? To compress the token length

And some simple questions may be directly sent to GPT, which can return the correct SQL without excessive prompt word encapsulation.

In addition, while using my own database (department+staff table), I asked Wren AI, "Which department has the highest average age? What do you think might be the reason for this?" This open-ended question, Wren AI actually reported an error and was unable to answer

@555chy 請問您方便加入discord server讓我做後續的了解嗎？

https://discord.gg/5DvshJqG8Z

[wmywbyt2]

请问您是使用deepseek么？能不能提供一下config.yaml。谢谢