Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I combine text2sql with python code generation to reduce memory usage #1503

Open
XJTU-JP opened this issue Jan 4, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@XJTU-JP
Copy link

XJTU-JP commented Jan 4, 2025

🚀 The feature

Great job! I find that in the case of a large amount of data in the database, it doesn't seem like a good idea to select all the data directly to the agent. If you combine sql and python code generation. Let some of the aggregation and filtering be done in sql, and then send the filtered data to python code as a dataframe. If you do this, you can certainly save a lot of memory (in the case of large amounts of data) :

Motivation, pitch

I am studying the use of text2sql and python code generation for data analysis under massive data

Alternatives

No response

Additional context

No response

@dosubot dosubot bot added the enhancement New feature or request label Jan 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant