Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HumanEval-X generation appears to not time out called subproceses #290

Open
nielstron opened this issue Nov 24, 2024 · 0 comments
Open

HumanEval-X generation appears to not time out called subproceses #290

nielstron opened this issue Nov 24, 2024 · 0 comments

Comments

@nielstron
Copy link

After running the humaneval-sythesize on a large number of samles, I noticed that many processes "a.out" and "node test.js" are runing, with runtimes of several hours and huge memory consumption. Appearently the evaluation does not kill processes after a set timeout.

Example command

accelerate launch --main_process_port 0 main.py --tasks humanevalsynthesize-cpp --load_generations_path ../.. --metric_output_path ../.. --allow_code_execution --n_samples 1000
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant