Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ux: Improve message on how to deal with an error from a worker #128

Merged
merged 4 commits into from
Aug 5, 2024

Conversation

eddiebergman
Copy link
Contributor

@eddiebergman eddiebergman commented Aug 5, 2024

When an error happens in a worker which does not ignore errors.

Traceback (most recent call last):
  File "/home/skantify/code/neps/neps/state/_eval.py", line 125, in _eval_trial
    user_result = fn(**kwargs, **trial.config)
  File "/home/skantify/code/neps/neps_examples/basic_usage/hyperparameters.py", line 11, in run_pipeline
    raise ValueError("Something went wrong!")
ValueError: Something went wrong!

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/skantify/code/neps/neps_examples/basic_usage/hyperparameters.py", line 25, in <module>
    neps.run(
  File "/home/skantify/code/neps/neps/api.py", line 232, in run
    _launch_runtime(
  File "/home/skantify/code/neps/neps/runtime.py", line 556, in _launch_runtime
    worker.run()
  File "/home/skantify/code/neps/neps/runtime.py", line 378, in run
    should_stop = self._check_if_should_stop(
  File "/home/skantify/code/neps/neps/runtime.py", line 228, in _check_if_should_stop
    raise WorkerRaiseError(msg) from error_from_this_worker
neps.exceptions.WorkerRaiseError: Error occurred while evaluating a configuration with this worker and the worker is set to stop with OnErrorPossibilities.RAISE_ANY_ERROR.

If this was a bug in the evaluation code while you were developing your pipeline and you have set ignore_errors=True, please delete your results folder and fix the error before re-running.
If this is an issue specifically with the configuration, considering setting `ignore_errors=False` to allow the worker to continue evaluating other configurations, even if this one failed.

When you run another worker which is set not to ignore errors.

neps.state.err_dump.SerializedError: An error occurred during the evaluation of a trial '1' which was evaluted by worker '183039-2024-08-05T16:50:06.087931+00:00'. The original error could not be deserialized but had the following information:
ValueError: Something went wrong!

Traceback (most recent call last):
  File "/home/skantify/code/neps/neps/state/_eval.py", line 125, in _eval_trial
    user_result = fn(**kwargs, **trial.config)
  File "/home/skantify/code/neps/neps_examples/basic_usage/hyperparameters.py", line 11, in run_pipeline
    raise ValueError("Something went wrong!")
ValueError: Something went wrong!


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/skantify/code/neps/neps_examples/basic_usage/hyperparameters.py", line 25, in <module>
    neps.run(
  File "/home/skantify/code/neps/neps/api.py", line 232, in run
    _launch_runtime(
  File "/home/skantify/code/neps/neps/runtime.py", line 556, in _launch_runtime
    worker.run()
  File "/home/skantify/code/neps/neps/runtime.py", line 378, in run
    should_stop = self._check_if_should_stop(
  File "/home/skantify/code/neps/neps/runtime.py", line 295, in _check_if_should_stop
    raise WorkerRaiseError(msg) from err
neps.exceptions.WorkerRaiseError: An error occurred in another worker and this worker is set to stop with OnErrorPossibilities.RAISE_ANY_ERROR.
If this was a bug in the evaluation code while you were developing your pipeline and you have set ignore_errors=True, please delete your results folder and fix the error before re-running.
If this is an issue specifically with the configuration, considering setting `ignore_errors=False` to allow the worker to continue evaluating other configurations, even if any worker fails.

@eddiebergman eddiebergman added the ux Something that would improve user experience label Aug 5, 2024
@eddiebergman eddiebergman merged commit a55d121 into master Aug 5, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ux Something that would improve user experience
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[UX] Show error and traceback when something goes wrong
1 participant