Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Select model based on eval loss by default and display eval metric as well #648

Open
WenjieDu opened this issue Mar 17, 2025 · 0 comments
Open
Assignees
Labels
enhancement New feature or request new feature Proposing to add a new feature

Comments

@WenjieDu
Copy link
Owner

1. Feature description

Training models should be selected based on evaluation loss on the validation set. As for the eval metric, it should be applied and displayed for users' further information as well.

2. Motivation

The criterion for selecting the best model could be the higher the better, or the lower the better, making things complex. To unify and simplify this, just apply loss on the validation set for model selection. If eval metric is provided, calculate the value and display it for users as well.

3. Your contribution

Will make a PR

@WenjieDu WenjieDu added enhancement New feature or request new feature Proposing to add a new feature labels Mar 17, 2025
@WenjieDu WenjieDu self-assigned this Mar 17, 2025
@WenjieDu WenjieDu mentioned this issue Mar 21, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request new feature Proposing to add a new feature
Projects
None yet
Development

No branches or pull requests

1 participant