We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.
Typo exists in one of the prompts from the MMLU-Pro benchmark (math) provided for llama-3.1-8B-Instruct-evals.
This could potentially lead to inaccuracies during model evaluation and reduce the quality of the benchmark.
It seems that \f should be replaced with \f There also seem to be issues with the use of \ in other parts as well.
The correct version of the prompt should be:
The text was updated successfully, but these errors were encountered:
@dongseok1220 Thank you so much for reporting this. I will forward this to our internal team.
Sorry, something went wrong.
wukaixingxp
No branches or pull requests
System Info
.
Information
🐛 Describe the bug
Typo exists in one of the prompts from the MMLU-Pro benchmark (math) provided for llama-3.1-8B-Instruct-evals.
This could potentially lead to inaccuracies during model evaluation and reduce the quality of the benchmark.
It seems that \f should be replaced with \f
There also seem to be issues with the use of \ in other parts as well.
Error logs
.
Expected behavior
The correct version of the prompt should be:
The text was updated successfully, but these errors were encountered: