Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exact commandline to reproduce leaderboard #162

Open
Derekglk opened this issue Nov 13, 2023 · 2 comments
Open

exact commandline to reproduce leaderboard #162

Derekglk opened this issue Nov 13, 2023 · 2 comments

Comments

@Derekglk
Copy link

Wonderful Bigcode collaborators,

Can we add a new column to the Big Code Leaderboard which describes the exact command line to reproduce the scores?
I saw many similar questions asking about how to reproduce the scores, and the differences against the paper.
This new column will definitely help.

Thanks!

@loubnabnl
Copy link
Collaborator

Good suggestion, will add it to the todo list

@Anindyadeep
Copy link

Hi, I just saw this issue, but does the only variable is the model_name here to reproduce results? And if that is the case, do we require a whole column?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants