You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Oh, there is more than just this one... lookup the degrees/radians problem in MBPP--sorry I forget the number. We actually don't use the canonical solutions at all in MultiPL-E. So, we should be okay about this.
The principle we've been following: we want to fix bugs in MultiPL-E, but preserve bugs in the underlying benchmarks. Hopefully, that will make comparisons easier to do. Let me know if you have other ideas.
openai/human-eval#22 (comment)
The text was updated successfully, but these errors were encountered: