You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have implemented metadata for most of these models, some of them don't show up because we know that they are not zero-shot on the benchmark. Others have problems with results I think, we should probably look into those (@Samoed) (I have no idea where xiabou-v2 is, but apparently we have no results on it , otherwise it should show up).
I have not included some other models, like IYUN or AGE_Hybrid because there is absolutely no information available about them. With IYUN for instance, @Muennighoff started a discussion, which asks where the model is (cause it's missing from the repo).
AGE has nothing in its readme, and I believe we should probably not have models on the leaderboard, which we (or any user) know nothing about.
This is especially a problem with the Chinese leaderboard. I went through most models by hand, and a lot of them had problems like this.
Not implementing gme-Qwen is an oversight, we'll have to fix it, sorry :D
I will take a look at the old leaderboard and see which models are missing, and if they are, is there a good reason for it.
The new leaderboard only has 108 models on mteb classic, while the old has 382
The text was updated successfully, but these errors were encountered: