Merge v2.0.0: Overview issue

[KennethEnevoldsen]

The plan for the merge is the following:

•  MIEB -> main. MIEB is aligned with main, and can be easily merged in. This is dependent on:
  • MIEB: Remove results-mieb folder and figure out where to store results #1338
  •  [MIEB] Figure out folder structure #1337
  • [MIEB] Integrate docs #1336 (todo: link MIEB docs from README)
  • Non-blocking issues
    • [MIEB] Fill out all task description stats metadata #1799
    • [MIEB] Complete all model meta based on main schema #1800
    • [MIEB] migrate results from tmp repo to results repo #1823
•  v2.0.0 -> main, which is dependent on:
  • [v2] Updated ModelMeta parameters  #1779
  •  [v2] remove save_suffix from AbsTask #1770
  • [v2] remove metadata_dict #1820
  •  [v2] Getting password prompt for task and tests #1766
  •  [v2] convert category from length descriptor to modality in task metadata #1767
  •  Remove encode_corpus and encode_queries and implement a "document" class #1284
  •  New benchmark interface #1272
  • Evaluators ambiguity: class/module #1124
  • [v2] Remove MultilingualTask #1832
  • [v2] Reupload Reranking datasets in old format #1878
  • Add support for STS evaluation for Cross-Encoders #1725
  • Non-blocking
    • [v2] Fill dataset Metadata #1889
    • [v2] Reupload datasets with trust_remote_code #1877

If someone has the time for any of these merged do feel free to grab them.

[isaac-chung]

@KennethEnevoldsen Before we merge MIEB into v2.0.0, should we merge main into MIEB?

[isaac-chung]

@KennethEnevoldsen Before we merge MIEB into v2.0.0, should we merge main into MIEB?

Updated the description based on our DM.