FLORES-200 and NLLB Professionally Translated Datasets: NLLB-Seed, NLLB-MD, and Toxicity-200

⚠️ This repository is no longer being updated ⚠️

Newer versions of the FLORES and NLLB-Seed datasets managed by the Open Language Data Initiative are available here:

• FLORES
• NLLB-Seed

Quick-access to the original READMEs:

• FLORES-200
• NLLB-Seed
• NLLB-MD
• Toxicity-200

Citation

If you use any of this data in your work, please cite:

@article{nllb2022,
  author    = {NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi,  Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, Jeff Wang},
  title     = {No Language Left Behind: Scaling Human-Centered Machine Translation},
  year      = {2022}
}

Changelog

• 2022-06-30: Released FLORES-200, NLLB-Seed, NLLB-MD, and Toxicity-200

• 2021-06-04: Released FLORES-101

Licenses

• FLORES-200: CC-BY-SA 4.0
• NLLB-SEED: CC-BY-SA 4.0
• NLLB-MD: CC-BY-NC 4.0
• Toxicity-200: CC-BY-SA 4.0