afro-xlmr-large-61L

AfroXLMR-large was created by MLM adaptation of XLM-R-large model on 61 languages widely spoken in Africa including 4 high-resource languages.

Pre-training corpus

A mix of mC4, Wikipedia and OPUS data

Languages

There are 61 languages available :

  • English (eng)
  • Amharic (amh)
  • Arabic (ara)
  • Somali (som)
  • Kiswahili (swa)
  • Portuguese (por)
  • Afrikaans (afr)
  • French (fra)
  • isiZulu (zul)
  • Malagasy (mlg)
  • Hausa (hau)
  • chiShona (sna)
  • Egyptian Arabic (arz)
  • Chichewa (nya)
  • Igbo (ibo)
  • isiXhosa (xho)
  • Yorùbá (yor)
  • Sesotho (sot)
  • Kinyarwanda (kin)
  • Tigrinya (tir)
  • Tsonga (tso)
  • Oromo (orm)
  • Rundi (run)
  • Northern Sotho (nso)
  • Ewe (ewe)
  • Lingala (lin)
  • Twi (twi)
  • Nigerian Pidgin (pcm)
  • Ga (gaa)
  • Lozi (loz)
  • Luganda (lug)
  • Gun (guw)
  • Bemba (bem)
  • Efik (efi)
  • Luvale (lue)
  • Luba-Lulua (lua)
  • Tonga (toi)
  • Tshivenḓa (ven)
  • Tumbuka (tum)
  • Tetela (tll)
  • Isoko (iso)
  • Kaonde (kqn)
  • Zande (zne)
  • Umbundu (umb)
  • Mossi (mos)
  • Tiv (tiv)
  • Luba-Katanga (lub)
  • Fula (fuv)
  • San Salvador Kongo (kwy)
  • Baoulé (bci)
  • Ruund (rnd)
  • Luo (luo)
  • Wolaitta (wal)
  • Swazi (ssw)
  • Lunda (lun)
  • Wolof (wol)
  • Nyaneka (nyk)
  • Kwanyama (kua)
  • Kikuyu (kik)
  • Fon (fon)

Acknowledgment

We would like to thank Google Cloud for providing us access to TPU v3-8 through the free cloud credits. Model trained using flax, before converted to pytorch.

BibTeX entry and citation info.

@misc{adelani2023sib200,
      title={SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects}, 
      author={David Ifeoluwa Adelani and Hannah Liu and Xiaoyu Shen and Nikita Vassilyev and Jesujoba O. Alabi and Yanke Mao and Haonan Gao and Annie En-Shiun Lee},
      year={2023},
      eprint={2309.07445},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
Downloads last month
22
Safetensors
Model size
560M params
Tensor type
I64
·
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.