metadata
language:
- rw
dataset:
- kinyabert by antoine nzeyimana
metric:
- loss
pipeline_tag: text-generation
Kinyarwanda GPT-2 model based on Andrej Karpathy's nanoGPT. It was trained on a mixture of news data and diverse computer-generated datasets in Kinyarwanda.
Model configuration
- number of layers = 6
- number of heads = 6
- embeddings = 384
- block size = 256
Model dependencies
pip install transformers datasets tiktoken wandb tqdm
Usage
To use this model to generate text, download all the files in the repo and put them in the same directory, then run the following command:
python sample.py --out_dir=.