|
--- |
|
language: |
|
- rw |
|
dataset: |
|
- kinyabert by antoine nzeyimana |
|
metric: |
|
- loss |
|
pipeline_tag: text-generation |
|
library_name: transformers |
|
--- |
|
|
|
Kinyarwanda GPT-2 model based on Andrej Karpathy's nanoGPT. It was trained on a mixture of news data and diverse computer-generated datasets in Kinyarwanda. |
|
|
|
## Model configuration |
|
|
|
- number of layers = 6 |
|
- number of heads = 6 |
|
- embeddings = 384 |
|
- block size = 256 |
|
|
|
## Model dependencies |
|
|
|
``` |
|
pip install transformers datasets tiktoken wandb tqdm |
|
|
|
``` |
|
|
|
## Usage |
|
|
|
To use this model to generate text, download all the files in the repo and put them in the same directory, then run the following command: |
|
|
|
``` |
|
python sample.py --out_dir=. |
|
``` |
|
|
