---
language:
  - rw
dataset:
  - kinyabert by antoine nzeyimana
metric:
  - loss
pipeline_tag: text-generation
---

Kinyarwanda GPT-2 model based on Andrej Karpathy's nanoGPT. It was trained on a mixture of news data and diverse computer-generated datasets in Kinyarwanda. 

## Model configuration

- number of layers = 6
- number of heads = 6
- embeddings = 384
- block size = 256

## Model dependencies

``` 
pip install transformers datasets tiktoken wandb tqdm

```

## Usage

To use this model to generate text, download all the files in the repo and put them in the same directory, then run the following command:

```
python sample.py --out_dir=.
```