---
license: mit
datasets:
- wikipedia
- oscar
language:
- ja
- ko
tags:
- kenlm
- perplexity
- n-gram
- kneser-ney
- bigscience
---

# KenLM models
This repo is a copy of [edugp/kenlm](https://huggingface.co./edugp/kenlm) but for the Japanese and Korean languages.

The Wikipedia models were trained using the `20231106` dump.