This is only a tokenizer.

This tokenizer is a PreTrainedTokenizerFast which is trained on raygx/Nepali-Extended-Corpus datasets.
This tokenizer is trained from scratch using Tokenizers library.
This tokenizer uses
- Model: Tokenizer(WordPiece(unk_token="[UNK]"))
- Normalizer: normalizers.Sequence([NFD(),Strip()])
- Pre-processor: pre_tokenizers.Sequence([Whitespace(),Digits(individual_digits=True), Punctuation()])
- Post-processor: BertProcessing

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API was unable to determine this model's library.