Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
versae
/
scandeng-tokenizer
like
0
Text Generation
Transformers
mistral
Inference Endpoints
License:
apache-2.0
Model card
Files
Files and versions
Community
Train
Deploy
Use this model
main
scandeng-tokenizer
1 contributor
History:
4 commits
versae
Add HF tokenizer converted from SentencePiece
04b8476
10 months ago
.gitattributes
Safe
1.52 kB
initial commit
10 months ago
README.md
Safe
28 Bytes
initial commit
10 months ago
config.json
Safe
595 Bytes
Create config.json
10 months ago
convert.sh
Safe
1.1 kB
Add HF tokenizer converted from SentencePiece
10 months ago
merges.txt
Safe
568 kB
Add HF tokenizer converted from SentencePiece
10 months ago
sentencepiece.model
Safe
770 kB
LFS
SentencePiece byte fallback 32k
10 months ago
sentencepiece.vocab.bak
Safe
486 kB
Add HF tokenizer converted from SentencePiece
10 months ago
special_tokens_map.json
Safe
72 Bytes
Add HF tokenizer converted from SentencePiece
10 months ago
tokenizer.json
Safe
1.85 MB
Add HF tokenizer converted from SentencePiece
10 months ago
tokenizer_config.json
Safe
826 Bytes
Add HF tokenizer converted from SentencePiece
10 months ago
vocab.json
Safe
687 kB
Add HF tokenizer converted from SentencePiece
10 months ago
