Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
yhavinga
/
dutch-tokenizer-arena
like
8
Running
App
Files
Files
Community
1
Fetching metadata from the HF Docker repository...
c78da21
dutch-tokenizer-arena
3 contributors
History:
79 commits
yhavinga
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
c78da21
10 months ago
css
add compression leaderboard
10 months ago
images
update
over 1 year ago
js
add grok mixtral
10 months ago
patcher
add compression leaderboard
10 months ago
stats
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
10 months ago
utils
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
10 months ago
vocab
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
10 months ago
.gitattributes
Safe
2.32 kB
add grok mixtral
10 months ago
.gitignore
Safe
171 Bytes
update
over 1 year ago
README.md
Safe
25.5 kB
Hack some Dutch tokenizers into it
about 1 year ago
app.py
Safe
565 Bytes
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
10 months ago
app_compression.py
Safe
4.75 kB
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
10 months ago
app_playground.py
Safe
9.81 kB
update
10 months ago
config.py
Safe
1.72 kB
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
10 months ago
examples.py
Safe
3.16 kB
add compression leaderboard
10 months ago
requirements.txt
Safe
96 Bytes
Pin protobuf dep
11 months ago
util.py
Safe
8.15 kB
Hack some Dutch tokenizers into it
about 1 year ago
