library_name: transformers datasets: - HuggingFaceFW/fineweb
This is a BPE tokenizer with 10,048 tokens trained on a portion of FineWeb's 10B token sample.
