mtasic85's picture
tokenizer
3cb4a20
|
raw
history blame
183 Bytes
# Train
## Environment
```bash
cd scripts
python -m venv venv
source venv/bin/activate
pip install -U -r requirements.in
```
## Tokenizer
```bash
python -B train_tokenizer.py
```