Commit
·
9ec253c
1
Parent(s):
21fa119
Update README.md
Browse files
README.md
CHANGED
@@ -66,33 +66,6 @@ tokenizer.tokenize(preprocessed_text)
|
|
66 |
|
67 |
We are working on integrating this preprocessing step into a Tokenizer within `transformers` library
|
68 |
|
69 |
-
## Development
|
70 |
-
|
71 |
-
### Installing
|
72 |
-
|
73 |
-
We use `python==3.7` and `poetry` to manage dependencies.
|
74 |
-
|
75 |
-
```bash
|
76 |
-
pip install poetry
|
77 |
-
poetry install
|
78 |
-
```
|
79 |
-
|
80 |
-
### Benchmarking
|
81 |
-
|
82 |
-
To run benchmarks
|
83 |
-
|
84 |
-
```bash
|
85 |
-
python bin/run_benchmark.py <model_name> --times 5 --output_path <output_path>
|
86 |
-
```
|
87 |
-
|
88 |
-
Check [RUN_BENCHMARKS](RUN_BENCHMARKS.md) for all experiments
|
89 |
-
|
90 |
-
### Smoke test
|
91 |
-
Test the benchmark running
|
92 |
-
|
93 |
-
```
|
94 |
-
./smoke_test.sh
|
95 |
-
```
|
96 |
## Citation
|
97 |
|
98 |
If you use *RoBERTuito*, please cite our paper:
|
|
|
66 |
|
67 |
We are working on integrating this preprocessing step into a Tokenizer within `transformers` library
|
68 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
69 |
## Citation
|
70 |
|
71 |
If you use *RoBERTuito*, please cite our paper:
|