manueltonneau
commited on
Commit
•
7dcf752
1
Parent(s):
6e5d6ad
Update README.md
Browse files
README.md
CHANGED
@@ -46,7 +46,7 @@ The model was further pre-trained on 2.2 billion tweets posted between March 200
|
|
46 |
|
47 |
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
48 |
We performed an adaptive fine tuning of XLM-R on the Nigerian Twitter dataset.
|
49 |
-
We kept the same vocabulary as XLM-R and trained the model for one epoch, using 1\% of the dataset as validation set. The training procedure was conducted in a distributed environment, for approximately 10 days, using 4 nodes with 4 RTX 8000 GPUs each, with a total batch size of 576.
|
50 |
|
51 |
|
52 |
## Evaluation
|
|
|
46 |
|
47 |
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
|
48 |
We performed an adaptive fine tuning of XLM-R on the Nigerian Twitter dataset.
|
49 |
+
We kept the same vocabulary as XLM-R and trained the model until convergence for a total of one epoch, using 1\% of the dataset as validation set. The training procedure was conducted in a distributed environment, for approximately 10 days, using 4 nodes with 4 RTX 8000 GPUs each, with a total batch size of 576.
|
50 |
|
51 |
|
52 |
## Evaluation
|