Update README.md
Browse files
README.md
CHANGED
@@ -4,3 +4,4 @@ This model was initialized with a word2vec token embedding matrix with 256k entr
|
|
4 |
|
5 |
Then the model was trained on this dataset with MLM for 250k steps (batch size 64). The token embeddings were updated during MLM.
|
6 |
|
|
|
|
4 |
|
5 |
Then the model was trained on this dataset with MLM for 250k steps (batch size 64). The token embeddings were updated during MLM.
|
6 |
|
7 |
+
For the same model but with frozen token embeddings while MLM training see: https://huggingface.co/vocab-transformers/distilbert-word2vec_256k-MLM_250k
|