Update README.md
Browse files
README.md
CHANGED
@@ -64,7 +64,7 @@ python3 pretrain.py --dataset_path couplet_dataset.pt \
|
|
64 |
--learning_rate 5e-4 --batch_size 64 \
|
65 |
--embedding word_pos --remove_embedding_layernorm \
|
66 |
--encoder transformer --mask causal --layernorm_positioning pre \
|
67 |
-
--target lm --
|
68 |
```
|
69 |
|
70 |
Finally, we convert the pre-trained model into Huggingface's format:
|
|
|
64 |
--learning_rate 5e-4 --batch_size 64 \
|
65 |
--embedding word_pos --remove_embedding_layernorm \
|
66 |
--encoder transformer --mask causal --layernorm_positioning pre \
|
67 |
+
--target lm --tie_weights
|
68 |
```
|
69 |
|
70 |
Finally, we convert the pre-trained model into Huggingface's format:
|