cloudyu
/

mistral_pretrain_demo

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

cloudyu commited on Mar 7

Commit

0158d31

•

1 Parent(s): 94ad50a

Update README.md

Files changed (1) hide show

README.md +3 -3

README.md CHANGED Viewed

@@ -59,12 +59,12 @@ trainer = SFTTrainer(
             max_steps=10000,
             learning_rate=1e-4,
             logging_steps=1,
-            output_dir="6B_outputs", overwrite_output_dir=True,save_steps=1000,
             optim="paged_adamw_32bit",report_to="none"
         )
 )
 trainer.train()
-trainer.model.save_pretrained("6B-final", dtype=torch.float32)
-trainer.tokenizer.save_pretrained("6B-final")
 ```

             max_steps=10000,
             learning_rate=1e-4,
             logging_steps=1,
+            output_dir="1B_outputs", overwrite_output_dir=True,save_steps=1000,
             optim="paged_adamw_32bit",report_to="none"
         )
 )
 trainer.train()
+trainer.model.save_pretrained("1B-final", dtype=torch.float32)
+trainer.tokenizer.save_pretrained("1B-final")
 ```