Hugging Face's logo

imdatta0
/
mistral_7b_v_MetaMathQA_40K_reverse

PEFT
Safetensors
unsloth
Generated from Trainer
Model card Files Community
mistral_7b_v_MetaMathQA_40K_reverse / README.md
imdatta0's picture
imdatta0
End of training
56726a7 verified
preview code
|
raw
history blame
3.88 kB
metadata
license: apache-2.0
library_name: peft
tags:
  - unsloth
  - generated_from_trainer
base_model: mistralai/Mistral-7B-v0.3
model-index:
  - name: mistral_7b_v_MetaMathQA_40K_reverse
    results: []

mistral_7b_v_MetaMathQA_40K_reverse

This model is a fine-tuned version of mistralai/Mistral-7B-v0.3 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4421

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 0.02
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
0.7252 0.0211 13 0.6225
0.5814 0.0421 26 0.5900
0.5599 0.0632 39 0.5751
0.548 0.0842 52 0.5708
0.5371 0.1053 65 0.5625
0.5347 0.1264 78 0.5600
0.5232 0.1474 91 0.5529
0.5382 0.1685 104 0.5478
0.5178 0.1896 117 0.5482
0.5272 0.2106 130 0.5423
0.5135 0.2317 143 0.5397
0.4943 0.2527 156 0.5321
0.5012 0.2738 169 0.5323
0.5077 0.2949 182 0.5300
0.5031 0.3159 195 0.5233
0.506 0.3370 208 0.5238
0.4851 0.3580 221 0.5180
0.4915 0.3791 234 0.5146
0.4826 0.4002 247 0.5150
0.4964 0.4212 260 0.5096
0.4989 0.4423 273 0.5050
0.4846 0.4633 286 0.5021
0.4776 0.4844 299 0.5006
0.4725 0.5055 312 0.4927
0.4752 0.5265 325 0.4898
0.4719 0.5476 338 0.4862
0.4689 0.5687 351 0.4817
0.4573 0.5897 364 0.4772
0.4536 0.6108 377 0.4754
0.4536 0.6318 390 0.4700
0.4519 0.6529 403 0.4664
0.4448 0.6740 416 0.4633
0.4327 0.6950 429 0.4618
0.4528 0.7161 442 0.4586
0.4379 0.7371 455 0.4557
0.4504 0.7582 468 0.4537
0.4436 0.7793 481 0.4525
0.4451 0.8003 494 0.4497
0.435 0.8214 507 0.4482
0.4247 0.8424 520 0.4466
0.4295 0.8635 533 0.4455
0.4204 0.8846 546 0.4444
0.4381 0.9056 559 0.4433
0.4355 0.9267 572 0.4430
0.4234 0.9478 585 0.4424
0.4261 0.9688 598 0.4421
0.4266 0.9899 611 0.4421

Framework versions

  • PEFT 0.7.1
  • Transformers 4.40.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1