Edit model card

Qwen2-7B_magiccoder_default

This model is a fine-tuned version of unsloth/Qwen2-7B on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.9894

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.02
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
0.8215 0.0261 4 0.9220
0.9247 0.0522 8 0.9780
0.9611 0.0783 12 0.9693
0.9392 0.1044 16 0.9867
1.0135 0.1305 20 1.0108
0.9152 0.1566 24 1.0167
0.9298 0.1827 28 1.0251
1.0625 0.2088 32 1.0349
0.9695 0.2349 36 1.0332
1.0104 0.2610 40 1.0390
1.0721 0.2871 44 1.0406
1.0397 0.3132 48 1.0449
0.9623 0.3393 52 1.0448
0.9735 0.3654 56 1.0436
1.0016 0.3915 60 1.0431
1.0557 0.4176 64 1.0401
1.0377 0.4437 68 1.0373
1.0022 0.4698 72 1.0361
1.0193 0.4959 76 1.0328
0.9806 0.5220 80 1.0301
1.0542 0.5481 84 1.0263
0.9692 0.5742 88 1.0244
1.0464 0.6003 92 1.0215
0.9771 0.6264 96 1.0166
1.0659 0.6525 100 1.0146
0.9476 0.6786 104 1.0106
0.983 0.7047 108 1.0074
0.9585 0.7308 112 1.0035
0.9193 0.7569 116 0.9997
0.9041 0.7830 120 0.9975
0.9697 0.8091 124 0.9954
0.9464 0.8352 128 0.9933
1.0252 0.8613 132 0.9917
0.9665 0.8874 136 0.9909
0.9948 0.9135 140 0.9904
0.946 0.9396 144 0.9897
1.0095 0.9657 148 0.9896
0.9675 0.9918 152 0.9894

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.0
  • Pytorch 2.4.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
0
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for imdatta0/Qwen2-7B_magiccoder_default

Base model

unsloth/Qwen2-7B
Adapter
(12)
this model