Hugging Face's logo

imdatta0
/
Qwen2-7B_metamath_ortho

PEFT
Safetensors
unsloth
Generated from Trainer
Model card Files Community
Qwen2-7B_metamath_ortho / README.md
imdatta0's picture
imdatta0
End of training
f2679e7 verified
preview code
|
raw
history blame
3.85 kB
metadata
base_model: unsloth/qwen2-7b-bnb-4bit
library_name: peft
license: apache-2.0
tags:
  - unsloth
  - generated_from_trainer
model-index:
  - name: Qwen2-7B_metamath_ortho
    results: []

Qwen2-7B_metamath_ortho

This model is a fine-tuned version of unsloth/qwen2-7b-bnb-4bit on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2209

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.02
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
0.2058 0.0211 13 0.2001
0.2083 0.0421 26 0.2441
0.247 0.0632 39 0.2670
0.2629 0.0842 52 0.2797
0.2692 0.1053 65 0.2921
0.2867 0.1264 78 0.2992
0.2939 0.1474 91 0.3022
0.2839 0.1685 104 0.3037
0.2922 0.1896 117 0.3071
0.2937 0.2106 130 0.3114
0.3016 0.2317 143 0.3096
0.298 0.2527 156 0.3078
0.3059 0.2738 169 0.3083
0.2883 0.2949 182 0.3052
0.3013 0.3159 195 0.3010
0.2874 0.3370 208 0.2995
0.286 0.3580 221 0.2979
0.2877 0.3791 234 0.2981
0.2858 0.4002 247 0.2921
0.2853 0.4212 260 0.2906
0.2887 0.4423 273 0.2882
0.2772 0.4633 286 0.2837
0.2867 0.4844 299 0.2799
0.2659 0.5055 312 0.2776
0.2688 0.5265 325 0.2746
0.2564 0.5476 338 0.2690
0.2587 0.5687 351 0.2657
0.2615 0.5897 364 0.2619
0.2517 0.6108 377 0.2576
0.2453 0.6318 390 0.2516
0.2473 0.6529 403 0.2475
0.2494 0.6740 416 0.2444
0.2397 0.6950 429 0.2405
0.2438 0.7161 442 0.2369
0.2288 0.7371 455 0.2344
0.2293 0.7582 468 0.2319
0.2372 0.7793 481 0.2310
0.2296 0.8003 494 0.2284
0.2163 0.8214 507 0.2263
0.2249 0.8424 520 0.2241
0.219 0.8635 533 0.2234
0.2154 0.8846 546 0.2230
0.2107 0.9056 559 0.2221
0.2132 0.9267 572 0.2215
0.2202 0.9478 585 0.2212
0.2197 0.9688 598 0.2211
0.2168 0.9899 611 0.2209

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.0
  • Pytorch 2.4.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1