llama381binstruct_summarize_short

This model is a fine-tuned version of NousResearch/Meta-Llama-3.1-8B-Instruct on the generator dataset. It achieves the following results on the evaluation set:

  • Loss: 2.6704

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 30
  • training_steps: 500

Training results

Training Loss Epoch Step Validation Loss
1.7321 1.25 25 1.6900
0.8071 2.5 50 1.8501
0.3599 3.75 75 1.8955
0.1408 5.0 100 2.0946
0.1067 6.25 125 2.1324
0.0366 7.5 150 2.4187
0.025 8.75 175 2.3309
0.0215 10.0 200 2.3936
0.0094 11.25 225 2.5407
0.0063 12.5 250 2.4951
0.0035 13.75 275 2.5288
0.0051 15.0 300 2.5601
0.0042 16.25 325 2.5644
0.002 17.5 350 2.6005
0.002 18.75 375 2.6281
0.002 20.0 400 2.6451
0.0019 21.25 425 2.6574
0.0018 22.5 450 2.6649
0.0017 23.75 475 2.6694
0.0018 25.0 500 2.6704

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.2
  • Pytorch 2.4.0+cu121
  • Datasets 3.0.0
  • Tokenizers 0.19.1
Downloads last month
0
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for dgutierrez/llama381binstruct_summarize_short

Adapter
(69)
this model