lenatr99's picture
lora_fine_tuned_cb_sloroberta
072b67c verified
|
raw
history blame
1.98 kB
metadata
license: cc-by-sa-4.0
library_name: peft
tags:
  - generated_from_trainer
base_model: EMBEDDIA/sloberta
metrics:
  - accuracy
  - f1
model-index:
  - name: lora_fine_tuned_cb_sloroberta
    results: []

lora_fine_tuned_cb_sloroberta

This model is a fine-tuned version of EMBEDDIA/sloberta on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4857
  • Accuracy: 0.3182
  • F1: 0.1536

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • training_steps: 400

Training results

Training Loss Epoch Step Validation Loss Accuracy F1
0.8917 3.5714 50 1.2659 0.3182 0.1536
0.7167 7.1429 100 1.4722 0.3182 0.1536
0.7436 10.7143 150 1.4516 0.3182 0.1536
0.6926 14.2857 200 1.4899 0.3182 0.1536
0.7165 17.8571 250 1.4917 0.3182 0.1536
0.7161 21.4286 300 1.4901 0.3182 0.1536
0.6938 25.0 350 1.4871 0.3182 0.1536
0.7145 28.5714 400 1.4857 0.3182 0.1536

Framework versions

  • PEFT 0.11.1
  • Transformers 4.41.0
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1