Hugging Face's logo

AntoineD
/
MiniLM_uncased_classification_tools_qlora_fr

Generated from Trainer
Model card Files Community
MiniLM_uncased_classification_tools_qlora_fr / README.md
AntoineD's picture
AntoineD
End of training
613a73a
preview code
|
raw
history blame
5.69 kB
metadata
license: mit
base_model: microsoft/MiniLM-L12-H384-uncased
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: MiniLM_uncased_classification_tools_qlora_fr
    results: []

MiniLM_uncased_classification_tools_qlora_fr

This model is a fine-tuned version of microsoft/MiniLM-L12-H384-uncased on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.7495
  • Accuracy: 0.5
  • Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 24
  • eval_batch_size: 192
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 60

Training results

Training Loss Epoch Step Validation Loss Accuracy Rate
No log 1.0 7 2.0845 0.075 0.0001
No log 2.0 14 2.0852 0.075 0.0001
No log 3.0 21 2.0851 0.075 0.0001
No log 4.0 28 2.0855 0.075 0.0001
No log 5.0 35 2.0858 0.075 0.0001
No log 6.0 42 2.0861 0.075 9e-05
No log 7.0 49 2.0863 0.075 0.0001
No log 8.0 56 2.0859 0.075 0.0001
No log 9.0 63 2.0856 0.075 0.0001
No log 10.0 70 2.0855 0.075 0.0001
No log 11.0 77 2.0848 0.075 0.0001
No log 12.0 84 2.0844 0.075 8e-05
No log 13.0 91 2.0829 0.075 0.0001
No log 14.0 98 2.0825 0.075 0.0001
No log 15.0 105 2.0811 0.1 0.0001
No log 16.0 112 2.0793 0.175 0.0001
No log 17.0 119 2.0771 0.125 0.0001
No log 18.0 126 2.0750 0.175 7e-05
No log 19.0 133 2.0723 0.175 0.0001
No log 20.0 140 2.0683 0.175 0.0001
No log 21.0 147 2.0637 0.175 0.0001
No log 22.0 154 2.0574 0.175 0.0001
No log 23.0 161 2.0507 0.2 0.0001
No log 24.0 168 2.0419 0.325 6e-05
No log 25.0 175 2.0318 0.35 0.0001
No log 26.0 182 2.0214 0.4 0.0001
No log 27.0 189 2.0083 0.4 0.0001
No log 28.0 196 1.9949 0.4 0.0001
No log 29.0 203 1.9781 0.4 0.0001
No log 30.0 210 1.9609 0.4 5e-05
No log 31.0 217 1.9475 0.425 0.0000
No log 32.0 224 1.9317 0.45 0.0000
No log 33.0 231 1.9131 0.45 0.0000
No log 34.0 238 1.9015 0.475 0.0000
No log 35.0 245 1.8906 0.5 0.0000
No log 36.0 252 1.8740 0.475 4e-05
No log 37.0 259 1.8613 0.5 0.0000
No log 38.0 266 1.8552 0.525 0.0000
No log 39.0 273 1.8389 0.5 0.0000
No log 40.0 280 1.8302 0.5 0.0000
No log 41.0 287 1.8228 0.5 0.0000
No log 42.0 294 1.8244 0.525 3e-05
No log 43.0 301 1.8048 0.5 0.0000
No log 44.0 308 1.7944 0.525 0.0000
No log 45.0 315 1.7929 0.5 0.0000
No log 46.0 322 1.7904 0.5 0.0000
No log 47.0 329 1.7810 0.5 0.0000
No log 48.0 336 1.7790 0.5 2e-05
No log 49.0 343 1.7758 0.5 0.0000
No log 50.0 350 1.7677 0.525 0.0000
No log 51.0 357 1.7626 0.525 0.0000
No log 52.0 364 1.7579 0.525 0.0000
No log 53.0 371 1.7552 0.525 0.0000
No log 54.0 378 1.7544 0.525 1e-05
No log 55.0 385 1.7523 0.525 0.0000
No log 56.0 392 1.7510 0.525 0.0000
No log 57.0 399 1.7501 0.525 5e-06
No log 58.0 406 1.7498 0.525 0.0000
No log 59.0 413 1.7496 0.525 0.0000
No log 60.0 420 1.7495 0.5 0.0

Framework versions

  • Transformers 4.34.0
  • Pytorch 2.0.1+cu117
  • Datasets 2.14.5
  • Tokenizers 0.14.1