Hugging Face's logo

wcvz
/
esm2_t130_150M-lora-classifier_2024-04-26_10-31-42

PEFT
Safetensors
Generated from Trainer
Model card Files Community
esm2_t130_150M-lora-classifier_2024-04-26_10-31-42 / README.md
wcvz's picture
wcvz
Model save
e607087 verified
preview code
|
raw
history blame
7.05 kB
metadata
license: mit
library_name: peft
tags:
  - generated_from_trainer
base_model: facebook/esm2_t30_150M_UR50D
metrics:
  - accuracy
model-index:
  - name: esm2_t130_150M-lora-classifier_2024-04-26_10-31-42
    results: []

esm2_t130_150M-lora-classifier_2024-04-26_10-31-42

This model is a fine-tuned version of facebook/esm2_t30_150M_UR50D on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.8800
  • Accuracy: 0.8945

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0008701568055793088
  • train_batch_size: 28
  • eval_batch_size: 28
  • seed: 8893
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 90
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
0.659 1.0 55 0.6868 0.5820
0.5296 2.0 110 0.6101 0.6777
0.5532 3.0 165 0.5395 0.7402
0.505 4.0 220 0.3482 0.8613
0.1246 5.0 275 0.3447 0.8535
0.3953 6.0 330 0.2982 0.8789
0.169 7.0 385 0.3257 0.875
0.2096 8.0 440 0.4022 0.8438
0.2339 9.0 495 0.3090 0.8848
0.1876 10.0 550 0.2723 0.8926
0.0631 11.0 605 0.3260 0.875
0.2843 12.0 660 0.4296 0.8438
0.0502 13.0 715 0.3301 0.8945
0.185 14.0 770 0.3273 0.8984
0.098 15.0 825 0.3355 0.8945
0.2528 16.0 880 0.4209 0.8926
0.2165 17.0 935 0.4467 0.8809
0.5202 18.0 990 0.4732 0.8984
0.0746 19.0 1045 0.4913 0.8926
0.0107 20.0 1100 0.4292 0.8906
0.0263 21.0 1155 0.6244 0.8555
0.011 22.0 1210 0.5228 0.8809
0.093 23.0 1265 0.5670 0.8828
0.058 24.0 1320 0.8912 0.8379
0.0007 25.0 1375 1.0458 0.8594
0.082 26.0 1430 0.7712 0.8848
0.0172 27.0 1485 0.7106 0.8945
0.1386 28.0 1540 0.9548 0.8535
0.1145 29.0 1595 0.8496 0.8945
0.001 30.0 1650 0.9245 0.8770
0.0004 31.0 1705 0.9368 0.8867
0.0039 32.0 1760 0.9754 0.8828
0.0165 33.0 1815 1.0050 0.8770
0.0028 34.0 1870 1.0051 0.8848
0.0006 35.0 1925 0.9673 0.8652
0.0 36.0 1980 0.9794 0.8906
0.0 37.0 2035 0.9294 0.8984
0.1004 38.0 2090 0.9621 0.8965
0.0 39.0 2145 0.9699 0.8965
0.0001 40.0 2200 1.0551 0.8926
0.0001 41.0 2255 0.9521 0.8965
0.1139 42.0 2310 1.0807 0.8828
0.0291 43.0 2365 0.9925 0.8965
0.0001 44.0 2420 1.0462 0.8867
0.0001 45.0 2475 0.9989 0.8848
0.0 46.0 2530 0.9005 0.8945
0.0005 47.0 2585 1.0845 0.8809
0.0 48.0 2640 0.9892 0.8965
0.0001 49.0 2695 0.9311 0.8887
0.0 50.0 2750 0.9819 0.8887
0.0 51.0 2805 1.0463 0.8887
0.0 52.0 2860 1.0672 0.8828
0.0 53.0 2915 1.0893 0.8926
0.0 54.0 2970 1.1496 0.8848
0.0002 55.0 3025 1.1330 0.8809
0.0009 56.0 3080 1.0782 0.8828
0.0046 57.0 3135 0.9937 0.8887
0.0009 58.0 3190 0.9710 0.8945
0.001 59.0 3245 1.0381 0.8848
0.0001 60.0 3300 0.9837 0.8887
0.0 61.0 3355 0.9552 0.8926
0.0002 62.0 3410 1.0600 0.8730
0.0 63.0 3465 0.9684 0.8887
0.0 64.0 3520 0.9498 0.8926
0.0003 65.0 3575 0.9644 0.8926
0.0 66.0 3630 0.9054 0.8887
0.0 67.0 3685 0.9370 0.8945
0.0001 68.0 3740 1.0082 0.8789
0.0001 69.0 3795 0.9378 0.8945
0.0048 70.0 3850 0.9371 0.8945
0.0002 71.0 3905 1.0431 0.8730
0.0007 72.0 3960 0.9235 0.8828
0.0011 73.0 4015 0.9624 0.8867
0.0 74.0 4070 0.9465 0.8926
0.0 75.0 4125 0.9266 0.8906
0.0 76.0 4180 0.9872 0.8867
0.0 77.0 4235 0.9488 0.8887
0.0002 78.0 4290 0.9376 0.8906
0.0 79.0 4345 0.9632 0.8867
0.0001 80.0 4400 0.9373 0.8926
0.0001 81.0 4455 0.9352 0.8848
0.0 82.0 4510 0.8856 0.8906
0.0001 83.0 4565 0.8813 0.8926
0.0001 84.0 4620 0.8822 0.8887
0.0 85.0 4675 0.8911 0.8887
0.0 86.0 4730 0.8834 0.8945
0.0001 87.0 4785 0.8747 0.8945
0.0 88.0 4840 0.8823 0.8926
0.0 89.0 4895 0.8824 0.8926
0.0 90.0 4950 0.8800 0.8945

Framework versions

  • PEFT 0.10.0
  • Transformers 4.39.3
  • Pytorch 2.2.1
  • Datasets 2.16.1
  • Tokenizers 0.15.2